diff --git "a/datasets/computer_vision.gexf" "b/datasets/computer_vision.gexf"
deleted file mode 100644--- "a/datasets/computer_vision.gexf"
+++ /dev/null
@@ -1,38736 +0,0 @@
-<?xml version='1.0' encoding='utf-8'?>
-<gexf xmlns="http://www.gexf.net/1.2draft" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gexf.net/1.2draft http://www.gexf.net/1.2draft/gexf.xsd" version="1.2">
-  <meta lastmodifieddate="2025-02-28">
-    <creator>NetworkX 3.3</creator>
-  </meta>
-  <graph defaultedgetype="directed" mode="static" name="">
-    <attributes mode="static" class="edge">
-      <attribute id="5" title="sentence" type="string" />
-    </attributes>
-    <attributes mode="static" class="node">
-      <attribute id="0" title="title" type="string" />
-      <attribute id="1" title="abstract" type="string" />
-      <attribute id="2" title="introduction" type="string" />
-      <attribute id="3" title="related" type="string" />
-      <attribute id="4" title="concepts" type="string" />
-    </attributes>
-    <nodes>
-      <node id="2003.09852" label="2003.09852">
-        <attvalues>
-          <attvalue for="0" value="Multiview Neural Surface Reconstruction by Disentangling Geometry and&#10;  Appearance" />
-          <attvalue for="1" value="  In this work we address the challenging problem of multiview 3D surface&#10;reconstruction. We introduce a neural network architecture that simultaneously&#10;learns the unknown geometry, camera parameters, and a neural renderer that&#10;approximates the light reflected from the surface towards the camera. The&#10;geometry is represented as a zero level-set of a neural network, while the&#10;neural renderer, derived from the rendering equation, is capable of&#10;(implicitly) modeling a wide set of lighting conditions and materials. We&#10;trained our network on real world 2D images of objects with different material&#10;properties, lighting conditions, and noisy camera initializations from the DTU&#10;MVS dataset. We found our model to produce state of the art 3D surface&#10;reconstructions with high fidelity, resolution and detail.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Differentiable rendering systems for learning geometry comes (mostly) in two flavors: differentiable rasterization \cite{loper2014opendr,kato2018neural,genova2018unsupervised,liu2019soft_b,chen2019learning}, and differentiable ray casting. Since the current work falls into the second category we first concentrate on that branch of works. Then, we will describe related works for multi-view surface reconstruction and neural view synthesis. &#10;&#10;Implicit surface differentiable ray casting.&#10;Differentiable ray casting is mostly used with implicit shape representations such as implicit function defined over a volumetric grid or implicit neural representation, where the implicit function can be the occupancy function \cite{mescheder2019occupancy,chen2019learningimplicit}, signed distance function (SDF) \cite{park2019deepsdf} or any other signed implicit \cite{atzmon2019sal}. &#10;In a related work, \cite{jiang2019sdfdiff} use a volumetric grid to represent an SDF and implement a ray casting differentiable renderer. They approximate the SDF value and the surface normals in each volumetric cell. &#10;\cite{liu2019dist} use sphere tracing of pre-trained DeepSDF model \cite{park2019deepsdf} and approximates the depth gradients w.r.t.~the latent code of the DeepSDF network by differentiating the individual steps of the sphere tracing algorithm; \cite{liu2019learning} use field probing to facilitate differentiable ray casting.&#10;In contrast to these works, IDR utilize exact and differentiable surface point and normal of the implicit surface, and considers a more general appearance model, as well as handle noisy cameras.&#10;&#10;Multi-view surface reconstruction.&#10;During the capturing process of an image, the depth information is lost. Assuming known cameras, classic Multi-View Stereo (MVS) methods \cite{furukawa2009accurate,schonberger2016pixelwise,campbell2008using,tola2012efficient} try to reproduce the depth information by matching features points across views. However, a post-processing steps of depth fusion \cite{curless1996volumetric,merrell2007real} followed by the Poisson Surface Reconstruction algorithm \cite{kazhdan2006poisson} are required for producing a valid 3D watertight surface reconstruction. &#10;Recent methods use a collection of scenes to train a deep neural models for either sub-tasks of the MVS pipeline, \eg, feature matching \cite{leroy2018shape}, or depth fusion \cite{donne2019learning,riegler2017octnetfusion}, or for an End-to-End MVS pipeline \cite{huang2018deepmvs,yao2018mvsnet,yao2019recurrent}. &#10;When the camera parameters are unavailable, and given a set of images from a specific scene, Structure From Motion (SFM) methods \cite{snavely2006photo,Schonberger_2016_CVPR,kasten2019algebraic,jiang2013global} are applied for reproducing the cameras and a sparse 3D reconstruction. Tang and Tan \cite{Tang2019BANetDB} use a deep neural architecture with an integrated differentiable Bundle Adjustment \cite{triggs1999bundle} layer to extract a linear basis for the depth of a reference frame, and features from nearby images and to optimize for the depth and the camera parameters in each forward pass. In contrast to these works, IDR is trained with images from a single target scene, producing an accurate watertight 3D surface reconstruction.&#10;&#10;Neural representation for view synthesis.&#10;Recent works trained neural networks to predict novel views and some geometric representation of 3D scenes or objects, from a limited set of images with known cameras. \cite{sitzmann2019scene} encode the scene geometry using an LSTM to simulate the ray marching process. \cite{mildenhall2020nerf} use a neural network to predict volume density and view dependent emitted radiance to synthesis new views from a set of images with known cameras. \cite{oechsle2020learning} use a neural network to learns the surface light fields from an input image and geometry and predicting unknown views and/or scene lighting. &#10;Differently from IDR, these methods do not produce a 3D surface reconstruction of the scene's geometry nor handle unknown cameras. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Neural Network Architecture, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2009.07506" label="2009.07506">
-        <attvalues>
-          <attvalue for="0" value="The 1st Tiny Object Detection Challenge:Methods and Results" />
-          <attvalue for="1" value="  The 1st Tiny Object Detection (TOD) Challenge aims to encourage research in&#10;developing novel and accurate methods for tiny object detection in images which&#10;have wide views, with a current focus on tiny person detection. The TinyPerson&#10;dataset was used for the TOD Challenge and is publicly released. It has 1610&#10;images and 72651 box-levelannotations. Around 36 participating teams from the&#10;globe competed inthe 1st TOD Challenge. In this paper, we provide a brief&#10;summary of the1st TOD Challenge including brief introductions to the top three&#10;methods.The submission leaderboard will be reopened for researchers that&#10;areinterested in the TOD challenge. The benchmark dataset and other information&#10;can be found at: https://github.com/ucas-vg/TinyBenchmark.&#10;" />
-          <attvalue for="2" value="&#10;Object detection is an important topic in the computer vision community. With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}. Nevertheless, detecting tiny objects remains challenging and far from well-explored. One possible reason for this is because there is a lack of datasets and benchmarks for tiny object detection, and researchers thus pay much less attention to challenges in tiny object detection compared to general object detection.&#10;&#10;Tiny object detection is very important for real-world vision application and differs from general object detection in several aspects. For example, since the objects are extremely small while the whole input image has relatively large field-of-view, there is much less information from the targeting objects and much more from background distractions. In addition, the large field-of-view characteristic of input images usually means that the tiny objects are imaged from a long distance, this makes detection of tiny objects with various poses and viewpoints even more difficult. All these distinctions make tiny object detection a uniquely challenging task.&#10;&#10;To encourage researchers to develop better methods to solve the tiny object detection problem, we held the first Tiny Object Detection Challenge. The TinyPerson dataset~\cite{yu2020scale} was adopted in this challenge. The dataset contains 1610 images with 72651 box-level annotations and is collected from real-world scenarios. The persons in the challenge dataset are very small, and their aspect ratios have a large variance so that they are representative of different type of objects. In the following section, we will summarize the detail information about the challenge, the methods and results.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Benchmark Dataset Development, Artificial Intelligence, Computer Science, Computer Vision Challenges, Object Detection Methods" />
-        </attvalues>
-      </node>
-      <node id="1405.0312" label="1405.0312">
-        <attvalues>
-          <attvalue for="0" value="Microsoft COCO: Common Objects in Context" />
-          <attvalue for="1" value="  We present a new dataset with the goal of advancing the state-of-the-art in&#10;object recognition by placing the question of object recognition in the context&#10;of the broader question of scene understanding. This is achieved by gathering&#10;images of complex everyday scenes containing common objects in their natural&#10;context. Objects are labeled using per-instance segmentations to aid in precise&#10;object localization. Our dataset contains photos of 91 objects types that would&#10;be easily recognizable by a 4 year old. With a total of 2.5 million labeled&#10;instances in 328k images, the creation of our dataset drew upon extensive crowd&#10;worker involvement via novel user interfaces for category detection, instance&#10;spotting and instance segmentation. We present a detailed statistical analysis&#10;of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide&#10;baseline performance analysis for bounding box and segmentation detection&#10;results using a Deformable Parts Model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.01497" label="1506.01497">
-        <attvalues>
-          <attvalue for="0" value="Faster R-CNN: Towards Real-Time Object Detection with Region Proposal&#10;  Networks" />
-          <attvalue for="1" value="  State-of-the-art object detection networks depend on region proposal&#10;algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN&#10;have reduced the running time of these detection networks, exposing region&#10;proposal computation as a bottleneck. In this work, we introduce a Region&#10;Proposal Network (RPN) that shares full-image convolutional features with the&#10;detection network, thus enabling nearly cost-free region proposals. An RPN is a&#10;fully convolutional network that simultaneously predicts object bounds and&#10;objectness scores at each position. The RPN is trained end-to-end to generate&#10;high-quality region proposals, which are used by Fast R-CNN for detection. We&#10;further merge RPN and Fast R-CNN into a single network by sharing their&#10;convolutional features---using the recently popular terminology of neural&#10;networks with 'attention' mechanisms, the RPN component tells the unified&#10;network where to look. For the very deep VGG-16 model, our detection system has&#10;a frame rate of 5fps (including all steps) on a GPU, while achieving&#10;state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS&#10;COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015&#10;competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning&#10;entries in several tracks. Code has been made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.06799" label="1803.06799">
-        <attvalues>
-          <attvalue for="0" value="Revisiting RCNN: On Awakening the Classification Power of Faster RCNN" />
-          <attvalue for="1" value="  Recent region-based object detectors are usually built with separate&#10;classification and localization branches on top of shared feature extraction&#10;networks. In this paper, we analyze failure cases of state-of-the-art detectors&#10;and observe that most hard false positives result from classification instead&#10;of localization. We conjecture that: (1) Shared feature representation is not&#10;optimal due to the mismatched goals of feature learning for classification and&#10;localization; (2) multi-task learning helps, yet optimization of the multi-task&#10;loss may result in sub-optimal for individual tasks; (3) large receptive field&#10;for different scales leads to redundant context information for small&#10;objects.We demonstrate the potential of detector classification power by a&#10;simple, effective, and widely-applicable Decoupled Classification Refinement&#10;(DCR) network. DCR samples hard false positives from the base classifier in&#10;Faster RCNN and trains a RCNN-styled strong classifier. Experiments show new&#10;state-of-the-art results on PASCAL VOC and COCO without any bells and whistles.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.04002" label="1810.04002">
-        <attvalues>
-          <attvalue for="0" value="Decoupled Classification Refinement: Hard False Positive Suppression for&#10;  Object Detection" />
-          <attvalue for="1" value="  In this paper, we analyze failure cases of state-of-the-art detectors and&#10;observe that most hard false positives result from classification instead of&#10;localization and they have a large negative impact on the performance of object&#10;detectors. We conjecture there are three factors: (1) Shared feature&#10;representation is not optimal due to the mismatched goals of feature learning&#10;for classification and localization; (2) multi-task learning helps, yet&#10;optimization of the multi-task loss may result in sub-optimal for individual&#10;tasks; (3) large receptive field for different scales leads to redundant&#10;context information for small objects. We demonstrate the potential of detector&#10;classification power by a simple, effective, and widely-applicable Decoupled&#10;Classification Refinement (DCR) network. In particular, DCR places a separate&#10;classification network in parallel with the localization network (base&#10;detector). With ROI Pooling placed on the early stage of the classification&#10;network, we enforce an adaptive receptive field in DCR. During training, DCR&#10;samples hard false positives from the base detector and trains a strong&#10;classifier to refine classification results. During testing, DCR refines all&#10;boxes from the base detector. Experiments show competitive results on PASCAL&#10;VOC and COCO without any bells and whistles. Our codes are available at:&#10;https://github.com/bowenc0221/Decoupled-Classification-Refinement.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Object Detection. &#10;Recent CNN based object detectors can generally be categorized into two-stage and single stage. One of the first two-stage detector is RCNN \cite{girshick2014rich}, where selective search \cite{uijlings2013selective} is used to generate a set of region proposals for object candidates, then a deep neural network to extract feature vector of each region followed by SVM classifiers. SPPNet \cite{he2014spatial} improves the efficiency of RCNN by sharing feature extraction stage and use spatial pyramid pooling to extract fixed length feature for each proposal. Fast RCNN \cite{girshick2015fast} improves over SPPNet by introducing an differentiable ROI Pooling operation to train the network end-to-end. Faster RCNN \cite{ren2015faster} embeds the region proposal step into a Region Proposal Network (RPN) that further reduce the proposal generation time. R-FCN \cite{dai2016r} proposed a position sensitive ROI Pooling (PSROI Pooling) that can share computation among classification branch and bounding box regression branch. Deformable ConvNets (DCN) \cite{dai2017deformable} further add deformable convolutions and deformable ROI Pooling operations, that use learned offsets to adjust position of each sampling bin in naive convolutions and ROI Pooling, to Faster RCNN. Feature Pyramid Networks (FPN) \cite{lin2017feature} add a top-down path with lateral connections to build a pyramid of features with different resolutions and attach detection heads to each level of the feature pyramid for making prediction. Finer feature maps are more useful for detecting small objects and thus a significant boost in small object detection is observed with FPN. Most of the current state-of-the-art object detectors are two-stage detectors based of Faster RCNN, because two-stage object detectors produce more accurate results and are easier to optimize. However, two-stage detectors are slow in speed and require very large input sizes due to the ROI Pooling operation. Aimed at achieving real time object detectors, one-stage method, such as OverFeat \cite{sermanet2013overfeat}, SSD \cite{liu2016ssd,fu2017dssd} and YOLO \cite{redmon2016you,redmon2017yolo9000}, predict object classes and locations directly. Though single stage methods are much faster than two-stage methods, their results are inferior and they need more extra data and extensive data augmentation to get better results. Recently, there advances a new group of one-stage object detectors named anchor-free methods. One type of anchor-free methods~\cite{Law_2018_ECCV,zhou2019bottom,duan2019centernet,zhou2019objects,cheng2019panoptic_1,cheng2019panoptic_2} are motivated by human pose estimation~\cite{cao2017realtime,papandreou2017towards,cheng2019bottom}. They first define ``keypoints'' for an object (e.g. center point~\cite{zhou2019objects,cheng2019panoptic_1,cheng2019panoptic_2}, corner point~\cite{Law_2018_ECCV} or extreme point~\cite{zhou2019bottom}) and regress to the object bounding box. Another type of anchor-free methods~\cite{tian2019fcos,zhu2019feature} are motivated by fully convolutional networks (FCN) for semantic segmentation~\cite{long2015fully,deeplabv3plus2018,cheng2019spgnet}. They directly predict the bounding box on every pixel that belongs to an instance. Our paper follows the method of two-stage detectors \cite{girshick2014rich,girshick2015fast,ren2015faster}, but with a main focus on analyzing reasons why detectors make mistakes.&#10;&#10;Classifier Cascade. &#10;The method of classifier cascade commonly trains a stage classifier using misclassified examples from a previous classifier. This has been used a lot for object detection in the past. The Viola Jones Algorithm \cite{viola2004robust} for face detection used a hard cascades by Adaboost \cite{freund1997decision}, where a strong region classifier is built with cascade of many weak classifier focusing attentions on different features and if any of the weak classifier rejects the window, there will be no more process. Soft cascades \cite{bourdev2005robust} improved \cite{viola2004robust} built each weak classifier based on the output of all previous classifiers. Deformable Part Model (DPM) \cite{felzenszwalb2010object} used a cascade of parts method where a root filter on coarse feature covering the entire object is combined with some part filters on fine feature with greater localization accuracy. More recently, Li et al. \cite{li2015convolutional} proposed the Convolutional Neural Network Cascade for fast face detection. Our paper proposed a method similar to the classifier cascade idea, however, they are different in the following aspects. The classifier cascade aims at producing an efficient classifier (mainly in speed) by cascade weak but fast classifiers and the weak classifiers are used to reject examples. In comparison, our method aims at improving the overall system accuracy, where exactly two strong classifiers are cascaded and they work together to make more accurate predictions. More recent Cascade RCNN \cite{cai2017cascade} proposes training object detector in a cascade manner with gradually increased IoU threshold to assign ground truth labels to align the testing metric i.e. average mAP with IOU 0.5:0.05:0.95.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1712.00726" label="1712.00726">
-        <attvalues>
-          <attvalue for="0" value="Cascade R-CNN: Delving into High Quality Object Detection" />
-          <attvalue for="1" value="  In object detection, an intersection over union (IoU) threshold is required&#10;to define positives and negatives. An object detector, trained with low IoU&#10;threshold, e.g. 0.5, usually produces noisy detections. However, detection&#10;performance tends to degrade with increasing the IoU thresholds. Two main&#10;factors are responsible for this: 1) overfitting during training, due to&#10;exponentially vanishing positive samples, and 2) inference-time mismatch&#10;between the IoUs for which the detector is optimal and those of the input&#10;hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is&#10;proposed to address these problems. It consists of a sequence of detectors&#10;trained with increasing IoU thresholds, to be sequentially more selective&#10;against close false positives. The detectors are trained stage by stage,&#10;leveraging the observation that the output of a detector is a good distribution&#10;for training the next higher quality detector. The resampling of progressively&#10;improved hypotheses guarantees that all detectors have a positive set of&#10;examples of equivalent size, reducing the overfitting problem. The same cascade&#10;procedure is applied at inference, enabling a closer match between the&#10;hypotheses and the detector quality of each stage. A simple implementation of&#10;the Cascade R-CNN is shown to surpass all single-model object detectors on the&#10;challenging COCO dataset. Experiments also show that the Cascade R-CNN is&#10;widely applicable across detector architectures, achieving consistent gains&#10;independently of the baseline detector strength. The code will be made&#10;available at https://github.com/zhaoweicai/cascade-rcnn.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.00886" label="1712.00886">
-        <attvalues>
-          <attvalue for="0" value="Improving Object Detection from Scratch via Gated Feature Reuse" />
-          <attvalue for="1" value="  In this paper, we present a simple and parameter-efficient drop-in module for&#10;one-stage object detectors like SSD when learning from scratch (i.e., without&#10;pre-trained models). We call our module GFR (Gated Feature Reuse), which&#10;exhibits two main advantages. First, we introduce a novel gate-controlled&#10;prediction strategy enabled by Squeeze-and-Excitation to adaptively enhance or&#10;attenuate supervision at different scales based on the input object size. As a&#10;result, our model is more effective in detecting diverse sizes of objects.&#10;Second, we propose a feature-pyramids structure to squeeze rich spatial and&#10;semantic features into a single prediction layer, which strengthens feature&#10;representation and reduces the number of parameters to learn. We apply the&#10;proposed structure on DSOD and SSD detection frameworks, and evaluate the&#10;performance on PASCAL VOC 2007, 2012 and COCO datasets. With fewer model&#10;parameters, GFR-DSOD outperforms the baseline DSOD by 1.4%, 1.1%, 1.7% and&#10;0.6%, respectively. GFR-SSD also outperforms the original SSD and SSD with&#10;dense prediction by 3.6% and 2.8% on VOC 2007 dataset. Code is available at:&#10;https://github.com/szq0214/GFR-DSOD .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.09709" label="1909.09709">
-        <attvalues>
-          <attvalue for="0" value="SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on&#10;  Embedded Systems" />
-          <attvalue for="1" value="  Object detection and tracking are challenging tasks for resource-constrained&#10;embedded systems. While these tasks are among the most compute-intensive tasks&#10;from the artificial intelligence domain, they are only allowed to use limited&#10;computation and memory resources on embedded devices. In the meanwhile, such&#10;resource-constrained implementations are often required to satisfy additional&#10;demanding requirements such as real-time response, high-throughput performance,&#10;and reliable inference accuracy. To overcome these challenges, we propose&#10;SkyNet, a hardware-efficient neural network to deliver the state-of-the-art&#10;detection accuracy and speed for embedded systems. Instead of following the&#10;common top-down flow for compact DNN (Deep Neural Network) design, SkyNet&#10;provides a bottom-up DNN design approach with comprehensive understanding of&#10;the hardware constraints at the very beginning to deliver hardware-efficient&#10;DNNs. The effectiveness of SkyNet is demonstrated by winning the competitive&#10;System Design Contest for low power object detection in the 56th IEEE/ACM&#10;Design Automation Conference (DAC-SDC), where our SkyNet significantly&#10;outperforms all other 100+ competitors: it delivers 0.731 Intersection over&#10;Union (IoU) and 67.33 frames per second (FPS) on a TX2 embedded GPU; and 0.716&#10;IoU and 25.05 FPS on an Ultra96 embedded FPGA. The evaluation of SkyNet is also&#10;extended to GOT-10K, a recent large-scale high-diversity benchmark for generic&#10;object tracking in the wild. For state-of-the-art object trackers SiamRPN++ and&#10;SiamMask, where ResNet-50 is employed as the backbone, implementations using&#10;our SkyNet as the backbone DNN are 1.60X and 1.73X faster with better or&#10;similar accuracy when running on a 1080Ti GPU, and 37.20X smaller in terms of&#10;parameter size for significantly better memory and storage footprint.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.10664" label="1912.10664">
-        <attvalues>
-          <attvalue for="0" value="Scale Match for Tiny Person Detection" />
-          <attvalue for="1" value="  Visual object detection has achieved unprecedented ad-vance with the rise of&#10;deep convolutional neural networks.However, detecting tiny objects (for example&#10;tiny per-sons less than 20 pixels) in large-scale images remainsnot well&#10;investigated. The extremely small objects raisea grand challenge about feature&#10;representation while themassive and complex backgrounds aggregate the risk&#10;offalse alarms. In this paper, we introduce a new benchmark,referred to as&#10;TinyPerson, opening up a promising directionfor tiny object detection in a long&#10;distance and with mas-sive backgrounds. We experimentally find that the scale&#10;mis-match between the dataset for network pre-training and thedataset for&#10;detector learning could deteriorate the featurerepresentation and the&#10;detectors. Accordingly, we proposea simple yet effective Scale Match approach&#10;to align theobject scales between the two datasets for favorable tiny-object&#10;representation. Experiments show the significantperformance gain of our&#10;proposed approach over state-of-the-art detectors, and the challenging aspects&#10;of TinyPersonrelated to real-world scenarios. The TinyPerson benchmarkand the&#10;code for our approach will be publicly&#10;available(https://github.com/ucas-vg/TinyBenchmark).(Attention: evaluation&#10;rules of AP have updated in benchmark after this paper accepted, So this paper&#10;use old rules. we will keep old rules of AP in benchmark, but we recommand the&#10;new and we will use the new in latter research.)&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.01923" label="2203.01923">
-        <attvalues>
-          <attvalue for="0" value="Recovering 3D Human Mesh from Monocular Images: A Survey" />
-          <attvalue for="1" value="  Estimating human pose and shape from monocular images is a long-standing&#10;problem in computer vision. Since the release of statistical body models, 3D&#10;human mesh recovery has been drawing broader attention. With the same goal of&#10;obtaining well-aligned and physically plausible mesh results, two paradigms&#10;have been developed to overcome challenges in the 2D-to-3D lifting process: i)&#10;an optimization-based paradigm, where different data terms and regularization&#10;terms are exploited as optimization objectives; and ii) a regression-based&#10;paradigm, where deep learning techniques are embraced to solve the problem in&#10;an end-to-end fashion. Meanwhile, continuous efforts are devoted to improving&#10;the quality of 3D mesh labels for a wide range of datasets. Though remarkable&#10;progress has been achieved in the past decade, the task is still challenging&#10;due to flexible body motions, diverse appearances, complex environments, and&#10;insufficient in-the-wild annotations. To the best of our knowledge, this is the&#10;first survey that focuses on the task of monocular 3D human mesh recovery. We&#10;start with the introduction of body models and then elaborate recovery&#10;frameworks and training objectives by providing in-depth analyses of their&#10;strengths and weaknesses. We also summarize datasets, evaluation metrics, and&#10;benchmark results. Open issues and future directions are discussed in the end,&#10;hoping to motivate researchers and facilitate their research in this area. A&#10;regularly updated project page can be found at&#10;https://github.com/tinatiansjz/hmr-survey.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Understanding humans from monocular images is one of the fundamental tasks in computer vision.&#10;Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images.&#10;With these advances, researchers further seek to estimate human pose in 3D space~\cite{grauman2003inferring,agarwal2005recovering, martinez_2017_3dbaseline, pavlakos2017coarse, sun2018integral, mehta2020xnect,Weinzaepfel2020_dope}.&#10;Although simple movements can be represented relatively clearly by 2D contents or a few sparse 3D joints, complex human behaviors require more descriptions of the human body with a finer granularity.&#10;Moreover, it is critical to reason about body shape, contact, gesture, and expression since we interact with the world using our surface skin instead of unobserved joints.&#10;&#10;In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}.&#10;This trend is inseparable from the success of statistical human models. As shown in Fig.~\ref{fig:citations},&#10;since the release of the SMPL model~\cite{loper2015smpl} in 2015 and the SMPL-X model~\cite{pavlakos2019expressive} in 2019, they have gained increasing interest as their annual citations grow rapidly year by year.&#10;The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}.&#10;It is also involved in widespread applications such as VR/AR content creation, virtual try-on, and computer-assistant coaching, as depicted in Fig.~\ref{fig:application}.&#10;&#10;Recovering 3D human mesh from monocular images is quite challenging, owing to the issues such as inherent ambiguities in lifting 2D observations to 3D space, flexible body kinematic structures, complex intersections with the environments, and insufficient annotated 3D data.&#10;To address these issues, two different paradigms have been investigated in this field for the recovery of well-aligned&#10;and physically plausible results.&#10;Following the optimization-based paradigm~\cite{bogo2016keep,lassner2017unite,zanfir2018monocular}, methods explicitly fit body models to 2D observations in an iterative manner. Various data terms and regularization terms are explored as optimization objectives.&#10;Alternatively, the regression-based paradigm~\cite{tung2017self, kanazawa2018end, pavlakos2018learning, omran2018neural, kolotouros2019learning} takes advantage of the powerful nonlinear mapping capability of neural networks and directly predicts model parameters from raw image pixels. Different network architectures and regression targets are designed to achieve better performances. &#10;Meanwhile, significant efforts have also been devoted to creating various datasets to facilitate the research of this task. Despite the remarkable progress achieved in recent years, the research community still faces challenges toward the ultimate goal of robust, accurate, and efficient human mesh recovery.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Human Pose Estimation, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1812.08008" label="1812.08008">
-        <attvalues>
-          <attvalue for="0" value="OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity&#10;  Fields" />
-          <attvalue for="1" value="  Realtime multi-person 2D pose estimation is a key component in enabling&#10;machines to have an understanding of people in images and videos. In this work,&#10;we present a realtime approach to detect the 2D pose of multiple people in an&#10;image. The proposed method uses a nonparametric representation, which we refer&#10;to as Part Affinity Fields (PAFs), to learn to associate body parts with&#10;individuals in the image. This bottom-up system achieves high accuracy and&#10;realtime performance, regardless of the number of people in the image. In&#10;previous work, PAFs and body part location estimation were refined&#10;simultaneously across training stages. We demonstrate that a PAF-only&#10;refinement rather than both PAF and body part location refinement results in a&#10;substantial increase in both runtime performance and accuracy. We also present&#10;the first combined body and foot keypoint detector, based on an internal&#10;annotated foot dataset that we have publicly released. We show that the&#10;combined detector not only reduces the inference time compared to running them&#10;sequentially, but also maintains the accuracy of each component individually.&#10;This work has culminated in the release of OpenPose, the first open-source&#10;realtime system for multi-person 2D pose detection, including body, foot, hand,&#10;and facial keypoints.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.00137" label="1612.00137">
-        <attvalues>
-          <attvalue for="0" value="RMPE: Regional Multi-person Pose Estimation" />
-          <attvalue for="1" value="  Multi-person pose estimation in the wild is challenging. Although&#10;state-of-the-art human detectors have demonstrated good performance, small&#10;errors in localization and recognition are inevitable. These errors can cause&#10;failures for a single-person pose estimator (SPPE), especially for methods that&#10;solely depend on human detection results. In this paper, we propose a novel&#10;regional multi-person pose estimation (RMPE) framework to facilitate pose&#10;estimation in the presence of inaccurate human bounding boxes. Our framework&#10;consists of three components: Symmetric Spatial Transformer Network (SSTN),&#10;Parametric Pose Non-Maximum-Suppression (NMS), and Pose-Guided Proposals&#10;Generator (PGPG). Our method is able to handle inaccurate bounding boxes and&#10;redundant detections, allowing it to achieve a 17% increase in mAP over the&#10;state-of-the-art methods on the MPII (multi person) dataset.Our model and&#10;source codes are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.02440" label="2103.02440">
-        <attvalues>
-          <attvalue for="0" value="OpenPifPaf: Composite Fields for Semantic Keypoint Detection and&#10;  Spatio-Temporal Association" />
-          <attvalue for="1" value="  Many image-based perception tasks can be formulated as detecting, associating&#10;and tracking semantic keypoints, e.g., human body pose estimation and tracking.&#10;In this work, we present a general framework that jointly detects and forms&#10;spatio-temporal keypoint associations in a single stage, making this the first&#10;real-time pose detection and tracking algorithm. We present a generic neural&#10;network architecture that uses Composite Fields to detect and construct a&#10;spatio-temporal pose which is a single, connected graph whose nodes are the&#10;semantic keypoints (e.g., a person's body joints) in multiple frames. For the&#10;temporal associations, we introduce the Temporal Composite Association Field&#10;(TCAF) which requires an extended network architecture and training method&#10;beyond previous Composite Fields. Our experiments show competitive accuracy&#10;while being an order of magnitude faster on multiple publicly available&#10;datasets such as COCO, CrowdPose and the PoseTrack 2017 and 2018 datasets. We&#10;also show that our method generalizes to any class of semantic keypoints such&#10;as car and animal parts to provide a holistic perception framework that is well&#10;suited for urban mobility such as self-driving cars and delivery robots.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1809.01354" label="1809.01354">
-        <attvalues>
-          <attvalue for="0" value="Semantic Human Matting" />
-          <attvalue for="1" value="  Human matting, high quality extraction of humans from natural images, is&#10;crucial for a wide variety of applications. Since the matting problem is&#10;severely under-constrained, most previous methods require user interactions to&#10;take user designated trimaps or scribbles as constraints. This user-in-the-loop&#10;nature makes them difficult to be applied to large scale data or time-sensitive&#10;scenarios. In this paper, instead of using explicit user input constraints, we&#10;employ implicit semantic constraints learned from data and propose an automatic&#10;human matting algorithm (SHM). SHM is the first algorithm that learns to&#10;jointly fit both semantic information and high quality details with deep&#10;networks. In practice, simultaneously learning both coarse semantics and fine&#10;details is challenging. We propose a novel fusion strategy which naturally&#10;gives a probabilistic estimation of the alpha matte. We also construct a very&#10;large dataset with high quality annotations consisting of 35,513 unique&#10;foregrounds to facilitate the learning and evaluation of human matting.&#10;Extensive experiments on this dataset and plenty of real images show that SHM&#10;achieves comparable results with state-of-the-art interactive matting methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.03287" label="1804.03287">
-        <attvalues>
-          <attvalue for="0" value="Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning&#10;  and A New Benchmark for Multi-Human Parsing" />
-          <attvalue for="1" value="  Despite the noticeable progress in perceptual tasks like detection, instance&#10;segmentation and human parsing, computers still perform unsatisfactorily on&#10;visually understanding humans in crowded scenes, such as group behavior&#10;analysis, person re-identification and autonomous driving, etc. To this end,&#10;models need to comprehensively perceive the semantic information and the&#10;differences between instances in a multi-human image, which is recently defined&#10;as the multi-human parsing task. In this paper, we present a new large-scale&#10;database &quot;Multi-Human Parsing (MHP)&quot; for algorithm development and evaluation,&#10;and advances the state-of-the-art in understanding humans in crowded scenes.&#10;MHP contains 25,403 elaborately annotated images with 58 fine-grained semantic&#10;category labels, involving 2-26 persons per image and captured in real-world&#10;scenes from various viewpoints, poses, occlusion, interactions and background.&#10;We further propose a novel deep Nested Adversarial Network (NAN) model for&#10;multi-human parsing. NAN consists of three Generative Adversarial Network&#10;(GAN)-like sub-nets, respectively performing semantic saliency prediction,&#10;instance-agnostic parsing and instance-aware clustering. These sub-nets form a&#10;nested structure and are carefully designed to learn jointly in an end-to-end&#10;way. NAN consistently outperforms existing state-of-the-art solutions on our&#10;MHP and several other datasets, and serves as a strong baseline to drive the&#10;future research for multi-human parsing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.09457" label="2008.09457">
-        <attvalues>
-          <attvalue for="0" value="DOPE: Distillation Of Part Experts for whole-body 3D pose estimation in&#10;  the wild" />
-          <attvalue for="1" value="  We introduce DOPE, the first method to detect and estimate whole-body 3D&#10;human poses, including bodies, hands and faces, in the wild. Achieving this&#10;level of details is key for a number of applications that require understanding&#10;the interactions of the people with each other or with the environment. The&#10;main challenge is the lack of in-the-wild data with labeled whole-body 3D&#10;poses. In previous work, training data has been annotated or generated for&#10;simpler tasks focusing on bodies, hands or faces separately. In this work, we&#10;propose to take advantage of these datasets to train independent experts for&#10;each part, namely a body, a hand and a face expert, and distill their knowledge&#10;into a single deep network designed for whole-body 2D-3D pose detection. In&#10;practice, given a training image with partial or no annotation, each part&#10;expert detects its subset of keypoints in 2D and 3D and the resulting&#10;estimations are combined to obtain whole-body pseudo ground-truth poses. A&#10;distillation loss encourages the whole-body predictions to mimic the experts'&#10;outputs. Our results show that this approach significantly outperforms the same&#10;whole-body model trained without distillation while staying close to the&#10;performance of the experts. Importantly, DOPE is computationally less demanding&#10;than the ensemble of experts and can achieve real-time performance. Test code&#10;and models are available at&#10;https://europe.naverlabs.com/research/computer-vision/dope.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The problem of 3D human whole-body pose estimation has been mainly tackled by breaking the body into parts and focusing on the pose inference of these parts separately. In the following, we briefly review the state of the art for each of these subtasks, before summarizing the few approaches that predict the 3D pose of the entire body, and finally discussing existing distillation methods.&#10;&#10;3D body pose estimation.&#10;Two basic categories of work can be found in the recent literature: (a) approaches that directly estimate the 3D body keypoints from an input image~\cite{RogezS16,PavlakosZDD17a,lcrnet++,xnect,moon2019camera} and (b) methods that leverage 2D human pose estimation~\cite{MartinezHRL17,ChenR17,BogoKLG0B16,up3d}. &#10;The latter ones rely on a previous localization of the body keypoints in the image, through an off-the-shelf 2D pose detector~\cite{openpose,alphapose,maskrcnn}, and lift them to 3D space~\cite{MartinezHRL17,ChenR17} or, as in \cite{BogoKLG0B16,up3d}, use them to initialize the optimization procedure of a parametric model of the human body such as SMPL~\cite{smpl}. For our body expert, we employ LCR-Net++~\cite{lcrnet++} that jointly estimates 2D and 3D body poses from the image and has demonstrated robustness to challenging in-the-wild scenarios, \ie, showing multiple interacting persons, with cluttered backgrounds and/or captured under severe occlusions and truncations.&#10;&#10;3D hand pose estimation.&#10;3D hand pose estimation from depth data has been studied for many years and state-of-the-art results on this task are now impressive as shown in a recent survey~\cite{supancic2018}. &#10;RGB-based 3D hand pose estimation is more challenging, and has gained interest in recent years. &#10;Regression-based techniques try to directly predict 3D location of hand keypoints~\cite{yang2019} or even the vertices of a mesh~\cite{ge2019} from an input image. &#10;Some methods incorporate priors by regressing parameters of a deformable hand model such as MANO~\cite{romero2017,boukhayma2019,hasson2019},&#10;and many techniques leverage intermediate representations such as 2D keypoints heatmaps to perform 3D predictions~\cite{renderedhand,cai2018,ganerated,zhang2019}.&#10;However, pose estimation is often performed on an image cropped around a single hand, and hand detection is performed independently. For our hand expert, we therefore use the detector of~\cite{lcrnet++} (adapted to hands) that recently achieved outstanding performances in RGB-based 3D hand pose estimation under hand-object interaction~\cite{ICCV19HandChallenge}.&#10;&#10;3D face pose estimation.&#10;As with hands, the recovery of the pose of a face is typically performed from an image crop, by detecting particular 2D facial landmarks~\cite{wu2019}. To better perceive the 3D pose and shape of a face, some works propose to fit a 3D Morphable Model~\cite{face3dmm_blanz, face3dmm_booth, zhu2017} or to regress dense 3D face representations~\cite{pix2face, jackson2017, feng2018}. In this work, we also adopt~\cite{lcrnet++} as face expert, resulting in an hybrid model-free approach that regresses 3D facial landmarks independently from their visibility, as in the approach introduced for the Menpo 3D benchmark~\cite{menpo}.&#10;&#10;3D Whole-body pose estimation.&#10;The few existing methods~\cite{totalcapture,smplx} that predict the 3D pose of the whole-body all rely on parametric models of the human body, namely Adam~\cite{adam} and SMPL-X~\cite{smplx}. These models are obtained by combining body, hand and face parametric models. Adam stitches together three different models: a simpler version of SMPL for the body, an artist-created rig for the hands, and the FaceWarehouse model~\cite{FaceWarehouse} for the face. In the case of SMPL-X, the SMPL body model is augmented with the FLAME head model~\cite{Flame} and MANO~\cite{romero2017}. A more realistic model is obtained in the case of SMPL-X by learning the shape and pose-dependent blend shapes. Both methods are based on an optimization scheme guided by 2D joint locations or 3D part orientations. Monocular Total Capture~\cite{totalcapture} remains limited to a single person while for SMPL-X~\cite{smplx}, the optimization strategy is applied independently on each person detected by OpenPose~\cite{openpose}. Optimizing over the parameters of such models can be time-consuming and the performance often depends on a correct initialization. Our approach is the first one that predicts whole-body 3D pose without relying on the optimization of a parametric model and can make real-time predictions of multiple 3D whole-body poses in real-world scenes. In addition, our DOPE training scheme can leverage datasets that do not contain ground-truth for all the parts at once.&#10;&#10;Distillation.&#10;Our learning procedure is based on the concept of distillation which was proposed in the context of efficient neural network computation by using class probabilities of a higher-capacity model as soft targets of a smaller and faster model~\cite{hinton2015distilling}. &#10;Distillation has been successfully employed for several problems in computer vision such as object detection~\cite{chen2017learning}, video classification~\cite{BhardwajSK19}, action recognition~\cite{CrastoWAS19}, multi-task learning~\cite{liu2019improving} or lifelong learning~\cite{Hou2018lifelong}. &#10;In addition to training a compact model~\cite{chen2017learning,BhardwajSK19}, &#10;several works \cite{CrastoWAS19,HoffmanHallucination} have shown that distillation can be combined with privileged information~\cite{VapnikTransferLearning}, also called generalized distillation~\cite{lopez2015unifying} in order to train a network while leveraging extra modalities available for training, \eg training on RGB and depth data while only RGB is available at test time. &#10;In this paper, we propose to use distillation in order to transfer the knowledge of several body-part experts into a unified network that outputs a more complete representation of the whole human body. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1607.08128" label="1607.08128">
-        <attvalues>
-          <attvalue for="0" value="Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a&#10;  Single Image" />
-          <attvalue for="1" value="  We describe the first method to automatically estimate the 3D pose of the&#10;human body as well as its 3D shape from a single unconstrained image. We&#10;estimate a full 3D mesh and show that 2D joints alone carry a surprising amount&#10;of information about body shape. The problem is challenging because of the&#10;complexity of the human body, articulation, occlusion, clothing, lighting, and&#10;the inherent ambiguity in inferring 3D from 2D. To solve this, we first use a&#10;recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D&#10;body joint locations. We then fit (top-down) a recently published statistical&#10;body shape model, called SMPL, to the 2D joints. We do so by minimizing an&#10;objective function that penalizes the error between the projected 3D model&#10;joints and detected 2D joints. Because SMPL captures correlations in human&#10;shape across the population, we are able to robustly fit it to very little&#10;data. We further leverage the 3D model to prevent solutions that cause&#10;interpenetration. We evaluate our method, SMPLify, on the Leeds Sports,&#10;HumanEva, and Human3.6M datasets, showing superior pose accuracy with respect&#10;to the state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.06584" label="1712.06584">
-        <attvalues>
-          <attvalue for="0" value="End-to-end Recovery of Human Shape and Pose" />
-          <attvalue for="1" value="  We describe Human Mesh Recovery (HMR), an end-to-end framework for&#10;reconstructing a full 3D mesh of a human body from a single RGB image. In&#10;contrast to most current methods that compute 2D or 3D joint locations, we&#10;produce a richer and more useful mesh representation that is parameterized by&#10;shape and 3D joint angles. The main objective is to minimize the reprojection&#10;loss of keypoints, which allow our model to be trained using images in-the-wild&#10;that only have ground truth 2D annotations. However, the reprojection loss&#10;alone leaves the model highly under constrained. In this work we address this&#10;problem by introducing an adversary trained to tell whether a human body&#10;parameter is real or not using a large database of 3D human meshes. We show&#10;that HMR can be trained with and without using any paired 2D-to-3D supervision.&#10;We do not rely on intermediate 2D keypoint detections and infer 3D pose and&#10;shape parameters directly from image pixels. Our model runs in real-time given&#10;a bounding box containing the person. We demonstrate our approach on various&#10;images in-the-wild and out-perform previous optimization based methods that&#10;output 3D meshes and show competitive results on tasks such as 3D joint&#10;location estimation and part segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04092" label="1805.04092">
-        <attvalues>
-          <attvalue for="0" value="Learning to Estimate 3D Human Pose and Shape from a Single Color Image" />
-          <attvalue for="1" value="  This work addresses the problem of estimating the full body 3D human pose and&#10;shape from a single color image. This is a task where iterative&#10;optimization-based solutions have typically prevailed, while Convolutional&#10;Networks (ConvNets) have suffered because of the lack of training data and&#10;their low resolution 3D predictions. Our work aims to bridge this gap and&#10;proposes an efficient and effective direct prediction method based on ConvNets.&#10;Central part to our approach is the incorporation of a parametric statistical&#10;body shape model (SMPL) within our end-to-end framework. This allows us to get&#10;very detailed 3D mesh results, while requiring estimation only of a small&#10;number of parameters, making it friendly for direct network prediction.&#10;Interestingly, we demonstrate that these parameters can be predicted reliably&#10;only from 2D keypoints and masks. These are typical outputs of generic 2D human&#10;analysis ConvNets, allowing us to relax the massive requirement that images&#10;with 3D shape ground truth are available for training. Simultaneously, by&#10;maintaining differentiability, at training time we generate the 3D mesh from&#10;the estimated parameters and optimize explicitly for the surface using a 3D&#10;per-vertex loss. Finally, a differentiable renderer is employed to project the&#10;3D mesh to the image, which enables further refinement of the network, by&#10;optimizing for the consistency of the projection with 2D annotations (i.e., 2D&#10;keypoints or masks). The proposed approach outperforms previous baselines on&#10;this task and offers an attractive solution for direct prediction of 3D shape&#10;from a single color image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.05942" label="1808.05942">
-        <attvalues>
-          <attvalue for="0" value="Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose&#10;  and Shape Estimation" />
-          <attvalue for="1" value="  Direct prediction of 3D body pose and shape remains a challenge even for&#10;highly parameterized deep learning models. Mapping from the 2D image space to&#10;the prediction space is difficult: perspective ambiguities make the loss&#10;function noisy and training data is scarce. In this paper, we propose a novel&#10;approach (Neural Body Fitting (NBF)). It integrates a statistical body model&#10;within a CNN, leveraging reliable bottom-up semantic body part segmentation and&#10;robust top-down body model constraints. NBF is fully differentiable and can be&#10;trained using 2D and 3D annotations. In detailed experiments, we analyze how&#10;the components of our model affect performance, especially the use of part&#10;segmentations as an explicit intermediate representation, and present a robust,&#10;efficiently trainable framework for 3D human pose estimation from 2D images&#10;with competitive results on standard benchmarks. Code will be made available at&#10;http://github.com/mohomran/neural_body_fitting&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.16507" label="2103.16507">
-        <attvalues>
-          <attvalue for="0" value="PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment&#10;  Feedback Loop" />
-          <attvalue for="1" value="  Regression-based methods have recently shown promising results in&#10;reconstructing human meshes from monocular images. By directly mapping raw&#10;pixels to model parameters, these methods can produce parametric models in a&#10;feed-forward manner via neural networks. However, minor deviation in parameters&#10;may lead to noticeable misalignment between the estimated meshes and image&#10;evidences. To address this issue, we propose a Pyramidal Mesh Alignment&#10;Feedback (PyMAF) loop to leverage a feature pyramid and rectify the predicted&#10;parameters explicitly based on the mesh-image alignment status in our deep&#10;regressor. In PyMAF, given the currently predicted parameters, mesh-aligned&#10;evidences will be extracted from finer-resolution features accordingly and fed&#10;back for parameter rectification. To reduce noise and enhance the reliability&#10;of these evidences, an auxiliary pixel-wise supervision is imposed on the&#10;feature encoder, which provides mesh-image correspondence guidance for our&#10;network to preserve the most related information in spatial features. The&#10;efficacy of our approach is validated on several benchmarks, including&#10;Human3.6M, 3DPW, LSP, and COCO, where experimental results show that our&#10;approach consistently improves the mesh-image alignment of the reconstruction.&#10;The project page with code and video results can be found at&#10;https://hongwenzhang.github.io/pymaf.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.08527" label="2104.08527">
-        <attvalues>
-          <attvalue for="0" value="PARE: Part Attention Regressor for 3D Human Body Estimation" />
-          <attvalue for="1" value="  Despite significant progress, we show that state of the art 3D human pose and&#10;shape estimation methods remain sensitive to partial occlusion and can produce&#10;dramatically wrong predictions although much of the body is observable. To&#10;address this, we introduce a soft attention mechanism, called the Part&#10;Attention REgressor (PARE), that learns to predict body-part-guided attention&#10;masks. We observe that state-of-the-art methods rely on global feature&#10;representations, making them sensitive to even small occlusions. In contrast,&#10;PARE's part-guided attention mechanism overcomes these issues by exploiting&#10;information about the visibility of individual body parts while leveraging&#10;information from neighboring body-parts to predict occluded parts. We show&#10;qualitatively that PARE learns sensible attention masks, and quantitative&#10;evaluation confirms that PARE achieves more accurate and robust reconstruction&#10;results than existing approaches on both occlusion-specific and standard&#10;benchmarks. The code and data are available for research purposes at {\small&#10;\url{https://pare.is.tue.mpg.de/}}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05866" label="1904.05866">
-        <attvalues>
-          <attvalue for="0" value="Expressive Body Capture: 3D Hands, Face, and Body from a Single Image" />
-          <attvalue for="1" value="  To facilitate the analysis of human actions, interactions and emotions, we&#10;compute a 3D model of human body pose, hand pose, and facial expression from a&#10;single monocular image. To achieve this, we use thousands of 3D scans to train&#10;a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with&#10;fully articulated hands and an expressive face. Learning to regress the&#10;parameters of SMPL-X directly from images is challenging without paired images&#10;and 3D ground truth. Consequently, we follow the approach of SMPLify, which&#10;estimates 2D features and then optimizes model parameters to fit the features.&#10;We improve on SMPLify in several significant ways: (1) we detect 2D features&#10;corresponding to the face, hands, and feet and fit the full SMPL-X model to&#10;these; (2) we train a new neural network pose prior using a large MoCap&#10;dataset; (3) we define a new interpenetration penalty that is both fast and&#10;accurate; (4) we automatically detect gender and the appropriate body models&#10;(male, female, or neutral); (5) our PyTorch implementation achieves a speedup&#10;of more than 8x over Chumpy. We use the new method, SMPLify-X, to fit SMPL-X to&#10;both controlled images and images in the wild. We evaluate 3D accuracy on a new&#10;curated dataset comprising 100 images with pseudo ground-truth. This is a step&#10;towards automatic expressive human capture from monocular RGB data. The models,&#10;code, and data are available for research purposes at&#10;https://smpl-x.is.tue.mpg.de.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.10378" label="2108.10378">
-        <attvalues>
-          <attvalue for="0" value="Lightweight Multi-person Total Motion Capture Using Sparse Multi-view&#10;  Cameras" />
-          <attvalue for="1" value="  Multi-person total motion capture is extremely challenging when it comes to&#10;handle severe occlusions, different reconstruction granularities from body to&#10;face and hands, drastically changing observation scales and fast body&#10;movements. To overcome these challenges above, we contribute a lightweight&#10;total motion capture system for multi-person interactive scenarios using only&#10;sparse multi-view cameras. By contributing a novel hand and face bootstrapping&#10;algorithm, our method is capable of efficient localization and accurate&#10;association of the hands and faces even on severe occluded occasions. We&#10;leverage both pose regression and keypoints detection methods and further&#10;propose a unified two-stage parametric fitting method for achieving&#10;pixel-aligned accuracy. Moreover, for extremely self-occluded poses and close&#10;interactions, a novel feedback mechanism is proposed to propagate the&#10;pixel-aligned reconstructions into the next frame for more accurate&#10;association. Overall, we propose the first light-weight total capture system&#10;and achieves fast, robust and accurate multi-person total motion capture&#10;performance. The results and experiments show that our method achieves more&#10;accurate results than existing methods under sparse-view setups.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1804.06023" label="1804.06023">
-        <attvalues>
-          <attvalue for="0" value="DoubleFusion: Real-time Capture of Human Performances with Inner Body&#10;  Shapes from a Single Depth Sensor" />
-          <attvalue for="1" value="  We propose DoubleFusion, a new real-time system that combines volumetric&#10;dynamic reconstruction with data-driven template fitting to simultaneously&#10;reconstruct detailed geometry, non-rigid motion and the inner human body shape&#10;from a single depth camera. One of the key contributions of this method is a&#10;double layer representation consisting of a complete parametric body shape&#10;inside, and a gradually fused outer surface layer. A pre-defined node graph on&#10;the body surface parameterizes the non-rigid deformations near the body, and a&#10;free-form dynamically changing graph parameterizes the outer surface layer far&#10;from the body, which allows more general reconstruction. We further propose a&#10;joint motion tracking method based on the double layer representation to enable&#10;robust and fast motion tracking performance. Moreover, the inner body shape is&#10;optimized online and forced to fit inside the outer surface layer. Overall, our&#10;method enables increasingly denoised, detailed and complete surface&#10;reconstructions, fast motion tracking performance and plausible inner body&#10;shape reconstruction in real-time. In particular, experiments show improved&#10;fast motion tracking and loop closure performance on more challenging&#10;scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.03858" label="2007.03858">
-        <attvalues>
-          <attvalue for="0" value="PaMIR: Parametric Model-Conditioned Implicit Representation for&#10;  Image-based Human Reconstruction" />
-          <attvalue for="1" value="  Modeling 3D humans accurately and robustly from a single image is very&#10;challenging, and the key for such an ill-posed problem is the 3D representation&#10;of the human models. To overcome the limitations of regular 3D representations,&#10;we propose Parametric Model-Conditioned Implicit Representation (PaMIR), which&#10;combines the parametric body model with the free-form deep implicit function.&#10;In our PaMIR-based reconstruction framework, a novel deep neural network is&#10;proposed to regularize the free-form deep implicit function using the semantic&#10;features of the parametric model, which improves the generalization ability&#10;under the scenarios of challenging poses and various clothing topologies.&#10;Moreover, a novel depth-ambiguity-aware training loss is further integrated to&#10;resolve depth ambiguities and enable successful surface detail reconstruction&#10;with imperfect body reference. Finally, we propose a body reference&#10;optimization method to improve the parametric model estimation accuracy and to&#10;enhance the consistency between the parametric model and the implicit function.&#10;With the PaMIR representation, our framework can be easily extended to&#10;multi-image input scenarios without the need of multi-camera calibration and&#10;pose synchronization. Experimental results demonstrate that our method achieves&#10;state-of-the-art performance for image-based 3D human reconstruction in the&#10;cases of challenging poses and clothing types.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.00261" label="2105.00261">
-        <attvalues>
-          <attvalue for="0" value="DeepMultiCap: Performance Capture of Multiple Characters Using Sparse&#10;  Multiview Cameras" />
-          <attvalue for="1" value="  We propose DeepMultiCap, a novel method for multi-person performance capture&#10;using sparse multi-view cameras. Our method can capture time varying surface&#10;details without the need of using pre-scanned template models. To tackle with&#10;the serious occlusion challenge for close interacting scenes, we combine a&#10;recently proposed pixel-aligned implicit function with parametric model for&#10;robust reconstruction of the invisible surface areas. An effective&#10;attention-aware module is designed to obtain the fine-grained geometry details&#10;from multi-view images, where high-fidelity results can be generated. In&#10;addition to the spatial attention method, for video inputs, we further propose&#10;a novel temporal fusion method to alleviate the noise and temporal&#10;inconsistencies for moving character reconstruction. For quantitative&#10;evaluation, we contribute a high quality multi-person dataset, MultiHuman,&#10;which consists of 150 static scenes with different levels of occlusions and&#10;ground truth 3D human models. Experimental results demonstrate the&#10;state-of-the-art performance of our method and the well generalization to real&#10;multiview video data, which outperforms the prior works by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.02194" label="2206.02194">
-        <attvalues>
-          <attvalue for="0" value="FOF: Learning Fourier Occupancy Field for Monocular Real-time Human&#10;  Reconstruction" />
-          <attvalue for="1" value="  The advent of deep learning has led to significant progress in monocular&#10;human reconstruction. However, existing representations, such as parametric&#10;models, voxel grids, meshes and implicit neural representations, have&#10;difficulties achieving high-quality results and real-time speed at the same&#10;time. In this paper, we propose Fourier Occupancy Field (FOF), a novel&#10;powerful, efficient and flexible 3D representation, for monocular real-time and&#10;accurate human reconstruction. The FOF represents a 3D object with a 2D field&#10;orthogonal to the view direction where at each 2D position the occupancy field&#10;of the object along the view direction is compactly represented with the first&#10;few terms of Fourier series, which retains the topology and neighborhood&#10;relation in the 2D domain. A FOF can be stored as a multi-channel image, which&#10;is compatible with 2D convolutional neural networks and can bridge the gap&#10;between 3D geometries and 2D images. The FOF is very flexible and extensible,&#10;e.g., parametric models can be easily integrated into a FOF as a prior to&#10;generate more robust results. Based on FOF, we design the first 30+FPS&#10;high-fidelity real-time monocular human reconstruction framework. We&#10;demonstrate the potential of FOF on both public dataset and real captured data.&#10;The code will be released for research purposes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.09127" label="2112.09127">
-        <attvalues>
-          <attvalue for="0" value="ICON: Implicit Clothed humans Obtained from Normals" />
-          <attvalue for="1" value="  Current methods for learning realistic and animatable 3D clothed avatars need&#10;either posed 3D scans or 2D images with carefully controlled user poses. In&#10;contrast, our goal is to learn an avatar from only 2D images of people in&#10;unconstrained poses. Given a set of images, our method estimates a detailed 3D&#10;surface from each image and then combines these into an animatable avatar.&#10;Implicit functions are well suited to the first task, as they can capture&#10;details like hair and clothes. Current methods, however, are not robust to&#10;varied human poses and often produce 3D surfaces with broken or disembodied&#10;limbs, missing details, or non-human shapes. The problem is that these methods&#10;use global feature encoders that are sensitive to global pose. To address this,&#10;we propose ICON (&quot;Implicit Clothed humans Obtained from Normals&quot;), which,&#10;instead, uses local features. ICON has two main modules, both of which exploit&#10;the SMPL(-X) body model. First, ICON infers detailed clothed-human normals&#10;(front/back) conditioned on the SMPL(-X) normals. Second, a visibility-aware&#10;implicit surface regressor produces an iso-surface of a human occupancy field.&#10;Importantly, at inference time, a feedback loop alternates between refining the&#10;SMPL(-X) mesh using the inferred clothed normals and then refining the normals.&#10;Given multiple reconstructed frames of a subject in varied poses, we use&#10;SCANimate to produce an animatable avatar from them. Evaluation on the AGORA&#10;and CAPE datasets shows that ICON outperforms the state of the art in&#10;reconstruction, even with heavily limited training data. Additionally, it is&#10;much more robust to out-of-distribution samples, e.g., in-the-wild poses/images&#10;and out-of-frame cropping. ICON takes a step towards robust 3D clothed human&#10;reconstruction from in-the-wild images. This enables creating avatars directly&#10;from video with personalized and natural pose-dependent cloth deformation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.07422" label="2212.07422">
-        <attvalues>
-          <attvalue for="0" value="ECON: Explicit Clothed humans Optimized via Normal integration" />
-          <attvalue for="1" value="  The combination of deep learning, artist-curated scans, and Implicit&#10;Functions (IF), is enabling the creation of detailed, clothed, 3D humans from&#10;images. However, existing methods are far from perfect. IF-based methods&#10;recover free-form geometry, but produce disembodied limbs or degenerate shapes&#10;for novel poses or clothes. To increase robustness for these cases, existing&#10;work uses an explicit parametric body model to constrain surface&#10;reconstruction, but this limits the recovery of free-form surfaces such as&#10;loose clothing that deviates from the body. What we want is a method that&#10;combines the best properties of implicit representation and explicit body&#10;regularization. To this end, we make two key observations: (1) current networks&#10;are better at inferring detailed 2D maps than full-3D surfaces, and (2) a&#10;parametric model can be seen as a &quot;canvas&quot; for stitching together detailed&#10;surface patches. Based on these, our method, ECON, has three main steps: (1) It&#10;infers detailed 2D normal maps for the front and back side of a clothed person.&#10;(2) From these, it recovers 2.5D front and back surfaces, called d-BiNI, that&#10;are equally detailed, yet incomplete, and registers these w.r.t. each other&#10;with the help of a SMPL-X body mesh recovered from the image. (3) It &quot;inpaints&quot;&#10;the missing geometry between d-BiNI surfaces. If the face and hands are noisy,&#10;they can optionally be replaced with the ones of SMPL-X. As a result, ECON&#10;infers high-fidelity 3D humans even in loose clothes and challenging poses.&#10;This goes beyond previous methods, according to the quantitative evaluation on&#10;the CAPE and Renderpeople datasets. Perceptual studies also show that ECON's&#10;perceived realism is better by a large margin. Code and models are available&#10;for research purposes at econ.is.tue.mpg.de&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.15838" label="2012.15838">
-        <attvalues>
-          <attvalue for="0" value="Neural Body: Implicit Neural Representations with Structured Latent&#10;  Codes for Novel View Synthesis of Dynamic Humans" />
-          <attvalue for="1" value="  This paper addresses the challenge of novel view synthesis for a human&#10;performer from a very sparse set of camera views. Some recent works have shown&#10;that learning implicit neural representations of 3D scenes achieves remarkable&#10;view synthesis quality given dense input views. However, the representation&#10;learning will be ill-posed if the views are highly sparse. To solve this&#10;ill-posed problem, our key idea is to integrate observations over video frames.&#10;To this end, we propose Neural Body, a new human body representation which&#10;assumes that the learned neural representations at different frames share the&#10;same set of latent codes anchored to a deformable mesh, so that the&#10;observations across frames can be naturally integrated. The deformable mesh&#10;also provides geometric guidance for the network to learn 3D representations&#10;more efficiently. To evaluate our approach, we create a multi-view dataset&#10;named ZJU-MoCap that captures performers with complex motions. Experiments on&#10;ZJU-MoCap show that our approach outperforms prior works by a large margin in&#10;terms of novel view synthesis quality. We also demonstrate the capability of&#10;our approach to reconstruct a moving person from a monocular video on the&#10;People-Snapshot dataset. The code and dataset are available at&#10;https://zju3dv.github.io/neuralbody/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.10203" label="2112.10203">
-        <attvalues>
-          <attvalue for="0" value="HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars" />
-          <attvalue for="1" value="  We propose a novel neural rendering pipeline, Hybrid Volumetric-Textural&#10;Rendering (HVTR), which synthesizes virtual human avatars from arbitrary poses&#10;efficiently and at high quality. First, we learn to encode articulated human&#10;motions on a dense UV manifold of the human body surface. To handle complicated&#10;motions (e.g., self-occlusions), we then leverage the encoded information on&#10;the UV manifold to construct a 3D volumetric representation based on a dynamic&#10;pose-conditioned neural radiance field. While this allows us to represent 3D&#10;geometry with changing topology, volumetric rendering is computationally heavy.&#10;Hence we employ only a rough volumetric representation using a pose-conditioned&#10;downsampled neural radiance field (PD-NeRF), which we can render efficiently at&#10;low resolutions. In addition, we learn 2D textural features that are fused with&#10;rendered volumetric features in image space. The key advantage of our approach&#10;is that we can then convert the fused features into a high-resolution,&#10;high-quality avatar by a fast GAN-based textural renderer. We demonstrate that&#10;hybrid rendering enables HVTR to handle complicated motions, render&#10;high-quality avatars under user-controlled poses/shapes and even loose&#10;clothing, and most importantly, be efficient at inference time. Our&#10;experimental results also demonstrate state-of-the-art quantitative results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.04572" label="2004.04572">
-        <attvalues>
-          <attvalue for="0" value="ARCH: Animatable Reconstruction of Clothed Humans" />
-          <attvalue for="1" value="  In this paper, we propose ARCH (Animatable Reconstruction of Clothed Humans),&#10;a novel end-to-end framework for accurate reconstruction of animation-ready 3D&#10;clothed humans from a monocular image. Existing approaches to digitize 3D&#10;humans struggle to handle pose variations and recover details. Also, they do&#10;not produce models that are animation ready. In contrast, ARCH is a learned&#10;pose-aware model that produces detailed 3D rigged full-body human avatars from&#10;a single unconstrained RGB image. A Semantic Space and a Semantic Deformation&#10;Field are created using a parametric 3D body estimator. They allow the&#10;transformation of 2D/3D clothed humans into a canonical space, reducing&#10;ambiguities in geometry caused by pose variations and occlusions in training&#10;data. Detailed surface geometry and appearance are learned using an implicit&#10;function representation with spatial local features. Furthermore, we propose&#10;additional per-pixel supervision on the 3D reconstruction using opacity-aware&#10;differentiable rendering. Our experiments indicate that ARCH increases the&#10;fidelity of the reconstructed humans. We obtain more than 50% lower&#10;reconstruction errors for standard metrics compared to state-of-the-art methods&#10;on public datasets. We also show numerous qualitative examples of animated,&#10;high-quality reconstructed avatars unseen in the literature so far.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.01137" label="2109.01137">
-        <attvalues>
-          <attvalue for="0" value="The Power of Points for Modeling Humans in Clothing" />
-          <attvalue for="1" value="  Currently it requires an artist to create 3D human avatars with realistic&#10;clothing that can move naturally. Despite progress on 3D scanning and modeling&#10;of human bodies, there is still no technology that can easily turn a static&#10;scan into an animatable avatar. Automating the creation of such avatars would&#10;enable many applications in games, social networking, animation, and AR/VR to&#10;name a few. The key problem is one of representation. Standard 3D meshes are&#10;widely used in modeling the minimally-clothed body but do not readily capture&#10;the complex topology of clothing. Recent interest has shifted to implicit&#10;surface models for this task but they are computationally heavy and lack&#10;compatibility with existing 3D tools. What is needed is a 3D representation&#10;that can capture varied topology at high resolution and that can be learned&#10;from data. We argue that this representation has been with us all along -- the&#10;point cloud. Point clouds have properties of both implicit and explicit&#10;representations that we exploit to model 3D garment geometry on a human body.&#10;We train a neural network with a novel local clothing geometric feature to&#10;represent the shape of different outfits. The network is trained from 3D point&#10;clouds of many types of clothing, on many bodies, in many poses, and learns to&#10;model pose-dependent clothing deformations. The geometry feature can be&#10;optimized to fit a previously unseen scan of a person in clothing, enabling the&#10;scan to be reposed realistically. Our model demonstrates superior quantitative&#10;and qualitative results in both multi-outfit modeling and unseen outfit&#10;animation. The code is available for research purposes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.14478" label="2203.14478">
-        <attvalues>
-          <attvalue for="0" value="Structured Local Radiance Fields for Human Avatar Modeling" />
-          <attvalue for="1" value="  It is extremely challenging to create an animatable clothed human avatar from&#10;RGB videos, especially for loose clothes due to the difficulties in motion&#10;modeling. To address this problem, we introduce a novel representation on the&#10;basis of recent neural scene rendering techniques. The core of our&#10;representation is a set of structured local radiance fields, which are anchored&#10;to the pre-defined nodes sampled on a statistical human body template. These&#10;local radiance fields not only leverage the flexibility of implicit&#10;representation in shape and appearance modeling, but also factorize cloth&#10;deformations into skeleton motions, node residual translations and the dynamic&#10;detail variations inside each individual radiance field. To learn our&#10;representation from RGB data and facilitate pose generalization, we propose to&#10;learn the node translations and the detail variations in a conditional&#10;generative latent space. Overall, our method enables automatic construction of&#10;animatable human avatars for various types of clothes without the need for&#10;scanning subject-specific templates, and can generate realistic images with&#10;dynamic details for novel poses. Experiment show that our method outperforms&#10;state-of-the-art methods both qualitatively and quantitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.04789" label="2305.04789">
-        <attvalues>
-          <attvalue for="0" value="AvatarReX: Real-time Expressive Full-body Avatars" />
-          <attvalue for="1" value="  We present AvatarReX, a new method for learning NeRF-based full-body avatars&#10;from video data. The learnt avatar not only provides expressive control of the&#10;body, hands and the face together, but also supports real-time animation and&#10;rendering. To this end, we propose a compositional avatar representation, where&#10;the body, hands and the face are separately modeled in a way that the&#10;structural prior from parametric mesh templates is properly utilized without&#10;compromising representation flexibility. Furthermore, we disentangle the&#10;geometry and appearance for each part. With these technical designs, we propose&#10;a dedicated deferred rendering pipeline, which can be executed in real-time&#10;framerate to synthesize high-quality free-view images. The disentanglement of&#10;geometry and appearance also allows us to design a two-pass training strategy&#10;that combines volume rendering and surface rendering for network training. In&#10;this way, patch-level supervision can be applied to force the network to learn&#10;sharp appearance details on the basis of geometry estimation. Overall, our&#10;method enables automatic construction of expressive full-body avatars with&#10;real-time rendering capability, and can generate photo-realistic images with&#10;dynamic details for novel body motions and facial expressions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.02468" label="1701.02468">
-        <attvalues>
-          <attvalue for="0" value="Unite the People: Closing the Loop Between 3D and 2D Human&#10;  Representations" />
-          <attvalue for="1" value="  3D models provide a common ground for different representations of human&#10;bodies. In turn, robust 2D estimation has proven to be a powerful tool to&#10;obtain 3D fits &quot;in-the- wild&quot;. However, depending on the level of detail, it&#10;can be hard to impossible to acquire labeled data for training 2D estimators on&#10;large scale. We propose a hybrid approach to this problem: with an extended&#10;version of the recently introduced SMPLify method, we obtain high quality 3D&#10;body model fits for multiple human pose datasets. Human annotators solely sort&#10;good and bad fits. This procedure leads to an initial dataset, UP-3D, with rich&#10;annotations. With a comprehensive set of experiments, we show how this data can&#10;be used to train discriminative models that produce results with an&#10;unprecedented level of detail: our models predict 31 segments and 91 landmark&#10;locations on the body. Using the 91 landmark pose estimator, we present&#10;state-of-the art results for 3D human pose and shape estimation using an order&#10;of magnitude less training data and without assumptions about gender or pose in&#10;the fitting procedure. We show that UP-3D can be enhanced with these improved&#10;fits to grow in quantity and quality, which makes the system deployable on&#10;large scale. The data, code and models are available for research purposes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.01337" label="1712.01337">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Learning of Motion Capture" />
-          <attvalue for="1" value="  Current state-of-the-art solutions for motion capture from a single camera&#10;are optimization driven: they optimize the parameters of a 3D human model so&#10;that its re-projection matches measurements in the video (e.g. person&#10;segmentation, optical flow, keypoint detections etc.). Optimization models are&#10;susceptible to local minima. This has been the bottleneck that forced using&#10;clean green-screen like backgrounds at capture time, manual initialization, or&#10;switching to multiple cameras as input resource. In this work, we propose a&#10;learning based motion capture model for single camera input. Instead of&#10;optimizing mesh and skeleton parameters directly, our model optimizes neural&#10;network weights that predict 3D shape and skeleton configurations given a&#10;monocular RGB video. Our model is trained using a combination of strong&#10;supervision from synthetic data, and self-supervision from differentiable&#10;rendering of (a) skeletal keypoints, (b) dense 3D mesh motion, and (c)&#10;human-background segmentation, in an end-to-end framework. Empirically we show&#10;our model combines the best of both worlds of supervised learning and test-time&#10;optimization: supervised learning initializes the model parameters in the right&#10;regime, ensuring good pose and surface initialization at test time, without&#10;manual effort. Self-supervision by back-propagating through differentiable&#10;rendering allows (unsupervised) adaptation of the model to the test data, and&#10;offers much tighter fit than a pretrained fixed model. We show that the&#10;proposed model improves with experience and converges to low-error solutions&#10;where previous optimization methods fail.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.13794" label="2303.13794">
-        <attvalues>
-          <attvalue for="0" value="Efficient and Accurate Co-Visible Region Localization with Matching&#10;  Key-Points Crop (MKPC): A Two-Stage Pipeline for Enhancing Image Matching&#10;  Performance" />
-          <attvalue for="1" value="  Image matching is a classic and fundamental task in computer vision. In this&#10;paper, under the hypothesis that the areas outside the co-visible regions carry&#10;little information, we propose a matching key-points crop (MKPC) algorithm. The&#10;MKPC locates, proposes and crops the critical regions, which are the co-visible&#10;areas with great efficiency and accuracy. Furthermore, building upon MKPC, we&#10;propose a general two-stage pipeline for image matching, which is compatible to&#10;any image matching models or combinations. We experimented with plugging&#10;SuperPoint + SuperGlue into the two-stage pipeline, whose results show that our&#10;method enhances the performance for outdoor pose estimations. What's more, in a&#10;fair comparative condition, our method outperforms the SOTA on Image Matching&#10;Challenge 2022 Benchmark, which represents the hardest outdoor benchmark of&#10;image matching currently.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image matching is a fundamental and classic task in computer vision, playing a crucial role in various downstream tasks such as Simultaneous Localization and Mapping (SLAM), Structure-from-Motion (SfM), and Localization. In the task of image matching, the objective is to establish the relationship between two given images by employing methods such as feature point extraction~\cite{detone2018superpoint, tyszkiewicz2020disk}, feature point description~\cite{detone2018superpoint, tyszkiewicz2020disk}, and feature point matching~\cite{sarlin2020superglue}. This relationship can be described using a fundamental matrix, an essential matrix, or a homography matrix. Recently, deep learning models have been developed to implement end-to-end keypoint matching, capable of outputting matched key-points by simply inputting two images~\cite{edstedt2022deep, sun2021loftr, tang2022quadtree, chen2022aspanformer}.&#10;&#10;While most current works focus on interest point detectors~\cite{detone2018superpoint, tyszkiewicz2020disk}, descriptors~\cite{detone2018superpoint, tyszkiewicz2020disk}, matching, or implementing in an end-to-end way~\cite{edstedt2022deep, sun2021loftr, tang2022quadtree, chen2022aspanformer}. we found that the input images have a significant impact on the performance of image matching. Essentially, improving the performance of image matching primarily involves reducing the number of incorrect matches while increasing the number of correct matches. Under the assumption that areas outside the co-visible regions contain little valuable information and the matching points outside these regions are incorrect matches, we define the co-visible area of two images as the &quot;critical region&quot; (Figure \ref{demo}). By focusing on the critical region as the matching target, we aim to enhance image matching performance. Identifying the critical regions shared between two images and applying fine-grained matching within these regions offers two significant advantages: (i) it refines the matching of critical regions and (ii) it rejects the majority of outliers or noisy points. Based on these considerations, we propose a simple but effective algorithm in image matching, called Matching Key-Points Crop (MKPC). The essence of MKPC is a DBSCAN clustering algorithm. The MKPC algorithm leverages matching key-points obtained from arbitrary matching models to efficiently and accurately crop the critical regions of both images with great efficiency and remarkable accuracy, while the key-points produced by upstream models are representative. &#10;&#10;Building upon MKPC, we propose a two-stage pipeline for image matching that can be easily extended for any image matching models while preserving superior performance. In the first stage, arbitrary image matching models generate matching points for the two images. Subsequently, the MKPC algorithm clusters these matching points and proposes a Region of Interest (ROI) area, which represents the critical regions for both images. A naive array slicing operation crops the critical regions, which are then fed into arbitrary image matching models after necessary pre-processing. This constitutes the second stage of our two-stage pipeline. Our experiments demonstrate that the two-stage pipeline with MKPC as the core consistently improves image matching performance on the PhotoTourism dataset, which is a subset of YFCC100M dataset~\cite{thomee2016yfcc100m} with ground truth poses. The experimental results not only validate the effectiveness of our proposed method, including MKPC and the two-stage pipeline but also support the theoretical foundation that the key to enhancing image matching performance lies in accurately locating the co-visible areas. Although our method is also applicable to indoor scenarios, it requires higher computational complexity. Furthermore, our approach outperforms the state-of-the-art (SOTA) on the Image Matching Challenge 2022 benchmark, which is considered the most challenging outdoor pose estimation dataset.&#10;&#10;The contributions of this paper are as follows: (i) We propose that the key to enhancing the performance of image matching lies in identifying critical regions, or the co-visible areas in both images. (ii) We introduce Matching Key-points Crop(MKPC), a critical region finder that efficiently crops these areas with remarkable accuracy, and (iii) Building upon MKPC, we present a two-stage pipeline that enhances the effectiveness of arbitrary image matching models for outdoor pose estimations. In addition, our proposed two-stage pipeline achieved a higher score than the SOTA score on the Image Matching Challenge 2022 benchmark, which represents the most difficult benchmark for outdoor estimation currently.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Pose Estimation, Image Matching, Mathematics, Artificial Intelligence, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1712.07629" label="1712.07629">
-        <attvalues>
-          <attvalue for="0" value="SuperPoint: Self-Supervised Interest Point Detection and Description" />
-          <attvalue for="1" value="  This paper presents a self-supervised framework for training interest point&#10;detectors and descriptors suitable for a large number of multiple-view geometry&#10;problems in computer vision. As opposed to patch-based neural networks, our&#10;fully-convolutional model operates on full-sized images and jointly computes&#10;pixel-level interest point locations and associated descriptors in one forward&#10;pass. We introduce Homographic Adaptation, a multi-scale, multi-homography&#10;approach for boosting interest point detection repeatability and performing&#10;cross-domain adaptation (e.g., synthetic-to-real). Our model, when trained on&#10;the MS-COCO generic image dataset using Homographic Adaptation, is able to&#10;repeatedly detect a much richer set of interest points than the initial&#10;pre-adapted deep model and any other traditional corner detector. The final&#10;system gives rise to state-of-the-art homography estimation results on HPatches&#10;when compared to LIFT, SIFT and ORB.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.11763" label="1911.11763">
-        <attvalues>
-          <attvalue for="0" value="SuperGlue: Learning Feature Matching with Graph Neural Networks" />
-          <attvalue for="1" value="  This paper introduces SuperGlue, a neural network that matches two sets of&#10;local features by jointly finding correspondences and rejecting non-matchable&#10;points. Assignments are estimated by solving a differentiable optimal transport&#10;problem, whose costs are predicted by a graph neural network. We introduce a&#10;flexible context aggregation mechanism based on attention, enabling SuperGlue&#10;to reason about the underlying 3D scene and feature assignments jointly.&#10;Compared to traditional, hand-designed heuristics, our technique learns priors&#10;over geometric transformations and regularities of the 3D world through&#10;end-to-end training from image pairs. SuperGlue outperforms other learned&#10;approaches and achieves state-of-the-art results on the task of pose estimation&#10;in challenging real-world indoor and outdoor environments. The proposed method&#10;performs matching in real-time on a modern GPU and can be readily integrated&#10;into modern SfM or SLAM systems. The code and trained weights are publicly&#10;available at https://github.com/magicleap/SuperGluePretrainedNetwork.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.01817" label="1503.01817">
-        <attvalues>
-          <attvalue for="0" value="YFCC100M: The New Data in Multimedia Research" />
-          <attvalue for="1" value="  We present the Yahoo Flickr Creative Commons 100 Million Dataset (YFCC100M),&#10;the largest public multimedia collection that has ever been released. The&#10;dataset contains a total of 100 million media objects, of which approximately&#10;99.2 million are photos and 0.8 million are videos, all of which carry a&#10;Creative Commons license. Each media object in the dataset is represented by&#10;several pieces of metadata, e.g. Flickr identifier, owner name, camera, title,&#10;tags, geo, media source. The collection provides a comprehensive snapshot of&#10;how photos and videos were taken, described, and shared over the years, from&#10;the inception of Flickr in 2004 until early 2014. In this article we explain&#10;the rationale behind its creation, as well as the implications the dataset has&#10;for science, research, engineering, and development. We further present several&#10;new challenges in multimedia research that can now be expanded upon with our&#10;dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.03367" label="2011.03367">
-        <attvalues>
-          <attvalue for="0" value="Disentangling 3D Prototypical Networks For Few-Shot Concept Learning" />
-          <attvalue for="1" value="  We present neural architectures that disentangle RGB-D images into objects'&#10;shapes and styles and a map of the background scene, and explore their&#10;applications for few-shot 3D object detection and few-shot concept&#10;classification. Our networks incorporate architectural biases that reflect the&#10;image formation process, 3D geometry of the world scene, and shape-style&#10;interplay. They are trained end-to-end self-supervised by predicting views in&#10;static scenes, alongside a small number of 3D object boxes. Objects and scenes&#10;are represented in terms of 3D feature grids in the bottleneck of the network.&#10;We show that the proposed 3D neural representations are compositional: they can&#10;generate novel 3D scene feature maps by mixing object shapes and styles,&#10;resizing and adding the resulting object 3D feature maps over background scene&#10;feature maps. We show that classifiers for object categories, color, materials,&#10;and spatial relationships trained over the disentangled 3D feature sub-spaces&#10;generalize better with dramatically fewer examples than the current&#10;state-of-the-art, and enable a visual question answering system that uses them&#10;as its modules to generalize one-shot to novel objects in the scene.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Humans can learn new concepts from just one or a few samples. Consider the example in Figure 1. Assuming there is a person who has no prior knowledge about {blue} and {carrot}, by showing this person an image of a blue carrot and telling him ``this is an {carrot} with {blue} color'', the person can easily generalize from this example to (1) recognizing {carrots} of varying colors, 3D poses and viewing conditions and under novel background scenes, (2) recognizing the color {blue} on different objects, (3) combine these two concepts with other concepts to form a novel object coloring he/she has never seen before, e.g., red carrot or blue tomato and (4) using the newly learned concepts to answer questions regarding the visual scene.&#10;Motivated by this, we explore computational models that can achieve these four types of generalization for visual concept learning.&#10;&#10;We propose disentangling 3D prototypical networks (\model), a model that learns to disentangle RGB-D images into objects, their 3D locations, sizes, 3D shapes and styles, and the background scene, as shown in Figure \ref{fig:model}. Our model &#10;can learn to detect objects from a few 3D object bounding box annotations and can further disentangle objects into different attributes through a self-supervised view prediction task. Specifically, \model~ uses differentiable unprojection and rendering operations to go back and forth between the input RGB-D (2.5D) image and a 3D scene feature map. From the scene feature map, our model learns to detect objects and disentangles each object into a 3D shape code and an 1D style code through a shape/style disentangling antoencoder. We use adaptive instance normalization layers \cite{DBLP:journals/corr/HuangB17} to encourage shape/style disentanglement within each object. Our key intuition is to represent objects and their shapes in terms of {3D feature representations disentangled from style variability} so that the model can correspond objects with similar shape by explicitly rotating and scaling their 3D shape representations during matching. &#10;&#10;With the disentangled representations, \model~can recognize new concepts regarding object shapes, styles and spatial arrangements from a few human-supplied labels by training concept classifiers only on the relevant feature subspace. Our model learns object shapes on shape codes, object colors and textures on style codes, and object spatial arrangements on object 3D locations. We show in the supplementary how the features relevant for each linguistic concept can be inferred from a few contrasting examples. Thus the classifiers attend only to the essential property of the concept and ignore irrelevant visual features. This allows them to generalize with far fewer examples and can recognize novel attribute compositions not present in the training data. &#10;&#10;We test \model~in few-shot concept learning, visual question answering (VQA) and scene generation. We train concept classifiers for object shapes, object colors/materials, and spatial relationships on our inferred disentangled feature spaces, and show they outperform current state-of-the-art \cite{Mao2019NeuroSymbolic,DBLP:journals/corr/HuRADS16}, which use 2D representations. We show that a VQA modular network that incorporates our concept classifiers shows improved generalization over the state-of-the-art \cite{Mao2019NeuroSymbolic} with dramatically fewer examples.&#10;Last, we empirically show that \model~generalize their view predictions to scenes with novel number, category and styles of objects, and compare against state-of-the-art view predictive architectures of \cite{Eslami1204}.&#10;&#10;The main contribution of this paper is to identify the importance of using disentangled 3D feature representations for few-shot concept learning. We show the disentangled 3D feature representations can be learned using self-supervised view prediction, and they are useful for detecting and classifying language concepts by training them over the relevant only feature subsets. The proposed model outperforms the current state-of-the-art in VQA in the low data regime and the proposed 3D disentangled representation outperforms similar 2D or 2.5D ones in few-shot concept classification. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Neural Network Architectures, Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Artificial Intelligence, Physics, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1703.06868" label="1703.06868">
-        <attvalues>
-          <attvalue for="0" value="Arbitrary Style Transfer in Real-time with Adaptive Instance&#10;  Normalization" />
-          <attvalue for="1" value="  Gatys et al. recently introduced a neural algorithm that renders a content&#10;image in the style of another image, achieving so-called style transfer.&#10;However, their framework requires a slow iterative optimization process, which&#10;limits its practical application. Fast approximations with feed-forward neural&#10;networks have been proposed to speed up neural style transfer. Unfortunately,&#10;the speed improvement comes at a cost: the network is usually tied to a fixed&#10;set of styles and cannot adapt to arbitrary new styles. In this paper, we&#10;present a simple yet effective approach that for the first time enables&#10;arbitrary style transfer in real-time. At the heart of our method is a novel&#10;adaptive instance normalization (AdaIN) layer that aligns the mean and variance&#10;of the content features with those of the style features. Our method achieves&#10;speed comparable to the fastest existing approach, without the restriction to a&#10;pre-defined set of styles. In addition, our approach allows flexible user&#10;controls such as content-style trade-off, style interpolation, color &amp; spatial&#10;controls, all using a single feed-forward neural network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.12584" label="1904.12584">
-        <attvalues>
-          <attvalue for="0" value="The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and&#10;  Sentences From Natural Supervision" />
-          <attvalue for="1" value="  We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns&#10;visual concepts, words, and semantic parsing of sentences without explicit&#10;supervision on any of them; instead, our model learns by simply looking at&#10;images and reading paired questions and answers. Our model builds an&#10;object-based scene representation and translates sentences into executable,&#10;symbolic programs. To bridge the learning of two modules, we use a&#10;neuro-symbolic reasoning module that executes these programs on the latent&#10;scene representation. Analogical to human concept learning, the perception&#10;module learns visual concepts based on the language description of the object&#10;being referred to. Meanwhile, the learned visual concepts facilitate learning&#10;new words and parsing new sentences. We use curriculum learning to guide the&#10;searching over the large compositional space of images and language. Extensive&#10;experiments demonstrate the accuracy and efficiency of our model on learning&#10;visual concepts, word representations, and semantic parsing of sentences.&#10;Further, our method allows easy generalization to new object attributes,&#10;compositions, language concepts, scenes and questions, and even new program&#10;domains. It also empowers applications including visual question answering and&#10;bidirectional image-text retrieval.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.09978" label="1611.09978">
-        <attvalues>
-          <attvalue for="0" value="Modeling Relationships in Referential Expressions with Compositional&#10;  Modular Networks" />
-          <attvalue for="1" value="  People often refer to entities in an image in terms of their relationships&#10;with other entities. For example, &quot;the black cat sitting under the table&quot;&#10;refers to both a &quot;black cat&quot; entity and its relationship with another &quot;table&quot;&#10;entity. Understanding these relationships is essential for interpreting and&#10;grounding such natural language expressions. Most prior work focuses on either&#10;grounding entire referential expressions holistically to one region, or&#10;localizing relationships based on a fixed set of categories. In this paper we&#10;instead present a modular deep architecture capable of analyzing referential&#10;expressions into their component parts, identifying entities and relationships&#10;mentioned in the input expression and grounding them all in the scene. We call&#10;this approach Compositional Modular Networks (CMNs): a novel architecture that&#10;learns linguistic analysis and visual inference end-to-end. Our approach is&#10;built around two types of neural modules that inspect local regions and&#10;pairwise interactions between regions. We evaluate CMNs on multiple referential&#10;expression datasets, outperforming state-of-the-art approaches on all tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.01203" label="1909.01203">
-        <attvalues>
-          <attvalue for="0" value="Cross View Fusion for 3D Human Pose Estimation" />
-          <attvalue for="1" value="  We present an approach to recover absolute 3D human poses from multi-view&#10;images by incorporating multi-view geometric priors in our model. It consists&#10;of two separate steps: (1) estimating the 2D poses in multi-view images and (2)&#10;recovering the 3D poses from the multi-view 2D poses. First, we introduce a&#10;cross-view fusion scheme into CNN to jointly estimate 2D poses for multiple&#10;views. Consequently, the 2D pose estimation for each view already benefits from&#10;other views. Second, we present a recursive Pictorial Structure Model to&#10;recover the 3D pose from the multi-view 2D poses. It gradually improves the&#10;accuracy of 3D pose with affordable computational cost. We test our method on&#10;two public datasets H36M and Total Capture. The Mean Per Joint Position Errors&#10;on the two datasets are 26mm and 29mm, which outperforms the state-of-the-arts&#10;remarkably (26mm vs 52mm, 29mm vs 35mm). Our code is released at&#10;\url{https://github.com/microsoft/multiview-human-pose-estimation-pytorch}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;We first review the related work on multi-view $3$D pose estimation and discuss how they differ from our work. Then we discuss some techniques on feature fusion. &#10;&#10;Multi-view $3$D Pose Estimation Many approaches \cite{liu2011markerless,gall2010optimization,burenius20133D,PavlakosZDD17,belagiannis20143D,Rhodin_2018_ECCV,rhodin2018learning} are proposed for multi-view pose estimation. They first define a body model represented as simple primitives, and then optimize the model parameters to align the projections of the body model with the image features. These approaches differ in terms of the used image features and optimization algorithms.&#10;&#10;We focus on the Pictorial Structure Model (PSM) which is widely used in object detection \cite{felzenszwalb2005pictorial,fischler1973representation} to model the spatial dependence between the object parts. This technique is also used for $2$D \cite{yang2011articulated,chen2014articulated,amin2013multi} and $3$D \cite{burenius20133D,PavlakosZDD17} pose estimation where the parts are the body joints or limbs. In \cite{amin2013multi}, Amin \etal first estimate the $2$D poses in a multi-view setup with PSM and then obtain the $3$D poses by direct triangulation. Later Burenius \etal \cite{burenius20133D} and Pavlakos \etal \cite{PavlakosZDD17} extend PSM to multi-view $3$D human pose estimation. For example, in \cite{PavlakosZDD17}, they first estimate $2$D poses independently for each view and then recover the $3$D pose using PSM. Our work differs from \cite{PavlakosZDD17} in that we extend PSM to a recursive version, \ie RPSM, which efficiently refines the 3D pose estimations step by step. In addition, they \cite{PavlakosZDD17} do not perform cross-view feature fusion as we do.&#10;&#10;Multi-image Feature Fusion Fusing features from different sources is a common practice in the computer vision literature. For example, in \cite{zhu2017flow}, Zhu \etal propose to warp the features of the neighboring frames (in a video sequence) to the current frame according to optical flow in order to robustly detect the objects. Ding \etal \cite{Ding_2018_CVPR} propose to aggregate the multi-scale features which achieves better segmentation accuracy for both large and small objects. Amin \etal \cite{amin2013multi} propose to estimate $2$D poses by exploring the geometric relation between multi-view images. It differs from our work in that it does not fuse features from other views to obtain better $2$D heatmaps. Instead, they use the multi-view $3$D geometric relation to select the joint locations from the ``imperfect'' heatmaps. In \cite{jafarian2018monet}, multi-view consistency is used as a source of supervision to train the pose estimation network. To the best of our knowledge, there is no previous work which fuses multi-view features so as to obtain better $2$D pose heatmaps because it is a challenging task to find the corresponding features across different views which is one of our key contributions of this work.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Human Pose Estimation, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2309.14865" label="2309.14865">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Multi-Person 3D Human Pose Estimation From 2D Poses Alone" />
-          <attvalue for="1" value="  Current unsupervised 2D-3D human pose estimation (HPE) methods do not work in&#10;multi-person scenarios due to perspective ambiguity in monocular images.&#10;Therefore, we present one of the first studies investigating the feasibility of&#10;unsupervised multi-person 2D-3D HPE from just 2D poses alone, focusing on&#10;reconstructing human interactions. To address the issue of perspective&#10;ambiguity, we expand upon prior work by predicting the cameras' elevation angle&#10;relative to the subjects' pelvis. This allows us to rotate the predicted poses&#10;to be level with the ground plane, while obtaining an estimate for the vertical&#10;offset in 3D between individuals. Our method involves independently lifting&#10;each subject's 2D pose to 3D, before combining them in a shared 3D coordinate&#10;system. The poses are then rotated and offset by the predicted elevation angle&#10;before being scaled. This by itself enables us to retrieve an accurate 3D&#10;reconstruction of their poses. We present our results on the CHI3D dataset,&#10;introducing its use for unsupervised 2D-3D pose estimation with three new&#10;quantitative metrics, and establishing a benchmark for future research.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Monocular 3D human pose estimation (HPE) is known to be an ill-posed inverse problem, as multiple different 2D poses can correspond to the same 3D pose. Despite this, unsupervised algorithms have developed rapidly in tackling single-person 2D-3D HPE from a single image, where they attempt to lift a 2D skeleton to 3D via some form of reprojected 2D pose likelihood \cite{hardy, Wandt_2022_CVPR, amazon_paper_2, Yu_2021_ICCV, Drover_2018_ECCV_Amazon}. Due to fundamental perspective ambiguity however, the absolute metric of depth cannot be obtained from a single view alone \cite{amazon_paper_2, depth_impossible}. To deal with this, unsupervised approaches centre the detected 2D pose around a root joint (typically the pelvis), while also setting the 3D prediction of the root to be a fixed unit of $c$ from the camera. This means that instead of absolute depth, these models learn to predict the 3D depth offset from the root joint when the root is assumed to be $c$ units from the camera. Although this works well in scenarios where we want to lift a single person, if we adapt this approach to lift multiple people simultaneously we obtain errors both in terms of the pose, as well as in the 3D distance between the two people as seen in Fig. \ref{current_problems}. Therefore, the aim of this paper is to obtain an accurate reconstruction of 3D human poses interacting within a shared coordinate system, relying solely on their 2D poses obtained from a single image. Additionally, in our extensive survey of prior literature, we found no prior work tackling unsupervised multi-person 2D-3D HPE from 2D poses alone. Therefore, we take the first leap in exploring if it is feasible to reconstruct an accurate 3D estimate of two people interacting from their 2D poses alone. &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1904.04812" label="1904.04812">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised 3D Pose Estimation with Geometric Self-Supervision" />
-          <attvalue for="1" value="  We present an unsupervised learning approach to recover 3D human pose from 2D&#10;skeletal joints extracted from a single image. Our method does not require any&#10;multi-view image data, 3D skeletons, correspondences between 2D-3D points, or&#10;use previously learned 3D priors during training. A lifting network accepts 2D&#10;landmarks as inputs and generates a corresponding 3D skeleton estimate. During&#10;training, the recovered 3D skeleton is reprojected on random camera viewpoints&#10;to generate new &quot;synthetic&quot; 2D poses. By lifting the synthetic 2D poses back to&#10;3D and re-projecting them in the original camera view, we can define&#10;self-consistency loss both in 3D and in 2D. The training can thus be self&#10;supervised by exploiting the geometric self-consistency of the&#10;lift-reproject-lift process. We show that self-consistency alone is not&#10;sufficient to generate realistic skeletons, however adding a 2D pose&#10;discriminator enables the lifter to output valid 3D poses. Additionally, to&#10;learn from 2D poses &quot;in the wild&quot;, we train an unsupervised 2D domain adapter&#10;network to allow for an expansion of 2D data. This improves results and&#10;demonstrates the usefulness of 2D pose data for unsupervised 3D lifting.&#10;Results on Human3.6M dataset for 3D human pose estimation demonstrate that our&#10;approach improves upon the previous unsupervised methods by 30% and outperforms&#10;many weakly supervised approaches that explicitly use 3D data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.02743" label="2012.02743">
-        <attvalues>
-          <attvalue for="0" value="SMPLy Benchmarking 3D Human Pose Estimation in the Wild" />
-          <attvalue for="1" value="  Predicting 3D human pose from images has seen great recent improvements.&#10;Novel approaches that can even predict both pose and shape from a single input&#10;image have been introduced, often relying on a parametric model of the human&#10;body such as SMPL. While qualitative results for such methods are often shown&#10;for images captured in-the-wild, a proper benchmark in such conditions is still&#10;missing, as it is cumbersome to obtain ground-truth 3D poses elsewhere than in&#10;a motion capture room. This paper presents a pipeline to easily produce and&#10;validate such a dataset with accurate ground-truth, with which we benchmark&#10;recent 3D human pose estimation methods in-the-wild. We make use of the&#10;recently introduced Mannequin Challenge dataset which contains in-the-wild&#10;videos of people frozen in action like statues and leverage the fact that&#10;people are static and the camera moving to accurately fit the SMPL model on the&#10;sequences. A total of 24,428 frames with registered body models are then&#10;selected from 567 scenes at almost no cost, using only online RGB videos. We&#10;benchmark state-of-the-art SMPL-based human pose estimation methods on this&#10;dataset. Our results highlight that challenges remain, in particular for&#10;difficult poses or for scenes where the persons are partially truncated or&#10;occluded.&#10;" />
-          <attvalue for="2" value=" &#10;&#10;Human pose estimation is an important computer vision problem with many possible applications in robotics, virtual/augmented reality or human-computer interactions.&#10;Primarily driven by the availability (or absence) of training data, the problem has been originally tackled either as 2D pose estimation in-the-wild~\cite{mpii} or 3D pose estimation in either synthetic~\cite{unite} or constrained scenarios~\cite{h36,totalcapture}, \eg using Motion Capture (MoCap) rooms.&#10;Recently, some works have managed to predict 3D poses for in-the-wild images, and impressively, various approaches can even predict the 3D human shape from a single image. &#10;Most of these methods rely on a parametric model of the human body such as SMPL~\cite{smpl}, and estimate the parameters that control pose and shape deformations of the model~\cite{hmr,unite,graphcmr,texturepose,dct,zanfir2018deep}. &#10;A few recent works make direct predictions of the human 3D shape without any parametric representation~\cite{moulding,zhu2019detailed,deephuman}. &#10;The task of human 3D pose and shape estimation is mostly evaluated on datasets captured in a constrained environment~\cite{humaneva,h36}, and only qualitative results are usually shown for a few images captured in the wild. &#10;Even if some efforts have been made to produce outdoor evaluation datasets~\cite{mupots,3dpw,unite}, one can argue that the accuracy of the ground truth is not sufficient or that the variety in terms of scenes, subjects, and backgrounds is not large enough to properly benchmark state-of-the-art methods. &#10;Therefore, it is difficult to understand what the current state of 3D human pose estimation is, in particular in-the-wild, and what challenges still need to be addressed.&#10;&#10;To answer these questions, we present a new dataset of images captured in-the-wild associated with accurate ground-truth body poses.&#10;Some approaches were proposed to annotate images with the SMPL model constraining its optimization with 2D pose detections combined with manual curation~\cite{unite}, Inertial Measurement Units (IMUs)~\cite{3dpw} or using multiple views~\cite{HuangBLKGRAB17}.&#10;One way to easily obtain many views of a person in a certain in-the-wild pose is to capture a video turning around that person while he/she stays still. &#10;The Mannequin Challenge dataset~\cite{mannequinchallenge}, recently introduced to predict the depth of humans in an image, shows in-the-wild videos of people, often several persons, frozen in action like mannequins and captured from a moving camera. &#10;We propose to use this dataset and leverage the static posture of the people with the motion of the camera to apply structure-from-motion techniques and accurately fit the SMPL~\cite{smpl} model on the resulting point cloud. &#10;More concretely, our pipeline includes the following steps: &#10;(a) the human instances are tracked and segmented over the frames to find individual instances,&#10;(b) we run a structure-from-motion reconstruction to obtain a 3D point cloud and clean it using the human segmentations,&#10;(c) we use DensePose~\cite{densepose} to obtain correspondences between pixels and SMPL surface, and fit SMPL with the constraint of the pose being constant over frames while minimizing the DensePose reconstruction error. &#10;After curating the resulting data and checking the quality of our body model registrations with both automatic and manual procedures, we selected for our dataset a total of 742 human subjects in 567 different scenes. We generate in total 24,428 crops around the detected subjects at each frame.&#10;To validate the quality of our reconstruction pipeline, we captured a set of RGB-D sequences and compared the reconstructions produced by our pipeline to the depth data, obtaining a negligible difference.&#10;&#10;As a second contribution, we benchmark recent methods that predict SMPL parameters in images~\cite{hmr,nbf,spin,texturepose,graphcmr,dct} or videos~\cite{vibe}. &#10;We find that the best method regarding 3D pose estimation is VIBE~\cite{vibe} which benefits from leveraging temporal information compared to the other ones that predict SMPL parameters from a static image.&#10;Interestingly, when evaluating the 2D reprojection error of the body joints, HMR~\cite{hmr} outperforms more recent approaches. This can be explained by the fact that HMR was trained on more in-the-wild datasets with 2D annotations.&#10;Despite impressive results, all state-of-the-art methods tend to catastrophically fail when humans are either occluded by objects in the scene in close-up or truncated by image boundaries. The presented benchmark helps understanding such failure cases and we believe it is of great importance to improve robustness of future works.&#10;&#10;After reviewing related benchmarks in Section~\ref{sec:related}, we present our method to reconstruct static people in Section~\ref{sec:reconstruct} and validate it in Section~\ref{sec:validation}.&#10;Section~\ref{sec:dataset} then summarizes our SMPL Mannequin Benchmark dataset while we benchmark state-of-the-art methods in Section~\ref{sec:results}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1801.01615" label="1801.01615">
-        <attvalues>
-          <attvalue for="0" value="Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and&#10;  Bodies" />
-          <attvalue for="1" value="  We present a unified deformation model for the markerless capture of multiple&#10;scales of human movement, including facial expressions, body motion, and hand&#10;gestures. An initial model is generated by locally stitching together models of&#10;the individual parts of the human body, which we refer to as the &quot;Frankenstein&quot;&#10;model. This model enables the full expression of part movements, including face&#10;and hands by a single seamless model. Using a large-scale capture of people&#10;wearing everyday clothes, we optimize the Frankenstein model to create &quot;Adam&quot;.&#10;Adam is a calibrated model that shares the same skeleton hierarchy as the&#10;initial model but can express hair and clothing geometry, making it directly&#10;usable for fitting people as they normally appear in everyday life. Finally, we&#10;demonstrate the use of these models for total motion tracking, simultaneously&#10;capturing the large-scale body movements and the subtle face and hand motion of&#10;a social group of people.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.03244" label="1905.03244">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Mesh Regression for Single-Image Human Shape&#10;  Reconstruction" />
-          <attvalue for="1" value="  This paper addresses the problem of 3D human pose and shape estimation from a&#10;single image. Previous approaches consider a parametric model of the human&#10;body, SMPL, and attempt to regress the model parameters that give rise to a&#10;mesh consistent with image evidence. This parameter regression has been a very&#10;challenging task, with model-based approaches underperforming compared to&#10;nonparametric solutions in terms of pose estimation. In our work, we propose to&#10;relax this heavy reliance on the model's parameter space. We still retain the&#10;topology of the SMPL template mesh, but instead of predicting model parameters,&#10;we directly regress the 3D location of the mesh vertices. This is a heavy task&#10;for a typical network, but our key insight is that the regression becomes&#10;significantly easier using a Graph-CNN. This architecture allows us to&#10;explicitly encode the template mesh structure within the network and leverage&#10;the spatial locality the mesh has to offer. Image-based features are attached&#10;to the mesh vertices and the Graph-CNN is responsible to process them on the&#10;mesh structure, while the regression target for each vertex is its 3D location.&#10;Having recovered the complete 3D geometry of the mesh, if we still require a&#10;specific model parametrization, this can be reliably regressed from the&#10;vertices locations. We demonstrate the flexibility and the effectiveness of our&#10;proposed graph-based mesh regression by attaching different types of features&#10;on the mesh vertices. In all cases, we outperform the comparable baselines&#10;relying on model parameter regression, while we also achieve state-of-the-art&#10;results among model-based pose estimation approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.11322" label="1910.11322">
-        <attvalues>
-          <attvalue for="0" value="TexturePose: Supervising Human Mesh Estimation with Texture Consistency" />
-          <attvalue for="1" value="  This work addresses the problem of model-based human pose estimation. Recent&#10;approaches have made significant progress towards regressing the parameters of&#10;parametric human body models directly from images. Because of the absence of&#10;images with 3D shape ground truth, relevant approaches rely on 2D annotations&#10;or sophisticated architecture designs. In this work, we advocate that there are&#10;more cues we can leverage, which are available for free in natural images,&#10;i.e., without getting more annotations, or modifying the network architecture.&#10;We propose a natural form of supervision, that capitalizes on the appearance&#10;constancy of a person among different frames (or viewpoints). This seemingly&#10;insignificant and often overlooked cue goes a long way for model-based pose&#10;estimation. The parametric model we employ allows us to compute a texture map&#10;for each frame. Assuming that the texture of the person does not change&#10;dramatically between frames, we can apply a novel texture consistency loss,&#10;which enforces that each point in the texture map has the same texture value&#10;across all frames. Since the texture is transferred in this common texture map&#10;space, no camera motion computation is necessary, or even an assumption of&#10;smoothness among frames. This makes our proposed supervision applicable in a&#10;variety of settings, ranging from monocular video, to multi-view images. We&#10;benchmark our approach against strong baselines that require the same or even&#10;more annotations that we do and we consistently outperform them.&#10;Simultaneously, we achieve state-of-the-art results among model-based pose&#10;estimation approaches in different benchmarks. The project website with videos,&#10;results, and code can be found at&#10;https://seas.upenn.edu/~pavlakos/projects/texturepose.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.06442" label="1908.06442">
-        <attvalues>
-          <attvalue for="0" value="Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild" />
-          <attvalue for="1" value="  Though much progress has been achieved in single-image 3D human recovery,&#10;estimating 3D model for in-the-wild images remains a formidable challenge. The&#10;reason lies in the fact that obtaining high-quality 3D annotations for&#10;in-the-wild images is an extremely hard task that consumes enormous amount of&#10;resources and manpower. To tackle this problem, previous methods adopt a hybrid&#10;training strategy that exploits multiple heterogeneous types of annotations&#10;including 3D and 2D while leaving the efficacy of each annotation not&#10;thoroughly investigated. In this work, we aim to perform a comprehensive study&#10;on cost and effectiveness trade-off between different annotations.&#10;Specifically, we focus on the challenging task of in-the-wild 3D human recovery&#10;from single images when paired 3D annotations are not fully available. Through&#10;extensive experiments, we obtain several observations: 1) 3D annotations are&#10;efficient, whereas traditional 2D annotations such as 2D keypoints and body&#10;part segmentation are less competent in guiding 3D human recovery. 2) Dense&#10;Correspondence such as DensePose is effective. When there are no paired&#10;in-the-wild 3D annotations available, the model exploiting dense correspondence&#10;can achieve 92% of the performance compared to a model trained with paired 3D&#10;data. We show that incorporating dense correspondence into in-the-wild 3D human&#10;recovery is promising and competitive due to its high efficiency and relatively&#10;low annotating cost. Our model trained with dense correspondence can serve as a&#10;strong reference for future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00439" label="1908.00439">
-        <attvalues>
-          <attvalue for="0" value="Moulding Humans: Non-parametric 3D Human Shape Estimation from Single&#10;  Images" />
-          <attvalue for="1" value="  In this paper, we tackle the problem of 3D human shape estimation from single&#10;RGB images. While the recent progress in convolutional neural networks has&#10;allowed impressive results for 3D human pose estimation, estimating the full 3D&#10;shape of a person is still an open issue. Model-based approaches can output&#10;precise meshes of naked under-cloth human bodies but fail to estimate details&#10;and un-modelled elements such as hair or clothing. On the other hand,&#10;non-parametric volumetric approaches can potentially estimate complete shapes&#10;but, in practice, they are limited by the resolution of the output grid and&#10;cannot produce detailed estimates. In this work, we propose a non-parametric&#10;approach that employs a double depth map to represent the 3D shape of a person:&#10;a visible depth map and a &quot;hidden&quot; depth map are estimated and combined, to&#10;reconstruct the human 3D shape as done with a &quot;mould&quot;. This representation&#10;through 2D depth maps allows a higher resolution output with a much lower&#10;dimension than voxel-based volumetric representations. Additionally, our fully&#10;derivable depth-based model allows us to efficiently incorporate a&#10;discriminator in an adversarial fashion to improve the accuracy and &quot;humanness&quot;&#10;of the 3D output. We train and quantitatively validate our approach on SURREAL&#10;and on 3D-HUMANS, a new photorealistic dataset made of semi-synthetic in-house&#10;videos annotated with 3D ground truth surfaces.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.10506" label="1904.10506">
-        <attvalues>
-          <attvalue for="0" value="Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh&#10;  Deformation" />
-          <attvalue for="1" value="  This paper presents a novel framework to recover detailed human body shapes&#10;from a single image. It is a challenging task due to factors such as variations&#10;in human shapes, body poses, and viewpoints. Prior methods typically attempt to&#10;recover the human body shape using a parametric based template that lacks the&#10;surface details. As such the resulting body shape appears to be without&#10;clothing. In this paper, we propose a novel learning-based framework that&#10;combines the robustness of parametric model with the flexibility of free-form&#10;3D deformation. We use the deep neural networks to refine the 3D shape in a&#10;Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from&#10;body joints, silhouettes, and per-pixel shading information. We are able to&#10;restore detailed human body shapes beyond skinned models. Experiments&#10;demonstrate that our method has outperformed previous state-of-the-art&#10;approaches, achieving better accuracy in terms of both 2D IoU number and 3D&#10;metric distance. The code is available in https://github.com/zhuhao-nju/hmd.git&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06473" label="1903.06473">
-        <attvalues>
-          <attvalue for="0" value="DeepHuman: 3D Human Reconstruction from a Single Image" />
-          <attvalue for="1" value="  We propose DeepHuman, an image-guided volume-to-volume translation CNN for 3D&#10;human reconstruction from a single RGB image. To reduce the ambiguities&#10;associated with the surface geometry reconstruction, even for the&#10;reconstruction of invisible areas, we propose and leverage a dense semantic&#10;representation generated from SMPL model as an additional input. One key&#10;feature of our network is that it fuses different scales of image features into&#10;the 3D space through volumetric feature transformation, which helps to recover&#10;accurate surface geometry. The visible surface details are further refined&#10;through a normal refinement network, which can be concatenated with the volume&#10;generation network using our proposed volumetric normal projection layer. We&#10;also contribute THuman, a 3D real-world human model dataset containing about&#10;7000 models. The network is trained using training data generated from the&#10;dataset. Overall, due to the specific design of our network and the diversity&#10;in our dataset, our method enables 3D human model estimation given only a&#10;single image and outperforms state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.03453" label="1712.03453">
-        <attvalues>
-          <attvalue for="0" value="Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB" />
-          <attvalue for="1" value="  We propose a new single-shot method for multi-person 3D pose estimation in&#10;general scenes from a monocular RGB camera. Our approach uses novel&#10;occlusion-robust pose-maps (ORPM) which enable full body pose inference even&#10;under strong partial occlusions by other people and objects in the scene. ORPM&#10;outputs a fixed number of maps which encode the 3D joint locations of all&#10;people in the scene. Body part associations allow us to infer 3D pose for an&#10;arbitrary number of people without explicit bounding box prediction. To train&#10;our approach we introduce MuCo-3DHP, the first large scale training data set&#10;showing real images of sophisticated multi-person interactions and occlusions.&#10;We synthesize a large corpus of multi-person images by compositing images of&#10;individual people (with ground truth from mutli-view performance capture). We&#10;evaluate our method on our new challenging 3D annotated multi-person test set&#10;MuPoTs-3D where we achieve state-of-the-art performance. To further stimulate&#10;research in multi-person 3D pose estimation, we will make our new datasets, and&#10;associated code publicly available for research purposes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.07548" label="1707.07548">
-        <attvalues>
-          <attvalue for="0" value="Towards Accurate Markerless Human Shape and Pose Estimation over Time" />
-          <attvalue for="1" value="  Existing marker-less motion capture methods often assume known backgrounds,&#10;static cameras, and sequence specific motion priors, which narrows its&#10;application scenarios. Here we propose a fully automatic method that given&#10;multi-view video, estimates 3D human motion and body shape. We take recent&#10;SMPLify \cite{bogo2016keep} as the base method, and extend it in several ways.&#10;First we fit the body to 2D features detected in multi-view images. Second, we&#10;use a CNN method to segment the person in each image and fit the 3D body model&#10;to the contours to further improves accuracy. Third we utilize a generic and&#10;robust DCT temporal prior to handle the left and right side swapping issue&#10;sometimes introduced by the 2D pose estimator. Validation on standard&#10;benchmarks shows our results are comparable to the state of the art and also&#10;provide a realistic 3D shape avatar. We also demonstrate accurate results on&#10;HumanEva and on challenging dance sequences from YouTube in monocular case.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.11111" label="1904.11111">
-        <attvalues>
-          <attvalue for="0" value="Learning the Depths of Moving People by Watching Frozen People" />
-          <attvalue for="1" value="  We present a method for predicting dense depth in scenarios where both a&#10;monocular camera and people in the scene are freely moving. Existing methods&#10;for recovering depth for dynamic, non-rigid objects from monocular video impose&#10;strong assumptions on the objects' motion and may only recover sparse depth. In&#10;this paper, we take a data-driven approach and learn human depth priors from a&#10;new source of data: thousands of Internet videos of people imitating&#10;mannequins, i.e., freezing in diverse, natural poses, while a hand-held camera&#10;tours the scene. Because people are stationary, training data can be generated&#10;using multi-view stereo reconstruction. At inference time, our method uses&#10;motion parallax cues from the static areas of the scenes to guide the depth&#10;prediction. We demonstrate our method on real-world sequences of complex human&#10;actions captured by a moving hand-held camera, show improvement over&#10;state-of-the-art monocular depth prediction methods, and show various 3D&#10;effects produced using our predicted depth.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.00434" label="1802.00434">
-        <attvalues>
-          <attvalue for="0" value="DensePose: Dense Human Pose Estimation In The Wild" />
-          <attvalue for="1" value="  In this work, we establish dense correspondences between RGB image and a&#10;surface-based representation of the human body, a task we refer to as dense&#10;human pose estimation. We first gather dense correspondences for 50K persons&#10;appearing in the COCO dataset by introducing an efficient annotation pipeline.&#10;We then use our dataset to train CNN-based systems that deliver dense&#10;correspondence 'in the wild', namely in the presence of background, occlusions&#10;and scale variations. We improve our training set's effectiveness by training&#10;an 'inpainting' network that can fill in missing groundtruth values and report&#10;clear improvements with respect to the best results that would be achievable in&#10;the past. We experiment with fully-convolutional networks and region-based&#10;models and observe a superiority of the latter; we further improve accuracy&#10;through cascading, obtaining a system that delivers highly0accurate results in&#10;real time. Supplementary materials and videos are provided on the project page&#10;http://densepose.org&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.12828" label="1909.12828">
-        <attvalues>
-          <attvalue for="0" value="Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the&#10;  Loop" />
-          <attvalue for="1" value="  Model-based human pose estimation is currently approached through two&#10;different paradigms. Optimization-based methods fit a parametric body model to&#10;2D observations in an iterative manner, leading to accurate image-model&#10;alignments, but are often slow and sensitive to the initialization. In&#10;contrast, regression-based methods, that use a deep network to directly&#10;estimate the model parameters from pixels, tend to provide reasonable, but not&#10;pixel accurate, results while requiring huge amounts of supervision. In this&#10;work, instead of investigating which approach is better, our key insight is&#10;that the two paradigms can form a strong collaboration. A reasonable, directly&#10;regressed estimate from the network can initialize the iterative optimization&#10;making the fitting faster and more accurate. Similarly, a pixel accurate fit&#10;from iterative optimization can act as strong supervision for the network. This&#10;is the core of our proposed approach SPIN (SMPL oPtimization IN the loop). The&#10;deep network initializes an iterative optimization routine that fits the body&#10;model to 2D joints within the training loop, and the fitted estimate is&#10;subsequently used to supervise the network. Our approach is self-improving by&#10;nature, since better network estimates can lead the optimization to better&#10;solutions, while more accurate optimization fits provide better supervision for&#10;the network. We demonstrate the effectiveness of our approach in different&#10;settings, where 3D ground truth is scarce, or not available, and we&#10;consistently outperform the state-of-the-art model-based pose estimation&#10;approaches by significant margins. The project website with videos, results,&#10;and code can be found at https://seas.upenn.edu/~nkolot/projects/spin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.05656" label="1912.05656">
-        <attvalues>
-          <attvalue for="0" value="VIBE: Video Inference for Human Body Pose and Shape Estimation" />
-          <attvalue for="1" value="  Human motion is fundamental to understanding behavior. Despite progress on&#10;single-image 3D pose and shape estimation, existing video-based&#10;state-of-the-art methods fail to produce accurate and natural motion sequences&#10;due to a lack of ground-truth 3D motion data for training. To address this&#10;problem, we propose Video Inference for Body Pose and Shape Estimation (VIBE),&#10;which makes use of an existing large-scale motion capture dataset (AMASS)&#10;together with unpaired, in-the-wild, 2D keypoint annotations. Our key novelty&#10;is an adversarial learning framework that leverages AMASS to discriminate&#10;between real human motions and those produced by our temporal pose and shape&#10;regression networks. We define a temporal network architecture and show that&#10;adversarial training, at the sequence level, produces kinematically plausible&#10;motion sequences without in-the-wild ground-truth 3D labels. We perform&#10;extensive experimentation to analyze the importance of motion and demonstrate&#10;the effectiveness of VIBE on challenging 3D pose estimation datasets, achieving&#10;state-of-the-art performance. Code and pretrained models are available at&#10;https://github.com/mkocabas/VIBE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.08270" label="2409.08270">
-        <attvalues>
-          <attvalue for="0" value="FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally" />
-          <attvalue for="1" value="  This study addresses the challenge of accurately segmenting 3D Gaussian&#10;Splatting from 2D masks. Conventional methods often rely on iterative gradient&#10;descent to assign each Gaussian a unique label, leading to lengthy optimization&#10;and sub-optimal solutions. Instead, we propose a straightforward yet globally&#10;optimal solver for 3D-GS segmentation. The core insight of our method is that,&#10;with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially&#10;a linear function with respect to the labels of each Gaussian. As such, the&#10;optimal label assignment can be solved via linear programming in closed form.&#10;This solution capitalizes on the alpha blending characteristic of the splatting&#10;process for single step optimization. By incorporating the background bias in&#10;our objective function, our method shows superior robustness in 3D segmentation&#10;against noises. Remarkably, our optimization completes within 30 seconds, about&#10;50$\times$ faster than the best existing methods. Extensive experiments&#10;demonstrate the efficiency and robustness of our method in segmenting various&#10;scenes, and its superior performance in downstream tasks such as object removal&#10;and inpainting. Demos and code will be available at&#10;https://github.com/florinshen/FlashSplat.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The pursuit of understanding and interacting with 3D environments represents a formidable yet pivotal challenge in computer vision. Central to this endeavor is the task of accurately perceiving and segmenting 3D structures~\cite{NeRF, dvgo, tensorf, mipnerf360, InstantNGP, plenoxel}, a task that becomes increasingly complex as we delve into more sophisticated representations of 3D scenes. &#10;Recently, 3D Gaussian Splatting (3D-GS)~\cite{3dgs} emerges as a cutting-edge approach that promises to revolutionize how we render and reconstruct 3D spaces. By employing a multitude of colored 3D Gaussians, this method achieves a high fidelity representation of 3D scenes, offering a compelling blend of precision and visual quality that is particularly suited for complex object and scene rendering.&#10;&#10;Despite the potential of 3D-GS, a significant obstacle remains: the segmentation of these 3D Gaussian from mere 2D masks—a process crucial for a range of applications from object recognition to scene manipulation. Existing approaches~\cite{gsgroup, saga} to this challenge have largely depended on iterative gradient descent methods for labeling 3D Gaussian. However, these methods are marred by their slow convergence speed and frequent entrapment in suboptimal solutions, rendering them impractical for applications requiring real-time performance or high accuracy.&#10;&#10;Addressing this gap, our work introduces a simple but globally optimal solver designed explicitly for the segmentation of 3D Gaussian Splatting. Our approach capitalizes on the insight that the process of rendering segmented 2D images from a reconstructed 3D Gaussian Splatting can be simplified to a linear function concerning the accumulated contribution of each Gaussian. This realization allows us to frame the problem as a linear integer programming that can be solved in a closed form, relying solely on the alpha composition term inherent in the splatting process. This breakthrough significantly streamlines the segmentation task, bypassing the need for iterative optimization and directly leading to the optimal label assignment.&#10;&#10;Moreover, by integrating a background bias into our objective function, we further enhance the method's robustness against 2D masks noise in 3D segmentation. This refinement not only bolsters the robustness of our solution but also its applicability across a wider range of scenes and conditions. Impressively, our solver completes its optimization in approximately 30 seconds—an acceleration of 50$\times$ faster than existing methods—without sacrificing accuracy or global optimality.&#10;&#10;Through extensive experimentation, we have validated the superiority of our approach in efficiently segmenting a variety of scenes, demonstrating its enhanced performance in downstream tasks such as object removal and inpainting. These results underscore the potential of our method to significantly advance the field of 3D scene processing and understanding. &#10;&#10;The primary contributions of our work are summarized as follows:&#10;\begin{itemize}&#10; \item We introduce a globally optimal solver for 3D Gaussian&#10;Splatting segmentation, which significantly enhances the efficiency of lifting 2D segmentation results into 3D space.&#10; \item We simplify the 3D-GS rendering process through linearization, turning the 2D to 3D segmentation task into a linear integer optimization problem. This method is effective for both binary and scene segmentation.&#10; \item We introduce a background bias in the optimization, demonstrate superior robustness againt noises in 3D segmentation, showcasing our method's robustness and efficiency across a variety of scene segmentation.&#10; \item Our method achieves a remarkable optimization speed, completing the process within 30 seconds—approximately 50 times faster than existing methods—while ensuring global optimality for given 2D masks.&#10; \item Extensive experiments validate the superiority of our approach in downstream tasks, including object removal and inpainting, thus highlighting its potential to significantly impact 3D data processing and application.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Physics, Linear Programming Optimization, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2003.08934" label="2003.08934">
-        <attvalues>
-          <attvalue for="0" value="NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" />
-          <attvalue for="1" value="  We present a method that achieves state-of-the-art results for synthesizing&#10;novel views of complex scenes by optimizing an underlying continuous volumetric&#10;scene function using a sparse set of input views. Our algorithm represents a&#10;scene using a fully-connected (non-convolutional) deep network, whose input is&#10;a single continuous 5D coordinate (spatial location $(x,y,z)$ and viewing&#10;direction $(\theta, \phi)$) and whose output is the volume density and&#10;view-dependent emitted radiance at that spatial location. We synthesize views&#10;by querying 5D coordinates along camera rays and use classic volume rendering&#10;techniques to project the output colors and densities into an image. Because&#10;volume rendering is naturally differentiable, the only input required to&#10;optimize our representation is a set of images with known camera poses. We&#10;describe how to effectively optimize neural radiance fields to render&#10;photorealistic novel views of scenes with complicated geometry and appearance,&#10;and demonstrate results that outperform prior work on neural rendering and view&#10;synthesis. View synthesis results are best viewed as videos, so we urge readers&#10;to view our supplementary video for convincing comparisons.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.04079" label="2308.04079">
-        <attvalues>
-          <attvalue for="0" value="3D Gaussian Splatting for Real-Time Radiance Field Rendering" />
-          <attvalue for="1" value="  Radiance Field methods have recently revolutionized novel-view synthesis of&#10;scenes captured with multiple photos or videos. However, achieving high visual&#10;quality still requires neural networks that are costly to train and render,&#10;while recent faster methods inevitably trade off speed for quality. For&#10;unbounded and complete scenes (rather than isolated objects) and 1080p&#10;resolution rendering, no current method can achieve real-time display rates. We&#10;introduce three key elements that allow us to achieve state-of-the-art visual&#10;quality while maintaining competitive training times and importantly allow&#10;high-quality real-time (&gt;= 30 fps) novel-view synthesis at 1080p resolution.&#10;First, starting from sparse points produced during camera calibration, we&#10;represent the scene with 3D Gaussians that preserve desirable properties of&#10;continuous volumetric radiance fields for scene optimization while avoiding&#10;unnecessary computation in empty space; Second, we perform interleaved&#10;optimization/density control of the 3D Gaussians, notably optimizing&#10;anisotropic covariance to achieve an accurate representation of the scene;&#10;Third, we develop a fast visibility-aware rendering algorithm that supports&#10;anisotropic splatting and both accelerates training and allows realtime&#10;rendering. We demonstrate state-of-the-art visual quality and real-time&#10;rendering on several established datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.00732" label="2312.00732">
-        <attvalues>
-          <attvalue for="0" value="Gaussian Grouping: Segment and Edit Anything in 3D Scenes" />
-          <attvalue for="1" value="  The recent Gaussian Splatting achieves high-quality and real-time novel-view&#10;synthesis of the 3D scenes. However, it is solely concentrated on the&#10;appearance and geometry modeling, while lacking in fine-grained object-level&#10;scene understanding. To address this issue, we propose Gaussian Grouping, which&#10;extends Gaussian Splatting to jointly reconstruct and segment anything in&#10;open-world 3D scenes. We augment each Gaussian with a compact Identity&#10;Encoding, allowing the Gaussians to be grouped according to their object&#10;instance or stuff membership in the 3D scene. Instead of resorting to expensive&#10;3D labels, we supervise the Identity Encodings during the differentiable&#10;rendering by leveraging the 2D mask predictions by Segment Anything Model&#10;(SAM), along with introduced 3D spatial consistency regularization. Compared to&#10;the implicit NeRF representation, we show that the discrete and grouped 3D&#10;Gaussians can reconstruct, segment and edit anything in 3D with high visual&#10;quality, fine granularity and efficiency. Based on Gaussian Grouping, we&#10;further propose a local Gaussian Editing scheme, which shows efficacy in&#10;versatile scene editing applications, including 3D object removal, inpainting,&#10;colorization, style transfer and scene recomposition. Our code and models are&#10;at https://github.com/lkeab/gaussian-grouping.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.05014" label="2104.05014">
-        <attvalues>
-          <attvalue for="0" value="One Ring to Rule Them All: a simple solution to multi-view&#10;  3D-Reconstruction of shapes with unknown BRDF via a small Recurrent ResNet" />
-          <attvalue for="1" value="  This paper proposes a simple method which solves an open problem of&#10;multi-view 3D-Reconstruction for objects with unknown and generic surface&#10;materials, imaged by a freely moving camera and a freely moving point light&#10;source. The object can have arbitrary (e.g. non-Lambertian), spatially-varying&#10;(or everywhere different) surface reflectances (svBRDF). Our solution consists&#10;of two smallsized neural networks (dubbed the 'Shape-Net' and 'BRDFNet'), each&#10;having about 1,000 neurons, used to parameterize the unknown shape and unknown&#10;svBRDF, respectively. Key to our method is a special network design (namely, a&#10;ResNet with a global feedback or 'ring' connection), which has a provable&#10;guarantee for finding a valid diffeomorphic shape parameterization. Despite the&#10;underlying problem is highly non-convex hence impractical to solve by&#10;traditional optimization techniques, our method converges reliably to high&#10;quality solutions, even without initialization. Extensive experiments&#10;demonstrate the superiority of our method, and it naturally enables a wide&#10;range of special-effect applications including novel-view-synthesis,&#10;relighting, material retouching, and shape exchange without additional coding&#10;effort. We encourage the reader to view our demo video for better&#10;visualizations.&#10;" />
-          <attvalue for="2" value="&#10;Reconstructing the 3D shape of object or scene from their multi-view images is one of the central tasks in computer vision research. Traditional multi-view 3D-reconstruction methods often assume that the objects or scenes of interest are largely diffuse (\ie close to Lambertian) or texture-rich, therefore, allowing for reliable cross-view image correspondences. However, in reality, many commonly seen objects are made of generic materials possibly with glossy, metal-like appearances, violating the brightness-constancy assumption needed for establishing image correspondences. &#10;&#10;It remains an open challenge to estimate the 3D geometry of objects of unknown arbitrary materials. Furthermore, when the object is illuminated by a moving (active) light source, it further complicates the task, because the visual appearance of a non-Lambertian object is not only view-dependent but also light-dependent in general. Very few approaches have been attempted at this challenging task. The only existing ones, based on photometric stereo, are plagued by solving difficult mathematical optimization problems often involving a highly non-convex objective function derived from photometric image rendering equation. As a result, their performances critically depends on the quality of the initialization or require significant manual intervention (\eg parameter tuning \cite{park2016robust,nam2018practical}).&#10;&#10;In this paper, we propose a neural-network based feature-correspondence-free method for reconstructing both the shape of an object and its spatially-varying reflectance model in the form of a BRDF (Bidirectional Reflectance Distribution Function). This allows novel synthetic views of the object to be rendered with high realism.&#10;&#10;Key to our method is the representation of object shape by a smooth vector field on the ambient space $\mathbb{R}^3$ along which a canonical shape ``flows'' to the desired shape. We show mathematically that the resulting shape must &#10;be a smooth embedding of a sphere, and that all genus-zero shapes can be represented in this way. The vector field (\ie a smooth mapping from $\mathbb{R}^3$ to $\mathbb{R}^3$) is computed by a single MLP (Multi-layer perceptron), integrated via a recurrent architecture with a ``ring&quot; (feedback) connection. This cascade is implemented by a recursive residual network, which we call the {Shape-Net}. The input to our method is a set of views of an object for which the positions of the camera and the light source are known. The output consists of a watertight shape and { BRDF} parameters for each point on the surface, both embedded in the same sphere. Such a surface representation differs from explicit mesh or implicit level set used in many previous works \cite{wang2018pixel2mesh,jiang2020sdfdiff}. Given the success of this approach, we believe that this method of defining a shape via a vector field will have wide applications beyond $3D$-reconstruction. Representation of diffeomorphisms using flows has been used before in computer vision \cite{LMLDD}, but not with deep neural nets. &#10;&#10;By extensive experiments, we demonstrate that our method produces accurate and compelling shape and svBRDF reconstructions, even without initialization. In the sequel, we will first describe the problem setting, our new method and its theory, followed by experiment validations. We defer ``Related work&quot; to a later sectionfor the sake of smooth reading. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Neural Network Design, Mathematics, Physics, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1804.01654" label="1804.01654">
-        <attvalues>
-          <attvalue for="0" value="Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images" />
-          <attvalue for="1" value="  We propose an end-to-end deep learning architecture that produces a 3D shape&#10;in triangular mesh from a single color image. Limited by the nature of deep&#10;neural network, previous methods usually represent a 3D shape in volume or&#10;point cloud, and it is non-trivial to convert them to the more ready-to-use&#10;mesh model. Unlike the existing methods, our network represents 3D mesh in a&#10;graph-based convolutional neural network and produces correct geometry by&#10;progressively deforming an ellipsoid, leveraging perceptual features extracted&#10;from the input image. We adopt a coarse-to-fine strategy to make the whole&#10;deformation procedure stable, and define various of mesh related losses to&#10;capture properties of different levels to guarantee visually appealing and&#10;physically accurate 3D geometry. Extensive experiments show that our method not&#10;only qualitatively produces mesh model with better details, but also achieves&#10;higher 3D shape estimation accuracy compared to the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.07109" label="1912.07109">
-        <attvalues>
-          <attvalue for="0" value="SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape&#10;  Optimization" />
-          <attvalue for="1" value="  We propose SDFDiff, a novel approach for image-based shape optimization using&#10;differentiable rendering of 3D shapes represented by signed distance functions&#10;(SDFs). Compared to other representations, SDFs have the advantage that they&#10;can represent shapes with arbitrary topology, and that they guarantee&#10;watertight surfaces. We apply our approach to the problem of multi-view 3D&#10;reconstruction, where we achieve high reconstruction quality and can capture&#10;complex topology of 3D objects. In addition, we employ a multi-resolution&#10;strategy to obtain a robust optimization algorithm. We further demonstrate that&#10;our SDF-based differentiable renderer can be integrated with deep learning&#10;models, which opens up options for learning approaches on 3D objects without 3D&#10;supervision. In particular, we apply our method to single-view 3D&#10;reconstruction and achieve state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.11971" label="2211.11971">
-        <attvalues>
-          <attvalue for="0" value="Multi-View Neural Surface Reconstruction with Structured Light" />
-          <attvalue for="1" value="  Three-dimensional (3D) object reconstruction based on differentiable&#10;rendering (DR) is an active research topic in computer vision. DR-based methods&#10;minimize the difference between the rendered and target images by optimizing&#10;both the shape and appearance and realizing a high visual reproductivity.&#10;However, most approaches perform poorly for textureless objects because of the&#10;geometrical ambiguity, which means that multiple shapes can have the same&#10;rendered result in such objects. To overcome this problem, we introduce active&#10;sensing with structured light (SL) into multi-view 3D object reconstruction&#10;based on DR to learn the unknown geometry and appearance of arbitrary scenes&#10;and camera poses. More specifically, our framework leverages the&#10;correspondences between pixels in different views calculated by structured&#10;light as an additional constraint in the DR-based optimization of implicit&#10;surface, color representations, and camera poses. Because camera poses can be&#10;optimized simultaneously, our method realizes high reconstruction accuracy in&#10;the textureless region and reduces efforts for camera pose calibration, which&#10;is required for conventional SL-based methods. Experiment results on both&#10;synthetic and real data demonstrate that our system outperforms conventional&#10;DR- and SL-based methods in a high-quality surface reconstruction, particularly&#10;for challenging objects with textureless or shiny surfaces.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Importing 3D objects from the real world into virtual worlds is an essential technology in digital arts, extended reality (XR) applications, cultural heritage protection~\cite{LiR,Niven}, paleontology~\cite{ziegler2020applications}, visual inspection~\cite{huang2019improved}, and 3D printing~\cite{biehler20143d}.&#10;Conventional methods applied to achieve this automatically include photogrammetry, which integrates multiple 2D images from different views.&#10;&#10;Differentiable rendering (DR)~\cite{kato2020differentiable,zhao2020physics} is an emerging tool for multi-view stereo. In contrast to traditional methods that rely on matching features from different views~\cite{camp,furu,gipuma,tola}, in DR-based methods, 3D geometries are directly optimized using gradient descent, such that rendered images are close to the observed images. A high-fidelity reconstruction can be achieved~\cite{dvr,idr} when combined with 3D representations using neural networks ({neural fields}~\cite{neuralfields}).&#10;However, DR-based methods are subject to an inherent problem in terms of the geometrical ambiguity of the observations. In other words, the observed images can be explained by several different geometries, although which among them is more accurate cannot be determined. This ambiguity is high on concave or textureless surfaces, such as the result produced by NeuS, as shown in Fig.~\ref{fig:bowl}.&#10;&#10;Active vision alleviates this problem. A representative method is the use structured light (SL)~\cite{Caspi,GuQ,Nguyen,Salvi,li2019pro,Li7}, in which multiple texture patterns are projected onto objects using a projector. Although the SL system requires an additional device (the projector), it can measure the depth (3D point clouds) with high quality even for textureless objects owing to an active projection. However, standard SL systems have certain disadvantages. First, the camera and projector must be rigidly fixed (mounted on a rig) and precisely calibrated. second, the SL system provides a 3D reconstruction of poor quality to recover optically complex scenes such as shiny objects, because the appearance of highlights and inter-reflections lead to incomplete 3D point cloud result. Third, SL systems are sensitive to occlusions, and by extension to holes, because they rely on an optical disparity.&#10;&#10;In this work, we propose to introduce active sensing with SL into DR-based multi-view stereo.&#10;We follow the implicit differentiable renderer~\cite{idr,neus,volsdf} to represent the surface as a zero-level set of a signed distance field (SDF) and the scene appearance as a color field.&#10;To train these networks, a sequence of SL patterns is projected onto the object surface and captured by two cameras while the object is arbitrarily rotated to obtain multiple observations of SL patterns around the object (see Fig.~\ref{fig:configuration}). The correspondence between the two camera views extracted from the captured images with projected patterns is used to supervise the training of the object surface.&#10;In addition, images without a SL pattern for each camera view are captured for photometric supervision, which has been widely used in previous DR frameworks. Both the surface and appearance of the object are optimized by minimizing the loss between observations and rendered images.&#10;On the one hand, compared to previous DR frameworks~\cite{idr,neus,volsdf} which only use photometric data to supervise the training, the SL supervision can provide important cues to reduce the geometrical ambiguity. &#10;On the other hand, shiny objects that cannot be handled by SL technology can be approximately represented using the color field. Thus, their shapes can be optimized through photometric supervision.&#10;In addition, self-occlusion, which cannot be handled by SL technology, can be solved through the global optimization of the surface represented by an implicit SDF using photometric supervision.&#10;Furthermore, by establishing a geometric relationship between the neural implicit surface and camera poses, the camera extrinsic parameters are also refined during optimization; therefore, our method does not require strict calibration in contrast to conventional SL methods~\cite{Caspi,GuQ,Nguyen,Salvi}.&#10;To the best of our knowledge, our work is the first to combine SL and differentiable rendering.&#10;&#10;We experimentally validated the effectiveness of the proposed method under both synthetic and real-world scenarios.&#10;The results demonstrate that our method can reconstruct many types of challenging targets that are textureless, self-occluded, or shiny even with rough camera information. They also show that the proposed approach outperforms other state-of-the-art neural scene representation methods as well as conventional SL methods.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Physics, Computer Science, Structured Light Sensing, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2006.12057" label="2006.12057">
-        <attvalues>
-          <attvalue for="0" value="Differentiable Rendering: A Survey" />
-          <attvalue for="1" value="  Deep neural networks (DNNs) have shown remarkable performance improvements on&#10;vision-related tasks such as object detection or image segmentation. Despite&#10;their success, they generally lack the understanding of 3D objects which form&#10;the image, as it is not always possible to collect 3D information about the&#10;scene or to easily annotate it. Differentiable rendering is a novel field which&#10;allows the gradients of 3D objects to be calculated and propagated through&#10;images. It also reduces the requirement of 3D data collection and annotation,&#10;while enabling higher success rate in various applications. This paper reviews&#10;existing literature and discusses the current state of differentiable&#10;rendering, its applications and open research problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.07372" label="1912.07372">
-        <attvalues>
-          <attvalue for="0" value="Differentiable Volumetric Rendering: Learning Implicit 3D&#10;  Representations without 3D Supervision" />
-          <attvalue for="1" value="  Learning-based 3D reconstruction methods have shown impressive results.&#10;However, most methods require 3D supervision which is often hard to obtain for&#10;real-world datasets. Recently, several works have proposed differentiable&#10;rendering techniques to train reconstruction models from RGB images.&#10;Unfortunately, these approaches are currently restricted to voxel- and&#10;mesh-based representations, suffering from discretization or low resolution. In&#10;this work, we propose a differentiable rendering formulation for implicit shape&#10;and texture representations. Implicit representations have recently gained&#10;popularity as they represent shape and texture continuously. Our key insight is&#10;that depth gradients can be derived analytically using the concept of implicit&#10;differentiation. This allows us to learn implicit shape and texture&#10;representations directly from RGB images. We experimentally show that our&#10;single-view reconstructions rival those learned with full 3D supervision.&#10;Moreover, we find that our method can be used for multi-view 3D reconstruction,&#10;directly resulting in watertight meshes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.11426" label="2111.11426">
-        <attvalues>
-          <attvalue for="0" value="Neural Fields in Visual Computing and Beyond" />
-          <attvalue for="1" value="  Recent advances in machine learning have created increasing interest in&#10;solving visual computing problems using a class of coordinate-based neural&#10;networks that parametrize physical properties of scenes or objects across space&#10;and time. These methods, which we call neural fields, have seen successful&#10;application in the synthesis of 3D shapes and image, animation of human bodies,&#10;3D reconstruction, and pose estimation. However, due to rapid progress in a&#10;short time, many papers exist but a comprehensive review and formulation of the&#10;problem has not yet emerged. In this report, we address this limitation by&#10;providing context, mathematical grounding, and an extensive review of&#10;literature on neural fields. This report covers research along two dimensions.&#10;In Part I, we focus on techniques in neural fields by identifying common&#10;components of neural field methods, including different representations,&#10;architectures, forward mapping, and generalization methods. In Part II, we&#10;focus on applications of neural fields to different problems in visual&#10;computing, and beyond (e.g., robotics, audio). Our review shows the breadth of&#10;topics already covered in visual computing, both historically and in current&#10;incarnations, demonstrating the improved quality, flexibility, and capability&#10;brought by neural fields methods. Finally, we present a companion website that&#10;contributes a living version of this review that can be continually updated by&#10;the community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.6595" label="1406.6595">
-        <attvalues>
-          <attvalue for="0" value="3DUNDERWORLD-SLS: An Open-Source Structured-Light Scanning System for&#10;  Rapid Geometry Acquisition" />
-          <attvalue for="1" value="  Recently, there has been an increase in the demand of virtual 3D objects&#10;representing real-life objects. A plethora of methods and systems have already&#10;been proposed for the acquisition of the geometry of real-life objects ranging&#10;from those which employ active sensor technology, passive sensor technology or&#10;a combination of various techniques.&#10;  In this paper we present the development of a 3D scanning system which is&#10;based on the principle of structured-light, without having particular&#10;requirements for specialized equipment. We discuss the intrinsic details and&#10;inherent difficulties of structured-light scanning techniques and present our&#10;solutions. Finally, we introduce our open-source scanning software system&#10;&quot;3DUNDERWORLD-SLS&quot; which implements the proposed techniques both in CPU and&#10;GPU. We have performed extensive testing with a wide range of models and report&#10;the results. Furthermore, we present a comprehensive evaluation of the system&#10;and a comparison with a high-end commercial 3D scanner.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.08185" label="1908.08185">
-        <attvalues>
-          <attvalue for="0" value="Pro-Cam SSfM: Projector-Camera System for Structure and Spectral&#10;  Reflectance from Motion" />
-          <attvalue for="1" value="  In this paper, we propose a novel projector-camera system for practical and&#10;low-cost acquisition of a dense object 3D model with the spectral reflectance&#10;property. In our system, we use a standard RGB camera and leverage an&#10;off-the-shelf projector as active illumination for both the 3D reconstruction&#10;and the spectral reflectance estimation. We first reconstruct the 3D points&#10;while estimating the poses of the camera and the projector, which are&#10;alternately moved around the object, by combining multi-view structured light&#10;and structure-from-motion (SfM) techniques. We then exploit the projector for&#10;multispectral imaging and estimate the spectral reflectance of each 3D point&#10;based on a novel spectral reflectance estimation model considering the&#10;geometric relationship between the reconstructed 3D points and the estimated&#10;projector positions. Experimental results on several real objects demonstrate&#10;that our system can precisely acquire a dense 3D model with the full spectral&#10;reflectance property using off-the-shelf devices.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;{Structured light systems:}&#10;Structured light is a well-adopted technique to accurately reconstruct the 3D points irrespective of surface textures by projecting structured light patterns~\cite{Aliaga,Geng,Salvi,Michael}. While structured light methods are generally based on a pre-calibrated projector-camera system, some multi-view structured light methods~\cite{Furukawa2,Garrido,Li7} have realized self-calibrating reconstruction of the 3D points. The key of these methods is that the structured light patterns projected by one fixed projector is captured from more than two camera viewpoints having viewing angle overlaps, so that feature matching and tracking can be made to connect all projectors and cameras. This setup can be realized by simultaneously using multiple projectors and cameras~\cite{Furukawa2,Garrido} or alternately moving a projector and a camera~\cite{Li7}.&#10;&#10;In Pro-Cam SSfM, we extend the method~\cite{Li7} for realizing denser 3D reconstruction, as detailed in Section~\ref{ssec:3d}. We also exploit the estimated projector positions for modeling the spectral reflectance, while existing methods~\cite{Furukawa2,Garrido,Li7} only focus on the geometric reconstruction.&#10;&#10;{Multispectral imaging systems:}&#10;Existing hardware-based~\cite{Baek,Cao,Cui,Han2,Monno,Oh,Park} or software-based~\cite{Aeschbacher,Arad,Fu,Jia4,Nguyen2,Shi} multispectral imaging systems commonly apply a single-viewpoint image-based spectral reflectance estimation method ignoring scene's or object's geometric information. This means that they only achieve scene-dependent spectral reflectance estimation, where the viewpoint and scene-dependent shading or shadow is baked in the estimated spectral reflectance. &#10;&#10;In Pro-Cam SSfM, we use an off-the-shelf RGB camera and projector for multispectral imaging. Although this setup is the same as~\cite{Han2}, we propose a novel spectral reflectance estimation model for recovering the object's inherent spectral reflectance considering the geometric information that can be obtained at the 3D reconstruction step.&#10;&#10;{Spectral 3D acquisition systems:}&#10;Existing systems for spectral 3D acquisition are roughly classified into photometric stereo-based~\cite{Kitahara,Ozawa}, SfM and multi-view stereo-based~\cite{Ito,Zia}, and active lighting or scanner-based~\cite{Hirai,Kim} systems.&#10;The photometric stereo-based systems~\cite{Kitahara,Ozawa} can acquire dense surface normals for every pixels of the single-view image. However, they have a main limitation that the light positions should be calibrated.&#10;The SfM and multi-view stereo-based systems~\cite{Ito,Zia} enable self-calibrating 3D reconstruction using multi-view images. However, they only can provide a sparse point cloud especially for a texture-less object.&#10;The active lighting or scanner-based systems~\cite{Hirai,Kim} can provide an accurate and dense 3D model based on the active sensing. However, they require burdensome calibration of the entire system.&#10;Furthermore, all of the above-mentioned systems rely on a dedicated setup using a multispectral camera~\cite{Kim,Ozawa,Zia} or a multispectral light source~\cite{Hirai,Ito,Kitahara} for achieving multispectral imaging capability.&#10;Those limitations make the existing system impractical and expensive for most users, narrowing the range of applications.&#10;&#10;Pro-Cam SSfM overcomes those limitations because (i) it uses a low-cost off-the-shelf camera and projector, (ii) it does not require geometric calibration, and (iii) it generates a dense 3D model based on the structured light.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2106.10689" label="2106.10689">
-        <attvalues>
-          <attvalue for="0" value="NeuS: Learning Neural Implicit Surfaces by Volume Rendering for&#10;  Multi-view Reconstruction" />
-          <attvalue for="1" value="  We present a novel neural surface reconstruction method, called NeuS, for&#10;reconstructing objects and scenes with high fidelity from 2D image inputs.&#10;Existing neural surface reconstruction approaches, such as DVR and IDR, require&#10;foreground mask as supervision, easily get trapped in local minima, and&#10;therefore struggle with the reconstruction of objects with severe&#10;self-occlusion or thin structures. Meanwhile, recent neural methods for novel&#10;view synthesis, such as NeRF and its variants, use volume rendering to produce&#10;a neural scene representation with robustness of optimization, even for highly&#10;complex objects. However, extracting high-quality surfaces from this learned&#10;implicit representation is difficult because there are not sufficient surface&#10;constraints in the representation. In NeuS, we propose to represent a surface&#10;as the zero-level set of a signed distance function (SDF) and develop a new&#10;volume rendering method to train a neural SDF representation. We observe that&#10;the conventional volume rendering method causes inherent geometric errors (i.e.&#10;bias) for surface reconstruction, and therefore propose a new formulation that&#10;is free of bias in the first order of approximation, thus leading to more&#10;accurate surface reconstruction even without the mask supervision. Experiments&#10;on the DTU dataset and the BlendedMVS dataset show that NeuS outperforms the&#10;state-of-the-arts in high-quality surface reconstruction, especially for&#10;objects and scenes with complex structures and self-occlusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.12052" label="2106.12052">
-        <attvalues>
-          <attvalue for="0" value="Volume Rendering of Neural Implicit Surfaces" />
-          <attvalue for="1" value="  Neural volume rendering became increasingly popular recently due to its&#10;success in synthesizing novel views of a scene from a sparse set of input&#10;images. So far, the geometry learned by neural volume rendering techniques was&#10;modeled using a generic density function. Furthermore, the geometry itself was&#10;extracted using an arbitrary level set of the density function leading to a&#10;noisy, often low fidelity reconstruction. The goal of this paper is to improve&#10;geometry representation and reconstruction in neural volume rendering. We&#10;achieve that by modeling the volume density as a function of the geometry. This&#10;is in contrast to previous work modeling the geometry as a function of the&#10;volume density. In more detail, we define the volume density function as&#10;Laplace's cumulative distribution function (CDF) applied to a signed distance&#10;function (SDF) representation. This simple density representation has three&#10;benefits: (i) it provides a useful inductive bias to the geometry learned in&#10;the neural volume rendering process; (ii) it facilitates a bound on the opacity&#10;approximation error, leading to an accurate sampling of the viewing ray.&#10;Accurate sampling is important to provide a precise coupling of geometry and&#10;radiance; and (iii) it allows efficient unsupervised disentanglement of shape&#10;and appearance in volume rendering. Applying this new density representation to&#10;challenging scene multiview datasets produced high quality geometry&#10;reconstructions, outperforming relevant baselines. Furthermore, switching shape&#10;and appearance between scenes is possible due to the disentanglement of the&#10;two.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.01013" label="2307.01013">
-        <attvalues>
-          <attvalue for="0" value="A Synthetic Benchmarking Pipeline to Compare Camera Calibration&#10;  Algorithms" />
-          <attvalue for="1" value="  Accurate camera calibration is crucial for various computer vision&#10;applications. However, measuring calibration accuracy in the real world is&#10;challenging due to the lack of datasets with ground truth to evaluate them. In&#10;this paper, we present SynthCal, a synthetic camera calibration benchmarking&#10;pipeline that generates images of calibration patterns to measure and enable&#10;accurate quantification of calibration algorithm performance in camera&#10;parameter estimation. We present a SynthCal generated calibration dataset with&#10;four common patterns, two camera types, and two environments with varying view,&#10;distortion, lighting, and noise levels for both monocular and multi-camera&#10;systems. The dataset evaluates both single and multi-view calibration&#10;algorithms by measuring re-projection and root-mean-square errors for identical&#10;patterns and camera settings. Additionally, we analyze the significance of&#10;different patterns using different calibration configurations. The experimental&#10;results demonstrate the effectiveness of SynthCal in evaluating various&#10;calibration algorithms and patterns.&#10;" />
-          <attvalue for="2" value="&#10;When we capture an image using a camera, the captured digital image can differ from the real-world scene in terms of perspective, distortion, color, resolution, and other visual properties. This is because real-world scenes are three-dimensional and continuous, while digital images captured by a camera are two-dimensional and discrete, and contain distortion and other imperfections. To minimize these differences and improve the accuracy of image-based computer vision tasks, camera calibration is essential. &#10;&#10;Camera calibration involves calculating camera parameters that refer to its intrinsic and extrinsic characteristics for accurately mapping points in the 3D world to their corresponding 2D image coordinates. &#10;Once the camera is calibrated, it can accurately measure distances, angles, and sizes of objects in the 3D world and perform other image-based computer vision tasks such as object tracking \cite{zhang2023motrv2}, 3D reconstruction \cite{kang2020review,wu2023multiview}, medical imaging \cite{barbero2019smartphone}, and autonomous driving \cite{feng2019can}. &#10;&#10;Geometric camera calibration \cite{kikkawa2023accuracy,huai2023review} is one of the most widely used calibration methods. It involves using a calibration target with known geometric features, such as a calibration grid, to estimate the camera parameters. &#10;&#10;However, creating real camera calibration data with ground truth for calibration algorithms can be challenging because it is difficult to measure camera position and rotation accurately, and the camera's intrinsic parameters can change with changes in the zoom level, focus distance, or temperature. &#10;Moreover, cameras can have different intrinsic parameters, even if they are of the same make and model, because of manufacturing tolerances, assembly errors, and differences in lens quality.&#10;Observing the calibration pattern in the image along with the previous knowledge of the pattern, we can determine the intrinsic and extrinsic parameters using various calibration algorithms, such as Zhang \cite{zhang2000flexible}, Tsai \cite{tsai1987versatile}, or Bouguet calibration method \cite{bouguet2004camera}.&#10;Previous works have tried to compare different camera calibration algorithms \cite{usamentiaga2018comparison}. However, there is a need for a benchmarking procedure that can provide a quantitative comparison of calibration algorithms due to the unknown ground truth of the calibration dataset.&#10;&#10;To overcome these problems, we introduce an overall pipeline, named SynthCal, which generates a synthetic camera calibration dataset with user-defined intrinsic camera parameters while precisely measuring the extrinsic camera parameters. It enables the selection of the optimal camera calibration algorithm for specific configurations by considering all intrinsic, extrinsic, and distortion parameters.&#10;Additionally, it ensures that lighting conditions and noise are identical for the different captured datasets for accurate comparison of different calibration patterns which is not possible in the real world.&#10;The idea of generating synthetic calibration data has been previously applied in other works, such as sports-based synthetic calibration \cite{chen2019sports} and evaluating closed-form solutions of principal line calibration \cite{chuang2021geometry}, but not necessarily for comparing calibration algorithms.&#10;&#10;Our main contributions can be summarized as follows:&#10;\begin{itemize}&#10; \item We present a pipeline to generate a camera calibration dataset with ground truth parameters and select the optimal camera calibration algorithm for the specific configurations, as depicted in \ref{fig:1}. &#10; \item We validate the proposed pipeline on three different camera calibration algorithms which is consistent with previous works and then use SynthCal-generated dataset to compare four different calibration patterns given in \ref{fig:3} for monocular and multi-camera systems with two distinct camera configurations, and two different lighting and noise conditions.&#10;\end{itemize}&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Benchmarking Methodology, Image Processing, Camera Calibration, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2211.09791" label="2211.09791">
-        <attvalues>
-          <attvalue for="0" value="MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained&#10;  Object Detectors" />
-          <attvalue for="1" value="  In this paper, we propose MOTRv2, a simple yet effective pipeline to&#10;bootstrap end-to-end multi-object tracking with a pretrained object detector.&#10;Existing end-to-end methods, MOTR and TrackFormer are inferior to their&#10;tracking-by-detection counterparts mainly due to their poor detection&#10;performance. We aim to improve MOTR by elegantly incorporating an extra object&#10;detector. We first adopt the anchor formulation of queries and then use an&#10;extra object detector to generate proposals as anchors, providing detection&#10;prior to MOTR. The simple modification greatly eases the conflict between joint&#10;learning detection and association tasks in MOTR. MOTRv2 keeps the query&#10;propogation feature and scales well on large-scale benchmarks. MOTRv2 ranks the&#10;1st place (73.4% HOTA on DanceTrack) in the 1st Multiple People Tracking in&#10;Group Dance Challenge. Moreover, MOTRv2 reaches state-of-the-art performance on&#10;the BDD100K dataset. We hope this simple and effective pipeline can provide&#10;some new insights to the end-to-end MOT community. Code is available at&#10;\url{https://github.com/megvii-research/MOTRv2}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.08247" label="2301.08247">
-        <attvalues>
-          <attvalue for="0" value="Multiview Compressive Coding for 3D Reconstruction" />
-          <attvalue for="1" value="  A central goal of visual recognition is to understand objects and scenes from&#10;a single image. 2D recognition has witnessed tremendous progress thanks to&#10;large-scale learning and general-purpose representations. Comparatively, 3D&#10;poses new challenges stemming from occlusions not depicted in the image. Prior&#10;works try to overcome these by inferring from multiple views or rely on scarce&#10;CAD models and category-specific priors which hinder scaling to novel settings.&#10;In this work, we explore single-view 3D reconstruction by learning&#10;generalizable representations inspired by advances in self-supervised learning.&#10;We introduce a simple framework that operates on 3D points of single objects or&#10;whole scenes coupled with category-agnostic large-scale training from diverse&#10;RGB-D videos. Our model, Multiview Compressive Coding (MCC), learns to compress&#10;the input appearance and geometry to predict the 3D structure by querying a&#10;3D-aware decoder. MCC's generality and efficiency allow it to learn from&#10;large-scale and diverse data sources with strong generalization to novel&#10;objects imagined by DALL$\cdot$E 2 or captured in-the-wild with an iPhone.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.12358" label="1909.12358">
-        <attvalues>
-          <attvalue for="0" value="Can We Trust You? On Calibration of a Probabilistic Object Detector for&#10;  Autonomous Driving" />
-          <attvalue for="1" value="  Reliable uncertainty estimation is crucial for perception systems in safe&#10;autonomous driving. Recently, many methods have been proposed to model&#10;uncertainties in deep learning based object detectors. However, the estimated&#10;probabilities are often uncalibrated, which may lead to severe problems in&#10;safety critical scenarios. In this work, we identify such uncertainty&#10;miscalibration problems in a probabilistic LiDAR 3D object detection network,&#10;and propose three practical methods to significantly reduce errors in&#10;uncertainty calibration. Extensive experiments on several datasets show that&#10;our methods produce well-calibrated uncertainties, and generalize well between&#10;different datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.10658" label="1810.10658">
-        <attvalues>
-          <attvalue for="0" value="Sports Camera Calibration via Synthetic Data" />
-          <attvalue for="1" value="  Calibrating sports cameras is important for autonomous broadcasting and&#10;sports analysis. Here we propose a highly automatic method for calibrating&#10;sports cameras from a single image using synthetic data. First, we develop a&#10;novel camera pose engine. The camera pose engine has only three significant&#10;free parameters so that it can effectively generate a lot of camera poses and&#10;corresponding edge (i.e, field marking) images. Then, we learn compact deep&#10;features via a siamese network from paired edge image and camera pose and build&#10;a feature-pose database. After that, we use a novel two-GAN (generative&#10;adversarial network) model to detect field markings in real images. Finally, we&#10;query an initial camera pose from the feature-pose database and refine camera&#10;poses using truncated distance images. We evaluate our method on both synthetic&#10;and real data. Our method not only demonstrates the robustness on the synthetic&#10;data but also achieves the state-of-the-art accuracy on a standard soccer&#10;dataset and very high performance on a volleyball dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.01363" label="2303.01363">
-        <attvalues>
-          <attvalue for="0" value="Deep-NFA: a Deep $\textit{a contrario}$ Framework for Small Object&#10;  Detection" />
-          <attvalue for="1" value="  The detection of small objects is a challenging task in computer vision.&#10;Conventional object detection methods have difficulty in finding the balance&#10;between high detection and low false alarm rates. In the literature, some&#10;methods have addressed this issue by enhancing the feature map responses, but&#10;without guaranteeing robustness with respect to the number of false alarms&#10;induced by background elements. To tackle this problem, we introduce an&#10;$\textit{a contrario}$ decision criterion into the learning process to take&#10;into account the unexpectedness of small objects. This statistic criterion&#10;enhances the feature map responses while controlling the number of false alarms&#10;(NFA) and can be integrated into any semantic segmentation neural network. Our&#10;add-on NFA module not only allows us to obtain competitive results for small&#10;target and crack detection tasks respectively, but also leads to more robust&#10;and interpretable results.&#10;" />
-          <attvalue for="2" value="&#10;The detection of small objects, defined as having {at least} one small ($1-5$ pixels) dimension, is a great challenge in computer vision.&#10;This issue is of a key interest in many real-world applications, {e.g.} in the defense and security fields for surveillance or in the medical field for early and accurate diagnosis. &#10;The rise of deep learning methods has led to impressive progress in object detection in the past decades, mostly thanks to their ability to extract non-linear features well adapted to the downstream task \cite{ren2015faster,redmon2016you}. However, most state-of-the-art (SOTA) object detection methods perform poorly on small objects. {Indeed, neural networks (NN) based on bounding box regression either have anchors too large for detecting small objects, or require a significant decrease in the IoU threshold, which leads to {multiple detections of a same target~\cite{dai2021attentional}.}&#10;Semantic segmentation methods are then preferred, although their performance remains limited.} This is first{ly} due to the nature of the objects: their surface area is made of only few pixels, and they do not present a specific structure. Secondly, small objects are often {partially hidden in complex and highly textured backgrounds}, leading to many false alarms. &#10;Moreover, dealing with small object detection results in learning from highly class-imbalanced datasets. Thus, in {a} semantic segmentation scheme, their features cannot be learned {easily} {due to the small number of samples} in comparison with the background class. &#10;&#10;Few approaches have been proposed to improve small object detection. Some of them focus on augmenting or oversampling the dataset~\cite{kisantal2019augmentation, akyon2022sahi}, while others focus on improving small object feature-enhancement. {Among} the latter, we can cite Feature Pyramid Networks (FPN) and their variants~\cite{lin2017feature,deng2021extended}, {whose} multi-scale approach is beneficial for the detection of objects of various sizes. Other works include attention mechanisms to learn long-range dependencies~\cite{wu2022uiu}, or use super-resolution to enhance feature responses of small objects~\cite{deng2021extended}. However, these methods do not take advantage of the unexpectedness of small objects with respect to the background, as one could do in an anomaly detection approach with, for example, one{-}class classifiers~\cite{pmlr-v80-ruff18a}. This criterion is important to reduce the number of false alarms and thus find a better balance between precision and detection rate.&#10;&#10;We {therefore} propose a module, called NFA module, which takes into account the unexpectedness of small objects and enhances their feature responses by rejecting the background hypothesis.&#10;Our module relies on a contrario {decision} methods, introduced by {\cite{desolneux2003grouping}}. These methods allow to automatically {derive a decision criterion} with regards to hypothesis testing and {draw inspiration} from theories of perception, in particular the Gestalt theory~\cite{desolneux2007gestalt}. &#10;They are based on the Helmholtz principle, which states that a large deviation from a random pattern is probably due to the presence of a structure. &#10;Concretely, {the} a contrario detection methods consist in rejecting a naive model describing the background by {minimizing} the Number of False Alarms (NFA, defined in Section~\ref{generalite_NFA}). One of the benefits of this {kind of approaches} is that it does not rely on any prior knowledge about the image {structures: it only assumes a naive model for the noise.} {Moreover}, it allows {for} the control of the NFA. &#10;&#10;Many a contrario formulations have been proposed in the literature, relying on different naive models. {An important distinction is the kind of considered images, namely either binary or gray-level images. In the first case, a widely used naive model is the uniform spatial distribution of the ``true'' pixels in the image lattice, leading to binomial distribution for the number of ``true'' pixels falling within any given parametric shape~\cite{desolneux2003grouping, HegaratMascle2019}. In the second case, a widely used naive model is the Gaussian distribution of the pixel gray-level values, leading to {chi-square} distribution for the sum of the square{d} errors~\cite{Robin2010, ipol.2019.263}.} &#10;Our work will be based on{~\cite{ipol.2019.263}}. &#10;&#10;In the literature, the a contrario test is applied either on natural images directly or after extracting features from the image by classical image processing methods. &#10;This filtering step can easily be replaced by {NN}. Indeed, when {looking at feature maps of a NN} trained for detecting objects, {the} objects to detect stand out against a background made of noise. It is possible to apply a contrario detection on these feature maps, as done in{~\cite{ipol.2019.263}}, {but such a post-processing step appears suboptimal since it does not guide the training step.}&#10;We therefore propose to include {the chosen} a contrario criterion in the training, through our NFA module. This module can be {integrated into}&#10;any segmentation NN, and can even take advantage of multi-scale information if the backbone allows it. &#10;Our main contributions can be summarized as follows:&#10;\begin{enumerate}&#10; \item We propose a new module specifically designed for small object detection that takes into account the unexpectedness of an object thanks to an a contrario decision criterion.&#10; \item We demonstrate the effectiveness of our method for infrared small target detection, which achieves state-of-the-art performance while also leading to more interpretable results.&#10; \item We extend our experiments to other backbone and application, namely crack detection, in order to show the generalization ability of the proposed method. &#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Neural Network Enhancement, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Statistics, Computer Vision Challenges, Object Detection Methods" />
-        </attvalues>
-      </node>
-      <node id="1506.02640" label="1506.02640">
-        <attvalues>
-          <attvalue for="0" value="You Only Look Once: Unified, Real-Time Object Detection" />
-          <attvalue for="1" value="  We present YOLO, a new approach to object detection. Prior work on object&#10;detection repurposes classifiers to perform detection. Instead, we frame object&#10;detection as a regression problem to spatially separated bounding boxes and&#10;associated class probabilities. A single neural network predicts bounding boxes&#10;and class probabilities directly from full images in one evaluation. Since the&#10;whole detection pipeline is a single network, it can be optimized end-to-end&#10;directly on detection performance.&#10;  Our unified architecture is extremely fast. Our base YOLO model processes&#10;images in real-time at 45 frames per second. A smaller version of the network,&#10;Fast YOLO, processes an astounding 155 frames per second while still achieving&#10;double the mAP of other real-time detectors. Compared to state-of-the-art&#10;detection systems, YOLO makes more localization errors but is far less likely&#10;to predict false detections where nothing exists. Finally, YOLO learns very&#10;general representations of objects. It outperforms all other detection methods,&#10;including DPM and R-CNN, by a wide margin when generalizing from natural images&#10;to artwork on both the Picasso Dataset and the People-Art Dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.08573" label="2012.08573">
-        <attvalues>
-          <attvalue for="0" value="Attentional Local Contrast Networks for Infrared Small Target Detection" />
-          <attvalue for="1" value="  To mitigate the issue of minimal intrinsic features for pure data-driven&#10;methods, in this paper, we propose a novel model-driven deep network for&#10;infrared small target detection, which combines discriminative networks and&#10;conventional model-driven methods to make use of both labeled data and the&#10;domain knowledge. By designing a feature map cyclic shift scheme, we modularize&#10;a conventional local contrast measure method as a depth-wise parameterless&#10;nonlinear feature refinement layer in an end-to-end network, which encodes&#10;relatively long-range contextual interactions with clear physical&#10;interpretability. To highlight and preserve the small target features, we also&#10;exploit a bottom-up attentional modulation integrating the smaller scale subtle&#10;details of low-level features into high-level features of deeper layers. We&#10;conduct detailed ablation studies with varying network depths to empirically&#10;verify the effectiveness and efficiency of the design of each component in our&#10;network architecture. We also compare the performance of our network against&#10;other model-driven methods and deep networks on the open SIRST dataset as well.&#10;The results suggest that our network yields a performance boost over its&#10;competitors. Our code, trained models, and results are available online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.07296" label="1902.07296">
-        <attvalues>
-          <attvalue for="0" value="Augmentation for small object detection" />
-          <attvalue for="1" value="  In recent years, object detection has experienced impressive progress.&#10;Despite these improvements, there is still a significant gap in the performance&#10;between the detection of small and large objects. We analyze the current&#10;state-of-the-art model, Mask-RCNN, on a challenging dataset, MS COCO. We show&#10;that the overlap between small ground-truth objects and the predicted anchors&#10;is much lower than the expected IoU threshold. We conjecture this is due to two&#10;factors; (1) only a few images are containing small objects, and (2) small&#10;objects do not appear enough even within each image containing them. We thus&#10;propose to oversample those images with small objects and augment each of those&#10;images by copy-pasting small objects many times. It allows us to trade off the&#10;quality of the detector on large objects with that on small objects. We&#10;evaluate different pasting augmentation strategies, and ultimately, we achieve&#10;9.7\% relative improvement on the instance segmentation and 7.1\% on the object&#10;detection of small objects, compared to the current state of the art method on&#10;MS COCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.03144" label="1612.03144">
-        <attvalues>
-          <attvalue for="0" value="Feature Pyramid Networks for Object Detection" />
-          <attvalue for="1" value="  Feature pyramids are a basic component in recognition systems for detecting&#10;objects at different scales. But recent deep learning object detectors have&#10;avoided pyramid representations, in part because they are compute and memory&#10;intensive. In this paper, we exploit the inherent multi-scale, pyramidal&#10;hierarchy of deep convolutional networks to construct feature pyramids with&#10;marginal extra cost. A top-down architecture with lateral connections is&#10;developed for building high-level semantic feature maps at all scales. This&#10;architecture, called a Feature Pyramid Network (FPN), shows significant&#10;improvement as a generic feature extractor in several applications. Using FPN&#10;in a basic Faster R-CNN system, our method achieves state-of-the-art&#10;single-model results on the COCO detection benchmark without bells and&#10;whistles, surpassing all existing single-model entries including those from the&#10;COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU&#10;and thus is a practical and accurate solution to multi-scale object detection.&#10;Code will be made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.07021" label="2003.07021">
-        <attvalues>
-          <attvalue for="0" value="Extended Feature Pyramid Network for Small Object Detection" />
-          <attvalue for="1" value="  Small object detection remains an unsolved challenge because it is hard to&#10;extract information of small objects with only a few pixels. While scale-level&#10;corresponding detection in feature pyramid network alleviates this problem, we&#10;find feature coupling of various scales still impairs the performance of small&#10;objects. In this paper, we propose extended feature pyramid network (EFPN) with&#10;an extra high-resolution pyramid level specialized for small object detection.&#10;Specifically, we design a novel module, named feature texture transfer (FTT),&#10;which is used to super-resolve features and extract credible regional details&#10;simultaneously. Moreover, we design a foreground-background-balanced loss&#10;function to alleviate area imbalance of foreground and background. In our&#10;experiments, the proposed EFPN is efficient on both computation and memory, and&#10;yields state-of-the-art results on small traffic-sign dataset Tsinghua-Tencent&#10;100K and small category of general object detection dataset MS COCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.00968" label="2212.00968">
-        <attvalues>
-          <attvalue for="0" value="UIU-Net: U-Net in U-Net for Infrared Small Object Detection" />
-          <attvalue for="1" value="  Learning-based infrared small object detection methods currently rely heavily&#10;on the classification backbone network. This tends to result in tiny object&#10;loss and feature distinguishability limitations as the network depth increases.&#10;Furthermore, small objects in infrared images are frequently emerged bright and&#10;dark, posing severe demands for obtaining precise object contrast information.&#10;For this reason, we in this paper propose a simple and effective ``U-Net in&#10;U-Net'' framework, UIU-Net for short, and detect small objects in infrared&#10;images. As the name suggests, UIU-Net embeds a tiny U-Net into a larger U-Net&#10;backbone, enabling the multi-level and multi-scale representation learning of&#10;objects. Moreover, UIU-Net can be trained from scratch, and the learned&#10;features can enhance global and local contrast information effectively. More&#10;specifically, the UIU-Net model is divided into two modules: the&#10;resolution-maintenance deep supervision (RM-DS) module and the&#10;interactive-cross attention (IC-A) module. RM-DS integrates Residual U-blocks&#10;into a deep supervision network to generate deep multi-scale&#10;resolution-maintenance features while learning global context information.&#10;Further, IC-A encodes the local context information between the low-level&#10;details and high-level semantic features. Extensive experiments conducted on&#10;two infrared single-frame image datasets, i.e., SIRST and Synthetic datasets,&#10;show the effectiveness and superiority of the proposed UIU-Net in comparison&#10;with several state-of-the-art infrared small object detection methods. The&#10;proposed UIU-Net also produces powerful generalization performance for video&#10;sequence infrared small object datasets, e.g., ATR ground/air video sequence&#10;dataset. The codes of this work are available openly at&#10;\url{https://github.com/danfenghong/IEEE_TIP_UIU-Net}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.03881" label="2306.03881">
-        <attvalues>
-          <attvalue for="0" value="Emergent Correspondence from Image Diffusion" />
-          <attvalue for="1" value="  Finding correspondences between images is a fundamental problem in computer&#10;vision. In this paper, we show that correspondence emerges in image diffusion&#10;models without any explicit supervision. We propose a simple strategy to&#10;extract this implicit knowledge out of diffusion networks as image features,&#10;namely DIffusion FeaTures (DIFT), and use them to establish correspondences&#10;between real images. Without any additional fine-tuning or supervision on the&#10;task-specific data or annotations, DIFT is able to outperform both&#10;weakly-supervised methods and competitive off-the-shelf features in identifying&#10;semantic, geometric, and temporal correspondences. Particularly for semantic&#10;correspondence, DIFT from Stable Diffusion is able to outperform DINO and&#10;OpenCLIP by 19 and 14 accuracy points respectively on the challenging SPair-71k&#10;benchmark. It even outperforms the state-of-the-art supervised methods on 9 out&#10;of 18 categories while remaining on par for the overall performance. Project&#10;page: https://diffusionfeatures.github.io&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\mypara{Visual Correspondence.} Establishing visual correspondences between different images is crucial for various computer vision tasks such as Structure-from-Motion / 3D reconstruction~\cite{agarwal2011building, schoenberger2016sfm, ozyecsil2017surveysfm, schoenberger2016mvs}, object tracking~\cite{gao2022aiatrack, yan2022towards}, image recognition~\cite{peng2017object, tang2020revisiting, cao2020concept} and segmentation~\cite{liu2021unsupervised, li2019deep, rubio2012unsupervised, hamilton2022stego}.&#10;Traditionally, correspondences are established using hand-designed features, such as SIFT~\cite{lowe2004distinctive} and SURF~\cite{bay2006surf}. With the advent of deep learning, methods that learn to find correspondences in a supervised-learning regime have shown promising results~\cite{lee2021patchmatch, cho2021cats, kim2022transformatcher, huang2022learning}. However, these approaches are difficult to scale due to the reliance on ground-truth correspondence annotations. To overcome difficulties in collecting a large number of image pairs with annotated correspondences, recent works have started looking into how to build visual correspondence models with weak supervision~\cite{wang2020learning} or self-supervision~\cite{wang2019learning, jabri2020walk}. Meanwhile, recent works on self-supervised representation learning~\cite{dino} has yielded strong per-pixel features that could be used to identify visual correspondence~\cite{tumanyan2022splicing, amir2021deep, dino, hamilton2022stego}. In particular, recent work has also found that the internal representation of Generative Adversarial Networks (GAN) \cite{goodfellow2020generative} could be used for identifying visual correspondence~\cite{zhang21, gangealing, mu2022coordgan} within certain image categories. Our work shares similar spirits with these works: we show that diffusion models could generate features that are useful for identifying visual correspondence on general images. In addition, we show that features generated at different timesteps and different layers of the de-noising process encode different information that could be used for determining correspondences needed for different downstream tasks.&#10;&#10;\mypara{Diffusion Model}~\cite{song2019generative, ho2020denoising, song2020score, karras2022elucidating} is a powerful family of generative models. Ablated Diffusion Model~\cite{adm} first showed that diffusion could surpass GAN's image generation quality on ImageNet~\cite{deng2009imagenet}. Subsequently, the introduction of classifier-free guidance~\cite{ho2022classifier} and latent diffusion model~\cite{ldm} made it scale up to billions of text-image pairs~\cite{laion}, leading to the popular open-sourced text-to-image diffusion model, i.e., Stable Diffusion. With its superior generation ability, recently people also start investigating the internal representation of diffusion models. For example, previous works~\cite{tumanyan2022plug, hertz2022prompt} found that the intermediate-layer features and attention maps of diffusion models are crucial for controllable generations; other works~\cite{baranchuk2021label, xu2023open, zhao2023unleashing} explored adapting pre-trained diffusion models for various downstream visual recognition tasks. &#10;Different from these works, we are the first to directly evaluate the efficacy of features inherent to pre-trained diffusion models on various visual correspondence tasks. &#10; " />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Image Diffusion Models, Computer Science, Correspondence Analysis" />
-        </attvalues>
-      </node>
-      <node id="2109.11077" label="2109.11077">
-        <attvalues>
-          <attvalue for="0" value="A Novel Factor Graph-Based Optimization Technique for Stereo&#10;  Correspondence Estimation" />
-          <attvalue for="1" value="  Dense disparities among multiple views is essential for estimating the 3D&#10;architecture of a scene based on the geometrical relationship among the scene&#10;and the views or cameras. Scenes with larger extents of heterogeneous textures,&#10;differing scene illumination among the multiple views and with occluding&#10;objects affect the accuracy of the estimated disparities. Markov random fields&#10;(MRF) based methods for disparity estimation address these limitations using&#10;spatial dependencies among the observations and among the disparity estimates.&#10;These methods, however, are limited by spatially fixed and smaller neighborhood&#10;systems or cliques. In this work, we present a new factor graph-based&#10;probabilistic graphical model for disparity estimation that allows a larger and&#10;a spatially variable neighborhood structure determined based on the local scene&#10;characteristics. We evaluated our method using the Middlebury benchmark stereo&#10;datasets and the Middlebury evaluation dataset version 3.0 and compared its&#10;performance with recent state-of-the-art disparity estimation algorithms. The&#10;new factor graph-based method provided disparity estimates with higher accuracy&#10;when compared to the recent non-learning- and learning-based disparity&#10;estimation algorithms. In addition to disparity estimation, our factor graph&#10;formulation can be useful for obtaining maximum a posteriori solution to&#10;optimization problems with complex and variable dependency structures as well&#10;as for other dense estimation problems such as optical flow estimation.&#10;" />
-          <attvalue for="2" value="&#10;The 3D architecture of an object or a scene can be estimated from two or more views of the scene by determining dense correspondences among the multiple views of the scene. A disparity map comprised of dense pixel-level correspondences between images in a stereo pair can be estimated using stereo matching algorithms. Stereo disparity estimation has wide-ranging applications such as robot navigation \cite{desouza2002vision}, aerial data analysis \cite{svensk2017evaluation}, image sequence analysis \cite{huang2008binocular}, and 3-D surface reconstruction \cite{remondino2008turning}. The presence of large untextured regions (homogeneous intensity), occluding objects and uneven intensity distributions in the stereo pair of images introduce significant challenges in estimating stereo disparity maps.&#10;&#10;Two broad categories of stereo matching methods are window-based and energy-based algorithms \cite{scharstein2002taxonomy}. The energy-based algorithms \cite{gu2008local} are global methods with a cost function defined as a function of the entire image extent \cite{yang2008stereo}. Window-based or local methods utilize a finite support window to define the cost function and are suitable for real-time applications. However, regions with homogeneous texture and occlusion affect the accuracy of the disparity estimated using local methods.&#10;&#10;In general, stereo disparity estimation procedures include an initial cost calculation step, a cost aggregation step, an optimal disparity estimation step, and a disparity refinement step. Some of the successful and commonly used disparity cost measures are normalized cross-correlation (NCC) \cite{roma2002comparative}, sum of squared differences (SSD) \cite{marghany20113d}, sum of absolute differences (SAD) \cite{parvathysurvey}, gradient-based measures \cite{de2011stereo}, feature-based measures \cite{wang2014feature}, rank transform (RT) \cite{gac2009high}, and census transform (CT) \cite{mei2011building}. Cost aggregation is useful for minimizing matching uncertainties and improving the accuracy of the estimates \cite{fang2012accelerating}. During cost aggregation, initial disparity cost measures within the support region of each pixel are aggregated \cite{hamid2020stereo} such as using a low pass filter with a fixed kernel size or a variable support window (VSW) \cite{tombari2008classification}, using adaptive support weight (ASW) \cite{yoon2006adaptive}, and using a cross-based support window \cite{zhang2009cross}. Using edge-preserving filters such as a bilateral filter (BF) \cite{yoon2006adaptive,zhu2015edge} and guided image filter (GIF) \cite{zhu2016edge} for cost aggregation provides a significant improvement on the final disparity estimates over the initial disparity estimates. An optimal disparity map or disparity estimates are obtained from the aggregated cost volume using optimization procedures such as the winner-take-all (WTA) \cite{chang2018real} or using global optimization procedures such as dynamic programming(DP) \cite{arranz2012multiresolution}, graph cut (GC) \cite{wang2013effective}, intrinsic curves \cite{tomasi1998stereo}, and belief propagation \cite{liang2011hardware} methods. &#10;&#10;Establishing dense correspondences among geometrical coordinates of multiple views is an ill-posed problem due to scene occlusion. In addition, larger scene extents with smoother texture characteristics and differences in scene illumination with respect to the multiple views/observers are sources of difficulties in disparity estimation. A successful strategy for improving the accuracy of the disparity estimates is to utilize the spatial dependencies of the scene characteristics as well as of the disparity estimates. Among the probabilistic inference formulations for estimating dense geometric correspondences \cite{sun2003stereo}, Markov Random Fields (MRF) based approaches have been successful in modeling spatial geometrical dependencies \cite{kim2015error}. In brief, unknown true disparities among multiple views of a scene are modeled as random variables on the pixel lattice (random field) and dependence among random variables is assumed to follow Markov property. Thus, MRF represents a joint distribution of the random field using conditional distributions of each of the random variables. In learning-based MRF models, parameters of the MRF potential functions are either learned separately from training data \cite{lan2006efficient,roth2009fields} or along with the unknown states of the random variables \cite{zhang2005parameter}. &#10;&#10;One of the limitations of the MRF models is that the neighborhood system used for enforcing spatial dependencies needs to be maximal. Further, the chosen dependency structure is uniformly enforced for all the random variables. Therefore, pairwise cliques or $2 \times 2$ cliques are more commonly used in MRF models. While learning methods are available for optimizing MRF parameters and neighborhood structure, they are generally limited to specific tasks.&#10;&#10;In this paper, we present a new factor graph-based probabilistic graphical model (FGS algorithm) for disparity estimation that addresses the aforementioned MRF limitations. Specifically, our model allows a larger neighborhood system and a spatially variable neighborhood structure dependent on the local scene characteristics. The proposed factor graph framework can also be used for solving general-purpose optimization problems from its posterior distributions. Further, we present strategies for reducing the computational cost and accelerating convergence of factor graph messages namely by using a priori disparity distributions with smaller support and using factor node potential functions that significantly reduce marginalization calculations. A priori disparity probabilities were estimated using a previously developed disparity cost calculation framework \cite{Shabanian2021Hybrid}. We demonstrate the performance of the proposed probabilistic factor graph model by conducting extensive experiments using the Middlebury benchmark stereo datasets \cite{scharstein2003high}, \cite{scharstein2007learning}, \cite{hirschmuller2007evaluation}, \cite{scharstein2014high} and compare its performance with other state-of-the-art disparity estimation algorithms using Middlebury evaluation dataset version 3.0 \cite{scharstein2002taxonomy}.&#10;&#10;The remainder of the paper is structured as follows. In Section~\ref{sec:FG model}, we present a detailed description of the new probabilistic factor graph-based stereo disparity estimation (FGS) algorithm. We present our experimental results in Section~\ref{Experiments} and conclude this work in Section~\ref{Conclusions}.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1708.07987" label="1708.07987">
-        <attvalues>
-          <attvalue for="0" value="Stereo Matching With Color-Weighted Correlation, Hierarchical Belief&#10;  Propagation And Occlusion Handling" />
-          <attvalue for="1" value="  In this paper, we contrive a stereo matching algorithm with careful handling&#10;of disparity, discontinuity and occlusion. This algorithm works a worldwide&#10;matching stereo model which is based on minimization of energy. The global&#10;energy comprises two terms, firstly the data term and secondly the smoothness&#10;term. The data term is approximated by a color-weighted correlation, then&#10;refined in obstruct and low-texture areas in many applications of hierarchical&#10;loopy belief propagation algorithm. The results during the experiment are&#10;evaluated on the Middlebury data sets, showing that out algorithm is the top&#10;performer among all the algorithms listed there&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00605" label="2004.00605">
-        <attvalues>
-          <attvalue for="0" value="EPOS: Estimating 6D Pose of Objects with Symmetries" />
-          <attvalue for="1" value="  We present a new method for estimating the 6D pose of rigid objects with&#10;available 3D models from a single RGB input image. The method is applicable to&#10;a broad range of objects, including challenging ones with global or partial&#10;symmetries. An object is represented by compact surface fragments which allow&#10;handling symmetries in a systematic manner. Correspondences between densely&#10;sampled pixels and the fragments are predicted using an encoder-decoder&#10;network. At each pixel, the network predicts: (i) the probability of each&#10;object's presence, (ii) the probability of the fragments given the object's&#10;presence, and (iii) the precise 3D location on each fragment. A data-dependent&#10;number of corresponding 3D locations is selected per pixel, and poses of&#10;possibly multiple object instances are estimated using a robust and efficient&#10;variant of the PnP-RANSAC algorithm. In the BOP Challenge 2019, the method&#10;outperforms all RGB and most RGB-D and D methods on the T-LESS and LM-O&#10;datasets. On the YCB-V dataset, it is superior to all competitors, with a large&#10;margin over the second-best RGB method. Source code is at:&#10;cmp.felk.cvut.cz/epos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Classical Methods.&#10;In the early attempt, Roberts~\cite{roberts1963machine} assumed that objects can be constructed from transformations of known simple 3D models which were fit to edges extracted from a grayscale input image.&#10;The first practical approaches were relying on local image features~\cite{lowe1999object,collet2011moped} or template matching~\cite{brunelli2009template}, and assumed a grayscale or RGB input image.&#10;Later, with the introduction of the consumer-grade Kinect-like sensors, the attention of the research field was steered towards estimating the object pose from RGB-D images.&#10;Methods based on RGB-D template matching~\cite{hinterstoisser2012accv,hodan2015detection}, point-pair features~\cite{drost2010model,hinterstoisser2016going,vidal2018method}, 3D local features~\cite{guo2016comprehensive}, and learning-based methods~\cite{brachmann2014learning,tejani2014latent,krull2015learning} demonstrated a superior performance over RGB-only counterparts.&#10;&#10;\customparagraph{CNN-Based Methods.}&#10;Recent methods are based on convolutional neural networks (CNN's) and focus primarily on estimating the object pose from RGB images.&#10;A popular approach is to establish 2D-3D correspondences by predicting the 2D projections of a fixed set of 3D keypoints, which are pre-selected for each object model, and solve for the object pose using PnP-RANSAC~\cite{rad2017bb8,pavlakos20176,oberweger2018making,tekin2018real,tremblay2018deep,fu2019deephmap++,hu2019segmentation,peng2019pvnet}.&#10;Methods establishing the correspondences in the opposite direction, \ie~by predicting the 3D object coordinates~\cite{brachmann2014learning} for a densely sampled set of pixels, have been also proposed~\cite{jafari2018ipose,nigam2018detect,zakharov2019dpod,park2019pix2pose,li2019cdpn}.&#10;As discussed below, none of the existing correspondence-based methods can reliably handle pose ambiguity due to object symmetries.&#10;&#10;Another approach is to localize the objects with 2D bounding boxes, and for each box predict the pose by regression~\cite{xiang2017posecnn,li2018unified,manhardt2019explaining} or by classification into discrete viewpoints~\cite{kehl2017ssd,corona2018pose,sundermeyer2019augmented}.&#10;However, in the case of occlusion, estimating accurate 2D bounding boxes covering the whole object (including the invisible parts) is problematic~\cite{kehl2017ssd}.&#10;&#10;Despite promising results, the recent CNN-based RGB methods are inferior to the classical RGB-D and D methods, as reported in~\cite{bop19challenge,hodan2018bop}.&#10;Using the depth image as an additional input of the CNN is a promising research direction~\cite{li2018unified,sock2018multi,wang2019densefusion}, but with a limited range of applications.&#10;&#10;\customparagraph{Handling Object Symmetries.}&#10;The many-to-many relationship of corresponding 2D and 3D locations, which arises in the case of object symmetries (Sec.~\ref{sec:intro}), degrades the performance of correspondence-based methods which assume a one-to-one relationship.&#10;In particular, classification-based methods predict for each pixel up to one corresponding 3D location~\cite{brachmann2014learning,nigam2018detect}, or for each 3D keypoint up to one 2D location which is typically given by the maximum response in a predicted heatmap~\cite{pavlakos20176,oberweger2018making,fu2019deephmap++}.&#10;This may result in a set of correspondences which carries only a limited support for each of the possible poses.&#10;On the other hand, regression-based methods~\cite{tekin2018real,zakharov2019dpod,peng2019pvnet} need to compromise among the possible corresponding locations and tend to return the average, which is often not a valid solution.&#10;For example, the average of all points on a sphere is the center of the sphere, which is not a valid surface location.&#10;&#10;The problem of pose ambiguity due to object symmetries has been approached by several methods.&#10;Rad and Lepetit~\cite{rad2017bb8} assume that the global object symmetries are known and propose a pose normalization applicable to the case when the projection of the axis of symmetry is close to vertical.&#10;Pitteri~\etal~\cite{pitteri2019object} introduce a pose normalization that is not limited to this special case.&#10;Kehl~\etal~\cite{kehl2017ssd} train a classifier for only a subset of viewpoints defined by global object symmetries.&#10;Corona~\etal~\cite{corona2018pose} show that predicting the order of rotational symmetry can improve the accuracy of pose estimation.&#10;Xiang~\etal~\cite{xiang2017posecnn} optimize a loss function that is invariant to global object symmetries.&#10;Park~\etal~\cite{park2019pix2pose} guide pose regression by calculating the loss \wrt to the closest symmetric pose.&#10;However, all of these approaches cover only pose ambiguities due to global object symmetries. Ambiguities due to partial object symmetries (\ie when the visible object part has multiple possible fits to the entire object surface) are not covered.&#10;&#10;As EPOS, the methods by Manhardt~\etal~\cite{manhardt2019explaining} and Li~\etal~\cite{li2018unified} can handle pose ambiguities due to both global and partial object symmetries without requiring any a priori information about the symmetries.&#10;The first~\cite{manhardt2019explaining} predicts multiple poses for each object instance to estimate the distribution of possible poses induced by symmetries.&#10;The second~\cite{li2018unified} deals with the possibly non-unimodal pose distribution by a classification and regression scheme applied to the rotation and translation space.&#10;Nevertheless, both methods rely on estimating accurate 2D bounding boxes which is problematic when the objects are occluded~\cite{kehl2017ssd}.&#10;&#10;\customparagraph{Object Representation.}&#10;To increase the robustness of 6D object pose tracking against occlusion, Crivellaro~\etal~\cite{crivellaro2017robust} represent an object by a set of parts and estimate the 6D pose of each part by predicting the 2D projections of pre-selected 3D keypoints.&#10;Brachmann~\etal~\cite{brachmann2014learning} and Nigam~\etal~\cite{nigam2018detect} split the 3D bounding box of the object model into uniform bins and predict up to one corresponding bin per pixel.&#10;They represent each bin with its center which yields correspondences with limited precision.&#10;&#10;For human pose estimation, G\&quot;{u}ler~\etal~\cite{alp2018densepose} segment the 3D surface of the human body into&#10;semantically-defined parts.&#10;At each pixel, they predict a label of the corresponding part and the UV texture coordinates defining the precise location on the part.&#10;In contrast, to effectively capture the partial object symmetries, we represent an object by a set of compact surface fragments of near-uniform size and predict possibly multiple labels of the corresponding fragments per pixel.&#10;Besides, we regress the precise location in local 3D coordinates of the fragment instead of the UV coordinates.&#10;Using the UV coordinates requires a well-defined topology of the mesh model, which may need manual intervention, and is problematic for objects with a complicated surface such as a coil or an engine~\cite{drost2017introducing}.&#10;&#10;\customparagraph{Model Fitting.}&#10;Many of the recent correspondence-based methods, \eg~\cite{park2019pix2pose,zakharov2019dpod,rad2017bb8,tekin2018real}, estimate the pose using the vanilla PnP-RANSAC algorithm~\cite{fischler1981random,lepetit2009epnp} implemented in the OpenCV function solvePnPRansac.&#10;We show that a noticeable improvement can be achieved by replacing the vanilla with a modern robust estimator.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2108.03819" label="2108.03819">
-        <attvalues>
-          <attvalue for="0" value="DistillPose: Lightweight Camera Localization Using Auxiliary Learning" />
-          <attvalue for="1" value="  We propose a lightweight retrieval-based pipeline to predict 6DOF camera&#10;poses from RGB images. Our pipeline uses a convolutional neural network (CNN)&#10;to encode a query image as a feature vector. A nearest neighbor lookup finds&#10;the pose-wise nearest database image. A siamese convolutional neural network&#10;regresses the relative pose from the nearest neighboring database image to the&#10;query image. The relative pose is then applied to the nearest neighboring&#10;absolute pose to obtain the query image's final absolute pose prediction. Our&#10;model is a distilled version of NN-Net that reduces its parameters by 98.87%,&#10;information retrieval feature vector size by 87.5%, and inference time by&#10;89.18% without a significant decrease in localization accuracy.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Camera pose estimation is used in a variety of applications such as autonomous navigation, augmented reality, structure from motion, and simultaneous localization and mapping \cite{shavit2019introduction}.&#10;Given a source of information such as RGB images or depth information, it aims at inferring the 6D camera pose, describing its orientation and position in the scene.&#10;&#10;Classically, a feature descriptor (\cite{lowe2004Sift,bay2008Surf,calonder2010BRIEFDescriptor,rublee2011ORB,yi2016LIFT}) would be used to match 2D points to 3D points.&#10;Assuming the camera is calibrated, solving the Perspective-n-Point (PnP) problem on the correspondences leads to an estimated camera pose \cite{szeliski2010computerVisionBook}.&#10;Such pose estimation approaches are usually very accurate, however, they are often based on prohibitive assumptions: an accurate 3D map of the world is present and a robust key-point descriptor is used.&#10;Further, establishing correct 2D-3D correspondences is a significant challenge as key-point descriptors are sensitive to occlusions, different lighting conditions, and texture.&#10;&#10;Several machine learning-based approaches have been proposed that aim to solve this problem by predicting the 3D coordinate for a given 2D point in an image.&#10;\cite{shotton2013SCoReForest} uses random forests \cite{criminisi2013decisionForestsBook} to predict 3D coordinates using RGB images, depth images or both; It does not use the entire image to predict the 3D points but rather takes single pixels as input.&#10;The forest regressors are used in a RANSAC loop \cite{fischler1981randomRansac} where a pose is refined iteratively.&#10;\cite{brachmann2017Dsac} develops a differentiable RANSAC \cite{fischler1981randomRansac} pipeline for pose estimation.&#10;Such methods yield very impressive results.&#10;Their flaw is the need for scene-specific training--- thus, requiring a new predictor to be trained to adopt to any new scene.&#10;&#10;With the recent advancements in deep learning, the pose estimation task has been formulated as a regression problem where a 7-dimensional vector representing the absolute pose would be regressed.&#10;\cite{kendall2015posenet} introduced the first absolute pose regression model on top of CNNs.&#10;They use a GoogLeNet \cite{szegedy2015goingGoogLeNet} backbone to transform scene images to feature vectors;&#10;A fully-connected layer regresses both absolute translations and rotation unit quaternions.&#10;\cite{kendall2016modellingBayesianPoseNet,kendall2017geometricLossFunctions,walch2017LSTMPose} proposed small variations to the PoseNet \cite{kendall2015posenet} architecture to reduce median localization error.&#10;Pose regression from RGB images is much faster than 2D-3D correspondence-based approaches.&#10;However, it suffers from two drawbacks: 1) The network is scene-specific and requires a significant amount of offline training and 2) it lacks in performance compared to correspondence-based approaches.&#10;NN-Net \cite{laskar2017camera} address these limitations by decomposing the absolute pose regression problem into two components: information retrieval and relative pose regression.&#10;The information retrieval component finds a reference image (with corresponding absolute pose) that is closest to the query image (See Fig. \ref{figure:quadruplets-batch}).&#10;The relative pose regression component then regresses the relative pose and applies it to the reference image's absolute pose.&#10;RelocNet \cite{balntas2018relocnet} correlates learned features to changes in the viewing camera frustum to improve nearest neighbor retrieval.&#10;CamNet \cite{ding2019camnet} introduces auxiliary learning in a coarse-to-fine framework to significantly reduce median localiztion error.&#10;This setup is able to generalize to new and unseen scenes-- as relative pose regression is not dependent on the scene's world coordinates.&#10;It also outperforms most absolute pose regression solutions.&#10;These improvements come at a computational and storage cost; Running deep siamese networks on edge devices (i.e. mobile phones, robots) is slow.&#10;Using five nearest neighbors (as in \cite{ding2019camnet} and \cite{laskar2017camera}) requires five inference steps.&#10;Storing thousands of database images requires plenty of on-device memory.&#10;This renders on-device camera localization in realtime unpractical for many robotic applications.&#10;In this paper, our contributions are twofold: we propose a novel layerwise training approach that enforces earlier layers to learn better pose-aware embeddings, and we use angle-based and frustum-based auxiliary losses in a fine-tuning step to:&#10;&#10;\begin{itemize}&#10; \item Use a single nearest neighbor (instead of 5 in NN-Net \cite{laskar2017camera}) and remove the need for a RANSAC pose filtering algorithm,&#10; \item Reduce the model size by 98.87\% (from 22.3M parameters in NN-Net \cite{laskar2017camera} to 248K parameters),&#10; \item Reduce the information retrieval feature vector size by 87.5\% (from a 512-dimensional vector to a 64-dimensional one),&#10; \item Reduce our inference time by 89.18\% (from 37ms to 4ms),&#10; \item Outperform NN-Net \cite{laskar2017camera} on information retrieval while retaining the performance on pose regression.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Pose Estimation, Mathematics, Computer Science, Neural Networks, Information Retrieval" />
-        </attvalues>
-      </node>
-      <node id="1907.05272" label="1907.05272">
-        <attvalues>
-          <attvalue for="0" value="Introduction to Camera Pose Estimation with Deep Learning" />
-          <attvalue for="1" value="  Over the last two decades, deep learning has transformed the field of&#10;computer vision. Deep convolutional networks were successfully applied to learn&#10;different vision tasks such as image classification, image segmentation, object&#10;detection and many more. By transferring the knowledge learned by deep models&#10;on large generic datasets, researchers were further able to create fine-tuned&#10;models for other more specific tasks. Recently this idea was applied for&#10;regressing the absolute camera pose from an RGB image. Although the resulting&#10;accuracy was sub-optimal, compared to classic feature-based solutions, this&#10;effort led to a surge of learning-based pose estimation methods. Here, we&#10;review deep learning approaches for camera pose estimation. We describe key&#10;methods in the field and identify trends aiming at improving the original deep&#10;pose regression solution. We further provide an extensive cross-comparison of&#10;existing learning-based pose estimators, together with practical notes on their&#10;execution for reproducibility purposes. Finally, we discuss emerging solutions&#10;and potential future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.09114" label="1603.09114">
-        <attvalues>
-          <attvalue for="0" value="LIFT: Learned Invariant Feature Transform" />
-          <attvalue for="1" value="  We introduce a novel Deep Network architecture that implements the full&#10;feature point handling pipeline, that is, detection, orientation estimation,&#10;and feature description. While previous works have successfully tackled each&#10;one of these problems individually, we show how to learn to do all three in a&#10;unified manner while preserving end-to-end differentiability. We then&#10;demonstrate that our Deep pipeline outperforms state-of-the-art methods on a&#10;number of benchmark datasets, without the need of retraining.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.07427" label="1505.07427">
-        <attvalues>
-          <attvalue for="0" value="PoseNet: A Convolutional Network for Real-Time 6-DOF Camera&#10;  Relocalization" />
-          <attvalue for="1" value="  We present a robust and real-time monocular six degree of freedom&#10;relocalization system. Our system trains a convolutional neural network to&#10;regress the 6-DOF camera pose from a single RGB image in an end-to-end manner&#10;with no need of additional engineering or graph optimisation. The algorithm can&#10;operate indoors and outdoors in real time, taking 5ms per frame to compute. It&#10;obtains approximately 2m and 6 degree accuracy for large scale outdoor scenes&#10;and 0.5m and 10 degree accuracy indoors. This is achieved using an efficient 23&#10;layer deep convnet, demonstrating that convnets can be used to solve&#10;complicated out of image plane regression problems. This was made possible by&#10;leveraging transfer learning from large scale classification data. We show the&#10;convnet localizes from high level features and is robust to difficult lighting,&#10;motion blur and different camera intrinsics where point based SIFT registration&#10;fails. Furthermore we show how the pose feature that is produced generalizes to&#10;other scenes allowing us to regress pose with only a few dozen training&#10;examples. PoseNet code, dataset and an online demonstration is available on our&#10;project webpage, at http://mi.eng.cam.ac.uk/projects/relocalisation/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1409.4842" label="1409.4842">
-        <attvalues>
-          <attvalue for="0" value="Going Deeper with Convolutions" />
-          <attvalue for="1" value="  We propose a deep convolutional neural network architecture codenamed&#10;&quot;Inception&quot;, which was responsible for setting the new state of the art for&#10;classification and detection in the ImageNet Large-Scale Visual Recognition&#10;Challenge 2014 (ILSVRC 2014). The main hallmark of this architecture is the&#10;improved utilization of the computing resources inside the network. This was&#10;achieved by a carefully crafted design that allows for increasing the depth and&#10;width of the network while keeping the computational budget constant. To&#10;optimize quality, the architectural decisions were based on the Hebbian&#10;principle and the intuition of multi-scale processing. One particular&#10;incarnation used in our submission for ILSVRC 2014 is called GoogLeNet, a 22&#10;layers deep network, the quality of which is assessed in the context of&#10;classification and detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1509.05909" label="1509.05909">
-        <attvalues>
-          <attvalue for="0" value="Modelling Uncertainty in Deep Learning for Camera Relocalization" />
-          <attvalue for="1" value="  We present a robust and real-time monocular six degree of freedom visual&#10;relocalization system. We use a Bayesian convolutional neural network to&#10;regress the 6-DOF camera pose from a single RGB image. It is trained in an&#10;end-to-end manner with no need of additional engineering or graph optimisation.&#10;The algorithm can operate indoors and outdoors in real time, taking under 6ms&#10;to compute. It obtains approximately 2m and 6 degrees accuracy for very large&#10;scale outdoor scenes and 0.5m and 10 degrees accuracy indoors. Using a Bayesian&#10;convolutional neural network implementation we obtain an estimate of the&#10;model's relocalization uncertainty and improve state of the art localization&#10;accuracy on a large scale outdoor dataset. We leverage the uncertainty measure&#10;to estimate metric relocalization error and to detect the presence or absence&#10;of the scene in the input image. We show that the model's uncertainty is caused&#10;by images being dissimilar to the training dataset in either pose or&#10;appearance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.00390" label="1704.00390">
-        <attvalues>
-          <attvalue for="0" value="Geometric Loss Functions for Camera Pose Regression with Deep Learning" />
-          <attvalue for="1" value="  Deep learning has shown to be effective for robust and real-time monocular&#10;image relocalisation. In particular, PoseNet is a deep convolutional neural&#10;network which learns to regress the 6-DOF camera pose from a single image. It&#10;learns to localize using high level features and is robust to difficult&#10;lighting, motion blur and unknown camera intrinsics, where point based SIFT&#10;registration fails. However, it was trained using a naive loss function, with&#10;hyper-parameters which require expensive tuning. In this paper, we give the&#10;problem a more fundamental theoretical treatment. We explore a number of novel&#10;loss functions for learning camera pose which are based on geometry and scene&#10;reprojection error. Additionally we show how to automatically learn an optimal&#10;weighting to simultaneously regress position and orientation. By leveraging&#10;geometry, we demonstrate that our technique significantly improves PoseNet's&#10;performance across datasets ranging from indoor rooms to a small city.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07890" label="1611.07890">
-        <attvalues>
-          <attvalue for="0" value="Image-based localization using LSTMs for structured feature correlation" />
-          <attvalue for="1" value="  In this work we propose a new CNN+LSTM architecture for camera pose&#10;regression for indoor and outdoor scenes. CNNs allow us to learn suitable&#10;feature representations for localization that are robust against motion blur&#10;and illumination changes. We make use of LSTM units on the CNN output, which&#10;play the role of a structured dimensionality reduction on the feature vector,&#10;leading to drastic improvements in localization performance. We provide&#10;extensive quantitative comparison of CNN-based and SIFT-based localization&#10;methods, showing the weaknesses and strengths of each. Furthermore, we present&#10;a new large-scale indoor dataset with accurate ground truth from a laser&#10;scanner. Experimental results on both indoor and outdoor public datasets show&#10;our method outperforms existing deep architectures, and can localize images in&#10;hard conditions, e.g., in the presence of mostly textureless surfaces, where&#10;classic SIFT-based methods fail.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.09733" label="1707.09733">
-        <attvalues>
-          <attvalue for="0" value="Camera Relocalization by Computing Pairwise Relative Poses Using&#10;  Convolutional Neural Network" />
-          <attvalue for="1" value="  We propose a new deep learning based approach for camera relocalization. Our&#10;approach localizes a given query image by using a convolutional neural network&#10;(CNN) for first retrieving similar database images and then predicting the&#10;relative pose between the query and the database images, whose poses are known.&#10;The camera location for the query image is obtained via triangulation from two&#10;relative translation estimates using a RANSAC based approach. Each relative&#10;pose estimate provides a hypothesis for the camera orientation and they are&#10;fused in a second RANSAC scheme. The neural network is trained for relative&#10;pose estimation in an end-to-end manner using training image pairs. In contrast&#10;to previous work, our approach does not require scene-specific training of the&#10;network, which improves scalability, and it can also be applied to scenes which&#10;are not available during the training of the network. As another main&#10;contribution, we release a challenging indoor localisation dataset covering 5&#10;different scenes registered to a common coordinate frame. We evaluate our&#10;approach using both our own dataset and the standard 7 Scenes benchmark. The&#10;results show that the proposed approach generalizes well to previously unseen&#10;scenes and compares favourably to other recent CNN-based methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.13508" label="2010.13508">
-        <attvalues>
-          <attvalue for="0" value="SHARP 2020: The 1st Shape Recovery from Partial Textured 3D Scans&#10;  Challenge Results" />
-          <attvalue for="1" value="  The SHApe Recovery from Partial textured 3D scans challenge, SHARP 2020, is&#10;the first edition of a challenge fostering and benchmarking methods for&#10;recovering complete textured 3D scans from raw incomplete data. SHARP 2020 is&#10;organised as a workshop in conjunction with ECCV 2020. There are two&#10;complementary challenges, the first one on 3D human scans, and the second one&#10;on generic objects. Challenge 1 is further split into two tracks, focusing,&#10;first, on large body and clothing regions, and, second, on fine body details. A&#10;novel evaluation metric is proposed to quantify jointly the shape&#10;reconstruction, the texture reconstruction and the amount of completed data.&#10;Additionally, two unique datasets of 3D scans are proposed, to provide raw&#10;ground-truth data for the benchmarks. The datasets are released to the&#10;scientific community. Moreover, an accompanying custom library of software&#10;routines is also released to the scientific community. It allows for processing&#10;3D scans, generating partial data and performing the evaluation. Results of the&#10;competition, analysed in comparison to baselines, show the validity of the&#10;proposed evaluation metrics, and highlight the challenging aspects of the task&#10;and of the datasets. Details on the SHARP 2020 challenge can be found at&#10;https://cvi2.uni.lu/sharp2020/.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Representing the physical world in 3D, including shape and&#10;colour, is key for industrial and research purposes~\cite{anguelov2005scape,yang2014semantic,hasler2009statistical,pishchulin2017building,bogo2014faust,xu2018multilevel,bronstein2008numerical,robinette1999caesar}.&#10;It includes, for example, areas from virtual reality to heritage conservation,&#10;or from medical treatment to fitness, entertainment and fashion.&#10;3D scanning allows to digitise the physical world, \eg objects and humans.&#10;Acquired 3D scans vary in quality depending&#10;on the scanning system used,&#10;the properties of the target and the environment.&#10;For example, a high-end photogrammetric scanning system with a fixed camera&#10;array might capture high-quality data at a high frame rate&#10;but might be bulky, have a fixed structure, suffer from occlusion&#10;and limited in scanning volume.&#10;Other high-end systems such as hand-held devices might produce accurate&#10;results, while easily transported and easily oriented to limit occlusion,&#10;but cannot handle movable targets and are time-consuming.&#10;On the other hand, low-end scanning systems might be flexible and easy to manipulate but produce&#10;low-quality scans.&#10;Limiting factors due to the target or the environment include&#10;varying levels of details (\eg finer anatomical parts),&#10;occlusion, non-rigidity, movement, and optical properties&#10;(\eg fabric, material, hair and reflection).&#10;Moreover, for time-consuming acquisition systems or moving targets, it might&#10;be desirable or only possible to capture partial data.&#10;In this work, defective and/or partial acquisitions, are both viewed as&#10;data with missing information that must be completed.&#10;&#10;The SHARP 2020 challenge for&#10;SHApe Recovery from Partial textured 3D scans&#10;is proposed to foster research and provide a benchmark on 3D shape and texture&#10;completion from partial 3D scan data.&#10;First, two new unique datasets of 3D textured scans are proposed to serve&#10;as reference data.&#10;These datasets contain thousands of scans of humans and generic objects with varied&#10;identities, clothing, colours, shapes and categories.&#10;One challenge is proposed per dataset,&#10;challenge 1 focusing on human scans&#10;and challenge 2 on object scans.&#10;Second, partial scans are generated synthetically, but randomly, to simulate&#10;a general pattern of partial data acquisition while still having access&#10;to ground truth data.&#10;Third, specific evaluation metrics are proposed to quantitatively measure&#10;the quality of the shape and texture reconstructions,&#10;and the amount of completed data.&#10;Fourth, reusable software libraries developed for the challenge are also&#10;made available.&#10;These contain routines&#10;to process 3D scans,&#10;to generate partial data,&#10;and to evaluate and analyse the submissions on the proposed benchmark.&#10;This paper summaries the SHARP challenge&#10;with a presentation of the proposed datasets, benchmark and evaluation method,&#10;as well as the results of the submitted methods&#10;and an analysis of the results.&#10;SHARP 2020 is the first edition of the challenge,&#10;held in conjunction with the 16th European Conference on Computer Vision (ECCV).&#10;&#10;In the following,&#10;Section~\ref{sec:datasets} describes the challenge and the proposed datasets.&#10;Section~\ref{sec:evaluation} describes the proposed evaluation protocol and,&#10;in Section~\ref{sec:results}, an extensive analysis on the results is presented.&#10;To conclude, the results and outcomes of the challenge are discussed in&#10;Section~\ref{sec:discussion}.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Data Benchmarking, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1503.05860" label="1503.05860">
-        <attvalues>
-          <attvalue for="0" value="Building Statistical Shape Spaces for 3D Human Modeling" />
-          <attvalue for="1" value="  Statistical models of 3D human shape and pose learned from scan databases&#10;have developed into valuable tools to solve a variety of vision and graphics&#10;problems. Unfortunately, most publicly available models are of limited&#10;expressiveness as they were learned on very small databases that hardly reflect&#10;the true variety in human body shapes. In this paper, we contribute by&#10;rebuilding a widely used statistical body representation from the largest&#10;commercially available scan database, and making the resulting model available&#10;to the community (visit http://humanshape.mpi-inf.mpg.de). As preprocessing&#10;several thousand scans for learning the model is a challenge in itself, we&#10;contribute by developing robust best practice solutions for scan alignment that&#10;quantitatively lead to the best learned models. We make implementations of&#10;these preprocessing steps also publicly available. We extensively evaluate the&#10;improved accuracy and generality of our new model, and show its improved&#10;performance for human body reconstruction from sparse input data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.10175" label="1811.10175">
-        <attvalues>
-          <attvalue for="0" value="Multilevel active registration for kinect human body scans: from low&#10;  quality to high quality" />
-          <attvalue for="1" value="  Registration of 3D human body has been a challenging research topic for over&#10;decades. Most of the traditional human body registration methods require manual&#10;assistance, or other auxiliary information such as texture and markers. The&#10;majority of these methods are tailored for high-quality scans from expensive&#10;scanners. Following the introduction of the low-quality scans from&#10;cost-effective devices such as Kinect, the 3D data capturing of human body&#10;becomes more convenient and easier. However, due to the inevitable holes,&#10;noises and outliers in the low-quality scan, the registration of human body&#10;becomes even more challenging. To address this problem, we propose a fully&#10;automatic active registration method which deforms a high-resolution template&#10;mesh to match the low-quality human body scans. Our registration method&#10;operates on two levels of statistical shape models: (1) the first level is a&#10;holistic body shape model that defines the basic figure of human; (2) the&#10;second level includes a set of shape models for every body part, aiming at&#10;capturing more body details. Our fitting procedure follows a coarse-to-fine&#10;approach that is robust and efficient. Experiments show that our method is&#10;comparable with the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.09633" label="2009.09633">
-        <attvalues>
-          <attvalue for="0" value="3D-FUTURE: 3D Furniture shape with TextURE" />
-          <attvalue for="1" value="  The 3D CAD shapes in current 3D benchmarks are mostly collected from online&#10;model repositories. Thus, they typically have insufficient geometric details&#10;and less informative textures, making them less attractive for comprehensive&#10;and subtle research in areas such as high-quality 3D mesh and texture recovery.&#10;This paper presents 3D Furniture shape with TextURE (3D-FUTURE): a&#10;richly-annotated and large-scale repository of 3D furniture shapes in the&#10;household scenario. At the time of this technical report, 3D-FUTURE contains&#10;20,240 clean and realistic synthetic images of 5,000 different rooms. There are&#10;9,992 unique detailed 3D instances of furniture with high-resolution textures.&#10;Experienced designers developed the room scenes, and the 3D CAD shapes in the&#10;scene are used for industrial production. Given the well-organized 3D-FUTURE,&#10;we provide baseline experiments on several widely studied tasks, such as joint&#10;2D instance segmentation and 3D object pose estimation, image-based 3D shape&#10;retrieval, 3D object reconstruction from a single image, and texture recovery&#10;for 3D shapes, to facilitate related future researches on our database.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Lots of 3D benchmarks have been established and made publicly available over the past decades \cite{chang2015shapenet,wu20153d,xiao2013sun3d,song2015sun,xiao2016sun,sun2018pix3d,xiang2014beyond,xiang2016objectnet3d,silberman2012indoor,dai2017scannet,hua2016scenenn,choi2016large,shilane2004princeton}. These datasets can be mainly divided into two groups, including 3D models and RGB-D scenes. We will briefly review some representative 3D benchmarks in the following. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Texture Analysis, Computer Science, Computer-Aided Design" />
-        </attvalues>
-      </node>
-      <node id="2108.06649" label="2108.06649">
-        <attvalues>
-          <attvalue for="0" value="Semi-supervised 3D Object Detection via Adaptive Pseudo-Labeling" />
-          <attvalue for="1" value="  3D object detection is an important task in computer vision. Most existing&#10;methods require a large number of high-quality 3D annotations, which are&#10;expensive to collect. Especially for outdoor scenes, the problem becomes more&#10;severe due to the sparseness of the point cloud and the complexity of urban&#10;scenes. Semi-supervised learning is a promising technique to mitigate the data&#10;annotation issue. Inspired by this, we propose a novel semi-supervised&#10;framework based on pseudo-labeling for outdoor 3D object detection tasks. We&#10;design the Adaptive Class Confidence Selection module (ACCS) to generate&#10;high-quality pseudo-labels. Besides, we propose Holistic Point Cloud&#10;Augmentation (HPCA) for unlabeled data to improve robustness. Experiments on&#10;the KITTI benchmark demonstrate the effectiveness of our method.&#10;" />
-          <attvalue for="2" value="&#10;3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}. The goal of this task is to estimate the categories and corresponding 3D bounding boxes of all targets in the scene. In recent years, great success has been made with the help of deep networks and large-scale datasets&#10;~\cite{qi2020imvotenet,yang2018pixor,yan2018second,zhou2018voxelnet}. All of these methods adopt a fully-supervised scheme and require a large number of annotated data, which makes this important task costly. In outdoor scenes, the cost of the annotation is much higher than that of indoor scenes due to the sparseness of the point cloud and the complexity of urban scenes. Therefore, a more effective solution is required to improve the network performance for the outdoor scene without a large quantity of annotated data.&#10;&#10;To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution. SSL allows us to exploit more information from unlabeled data. We can achieve decent performance with only a small amount of labeled data. To the best of our knowledge,~\cite{zhao2020sess} is the only deep learning method for 3D object detection under semi-supervised conditions. Promising results have been achieved on indoor scenes~\cite{sunrgb-d,scannet}. However, the consistency strategy they adopted may face challenges in those more sparse outdoor point cloud scenes. &#10;&#10;Motivated by these facts, in this work, we propose a novel semi-supervised 3D object detection method based on pseudo-labeling for outdoor scenes.Considering that the direct use of pseudo-labeling~\cite{iscen2019label} will produce inevitable noise that may accumulate errors and affect the performance, we adopt Adaptive Class Confidence Selection scheme (ACCS) for pseudo-label filtering. Specifically, we present ACCS by sorting predicted confidence and gradually increasing the ratio of pseudo labels. With the adaptive selection, we can greatly alleviate the over-confident errors of pseudo-labeling.&#10;Besides, we design Holistic Point Cloud Augmentation (HPCA) to unlabeled data to improve robustness by applies a combination of strong data augmentations, which includes pseudo label augmentation, global augmentation and object augmentation. These augmentations on unlabeled point clouds can further improve the robustness of the neural networks.&#10;&#10;Our main contributions can be summarized as follows:&#10;\begin{itemize}&#10;\item We propose a novel semi-supervised 3D object detection framework via adaptive pseudo-labeling for outdoor 3D scenes. The effectiveness is validated on KITTI benchmark.&#10;\item We present Adaptive Class Confidence Selection scheme (ACCS) to adaptively implement pseudo-label filtering on each class.&#10;\item We design Holistic Point Cloud Augmentation scheme (HPCA) to regularize unlabeled data and the detector in each iteration of pseudo-labeling.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Semi-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1904.08963" label="1904.08963">
-        <attvalues>
-          <attvalue for="0" value="VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas&#10;  Segmentation" />
-          <attvalue for="1" value="  Deep learning (DL) approaches are state-of-the-art for many medical image&#10;segmentation tasks. They offer a number of advantages: they can be trained for&#10;specific tasks, computations are fast at test time, and segmentation quality is&#10;typically high. In contrast, previously popular multi-atlas segmentation (MAS)&#10;methods are relatively slow (as they rely on costly registrations) and even&#10;though sophisticated label fusion strategies have been proposed, DL approaches&#10;generally outperform MAS. In this work, we propose a DL-based label fusion&#10;strategy (VoteNet) which locally selects a set of reliable atlases whose labels&#10;are then fused via plurality voting. Experiments on 3D brain MRI data show that&#10;by selecting a good initial atlas set MAS with VoteNet significantly&#10;outperforms a number of other label fusion strategies as well as a direct DL&#10;segmentation approach. We also provide an experimental analysis of the upper&#10;performance bound achievable by our method. While unlikely achievable in&#10;practice, this bound suggests room for further performance improvements.&#10;Lastly, to address the runtime disadvantage of standard MAS, all our results&#10;make use of a fast DL registration approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.02294" label="1712.02294">
-        <attvalues>
-          <attvalue for="0" value="Joint 3D Proposal Generation and Object Detection from View Aggregation" />
-          <attvalue for="1" value="  We present AVOD, an Aggregate View Object Detection network for autonomous&#10;driving scenarios. The proposed neural network architecture uses LIDAR point&#10;clouds and RGB images to generate features that are shared by two subnetworks:&#10;a region proposal network (RPN) and a second stage detector network. The&#10;proposed RPN uses a novel architecture capable of performing multimodal feature&#10;fusion on high resolution feature maps to generate reliable 3D object proposals&#10;for multiple object classes in road scenes. Using these proposals, the second&#10;stage detection network performs accurate oriented 3D bounding box regression&#10;and category classification to predict the extents, orientation, and&#10;classification of objects in 3D space. Our proposed architecture is shown to&#10;produce state of the art results on the KITTI 3D object detection benchmark&#10;while running in real time with a low memory footprint, making it a suitable&#10;candidate for deployment on autonomous vehicles. Code is at:&#10;https://github.com/kujason/avod&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10275" label="1711.10275">
-        <attvalues>
-          <attvalue for="0" value="3D Semantic Segmentation with Submanifold Sparse Convolutional Networks" />
-          <attvalue for="1" value="  Convolutional networks are the de-facto standard for analyzing&#10;spatio-temporal data such as images, videos, and 3D shapes. Whilst some of this&#10;data is naturally dense (e.g., photos), many other data sources are inherently&#10;sparse. Examples include 3D point clouds that were obtained using a LiDAR&#10;scanner or RGB-D camera. Standard &quot;dense&quot; implementations of convolutional&#10;networks are very inefficient when applied on such sparse data. We introduce&#10;new sparse convolutional operations that are designed to process&#10;spatially-sparse data more efficiently, and use them to develop&#10;spatially-sparse convolutional networks. We demonstrate the strong performance&#10;of the resulting models, called submanifold sparse convolutional networks&#10;(SSCNs), on two tasks involving semantic segmentation of 3D point clouds. In&#10;particular, our models outperform all prior state-of-the-art on the test set of&#10;a recent semantic segmentation competition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07759" label="1611.07759">
-        <attvalues>
-          <attvalue for="0" value="Multi-View 3D Object Detection Network for Autonomous Driving" />
-          <attvalue for="1" value="  This paper aims at high-accuracy 3D object detection in autonomous driving&#10;scenario. We propose Multi-View 3D networks (MV3D), a sensory-fusion framework&#10;that takes both LIDAR point cloud and RGB images as input and predicts oriented&#10;3D bounding boxes. We encode the sparse 3D point cloud with a compact&#10;multi-view representation. The network is composed of two subnetworks: one for&#10;3D object proposal generation and another for multi-view feature fusion. The&#10;proposal network generates 3D candidate boxes efficiently from the bird's eye&#10;view representation of 3D point cloud. We design a deep fusion scheme to&#10;combine region-wise features from multiple views and enable interactions&#10;between intermediate layers of different paths. Experiments on the challenging&#10;KITTI benchmark show that our approach outperforms the state-of-the-art by&#10;around 25% and 30% AP on the tasks of 3D localization and 3D detection. In&#10;addition, for 2D detection, our approach obtains 10.3% higher AP than the&#10;state-of-the-art on the hard data among the LIDAR-based methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.03194" label="2012.03194">
-        <attvalues>
-          <attvalue for="0" value="Computer Stereo Vision for Autonomous Driving" />
-          <attvalue for="1" value="  As an important component of autonomous systems, autonomous car perception&#10;has had a big leap with recent advances in parallel computing architectures.&#10;With the use of tiny but full-feature embedded supercomputers, computer stereo&#10;vision has been prevalently applied in autonomous cars for depth perception.&#10;The two key aspects of computer stereo vision are speed and accuracy. They are&#10;both desirable but conflicting properties, as the algorithms with better&#10;disparity accuracy usually have higher computational complexity. Therefore, the&#10;main aim of developing a computer stereo vision algorithm for resource-limited&#10;hardware is to improve the trade-off between speed and accuracy. In this&#10;chapter, we introduce both the hardware and software aspects of computer stereo&#10;vision for autonomous car systems. Then, we discuss four autonomous car&#10;perception tasks, including 1) visual feature detection, description and&#10;matching, 2) 3D information acquisition, 3) object detection/recognition and 4)&#10;semantic image segmentation. The principles of computer stereo vision and&#10;parallel computing on multi-threading CPU and GPU architectures are then&#10;detailed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.00072" label="2008.00072">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Object Tracking and Masking for Visual SLAM" />
-          <attvalue for="1" value="  In dynamic environments, performance of visual SLAM techniques can be&#10;impaired by visual features taken from moving objects. One solution is to&#10;identify those objects so that their visual features can be removed for&#10;localization and mapping. This paper presents a simple and fast pipeline that&#10;uses deep neural networks, extended Kalman filters and visual SLAM to improve&#10;both localization and mapping in dynamic environments (around 14 fps on a GTX&#10;1080). Results on the dynamic sequences from the TUM dataset using RTAB-Map as&#10;visual SLAM suggest that the approach achieves similar localization performance&#10;compared to other state-of-the-art methods, while also providing the position&#10;of the tracked dynamic objects, a 3D map free of those dynamic objects, better&#10;loop closure detection with the whole pipeline able to run on a robot moving at&#10;moderate speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.10692" label="2001.10692">
-        <attvalues>
-          <attvalue for="0" value="ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes" />
-          <attvalue for="1" value="  3D object detection has seen quick progress thanks to advances in deep&#10;learning on point clouds. A few recent works have even shown state-of-the-art&#10;performance with just point clouds input (e.g. VoteNet). However, point cloud&#10;data have inherent limitations. They are sparse, lack color information and&#10;often suffer from sensor noise. Images, on the other hand, have high resolution&#10;and rich texture. Thus they can complement the 3D geometry provided by point&#10;clouds. Yet how to effectively use image information to assist point cloud&#10;based detection is still an open question. In this work, we build on top of&#10;VoteNet and propose a 3D detection architecture called ImVoteNet specialized&#10;for RGB-D scenes. ImVoteNet is based on fusing 2D votes in images and 3D votes&#10;in point clouds. Compared to prior work on multi-modal detection, we explicitly&#10;extract both geometric and semantic features from the 2D images. We leverage&#10;camera parameters to lift these features to 3D. To improve the synergy of 2D-3D&#10;feature fusion, we also propose a multi-tower training scheme. We validate our&#10;model on the challenging SUN RGB-D dataset, advancing state-of-the-art results&#10;by 5.7 mAP. We also provide rich ablation studies to analyze the contribution&#10;of each design choice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.06326" label="1902.06326">
-        <attvalues>
-          <attvalue for="0" value="PIXOR: Real-time 3D Object Detection from Point Clouds" />
-          <attvalue for="1" value="  We address the problem of real-time 3D object detection from point clouds in&#10;the context of autonomous driving. Computation speed is critical as detection&#10;is a necessary component for safety. Existing approaches are, however,&#10;expensive in computation due to high dimensionality of point clouds. We utilize&#10;the 3D data more efficiently by representing the scene from the Bird's Eye View&#10;(BEV), and propose PIXOR, a proposal-free, single-stage detector that outputs&#10;oriented 3D object estimates decoded from pixel-wise neural network&#10;predictions. The input representation, network architecture, and model&#10;optimization are especially designed to balance high accuracy and real-time&#10;efficiency. We validate PIXOR on two datasets: the KITTI BEV object detection&#10;benchmark, and a large-scale 3D vehicle detection benchmark. In both datasets&#10;we show that the proposed detector surpasses other state-of-the-art methods&#10;notably in terms of Average Precision (AP), while still runs at &gt;28 FPS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.06396" label="1711.06396">
-        <attvalues>
-          <attvalue for="0" value="VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection" />
-          <attvalue for="1" value="  Accurate detection of objects in 3D point clouds is a central problem in many&#10;applications, such as autonomous navigation, housekeeping robots, and&#10;augmented/virtual reality. To interface a highly sparse LiDAR point cloud with&#10;a region proposal network (RPN), most existing efforts have focused on&#10;hand-crafted feature representations, for example, a bird's eye view&#10;projection. In this work, we remove the need of manual feature engineering for&#10;3D point clouds and propose VoxelNet, a generic 3D detection network that&#10;unifies feature extraction and bounding box prediction into a single stage,&#10;end-to-end trainable deep network. Specifically, VoxelNet divides a point cloud&#10;into equally spaced 3D voxels and transforms a group of points within each&#10;voxel into a unified feature representation through the newly introduced voxel&#10;feature encoding (VFE) layer. In this way, the point cloud is encoded as a&#10;descriptive volumetric representation, which is then connected to a RPN to&#10;generate detections. Experiments on the KITTI car detection benchmark show that&#10;VoxelNet outperforms the state-of-the-art LiDAR based 3D detection methods by a&#10;large margin. Furthermore, our network learns an effective discriminative&#10;representation of objects with various geometries, leading to encouraging&#10;results in 3D detection of pedestrians and cyclists, based on only LiDAR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1507.02672" label="1507.02672">
-        <attvalues>
-          <attvalue for="0" value="Semi-Supervised Learning with Ladder Networks" />
-          <attvalue for="1" value="  We combine supervised learning with unsupervised learning in deep neural&#10;networks. The proposed model is trained to simultaneously minimize the sum of&#10;supervised and unsupervised cost functions by backpropagation, avoiding the&#10;need for layer-wise pre-training. Our work builds on the Ladder network&#10;proposed by Valpola (2015), which we extend by combining the model with&#10;supervision. We show that the resulting model reaches state-of-the-art&#10;performance in semi-supervised MNIST and CIFAR-10 classification, in addition&#10;to permutation-invariant MNIST classification with all labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.02242" label="1610.02242">
-        <attvalues>
-          <attvalue for="0" value="Temporal Ensembling for Semi-Supervised Learning" />
-          <attvalue for="1" value="  In this paper, we present a simple and efficient method for training deep&#10;neural networks in a semi-supervised setting where only a small portion of&#10;training data is labeled. We introduce self-ensembling, where we form a&#10;consensus prediction of the unknown labels using the outputs of the&#10;network-in-training on different epochs, and most importantly, under different&#10;regularization and input augmentation conditions. This ensemble prediction can&#10;be expected to be a better predictor for the unknown labels than the output of&#10;the network at the most recent training epoch, and can thus be used as a target&#10;for training. Using our method, we set new records for two standard&#10;semi-supervised learning benchmarks, reducing the (non-augmented)&#10;classification error rate from 18.44% to 7.05% in SVHN with 500 labels and from&#10;18.63% to 16.55% in CIFAR-10 with 4000 labels, and further to 5.12% and 12.16%&#10;by enabling the standard augmentations. We additionally obtain a clear&#10;improvement in CIFAR-100 classification accuracy by using random images from&#10;the Tiny Images dataset as unlabeled extra inputs during training. Finally, we&#10;demonstrate good tolerance to incorrect labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.03976" label="1704.03976">
-        <attvalues>
-          <attvalue for="0" value="Virtual Adversarial Training: A Regularization Method for Supervised and&#10;  Semi-Supervised Learning" />
-          <attvalue for="1" value="  We propose a new regularization method based on virtual adversarial loss: a&#10;new measure of local smoothness of the conditional label distribution given&#10;input. Virtual adversarial loss is defined as the robustness of the conditional&#10;label distribution around each input data point against local perturbation.&#10;Unlike adversarial training, our method defines the adversarial direction&#10;without label information and is hence applicable to semi-supervised learning.&#10;Because the directions in which we smooth the model are only &quot;virtually&quot;&#10;adversarial, we call our method virtual adversarial training (VAT). The&#10;computational cost of VAT is relatively low. For neural networks, the&#10;approximated gradient of virtual adversarial loss can be computed with no more&#10;than two pairs of forward- and back-propagations. In our experiments, we&#10;applied VAT to supervised and semi-supervised learning tasks on multiple&#10;benchmark datasets. With a simple enhancement of the algorithm based on the&#10;entropy minimization principle, our VAT achieves state-of-the-art performance&#10;for semi-supervised learning tasks on SVHN and CIFAR-10.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.08514" label="2004.08514">
-        <attvalues>
-          <attvalue for="0" value="DMT: Dynamic Mutual Training for Semi-Supervised Learning" />
-          <attvalue for="1" value="  Recent semi-supervised learning methods use pseudo supervision as core idea,&#10;especially self-training methods that generate pseudo labels. However, pseudo&#10;labels are unreliable. Self-training methods usually rely on single model&#10;prediction confidence to filter low-confidence pseudo labels, thus remaining&#10;high-confidence errors and wasting many low-confidence correct labels. In this&#10;paper, we point out it is difficult for a model to counter its own errors.&#10;Instead, leveraging inter-model disagreement between different models is a key&#10;to locate pseudo label errors. With this new viewpoint, we propose mutual&#10;training between two different models by a dynamically re-weighted loss&#10;function, called Dynamic Mutual Training (DMT). We quantify inter-model&#10;disagreement by comparing predictions from two different models to dynamically&#10;re-weight loss in training, where a larger disagreement indicates a possible&#10;error and corresponds to a lower loss value. Extensive experiments show that&#10;DMT achieves state-of-the-art performance in both image classification and&#10;semantic segmentation. Our codes are released at&#10;https://github.com/voldemortX/DST-CBC .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.08878" label="2004.08878">
-        <attvalues>
-          <attvalue for="0" value="Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Unsupervised domain adaptation (UDA) aims to adapt existing models of the&#10;source domain to a new target domain with only unlabeled data. Most existing&#10;methods suffer from noticeable negative transfer resulting from either the&#10;error-prone discriminator network or the unreasonable teacher model. Besides,&#10;the local regional consistency in UDA has been largely neglected, and only&#10;extracting the global-level pattern information is not powerful enough for&#10;feature alignment due to the abuse use of contexts. To this end, we propose an&#10;uncertainty-aware consistency regularization method for cross-domain semantic&#10;segmentation. Firstly, we introduce an uncertainty-guided consistency loss with&#10;a dynamic weighting scheme by exploiting the latent uncertainty information of&#10;the target samples. As such, more meaningful and reliable knowledge from the&#10;teacher model can be transferred to the student model. We further reveal the&#10;reason why the current consistency regularization is often unstable in&#10;minimizing the domain discrepancy. Besides, we design a ClassDrop mask&#10;generation algorithm to produce strong class-wise perturbations. Guided by this&#10;mask, we propose a ClassOut strategy to realize effective regional consistency&#10;in a fine-grained manner. Experiments demonstrate that our method outperforms&#10;the state-of-the-art methods on four domain adaptation benchmarks, i.e., GTAV&#10;$\rightarrow $ Cityscapes and SYNTHIA $\rightarrow $ Cityscapes, Virtual KITTI&#10;$\rightarrow$ KITTI and Cityscapes $\rightarrow$ KITTI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.11803" label="1912.11803">
-        <attvalues>
-          <attvalue for="0" value="SESS: Self-Ensembling Semi-Supervised 3D Object Detection" />
-          <attvalue for="1" value="  The performance of existing point cloud-based 3D object detection methods&#10;heavily relies on large-scale high-quality 3D annotations. However, such&#10;annotations are often tedious and expensive to collect. Semi-supervised&#10;learning is a good alternative to mitigate the data annotation issue, but has&#10;remained largely unexplored in 3D object detection. Inspired by the recent&#10;success of self-ensembling technique in semi-supervised image classification&#10;task, we propose SESS, a self-ensembling semi-supervised 3D object detection&#10;framework. Specifically, we design a thorough perturbation scheme to enhance&#10;generalization of the network on unlabeled and new unseen data. Furthermore, we&#10;propose three consistency losses to enforce the consistency between two sets of&#10;predicted 3D object proposals, to facilitate the learning of structure and&#10;semantic invariances of objects. Extensive experiments conducted on SUN RGB-D&#10;and ScanNet datasets demonstrate the effectiveness of SESS in both inductive&#10;and transductive semi-supervised 3D object detection. Our SESS achieves&#10;competitive performance compared to the state-of-the-art fully-supervised&#10;method by using only 50% labeled data. Our code is available at&#10;https://github.com/Na-Z/sess.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.04405" label="1702.04405">
-        <attvalues>
-          <attvalue for="0" value="ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes" />
-          <attvalue for="1" value="  A key requirement for leveraging supervised deep learning methods is the&#10;availability of large, labeled datasets. Unfortunately, in the context of RGB-D&#10;scene understanding, very little data is available -- current datasets cover a&#10;small range of scene views and have limited semantic annotations. To address&#10;this issue, we introduce ScanNet, an RGB-D video dataset containing 2.5M views&#10;in 1513 scenes annotated with 3D camera poses, surface reconstructions, and&#10;semantic segmentations. To collect this data, we designed an easy-to-use and&#10;scalable RGB-D capture system that includes automated surface reconstruction&#10;and crowdsourced semantic annotation. We show that using this data helps&#10;achieve state-of-the-art performance on several 3D scene understanding tasks,&#10;including 3D object classification, semantic voxel labeling, and CAD model&#10;retrieval. The dataset is freely available at http://www.scan-net.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04717" label="1904.04717">
-        <attvalues>
-          <attvalue for="0" value="Label Propagation for Deep Semi-supervised Learning" />
-          <attvalue for="1" value="  Semi-supervised learning is becoming increasingly important because it can&#10;combine data carefully labeled by humans with abundant unlabeled data to train&#10;deep neural networks. Classic methods on semi-supervised learning that have&#10;focused on transductive learning have not been fully exploited in the inductive&#10;framework followed by modern deep learning. The same holds for the manifold&#10;assumption---that similar examples should get the same prediction. In this&#10;work, we employ a transductive label propagation method that is based on the&#10;manifold assumption to make predictions on the entire dataset and use these&#10;predictions to generate pseudo-labels for the unlabeled data and train a deep&#10;neural network. At the core of the transductive method lies a nearest neighbor&#10;graph of the dataset that we create based on the embeddings of the same&#10;network.Therefore our learning process iterates between these two steps. We&#10;improve performance on several datasets especially in the few labels regime and&#10;show that our work is complementary to current state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.04628" label="2210.04628">
-        <attvalues>
-          <attvalue for="0" value="Novel View Synthesis with Diffusion Models" />
-          <attvalue for="1" value="  We present 3DiM, a diffusion model for 3D novel view synthesis, which is able&#10;to translate a single input view into consistent and sharp completions across&#10;many views. The core component of 3DiM is a pose-conditional image-to-image&#10;diffusion model, which takes a source view and its pose as inputs, and&#10;generates a novel view for a target pose as output. 3DiM can generate multiple&#10;views that are 3D consistent using a novel technique called stochastic&#10;conditioning. The output views are generated autoregressively, and during the&#10;generation of each novel view, one selects a random conditioning view from the&#10;set of available views at each denoising step. We demonstrate that stochastic&#10;conditioning significantly improves the 3D consistency of a naive sampler for&#10;an image-to-image diffusion model, which involves conditioning on a single&#10;fixed view. We compare 3DiM to prior work on the SRN ShapeNet dataset,&#10;demonstrating that 3DiM's generated completions from a single view achieve much&#10;higher fidelity, while being approximately 3D consistent. We also introduce a&#10;new evaluation methodology, 3D consistency scoring, to measure the 3D&#10;consistency of a generated object by training a neural field on the model's&#10;output views. 3DiM is geometry free, does not rely on hyper-networks or&#10;test-time optimization for novel view synthesis, and allows a single model to&#10;easily scale to a large number of scenes.&#10;" />
-          <attvalue for="2" value="&#10;Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}. Diffusion models have achieved impressive empirical results in a variety of image-to-image translation tasks not limited to text-to-image, super-resolution, inpainting, colorization, uncropping, and artifact removal \cite{song2020score,saharia2021palette,ramesh2022hierarchical, saharia2022photorealistic}.&#10;&#10;One particular image-to-image translation problem where diffusion models have not been investigated is novel view synthesis, where, given a set of images of a given 3D scene, the task is to infer how the scene looks from novel viewpoints.&#10;Before the recent emergence of Scene Representation Networks (SRN)~\cite{sitzmann2019scene} and Neural Radiance Fields (NeRF)~\cite{mildenhall2020nerf}, state-of-the-art approaches to novel view synthesis were typically built on generative models~\cite{sun2018multi} or more classical techniques on interpolation or disparity estimation \cite{park2017transformation,zhou2018stereo}. Today, these models have been outperformed by NeRF-class models~\cite{yu2021pixelnerf,niemeyer2021regnerf,jang2021codenerf}, where 3D consistency is guaranteed by construction, as images are generated by volume rendering of a single underlying 3D representation (a.k.a. ``geometry-aware'' models).&#10;&#10;Still, these approaches feature different limitations.&#10;Heavily regularized NeRFs for novel view synthesis with few images such as RegNeRF~\cite{niemeyer2021regnerf} produce undesired artifacts when given very few images, and fail to leverage knowledge from multiple scenes (recall NeRFs are trained on a single scene, i.e., one model per scene), and given one or very few views of a novel scene, a reasonable model must extrapolate to complete the occluded parts of the scene.&#10;PixelNeRF~\cite{yu2021pixelnerf} and VisionNeRF~\cite{lin2022vision} address this by training NeRF-like models conditioned on feature maps that encode the novel input view(s). However, these approaches are regressive rather than generative, and as a result, they cannot yield different plausible modes and are prone to blurriness. This type of failure has also been previously observed in regression-based models~\cite{saharia2021image}. Other works such as CodeNeRF~\cite{jang2021codenerf} and LoLNeRF~\cite{rebain2021lolnerf} instead employ test-time optimization to handle novel scenes, but still have issues with sample quality.&#10;&#10;In recent literature, geometry-free approaches (i.e., methods without explicit geometric inductive biases like those introduced by volume rendering) such as Light Field Networks (LFN)~\cite{sitzmann2021light} and Scene Representation Transformers (SRT)~\cite{sajjadi2021scene} have achieved results competitive with 3D-aware methods in the ``few-shot'' setting, where the number of conditioning views is limited (i.e., 1-10 images vs. dozens of images as in the usual NeRF setting). Similarly to our approach, EG3D \cite{chan2022efficient} provides approximate 3D consistency by leveraging generative models. EG3D employs a StyleGAN \cite{karras2019style} with volumetric rendering, followed by generative super-resolution (the latter being responsible for the approximation). In comparison to this complex setup, diffusion not only provides a significantly simpler architecture, but also a simpler hyper-parameter tuning experience compared to GANs, which are well-known to be notoriously difficult to tune \cite{mescheder2018training}. Notably, diffusion models have already seen some success on 3D point-cloud generation \cite{luo2021diffusion,waibel2022diffusion}.&#10;&#10;Motivated by these observations and the success of diffusion models in image-to-image tasks, we introduce 3D Diffusion Models~(3DiMs).&#10;3DiMs are image-to-image diffusion models trained on pairs of images of the same scene, where we assume the poses of the two images are known.&#10;Drawing inspiration from Scene Representation Transformers~\cite{sajjadi2021scene}, 3DiMs are trained to build a conditional generative model of one view given another view and their poses.&#10;Our key discovery is that we can turn this image-to-image model into a model that can produce an entire set of 3D-consistent frames through autoregressive generation, which we enable with our novel stochastic conditioning sampling algorithm.&#10;We cover stochastic conditioning in more detail in Section \ref{section:sampler} and provide an illustration in Figure \ref{fig:sampler}.&#10;Compared to prior work, 3DiMs are generative (vs. regressive) geometry free models, they allow training to scale to a large number of scenes, and offer a simple end-to-end approach.&#10;&#10;We now summarize our core contributions:&#10;\begin{enumerate}&#10; \item We introduce 3DiM, a geometry-free image-to-image diffusion model for novel view synthesis.&#10; \item We introduce the stochastic conditioning sampling algorithm, which encourages 3DiM to generate 3D-consistent outputs.&#10; \item We introduce X-UNet, a new UNet architecture~\cite{ronneberger2015u} variant for 3D novel view synthesis, demonstrating that changes in architecture are critical for high fidelity results.&#10; \item We introduce an evaluation scheme for geometry-free view synthesis models, 3D consistency scoring, that can numerically capture 3D consistency by training neural fields on model outputs.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Diffusion Modeling, Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1503.03585" label="1503.03585">
-        <attvalues>
-          <attvalue for="0" value="Deep Unsupervised Learning using Nonequilibrium Thermodynamics" />
-          <attvalue for="1" value="  A central problem in machine learning involves modeling complex data-sets&#10;using highly flexible families of probability distributions in which learning,&#10;sampling, inference, and evaluation are still analytically or computationally&#10;tractable. Here, we develop an approach that simultaneously achieves both&#10;flexibility and tractability. The essential idea, inspired by non-equilibrium&#10;statistical physics, is to systematically and slowly destroy structure in a&#10;data distribution through an iterative forward diffusion process. We then learn&#10;a reverse diffusion process that restores structure in data, yielding a highly&#10;flexible and tractable generative model of the data. This approach allows us to&#10;rapidly learn, sample from, and evaluate probabilities in deep generative&#10;models with thousands of layers or time steps, as well as to compute&#10;conditional and posterior probabilities under the learned model. We&#10;additionally release an open source reference implementation of the algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.00713" label="2009.00713">
-        <attvalues>
-          <attvalue for="0" value="WaveGrad: Estimating Gradients for Waveform Generation" />
-          <attvalue for="1" value="  This paper introduces WaveGrad, a conditional model for waveform generation&#10;which estimates gradients of the data density. The model is built on prior work&#10;on score matching and diffusion probabilistic models. It starts from a Gaussian&#10;white noise signal and iteratively refines the signal via a gradient-based&#10;sampler conditioned on the mel-spectrogram. WaveGrad offers a natural way to&#10;trade inference speed for sample quality by adjusting the number of refinement&#10;steps, and bridges the gap between non-autoregressive and autoregressive models&#10;in terms of audio quality. We find that it can generate high fidelity audio&#10;samples using as few as six iterations. Experiments reveal WaveGrad to generate&#10;high fidelity audio, outperforming adversarial non-autoregressive baselines and&#10;matching a strong likelihood-based autoregressive baseline using fewer&#10;sequential operations. Audio samples are available at&#10;https://wavegrad.github.io/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.05233" label="2105.05233">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Models Beat GANs on Image Synthesis" />
-          <attvalue for="1" value="  We show that diffusion models can achieve image sample quality superior to&#10;the current state-of-the-art generative models. We achieve this on&#10;unconditional image synthesis by finding a better architecture through a series&#10;of ablations. For conditional image synthesis, we further improve sample&#10;quality with classifier guidance: a simple, compute-efficient method for&#10;trading off diversity for fidelity using gradients from a classifier. We&#10;achieve an FID of 2.97 on ImageNet 128$\times$128, 4.59 on ImageNet&#10;256$\times$256, and 7.72 on ImageNet 512$\times$512, and we match BigGAN-deep&#10;even with as few as 25 forward passes per sample, all while maintaining better&#10;coverage of the distribution. Finally, we find that classifier guidance&#10;combines well with upsampling diffusion models, further improving FID to 3.94&#10;on ImageNet 256$\times$256 and 3.85 on ImageNet 512$\times$512. We release our&#10;code at https://github.com/openai/guided-diffusion&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.09258" label="2101.09258">
-        <attvalues>
-          <attvalue for="0" value="Maximum Likelihood Training of Score-Based Diffusion Models" />
-          <attvalue for="1" value="  Score-based diffusion models synthesize samples by reversing a stochastic&#10;process that diffuses data to noise, and are trained by minimizing a weighted&#10;combination of score matching losses. The log-likelihood of score-based&#10;diffusion models can be tractably computed through a connection to continuous&#10;normalizing flows, but log-likelihood is not directly optimized by the weighted&#10;combination of score matching losses. We show that for a specific weighting&#10;scheme, the objective upper bounds the negative log-likelihood, thus enabling&#10;approximate maximum likelihood training of score-based diffusion models. We&#10;empirically observe that maximum likelihood training consistently improves the&#10;likelihood of score-based diffusion models across multiple datasets, stochastic&#10;processes, and model architectures. Our best models achieve negative&#10;log-likelihoods of 2.83 and 3.76 bits/dim on CIFAR-10 and ImageNet 32x32&#10;without any data augmentation, on a par with state-of-the-art autoregressive&#10;models on these tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.00630" label="2107.00630">
-        <attvalues>
-          <attvalue for="0" value="Variational Diffusion Models" />
-          <attvalue for="1" value="  Diffusion-based generative models have demonstrated a capacity for&#10;perceptually impressive synthesis, but can they also be great likelihood-based&#10;models? We answer this in the affirmative, and introduce a family of&#10;diffusion-based generative models that obtain state-of-the-art likelihoods on&#10;standard image density estimation benchmarks. Unlike other diffusion-based&#10;models, our method allows for efficient optimization of the noise schedule&#10;jointly with the rest of the model. We show that the variational lower bound&#10;(VLB) simplifies to a remarkably short expression in terms of the&#10;signal-to-noise ratio of the diffused data, thereby improving our theoretical&#10;understanding of this model class. Using this insight, we prove an equivalence&#10;between several models proposed in the literature. In addition, we show that&#10;the continuous-time VLB is invariant to the noise schedule, except for the&#10;signal-to-noise ratio at its endpoints. This enables us to learn a noise&#10;schedule that minimizes the variance of the resulting VLB estimator, leading to&#10;faster optimization. Combining these advances with architectural improvements,&#10;we obtain state-of-the-art likelihoods on image density estimation benchmarks,&#10;outperforming autoregressive models that have dominated these benchmarks for&#10;many years, with often significantly faster optimization. In addition, we show&#10;how to use the model as part of a bits-back compression scheme, and demonstrate&#10;lossless compression rates close to the theoretical optimum. Code is available&#10;at https://github.com/google-research/vdm .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.13456" label="2011.13456">
-        <attvalues>
-          <attvalue for="0" value="Score-Based Generative Modeling through Stochastic Differential&#10;  Equations" />
-          <attvalue for="1" value="  Creating noise from data is easy; creating data from noise is generative&#10;modeling. We present a stochastic differential equation (SDE) that smoothly&#10;transforms a complex data distribution to a known prior distribution by slowly&#10;injecting noise, and a corresponding reverse-time SDE that transforms the prior&#10;distribution back into the data distribution by slowly removing the noise.&#10;Crucially, the reverse-time SDE depends only on the time-dependent gradient&#10;field (\aka, score) of the perturbed data distribution. By leveraging advances&#10;in score-based generative modeling, we can accurately estimate these scores&#10;with neural networks, and use numerical SDE solvers to generate samples. We&#10;show that this framework encapsulates previous approaches in score-based&#10;generative modeling and diffusion probabilistic modeling, allowing for new&#10;sampling procedures and new modeling capabilities. In particular, we introduce&#10;a predictor-corrector framework to correct errors in the evolution of the&#10;discretized reverse-time SDE. We also derive an equivalent neural ODE that&#10;samples from the same distribution as the SDE, but additionally enables exact&#10;likelihood computation, and improved sampling efficiency. In addition, we&#10;provide a new way to solve inverse problems with score-based models, as&#10;demonstrated with experiments on class-conditional generation, image&#10;inpainting, and colorization. Combined with multiple architectural&#10;improvements, we achieve record-breaking performance for unconditional image&#10;generation on CIFAR-10 with an Inception score of 9.89 and FID of 2.20, a&#10;competitive likelihood of 2.99 bits/dim, and demonstrate high fidelity&#10;generation of 1024 x 1024 images for the first time from a score-based&#10;generative model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.05826" label="2111.05826">
-        <attvalues>
-          <attvalue for="0" value="Palette: Image-to-Image Diffusion Models" />
-          <attvalue for="1" value="  This paper develops a unified framework for image-to-image translation based&#10;on conditional diffusion models and evaluates this framework on four&#10;challenging image-to-image translation tasks, namely colorization, inpainting,&#10;uncropping, and JPEG restoration. Our simple implementation of image-to-image&#10;diffusion models outperforms strong GAN and regression baselines on all tasks,&#10;without task-specific hyper-parameter tuning, architecture customization, or&#10;any auxiliary loss or sophisticated new techniques needed. We uncover the&#10;impact of an L2 vs. L1 loss in the denoising diffusion objective on sample&#10;diversity, and demonstrate the importance of self-attention in the neural&#10;architecture through empirical studies. Importantly, we advocate a unified&#10;evaluation protocol based on ImageNet, with human evaluation and sample quality&#10;scores (FID, Inception Score, Classification Accuracy of a pre-trained&#10;ResNet-50, and Perceptual Distance against original images). We expect this&#10;standardized evaluation protocol to play a role in advancing image-to-image&#10;translation research. Finally, we show that a generalist, multi-task diffusion&#10;model performs as well or better than task-specific specialist counterparts.&#10;Check out https://diffusion-palette.github.io for an overview of the results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.06125" label="2204.06125">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Text-Conditional Image Generation with CLIP Latents" />
-          <attvalue for="1" value="  Contrastive models like CLIP have been shown to learn robust representations&#10;of images that capture both semantics and style. To leverage these&#10;representations for image generation, we propose a two-stage model: a prior&#10;that generates a CLIP image embedding given a text caption, and a decoder that&#10;generates an image conditioned on the image embedding. We show that explicitly&#10;generating image representations improves image diversity with minimal loss in&#10;photorealism and caption similarity. Our decoders conditioned on image&#10;representations can also produce variations of an image that preserve both its&#10;semantics and style, while varying the non-essential details absent from the&#10;image representation. Moreover, the joint embedding space of CLIP enables&#10;language-guided image manipulations in a zero-shot fashion. We use diffusion&#10;models for the decoder and experiment with both autoregressive and diffusion&#10;models for the prior, finding that the latter are computationally more&#10;efficient and produce higher-quality samples.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.01618" label="1906.01618">
-        <attvalues>
-          <attvalue for="0" value="Scene Representation Networks: Continuous 3D-Structure-Aware Neural&#10;  Scene Representations" />
-          <attvalue for="1" value="  Unsupervised learning with generative models has the potential of discovering&#10;rich representations of 3D scenes. While geometric deep learning has explored&#10;3D-structure-aware representations of scene geometry, these models typically&#10;require explicit 3D supervision. Emerging neural scene representations can be&#10;trained only with posed 2D images, but existing methods ignore the&#10;three-dimensional structure of scenes. We propose Scene Representation Networks&#10;(SRNs), a continuous, 3D-structure-aware scene representation that encodes both&#10;geometry and appearance. SRNs represent scenes as continuous functions that map&#10;world coordinates to a feature representation of local scene properties. By&#10;formulating the image formation as a differentiable ray-marching algorithm,&#10;SRNs can be trained end-to-end from only 2D images and their camera poses,&#10;without access to depth or shape. This formulation naturally generalizes across&#10;scenes, learning powerful geometry and appearance priors in the process. We&#10;demonstrate the potential of SRNs by evaluating them for novel view synthesis,&#10;few-shot reconstruction, joint shape and appearance interpolation, and&#10;unsupervised discovery of a non-rigid face model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.02921" label="1703.02921">
-        <attvalues>
-          <attvalue for="0" value="Transformation-Grounded Image Generation Network for Novel 3D View&#10;  Synthesis" />
-          <attvalue for="1" value="  We present a transformation-grounded image generation network for novel 3D&#10;view synthesis from a single image. Instead of taking a 'blank slate' approach,&#10;we first explicitly infer the parts of the geometry visible both in the input&#10;and novel views and then re-cast the remaining synthesis problem as image&#10;completion. Specifically, we both predict a flow to move the pixels from the&#10;input to the novel view along with a novel visibility map that helps deal with&#10;occulsion/disocculsion. Next, conditioned on those intermediate results, we&#10;hallucinate (infer) parts of the object invisible in the input image. In&#10;addition to the new network structure, training with a combination of&#10;adversarial and perceptual loss results in a reduction in common artifacts of&#10;novel view synthesis such as distortions and holes, while successfully&#10;generating high frequency details and preserving visual aspects of the input&#10;image. We evaluate our approach on a wide range of synthetic and real examples.&#10;Both qualitative and quantitative results show our method achieves&#10;significantly better results compared to existing methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.09817" label="1805.09817">
-        <attvalues>
-          <attvalue for="0" value="Stereo Magnification: Learning View Synthesis using Multiplane Images" />
-          <attvalue for="1" value="  The view synthesis problem--generating novel views of a scene from known&#10;imagery--has garnered recent attention due in part to compelling applications&#10;in virtual and augmented reality. In this paper, we explore an intriguing&#10;scenario for view synthesis: extrapolating views from imagery captured by&#10;narrow-baseline stereo cameras, including VR cameras and now-widespread&#10;dual-lens camera phones. We call this problem stereo magnification, and propose&#10;a learning framework that leverages a new layered representation that we call&#10;multiplane images (MPIs). Our method also uses a massive new data source for&#10;learning view extrapolation: online videos on YouTube. Using data mined from&#10;such videos, we train a deep network that predicts an MPI from an input stereo&#10;image pair. This inferred MPI can then be used to synthesize a range of novel&#10;views of the scene, including views that extrapolate significantly beyond the&#10;input baseline. We show that our method compares favorably with several recent&#10;view synthesis methods, and demonstrate applications in magnifying&#10;narrow-baseline stereo images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.02190" label="2012.02190">
-        <attvalues>
-          <attvalue for="0" value="pixelNeRF: Neural Radiance Fields from One or Few Images" />
-          <attvalue for="1" value="  We propose pixelNeRF, a learning framework that predicts a continuous neural&#10;scene representation conditioned on one or few input images. The existing&#10;approach for constructing neural radiance fields involves optimizing the&#10;representation to every scene independently, requiring many calibrated views&#10;and significant compute time. We take a step towards resolving these&#10;shortcomings by introducing an architecture that conditions a NeRF on image&#10;inputs in a fully convolutional manner. This allows the network to be trained&#10;across multiple scenes to learn a scene prior, enabling it to perform novel&#10;view synthesis in a feed-forward manner from a sparse set of views (as few as&#10;one). Leveraging the volume rendering approach of NeRF, our model can be&#10;trained directly from images with no explicit 3D supervision. We conduct&#10;extensive experiments on ShapeNet benchmarks for single image novel view&#10;synthesis tasks with held-out objects as well as entire unseen categories. We&#10;further demonstrate the flexibility of pixelNeRF by demonstrating it on&#10;multi-object ShapeNet scenes and real scenes from the DTU dataset. In all&#10;cases, pixelNeRF outperforms current state-of-the-art baselines for novel view&#10;synthesis and single image 3D reconstruction. For the video and code, please&#10;visit the project website: https://alexyu.net/pixelnerf&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.00724" label="2112.00724">
-        <attvalues>
-          <attvalue for="0" value="RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from&#10;  Sparse Inputs" />
-          <attvalue for="1" value="  Neural Radiance Fields (NeRF) have emerged as a powerful representation for&#10;the task of novel view synthesis due to their simplicity and state-of-the-art&#10;performance. Though NeRF can produce photorealistic renderings of unseen&#10;viewpoints when many input views are available, its performance drops&#10;significantly when this number is reduced. We observe that the majority of&#10;artifacts in sparse input scenarios are caused by errors in the estimated scene&#10;geometry, and by divergent behavior at the start of training. We address this&#10;by regularizing the geometry and appearance of patches rendered from unobserved&#10;viewpoints, and annealing the ray sampling space during training. We&#10;additionally use a normalizing flow model to regularize the color of unobserved&#10;viewpoints. Our model outperforms not only other methods that optimize over a&#10;single scene, but in many cases also conditional models that are extensively&#10;pre-trained on large multi-view datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.01750" label="2109.01750">
-        <attvalues>
-          <attvalue for="0" value="CodeNeRF: Disentangled Neural Radiance Fields for Object Categories" />
-          <attvalue for="1" value="  CodeNeRF is an implicit 3D neural representation that learns the variation of&#10;object shapes and textures across a category and can be trained, from a set of&#10;posed images, to synthesize novel views of unseen objects. Unlike the original&#10;NeRF, which is scene specific, CodeNeRF learns to disentangle shape and texture&#10;by learning separate embeddings. At test time, given a single unposed image of&#10;an unseen object, CodeNeRF jointly estimates camera viewpoint, and shape and&#10;appearance codes via optimization. Unseen objects can be reconstructed from a&#10;single image, and then rendered from new viewpoints or their shape and texture&#10;edited by varying the latent codes. We conduct experiments on the SRN&#10;benchmark, which show that CodeNeRF generalises well to unseen objects and&#10;achieves on-par performance with methods that require known camera pose at test&#10;time. Our results on real-world images demonstrate that CodeNeRF can bridge the&#10;sim-to-real gap. Project page: \url{https://github.com/wayne1123/code-nerf}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.05736" label="2207.05736">
-        <attvalues>
-          <attvalue for="0" value="Vision Transformer for NeRF-Based View Synthesis from a Single Input&#10;  Image" />
-          <attvalue for="1" value="  Although neural radiance fields (NeRF) have shown impressive advances for&#10;novel view synthesis, most methods typically require multiple input images of&#10;the same scene with accurate camera poses. In this work, we seek to&#10;substantially reduce the inputs to a single unposed image. Existing approaches&#10;condition on local image features to reconstruct a 3D object, but often render&#10;blurry predictions at viewpoints that are far away from the source view. To&#10;address this issue, we propose to leverage both the global and local features&#10;to form an expressive 3D representation. The global features are learned from a&#10;vision transformer, while the local features are extracted from a 2D&#10;convolutional network. To synthesize a novel view, we train a multilayer&#10;perceptron (MLP) network conditioned on the learned 3D representation to&#10;perform volume rendering. This novel 3D representation allows the network to&#10;reconstruct unseen regions without enforcing constraints like symmetry or&#10;canonical coordinate systems. Our method can render novel views from only a&#10;single input image and generalize across multiple object categories using a&#10;single model. Quantitative and qualitative evaluations demonstrate that the&#10;proposed method achieves state-of-the-art performance and renders richer&#10;details than existing approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.07636" label="2104.07636">
-        <attvalues>
-          <attvalue for="0" value="Image Super-Resolution via Iterative Refinement" />
-          <attvalue for="1" value="  We present SR3, an approach to image Super-Resolution via Repeated&#10;Refinement. SR3 adapts denoising diffusion probabilistic models to conditional&#10;image generation and performs super-resolution through a stochastic denoising&#10;process. Inference starts with pure Gaussian noise and iteratively refines the&#10;noisy output using a U-Net model trained on denoising at various noise levels.&#10;SR3 exhibits strong performance on super-resolution tasks at different&#10;magnification factors, on faces and natural images. We conduct human evaluation&#10;on a standard 8X face super-resolution task on CelebA-HQ, comparing with SOTA&#10;GAN methods. SR3 achieves a fool rate close to 50%, suggesting photo-realistic&#10;outputs, while GANs do not exceed a fool rate of 34%. We further show the&#10;effectiveness of SR3 in cascaded image generation, where generative models are&#10;chained with super-resolution models, yielding a competitive FID score of 11.3&#10;on ImageNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09996" label="2111.09996">
-        <attvalues>
-          <attvalue for="0" value="LOLNeRF: Learn from One Look" />
-          <attvalue for="1" value="  We present a method for learning a generative 3D model based on neural&#10;radiance fields, trained solely from data with only single views of each&#10;object. While generating realistic images is no longer a difficult task,&#10;producing the corresponding 3D structure such that they can be rendered from&#10;different views is non-trivial. We show that, unlike existing methods, one does&#10;not need multi-view data to achieve this goal. Specifically, we show that by&#10;reconstructing many images aligned to an approximate canonical pose with a&#10;single network conditioned on a shared latent space, you can learn a space of&#10;radiance fields that models shape and appearance for a class of objects. We&#10;demonstrate this by training models to reconstruct object categories using&#10;datasets that contain only one view of each subject without depth or geometry&#10;information. Our experiments show that we achieve state-of-the-art results in&#10;novel view synthesis and high-quality results for monocular depth prediction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02634" label="2106.02634">
-        <attvalues>
-          <attvalue for="0" value="Light Field Networks: Neural Scene Representations with&#10;  Single-Evaluation Rendering" />
-          <attvalue for="1" value="  Inferring representations of 3D scenes from 2D observations is a fundamental&#10;problem of computer graphics, computer vision, and artificial intelligence.&#10;Emerging 3D-structured neural scene representations are a promising approach to&#10;3D scene understanding. In this work, we propose a novel neural scene&#10;representation, Light Field Networks or LFNs, which represent both geometry and&#10;appearance of the underlying 3D scene in a 360-degree, four-dimensional light&#10;field parameterized via a neural implicit representation. Rendering a ray from&#10;an LFN requires only a single network evaluation, as opposed to hundreds of&#10;evaluations per ray for ray-marching or volumetric based renderers in&#10;3D-structured neural scene representations. In the setting of simple scenes, we&#10;leverage meta-learning to learn a prior over LFNs that enables multi-view&#10;consistent light field reconstruction from as little as a single image&#10;observation. This results in dramatic reductions in time and memory complexity,&#10;and enables real-time rendering. The cost of storing a 360-degree light field&#10;via an LFN is two orders of magnitude lower than conventional methods such as&#10;the Lumigraph. Utilizing the analytical differentiability of neural implicit&#10;representations and a novel parameterization of light space, we further&#10;demonstrate the extraction of sparse depth maps from LFNs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.13152" label="2111.13152">
-        <attvalues>
-          <attvalue for="0" value="Scene Representation Transformer: Geometry-Free Novel View Synthesis&#10;  Through Set-Latent Scene Representations" />
-          <attvalue for="1" value="  A classical problem in computer vision is to infer a 3D scene representation&#10;from few images that can be used to render novel views at interactive rates.&#10;Previous work focuses on reconstructing pre-defined 3D representations, e.g.&#10;textured meshes, or implicit representations, e.g. radiance fields, and often&#10;requires input images with precise camera poses and long processing times for&#10;each novel scene.&#10;  In this work, we propose the Scene Representation Transformer (SRT), a method&#10;which processes posed or unposed RGB images of a new area, infers a &quot;set-latent&#10;scene representation&quot;, and synthesises novel views, all in a single&#10;feed-forward pass. To calculate the scene representation, we propose a&#10;generalization of the Vision Transformer to sets of images, enabling global&#10;information integration, and hence 3D reasoning. An efficient decoder&#10;transformer parameterizes the light field by attending into the scene&#10;representation to render novel views. Learning is supervised end-to-end by&#10;minimizing a novel-view reconstruction error.&#10;  We show that this method outperforms recent baselines in terms of PSNR and&#10;speed on synthetic datasets, including a new dataset created for the paper.&#10;Further, we demonstrate that SRT scales to support interactive visualization&#10;and semantic segmentation of real-world outdoor environments using Street View&#10;imagery.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.07945" label="2112.07945">
-        <attvalues>
-          <attvalue for="0" value="Efficient Geometry-aware 3D Generative Adversarial Networks" />
-          <attvalue for="1" value="  Unsupervised generation of high-quality multi-view-consistent images and 3D&#10;shapes using only collections of single-view 2D photographs has been a&#10;long-standing challenge. Existing 3D GANs are either compute-intensive or make&#10;approximations that are not 3D-consistent; the former limits quality and&#10;resolution of the generated images and the latter adversely affects multi-view&#10;consistency and shape quality. In this work, we improve the computational&#10;efficiency and image quality of 3D GANs without overly relying on these&#10;approximations. We introduce an expressive hybrid explicit-implicit network&#10;architecture that, together with other design choices, synthesizes not only&#10;high-resolution multi-view-consistent images in real time but also produces&#10;high-quality 3D geometry. By decoupling feature generation and neural&#10;rendering, our framework is able to leverage state-of-the-art 2D CNN&#10;generators, such as StyleGAN2, and inherit their efficiency and expressiveness.&#10;We demonstrate state-of-the-art 3D-aware synthesis with FFHQ and AFHQ Cats,&#10;among other experiments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.04948" label="1812.04948">
-        <attvalues>
-          <attvalue for="0" value="A Style-Based Generator Architecture for Generative Adversarial Networks" />
-          <attvalue for="1" value="  We propose an alternative generator architecture for generative adversarial&#10;networks, borrowing from style transfer literature. The new architecture leads&#10;to an automatically learned, unsupervised separation of high-level attributes&#10;(e.g., pose and identity when trained on human faces) and stochastic variation&#10;in the generated images (e.g., freckles, hair), and it enables intuitive,&#10;scale-specific control of the synthesis. The new generator improves the&#10;state-of-the-art in terms of traditional distribution quality metrics, leads to&#10;demonstrably better interpolation properties, and also better disentangles the&#10;latent factors of variation. To quantify interpolation quality and&#10;disentanglement, we propose two new, automated methods that are applicable to&#10;any generator architecture. Finally, we introduce a new, highly varied and&#10;high-quality dataset of human faces.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.04406" label="1801.04406">
-        <attvalues>
-          <attvalue for="0" value="Which Training Methods for GANs do actually Converge?" />
-          <attvalue for="1" value="  Recent work has shown local convergence of GAN training for absolutely&#10;continuous data and generator distributions. In this paper, we show that the&#10;requirement of absolute continuity is necessary: we describe a simple yet&#10;prototypical counterexample showing that in the more realistic case of&#10;distributions that are not absolutely continuous, unregularized GAN training is&#10;not always convergent. Furthermore, we discuss regularization strategies that&#10;were recently proposed to stabilize GAN training. Our analysis shows that GAN&#10;training with instance noise or zero-centered gradient penalties converges. On&#10;the other hand, we show that Wasserstein-GANs and WGAN-GP with a finite number&#10;of discriminator updates per generator update do not always converge to the&#10;equilibrium point. We discuss these results, leading us to a new explanation&#10;for the stability problems of GAN training. Based on our analysis, we extend&#10;our convergence results to more general GANs and prove local convergence for&#10;simplified gradient penalties even if the generator and data distribution lie&#10;on lower dimensional manifolds. We find these penalties to work well in&#10;practice and use them to learn high-resolution generative image models for a&#10;variety of datasets with little hyperparameter tuning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.01458" label="2103.01458">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Probabilistic Models for 3D Point Cloud Generation" />
-          <attvalue for="1" value="  We present a probabilistic model for point cloud generation, which is&#10;fundamental for various 3D vision tasks such as shape completion, upsampling,&#10;synthesis and data augmentation. Inspired by the diffusion process in&#10;non-equilibrium thermodynamics, we view points in point clouds as particles in&#10;a thermodynamic system in contact with a heat bath, which diffuse from the&#10;original distribution to a noise distribution. Point cloud generation thus&#10;amounts to learning the reverse diffusion process that transforms the noise&#10;distribution to the distribution of a desired shape. Specifically, we propose&#10;to model the reverse diffusion process for point clouds as a Markov chain&#10;conditioned on certain shape latent. We derive the variational bound in closed&#10;form for training and provide implementations of the model. Experimental&#10;results demonstrate that our model achieves competitive performance in point&#10;cloud generation and auto-encoding. The code is available at&#10;\url{https://github.com/luost26/diffusion-point-cloud}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.14125" label="2208.14125">
-        <attvalues>
-          <attvalue for="0" value="A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images" />
-          <attvalue for="1" value="  Diffusion models are a special type of generative model, capable of&#10;synthesising new data from a learnt distribution. We introduce DISPR, a&#10;diffusion-based model for solving the inverse problem of three-dimensional (3D)&#10;cell shape prediction from two-dimensional (2D) single cell microscopy images.&#10;Using the 2D microscopy image as a prior, DISPR is conditioned to predict&#10;realistic 3D shape reconstructions. To showcase the applicability of DISPR as a&#10;data augmentation tool in a feature-based single cell classification task, we&#10;extract morphological features from the red blood cells grouped into six highly&#10;imbalanced classes. Adding features from the DISPR predictions to the three&#10;minority classes improved the macro F1 score from $F1_\text{macro} = 55.2 \pm&#10;4.6\%$ to $F1_\text{macro} = 72.2 \pm 4.9\%$. We thus demonstrate that&#10;diffusion models can be successfully applied to inverse biomedical problems,&#10;and that they learn to reconstruct 3D shapes with realistic morphological&#10;features from 2D microscopy images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.04597" label="1505.04597">
-        <attvalues>
-          <attvalue for="0" value="U-Net: Convolutional Networks for Biomedical Image Segmentation" />
-          <attvalue for="1" value="  There is large consent that successful training of deep networks requires&#10;many thousand annotated training samples. In this paper, we present a network&#10;and training strategy that relies on the strong use of data augmentation to use&#10;the available annotated samples more efficiently. The architecture consists of&#10;a contracting path to capture context and a symmetric expanding path that&#10;enables precise localization. We show that such a network can be trained&#10;end-to-end from very few images and outperforms the prior best method (a&#10;sliding-window convolutional network) on the ISBI challenge for segmentation of&#10;neuronal structures in electron microscopic stacks. Using the same network&#10;trained on transmitted light microscopy images (phase contrast and DIC) we won&#10;the ISBI cell tracking challenge 2015 in these categories by a large margin.&#10;Moreover, the network is fast. Segmentation of a 512x512 image takes less than&#10;a second on a recent GPU. The full implementation (based on Caffe) and the&#10;trained networks are available at&#10;http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.06267" label="2205.06267">
-        <attvalues>
-          <attvalue for="0" value="Topologically-Aware Deformation Fields for Single-View 3D Reconstruction" />
-          <attvalue for="1" value="  We present a framework for learning 3D object shapes and dense cross-object&#10;3D correspondences from just an unaligned category-specific image collection.&#10;The 3D shapes are generated implicitly as deformations to a category-specific&#10;signed distance field and are learned in an unsupervised manner solely from&#10;unaligned image collections and their poses without any 3D supervision.&#10;Generally, image collections on the internet contain several intra-category&#10;geometric and topological variations, for example, different chairs can have&#10;different topologies, which makes the task of joint shape and correspondence&#10;estimation much more challenging. Because of this, prior works either focus on&#10;learning each 3D object shape individually without modeling cross-instance&#10;correspondences or perform joint shape and correspondence estimation on&#10;categories with minimal intra-category topological variations. We overcome&#10;these restrictions by learning a topologically-aware implicit deformation field&#10;that maps a 3D point in the object space to a higher dimensional point in the&#10;category-specific canonical space. At inference time, given a single image, we&#10;reconstruct the underlying 3D shape by first implicitly deforming each 3D point&#10;in the object space to the learned category-specific canonical space using the&#10;topologically-aware deformation field and then reconstructing the 3D shape as a&#10;canonical signed distance field. Both canonical shape and deformation field are&#10;learned end-to-end in an inverse-graphics fashion using a learned recurrent ray&#10;marcher (SRN) as a differentiable rendering module. Our approach, dubbed TARS,&#10;achieves state-of-the-art reconstruction fidelity on several datasets:&#10;ShapeNet, Pascal3D+, CUB, and Pix3D chairs. Result videos and code at&#10;https://shivamduggal4.github.io/tars-3D/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Reconstructing 3D from 2D observations has been an actively studied problem \cite{schoenberger2016mvs,Newcombe2011KinectFusionRD,shape_from_silhouette,park2019deepsdf,ONet,chen2018implicit_decoder}. &#10;Until recently, the majority of the high-fidelity reconstruction results were credited to the availability of some form of 3D data \cite{qixing_huang_retrieval,Newcombe2011KinectFusionRD}, and because of this, the majority of the success had been restricted to synthetic datasets \cite{Chang2015ShapeNetAI}. Reconstruction of real-world 3D shapes was either done by transferring the learned synthetic models to real-world objects \cite{Bechtold2021HPN,sdf_secrets,marrnet,shapehd} or required special 3D sensors \cite{sdflabel,sdf_secrets,Newcombe2011KinectFusionRD}. However, collecting dense 3D data is cumbersome, challenging, and even not possible for certain categories (eg: birds). With advancements in inverse graphics and differentiable rendering \cite{Loper2014OpenDRAA,kato2018renderer,liu2019softras, liu2020dist, drcTulsiani17,chen2019dibrender}, the requirement of 3D supervision has been significantly reduced. More recently, significant progress has been made in this direction &#10;and the reconstruction quality has reached its golden standard, particularly thanks to the combination of neural implicit representations and differentiable rendering \cite{mildenhall2020nerf, SRN, DVR, yariv2020multiview}. However, the majority of these works still require dense supervision in form of paired multi-view images. Such a setting may not be possible for the internet of images. Our work, TARS, focuses on further mitigating the dependency on dense supervision by operating only on single-view data.&#10;&#10;3D Reconstruction with Single-view Supervision:&#10;The task of single-view 3D reconstruction has been comparatively less-explored. Kar \etal \cite{Kar2015CategoryspecificOR}, Kanazawa \etal \cite{cmrKanazawa18} took a major step in this direction by learning 3D structures from a large collection of unpaired images. They learned to reconstruct the underlying shape by learning the deformations on top of a (learned) category-specific mean mesh. Further research along this direction focused on reducing supervision \cite{ucmrGoel20, umr2020}, enhancing geometry \cite{implciit_mesh_shubham, chen2019dibrender} and texture fidelity \cite{bhattad2021view}. However, these works are restricted to the reconstruction of object categories with topologically similar instances (eg: birds). Like CMR \cite{cmrKanazawa18}, we leverage the structural knowledge embedded in image collection in form of learned deformations to a learned category-specific mean shape but overcome their topological restrictions. Recently, Lin \etal \cite{SDF-SRN} directed the success of neural implicit modeling \cite{SRN} to the task of single-view 3D reconstruction and achieved state of the art results in terms of geometric fidelity. Our work further boosts the fidelity standards by jointly learning category-specific deformations and SDF fields.&#10;&#10;Neural Rendering:&#10;Recent works \cite{mildenhall2020nerf, wang2021neus, DVR, yariv2020multiview,Oechsle2021ICCV,dist, SRN} for rendering implicit surfaces have majorly leveraged some form of ray-tracing (ray marching, volumetric or surface rendering). The recent survey on neural rendering \cite{Tewari2020NeuralSTAR} classifies the rendering as: (a) image-based rendering approaches, which generate 2D content without explicitly modeling 3D (by transforming/ warping the input images) (b) explicit 3D based approaches. In our work, we utilize SRN \cite{SRN} as our neural renderer. SRN \cite{SRN} performs LSTM-based ray marching to implicitly generate a 2.5D depth map corresponding to the input image. Therefore, SRN lies at the intersectionof image-based and explicit rendering approaches. By using SRN \cite{SRN} as image-based renderer, we learn shapes in two ways: image to depth map translation learned using object-space features, and image to SDF learned using canonical-space features. Consistency between the two shape representations is the key contributor to our performance.&#10;&#10;3D Reconstruction with Dense Correspondences:&#10;Learning category-specific deformable shapes have been found to be prominently useful for 3D reconstruction \cite{3DMM, SMPL:2015, Kar2015CategoryspecificOR, cmrKanazawa18,Engelmann2017SAMPSA}. These approaches generally learn instance shapes as deformations to the initial shape bases. Prior works along this line (reconstruction via deformation) have reconstructed 3D shapes either in volumetric grid representation \cite{Engelmann2017SAMPSA,Wang2019DirectShapePA} or mesh representation \cite{SMPL, Kar2015CategoryspecificOR, cmrKanazawa18}. We learn both the deformation field and the 3D shape (signed distance field) implicitly. Unlike deformations to mesh, learning deformations to an implicit field is much more challenging, because of the loss of explicit structure (mesh connectivity). Recently, \cite{deng2021deformed, zheng2020dit} learned category specific deformation and signed distance fields implicitly. However, unlike our approach (TARS), they require dense 3D supervision during training.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Unsupervised Learning, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2201.09373" label="2201.09373">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a&#10;  Single-View Image" />
-          <attvalue for="1" value="  Much progress has been made in the supervised learning of 3D reconstruction&#10;of rigid objects from multi-view images or a video. However, it is more&#10;challenging to reconstruct severely deformed objects from a single-view RGB&#10;image in an unsupervised manner. Although training-based methods, such as&#10;specific category-level training, have been shown to successfully reconstruct&#10;rigid objects and slightly deformed objects like birds from a single-view&#10;image, they cannot effectively handle severely deformed objects and neither can&#10;be applied to some downstream tasks in the real world due to the inconsistent&#10;semantic meaning of vertices, which are crucial in defining the adopted 3D&#10;templates of objects to be reconstructed. In this work, we introduce a&#10;template-based method to infer 3D shapes from a single-view image and apply the&#10;reconstructed mesh to a downstream task, i.e., absolute length measurement.&#10;Without using 3D ground truth, our method faithfully reconstructs 3D meshes and&#10;achieves state-of-the-art accuracy in a length measurement task on a severely&#10;deformed fish dataset.&#10;" />
-          <attvalue for="2" value="&#10;Modeling the 3D geometry of objects is an open research problem in computer vision. 3D shape reconstruction enables machines to perceive the 3D world. Given 2D RGB inputs, predicting the 3D shape of objects looks natural to humans but quite challenging to machines. The key problem is the ill-posed nature of the task, i.e., with limited 2D observed views, the object's shape can be arbitrary in any unseen view.&#10;&#10;To deal with this ill-posed problem, a promising approach is to use supervised learning approaches requiring 3D ground truth, such as Mesh R-CNN~\cite{Gkioxari_2019_ICCV} and Pixel2Mesh~\cite{wang2018pixel2mesh}. Although supervised training has been quite successful in 2D tasks such as image classification~\cite{ mei2021videobased}, object detection~\cite{ren2016faster} and segmentation~\cite{ronneberger2015u}, it is much more time-consuming and expensive to collect 3D ground truth data for 3D supervised training. For example, SMPL~\cite{loper2015smpl} uses thousands of registered 3D scans of humans for training, SMAL~\cite{zuffi20173d} utilizes scans of animal toys and a manually rigged mesh model. Collecting such 3D ground truth data requires much more time and effort than collecting 2D data. Therefore, existing supervised training methods fail to reconstruct objects' shapes beyond the training categories, particularly for objects whose 3D ground truths are impossible to collect such as wild animals.&#10;&#10;Without supervised training, a recent trend to mitigate this ill-posed problem is leveraging category-specific image collections for unsupervised training. By using images from the same category for training, existing methods, such as CMR ~\cite{kanazawa2018learning} and ACSM~\cite{kulkarni2020articulationaware}, assume the model can learn the common shape structure in the category. However, such results are limited to rigid objects or slightly deformed objects such as birds. Besides, these methods can not be applied to some downstream tasks, such as size measurement, because they can not guarantee the consistent semantic meaning of each vertex in the 3D mesh template. &#10;&#10;We proposed an unsupervised learning method, which lies in an intermediate regime between the above two extremes, keeping the model's ability to reconstruct severely deformed objects and at the same time maintaining the semantic meaning of each mesh vertex. Our proposed method can thus ensure the reconstructed shape to be applied to some downstream tasks, that require consistent semantic meaning of mesh vertices, such as length measurement.&#10;&#10;Why single-view? 3D reconstruction models, such as Nerf~\cite{mildenhall2020nerf}, uses multi-view images as input while CMR~\cite{kanazawa2018learning}, ACSM~\cite{kulkarni2020articulationaware} and LASR~\cite{yang2021lasr} use a single view image. Though achieving better performance, multi-view data are commonly much harder to collect than single-view images or monocular videos. Previous methods~\cite{huang2018fish, mei2021absolute} also mention the challenges encountered in the synchronization of multi-view cameras. Therefore, our proposed method is based on a single-view image.&#10;&#10;Why a template? In order to maintain the consistent semantic meaning of each mesh vertex defined for 3D objects, prior works introduce some templates, such as the 3D human template in SMPL~\cite{loper2015smpl}. Because only when each mesh vertex has consistent semantic meaning, the reconstructed shape can be further applied in some downstream tasks. Our method also adopts a 3D template for unsupervised estimation of the 3D object poses under severely deformed scenarios.&#10;&#10;Proposed approach: Instead of predicting deformed poses and 3D shapes from category-specific image collections or multi-view inputs, we propose a shape model with the ability to severely deform for matching with a single view image of an object. We utilize a differentiable render~\cite{liu2019softras} to make whole learning unsupervised, given an initialized 3D template that can be deformed based on some learnable parameters. By comparing the forward rendered silhouette (segmentation mask) under these learnable parameters with the ground truth of object silhouette, we use Adam optimizer to optimize these deformation parameters so that we can reconstruct the object shape in an unsupervised manner without any 3D ground truth. An overview of the pipeline is shown in Fig.\ref{pipeline}&#10;&#10;Contributions We propose an unsupervised learning method for severely deformed shape reconstruction based on a 3D deformable template. With the consistent semantic meaning of each mesh vertex, we are able to apply the reconstructed mesh to a downstream task, length measurement. By only using a target's 2D segmentation mask, our model can reconstruct the severely deformed shape and further estimate the 3D pose of the object, and perform the length measurement task from a monocular video, which further allows averaging estimated lengths for all frames corresponding to one object.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Unsupervised Learning, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1906.02739" label="1906.02739">
-        <attvalues>
-          <attvalue for="0" value="Mesh R-CNN" />
-          <attvalue for="1" value="  Rapid advances in 2D perception have led to systems that accurately detect&#10;objects in real-world images. However, these systems make predictions in 2D,&#10;ignoring the 3D structure of the world. Concurrently, advances in 3D shape&#10;prediction have mostly focused on synthetic benchmarks and isolated objects. We&#10;unify advances in these two areas. We propose a system that detects objects in&#10;real-world images and produces a triangle mesh giving the full 3D shape of each&#10;detected object. Our system, called Mesh R-CNN, augments Mask R-CNN with a mesh&#10;prediction branch that outputs meshes with varying topological structure by&#10;first predicting coarse voxel representations which are converted to meshes and&#10;refined with a graph convolution network operating over the mesh's vertices and&#10;edges. We validate our mesh prediction branch on ShapeNet, where we outperform&#10;prior work on single-image shape prediction. We then deploy our full Mesh R-CNN&#10;system on Pix3D, where we jointly detect objects and predict their 3D shapes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07700" label="1611.07700">
-        <attvalues>
-          <attvalue for="0" value="3D Menagerie: Modeling the 3D shape and pose of animals" />
-          <attvalue for="1" value="  There has been significant work on learning realistic, articulated, 3D models&#10;of the human body. In contrast, there are few such models of animals, despite&#10;many applications. The main challenge is that animals are much less cooperative&#10;than humans. The best human body models are learned from thousands of 3D scans&#10;of people in specific poses, which is infeasible with live animals.&#10;Consequently, we learn our model from a small set of 3D scans of toy figurines&#10;in arbitrary poses. We employ a novel part-based shape model to compute an&#10;initial registration to the scans. We then normalize their pose, learn a&#10;statistical shape model, and refine the registrations and the model together.&#10;In this way, we accurately align animal scans from different quadruped families&#10;with very different shapes and poses. With the registration to a common&#10;template we learn a shape space representing animals including lions, cats,&#10;dogs, horses, cows and hippos. Animal shapes can be sampled from the model,&#10;posed, animated, and fit to data. We demonstrate generalization by fitting it&#10;to images of real animals including species not seen in training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07549" label="1803.07549">
-        <attvalues>
-          <attvalue for="0" value="Learning Category-Specific Mesh Reconstruction from Image Collections" />
-          <attvalue for="1" value="  We present a learning framework for recovering the 3D shape, camera, and&#10;texture of an object from a single image. The shape is represented as a&#10;deformable 3D mesh model of an object category where a shape is parameterized&#10;by a learned mean shape and per-instance predicted deformation. Our approach&#10;allows leveraging an annotated image collection for training, where the&#10;deformable model and the 3D prediction mechanism are learned without relying on&#10;ground-truth 3D or multi-view supervision. Our representation enables us to go&#10;beyond existing 3D prediction approaches by incorporating texture inference as&#10;prediction of an image in a canonical appearance space. Additionally, we show&#10;that semantic keypoints can be easily associated with the predicted shapes. We&#10;present qualitative and quantitative results of our approach on CUB and&#10;PASCAL3D datasets and show that we can learn to predict diverse shapes and&#10;textures across objects using only annotated image collections. The project&#10;website can be found at https://akanazawa.github.io/cmr/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00614" label="2004.00614">
-        <attvalues>
-          <attvalue for="0" value="Articulation-aware Canonical Surface Mapping" />
-          <attvalue for="1" value="  We tackle the tasks of: 1) predicting a Canonical Surface Mapping (CSM) that&#10;indicates the mapping from 2D pixels to corresponding points on a canonical&#10;template shape, and 2) inferring the articulation and pose of the template&#10;corresponding to the input image. While previous approaches rely on keypoint&#10;supervision for learning, we present an approach that can learn without such&#10;annotations. Our key insight is that these tasks are geometrically related, and&#10;we can obtain supervisory signal via enforcing consistency among the&#10;predictions. We present results across a diverse set of animal object&#10;categories, showing that our method can learn articulation and CSM prediction&#10;from image collections using only foreground mask labels for training. We&#10;empirically show that allowing articulation helps learn more accurate CSM&#10;prediction, and that enforcing the consistency with predicted CSM is similarly&#10;critical for learning meaningful articulation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.02976" label="2105.02976">
-        <attvalues>
-          <attvalue for="0" value="LASR: Learning Articulated Shape Reconstruction from a Monocular Video" />
-          <attvalue for="1" value="  Remarkable progress has been made in 3D reconstruction of rigid structures&#10;from a video or a collection of images. However, it is still challenging to&#10;reconstruct nonrigid structures from RGB inputs, due to its under-constrained&#10;nature. While template-based approaches, such as parametric shape models, have&#10;achieved great success in modeling the &quot;closed world&quot; of known object&#10;categories, they cannot well handle the &quot;open-world&quot; of novel object categories&#10;or outlier shapes. In this work, we introduce a template-free approach to learn&#10;3D shapes from a single video. It adopts an analysis-by-synthesis strategy that&#10;forward-renders object silhouette, optical flow, and pixel values to compare&#10;with video observations, which generates gradients to adjust the camera, shape&#10;and motion parameters. Without using a category-specific shape template, our&#10;method faithfully reconstructs nonrigid 3D structures from videos of human,&#10;animals, and objects of unknown classes. Code will be available at&#10;lasr-google.github.io .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Below and in Tab.~\ref{tab:taxonomy}, we discuss related work of nonrigid shape recovery according to priors being used (shape template, category prior, or generic shape and motion priors).&#10;&#10;Model-based reconstruction:&#10;Model-based reconstruction leverages a parametric shape model to solve the under-constrained 3D shape and pose estimation problem. A large body of work in 3D human and animal reconstruction uses such parametric shape models~\cite{SMPL:2015,SMPL-X:2019,xiang2019monocular,Zuffi:CVPR:2018,Zuffi:CVPR:2017}, which are learned from 3D scans of human or animal toys~\cite{SMPL:2015,Zuffi:CVPR:2017}, and allow one to recover the 3D shape given very few annotations at test time (2D keypoints and silhouettes). Recently, model-based regression methods are developed to predict model-specific shape and pose parameters from a single image or video~\cite{badger2020,biggs2020wldo,kocabas2019vibe,zuffi2019three}, usually trained with ground-truth 3D data generated by such parametric shape models. Although model-based methods achieve great success in reconstructing ``closed-world'' objects of known category and rich 3D data, it is challenging to apply to unknown object categories, or categories with limited 3D data. &#10;&#10;Category mesh reconstruction:&#10;A recent trend is to reduce supervision from 3D or multi-view capturing to 2D annotations, such as keypoints and object silhouettes~\cite{ucmrGoel20,cmrKanazawa18,li2020self}. Such methods often take advantage of category priors, including a collection of images from the same category, and category-specific shape templates~\cite{kulkarni2020articulation,tulsiani2020imr}. Recent progress makes single-view reconstruction of birds and other common categories possible without 3D annotation. However, the single view reconstruction is usually coarse and lacks instance-specific details. Recent work adapts category-specific models to a test-time video~\cite{vmr2020}, but still does not handle objects of unknown classes.&#10;&#10;Template-free reconstruction:&#10;Among the template-free methods, PIFu~\cite{saito2019pifu,saito2020pifuhd} learns to predict an implicit shape representation for clothed human reconstruction, but requires ground-truth 3D shapes to train. A3DC~\cite{Reinert:2016:10.20380/GI2016.17} reconstructs articulated animals from videos, but requires involved user stroke interactions. Without requiring 3D data or user interactions, NRSfM factorizes a set of 2D keypoints or point trajectories into the 3D object shape and pose in each view assuming ``low-rank'' shape or deformation~\cite{agudo2018image, dai2014simple, gotardo2011non}. Recently, deep networks have been applied to learn such factorization of specific categories from 2D annotations~\cite{kong2019deep, novotny2019c3dpo, wang2020deep}. Close to our approach, Neural Dense NRSfM (N-NRSfM)~\cite{sidhu2020neural} learns a video-specific shape and deformation model from dense 2D point tracks. However, such methods are limited by the accuracy of 2D trajectory inputs, which is challenging to estimate in real-world sequences when large motion occurs~\cite{sand2008particle,sidhu2020neural,sundaram2010dense}.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1904.01786" label="1904.01786">
-        <attvalues>
-          <attvalue for="0" value="Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning" />
-          <attvalue for="1" value="  Rendering bridges the gap between 2D vision and 3D scenes by simulating the&#10;physical process of image formation. By inverting such renderer, one can think&#10;of a learning approach to infer 3D information from 2D images. However,&#10;standard graphics renderers involve a fundamental discretization step called&#10;rasterization, which prevents the rendering process to be differentiable, hence&#10;able to be learned. Unlike the state-of-the-art differentiable renderers, which&#10;only approximate the rendering gradient in the back propagation, we propose a&#10;truly differentiable rendering framework that is able to (1) directly render&#10;colorized mesh using differentiable functions and (2) back-propagate efficient&#10;supervision signals to mesh vertices and their attributes from various forms of&#10;image representations, including silhouette, shading and color images. The key&#10;to our framework is a novel formulation that views rendering as an aggregation&#10;function that fuses the probabilistic contributions of all mesh triangles with&#10;respect to the rendered pixels. Such formulation enables our framework to flow&#10;gradients to the occluded and far-range vertices, which cannot be achieved by&#10;the previous state-of-the-arts. We show that by using the proposed renderer,&#10;one can achieve significant improvement in 3D unsupervised single-view&#10;reconstruction both qualitatively and quantitatively. Experiments also&#10;demonstrate that our approach is able to handle the challenging tasks in&#10;image-based shape fitting, which remain nontrivial to existing differentiable&#10;renderers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.04479" label="2002.04479">
-        <attvalues>
-          <attvalue for="0" value="Depth Extraction from Video Using Non-parametric Sampling" />
-          <attvalue for="1" value="  We describe a technique that automatically generates plausible depth maps&#10;from videos using non-parametric depth sampling. We demonstrate our technique&#10;in cases where past methods fail (non-translating cameras and dynamic scenes).&#10;Our technique is applicable to single images as well as videos. For videos, we&#10;use local motion cues to improve the inferred depth maps, while optical flow is&#10;used to ensure temporal depth consistency. For training and evaluation, we use&#10;a Kinect-based system to collect a large dataset containing stereoscopic videos&#10;with known depths. We show that our depth estimation technique outperforms the&#10;state-of-the-art on benchmark databases. Our technique can be used to&#10;automatically convert a monoscopic video into stereo for 3D visualization, and&#10;we demonstrate this through a variety of visually pleasing results for indoor&#10;and outdoor scenes, including results from the feature film Charade.&#10;" />
-          <attvalue for="2" value="&#10;&#10;While many reconstruction techniques for extracting depth from video sequences exist, they typically assume moving cameras and static scenes. They do not work for dynamic scenes or for stationary, purely rotating, or strictly variable focal length sequences. There are some exceptions, e.g., \cite{Zhang:2011pami}, which can handle some moving objects, but they still require camera motion to induce parallax and allow depth estimation.&#10;&#10;In this paper, we present a novel solution to generate depth maps from ordinary 2D videos; our solution also applies to single images. This technique is applicable to arbitrary videos, and works in cases where conventional depth recovery methods fail (static/rotating camera; change in focal length; dynamic scenes). Our primary contribution is the use of a non-parametric ``depth transfer'' approach for inferring temporally consistent depth maps without imposing requirements on the video (Sec~\ref{sec:depthest} and~\ref{sec:depthest:video}), including a method for improving the depth estimates of moving objects (Sec~\ref{sec:motion_est}). In addition, we introduce a new, ground truth stereo RGBD (RGB+depth) video dataset (Sec~\ref{sec:dataset}). We also describe how we synthesize stereo videos from ordinary 2D videos using the results of our technique (Sec~\ref{sec:stereoest}).&#10;&#10;Several reconstruction methods for single images of real, unknown scenes have also been proposed. One of the first methods, introduced by Hoiem~\ea~\cite{Hoiem:2005:APP:1186822.1073232}, created convincing reconstructions of outdoor images by assuming an image could be broken into a few planar surfaces; similarly, Delage~\ea~developed a Bayesian framework for reconstructing indoor scenes~\cite{Delage:06}. Saxena \ea~devised a supervised learning strategy for predicting depth from a single image~\cite{Saxena05learningdepth}, which was further improved to create realistic reconstructions for general scenes~\cite{Saxena:09}, and efficient learning strategies have since been proposed~\cite{Batra:CVPR:12}. Better depth estimates have been achieved by incorporating semantic labels~\cite{Liu:cvpr10}, or more sophisticated models~\cite{Li:2010}. Repetitive structures can also be used for stereo reconstruction from a single image~\cite{Wu:2011wf}. Single-image shape from shading is also possible for known (a priori) object classes~\cite{Han:2003,Hassner:06}. We not only focus on depth from a single image, but also present a framework for using temporal information for enhanced and time-coherent depth when multiple frames are available.&#10;&#10;One application of these methods is 2D-to-3D conversion; however, many existing techniques require user interaction to refine depth and stereo estimates \cite{Guttmann:ICCV:09,Ward:11,Liao:TVCG:12}. An exception is the contemporaneous work of Konrad et al., which uses non-parametric depth sampling to automatically convert monocular images into stereoscopic images~\cite{Konrad:3DCINE:12}. Our technique extends their inference procedure, and works for videos as well.&#10;&#10;Liu~\ea~\cite{Liu:09} showed that arbitrary scenes can be semantically labelled through non-parametric learning. Given an unlabeled input image and a database with known per-pixel labels (e.g., sky, car, tree, window), their method works by transferring the labels from the database to the input image based on SIFT features. We build on this work by transferring depth instead of semantic labels. Furthermore, we show that this ``transfer'' approach can be applied in a continuous optimization framework (Sec~\ref{sec:depthest:transfer}), whereas their method used a discrete optimization approach (MRFs). &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Depth Map Generation, Physics, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1110.5102" label="1110.5102">
-        <attvalues>
-          <attvalue for="0" value="Towards Holistic Scene Understanding: Feedback Enabled Cascaded&#10;  Classification Models" />
-          <attvalue for="1" value="  Scene understanding includes many related sub-tasks, such as scene&#10;categorization, depth estimation, object detection, etc. Each of these&#10;sub-tasks is often notoriously hard, and state-of-the-art classifiers already&#10;exist for many of them. These classifiers operate on the same raw image and&#10;provide correlated outputs. It is desirable to have an algorithm that can&#10;capture such correlation without requiring any changes to the inner workings of&#10;any classifier.&#10;  We propose Feedback Enabled Cascaded Classification Models (FE-CCM), that&#10;jointly optimizes all the sub-tasks, while requiring only a `black-box'&#10;interface to the original classifier for each sub-task. We use a two-layer&#10;cascade of classifiers, which are repeated instantiations of the original ones,&#10;with the output of the first layer fed into the second layer as input. Our&#10;training method involves a feedback step that allows later classifiers to&#10;provide earlier classifiers information about which error modes to focus on. We&#10;show that our method significantly improves performance in all the sub-tasks in&#10;the domain of scene understanding, where we consider depth estimation, scene&#10;categorization, event categorization, object detection, geometric labeling and&#10;saliency detection. Our method also improves performance in two robotic&#10;applications: an object-grasping robot and an object-finding robot.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.05301" label="2112.05301">
-        <attvalues>
-          <attvalue for="0" value="Self-Ensemling for 3D Point Cloud Domain Adaption" />
-          <attvalue for="1" value="  Recently 3D point cloud learning has been a hot topic in computer vision and&#10;autonomous driving. Due to the fact that it is difficult to manually annotate a&#10;qualitative large-scale 3D point cloud dataset, unsupervised domain adaptation&#10;(UDA) is popular in 3D point cloud learning which aims to transfer the learned&#10;knowledge from the labeled source domain to the unlabeled target domain.&#10;However, the generalization and reconstruction errors caused by domain shift&#10;with simply-learned model are inevitable which substantially hinder the model's&#10;capability from learning good representations. To address these issues, we&#10;propose an end-to-end self-ensembling network (SEN) for 3D point cloud domain&#10;adaption tasks. Generally, our SEN resorts to the advantages of Mean Teacher&#10;and semi-supervised learning, and introduces a soft classification loss and a&#10;consistency loss, aiming to achieve consistent generalization and accurate&#10;reconstruction. In SEN, a student network is kept in a collaborative manner&#10;with supervised learning and self-supervised learning, and a teacher network&#10;conducts temporal consistency to learn useful representations and ensure the&#10;quality of point clouds reconstruction. Extensive experiments on several 3D&#10;point cloud UDA benchmarks show that our SEN outperforms the state-of-the-art&#10;methods on both classification and segmentation tasks. Moreover, further&#10;analysis demonstrates that our SEN also achieves better reconstruction results.&#10;" />
-          <attvalue for="2" value="&#10;Recently, the rapid development of the 3D point cloud has shown crucial importance to many real-world applications in different areas, such as robotics~\cite{pomerleau2015review}, autonomous driving~\cite{chen2017multi}, augmented and virtual reality~\cite{reitinger2007augmented}. The point cloud, as a simple set of points data in a 3D space, is a very convenient format for understanding the 3D world. Compared with 2D data, 3D point clouds can provide detailed geometry position information and easily capture the 3D structure information for the scene, but it is laborious to be processed. The 3D point cloud learning has attracted increasing attention on computer vision areas to process the various vision tasks, such as classification~\cite{qi2017pointnet}, segmentation~\cite{ye20183d}, registration~\cite{lawin2018density} and detection~\cite{vora2020pointpainting}. &#10;&#10;{Related work.} Early studies are mostly based on the hand-crafted features to deal with the 3D point cloud. Driven by the breakthroughs brought by deep learning, many studies have actively explored this technique to address these 3D problems on computer vision and robotics tasks. For example, the traditional deep learning is used with the multi-view~\cite{su2015multi} or 3D voxels~\cite{maturana2015voxnet} methods on point cloud tasks. They usually transform 3D points to collections of images or regular 3D voxel before deep model training. However, those pre-processing will cause the loss of geometric information during the data conversion or suffer from high computation or memory consumption.&#10;PointNet firstly utilizes deep learning on raw point clouds, which is the bedrock for overcoming the challenge of the irregular nature of point cloud~\cite{qi2017pointnet}. But it is only designed to fuse all features extracted from each point into a global feature, which does not capture the local structural information around a small neighborhood for each point. Subsequently, Qi et al.~\cite{qi2017pointnet++} proposed a hierarchical network PointNet++ to extract local features from the different abstraction levels. Many follow up works such as PointCNN~\cite{li2018pointcnn}, PointConv~\cite{wu2019pointconv} and DGCNN~\cite{wang2019dynamic} also focus on the aggregation of local regions to further improve the feature representation. Despite their impressive success on the point cloud tasks, all of these methods rely on a large amount of labeled data that is time-consuming and expensive to annotate. These make it difficult to keep the model's generalization ability while facing the 3D point clouds in the real world other than the existing dataset. &#10;&#10;The unlabeled data is relatively easy to obtain abundantly in real applications, thus some recent studies attempt unsupervised learning for 3D point clouds~\cite{hassani2019unsupervised,chen2020unsupervised}. To get discriminative information in unsupervised manner, recent works focus on constructing a new structural representation in the form of 3D structure points~\cite{yang2018foldingnet, deng2018ppf}. However, the reconstruction error will eventually affect the quality of feature representation.To address this issue, some other works explore unsupervised domain adaptation (UDA) methods on 3D point clouds~\cite{qin2019pointdan, zhao2019multi}, which takes advantage of the labeled source domain dataset to transfer the learned knowledge to the unlabeled target domain dataset. Particularly, there are few research methods based on unsupervised domain adaptation for 3D point cloud classification and segmentation. Currently, PointDAN~\cite{qin2019pointdan} introduces a point-based unsupervised domain adaptation network by aligning the local and global distribution across the different domains for classification tasks. DefRec~\cite{achituve2021self} utilizes the self-supervised tasks to reconstruct deformed input point clouds that produce meaningful representations and obtain a higher classification accuracy than PointDAN on PointDA-10. It suggested that self-supervised learning (SL) is an effective way for the 3D point cloud domain adaptation. However, solely using the SL network often fails to produce promising results with the simple student net, which does not consider the reliability of the network for discriminate feature representations learned from the similar samples on target domain. If we use a teacher net to supervise the student net to give a more distinct reconstruction, we will improve the performance of the whole network. As described in Figure \ref{fig:motivation}, most of the SL methods only rely on a student network to predict the final results. For example, if the chair is similar to the sofa from the source domain, the student net is difficult to discriminate similar samples from the target domain by its ability. Once the teacher net is used to transfer the guidance knowledge to supervise the student net during the process of learning, it can give the accurate judgment of the sample to the chair. &#10;&#10;In this paper, we propose a self-ensembling network (named SEN) for 3D point cloud domain adaptation. Generally, our SEN leveraged the advantages of Mean Teacher and semi-supervised learning to transfer knowledge gained from a labeled source dataset to a different unlabeled target dataset. We introduce a soft classification loss for the source domain learning, which used the teacher model to provide reliable soft labels so that to naturally supervise the student model. We also introduce a consistency loss for target domain learning, which aims to guarantee the predictions to be consistent with the teacher and student network so that to improve the student network's robustness. Moreover, SEN easily bridges the domain gap between supervised and unsupervised representation learning and improves the whole network generalization and prediction by self-ensembling on 3D point clouds.&#10;&#10;The main contributions of our paper could be summarized as follows: 1) We propose a novel end-to-end self-ensembling network (SEN) for 3D point cloud domain adaptation tasks. 2) Our SEN resorts to the advantages of Mean Teacher and semi-supervised learning, in which, a student network is kept in a collaborative manner with supervised learning and self-supervised learning, and a teacher network conducts temporal consistency to learn useful representations and improve the accuracy of point clouds reconstruction. 3) We introduce a soft classification loss and a consistency loss to achieve consistent generalization and accurate reconstruction. 4) Our proposed method achieved significant results and gained over the state-of-the-art methods on PointDA-10 and PointSegDA datasets for 3D point cloud domain adaptation tasks. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Autonomous Systems, Computer Science, Unsupervised Domain Adaptation" />
-        </attvalues>
-      </node>
-      <node id="1612.00593" label="1612.00593">
-        <attvalues>
-          <attvalue for="0" value="PointNet: Deep Learning on Point Sets for 3D Classification and&#10;  Segmentation" />
-          <attvalue for="1" value="  Point cloud is an important type of geometric data structure. Due to its&#10;irregular format, most researchers transform such data to regular 3D voxel&#10;grids or collections of images. This, however, renders data unnecessarily&#10;voluminous and causes issues. In this paper, we design a novel type of neural&#10;network that directly consumes point clouds and well respects the permutation&#10;invariance of points in the input. Our network, named PointNet, provides a&#10;unified architecture for applications ranging from object classification, part&#10;segmentation, to scene semantic parsing. Though simple, PointNet is highly&#10;efficient and effective. Empirically, it shows strong performance on par or&#10;even better than state of the art. Theoretically, we provide analysis towards&#10;understanding of what the network has learnt and why the network is robust with&#10;respect to input perturbation and corruption.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.01495" label="1804.01495">
-        <attvalues>
-          <attvalue for="0" value="Density Adaptive Point Set Registration" />
-          <attvalue for="1" value="  Probabilistic methods for point set registration have demonstrated&#10;competitive results in recent years. These techniques estimate a probability&#10;distribution model of the point clouds. While such a representation has shown&#10;promise, it is highly sensitive to variations in the density of 3D points. This&#10;fundamental problem is primarily caused by changes in the sensor location&#10;across point sets. We revisit the foundations of the probabilistic registration&#10;paradigm. Contrary to previous works, we model the underlying structure of the&#10;scene as a latent probability distribution, and thereby induce invariance to&#10;point set density changes. Both the probabilistic model of the scene and the&#10;registration parameters are inferred by minimizing the Kullback-Leibler&#10;divergence in an Expectation Maximization based framework. Our density-adaptive&#10;registration successfully handles severe density variations commonly&#10;encountered in terrestrial Lidar applications. We perform extensive experiments&#10;on several challenging real-world Lidar datasets. The results demonstrate that&#10;our approach outperforms state-of-the-art probabilistic methods for multi-view&#10;registration, without the need of re-sampling. Code is available at&#10;https://github.com/felja633/DARE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.10150" label="1911.10150">
-        <attvalues>
-          <attvalue for="0" value="PointPainting: Sequential Fusion for 3D Object Detection" />
-          <attvalue for="1" value="  Camera and lidar are important sensor modalities for robotics in general and&#10;self-driving cars in particular. The sensors provide complementary information&#10;offering an opportunity for tight sensor-fusion. Surprisingly, lidar-only&#10;methods outperform fusion methods on the main benchmark datasets, suggesting a&#10;gap in the literature. In this work, we propose PointPainting: a sequential&#10;fusion method to fill this gap. PointPainting works by projecting lidar points&#10;into the output of an image-only semantic segmentation network and appending&#10;the class scores to each point. The appended (painted) point cloud can then be&#10;fed to any lidar-only method. Experiments show large improvements on three&#10;different state-of-the art methods, Point-RCNN, VoxelNet and PointPillars on&#10;the KITTI and nuScenes datasets. The painted version of PointRCNN represents a&#10;new state of the art on the KITTI leaderboard for the bird's-eye view detection&#10;task. In ablation, we study how the effects of Painting depends on the quality&#10;and format of the semantic segmentation output, and demonstrate how latency can&#10;be minimized through pipelining.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.00880" label="1505.00880">
-        <attvalues>
-          <attvalue for="0" value="Multi-view Convolutional Neural Networks for 3D Shape Recognition" />
-          <attvalue for="1" value="  A longstanding question in computer vision concerns the representation of 3D&#10;shapes for recognition: should 3D shapes be represented with descriptors&#10;operating on their native 3D formats, such as voxel grid or polygon mesh, or&#10;can they be effectively represented with view-based descriptors? We address&#10;this question in the context of learning to recognize 3D shapes from a&#10;collection of their rendered views on 2D images. We first present a standard&#10;CNN architecture trained to recognize the shapes' rendered views independently&#10;of each other, and show that a 3D shape can be recognized even from a single&#10;view at an accuracy far higher than using state-of-the-art 3D shape&#10;descriptors. Recognition rates further increase when multiple views of the&#10;shapes are provided. In addition, we present a novel CNN architecture that&#10;combines information from multiple views of a 3D shape into a single and&#10;compact shape descriptor offering even better recognition performance. The same&#10;architecture can be applied to accurately recognize human hand-drawn sketches&#10;of shapes. We conclude that a collection of 2D views can be highly informative&#10;for 3D shape recognition and is amenable to emerging CNN architectures and&#10;their derivatives.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.07246" label="1811.07246">
-        <attvalues>
-          <attvalue for="0" value="PointConv: Deep Convolutional Networks on 3D Point Clouds" />
-          <attvalue for="1" value="  Unlike images which are represented in regular dense grids, 3D point clouds&#10;are irregular and unordered, hence applying convolution on them can be&#10;difficult. In this paper, we extend the dynamic filter to a new convolution&#10;operation, named PointConv. PointConv can be applied on point clouds to build&#10;deep convolutional networks. We treat convolution kernels as nonlinear&#10;functions of the local coordinates of 3D points comprised of weight and density&#10;functions. With respect to a given point, the weight functions are learned with&#10;multi-layer perceptron networks and density functions through kernel density&#10;estimation. The most important contribution of this work is a novel&#10;reformulation proposed for efficiently computing the weight functions, which&#10;allowed us to dramatically scale up the network and significantly improve its&#10;performance. The learned convolution kernel can be used to compute&#10;translation-invariant and permutation-invariant convolution on any point set in&#10;the 3D space. Besides, PointConv can also be used as deconvolution operators to&#10;propagate features from a subsampled point cloud back to its original&#10;resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep&#10;convolutional neural networks built on PointConv are able to achieve&#10;state-of-the-art on challenging semantic segmentation benchmarks on 3D point&#10;clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed&#10;that networks built on PointConv can match the performance of convolutional&#10;networks in 2D images of a similar structure.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.07829" label="1801.07829">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Graph CNN for Learning on Point Clouds" />
-          <attvalue for="1" value="  Point clouds provide a flexible geometric representation suitable for&#10;countless applications in computer graphics; they also comprise the raw output&#10;of most 3D data acquisition devices. While hand-designed features on point&#10;clouds have long been proposed in graphics and vision, however, the recent&#10;overwhelming success of convolutional neural networks (CNNs) for image analysis&#10;suggests the value of adapting insight from CNN to the point cloud world. Point&#10;clouds inherently lack topological information so designing a model to recover&#10;topology can enrich the representation power of point clouds. To this end, we&#10;propose a new neural network module dubbed EdgeConv suitable for CNN-based&#10;high-level tasks on point clouds including classification and segmentation.&#10;EdgeConv acts on graphs dynamically computed in each layer of the network. It&#10;is differentiable and can be plugged into existing architectures. Compared to&#10;existing modules operating in extrinsic space or treating each point&#10;independently, EdgeConv has several appealing properties: It incorporates local&#10;neighborhood information; it can be stacked applied to learn global shape&#10;properties; and in multi-layer systems affinity in feature space captures&#10;semantic characteristics over potentially long distances in the original&#10;embedding. We show the performance of our model on standard benchmarks&#10;including ModelNet40, ShapeNetPart, and S3DIS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.08207" label="1910.08207">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Multi-Task Feature Learning on Point Clouds" />
-          <attvalue for="1" value="  We introduce an unsupervised multi-task model to jointly learn point and&#10;shape features on point clouds. We define three unsupervised tasks including&#10;clustering, reconstruction, and self-supervised classification to train a&#10;multi-scale graph-based encoder. We evaluate our model on shape classification&#10;and segmentation benchmarks. The results suggest that it outperforms prior&#10;state-of-the-art unsupervised models: In the ModelNet40 classification task, it&#10;achieves an accuracy of 89.1% and in ShapeNet segmentation task, it achieves an&#10;mIoU of 68.2 and accuracy of 88.6%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.01661" label="2003.01661">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Intrinsic Structural Representation Points" />
-          <attvalue for="1" value="  Learning structures of 3D shapes is a fundamental problem in the field of&#10;computer graphics and geometry processing. We present a simple yet&#10;interpretable unsupervised method for learning a new structural representation&#10;in the form of 3D structure points. The 3D structure points produced by our&#10;method encode the shape structure intrinsically and exhibit semantic&#10;consistency across all the shape instances with similar structures. This is a&#10;challenging goal that has not fully been achieved by other methods.&#10;Specifically, our method takes a 3D point cloud as input and encodes it as a&#10;set of local features. The local features are then passed through a novel point&#10;integration module to produce a set of 3D structure points. The chamfer&#10;distance is used as reconstruction loss to ensure the structure points lie&#10;close to the input point cloud. Extensive experiments have shown that our&#10;method outperforms the state-of-the-art on the semantic shape correspondence&#10;task and achieves comparable performance with the state-of-the-art on the&#10;segmentation label transfer task. Moreover, the PCA based shape embedding built&#10;upon consistent structure points demonstrates good performance in preserving&#10;the shape structures. Code is available at&#10;https://github.com/NolenChen/3DStructurePoints&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.07262" label="1712.07262">
-        <attvalues>
-          <attvalue for="0" value="FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation" />
-          <attvalue for="1" value="  Recent deep networks that directly handle points in a point set, e.g.,&#10;PointNet, have been state-of-the-art for supervised learning tasks on point&#10;clouds such as classification and segmentation. In this work, a novel&#10;end-to-end deep auto-encoder is proposed to address unsupervised learning&#10;challenges on point clouds. On the encoder side, a graph-based enhancement is&#10;enforced to promote local structures on top of PointNet. Then, a novel&#10;folding-based decoder deforms a canonical 2D grid onto the underlying 3D object&#10;surface of a point cloud, achieving low reconstruction errors even for objects&#10;with delicate structures. The proposed decoder only uses about 7% parameters of&#10;a decoder with fully-connected neural networks, yet leads to a more&#10;discriminative representation that achieves higher linear SVM classification&#10;accuracy than the benchmark. In addition, the proposed decoder structure is&#10;shown, in theory, to be a generic architecture that is able to reconstruct an&#10;arbitrary point cloud from a 2D grid. Our code is available at&#10;http://www.merl.com/research/license#FoldingNet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.02744" label="1911.02744">
-        <attvalues>
-          <attvalue for="0" value="PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud&#10;  Representation" />
-          <attvalue for="1" value="  Domain Adaptation (DA) approaches achieved significant improvements in a wide&#10;range of machine learning and computer vision tasks (i.e., classification,&#10;detection, and segmentation). However, as far as we are aware, there are few&#10;methods yet to achieve domain adaptation directly on 3D point cloud data. The&#10;unique challenge of point cloud data lies in its abundant spatial geometric&#10;information, and the semantics of the whole object is contributed by including&#10;regional geometric structures. Specifically, most general-purpose DA methods&#10;that struggle for global feature alignment and ignore local geometric&#10;information are not suitable for 3D domain alignment. In this paper, we propose&#10;a novel 3D Domain Adaptation Network for point cloud data (PointDAN). PointDAN&#10;jointly aligns the global and local features in multi-level. For local&#10;alignment, we propose Self-Adaptive (SA) node module with an adjusted receptive&#10;field to model the discriminative local structures for aligning domains. To&#10;represent hierarchically scaled features, node-attention module is further&#10;introduced to weight the relationship of SA nodes across objects and domains.&#10;For global alignment, an adversarial-training strategy is employed to learn and&#10;align global features across domains. Since there is no common evaluation&#10;benchmark for 3D point cloud DA scenario, we build a general benchmark (i.e.,&#10;PointDA-10) extracted from three popular 3D object/scene datasets (i.e.,&#10;ModelNet, ShapeNet and ScanNet) for cross-domain 3D objects classification&#10;fashion. Extensive experiments on PointDA-10 illustrate the superiority of our&#10;model over the state-of-the-art general-purpose DA methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.13090" label="2312.13090">
-        <attvalues>
-          <attvalue for="0" value="Perception Test 2023: A Summary of the First Challenge And Outcome" />
-          <attvalue for="1" value="  The First Perception Test challenge was held as a half-day workshop alongside&#10;the IEEE/CVF International Conference on Computer Vision (ICCV) 2023, with the&#10;goal of benchmarking state-of-the-art video models on the recently proposed&#10;Perception Test benchmark. The challenge had six tracks covering low-level and&#10;high-level tasks, with both a language and non-language interface, across&#10;video, audio, and text modalities, and covering: object tracking, point&#10;tracking, temporal action localisation, temporal sound localisation,&#10;multiple-choice video question-answering, and grounded video&#10;question-answering. We summarise in this report the task descriptions, metrics,&#10;baselines, and results.&#10;" />
-          <attvalue for="2" value="&#10;Making progress towards building perception systems that achieve human-level scene understanding requires robust and comprehensive evaluations to guide research. Following-up on our previous ECCV 2022 workshop on designing and evaluating computer perception systems, we organised this workshop-challenge to focus on evaluating SOTA perception models. &#10;&#10;Dataset: Most of the existing benchmarks have been designed to evaluate specialised models, e.g.\ image or video classifiers, objects detectors and so on. Using a collection of such benchmarks to evaluate general perception models results in expensive evaluations that are redundant in some areas (e.g.\ semantics), while missing other areas (e.g.\ memory or physics). The Perception Test~\cite{patraucean2023perception} is the first benchmark to use real world videos and focus on comprehensively diagnosing perception capabilities, like memory, understanding of intuitive physics and geometry, abstractions, or semantics. The benchmark consists of 11.6k videos, with audio, up to 35s long, filmed by diverse crowd-sourced participants following scripts designed to show perceptually-interesting situations. The videos have six types of annotations enabling language and non-language evaluations, across video, audio, and text modalities. The benchmark focuses on probing generalisation and transfer capabilities, so it only provides a relatively small training set to be used for fine-tuning or prompting, and the rest is used for evaluation. The goal of this workshop-challenge was to evaluate SOTA video models using the Perception Test benchmark. More details of the Perception Test and data samples are available on our github repository and workshop website.&#10;&#10;Challenge tracks: Ideally, a single model should be able to perform any evaluation task defined using the annotations provided in the Perception Test. However, such a model does not exist. For this first iteration of the challenge, we defined six individual challenge tracks and evaluate SOTA models whose interface specialises for each annotation type: object tracking, point tracking, temporal action localisation, temporal sound localisation, multiple-choice video QA, grounded video QA.&#10;&#10;Challenge setup: We relied on the open-source eval.ai platform to set up the different challenge tracks. Each track had 2 phases (validation and test), each phase using the corresponding validation and test splits of the Perception Test benchmark. For each submission, the participants had to indicate the evaluation mode (fine-tuning, few-shot, or zero-shot evaluation). In some tracks, the participants had to indicate if the model used the audio modality as well or not (for action and sound localisation, multiple-choice video QA). For test submissions, the participants were required to also upload a short report describing their method (architecture, pre-training datasets and tasks, etc.).&#10;The validation phase served as a sanity check for participants' submission pipelines. The number of submissions for the validation phase was not limited.&#10;&#10;The test set was made available 1.5 months before the submission deadline. &#10;Only the published results on the test set were considered for the competition. For the test phase, the limit was set to 2 submissions per day, 25 submissions in total. We received 475 submissions from 63 teams across all six tracks in both phases. We awarded 2 prizes per track (best and runner-up) to submissions that obtained the best (and second best) results in the leaderboard. To encourage submissions of novel models, but which are not (yet) competitive with SOTA models, we awarded two novelty awards across all tracks; see next sectionfor more details. The reports of the winning submissions are available on the workshop website.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Artificial Intelligence, Cognitive Science, Computer Science, Benchmarking Challenges, Multimodal Perception" />
-        </attvalues>
-      </node>
-      <node id="2305.13786" label="2305.13786">
-        <attvalues>
-          <attvalue for="0" value="Perception Test: A Diagnostic Benchmark for Multimodal Video Models" />
-          <attvalue for="1" value="  We propose a novel multimodal video benchmark - the Perception Test - to&#10;evaluate the perception and reasoning skills of pre-trained multimodal models&#10;(e.g. Flamingo, SeViLA, or GPT-4). Compared to existing benchmarks that focus&#10;on computational tasks (e.g. classification, detection or tracking), the&#10;Perception Test focuses on skills (Memory, Abstraction, Physics, Semantics) and&#10;types of reasoning (descriptive, explanatory, predictive, counterfactual)&#10;across video, audio, and text modalities, to provide a comprehensive and&#10;efficient evaluation tool. The benchmark probes pre-trained models for their&#10;transfer capabilities, in a zero-shot / few-shot or limited finetuning regime.&#10;For these purposes, the Perception Test introduces 11.6k real-world videos, 23s&#10;average length, designed to show perceptually interesting situations, filmed by&#10;around 100 participants worldwide. The videos are densely annotated with six&#10;types of labels (multiple-choice and grounded video question-answers, object&#10;and point tracks, temporal action and sound segments), enabling both language&#10;and non-language evaluations. The fine-tuning and validation splits of the&#10;benchmark are publicly available (CC-BY license), in addition to a challenge&#10;server with a held-out test split. Human baseline results compared to&#10;state-of-the-art video QA models show a substantial gap in performance (91.4%&#10;vs 46.2%), suggesting that there is significant room for improvement in&#10;multimodal video understanding.&#10;  Dataset, baseline code, and challenge server are available at&#10;https://github.com/deepmind/perception_test&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09464" label="1908.09464">
-        <attvalues>
-          <attvalue for="0" value="Shape-Aware Human Pose and Shape Reconstruction Using Multi-View Images" />
-          <attvalue for="1" value="  We propose a scalable neural network framework to reconstruct the 3D mesh of&#10;a human body from multi-view images, in the subspace of the SMPL model. Use of&#10;multi-view images can significantly reduce the projection ambiguity of the&#10;problem, increasing the reconstruction accuracy of the 3D human body under&#10;clothing. Our experiments show that this method benefits from the synthetic&#10;dataset generated from our pipeline since it has good flexibility of variable&#10;control and can provide ground-truth for validation. Our method outperforms&#10;existing methods on real-world images, especially on shape estimations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In this section, we survey recent works on human body pose and shape estimation, neural network techniques, and other related work that make use of synthetic data.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Neural Network Framework, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2012.14833" label="2012.14833">
-        <attvalues>
-          <attvalue for="0" value="Visual-Thermal Camera Dataset Release and Multi-Modal Alignment without&#10;  Calibration Information" />
-          <attvalue for="1" value="  This report accompanies a dataset release on visual and thermal camera data&#10;and details a procedure followed to align such multi-modal camera frames in&#10;order to provide pixel-level correspondence between the two without using&#10;intrinsic or extrinsic calibration information. To achieve this goal we benefit&#10;from progress in the domain of multi-modal image alignment and specifically&#10;employ the Mattes Mutual Information Metric to guide the registration process.&#10;In the released dataset we release both the raw visual and thermal camera data,&#10;as well as the aligned frames, alongside calibration parameters with the goal&#10;to better facilitate the investigation on common local/global features across&#10;such multi-modal image streams.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Autonomous robotic operation and scene understanding often requires the fusion of diverse sensing modalities. Especially when robot navigation and environment cognition in visually-degraded environments is considered~\cite{ARTIFACTS_AEROCONF2020,GBPLANNER_IROS_2019,sizintsev2019multi}, robotic systems may benefit if equipped with not only traditional camera systems but also sensors with complementary capabilities. Thermal vision in particular offers certain capabilities of interest including the fact that it can remain informative in conditions of darkness or certain types of obscurants such as dust. In previous work, the research community has investigated - among others - the benefits of thermal vision specifically for the problem of thermal-inertial odometry estimation~\cite{KTIO_ICRA_2019,VTIMUDVE_AEROCONF_2019,khattak2019robust,zhao2020tp,khattak2018marker} and object detection~\cite{treptow2006real,ciric2013computationally,yin2006moving}. When multiple modalities are available onboard an essential question relates to the best way that their data may be fused in order to allow maximum resilience and information-rich robotic operation. Motivated by the above, we seek to identify how to best fuse the data from visual and thermal vision. Towards that goal and with the aim to investigate both hand-designed and data-driven approaches for the underlying multi-modal data association problem, there is need to develop datasets that allow to study the cross-modality relations of visual and thermal cameras for robotic applications. &#10;&#10;This working paper serves to accompany a dataset release on visual and Long-Wave InfraRed (LWIR) camera data and to further demonstrate a process for their alignment assuming lack of intrinsic and extrinsic calibration parameters, alongside no knowledge of the depth associated with each pixel. The method employed considers the fact that visual and thermal camera are different in nature and thus the image intensities do not necessarily present high similarity - especially in a local sense. Reflecting this fact, the employed approach to achieve alignment is a multi-modal one and in particular it relates to the utilization of the Mattes Mutual Information Criterion in order to guide the alignment. Given the two image sets, visual and thermal, the method seeks to identify the geometric transformation $\mathbf{g}$ between each pair that maps every point $\mathbf{x}$ in the visual frame to the point $\mathbf{g}(\mathbf{x})$ on the thermal image. Notably, the dataset release does contain calibration information. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Image Processing, Physics, Computer Vision Dataset, Image Alignment Techniques, Computer Science, Multi-Modal Imaging" />
-        </attvalues>
-      </node>
-      <node id="1903.01656" label="1903.01656">
-        <attvalues>
-          <attvalue for="0" value="Visual-Thermal Landmarks and Inertial Fusion for Navigation in Degraded&#10;  Visual Environments" />
-          <attvalue for="1" value="  With an ever-widening domain of aerial robotic applications, including many&#10;mission critical tasks such as disaster response operations, search and rescue&#10;missions and infrastructure inspections taking place in GPS-denied&#10;environments, the need for reliable autonomous operation of aerial robots has&#10;become crucial. Operating in GPS-denied areas aerial robots rely on a multitude&#10;of sensors to localize and navigate. Visible spectrum cameras are the most&#10;commonly used sensors due to their low cost and weight. However, in&#10;environments that are visually-degraded such as in conditions of poor&#10;illumination, low texture, or presence of obscurants including fog, smoke and&#10;dust, the reliability of visible light cameras deteriorates significantly.&#10;Nevertheless, maintaining reliable robot navigation in such conditions is&#10;essential. In contrast to visible light cameras, thermal cameras offer&#10;visibility in the infrared spectrum and can be used in a complementary manner&#10;with visible spectrum cameras for robot localization and navigation tasks,&#10;without paying the significant weight and power penalty typically associated&#10;with carrying other sensors. Exploiting this fact, in this work we present a&#10;multi-sensor fusion algorithm for reliable odometry estimation in GPS-denied&#10;and degraded visual environments. The proposed method utilizes information from&#10;both the visible and thermal spectra for landmark selection and prioritizes&#10;feature extraction from informative image regions based on a metric over&#10;spatial entropy. Furthermore, inertial sensing cues are integrated to improve&#10;the robustness of the odometry estimation process. To verify our solution, a&#10;set of challenging experiments were conducted inside a) an obscurant filed&#10;machine shop-like industrial environment, as well as b) a dark subterranean&#10;mine in the presence of heavy airborne dust.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.03455" label="2012.03455">
-        <attvalues>
-          <attvalue for="0" value="TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPoint" />
-          <attvalue for="1" value="  To achieve robust motion estimation in visually degraded environments,&#10;thermal odometry has been an attraction in the robotics community. However,&#10;most thermal odometry methods are purely based on classical feature extractors,&#10;which is difficult to establish robust correspondences in successive frames due&#10;to sudden photometric changes and large thermal noise. To solve this problem,&#10;we propose ThermalPoint, a lightweight feature detection network specifically&#10;tailored for producing keypoints on thermal images, providing notable&#10;anti-noise improvements compared with other state-of-the-art methods. After&#10;that, we combine ThermalPoint with a novel radiometric feature tracking method,&#10;which directly makes use of full radiometric data and establishes reliable&#10;correspondences between sequential frames. Finally, taking advantage of an&#10;optimization-based visual-inertial framework, a deep feature-based&#10;thermal-inertial odometry (TP-TIO) framework is proposed and evaluated&#10;thoroughly in various visually degraded environments. Experiments show that our&#10;method outperforms state-of-the-art visual and laser odometry methods in&#10;smoke-filled environments and achieves competitive accuracy in normal&#10;environments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.00782" label="1903.00782">
-        <attvalues>
-          <attvalue for="0" value="Marker based Thermal-Inertial Localization for Aerial Robots in&#10;  Obscurant Filled Environments" />
-          <attvalue for="1" value="  For robotic inspection tasks in known environments fiducial markers provide a&#10;reliable and low-cost solution for robot localization. However, detection of&#10;such markers relies on the quality of RGB camera data, which degrades&#10;significantly in the presence of visual obscurants such as fog and smoke. The&#10;ability to navigate known environments in the presence of obscurants can be&#10;critical for inspection tasks especially, in the aftermath of a disaster.&#10;Addressing such a scenario, this work proposes a method for the design of&#10;fiducial markers to be used with thermal cameras for the pose estimation of&#10;aerial robots. Our low cost markers are designed to work in the long wave&#10;infrared spectrum, which is not affected by the presence of obscurants, and can&#10;be affixed to any object that has measurable temperature difference with&#10;respect to its surroundings. Furthermore, the estimated pose from the fiducial&#10;markers is fused with inertial measurements in an extended Kalman filter to&#10;remove high frequency noise and error present in the fiducial pose estimates.&#10;The proposed markers and the pose estimation method are experimentally&#10;evaluated in an obscurant filled environment using an aerial robot carrying a&#10;thermal camera.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.12163" label="2207.12163">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale RAFT: Combining Hierarchical Concepts for Learning-based&#10;  Optical FLow Estimation" />
-          <attvalue for="1" value="  Many classical and learning-based optical flow methods rely on hierarchical&#10;concepts to improve both accuracy and robustness. However, one of the currently&#10;most successful approaches -- RAFT -- hardly exploits such concepts. In this&#10;work, we show that multi-scale ideas are still valuable. More precisely, using&#10;RAFT as a baseline, we propose a novel multi-scale neural network that combines&#10;several hierarchical concepts within a single estimation framework. These&#10;concepts include (i) a partially shared coarse-to-fine architecture, (ii)&#10;multi-scale features, (iii) a hierarchical cost volume and (iv) a multi-scale&#10;multi-iteration loss. Experiments on MPI Sintel and KITTI clearly demonstrate&#10;the benefits of our approach. They show not only substantial improvements&#10;compared to RAFT, but also state-of-the-art results -- in particular in&#10;non-occluded regions. Code will be available at&#10;https://github.com/cv-stuttgart/MS_RAFT.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The estimation of the optical flow is one of the key problems in computer vision. &#10;To solve this task many approaches rely on matching characteristic image features by minimizing some local or global cost. In this context, hierarchical matching schemes have a long tradition, since they address both accuracy and robustness&#10;\cite{Moravec1979_VisMap,&#10;Anandan1989,&#10;Black1991,&#10;Brox2004,&#10;Xu2011_MDPFlow,Hu2016_CPM}. &#10;By considering flow initializations from coarser scales,&#10;they not only enable the estimation of larger motion &#10;but also render the underlying minimization more robust w.r.t.\ local minima.&#10;&#10;Meanwhile, hierarchical schemes also became popular in the context of neural networks. For instance, SpyNet &#10;\cite{Ranjan2017_SPyNet} proposes to use a spatial pyramid network, where the downsampled input frames are warped&#10;by the current estimate and &#10;iterative updates proceed in a coarse-to-fine manner \cite{Brox2004}.&#10;PWC-Net \cite{Sun2018_PWC} improved this idea by warping image features&#10;instead of input frames and by a joint training of the individual scales using&#10; a multi-scale loss. Finally, IRR \cite{Hur2019} suggests to share the estimation network across all scales, leading to a more compact architecture which can be trained more efficiently. &#10;&#10;In contrast, RAFT \cite{Teed2020_RAFT} follows a completely different ap- proach.&#10;It performs recurrent updates on a single scale with spatial size $\frac{1}{8}\times (h,w)$&#10;finally interpolating the results.&#10;When performing the updates, however, it relies on a sampling-based hierarchical cost volume, i.e.\ a correlation pyramid with multiple lookup levels.&#10;While the correlation pyramid adds non-local cost information to the matching process, the sampling allows to overcome some&#10;problems of &#10;warping&#10;\cite{Hofinger2020_IOFPL}.&#10;&#10;Given the orthogonal hierarchical concepts considered in PWC-Net / IRR and RAFT, it seems desirable to combine them within a single architecture. Moreover, in this context, it seems also desirable to integrate other hierarchical concepts, e.g.\ regarding the features and regarding the loss.&#10;&#10;{Contributions.} Hence, in this work, we propose a multi-scale neural network that builds upon RAFT and combines several established hierarchical concepts within a single approach. In this context, our contributions are fourfold:&#10;(i) We embed the &#10;recurrent &#10;estimation in a coarse-to-fine architecture &#10;-- using a shared sub-network -- &#10;which enables us to build upon results from coarser scales.&#10;(ii) Besides, similar to feature pyramid networks (FPNs) \cite{Lin2017_FPN}, we propose to use U-Net-style features that allow to exploit multi-scale information also during matching.&#10;(iii) Moreover, when combining our coarse-to-fine architecture with our multi-scale features, we suggest to still use RAFT's correlation pyramid as third hierarchical concept on top, since it provides additional information in terms of non-local costs.&#10;(iv) Finally, we propose a multi-scale multi-iteration loss that relies on a novel sample-wise robustification and hence allows for a more robust training. Experiments on Sintel \cite{Butler2012_Sintel} and KITTI \cite{Menze2015_KITTI} demonstrate that our multi-scale efforts pay off. They show clear improvements compared to RAFT and state-of-the-art results on both benchmarks - with a particularly high accuracy in non-occluded regions.&#10;&#10;{Related Work.} To the best of our knowledge, there is only the report of Wan {et al.\ }\cite{Wan2020_PRAFlow} that seeks to extend RAFT to more than one scale. &#10;However, in their approach only two scales are investigated, features &#10;are limited to a bottom-up &#10;approach (cf.\ \cite{Lin2017_FPN})&#10;and the actual loss is not specified.&#10;Regarding the use of multi-scale features,&#10;the&#10;methods of Saxena {et al.\ }\cite{Saxena2019_pwoc}, Rishav {et al.\ }\cite{Rishav2021_ResFPN} and Long and Lang \cite {Long2022}&#10;have to be mentioned. &#10;While &#10;\cite{Saxena2019_pwoc,Rishav2021_ResFPN} are also based on &#10;FPN ideas to &#10;integrate semantically important information from other scales,&#10;they rely on PWC-Net \cite{Sun2018_PWC} and hence neither exploit recurrent updates nor a non-local lookup in the cost volume. &#10;On the other hand, while &#10;\cite {Long2022} is &#10;based on RAFT, it estimates the flow on a single scale thereby using a feature extractor&#10;that &#10;aims at preserving fine-scale details&#10;via residual downsampling.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Computer Vision, Optical Flow Methods, Mathematics, Physics, Computer Science, Neural Network Architecture, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1611.00850" label="1611.00850">
-        <attvalues>
-          <attvalue for="0" value="Optical Flow Estimation using a Spatial Pyramid Network" />
-          <attvalue for="1" value="  We learn to compute optical flow by combining a classical spatial-pyramid&#10;formulation with deep learning. This estimates large motions in a&#10;coarse-to-fine approach by warping one image of a pair at each pyramid level by&#10;the current flow estimate and computing an update to the flow. Instead of the&#10;standard minimization of an objective function at each pyramid level, we train&#10;one deep network per level to compute the flow update. Unlike the recent&#10;FlowNet approach, the networks do not need to deal with large motions; these&#10;are dealt with by the pyramid. This has several advantages. First, our Spatial&#10;Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms&#10;of model parameters. This makes it more efficient and appropriate for embedded&#10;applications. Second, since the flow at each pyramid level is small (&lt; 1&#10;pixel), a convolutional approach applied to pairs of warped images is&#10;appropriate. Third, unlike FlowNet, the learned convolution filters appear&#10;similar to classical spatio-temporal filters, giving insight into the method&#10;and how to improve it. Our results are more accurate than FlowNet on most&#10;standard benchmarks, suggesting a new direction of combining classical flow&#10;methods with deep learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.02371" label="1709.02371">
-        <attvalues>
-          <attvalue for="0" value="PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume" />
-          <attvalue for="1" value="  We present a compact but effective CNN model for optical flow, called&#10;PWC-Net. PWC-Net has been designed according to simple and well-established&#10;principles: pyramidal processing, warping, and the use of a cost volume. Cast&#10;in a learnable feature pyramid, PWC-Net uses the cur- rent optical flow&#10;estimate to warp the CNN features of the second image. It then uses the warped&#10;features and features of the first image to construct a cost volume, which is&#10;processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in&#10;size and easier to train than the recent FlowNet2 model. Moreover, it&#10;outperforms all published optical flow methods on the MPI Sintel final pass and&#10;KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024x436)&#10;images. Our models are available on https://github.com/NVlabs/PWC-Net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05290" label="1904.05290">
-        <attvalues>
-          <attvalue for="0" value="Iterative Residual Refinement for Joint Optical Flow and Occlusion&#10;  Estimation" />
-          <attvalue for="1" value="  Deep learning approaches to optical flow estimation have seen rapid progress&#10;over the recent years. One common trait of many networks is that they refine an&#10;initial flow estimate either through multiple stages or across the levels of a&#10;coarse-to-fine representation. While leading to more accurate results, the&#10;downside of this is an increased number of parameters. Taking inspiration from&#10;both classical energy minimization approaches as well as residual networks, we&#10;propose an iterative residual refinement (IRR) scheme based on weight sharing&#10;that can be combined with several backbone networks. It reduces the number of&#10;parameters, improves the accuracy, or even achieves both. Moreover, we show&#10;that integrating occlusion prediction and bi-directional flow estimation into&#10;our IRR scheme can further boost the accuracy. Our full network achieves&#10;state-of-the-art results for both optical flow and occlusion estimation across&#10;several standard datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.12039" label="2003.12039">
-        <attvalues>
-          <attvalue for="0" value="RAFT: Recurrent All-Pairs Field Transforms for Optical Flow" />
-          <attvalue for="1" value="  We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network&#10;architecture for optical flow. RAFT extracts per-pixel features, builds&#10;multi-scale 4D correlation volumes for all pairs of pixels, and iteratively&#10;updates a flow field through a recurrent unit that performs lookups on the&#10;correlation volumes. RAFT achieves state-of-the-art performance. On KITTI, RAFT&#10;achieves an F1-all error of 5.10%, a 16% error reduction from the best&#10;published result (6.10%). On Sintel (final pass), RAFT obtains an&#10;end-point-error of 2.855 pixels, a 30% error reduction from the best published&#10;result (4.098 pixels). In addition, RAFT has strong cross-dataset&#10;generalization as well as high efficiency in inference time, training speed,&#10;and parameter count. Code is available at https://github.com/princeton-vl/RAFT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.10739" label="1912.10739">
-        <attvalues>
-          <attvalue for="0" value="Improving Optical Flow on a Pyramid Level" />
-          <attvalue for="1" value="  In this work we review the coarse-to-fine spatial feature pyramid concept,&#10;which is used in state-of-the-art optical flow estimation networks to make&#10;exploration of the pixel flow search space computationally tractable and&#10;efficient. Within an individual pyramid level, we improve the cost volume&#10;construction process by departing from a warping- to a sampling-based strategy,&#10;which avoids ghosting and hence enables us to better preserve fine flow&#10;details. We further amplify the positive effects through a level-specific, loss&#10;max-pooling strategy that adaptively shifts the focus of the learning process&#10;on under-performing predictions. Our second contribution revises the gradient&#10;flow across pyramid levels. The typical operations performed at each pyramid&#10;level can lead to noisy, or even contradicting gradients across levels. We show&#10;and discuss how properly blocking some of these gradient components leads to&#10;improved convergence and ultimately better performance. Finally, we introduce a&#10;distillation concept to counteract the issue of catastrophic forgetting and&#10;thus preserving knowledge over models sequentially trained on multiple&#10;datasets. Our findings are conceptually simple and easy to implement, yet&#10;result in compelling improvements on relevant error measures that we&#10;demonstrate via exhaustive ablations on datasets like Flying Chairs2, Flying&#10;Things, Sintel and KITTI. We establish new state-of-the-art results on the&#10;challenging Sintel and KITTI 2012 test datasets, and even show the portability&#10;of our findings to different optical flow and depth from stereo approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06116" label="1904.06116">
-        <attvalues>
-          <attvalue for="0" value="PWOC-3D: Deep Occlusion-Aware End-to-End Scene Flow Estimation" />
-          <attvalue for="1" value="  In the last few years, convolutional neural networks (CNNs) have demonstrated&#10;increasing success at learning many computer vision tasks including dense&#10;estimation problems such as optical flow and stereo matching. However, the&#10;joint prediction of these tasks, called scene flow, has traditionally been&#10;tackled using slow classical methods based on primitive assumptions which fail&#10;to generalize. The work presented in this paper overcomes these drawbacks&#10;efficiently (in terms of speed and accuracy) by proposing PWOC-3D, a compact&#10;CNN architecture to predict scene flow from stereo image sequences in an&#10;end-to-end supervised setting. Further, large motion and occlusions are&#10;well-known problems in scene flow estimation. PWOC-3D employs specialized&#10;design decisions to explicitly model these challenges. In this regard, we&#10;propose a novel self-supervised strategy to predict occlusions from images&#10;(learned without any labeled occlusion data). Leveraging several such&#10;constructs, our network achieves competitive results on the KITTI benchmark and&#10;the challenging FlyingThings3D dataset. Especially on KITTI, PWOC-3D achieves&#10;the second place among end-to-end deep learning methods with 48 times fewer&#10;parameters than the top-performing method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.12235" label="2006.12235">
-        <attvalues>
-          <attvalue for="0" value="ResFPN: Residual Skip Connections in Multi-Resolution Feature Pyramid&#10;  Networks for Accurate Dense Pixel Matching" />
-          <attvalue for="1" value="  Dense pixel matching is required for many computer vision algorithms such as&#10;disparity, optical flow or scene flow estimation. Feature Pyramid Networks&#10;(FPN) have proven to be a suitable feature extractor for CNN-based dense&#10;matching tasks. FPN generates well localized and semantically strong features&#10;at multiple scales. However, the generic FPN is not utilizing its full&#10;potential, due to its reasonable but limited localization accuracy. Thus, we&#10;present ResFPN -- a multi-resolution feature pyramid network with multiple&#10;residual skip connections, where at any scale, we leverage the information from&#10;higher resolution maps for stronger and better localized features. In our&#10;ablation study, we demonstrate the effectiveness of our novel architecture with&#10;clearly higher accuracy than FPN. In addition, we verify the superior accuracy&#10;of ResFPN in many different pixel matching applications on established datasets&#10;like KITTI, Sintel, and FlyingThings3D.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.17096" label="2303.17096">
-        <attvalues>
-          <attvalue for="0" value="ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing" />
-          <attvalue for="1" value="  Recent studies have shown that higher accuracy on ImageNet usually leads to&#10;better robustness against different corruptions. Therefore, in this paper,&#10;instead of following the traditional research paradigm that investigates new&#10;out-of-distribution corruptions or perturbations deep models may encounter, we&#10;conduct model debugging in in-distribution data to explore which object&#10;attributes a model may be sensitive to. To achieve this goal, we create a&#10;toolkit for object editing with controls of backgrounds, sizes, positions, and&#10;directions, and create a rigorous benchmark named ImageNet-E(diting) for&#10;evaluating the image classifier robustness in terms of object attributes. With&#10;our ImageNet-E, we evaluate the performance of current deep learning models,&#10;including both convolutional neural networks and vision transformers. We find&#10;that most models are quite sensitive to attribute changes. A small change in&#10;the background can lead to an average of 9.23\% drop on top-1 accuracy. We also&#10;evaluate some robust models including both adversarially trained models and&#10;other robust trained models and find that some models show worse robustness&#10;against attribute changes than vanilla models. Based on these findings, we&#10;discover ways to enhance attribute robustness with preprocessing, architecture&#10;designs, and training strategies. We hope this work can provide some insights&#10;to the community and open up a new avenue for research in robust computer&#10;vision. The code and dataset are available at&#10;https://github.com/alibaba/easyrobust.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The literature related to attribute robustness benchmarks can be broadly grouped into the following themes: robustness benchmarks and attribute editing datasets. &#10;Existing robustness benchmarks such as ImageNet-C(orruption)~\cite{hendrycks2019benchmarking}, ImageNet-R(endition)~\cite{DeepAugment}, ImageNet-Stylized~\cite{SIN} and ImageNet-3DCC~\cite{kar20223d} mainly focus on the exploration of the corrupted or out-of-distribution data that models may encounter in reality. &#10;For instance, the ImageNet-R dataset contains various renditions (e.g., paintings, embroidery) of ImageNet object classes. ImageNet-C analyzes image models in terms of various simulated image corruptions~(e.g., noise, blur, weather, JPEG compression, etc.).&#10;Attribute editing dataset creation is a new topic and few studies have explored it before. &#10;Among them, ObjectNet~\cite{barbu2019objectnet} and ImageNet-9~(a.k.a. background challenge)~\cite{xiao2020noise} can be the representative. &#10;Specifically, ObjectNet collects a large real-world test set for object recognition with controls where object backgrounds, rotations, and imaging viewpoints are random. The images in ObjectNet are collected by their workers who image objects in their homes. It consists of $313$ classes which are mainly household objects. ImageNet-9 mainly creates a suit of datasets that help disentangle the impact of foreground and background signals on classification. To achieve this goal, it uses coarse-grained classes with corresponding rectangular bounding boxes to remove the foreground and then paste the cut area with other backgrounds. It can be observed that there lacks a dataset that can smoothly edit the object attribute.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Image Classification, Model Robustness" />
-        </attvalues>
-      </node>
-      <node id="2402.15820" label="2402.15820">
-        <attvalues>
-          <attvalue for="0" value="DART: Depth-Enhanced Accurate and Real-Time Background Matting" />
-          <attvalue for="1" value="  Matting with a static background, often referred to as ``Background Matting&quot;&#10;(BGM), has garnered significant attention within the computer vision community&#10;due to its pivotal role in various practical applications like webcasting and&#10;photo editing. Nevertheless, achieving highly accurate background matting&#10;remains a formidable challenge, primarily owing to the limitations inherent in&#10;conventional RGB images. These limitations manifest in the form of&#10;susceptibility to varying lighting conditions and unforeseen shadows.&#10;  In this paper, we leverage the rich depth information provided by the&#10;RGB-Depth (RGB-D) cameras to enhance background matting performance in&#10;real-time, dubbed DART. Firstly, we adapt the original RGB-based BGM algorithm&#10;to incorporate depth information. The resulting model's output undergoes&#10;refinement through Bayesian inference, incorporating a background depth prior.&#10;The posterior prediction is then translated into a &quot;trimap,&quot; which is&#10;subsequently fed into a state-of-the-art matting algorithm to generate more&#10;precise alpha mattes. To ensure real-time matting capabilities, a critical&#10;requirement for many real-world applications, we distill the backbone of our&#10;model from a larger and more versatile BGM network. Our experiments demonstrate&#10;the superior performance of the proposed method. Moreover, thanks to the&#10;distillation operation, our method achieves a remarkable processing speed of 33&#10;frames per second (fps) on a mid-range edge-computing device. This high&#10;efficiency underscores DART's immense potential for deployment in mobile&#10;applications}&#10;" />
-          <attvalue for="2" value="&#10;&#10;I{mage} Image matting is a well-established problem in computer vision, with applications spanning image and video editing, video conferencing, and more. Researchers have dedicated substantial efforts to realizing automatic and high-quality image matting under real-world conditions, resulting in advancements such as deep learning-based approaches \cite{shen2016deep, zhu2017fast, chen2018semantic, liu2020boosting, sengupta2020background, li2022bridging, ke2022modnet, sun2022human, dai2022boosting, ma2023rethinking, li2023deep}. A significant challenge in automatic image matting lies in the requirement of a ``trimap,&quot; a crucial input for conventional matting algorithms. Without a proper trimap, the matting problem becomes ill-posed, as distinguishing between the ``foreground&quot; and ``background&quot; can be highly ambiguous \cite{li2023deep, xu2017deep, fang2022user, sun2023ultrahigh, park2022matteformer, liu2021long, zheng2022image, liu2021tripartite}. To address this issue, pioneering work such as the &quot;Background Matting&quot; (BGM) algorithm \cite{sengupta2020background, BGMv2} was introduced. BGM conducts image matting against a fixed background, ensuring a well-defined foreground that can be accurately extracted even without human-labeled trimaps. However, challenges persist, especially when dealing with shadows introduced by foreground objects or unexpected lighting variations. &#10;&#10;In this paper, we present an innovative approach that leverages depth information obtained from a standard RGB-D camera to significantly improve the accuracy and robustness of image matting with static backgrounds. This novel utilization of depth data addresses some of the limitations inherent in traditional methods like BGM (Background Matting) and enhances the precision and reliability of matting results. We provide a schematic overview of our proposed method in Figure~\ref{fig:concept}. Our approach builds upon the foundation of the BGMv$2$ algorithm \cite{BGMv2} but introduces several key modifications to incorporate depth information. We refer to this method as ``Depth-enhanced Accurate and Real-Time BackGround Matting,&quot; or simply ``DART&quot; for brevity. Specifically, we utilize depth information to correct the ``error map&quot; estimated by the base network of the original BGM model \cite{BGMv2} and refine the ``trimap&quot; used in the post-matting process. Additionally, we replace the use of traditional RGB patches with RGB-D patches in the refining network \cite{BGMv2}. Furthermore, we introduce the concept of distillation, where we derive a smaller base network \cite{hinton2015distilling} from the original ResNet$50$-based model used in BGMv$2$. These modifications collectively result in significantly improved matting performance compared to state-of-the-art methods, all while maintaining exceptional computational efficiency. Our DART algorithm can achieve a remarkable speed of $125$ FPS on an affordable desktop GPU and still maintains a respectable $33$ FPS on a mid-range edge-computing platform.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Depth Imaging, Machine Learning, Engineering, Computer Vision, Mathematics, Background Matting, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2304.04672" label="2304.04672">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Matting: A Comprehensive Survey" />
-          <attvalue for="1" value="  Image matting refers to extracting precise alpha matte from natural images,&#10;and it plays a critical role in various downstream applications, such as image&#10;editing. Despite being an ill-posed problem, traditional methods have been&#10;trying to solve it for decades. The emergence of deep learning has&#10;revolutionized the field of image matting and given birth to multiple new&#10;techniques, including automatic, interactive, and referring image matting. This&#10;paper presents a comprehensive review of recent advancements in image matting&#10;in the era of deep learning. We focus on two fundamental sub-tasks: auxiliary&#10;input-based image matting, which involves user-defined input to predict the&#10;alpha matte, and automatic image matting, which generates results without any&#10;manual intervention. We systematically review the existing methods for these&#10;two tasks according to their task settings and network structures and provide a&#10;summary of their advantages and disadvantages. Furthermore, we introduce the&#10;commonly used image matting datasets and evaluate the performance of&#10;representative matting methods both quantitatively and qualitatively. Finally,&#10;we discuss relevant applications of image matting and highlight existing&#10;challenges and potential opportunities for future research. We also maintain a&#10;public repository to track the rapid development of deep image matting at&#10;https://github.com/JizhiziLi/matting-survey.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00626" label="2004.00626">
-        <attvalues>
-          <attvalue for="0" value="Background Matting: The World is Your Green Screen" />
-          <attvalue for="1" value="  We propose a method for creating a matte -- the per-pixel foreground color&#10;and alpha -- of a person by taking photos or videos in an everyday setting with&#10;a handheld camera. Most existing matting methods require a green screen&#10;background or a manually created trimap to produce a good matte. Automatic,&#10;trimap-free methods are appearing, but are not of comparable quality. In our&#10;trimap free approach, we ask the user to take an additional photo of the&#10;background without the subject at the time of capture. This step requires a&#10;small amount of foresight but is far less time-consuming than creating a&#10;trimap. We train a deep network with an adversarial loss to predict the matte.&#10;We first train a matting network with supervised loss on ground truth data with&#10;synthetic composites. To bridge the domain gap to real imagery with no&#10;labeling, we train another matting network guided by the first network and by a&#10;discriminator that judges the quality of composites. We demonstrate results on&#10;a wide variety of photos and videos and show significant improvement over the&#10;state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.07810" label="2012.07810">
-        <attvalues>
-          <attvalue for="0" value="Real-Time High-Resolution Background Matting" />
-          <attvalue for="1" value="  We introduce a real-time, high-resolution background replacement technique&#10;which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our&#10;technique is based on background matting, where an additional frame of the&#10;background is captured and used in recovering the alpha matte and the&#10;foreground layer. The main challenge is to compute a high-quality alpha matte,&#10;preserving strand-level hair details, while processing high-resolution images&#10;in real-time. To achieve this goal, we employ two neural networks; a base&#10;network computes a low-resolution result which is refined by a second network&#10;operating at high-resolution on selective patches. We introduce two largescale&#10;video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our&#10;approach yields higher quality results compared to the previous&#10;state-of-the-art in background matting, while simultaneously yielding a&#10;dramatic boost in both speed and resolution.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.02531" label="1503.02531">
-        <attvalues>
-          <attvalue for="0" value="Distilling the Knowledge in a Neural Network" />
-          <attvalue for="1" value="  A very simple way to improve the performance of almost any machine learning&#10;algorithm is to train many different models on the same data and then to&#10;average their predictions. Unfortunately, making predictions using a whole&#10;ensemble of models is cumbersome and may be too computationally expensive to&#10;allow deployment to a large number of users, especially if the individual&#10;models are large neural nets. Caruana and his collaborators have shown that it&#10;is possible to compress the knowledge in an ensemble into a single model which&#10;is much easier to deploy and we develop this approach further using a different&#10;compression technique. We achieve some surprising results on MNIST and we show&#10;that we can significantly improve the acoustic model of a heavily used&#10;commercial system by distilling the knowledge in an ensemble of models into a&#10;single model. We also introduce a new type of ensemble composed of one or more&#10;full models and many specialist models which learn to distinguish fine-grained&#10;classes that the full models confuse. Unlike a mixture of experts, these&#10;specialist models can be trained rapidly and in parallel.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.08357" label="2401.08357">
-        <attvalues>
-          <attvalue for="0" value="SAMF: Small-Area-Aware Multi-focus Image Fusion for Object Detection" />
-          <attvalue for="1" value="  Existing multi-focus image fusion (MFIF) methods often fail to preserve the&#10;uncertain transition region and detect small focus areas within large defocused&#10;regions accurately. To address this issue, this study proposes a new&#10;small-area-aware MFIF algorithm for enhancing object detection capability.&#10;First, we enhance the pixel attributes within the small focus and boundary&#10;regions, which are subsequently combined with visual saliency detection to&#10;obtain the pre-fusion results used to discriminate the distribution of focused&#10;pixels. To accurately ensure pixel focus, we consider the source image as a&#10;combination of focused, defocused, and uncertain regions and propose a&#10;three-region segmentation strategy. Finally, we design an effective pixel&#10;selection rule to generate segmentation decision maps and obtain the final&#10;fusion results. Experiments demonstrated that the proposed method can&#10;accurately detect small and smooth focus areas while improving object detection&#10;performance, outperforming existing methods in both subjective and objective&#10;evaluations. The source code is available at https://github.com/ixilai/SAMF.&#10;" />
-          <attvalue for="2" value="&#10;The limited depth of field of optical lenses makes it difficult for optical imaging devices to satisfy the need for clear imaging of all objects in a scene, resulting in partially focused images. For some important practical applications, such as object detection, microscope imaging, and intelligent surveillance, all objects in an image must be in focus. This is achieved using a multi-focus image fusion (MFIF) technique \cite{R1,R3} that can fully extract focusing information from each source image. &#10;&#10;Existing image fusion algorithms can be divided into two main categories: deep learning (DL)-based methods \cite{R4,R5,R27} and traditional methods \cite{R7,R8,R9}. DL-based algorithms can be further categorized into decision map-based \cite{R4,R5,R10} and end-to-end-based \cite{R11,R12,R13} methods. The former utilize a decision map to determine the focusing attributes of pixels, allowing direct utilization of pixel information from the source image in the process of fusing multi-focus images, and the latter generate fusion images directly, without the need for postprocessing steps. The traditional methods can be further categorized into multi-scale transform (MST)-based \cite{R3,R14} and spatial domain-based \cite{R15,R16,R17} methods. MST-based methods mainly perform multi-scale decomposition of the source image, obtain a series of sub-bands representing different information from the source image, and finally design the fusion rules to integrate this information. Spatial domain-based algorithms construct a focusing decision map for fusion by detecting the saliency of pixels.&#10;&#10;In summary, although existing methods can achieve desirable fusion performance, they may fail in recognizing small and complex focus regions, which are particularly important for object detection tasks. In real-world autonomous driving scenarios, it is difficult to capture the complete scene focus information within different depths of field in a single image. The goal of MFIF is to integrate the focus information from all depths of field into a unified image. However, owing to the complexity of road conditions, it is difficult for existing MFIF algorithms to extract object information accurately from compact background regions, such as pedestrians located between two vehicles; consequently, automobiles can fail to detect pedestrians, as shown in Fig. 1. To address this problem, we propose a small-area-aware MFIF algorithm (SAMF) for object detection. This study makes the following contributions: (1) We propose an innovative small-region-aware MFIF methods designed to address optical imaging constraints and improve object detection capability. (2) We propose a new three-region segmentation strategy that divides an image into focused, defocused, and uncertain regions. (3) We construct a real-world MFIF dataset (Road-MF) with all the images taken against vehicles on a road. The Road-MF dataset comprises 80 pairs of images, and fused images can be used for object detection tasks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Image Processing, Computer Science, Image Fusion Techniques, Object Detection Methods, Computer Vision Algorithms" />
-        </attvalues>
-      </node>
-      <node id="2103.09943" label="2103.09943">
-        <attvalues>
-          <attvalue for="0" value="Fast and High-Quality Blind Multi-Spectral Image Pansharpening" />
-          <attvalue for="1" value="  Blind pansharpening addresses the problem of generating a high&#10;spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution&#10;multi-spectral (LRMS) image with the guidance of its associated spatially&#10;misaligned high spatial-resolution panchromatic (PAN) image without parametric&#10;side information. In this paper, we propose a fast approach to blind&#10;pansharpening and achieve state-of-the-art image reconstruction quality.&#10;Typical blind pansharpening algorithms are often computationally intensive&#10;since the blur kernel and the target HRMS image are often computed using&#10;iterative solvers and in an alternating fashion. To achieve fast blind&#10;pansharpening, we decouple the solution of the blur kernel and of the HRMS&#10;image. First, we estimate the blur kernel by computing the kernel coefficients&#10;with minimum total generalized variation that blur a downsampled version of the&#10;PAN image to approximate a linear combination of the LRMS image channels. Then,&#10;we estimate each channel of the HRMS image using local Laplacian prior to&#10;regularize the relationship between each HRMS channel and the PAN image.&#10;Solving the HRMS image is accelerated by both parallelizing across the channels&#10;and by fast numerical algorithms for each channel. Due to the fast scheme and&#10;the powerful priors we used on the blur kernel coefficients (total generalized&#10;variation) and on the cross-channel relationship (local Laplacian prior),&#10;numerical experiments demonstrate that our algorithm outperforms&#10;state-of-the-art model-based counterparts in terms of both computational time&#10;and reconstruction quality of the HRMS images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.13985" label="2405.13985">
-        <attvalues>
-          <attvalue for="0" value="LookHere: Vision Transformers with Directed Attention Generalize and&#10;  Extrapolate" />
-          <attvalue for="1" value="  High-resolution images offer more information about scenes that can improve&#10;model accuracy. However, the dominant model architecture in computer vision,&#10;the vision transformer (ViT), cannot effectively leverage larger images without&#10;finetuning -- ViTs poorly extrapolate to more patches at test time, although&#10;transformers offer sequence length flexibility. We attribute this shortcoming&#10;to the current patch position encoding methods, which create a distribution&#10;shift when extrapolating.&#10;  We propose a drop-in replacement for the position encoding of plain ViTs that&#10;restricts attention heads to fixed fields of view, pointed in different&#10;directions, using 2D attention masks. Our novel method, called LookHere,&#10;provides translation-equivariance, ensures attention head diversity, and limits&#10;the distribution shift that attention heads face when extrapolating. We&#10;demonstrate that LookHere improves performance on classification (avg. 1.6%),&#10;against adversarial attack (avg. 5.4%), and decreases calibration error (avg.&#10;1.5%) -- on ImageNet without extrapolation. With extrapolation, LookHere&#10;outperforms the current SoTA position encoding method, 2D-RoPE, by 21.7% on&#10;ImageNet when trained at $224^2$ px and tested at $1024^2$ px. Additionally, we&#10;release a high-resolution test set to improve the evaluation of high-resolution&#10;image classifiers, called ImageNet-HR.&#10;" />
-          <attvalue for="2" value="&#10;&#10;There is a decades-long trend in computer vision towards higher-resolution imagery, which contains more detailed scene information. Increasing resolution is a reliable way to improve model accuracy \cite{fixres, gpipe, efficientnet, threethings, revenge, apple_vlm, monkey, vstar}, but this comes at a cost; training models for hundreds of epochs on large-scale datasets is expensive, especially at high-resolutions. There are two ways to reduce this cost and still see accuracy benefits from high-resolutions: \circlednum{1} high-resolution finetuning, which pretrains models at a lower resolution, like $224^2$ px, then finetunes them at a higher resolution, like $384^2$ px; and \circlednum{2} extrapolating, which deploys models at a higher resolution, without further training. Of these two options, we should aim for models that can effectively extrapolate, as it presents a zero-cost solution that does not require finetuning at every target resolution. Finetuning costs aside, improvements to extrapolation should benefit high-resolution finetuning since models that are better at extrapolating can adapt to higher resolutions more easily. Although extrapolation is a significant and exciting challenge, state-of-the-art (SoTA) model architectures extrapolate poorly.&#10;&#10;Vision transformers (ViTs \cite{vit}) offer SoTA performance on many computer vision tasks. ViTs are simple; they split images into non-overlapping patches, linearly project pixels to form patch embeddings, and process these ``tokens'' with a stack of architecturally identical transformer layers --- maintaining a constant feature map size throughout. This non-hierarchical design enables learning patch representations, which are useful for dense prediction tasks \cite{segvit, open_vocab, plain_obj} and are fundamental for vision-language models \cite{vlm_1, vlm_2, vlm_3}. The design enables efficient processing of only a subset of patches, known as token dropping \cite{drop_token, mae}. Lastly, it enables model scaling by increasing the embedding size and the layer count \cite{scaling, sovit}.&#10;&#10;Image-size extrapolation with ViTs can be achieved in three ways: \circlednum{1} increasing the patch size, which packs more pixels into each patch embedding; \circlednum{2} increasing the ``patchification'' stride, which skips-over pixels; and \circlednum{3} increasing the number of patches. Of these three options, we should aim for models that can effectively ingest more patches --- called ``sequence length extrapolation'' in the natural language processing (NLP) community \cite{alibi_nlp} --- as a greater number of patches presents models with more (uncompressed) information that we hope to leverage into higher accuracy. Furthermore, methods that improve sequence length extrapolation, like our proposed method, can be fused with methods that adjust patch sizes, like FlexiViT \cite{flexivit}. We strongly believe that patch position encoding is a primary cause of the poor sequence length extrapolation ability of ViTs --- like it is in NLP, where significant advancements have been made by improving position encoding \cite{alibi_nlp, rope_scaling, nope, skip_pose}.&#10;&#10;Adding learnable or fixed sinusoidal position embeddings to patch embeddings before the first layer is the most common way ViTs encode positions. Recently, the rotary position embeddings (RoPE \cite{rope_nlp}) used in SoTA language models \cite{llama, mixtral} were extended to ViTs, as $2$D-RoPE \cite{rope}, showing exciting results. RoPE is a different approach to position encoding that injects positional information in each self-attention layer by rotating queries and keys with fixed sinusoidal embeddings. But for these methods to ingest more patches at test time, they must either introduce new position embeddings or modify existing embeddings --- both options create a significant distribution shift. Motivated by these observations and more, we make the following contributions:&#10;&#10;\circlednum{1} {LookHere} --- We introduce a novel position encoding method for plain ViTs that restricts attention heads to fixed fields of view (FOV) and points them in different directions via $2$D masks. This design provides: \circlednum{a} translation-equivariance, \circlednum{b} attention head diversity, \circlednum{c} improved interpretability, and \circlednum{d} limits the distribution shift that attention heads face when extrapolating.&#10;&#10;\circlednum{2} {Controlled Experiments} --- We perform an apples-to-apples comparison between seven position encoding methods for plain ViTs alongside our three LookHere variants. We demonstrate that LookHere: \circlednum{a} improves classification, segmentation, adversarial robustness, and model calibration when tested at the training resolution; \circlednum{b} significantly improves performance when tested beyond the training resolution; and \circlednum{c} increases its performance advantage after high-resolution finetuning.&#10;&#10;\circlednum{3} {Extrapolation Insights} --- We show that extrapolation: \circlednum{a} benefits images with small objects the most, as they occupy more patches at test time; \circlednum{b} produces class-level and dataset-level effects; and \circlednum{c} creates distribution shifts that can be visualized via attention maps. &#10;&#10;\circlednum{4} {ImageNet-HR} --- We introduce the first natively high-resolution ImageNet test set ($1024^2$ px) aimed to benchmark classifiers on images that were not upsampled to achieve the target image size.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Transformer Models, Mathematics, Artificial Intelligence, Computer Science, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1906.06423" label="1906.06423">
-        <attvalues>
-          <attvalue for="0" value="Fixing the train-test resolution discrepancy" />
-          <attvalue for="1" value="  Data-augmentation is key to the training of neural networks for image&#10;classification. This paper first shows that existing augmentations induce a&#10;significant discrepancy between the typical size of the objects seen by the&#10;classifier at train and test time. We experimentally validate that, for a&#10;target test resolution, using a lower train resolution offers better&#10;classification at test time.&#10;  We then propose a simple yet effective and efficient strategy to optimize the&#10;classifier performance when the train and test resolutions differ. It involves&#10;only a computationally cheap fine-tuning of the network at the test resolution.&#10;This enables training strong classifiers using small training images. For&#10;instance, we obtain 77.1% top-1 accuracy on ImageNet with a ResNet-50 trained&#10;on 128x128 images, and 79.8% with one trained on 224x224 image. In addition, if&#10;we use extra training data we get 82.5% with the ResNet-50 train with 224x224&#10;images.&#10;  Conversely, when training a ResNeXt-101 32x48d pre-trained in&#10;weakly-supervised fashion on 940 million public images at resolution 224x224&#10;and further optimizing for test resolution 320x320, we obtain a test top-1&#10;accuracy of 86.4% (top-5: 98.0%) (single-crop). To the best of our knowledge&#10;this is the highest ImageNet single-crop, top-1 and top-5 accuracy to date.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.11929" label="2010.11929">
-        <attvalues>
-          <attvalue for="0" value="An Image is Worth 16x16 Words: Transformers for Image Recognition at&#10;  Scale" />
-          <attvalue for="1" value="  While the Transformer architecture has become the de-facto standard for&#10;natural language processing tasks, its applications to computer vision remain&#10;limited. In vision, attention is either applied in conjunction with&#10;convolutional networks, or used to replace certain components of convolutional&#10;networks while keeping their overall structure in place. We show that this&#10;reliance on CNNs is not necessary and a pure transformer applied directly to&#10;sequences of image patches can perform very well on image classification tasks.&#10;When pre-trained on large amounts of data and transferred to multiple mid-sized&#10;or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision&#10;Transformer (ViT) attains excellent results compared to state-of-the-art&#10;convolutional networks while requiring substantially fewer computational&#10;resources to train.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.05844" label="2210.05844">
-        <attvalues>
-          <attvalue for="0" value="SegViT: Semantic Segmentation with Plain Vision Transformers" />
-          <attvalue for="1" value="  We explore the capability of plain Vision Transformers (ViTs) for semantic&#10;segmentation and propose the SegVit. Previous ViT-based segmentation networks&#10;usually learn a pixel-level representation from the output of the ViT.&#10;Differently, we make use of the fundamental component -- attention mechanism,&#10;to generate masks for semantic segmentation. Specifically, we propose the&#10;Attention-to-Mask (ATM) module, in which the similarity maps between a set of&#10;learnable class tokens and the spatial feature maps are transferred to the&#10;segmentation masks. Experiments show that our proposed SegVit using the ATM&#10;module outperforms its counterparts using the plain ViT backbone on the ADE20K&#10;dataset and achieves new state-of-the-art performance on COCO-Stuff-10K and&#10;PASCAL-Context datasets. Furthermore, to reduce the computational cost of the&#10;ViT backbone, we propose query-based down-sampling (QD) and query-based&#10;up-sampling (QU) to build a Shrunk structure. With the proposed Shrunk&#10;structure, the model can save up to $40\%$ computations while maintaining&#10;competitive performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.08485" label="2304.08485">
-        <attvalues>
-          <attvalue for="0" value="Visual Instruction Tuning" />
-          <attvalue for="1" value="  Instruction tuning large language models (LLMs) using machine-generated&#10;instruction-following data has improved zero-shot capabilities on new tasks,&#10;but the idea is less explored in the multimodal field. In this paper, we&#10;present the first attempt to use language-only GPT-4 to generate multimodal&#10;language-image instruction-following data. By instruction tuning on such&#10;generated data, we introduce LLaVA: Large Language and Vision Assistant, an&#10;end-to-end trained large multimodal model that connects a vision encoder and&#10;LLM for general-purpose visual and language understanding.Our early experiments&#10;show that LLaVA demonstrates impressive multimodel chat abilities, sometimes&#10;exhibiting the behaviors of multimodal GPT-4 on unseen images/instructions, and&#10;yields a 85.1% relative score compared with GPT-4 on a synthetic multimodal&#10;instruction-following dataset. When fine-tuned on Science QA, the synergy of&#10;LLaVA and GPT-4 achieves a new state-of-the-art accuracy of 92.53%. We make&#10;GPT-4 generated visual instruction tuning data, our model and code base&#10;publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.04560" label="2106.04560">
-        <attvalues>
-          <attvalue for="0" value="Scaling Vision Transformers" />
-          <attvalue for="1" value="  Attention-based neural networks such as the Vision Transformer (ViT) have&#10;recently attained state-of-the-art results on many computer vision benchmarks.&#10;Scale is a primary ingredient in attaining excellent results, therefore,&#10;understanding a model's scaling properties is a key to designing future&#10;generations effectively. While the laws for scaling Transformer language models&#10;have been studied, it is unknown how Vision Transformers scale. To address&#10;this, we scale ViT models and data, both up and down, and characterize the&#10;relationships between error rate, data, and compute. Along the way, we refine&#10;the architecture and training of ViT, reducing memory consumption and&#10;increasing accuracy of the resulting models. As a result, we successfully train&#10;a ViT model with two billion parameters, which attains a new state-of-the-art&#10;on ImageNet of 90.45% top-1 accuracy. The model also performs well for few-shot&#10;transfer, for example, reaching 84.86% top-1 accuracy on ImageNet with only 10&#10;examples per class.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.12409" label="2108.12409">
-        <attvalues>
-          <attvalue for="0" value="Train Short, Test Long: Attention with Linear Biases Enables Input&#10;  Length Extrapolation" />
-          <attvalue for="1" value="  Since the introduction of the transformer model by Vaswani et al. (2017), a&#10;fundamental question has yet to be answered: how does a model achieve&#10;extrapolation at inference time for sequences that are longer than it saw&#10;during training? We first show that extrapolation can be enabled by simply&#10;changing the position representation method, though we find that current&#10;methods do not allow for efficient extrapolation. We therefore introduce a&#10;simpler and more efficient position method, Attention with Linear Biases&#10;(ALiBi). ALiBi does not add positional embeddings to word embeddings; instead,&#10;it biases query-key attention scores with a penalty that is proportional to&#10;their distance. We show that this method trains a 1.3 billion parameter model&#10;on input sequences of length 1024 that extrapolates to input sequences of&#10;length 2048, achieving the same perplexity as a sinusoidal position embedding&#10;model trained on inputs of length 2048 but training 11% faster and using 11%&#10;less memory. ALiBi's inductive bias towards recency also leads it to outperform&#10;multiple strong position methods on the WikiText-103 benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.08013" label="2212.08013">
-        <attvalues>
-          <attvalue for="0" value="FlexiViT: One Model for All Patch Sizes" />
-          <attvalue for="1" value="  Vision Transformers convert images to sequences by slicing them into patches.&#10;The size of these patches controls a speed/accuracy tradeoff, with smaller&#10;patches leading to higher accuracy at greater computational cost, but changing&#10;the patch size typically requires retraining the model. In this paper, we&#10;demonstrate that simply randomizing the patch size at training time leads to a&#10;single set of weights that performs well across a wide range of patch sizes,&#10;making it possible to tailor the model to different compute budgets at&#10;deployment time. We extensively evaluate the resulting model, which we call&#10;FlexiViT, on a wide range of tasks, including classification, image-text&#10;retrieval, open-world detection, panoptic segmentation, and semantic&#10;segmentation, concluding that it usually matches, and sometimes outperforms,&#10;standard ViT models trained at a single patch size in an otherwise identical&#10;setup. Hence, FlexiViT training is a simple drop-in improvement for ViT that&#10;makes it easy to add compute-adaptive capabilities to most models relying on a&#10;ViT backbone architecture. Code and pre-trained models are available at&#10;https://github.com/google-research/big_vision&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.09864" label="2104.09864">
-        <attvalues>
-          <attvalue for="0" value="RoFormer: Enhanced Transformer with Rotary Position Embedding" />
-          <attvalue for="1" value="  Position encoding recently has shown effective in the transformer&#10;architecture. It enables valuable supervision for dependency modeling between&#10;elements at different positions of the sequence. In this paper, we first&#10;investigate various methods to integrate positional information into the&#10;learning process of transformer-based language models. Then, we propose a novel&#10;method named Rotary Position Embedding(RoPE) to effectively leverage the&#10;positional information. Specifically, the proposed RoPE encodes the absolute&#10;position with a rotation matrix and meanwhile incorporates the explicit&#10;relative position dependency in self-attention formulation. Notably, RoPE&#10;enables valuable properties, including the flexibility of sequence length,&#10;decaying inter-token dependency with increasing relative distances, and the&#10;capability of equipping the linear self-attention with relative position&#10;encoding. Finally, we evaluate the enhanced transformer with rotary position&#10;embedding, also called RoFormer, on various long text classification benchmark&#10;datasets. Our experiments show that it consistently overcomes its alternatives.&#10;Furthermore, we provide a theoretical analysis to explain some experimental&#10;results. RoFormer is already integrated into Huggingface:&#10;\url{https://huggingface.co/docs/transformers/model_doc/roformer}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.13971" label="2302.13971">
-        <attvalues>
-          <attvalue for="0" value="LLaMA: Open and Efficient Foundation Language Models" />
-          <attvalue for="1" value="  We introduce LLaMA, a collection of foundation language models ranging from&#10;7B to 65B parameters. We train our models on trillions of tokens, and show that&#10;it is possible to train state-of-the-art models using publicly available&#10;datasets exclusively, without resorting to proprietary and inaccessible&#10;datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks,&#10;and LLaMA-65B is competitive with the best models, Chinchilla-70B and&#10;PaLM-540B. We release all our models to the research community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.10408" label="2211.10408">
-        <attvalues>
-          <attvalue for="0" value="CroCo v2: Improved Cross-view Completion Pre-training for Stereo&#10;  Matching and Optical Flow" />
-          <attvalue for="1" value="  Despite impressive performance for high-level downstream tasks,&#10;self-supervised pre-training methods have not yet fully delivered on dense&#10;geometric vision tasks such as stereo matching or optical flow. The application&#10;of self-supervised concepts, such as instance discrimination or masked image&#10;modeling, to geometric tasks is an active area of research. In this work, we&#10;build on the recent cross-view completion framework, a variation of masked&#10;image modeling that leverages a second view from the same scene which makes it&#10;well suited for binocular downstream tasks. The applicability of this concept&#10;has so far been limited in at least two ways: (a) by the difficulty of&#10;collecting real-world image pairs -- in practice only synthetic data have been&#10;used -- and (b) by the lack of generalization of vanilla transformers to dense&#10;downstream tasks for which relative position is more meaningful than absolute&#10;position. We explore three avenues of improvement. First, we introduce a method&#10;to collect suitable real-world image pairs at large scale. Second, we&#10;experiment with relative positional embeddings and show that they enable vision&#10;transformers to perform substantially better. Third, we scale up vision&#10;transformer based cross-completion architectures, which is made possible by the&#10;use of large amounts of data. With these improvements, we show for the first&#10;time that state-of-the-art results on stereo matching and optical flow can be&#10;reached without using any classical task-specific techniques like correlation&#10;volume, iterative estimation, image warping or multi-scale reasoning, thus&#10;paving the way towards universal vision models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.06734" label="2012.06734">
-        <attvalues>
-          <attvalue for="0" value="PoP-Net: Pose over Parts Network for Multi-Person 3D Pose Estimation&#10;  from a Depth Image" />
-          <attvalue for="1" value="  In this paper, a real-time method called PoP-Net is proposed to predict&#10;multi-person 3D poses from a depth image. PoP-Net learns to predict bottom-up&#10;part representations and top-down global poses in a single shot. Specifically,&#10;a new part-level representation, called Truncated Part Displacement Field&#10;(TPDF), is introduced which enables an explicit fusion process to unify the&#10;advantages of bottom-up part detection and global pose detection. Meanwhile, an&#10;effective mode selection scheme is introduced to automatically resolve the&#10;conflicting cases between global pose and part detections. Finally, due to the&#10;lack of high-quality depth datasets for developing multi-person 3D pose&#10;estimation, we introduce Multi-Person 3D Human Pose Dataset (MP-3DHP) as a new&#10;benchmark. MP-3DHP is designed to enable effective multi-person and background&#10;data augmentation in model training, and to evaluate 3D human pose estimators&#10;under uncontrolled multi-person scenarios. We show that PoP-Net achieves the&#10;state-of-the-art results both on MP-3DHP and on the widely used ITOP dataset,&#10;and has significant advantages in efficiency for multi-person processing. To&#10;demonstrate one of the applications of our algorithm pipeline, we also show&#10;results of virtual avatars driven by our calculated 3D joint positions. MP-3DHP&#10;Dataset and the evaluation code have been made available at:&#10;https://github.com/oppo-us-research/PoP-Net.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Human pose estimation plays an important role in a wide variety of applications, and there is a rich pool of literature for human pose estimation methods. Categorizations of existing methods can be made from different dimensions. There are methods mostly relying on a single image to predict human poses~\cite{Wei:etal:CVPR2016:CPM,DBLP:conf/cvpr/PavlakosZDD17a,Mehta:etal:ACM2017:VNect} and others based on multiple cameras~\cite{DBLP:conf/eccv/RhodinRCRST16,Elhayek:etal:PAMI17}. Some methods are capable of predicting multiple poses~\cite{Cao:etal:cvpr17:openpose,Mehta:etal:XNECT:TOG2020} while others are focusing on single person~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PavlakosZDD17a}. Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}. Methods can also be differentiated by the input, while most methods use RGB images~\cite{Cao:etal:cvpr17:openpose,DBLP:conf/cvpr/PapandreouZKTTB17,Mehta:etal:ACM2017:VNect,BenzineCLPA:CVPR2020}, some use depth maps~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Wang:etal:ACM2016,Xiong:etal:iccv2019:a2j}. Specifically, this paper focuses on multi-person 3D pose estimation from a depth image.&#10;&#10;In the era of deep learning, a large pool of Deep Neural Networks (DNN)-based methods have been developed for multi-person pose estimation. Ideas from existing literature can be generally categorized into three prototypical trends. The simplest idea is to directly extend a single-shot object detector~\cite{Liu:etal:eccv2016:ssd,Redmon:etal:cvpr16:yolo,Redmon:Farhadi:cvpr2017:yolo2} with additional pose attributes, so that the network can output human poses. Such single-shot regression can be very efficient, but has low part accuracy, as shown in Figure~\ref{fig:visual:compare} (Yolo-Pose+), because a long-range inference for part locations is involved in a center-relative pose representation. The second type builds on a two-stage pipeline where the first stage detects object bounding boxes, and the second stage estimates the pose within each~\cite{DBLP:conf/cvpr/PapandreouZKTTB17,He:etal:iccv2017:mask:rcnn,Xiong:etal:iccv2019:a2j}. Two-stage methods can be very accurate, as shown in Figure~\ref{fig:visual:compare} (Yolo-A2J), but not as efficient when more human beings appear in an image. In addition, more sophisticated design is required to solve the compatibility issue between pose estimation and bounding box detection~\cite{He:etal:iccv2017:mask:rcnn,Redmon:etal:cvpr16:yolo}. The third idea is to detect human poses from part detection and association~\cite{DBLP:conf/eccv/IqbalG16,DBLP:conf/nips/NewellHD17,Cao:etal:cvpr17:openpose,Martiez-Gonzalez:etal:iros2018:rpm,Mehta:etal:XNECT:TOG2020}. Although part detection can be rather efficient, solving the part association problem is usually time consuming. OpenPose~\cite{Cao:etal:cvpr17:openpose} gained its popularity for introducing an efficient solution to solve the association, resulting in a network benefiting from both the single-shot pipeline with high efficiency and the part-based dense representation with high positional precision. However, a pure bottom-up method does not have a global sense, so that it is rather sensitive to occlusion, truncation, and ambiguities in symmetric limbs (Figure~\ref{fig:visual:compare} OpenPose+). Moreover, dependency on the bipartite matching in assembling parts presents the part predictions from differentiable integration with global context reasoning, which may systematically block a solution from end-to-end training~\cite{Mehta:etal:XNECT:TOG2020}.&#10;&#10;Developing a depth image-based multi-person 3D pose estimation from a well-established RGB image-based 2D method~\cite{DBLP:conf/cvpr/PapandreouZKTTB17,Ren:etal:pami2017:faster:rcnn,Cao:etal:cvpr17:openpose} is conceptually simple for two reasons: some schemes to handle multi-person detection can be shared; 3D information is partially available from the input~\cite{Haque:etal:eccv2016:itop,Xiong:etal:iccv2019:a2j}. As a result, a depth image-based approach may not necessarily require as many sophisticated designs as methods aiming to estimate 3D poses from a single RGB image~\cite{Mehta:etal:ACM2017:VNect,Mehta:etal:XNECT:TOG2020}. However, RGB-based methods and depth-based methods usually focus on different challenges. While some RGB-based methods spend more effort in differentiating people cluttered in a large scene~\cite{Cao:etal:cvpr17:openpose,BenzineCLPA:CVPR2020}, depth-based methods~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Xiong:etal:iccv2019:a2j} focus more on recovering highly accurate 3D poses of fewer people presented in a closer range. In practice, to develop a depth-based method that robustly predicts 3D poses in a multi-person scenario is very challenging due to noisy depth inputs and heavy occlusions. &#10;&#10;In this paper, we present a method called Pose-over-Parts Network (PoP-Net) to estimate multiple 3D poses from a depth image. As illustrated in Figure~\ref{fig:main:idea}, the main idea of PoP-Net is to explicitly fuse the predicted bottom-up parts and top-down global poses. This fusion process is enabled by a new intermediate representation, called Truncated-Part-Displacement-Field (TPDF), which is a vector field that records the vector pointing to the closest part location at every 2D position. TPDF is utilized to guide a structural valid global pose towards more positionally precise part location so that the advantages of global pose and local part detection can be naturally unified.&#10;&#10;At the same time, we release a comprehensive depth dataset, named Multi-Person 3D Human Pose Dataset (MP-3DHP), to facilitate the development of 3D pose estimation methods generalizable to novel background and unobserved multi-person configurations in real-world applications. Although there are a decent amount of RGB datasets~\cite{DBLP:conf/eccv/LinMBHPRDZ14,DBLP:journals/pami/IonescuPOS14,DBLP:conf/cvpr/AndrilukaPGS14:mpii} in prior art, there are limited high-quality depth datasets. The released dataset is constructed to cover most of the essential aspects of visual variance related to 3D human pose estimation, and particularly to promote the development of multi-person methods. &#10;&#10;The contribution of this paper is four fold. First, we introduce an efficient framework that predicts multiple 3D poses in a single shot. Second, we propose a new part-level representation called TPDF, which enables an explicit fusion of global poses and part-level representations. Third, we introduce a mode selection scheme that automatically resolves the conflicting cases between local and global predictions. Finally, we release a comprehensive depth image-based dataset to facilitate the development of multi-person 3D pose estimation methods applicable to real-world challenges.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1602.00134" label="1602.00134">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Pose Machines" />
-          <attvalue for="1" value="  Pose Machines provide a sequential prediction framework for learning rich&#10;implicit spatial models. In this work we show a systematic design for how&#10;convolutional networks can be incorporated into the pose machine framework for&#10;learning image features and image-dependent spatial models for the task of pose&#10;estimation. The contribution of this paper is to implicitly model long-range&#10;dependencies between variables in structured prediction tasks such as&#10;articulated pose estimation. We achieve this by designing a sequential&#10;architecture composed of convolutional networks that directly operate on belief&#10;maps from previous stages, producing increasingly refined estimates for part&#10;locations, without the need for explicit graphical model-style inference. Our&#10;approach addresses the characteristic difficulty of vanishing gradients during&#10;training by providing a natural learning objective function that enforces&#10;intermediate supervision, thereby replenishing back-propagated gradients and&#10;conditioning the learning procedure. We demonstrate state-of-the-art&#10;performance and outperform competing methods on standard benchmarks including&#10;the MPII, LSP, and FLIC datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07828" label="1611.07828">
-        <attvalues>
-          <attvalue for="0" value="Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose" />
-          <attvalue for="1" value="  This paper addresses the challenge of 3D human pose estimation from a single&#10;color image. Despite the general success of the end-to-end learning paradigm,&#10;top performing approaches employ a two-step solution consisting of a&#10;Convolutional Network (ConvNet) for 2D joint localization and a subsequent&#10;optimization step to recover 3D pose. In this paper, we identify the&#10;representation of 3D pose as a critical issue with current ConvNet approaches&#10;and make two important contributions towards validating the value of end-to-end&#10;learning for this task. First, we propose a fine discretization of the 3D space&#10;around the subject and train a ConvNet to predict per voxel likelihoods for&#10;each joint. This creates a natural representation for 3D pose and greatly&#10;improves performance over the direct regression of joint coordinates. Second,&#10;to further improve upon initial estimates, we employ a coarse-to-fine&#10;prediction scheme. This step addresses the large dimensionality increase and&#10;enables iterative refinement and repeated processing of the image features. The&#10;proposed approach outperforms all state-of-the-art methods on standard&#10;benchmarks achieving a relative error reduction greater than 30% on average.&#10;Additionally, we investigate using our volumetric representation in a related&#10;architecture which is suboptimal compared to our end-to-end approach, but is of&#10;practical interest, since it enables training when no image with corresponding&#10;3D groundtruth is available, and allows us to present compelling results for&#10;in-the-wild images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.01583" label="1705.01583">
-        <attvalues>
-          <attvalue for="0" value="VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera" />
-          <attvalue for="1" value="  We present the first real-time method to capture the full global 3D skeletal&#10;pose of a human in a stable, temporally consistent manner using a single RGB&#10;camera. Our method combines a new convolutional neural network (CNN) based pose&#10;regressor with kinematic skeleton fitting. Our novel fully-convolutional pose&#10;formulation regresses 2D and 3D joint positions jointly in real time and does&#10;not require tightly cropped input frames. A real-time kinematic skeleton&#10;fitting method uses the CNN output to yield temporally stable 3D global pose&#10;reconstructions on the basis of a coherent kinematic skeleton. This makes our&#10;approach the first monocular RGB method usable in real-time applications such&#10;as 3D character control---thus far, the only monocular methods for such&#10;applications employed specialized RGB-D cameras. Our method's accuracy is&#10;quantitatively on par with the best offline 3D monocular RGB pose estimation&#10;methods. Our results are qualitatively comparable to, and sometimes better&#10;than, results from monocular RGB-D approaches, such as the Kinect. However, we&#10;show that our approach is more broadly applicable than RGB-D solutions, i.e. it&#10;works for outdoor scenes, community videos, and low quality commodity RGB&#10;cameras.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1607.08659" label="1607.08659">
-        <attvalues>
-          <attvalue for="0" value="General Automatic Human Shape and Motion Capture Using Volumetric&#10;  Contour Cues" />
-          <attvalue for="1" value="  Markerless motion capture algorithms require a 3D body with properly&#10;personalized skeleton dimension and/or body shape and appearance to&#10;successfully track a person. Unfortunately, many tracking methods consider&#10;model personalization a different problem and use manual or semi-automatic&#10;model initialization, which greatly reduces applicability. In this paper, we&#10;propose a fully automatic algorithm that jointly creates a rigged actor model&#10;commonly used for animation - skeleton, volumetric shape, appearance, and&#10;optionally a body surface - and estimates the actor's motion from multi-view&#10;video input only. The approach is rigorously designed to work on footage of&#10;general outdoor scenes recorded with very few cameras and without background&#10;subtraction. Our method uses a new image formation model with analytic&#10;visibility and analytically differentiable alignment energy. For&#10;reconstruction, 3D body shape is approximated as Gaussian density field. For&#10;pose and shape estimation, we minimize a new edge-based alignment energy&#10;inspired by volume raycasting in an absorbing medium. We further propose a new&#10;statistical human body model that represents the body surface, volumetric&#10;Gaussian density, as well as variability in skeleton shape. Given any&#10;multi-view sequence, our method jointly optimizes the pose and shape parameters&#10;of this model fully automatically in a spatiotemporal way.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.08050" label="1611.08050">
-        <attvalues>
-          <attvalue for="0" value="Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" />
-          <attvalue for="1" value="  We present an approach to efficiently detect the 2D pose of multiple people&#10;in an image. The approach uses a nonparametric representation, which we refer&#10;to as Part Affinity Fields (PAFs), to learn to associate body parts with&#10;individuals in the image. The architecture encodes global context, allowing a&#10;greedy bottom-up parsing step that maintains high accuracy while achieving&#10;realtime performance, irrespective of the number of people in the image. The&#10;architecture is designed to jointly learn part locations and their association&#10;via two branches of the same sequential prediction process. Our method placed&#10;first in the inaugural COCO 2016 keypoints challenge, and significantly exceeds&#10;the previous state-of-the-art result on the MPII Multi-Person benchmark, both&#10;in performance and efficiency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.00837" label="1907.00837">
-        <attvalues>
-          <attvalue for="0" value="XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera" />
-          <attvalue for="1" value="  We present a real-time approach for multi-person 3D motion capture at over 30&#10;fps using a single RGB camera. It operates successfully in generic scenes which&#10;may contain occlusions by objects and by other people. Our method operates in&#10;subsequent stages. The first stage is a convolutional neural network (CNN) that&#10;estimates 2D and 3D pose features along with identity assignments for all&#10;visible joints of all individuals.We contribute a new architecture for this&#10;CNN, called SelecSLS Net, that uses novel selective long and short range skip&#10;connections to improve the information flow allowing for a drastically faster&#10;network without compromising accuracy. In the second stage, a fully connected&#10;neural network turns the possibly partial (on account of occlusion) 2Dpose and&#10;3Dpose features for each subject into a complete 3Dpose estimate per&#10;individual. The third stage applies space-time skeletal model fitting to the&#10;predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose,&#10;and enforce temporal coherence. Our method returns the full skeletal pose in&#10;joint angles for each subject. This is a further key distinction from previous&#10;work that do not produce joint angle results of a coherent skeleton in real&#10;time for multi-person scenes. The proposed system runs on consumer hardware at&#10;a previously unseen speed of more than 30 fps given 512x320 images as input&#10;while achieving state-of-the-art accuracy, which we will demonstrate on a range&#10;of challenging real-world scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.06937" label="1603.06937">
-        <attvalues>
-          <attvalue for="0" value="Stacked Hourglass Networks for Human Pose Estimation" />
-          <attvalue for="1" value="  This work introduces a novel convolutional network architecture for the task&#10;of human pose estimation. Features are processed across all scales and&#10;consolidated to best capture the various spatial relationships associated with&#10;the body. We show how repeated bottom-up, top-down processing used in&#10;conjunction with intermediate supervision is critical to improving the&#10;performance of the network. We refer to the architecture as a &quot;stacked&#10;hourglass&quot; network based on the successive steps of pooling and upsampling that&#10;are done to produce a final set of predictions. State-of-the-art results are&#10;achieved on the FLIC and MPII benchmarks outcompeting all recent methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.05180" label="1605.05180">
-        <attvalues>
-          <attvalue for="0" value="Structured Prediction of 3D Human Pose with Deep Neural Networks" />
-          <attvalue for="1" value="  Most recent approaches to monocular 3D pose estimation rely on Deep Learning.&#10;They either train a Convolutional Neural Network to directly regress from image&#10;to 3D pose, which ignores the dependencies between human joints, or model these&#10;dependencies via a max-margin structured learning framework, which involves a&#10;high computational cost at inference time.&#10;  In this paper, we introduce a Deep Learning regression architecture for&#10;structured prediction of 3D human pose from monocular images that relies on an&#10;overcomplete auto-encoder to learn a high-dimensional latent pose&#10;representation and account for joint dependencies. We demonstrate that our&#10;approach outperforms state-of-the-art ones both in terms of structure&#10;preservation and prediction accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.03098" label="1705.03098">
-        <attvalues>
-          <attvalue for="0" value="A simple yet effective baseline for 3d human pose estimation" />
-          <attvalue for="1" value="  Following the success of deep convolutional networks, state-of-the-art&#10;methods for 3d human pose estimation have focused on deep end-to-end systems&#10;that predict 3d joint locations given raw image pixels. Despite their excellent&#10;performance, it is often not easy to understand whether their remaining error&#10;stems from a limited 2d pose (visual) understanding, or from a failure to map&#10;2d poses into 3-dimensional positions. With the goal of understanding these&#10;sources of error, we set out to build a system that given 2d joint locations&#10;predicts 3d positions. Much to our surprise, we have found that, with current&#10;technology, &quot;lifting&quot; ground truth 2d joint locations to 3d space is a task&#10;that can be solved with a remarkably low error rate: a relatively simple deep&#10;feed-forward network outperforms the best reported result by about 30\% on&#10;Human3.6M, the largest publicly available 3d pose estimation benchmark.&#10;Furthermore, training our system on the output of an off-the-shelf&#10;state-of-the-art 2d detector (\ie, using images as input) yields state of the&#10;art results -- this includes an array of systems that have been trained&#10;end-to-end specifically for this task. Our results indicate that a large&#10;portion of the error of modern deep 3d pose estimation systems stems from their&#10;visual analysis, and suggests directions to further advance the state of the&#10;art in 3d human pose estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04095" label="1805.04095">
-        <attvalues>
-          <attvalue for="0" value="Ordinal Depth Supervision for 3D Human Pose Estimation" />
-          <attvalue for="1" value="  Our ability to train end-to-end systems for 3D human pose estimation from&#10;single images is currently constrained by the limited availability of 3D&#10;annotations for natural images. Most datasets are captured using Motion Capture&#10;(MoCap) systems in a studio setting and it is difficult to reach the&#10;variability of 2D human pose datasets, like MPII or LSP. To alleviate the need&#10;for accurate 3D ground truth, we propose to use a weaker supervision signal&#10;provided by the ordinal depths of human joints. This information can be&#10;acquired by human annotators for a wide range of images and poses. We showcase&#10;the effectiveness and flexibility of training Convolutional Networks (ConvNets)&#10;with these ordinal relations in different settings, always achieving&#10;competitive performance with ConvNets trained with accurate 3D joint&#10;coordinates. Additionally, to demonstrate the potential of the approach, we&#10;augment the popular LSP and MPII datasets with ordinal depth annotations. This&#10;extension allows us to present quantitative and qualitative evaluation in&#10;non-studio conditions. Simultaneously, these ordinal annotations can be easily&#10;incorporated in the training procedure of typical ConvNets for 3D human pose.&#10;Through this inclusion we achieve new state-of-the-art performance for the&#10;relevant benchmarks and validate the effectiveness of ordinal depth supervision&#10;for 3D human pose.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09999" label="1908.09999">
-        <attvalues>
-          <attvalue for="0" value="A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose&#10;  Estimation from a Single Depth Image" />
-          <attvalue for="1" value="  For 3D hand and body pose estimation task in depth image, a novel&#10;anchor-based approach termed Anchor-to-Joint regression network (A2J) with the&#10;end-to-end learning ability is proposed. Within A2J, anchor points able to&#10;capture global-local spatial context information are densely set on depth image&#10;as local regressors for the joints. They contribute to predict the positions of&#10;the joints in ensemble way to enhance generalization ability. The proposed 3D&#10;articulated pose estimation paradigm is different from the state-of-the-art&#10;encoder-decoder based FCN, 3D CNN and point-set based manners. To discover&#10;informative anchor points towards certain joint, anchor proposal procedure is&#10;also proposed for A2J. Meanwhile 2D CNN (i.e., ResNet-50) is used as backbone&#10;network to drive A2J, without using time-consuming 3D convolutional or&#10;deconvolutional layers. The experiments on 3 hand datasets and 2 body datasets&#10;verify A2J's superiority. Meanwhile, A2J is of high running speed around 100&#10;FPS on single NVIDIA 1080Ti GPU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.01779" label="1701.01779">
-        <attvalues>
-          <attvalue for="0" value="Towards Accurate Multi-person Pose Estimation in the Wild" />
-          <attvalue for="1" value="  We propose a method for multi-person detection and 2-D pose estimation that&#10;achieves state-of-art results on the challenging COCO keypoints task. It is a&#10;simple, yet powerful, top-down approach consisting of two stages.&#10;  In the first stage, we predict the location and scale of boxes which are&#10;likely to contain people; for this we use the Faster RCNN detector. In the&#10;second stage, we estimate the keypoints of the person potentially contained in&#10;each proposed bounding box. For each keypoint type we predict dense heatmaps&#10;and offsets using a fully convolutional ResNet. To combine these outputs we&#10;introduce a novel aggregation procedure to obtain highly localized keypoint&#10;predictions. We also use a novel form of keypoint-based Non-Maximum-Suppression&#10;(NMS), instead of the cruder box-level NMS, and a novel form of keypoint-based&#10;confidence score estimation, instead of box-level scoring.&#10;  Trained on COCO data alone, our final system achieves average precision of&#10;0.649 on the COCO test-dev set and the 0.643 test-standard sets, outperforming&#10;the winner of the 2016 COCO keypoints challenge and other recent state-of-art.&#10;Further, by using additional in-house labeled data we obtain an even higher&#10;average precision of 0.685 on the test-dev set and 0.673 on the test-standard&#10;set, more than 5% absolute improvement compared to the previous best performing&#10;method on the same dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.13911" label="1910.13911">
-        <attvalues>
-          <attvalue for="0" value="Real-time Convolutional Networks for Depth-based Human Pose Estimation" />
-          <attvalue for="1" value="  We propose to combine recent Convolutional Neural Networks (CNN) models with&#10;depth imaging to obtain a reliable and fast multi-person pose estimation&#10;algorithm applicable to Human Robot Interaction (HRI) scenarios. Our hypothesis&#10;is that depth images contain less structures and are easier to process than RGB&#10;images while keeping the required information for human detection and pose&#10;inference, thus allowing the use of simpler networks for the task. Our&#10;contributions are threefold. (i) we propose a fast and efficient network based&#10;on residual blocks (called RPM) for body landmark localization from depth&#10;images; (ii) we created a public dataset DIH comprising more than 170k&#10;synthetic images of human bodies with various shapes and viewpoints as well as&#10;real (annotated) data for evaluation; (iii) we show that our model trained on&#10;synthetic data from scratch can perform well on real data, obtaining similar&#10;results to larger models initialized with pre-trained networks. It thus&#10;provides a good trade-off between performance and computation. Experiments on&#10;real data demonstrate the validity of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.03932" label="1608.03932">
-        <attvalues>
-          <attvalue for="0" value="Human Pose Estimation from Depth Images via Inference Embedded&#10;  Multi-task Learning" />
-          <attvalue for="1" value="  Human pose estimation (i.e., locating the body parts / joints of a person) is&#10;a fundamental problem in human-computer interaction and multimedia&#10;applications. Significant progress has been made based on the development of&#10;depth sensors, i.e., accessible human pose prediction from still depth images&#10;[32]. However, most of the existing approaches to this problem involve several&#10;components/models that are independently designed and optimized, leading to&#10;suboptimal performances. In this paper, we propose a novel inference-embedded&#10;multi-task learning framework for predicting human pose from still depth&#10;images, which is implemented with a deep architecture of neural networks.&#10;Specifically, we handle two cascaded tasks: i) generating the heat (confidence)&#10;maps of body parts via a fully convolutional network (FCN); ii) seeking the&#10;optimal configuration of body parts based on the detected body part proposals&#10;via an inference built-in MatchNet [10], which measures the appearance and&#10;geometric kinematic compatibility of body parts and embodies the dynamic&#10;programming inference as an extra network layer. These two tasks are jointly&#10;optimized. Our extensive experiments show that the proposed deep model&#10;significantly improves the accuracy of human pose estimation over other several&#10;state-of-the-art methods or SDKs. We also release a large-scale dataset for&#10;comparison, which includes 100K depth images under challenging scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.02325" label="1512.02325">
-        <attvalues>
-          <attvalue for="0" value="SSD: Single Shot MultiBox Detector" />
-          <attvalue for="1" value="  We present a method for detecting objects in images using a single deep&#10;neural network. Our approach, named SSD, discretizes the output space of&#10;bounding boxes into a set of default boxes over different aspect ratios and&#10;scales per feature map location. At prediction time, the network generates&#10;scores for the presence of each object category in each default box and&#10;produces adjustments to the box to better match the object shape. Additionally,&#10;the network combines predictions from multiple feature maps with different&#10;resolutions to naturally handle objects of various sizes. Our SSD model is&#10;simple relative to methods that require object proposals because it completely&#10;eliminates proposal generation and subsequent pixel or feature resampling stage&#10;and encapsulates all computation in a single network. This makes SSD easy to&#10;train and straightforward to integrate into systems that require a detection&#10;component. Experimental results on the PASCAL VOC, MS COCO, and ILSVRC datasets&#10;confirm that SSD has comparable accuracy to methods that utilize an additional&#10;object proposal step and is much faster, while providing a unified framework&#10;for both training and inference. Compared to other single stage methods, SSD&#10;has much better accuracy, even with a smaller input image size. For $300\times&#10;300$ input, SSD achieves 72.1% mAP on VOC2007 test at 58 FPS on a Nvidia Titan&#10;X and for $500\times 500$ input, SSD achieves 75.1% mAP, outperforming a&#10;comparable state of the art Faster R-CNN model. Code is available at&#10;https://github.com/weiliu89/caffe/tree/ssd .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.08242" label="1612.08242">
-        <attvalues>
-          <attvalue for="0" value="YOLO9000: Better, Faster, Stronger" />
-          <attvalue for="1" value="  We introduce YOLO9000, a state-of-the-art, real-time object detection system&#10;that can detect over 9000 object categories. First we propose various&#10;improvements to the YOLO detection method, both novel and drawn from prior&#10;work. The improved model, YOLOv2, is state-of-the-art on standard detection&#10;tasks like PASCAL VOC and COCO. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At&#10;40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like&#10;Faster RCNN with ResNet and SSD while still running significantly faster.&#10;Finally we propose a method to jointly train on object detection and&#10;classification. Using this method we train YOLO9000 simultaneously on the COCO&#10;detection dataset and the ImageNet classification dataset. Our joint training&#10;allows YOLO9000 to predict detections for object classes that don't have&#10;labelled detection data. We validate our approach on the ImageNet detection&#10;task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite&#10;only having detection data for 44 of the 200 classes. On the 156 classes not in&#10;COCO, YOLO9000 gets 16.0 mAP. But YOLO can detect more than just 200 classes;&#10;it predicts detections for more than 9000 different object categories. And it&#10;still runs in real-time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.06870" label="1703.06870">
-        <attvalues>
-          <attvalue for="0" value="Mask R-CNN" />
-          <attvalue for="1" value="  We present a conceptually simple, flexible, and general framework for object&#10;instance segmentation. Our approach efficiently detects objects in an image&#10;while simultaneously generating a high-quality segmentation mask for each&#10;instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a&#10;branch for predicting an object mask in parallel with the existing branch for&#10;bounding box recognition. Mask R-CNN is simple to train and adds only a small&#10;overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to&#10;generalize to other tasks, e.g., allowing us to estimate human poses in the&#10;same framework. We show top results in all three tracks of the COCO suite of&#10;challenges, including instance segmentation, bounding-box object detection, and&#10;person keypoint detection. Without bells and whistles, Mask R-CNN outperforms&#10;all existing, single-model entries on every task, including the COCO 2016&#10;challenge winners. We hope our simple and effective approach will serve as a&#10;solid baseline and help ease future research in instance-level recognition.&#10;Code has been made available at: https://github.com/facebookresearch/Detectron&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.08526" label="1608.08526">
-        <attvalues>
-          <attvalue for="0" value="Multi-Person Pose Estimation with Local Joint-to-Person Associations" />
-          <attvalue for="1" value="  Despite of the recent success of neural networks for human pose estimation,&#10;current approaches are limited to pose estimation of a single person and cannot&#10;handle humans in groups or crowds. In this work, we propose a method that&#10;estimates the poses of multiple persons in an image in which a person can be&#10;occluded by another person or might be truncated. To this end, we consider&#10;multi-person pose estimation as a joint-to-person association problem. We&#10;construct a fully connected graph from a set of detected joint candidates in an&#10;image and resolve the joint-to-person association and outlier detection using&#10;integer linear programming. Since solving joint-to-person association jointly&#10;for all persons in an image is an NP-hard problem and even approximations are&#10;expensive, we solve the problem locally for each person. On the challenging&#10;MPII Human Pose Dataset for multiple persons, our approach achieves the&#10;accuracy of a state-of-the-art method, but it is 6,000 to 19,000 times faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.05424" label="1611.05424">
-        <attvalues>
-          <attvalue for="0" value="Associative Embedding: End-to-End Learning for Joint Detection and&#10;  Grouping" />
-          <attvalue for="1" value="  We introduce associative embedding, a novel method for supervising&#10;convolutional neural networks for the task of detection and grouping. A number&#10;of computer vision problems can be framed in this manner including multi-person&#10;pose estimation, instance segmentation, and multi-object tracking. Usually the&#10;grouping of detections is achieved with multi-stage pipelines, instead we&#10;propose an approach that teaches a network to simultaneously output detections&#10;and group assignments. This technique can be easily integrated into any&#10;state-of-the-art network architecture that produces pixel-wise predictions. We&#10;show how to apply this method to both multi-person pose estimation and instance&#10;segmentation and report state-of-the-art performance for multi-person pose on&#10;the MPII and MS-COCO datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.07076" label="1603.07076">
-        <attvalues>
-          <attvalue for="0" value="Towards Viewpoint Invariant 3D Human Pose Estimation" />
-          <attvalue for="1" value="  We propose a viewpoint invariant model for 3D human pose estimation from a&#10;single depth image. To achieve this, our discriminative model embeds local&#10;regions into a learned viewpoint invariant feature space. Formulated as a&#10;multi-task learning problem, our model is able to selectively predict partial&#10;poses in the presence of noise and occlusion. Our approach leverages a&#10;convolutional and recurrent network architecture with a top-down error feedback&#10;mechanism to self-correct previous pose estimates in an end-to-end manner. We&#10;evaluate our model on a previously published depth dataset and a newly&#10;collected human pose dataset containing 100K annotated depth images from&#10;extreme viewpoints. Experiments show that our model achieves competitive&#10;performance on frontal views while achieving state-of-the-art performance on&#10;alternate viewpoints.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.12238" label="2010.12238">
-        <attvalues>
-          <attvalue for="0" value="Hard Example Generation by Texture Synthesis for Cross-domain Shape&#10;  Similarity Learning" />
-          <attvalue for="1" value="  Image-based 3D shape retrieval (IBSR) aims to find the corresponding 3D shape&#10;of a given 2D image from a large 3D shape database. The common routine is to&#10;map 2D images and 3D shapes into an embedding space and define (or learn) a&#10;shape similarity measure. While metric learning with some adaptation techniques&#10;seems to be a natural solution to shape similarity learning, the performance is&#10;often unsatisfactory for fine-grained shape retrieval. In the paper, we&#10;identify the source of the poor performance and propose a practical solution to&#10;this problem. We find that the shape difference between a negative pair is&#10;entangled with the texture gap, making metric learning ineffective in pushing&#10;away negative pairs. To tackle this issue, we develop a geometry-focused&#10;multi-view metric learning framework empowered by texture synthesis. The&#10;synthesis of textures for 3D shape models creates hard triplets, which suppress&#10;the adverse effects of rich texture in 2D images, thereby push the network to&#10;focus more on discovering geometric characteristics. Our approach shows&#10;state-of-the-art performance on a recently released large-scale 3D-FUTURE[1]&#10;repository, as well as three widely studied benchmarks, including Pix3D[2],&#10;Stanford Cars[3], and Comp Cars[4]. Codes will be made publicly available at:&#10;https://github.com/3D-FRONT-FUTURE/IBSR-texture&#10;" />
-          <attvalue for="2" value="&#10;Cross-domain image-based 3D shape retrieval (IBSR) is to identify the matched 3D CAD model of the object contained in a query image. With the growing number of 3D shapes, the establishment of reliable IBSR platforms is significant. For example, it can help to build 3D virtual scenes for real-world houses from 2D images. High-performing IBSR systems may also inspire and benefit studies in shape collection based 3D object reconstruction. Unluckily, compared to content-based image retrieval \cite{wu2018unsupervised,babenko2014neural,radenovic2018fine,liu2016deep,caron2018deep}, IBSR is challenging due to the large appearance gap between 3D shapes and 2D images. &#10;&#10;There are mainly two directions to alleviate the issue, as shown in Fig.~\ref{fig:motivation}. For the first one, researchers take efforts to predict 2.5D sketches from images, such as surface normal, depth, and location filed, to bridge the gaps between 3D and 2D domains \cite{sun2018pix3d,huang2018holistic,wu2017marrnet,bansal2016marr,bachman1978data,grabner20183d,grabner2019location}. These methods achieve state-of-the-art (SOTA) retrieval accuracy and sequences. However, while 2.5D representations for 3D CAD models can be conveniently rendered, accurately estimating 2.5D sketches from 2D images is itself an open problem \cite{eigen2015predicting,wang2015designing,chen2016single,qi2018geonet,fu2018deep,wang2016surge}. &#10;The second routine is to map cross-domain representations into a unified constrained embedding space and define (or learn) a shape similarity measure \cite{li2015joint,aubry2014seeing,lee2018cross,massa2016deep,tasse2016shape2vec,girdhar2016learning,dalal2005histograms,izadinia2017im2cad,tulsiani2018factoring,aubry2015understanding,lin2018learning,3dor.20171048}. Standard metric learning with feature adaptation techniques seems to be a natural solution to shape similarity learning. However, their performance is often unsatisfactory for fine-grained shape retrieval. One of the possible reasons is that the shape difference between a negative pair is entangled with the texture gap, making metric learning ineffective in pushing away negative pairs. &#10;&#10;As the analysis, assuming that we have many hard triplets like $(x, x^{+}, x^{-})$, where the positive example $x^{+}$ and the anchor image $x$ are identical in shape, but differ in texture, the negative example $x^{-}$ and $x$ are similar in texture, but differ in shape. The metric learning procedure thus should suppress the adverse impact of texture in 2D images, thereby push the network to focus more on discovering geometric characteristics. Towards the goal, we develop a geometry-focused multi-view metric learning framework empowered by texture synthesis. The synthesis of textures for 3D models creates hard triplets. More specifically, we learn a conditional generative adversarial network that can assign a texture to a 3D model based on a texture code encoded from an example image. Then, we can generate hard samples in an online manner to improve the cross-domain shape similarity learning, as shown in Fig~\ref{fig:motivation}. Note that, in contrast to category-driven metric learning for image retrieval, each 3D shape is an individual instance (category). It's not easy to apply hard sample mining strategies in image retrieval to IBSR. We are the first to consider texture information to generate hard triplets based on the special properties of the IBSR subject.&#10;&#10;Furthermore, the geometry-focused multi-view metric learning framework also contains: (1) a mask attention mechanism to enhance the feature from saliency regions, and (2) a simple but effective viewpoint attention policy to guide the multi-view embedding aggregation. With these guidances, we find that the network can easily focus more on the learning of shape characteristics. Our approach shows state-of-the-art performance on a recently released large-scale 3D-FUTURE \cite{fu20203d} repository, as well as three widely studied benchmarks, including Pix3D \cite{sun2018pix3d}, Stanford Cars \cite{KrauseStarkDengFei-Fei_3DRR2013}, and Comp Cars \cite{wang20183d}. We also conduct various ablation studies to comprehensively discuss our method. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Computer Science, Metric Learning Techniques, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1404.1777" label="1404.1777">
-        <attvalues>
-          <attvalue for="0" value="Neural Codes for Image Retrieval" />
-          <attvalue for="1" value="  It has been shown that the activations invoked by an image within the top&#10;layers of a large convolutional neural network provide a high-level descriptor&#10;of the visual content of the image. In this paper, we investigate the use of&#10;such descriptors (neural codes) within the image retrieval application. In the&#10;experiments with several standard retrieval benchmarks, we establish that&#10;neural codes perform competitively even when the convolutional neural network&#10;has been trained for an unrelated classification task (e.g.\ Image-Net). We&#10;also evaluate the improvement in the retrieval performance of neural codes,&#10;when the network is retrained on a dataset of images that are similar to images&#10;encountered at test time.&#10;  We further evaluate the performance of the compressed neural codes and show&#10;that a simple PCA compression provides very good short codes that give&#10;state-of-the-art accuracy on a number of datasets. In general, neural codes&#10;turn out to be much more resilient to such compression in comparison other&#10;state-of-the-art descriptors. Finally, we show that discriminative&#10;dimensionality reduction trained on a dataset of pairs of matched photographs&#10;improves the performance of PCA-compressed neural codes even further. Overall,&#10;our quantitative experiments demonstrate the promise of neural codes as visual&#10;descriptors for image retrieval.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.02512" label="1711.02512">
-        <attvalues>
-          <attvalue for="0" value="Fine-tuning CNN Image Retrieval with No Human Annotation" />
-          <attvalue for="1" value="  Image descriptors based on activations of Convolutional Neural Networks&#10;(CNNs) have become dominant in image retrieval due to their discriminative&#10;power, compactness of representation, and search efficiency. Training of CNNs,&#10;either from scratch or fine-tuning, requires a large amount of annotated data,&#10;where a high quality of annotation is often crucial. In this work, we propose&#10;to fine-tune CNNs for image retrieval on a large collection of unordered images&#10;in a fully automated manner. Reconstructed 3D models obtained by the&#10;state-of-the-art retrieval and structure-from-motion methods guide the&#10;selection of the training data. We show that both hard-positive and&#10;hard-negative examples, selected by exploiting the geometry and the camera&#10;positions available from the 3D models, enhance the performance of&#10;particular-object retrieval. CNN descriptor whitening discriminatively learned&#10;from the same training data outperforms commonly used PCA whitening. We propose&#10;a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and&#10;average pooling and show that it boosts retrieval performance. Applying the&#10;proposed method to the VGG network achieves state-of-the-art performance on the&#10;standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.05520" label="1807.05520">
-        <attvalues>
-          <attvalue for="0" value="Deep Clustering for Unsupervised Learning of Visual Features" />
-          <attvalue for="1" value="  Clustering is a class of unsupervised learning methods that has been&#10;extensively applied and studied in computer vision. Little work has been done&#10;to adapt it to the end-to-end training of visual features on large scale&#10;datasets. In this work, we present DeepCluster, a clustering method that&#10;jointly learns the parameters of a neural network and the cluster assignments&#10;of the resulting features. DeepCluster iteratively groups the features with a&#10;standard clustering algorithm, k-means, and uses the subsequent assignments as&#10;supervision to update the weights of the network. We apply DeepCluster to the&#10;unsupervised training of convolutional neural networks on large datasets like&#10;ImageNet and YFCC100M. The resulting model outperforms the current state of the&#10;art by a significant margin on all the standard benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.04610" label="1804.04610">
-        <attvalues>
-          <attvalue for="0" value="Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling" />
-          <attvalue for="1" value="  We study 3D shape modeling from a single image and make contributions to it&#10;in three aspects. First, we present Pix3D, a large-scale benchmark of diverse&#10;image-shape pairs with pixel-level 2D-3D alignment. Pix3D has wide applications&#10;in shape-related tasks including reconstruction, retrieval, viewpoint&#10;estimation, etc. Building such a large-scale dataset, however, is highly&#10;challenging; existing datasets either contain only synthetic data, or lack&#10;precise alignment between 2D images and 3D shapes, or only have a small number&#10;of images. Second, we calibrate the evaluation criteria for 3D shape&#10;reconstruction through behavioral studies, and use them to objectively and&#10;systematically benchmark cutting-edge reconstruction algorithms on Pix3D.&#10;Third, we design a novel model that simultaneously performs 3D reconstruction&#10;and pose estimation; our multi-task learning approach achieves state-of-the-art&#10;performance on both tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.02201" label="1808.02201">
-        <attvalues>
-          <attvalue for="0" value="Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image" />
-          <attvalue for="1" value="  We propose a computational framework to jointly parse a single RGB image and&#10;reconstruct a holistic 3D configuration composed by a set of CAD models using a&#10;stochastic grammar model. Specifically, we introduce a Holistic Scene Grammar&#10;(HSG) to represent the 3D scene structure, which characterizes a joint&#10;distribution over the functional and geometric space of indoor scenes. The&#10;proposed HSG captures three essential and often latent dimensions of the indoor&#10;scenes: i) latent human context, describing the affordance and the&#10;functionality of a room arrangement, ii) geometric constraints over the scene&#10;configurations, and iii) physical constraints that guarantee physically&#10;plausible parsing and reconstruction. We solve this joint parsing and&#10;reconstruction problem in an analysis-by-synthesis fashion, seeking to minimize&#10;the differences between the input image and the rendered images generated by&#10;our 3D representation, over the space of depth, surface normal, and object&#10;segmentation map. The optimal configuration, represented by a parse graph, is&#10;inferred using Markov chain Monte Carlo (MCMC), which efficiently traverses&#10;through the non-differentiable solution space, jointly optimizing object&#10;localization, 3D layout, and hidden human context. Experimental results&#10;demonstrate that the proposed algorithm improves the generalization ability and&#10;significantly outperforms prior methods on 3D layout estimation, 3D object&#10;detection, and holistic scene understanding.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.01347" label="1604.01347">
-        <attvalues>
-          <attvalue for="0" value="Marr Revisited: 2D-3D Alignment via Surface Normal Prediction" />
-          <attvalue for="1" value="  We introduce an approach that leverages surface normal predictions, along&#10;with appearance cues, to retrieve 3D models for objects depicted in 2D still&#10;images from a large CAD object library. Critical to the success of our approach&#10;is the ability to recover accurate surface normals for objects in the depicted&#10;scene. We introduce a skip-network model built on the pre-trained Oxford VGG&#10;convolutional neural network (CNN) for surface normal prediction. Our model&#10;achieves state-of-the-art accuracy on the NYUv2 RGB-D dataset for surface&#10;normal prediction, and recovers fine object detail compared to previous&#10;methods. Furthermore, we develop a two-stream network over the input image and&#10;predicted surface normals that jointly learns pose and style for CAD model&#10;retrieval. When using the predicted surface normals, our two-stream network&#10;matches prior work using surface normals computed from RGB-D images on the task&#10;of pose prediction, and achieves state of the art when using RGB-D input.&#10;Finally, our two-stream network allows us to retrieve CAD models that better&#10;match the style and pose of a depicted object compared with baseline&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.11493" label="1803.11493">
-        <attvalues>
-          <attvalue for="0" value="3D Pose Estimation and 3D Model Retrieval for Objects in the Wild" />
-          <attvalue for="1" value="  We propose a scalable, efficient and accurate approach to retrieve 3D models&#10;for objects in the wild. Our contribution is twofold. We first present a 3D&#10;pose estimation approach for object categories which significantly outperforms&#10;the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior&#10;to retrieve 3D models which accurately represent the geometry of objects in RGB&#10;images. For this purpose, we render depth images from 3D models under our&#10;predicted pose and match learned image descriptors of RGB images against those&#10;of rendered depth images using a CNN-based multi-view metric learning approach.&#10;In this way, we are the first to report quantitative results for 3D model&#10;retrieval on Pascal3D+, where our method chooses the same models as human&#10;annotators for 50% of the validation images on average. In addition, we show&#10;that our method, which was trained purely on Pascal3D+, retrieves rich and&#10;accurate 3D models from ShapeNet given RGB images of objects in the wild.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.02853" label="1908.02853">
-        <attvalues>
-          <attvalue for="0" value="Location Field Descriptors: Single Image 3D Model Retrieval in the Wild" />
-          <attvalue for="1" value="  We present Location Field Descriptors, a novel approach for single image 3D&#10;model retrieval in the wild. In contrast to previous methods that directly map&#10;3D models and RGB images to an embedding space, we establish a common low-level&#10;representation in the form of location fields from which we compute pose&#10;invariant 3D shape descriptors. Location fields encode correspondences between&#10;2D pixels and 3D surface coordinates and, thus, explicitly capture 3D shape and&#10;3D pose information without appearance variations which are irrelevant for the&#10;task. This early fusion of 3D models and RGB images results in three main&#10;advantages: First, the bottleneck location field prediction acts as a&#10;regularizer during training. Second, major parts of the system benefit from&#10;training on a virtually infinite amount of synthetic data. Finally, the&#10;predicted location fields are visually interpretable and unblackbox the system.&#10;We evaluate our proposed approach on three challenging real-world datasets&#10;(Pix3D, Comp, and Stanford) with different object categories and significantly&#10;outperform the state-of-the-art by up to 20% absolute in multiple 3D retrieval&#10;metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.4734" label="1411.4734">
-        <attvalues>
-          <attvalue for="0" value="Predicting Depth, Surface Normals and Semantic Labels with a Common&#10;  Multi-Scale Convolutional Architecture" />
-          <attvalue for="1" value="  In this paper we address three different computer vision tasks using a single&#10;basic architecture: depth prediction, surface normal estimation, and semantic&#10;labeling. We use a multiscale convolutional network that is able to adapt&#10;easily to each task using only small modifications, regressing from the input&#10;image to the output map directly. Our method progressively refines predictions&#10;using a sequence of scales, and captures many image details without any&#10;superpixels or low-level segmentation. We achieve state-of-the-art performance&#10;on benchmarks for all three tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.4958" label="1411.4958">
-        <attvalues>
-          <attvalue for="0" value="Designing Deep Networks for Surface Normal Estimation" />
-          <attvalue for="1" value="  In the past few years, convolutional neural nets (CNN) have shown incredible&#10;promise for learning visual representations. In this paper, we use CNNs for the&#10;task of predicting surface normals from a single image. But what is the right&#10;architecture we should use? We propose to build upon the decades of hard work&#10;in 3D scene understanding, to design new CNN architecture for the task of&#10;surface normal estimation. We show by incorporating several constraints&#10;(man-made, manhattan world) and meaningful intermediate representations (room&#10;layout, edge labels) in the architecture leads to state of the art performance&#10;on surface normal estimation. We also show that our network is quite robust and&#10;show state of the art results on other datasets as well without any&#10;fine-tuning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.03901" label="1604.03901">
-        <attvalues>
-          <attvalue for="0" value="Single-Image Depth Perception in the Wild" />
-          <attvalue for="1" value="  This paper studies single-image depth perception in the wild, i.e.,&#10;recovering depth from a single image taken in unconstrained settings. We&#10;introduce a new dataset &quot;Depth in the Wild&quot; consisting of images in the wild&#10;annotated with relative depth between pairs of random points. We also propose a&#10;new algorithm that learns to estimate metric depth using annotations of&#10;relative depth. Compared to the state of the art, our algorithm is simpler and&#10;performs better. Experiments show that our algorithm, combined with existing&#10;RGB-D data and our new relative depth annotations, significantly improves&#10;single-image depth perception in the wild.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.02446" label="1806.02446">
-        <attvalues>
-          <attvalue for="0" value="Deep Ordinal Regression Network for Monocular Depth Estimation" />
-          <attvalue for="1" value="  Monocular depth estimation, which plays a crucial role in understanding 3D&#10;scene geometry, is an ill-posed problem. Recent methods have gained significant&#10;improvement by exploring image-level information and hierarchical features from&#10;deep convolutional neural networks (DCNNs). These methods model depth&#10;estimation as a regression problem and train the regression networks by&#10;minimizing mean squared error, which suffers from slow convergence and&#10;unsatisfactory local solutions. Besides, existing depth estimation networks&#10;employ repeated spatial pooling operations, resulting in undesirable&#10;low-resolution feature maps. To obtain high-resolution depth maps,&#10;skip-connections or multi-layer deconvolution networks are required, which&#10;complicates network training and consumes much more computations. To eliminate&#10;or at least largely reduce these problems, we introduce a spacing-increasing&#10;discretization (SID) strategy to discretize depth and recast depth network&#10;learning as an ordinal regression problem. By training the network using an&#10;ordinary regression loss, our method achieves much higher accuracy and&#10;\dd{faster convergence in synch}. Furthermore, we adopt a multi-scale network&#10;structure which avoids unnecessary spatial pooling and captures multi-scale&#10;information in parallel.&#10;  The method described in this paper achieves state-of-the-art results on four&#10;challenging benchmarks, i.e., KITTI [17], ScanNet [9], Make3D [50], and NYU&#10;Depth v2 [42], and win the 1st prize in Robust Vision Challenge 2018. Code has&#10;been made available at: https://github.com/hufu6371/DORN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.04314" label="1806.04314">
-        <attvalues>
-          <attvalue for="0" value="3D Pose Estimation for Fine-Grained Object Categories" />
-          <attvalue for="1" value="  Existing object pose estimation datasets are related to generic object types&#10;and there is so far no dataset for fine-grained object categories. In this&#10;work, we introduce a new large dataset to benchmark pose estimation for&#10;fine-grained objects, thanks to the availability of both 2D and 3D fine-grained&#10;data recently. Specifically, we augment two popular fine-grained recognition&#10;datasets (StanfordCars and CompCars) by finding a fine-grained 3D CAD model for&#10;each sub-category and manually annotating each object in images with 3D pose.&#10;We show that, with enough training data, a full perspective model with&#10;continuous parameters can be estimated using 2D appearance information alone.&#10;We achieve this via a framework based on Faster/Mask R-CNN. This goes beyond&#10;previous works on category-level pose estimation, which only estimate&#10;discrete/continuous viewpoint angles or recover rotation matrices often with&#10;the help of key points. Furthermore, with fine-grained 3D models available, we&#10;incorporate a dense 3D representation named as location field into the&#10;CNN-based pose estimation framework to further improve the performance. The new&#10;dataset is available at www.umiacs.umd.edu/~wym/3dpose.html&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.14335" label="2403.14335">
-        <attvalues>
-          <attvalue for="0" value="FFT-based Selection and Optimization of Statistics for Robust&#10;  Recognition of Severely Corrupted Images" />
-          <attvalue for="1" value="  Improving model robustness in case of corrupted images is among the key&#10;challenges to enable robust vision systems on smart devices, such as robotic&#10;agents. Particularly, robust test-time performance is imperative for most of&#10;the applications. This paper presents a novel approach to improve robustness of&#10;any classification model, especially on severely corrupted images. Our method&#10;(FROST) employs high-frequency features to detect input image corruption type,&#10;and select layer-wise feature normalization statistics. FROST provides the&#10;state-of-the-art results for different models and datasets, outperforming&#10;competitors on ImageNet-C by up to 37.1% relative gain, improving baseline of&#10;40.9% mCE on severe corruptions.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Achieving robustness of object recognition models on corrupted images is a challenging problem, which has been studied extensively in recent years \cite{hendrycks2019robustness,yucel2023hybridaugment,lim2023ttn,Chen_2021_ICCV,long2022frequency}. While the models have achieved impressive results on several benchmarks \cite{russakovsky2015imagenet, Krizhevsky2009LearningML}, recent works show that their performance is severely degraded when dealing with corrupted images. Addressing the robustness issues is important to ease the adoption of the models in practical applications on devices used in the wild, where such corruptions are frequently experienced.&#10;We tackle this problem considering a collection of image distortions that are commonly observed in real-world natural images.&#10;Object recognition using distorted image datasets can be posed as a dataset shift problem \cite{Sun2017ImprovingRO}.&#10;Although there has been substantial prior work investigating this problem, it is far from being fully understood, let alone solved. The most successful approaches make use of data augmentation \cite{wang2021augmax,autoaugment,Hendrycks2021PixMixDP,hendrycks2020augmix} and adversarial training \cite{bai2021transformers,long2022frequency,yucel2023hybridaugment}.&#10;&#10;Some methods focus on resolving shift of data statistics to improve model robustness, developing modules on top of feature normalization layers \cite{lim2023ttn,Schneider2020ImprovingRA,Khurana2021SITASI}.&#10;Recent studies leveraged frequency spectra insights to improve model robustness \cite{Chen_2021_ICCV,yucel2023hybridaugment,long2022frequency}. &#10;However, these methods often use cumbersome ensemble models \cite{Saikia_2021_ICCV} or formulate complex augmentation regimes \cite{Sun2022ASV,long2022frequency} at training time, with no possibility to adapt the models to test data. In addition, it is important to preserve, if not improve the accuracy of the models on clean samples while enhancing its robustness.&#10;&#10;We improve robustness of models against corrupted images by selecting layer-wise feature normalization statistics depending on the corruption via high-frequency features. &#10;At test-time, we determine the corruption present in an input image, by processing FFT features and matching them to corruption-specific prototypical features (prototypes). &#10;Then, we use codebooks mapping prototypes to corruption-specific statistics provided by normalization layers of models, to adapt the models to input image corruption. Our main contributions can be summarized as follows:&#10;(1) we propose a novel test-time \underline{F}FT-based \underline{RO}bust \underline{ST}atistics selection method (FROST) based on a codebook mapping FFT features to corruption-specific statistics;&#10;(2)&#09;we achieve model robustness to image corruptions at test-time, making it useful for mobile and robotic applications in the wild;&#10;(3)&#09;we improve accuracy against competitors on ImageNet-C \cite{hendrycks2019robustness}, especially in case of severe corruptions, while preserving accuracy on clean data;&#10;(4)&#09;our solution is portable to different models and architectures, optimizing statistics of feature normalization layers (Batch/Layer Normalization), with very limited storage requirements. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Image Classification, Model Robustness" />
-        </attvalues>
-      </node>
-      <node id="1903.12261" label="1903.12261">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking Neural Network Robustness to Common Corruptions and&#10;  Perturbations" />
-          <attvalue for="1" value="  In this paper we establish rigorous benchmarks for image classifier&#10;robustness. Our first benchmark, ImageNet-C, standardizes and expands the&#10;corruption robustness topic, while showing which classifiers are preferable in&#10;safety-critical applications. Then we propose a new dataset called ImageNet-P&#10;which enables researchers to benchmark a classifier's robustness to common&#10;perturbations. Unlike recent robustness research, this benchmark evaluates&#10;performance on common corruptions and perturbations not worst-case adversarial&#10;perturbations. We find that there are negligible changes in relative corruption&#10;robustness from AlexNet classifiers to ResNet classifiers. Afterward we&#10;discover ways to enhance corruption and perturbation robustness. We even find&#10;that a bypassed adversarial defense provides substantial common perturbation&#10;robustness. Together our benchmarks may aid future work toward networks that&#10;robustly generalize.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.11823" label="2307.11823">
-        <attvalues>
-          <attvalue for="0" value="HybridAugment++: Unified Frequency Spectra Perturbations for Model&#10;  Robustness" />
-          <attvalue for="1" value="  Convolutional Neural Networks (CNN) are known to exhibit poor generalization&#10;performance under distribution shifts. Their generalization have been studied&#10;extensively, and one line of work approaches the problem from a&#10;frequency-centric perspective. These studies highlight the fact that humans and&#10;CNNs might focus on different frequency components of an image. First, inspired&#10;by these observations, we propose a simple yet effective data augmentation&#10;method HybridAugment that reduces the reliance of CNNs on high-frequency&#10;components, and thus improves their robustness while keeping their clean&#10;accuracy high. Second, we propose HybridAugment++, which is a hierarchical&#10;augmentation method that attempts to unify various frequency-spectrum&#10;augmentations. HybridAugment++ builds on HybridAugment, and also reduces the&#10;reliance of CNNs on the amplitude component of images, and promotes phase&#10;information instead. This unification results in competitive to or better than&#10;state-of-the-art results on clean accuracy (CIFAR-10/100 and ImageNet),&#10;corruption benchmarks (ImageNet-C, CIFAR-10-C and CIFAR-100-C), adversarial&#10;robustness on CIFAR-10 and out-of-distribution detection on various datasets.&#10;HybridAugment and HybridAugment++ are implemented in a few lines of code, does&#10;not require extra data, ensemble models or additional networks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.05155" label="2302.05155">
-        <attvalues>
-          <attvalue for="0" value="TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation" />
-          <attvalue for="1" value="  This paper proposes a novel batch normalization strategy for test-time&#10;adaptation. Recent test-time adaptation methods heavily rely on the modified&#10;batch normalization, i.e., transductive batch normalization (TBN), which&#10;calculates the mean and the variance from the current test batch rather than&#10;using the running mean and variance obtained from the source data, i.e.,&#10;conventional batch normalization (CBN). Adopting TBN that employs test batch&#10;statistics mitigates the performance degradation caused by the domain shift.&#10;However, re-estimating normalization statistics using test data depends on&#10;impractical assumptions that a test batch should be large enough and be drawn&#10;from i.i.d. stream, and we observed that the previous methods with TBN show&#10;critical performance drop without the assumptions. In this paper, we identify&#10;that CBN and TBN are in a trade-off relationship and present a new test-time&#10;normalization (TTN) method that interpolates the statistics by adjusting the&#10;importance between CBN and TBN according to the domain-shift sensitivity of&#10;each BN layer. Our proposed TTN improves model robustness to shifted domains&#10;across a wide range of batch sizes and in various realistic evaluation&#10;scenarios. TTN is widely applicable to other test-time adaptation methods that&#10;rely on updating model parameters via backpropagation. We demonstrate that&#10;adopting TTN further improves their performance and achieves state-of-the-art&#10;performance in various standard benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.08487" label="2108.08487">
-        <attvalues>
-          <attvalue for="0" value="Amplitude-Phase Recombination: Rethinking Robustness of Convolutional&#10;  Neural Networks in Frequency Domain" />
-          <attvalue for="1" value="  Recently, the generalization behavior of Convolutional Neural Networks (CNN)&#10;is gradually transparent through explanation techniques with the frequency&#10;components decomposition. However, the importance of the phase spectrum of the&#10;image for a robust vision system is still ignored. In this paper, we notice&#10;that the CNN tends to converge at the local optimum which is closely related to&#10;the high-frequency components of the training images, while the amplitude&#10;spectrum is easily disturbed such as noises or common corruptions. In contrast,&#10;more empirical studies found that humans rely on more phase components to&#10;achieve robust recognition. This observation leads to more explanations of the&#10;CNN's generalization behaviors in both robustness to common perturbations and&#10;out-of-distribution detection, and motivates a new perspective on data&#10;augmentation designed by re-combing the phase spectrum of the current image and&#10;the amplitude spectrum of the distracter image. That is, the generated samples&#10;force the CNN to pay more attention to the structured information from phase&#10;components and keep robust to the variation of the amplitude. Experiments on&#10;several image datasets indicate that the proposed method achieves&#10;state-of-the-art performances on multiple generalizations and calibration&#10;tasks, including adaptability for common corruptions and surface variations,&#10;out-of-distribution detection, and adversarial attack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.05382" label="2207.05382">
-        <attvalues>
-          <attvalue for="0" value="Frequency Domain Model Augmentation for Adversarial Attack" />
-          <attvalue for="1" value="  For black-box attacks, the gap between the substitute model and the victim&#10;model is usually large, which manifests as a weak attack performance. Motivated&#10;by the observation that the transferability of adversarial examples can be&#10;improved by attacking diverse models simultaneously, model augmentation methods&#10;which simulate different models by using transformed images are proposed.&#10;However, existing transformations for spatial domain do not translate to&#10;significantly diverse augmented models. To tackle this issue, we propose a&#10;novel spectrum simulation attack to craft more transferable adversarial&#10;examples against both normally trained and defense models. Specifically, we&#10;apply a spectrum transformation to the input and thus perform the model&#10;augmentation in the frequency domain. We theoretically prove that the&#10;transformation derived from frequency domain leads to a diverse spectrum&#10;saliency map, an indicator we proposed to reflect the diversity of substitute&#10;models. Notably, our method can be generally combined with existing attacks.&#10;Extensive experiments on the ImageNet dataset demonstrate the effectiveness of&#10;our method, \textit{e.g.}, attacking nine state-of-the-art defense models with&#10;an average success rate of \textbf{95.4\%}. Our code is available in&#10;\url{https://github.com/yuyang-long/SSA}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1409.0575" label="1409.0575">
-        <attvalues>
-          <attvalue for="0" value="ImageNet Large Scale Visual Recognition Challenge" />
-          <attvalue for="1" value="  The ImageNet Large Scale Visual Recognition Challenge is a benchmark in&#10;object category classification and detection on hundreds of object categories&#10;and millions of images. The challenge has been run annually from 2010 to&#10;present, attracting participation from more than fifty institutions.&#10;  This paper describes the creation of this benchmark dataset and the advances&#10;in object recognition that have been possible as a result. We discuss the&#10;challenges of collecting large-scale ground truth annotation, highlight key&#10;breakthroughs in categorical object recognition, provide a detailed analysis of&#10;the current state of the field of large-scale image classification and object&#10;detection, and compare the state-of-the-art computer vision accuracy with human&#10;accuracy. We conclude with lessons learned in the five years of the challenge,&#10;and propose future directions and improvements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.07830" label="1707.07830">
-        <attvalues>
-          <attvalue for="0" value="Improving Robustness of Feature Representations to Image Deformations&#10;  using Powered Convolution in CNNs" />
-          <attvalue for="1" value="  In this work, we address the problem of improvement of robustness of feature&#10;representations learned using convolutional neural networks (CNNs) to image&#10;deformation. We argue that higher moment statistics of feature distributions&#10;could be shifted due to image deformations, and the shift leads to degrade of&#10;performance and cannot be reduced by ordinary normalization methods as observed&#10;in experimental analyses. In order to attenuate this effect, we apply&#10;additional non-linearity in CNNs by combining power functions with learnable&#10;parameters into convolution operation. In the experiments, we observe that CNNs&#10;which employ the proposed method obtain remarkable boost in both the&#10;generalization performance and the robustness under various types of&#10;deformations using large scale benchmark datasets. For instance, a model&#10;equipped with the proposed method obtains 3.3\% performance boost in mAP on&#10;Pascal Voc object detection task using deformed images, compared to the&#10;reference model, while both models provide the same performance using original&#10;images. To the best of our knowledge, this is the first work that studies&#10;robustness of deep features learned using CNNs to a wide range of deformations&#10;for object recognition and detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.13771" label="2110.13771">
-        <attvalues>
-          <attvalue for="0" value="AugMax: Adversarial Composition of Random Augmentations for Robust&#10;  Training" />
-          <attvalue for="1" value="  Data augmentation is a simple yet effective way to improve the robustness of&#10;deep neural networks (DNNs). Diversity and hardness are two complementary&#10;dimensions of data augmentation to achieve robustness. For example, AugMix&#10;explores random compositions of a diverse set of augmentations to enhance&#10;broader coverage, while adversarial training generates adversarially hard&#10;samples to spot the weakness. Motivated by this, we propose a data augmentation&#10;framework, termed AugMax, to unify the two aspects of diversity and hardness.&#10;AugMax first randomly samples multiple augmentation operators and then learns&#10;an adversarial mixture of the selected operators. Being a stronger form of data&#10;augmentation, AugMax leads to a significantly augmented input distribution&#10;which makes model training more challenging. To solve this problem, we further&#10;design a disentangled normalization module, termed DuBIN&#10;(Dual-Batch-and-Instance Normalization), that disentangles the instance-wise&#10;feature heterogeneity arising from AugMax. Experiments show that AugMax-DuBIN&#10;leads to significantly improved out-of-distribution robustness, outperforming&#10;prior arts by 3.03%, 3.49%, 1.82% and 0.71% on CIFAR10-C, CIFAR100-C, Tiny&#10;ImageNet-C and ImageNet-C. Codes and pretrained models are available:&#10;https://github.com/VITA-Group/AugMax.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.05135" label="2112.05135">
-        <attvalues>
-          <attvalue for="0" value="PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures" />
-          <attvalue for="1" value="  In real-world applications of machine learning, reliable and safe systems&#10;must consider measures of performance beyond standard test set accuracy. These&#10;other goals include out-of-distribution (OOD) robustness, prediction&#10;consistency, resilience to adversaries, calibrated uncertainty estimates, and&#10;the ability to detect anomalous inputs. However, improving performance towards&#10;these goals is often a balancing act that today's methods cannot achieve&#10;without sacrificing performance on other safety axes. For instance, adversarial&#10;training improves adversarial robustness but sharply degrades other classifier&#10;performance metrics. Similarly, strong data augmentation and regularization&#10;techniques often improve OOD robustness but harm anomaly detection, raising the&#10;question of whether a Pareto improvement on all existing safety measures is&#10;possible. To meet this challenge, we design a new data augmentation strategy&#10;utilizing the natural structural complexity of pictures such as fractals, which&#10;outperforms numerous baselines, is near Pareto-optimal, and roundly improves&#10;safety measures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.02781" label="1912.02781">
-        <attvalues>
-          <attvalue for="0" value="AugMix: A Simple Data Processing Method to Improve Robustness and&#10;  Uncertainty" />
-          <attvalue for="1" value="  Modern deep neural networks can achieve high accuracy when the training&#10;distribution and test distribution are identically distributed, but this&#10;assumption is frequently violated in practice. When the train and test&#10;distributions are mismatched, accuracy can plummet. Currently there are few&#10;techniques that improve robustness to unforeseen data shifts encountered during&#10;deployment. In this work, we propose a technique to improve the robustness and&#10;uncertainty estimates of image classifiers. We propose AugMix, a data&#10;processing technique that is simple to implement, adds limited computational&#10;overhead, and helps models withstand unforeseen corruptions. AugMix&#10;significantly improves robustness and uncertainty measures on challenging image&#10;classification benchmarks, closing the gap between previous methods and the&#10;best possible performance in some cases by more than half.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.05464" label="2111.05464">
-        <attvalues>
-          <attvalue for="0" value="Are Transformers More Robust Than CNNs?" />
-          <attvalue for="1" value="  Transformer emerges as a powerful tool for visual recognition. In addition to&#10;demonstrating competitive performance on a broad range of visual benchmarks,&#10;recent works also argue that Transformers are much more robust than&#10;Convolutions Neural Networks (CNNs). Nonetheless, surprisingly, we find these&#10;conclusions are drawn from unfair experimental settings, where Transformers and&#10;CNNs are compared at different scales and are applied with distinct training&#10;frameworks. In this paper, we aim to provide the first fair &amp; in-depth&#10;comparisons between Transformers and CNNs, focusing on robustness evaluations.&#10;  With our unified training setup, we first challenge the previous belief that&#10;Transformers outshine CNNs when measuring adversarial robustness. More&#10;surprisingly, we find CNNs can easily be as robust as Transformers on defending&#10;against adversarial attacks, if they properly adopt Transformers' training&#10;recipes. While regarding generalization on out-of-distribution samples, we show&#10;pre-training on (external) large-scale datasets is not a fundamental request&#10;for enabling Transformers to achieve better performance than CNNs. Moreover,&#10;our ablations suggest such stronger generalization is largely benefited by the&#10;Transformer's self-attention-like architectures per se, rather than by other&#10;training setups. We hope this work can help the community better understand and&#10;benchmark the robustness of Transformers and CNNs. The code and models are&#10;publicly available at https://github.com/ytongbai/ViTs-vs-CNNs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.16971" label="2006.16971">
-        <attvalues>
-          <attvalue for="0" value="Improving robustness against common corruptions by covariate shift&#10;  adaptation" />
-          <attvalue for="1" value="  Today's state-of-the-art machine vision models are vulnerable to image&#10;corruptions like blurring or compression artefacts, limiting their performance&#10;in many real-world applications. We here argue that popular benchmarks to&#10;measure model robustness against common corruptions (like ImageNet-C)&#10;underestimate model robustness in many (but not all) application scenarios. The&#10;key insight is that in many scenarios, multiple unlabeled examples of the&#10;corruptions are available and can be used for unsupervised online adaptation.&#10;Replacing the activation statistics estimated by batch normalization on the&#10;training set with the statistics of the corrupted images consistently improves&#10;the robustness across 25 different popular computer vision models. Using the&#10;corrected statistics, ResNet-50 reaches 62.2% mCE on ImageNet-C compared to&#10;76.7% without adaptation. With the more robust DeepAugment+AugMix model, we&#10;improve the state of the art achieved by a ResNet50 model up to date from 53.6%&#10;mCE to 45.4% mCE. Even adapting to a single sample improves robustness for the&#10;ResNet-50 and AugMix models, and 32 samples are sufficient to improve the&#10;current state of the art for a ResNet-50 architecture. We argue that results&#10;with adapted statistics should be included whenever reporting scores in&#10;corruption benchmarks and other out-of-distribution generalization settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The IN-C benchmark~\cite{hendrycks2018benchmarking} has been extended to MNIST~\cite{mu2019mnist}, several object detection datasets~\cite{michaelis2019benchmarking} and image segmentation~\cite{Kamann2019BenchmarkingTR} reflecting the interest of the robustness community.&#10;Most proposals for improving robustness involve special training protocols, requiring time and additional resources.&#10;This includes data augmentation like Gaussian noise~\cite{ford2019adversarial}, optimized mixtures of data augmentations in conjunction with a consistency loss~\cite{hendrycks2019augmix}, training on stylized images~\cite{geirhos2018imagenettrained, michaelis2019benchmarking, Mikoajczyk2018DataAF} or against adversarial noise distributions~\cite{Rusak2020IncreasingTR}.&#10;Other approaches tweak the architecture, e.g. by adding shift-equivariance with an anti-aliasing module,~\cite{zhang2019making} or assemble different training techniques~\cite{lee2020compounding}.&#10;&#10;Unsupervised domain adaptation (DA) is a form of transductive inference where additional information about the test dataset is used to adapt a model to the test distribution.&#10;Adapting feature statistics was proposed by~\cite{sun2017correlation} and follow up work evaluated the performance of adapting BN parameters in unsupervised \cite{DBLP:journals/corr/LiWSLH16, cariucci2017autodial} and supervised DA settings \cite{schneider2018multi}.&#10;As an application example in medical imaging,~\cite{bug2017context} show that adaptive normalization is useful for removing domain shifts on histopathological data.&#10;More involved methods for DA include self-supervised domain adaptation on single examples \cite{sun2019test} and pseudo-labeling \cite{french2017self}.&#10;\cite{xie2020self} achieve the state of the art on IN-C with pseudo-labeling.&#10;In work concurrent to ours, \cite{wang2020fully} also show BN adaptation results on IN-C. They also perform experiments on CIFAR10-C and CIFAR100-C and explore other domain adaptation techniques.&#10;&#10;Robustness scores obtained by adversarial training can be improved when separate BN or GroupNorm layers are used for clean and adversarial images \cite{xie2020intriguing}.&#10;The expressive power of adapting only affine BN parameters BN parameters was shown in multi-task \cite{rebuffi2017learning} and DA contexts \cite{schneider2018multi} and holds even for fine-tuning randomly initialized ResNets \cite{frankle2020training}. &#10;Concurrent work shows additional evidence that BN adaptation yields increased performance on ImageNet-C~\cite{nado2020evaluating}. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2112.02355" label="2112.02355">
-        <attvalues>
-          <attvalue for="0" value="SITA: Single Image Test-time Adaptation" />
-          <attvalue for="1" value="  In Test-time Adaptation (TTA), given a source model, the goal is to adapt it&#10;to make better predictions for test instances from a different distribution&#10;than the source. Crucially, TTA assumes no access to the source data or even&#10;any additional labeled/unlabeled samples from the target distribution to&#10;finetune the source model. In this work, we consider TTA in a more pragmatic&#10;setting which we refer to as SITA (Single Image Test-time Adaptation). Here,&#10;when making a prediction, the model has access only to the given single test&#10;instance, rather than a batch of instances, as typically been considered in the&#10;literature. This is motivated by the realistic scenarios where inference is&#10;needed on-demand instead of delaying for an incoming batch or the inference is&#10;happening on an edge device (like mobile phone) where there is no scope for&#10;batching. The entire adaptation process in SITA should be extremely fast as it&#10;happens at inference time. To address this, we propose a novel approach AugBN&#10;that requires only a single forward pass. It can be used on any off-the-shelf&#10;trained model to test single instances for both classification and segmentation&#10;tasks. AugBN estimates normalization statistics of the unseen test distribution&#10;from the given test image using only one forward pass with label-preserving&#10;transformations. Since AugBN does not involve any back-propagation, it is&#10;significantly faster compared to recent test time adaptation methods. We&#10;further extend AugBN to make the algorithm hyperparameter-free. Rigorous&#10;experimentation show that our simple algorithm is able to achieve significant&#10;performance gains for a variety of datasets, tasks, and network architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.16241" label="2103.16241">
-        <attvalues>
-          <attvalue for="0" value="Improving robustness against common corruptions with frequency biased&#10;  models" />
-          <attvalue for="1" value="  CNNs perform remarkably well when the training and test distributions are&#10;i.i.d, but unseen image corruptions can cause a surprisingly large drop in&#10;performance. In various real scenarios, unexpected distortions, such as random&#10;noise, compression artefacts, or weather distortions are common phenomena.&#10;Improving performance on corrupted images must not result in degraded i.i.d&#10;performance - a challenge faced by many state-of-the-art robust approaches.&#10;Image corruption types have different characteristics in the frequency spectrum&#10;and would benefit from a targeted type of data augmentation, which, however, is&#10;often unknown during training. In this paper, we introduce a mixture of two&#10;expert models specializing in high and low-frequency robustness, respectively.&#10;Moreover, we propose a new regularization scheme that minimizes the total&#10;variation (TV) of convolution feature-maps to increase high-frequency&#10;robustness. The approach improves on corrupted images without degrading&#10;in-distribution performance. We demonstrate this on ImageNet-C and also for&#10;real-world corruptions on an automotive dataset, both for object classification&#10;and object detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.08613" label="2409.08613">
-        <attvalues>
-          <attvalue for="0" value="Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse&#10;  Viewpoints" />
-          <attvalue for="1" value="  3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene&#10;synthesis and novel view synthesis tasks. Typically, the initialization of 3D&#10;Gaussian primitives relies on point clouds derived from Structure-from-Motion&#10;(SfM) methods. However, in scenarios requiring scene reconstruction from sparse&#10;viewpoints, the effectiveness of 3DGS is significantly constrained by the&#10;quality of these initial point clouds and the limited number of input images.&#10;In this study, we present Dust-GS, a novel framework specifically designed to&#10;overcome the limitations of 3DGS in sparse viewpoint conditions. Instead of&#10;relying solely on SfM, Dust-GS introduces an innovative point cloud&#10;initialization technique that remains effective even with sparse input data.&#10;Our approach leverages a hybrid strategy that integrates an adaptive&#10;depth-based masking technique, thereby enhancing the accuracy and detail of&#10;reconstructed scenes. Extensive experiments conducted on several benchmark&#10;datasets demonstrate that Dust-GS surpasses traditional 3DGS methods in&#10;scenarios with sparse viewpoints, achieving superior scene reconstruction&#10;quality with a reduced number of input images.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The synthesis of high-quality 3D scenes from sparse image data has emerged as a crucial research area in computer vision and graphics, driven by applications such as virtual reality, augmented reality, autonomous driving, and robotics \cite{tonderski2024neurad,pan2023uniocc,zhou2024drivinggaussian}. These applications require accurate 3D reconstructions from limited viewpoints, often with only a few input images available. While traditional methods like Neural Radiance Fields (NeRF) \cite{martin2021nerf} have shown impressive performance in generating detailed 3D scenes and novel views, they rely on implicit volumetric representations that demand substantial computational resources for pixel-level ray tracing and optimization \cite{mildenhall2021nerf,wang2023sparsenerf,truong2023sparf}. This dependency makes them less practical for scenarios with sparse input data where computational efficiency and rapid scene reconstruction are essential.&#10;&#10;Recently, 3D Gaussian Splatting (3DGS) \cite{kerbl20233d,li2024dngaussian,lu2024scaffold} has emerged as a promising alternative, offering explicit representations for 3D scene synthesis that bypass the computational burden of voxel-based rendering of NeRF. By utilizing Gaussian primitives, 3DGS facilitates faster and more efficient scene reconstruction and novel view synthesis \cite{li2024dngaussian,zhang2024pixel,yu2024mip}. The explicit nature of Gaussian representations allows for direct sampling and manipulation in space, enhancing both rendering efficiency and flexibility in editing 3D scenes. However, the effectiveness of 3DGS is inherently tied to the quality of the initial point clouds generated by Structure-from-Motion (SfM) \cite{schoenberger2016mvs,schonberger2016structure} methods. In scenarios with sparse viewpoints and limited input images, the initial point clouds often lack sufficient detail and accuracy, constraining the performance of 3DGS and limiting its applicability to sparse-viewpoint scene reconstruction \cite{xiong2023sparsegs,chen2024optimizing,fan2024instantsplat}.&#10;&#10;To address the limitations of 3DGS in sparse viewpoint conditions, we propose that the density of point clouds is crucial for high-quality scene reconstruction. We introduce Dust-GS, a novel framework designed to enhance point cloud initialization beyond traditional SfM methods. &#10;Dust-GS integrates a hybrid strategy that combines adaptive depth-based masking with point cloud optimization to generate denser and more accurate point clouds from sparse input data. This approach demonstrates that &quot;dense point clouds matter&quot; for achieving superior scene synthesis under limited viewpoints.&#10;The comparison of Dust-GS with other methods is shown in Fig.\ref{fig:mov}.&#10;To further enhance 3D reconstruction accuracy and stability in sparse data settings, Dust-GS introduces an adaptive depth processing technique and an improved point cloud initialization strategy that generates denser point clouds, overcoming the sparsity issues of SfM-based methods. Additionally, a dynamic depth masking mechanism is employed to filter high-frequency noise and artifacts from distant objects, retaining critical geometric information from mid- and near-range areas. This targeted depth management significantly improves scene reconstruction quality from sparse inputs, as demonstrated by our experimental results. &#10;&#10;In summary, our contributions are:&#10;\begin{itemize}&#10; \item We propose a framework that optimizes Gaussian splatting for detailed 3D scene reconstruction using sparse input data, enhancing performance in few-shot conditions.&#10; \item A novel point cloud initialization strategy is introduced, reducing reliance on traditional SfM methods and generating denser, more accurate point clouds for improved scene synthesis.&#10; \item We develop a dynamic depth masking mechanism to refine the control over Gaussian primitives, effectively filtering noise and improving geometric fidelity in the reconstructed scenes.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Computer Science, Scene Reconstruction, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2311.15260" label="2311.15260">
-        <attvalues>
-          <attvalue for="0" value="NeuRAD: Neural Rendering for Autonomous Driving" />
-          <attvalue for="1" value="  Neural radiance fields (NeRFs) have gained popularity in the autonomous&#10;driving (AD) community. Recent methods show NeRFs' potential for closed-loop&#10;simulation, enabling testing of AD systems, and as an advanced training data&#10;augmentation technique. However, existing methods often require long training&#10;times, dense semantic supervision, or lack generalizability. This, in turn,&#10;hinders the application of NeRFs for AD at scale. In this paper, we propose&#10;NeuRAD, a robust novel view synthesis method tailored to dynamic AD data. Our&#10;method features simple network design, extensive sensor modeling for both&#10;camera and lidar -- including rolling shutter, beam divergence and ray dropping&#10;-- and is applicable to multiple datasets out of the box. We verify its&#10;performance on five popular AD datasets, achieving state-of-the-art performance&#10;across the board. To encourage further development, we will openly release the&#10;NeuRAD source code. See https://github.com/georghess/NeuRAD .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.09117" label="2306.09117">
-        <attvalues>
-          <attvalue for="0" value="UniOcc: Unifying Vision-Centric 3D Occupancy Prediction with Geometric&#10;  and Semantic Rendering" />
-          <attvalue for="1" value="  In this technical report, we present our solution, named UniOCC, for the&#10;Vision-Centric 3D occupancy prediction track in the nuScenes Open Dataset&#10;Challenge at CVPR 2023. Existing methods for occupancy prediction primarily&#10;focus on optimizing projected features on 3D volume space using 3D occupancy&#10;labels. However, the generation process of these labels is complex and&#10;expensive (relying on 3D semantic annotations), and limited by voxel&#10;resolution, they cannot provide fine-grained spatial semantics. To address this&#10;limitation, we propose a novel Unifying Occupancy (UniOcc) prediction method,&#10;explicitly imposing spatial geometry constraint and complementing fine-grained&#10;semantic supervision through volume ray rendering. Our method significantly&#10;enhances model performance and demonstrates promising potential in reducing&#10;human annotation costs. Given the laborious nature of annotating 3D occupancy,&#10;we further introduce a Depth-aware Teacher Student (DTS) framework to enhance&#10;prediction accuracy using unlabeled data. Our solution achieves 51.27\% mIoU on&#10;the official leaderboard with single model, placing 3rd in this challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.07920" label="2312.07920">
-        <attvalues>
-          <attvalue for="0" value="DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic&#10;  Autonomous Driving Scenes" />
-          <attvalue for="1" value="  We present DrivingGaussian, an efficient and effective framework for&#10;surrounding dynamic autonomous driving scenes. For complex scenes with moving&#10;objects, we first sequentially and progressively model the static background of&#10;the entire scene with incremental static 3D Gaussians. We then leverage a&#10;composite dynamic Gaussian graph to handle multiple moving objects,&#10;individually reconstructing each object and restoring their accurate positions&#10;and occlusion relationships within the scene. We further use a LiDAR prior for&#10;Gaussian Splatting to reconstruct scenes with greater details and maintain&#10;panoramic consistency. DrivingGaussian outperforms existing methods in dynamic&#10;driving scene reconstruction and enables photorealistic surround-view synthesis&#10;with high-fidelity and multi-camera consistency. Our project page is at:&#10;https://github.com/VDIGPKU/DrivingGaussian.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.02268" label="2008.02268">
-        <attvalues>
-          <attvalue for="0" value="NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo&#10;  Collections" />
-          <attvalue for="1" value="  We present a learning-based method for synthesizing novel views of complex&#10;scenes using only unstructured collections of in-the-wild photographs. We build&#10;on Neural Radiance Fields (NeRF), which uses the weights of a multilayer&#10;perceptron to model the density and color of a scene as a function of 3D&#10;coordinates. While NeRF works well on images of static subjects captured under&#10;controlled settings, it is incapable of modeling many ubiquitous, real-world&#10;phenomena in uncontrolled images, such as variable illumination or transient&#10;occluders. We introduce a series of extensions to NeRF to address these issues,&#10;thereby enabling accurate reconstructions from unstructured image collections&#10;taken from the internet. We apply our system, dubbed NeRF-W, to internet photo&#10;collections of famous landmarks, and demonstrate temporally consistent novel&#10;view renderings that are significantly closer to photorealism than the prior&#10;state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.16196" label="2303.16196">
-        <attvalues>
-          <attvalue for="0" value="SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis" />
-          <attvalue for="1" value="  Neural Radiance Field (NeRF) significantly degrades when only a limited&#10;number of views are available. To complement the lack of 3D information,&#10;depth-based models, such as DSNeRF and MonoSDF, explicitly assume the&#10;availability of accurate depth maps of multiple views. They linearly scale the&#10;accurate depth maps as supervision to guide the predicted depth of few-shot&#10;NeRFs. However, accurate depth maps are difficult and expensive to capture due&#10;to wide-range depth distances in the wild.&#10;  In this work, we present a new Sparse-view NeRF (SparseNeRF) framework that&#10;exploits depth priors from real-world inaccurate observations. The inaccurate&#10;depth observations are either from pre-trained depth models or coarse depth&#10;maps of consumer-level depth sensors. Since coarse depth maps are not strictly&#10;scaled to the ground-truth depth maps, we propose a simple yet effective&#10;constraint, a local depth ranking method, on NeRFs such that the expected depth&#10;ranking of the NeRF is consistent with that of the coarse depth maps in local&#10;patches. To preserve the spatial continuity of the estimated depth of NeRF, we&#10;further propose a spatial continuity constraint to encourage the consistency of&#10;the expected depth continuity of NeRF with coarse depth maps. Surprisingly,&#10;with simple depth ranking constraints, SparseNeRF outperforms all&#10;state-of-the-art few-shot NeRF methods (including depth-based models) on&#10;standard LLFF and DTU datasets. Moreover, we collect a new dataset NVS-RGBD&#10;that contains real-world depth maps from Azure Kinect, ZED 2, and iPhone 13&#10;Pro. Extensive experiments on NVS-RGBD dataset also validate the superiority&#10;and generalizability of SparseNeRF. Code and dataset are available at&#10;https://sparsenerf.github.io/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Neural Radiance Fields. NeRF~\cite{mildenhall2020nerf,tao2023lidar,chen2023cunerf,peng2021pi} has made great success in synthesizing novel views of complex scenes due to good representation of neural networks \cite{he2016deep,wang2020grammatically,wang2018kalman,wang2019adaptively,dong2021efficientbert,li2022automated}. Block-NeRF \cite{tancik2022block}, CityNeRF \cite{xiangli2021citynerf}, and Mega-NeRF \cite{turki2022mega} scaled the standard NeRF up to city-scale or urban-scale scenes. NeRF$--$ \cite{wang2021nerf}, GNeRF \cite{meng2021gnerf}, and BARF \cite{lin2021barf} relaxed the requirements of NeRFs and synthesized novel views without perfect camera poses. Some works attempted to improve NeRFs by considering anti-aliasing \cite{barron2021mip}, sparse 3D grids with spherical harmonics \cite{Fridovich-Keil_2022_CVPR}. Some methods \cite{pumarola2021d,li2021neural,xian2021space} extended NeRFs to dynamic scenes. These methods do not focus on generating novel views with a few views. In this paper, we study sparse-view NeRF to reduce dense capture requirements in the real-world applications. &#10;&#10;Few-shot Novel View Synthesis. There are increasing studies on few-shot novel view synthesis. Basically, the existing few-shot NeRF methods can be categorized into three groups. First, some methods exploit continuity constraints on geometry or object semantics. For example, &#10;RegNeRF \cite{niemeyer2022regnerf} imposed a continuity constraint on geometry and regularized the appearance of patches from unobserved viewpoints with a flow model.&#10;InfoNeRF \cite{kim2022infonerf} proposed a ray entropy minimization regularization to encourage the density to be as sparse as possible along a ray, and used a ray information gain reduction regularization to constrain the continuous depth of neighbor rays.&#10;Second, some methods attempt to pe-train a NeRF on other similar scenes and fine-tune the NeRF on the target scene. For example, PixelNeRF \cite{yu2021pixelnerf} conditioned a NeRF on image inputs in a fully convolutional manner, allowing the model to learn scene priors from other scenes and reduce the requirement of dense views.&#10;MVSNeRF \cite{chen2021mvsnerf} leveraged plane-swept cost volumes for geometry-aware scene reasoning, and combined it with physically based volume rendering. Similar to PixelNeRF, MVSNeRF was first trained on other real scenes and was finetuned on target scenes to evaluate its effectiveness and generalizability. &#10;Third, depth-based models \cite{deng2022depth,yu2022monosdf} use available depth information to supervise the training of NeRFs. DSNeRF \cite{deng2022depth} exploited sparse 3D points generated by COLMAP \cite{schoenberger2016sfm} or accurate depth maps \cite{choi2016large} obtained by high-accuracy depth scanners. Different from these depth-based models, SparseNeRF distills robust depth ranking from pre-trained depth models or coarse depth maps from consumer-level sensors. As concurrent works, NeRDi \cite{deng2023nerdi} and NeuralLift-360 \cite{xu2023neurallift} also use ranking-based methods. However, they mainly focus on single-view setting while SparseNeRF focuses on sparse-view. SparseNeRF uses ranking loss to avoid inconsistent 3D geometry across different views while \cite{deng2023nerdi,xu2023neurallift} focuses on a soft geometric regularization on a single view (no cross-view consistency problem during training). Moreover, SparseNeRF introduces a new spatial continuity loss to distill spatial coherence from monocular depth estimators. In addition, some single-view synthesis methods \cite{xu2022sinnerf} either allow new generative objects from unseen views or focus on specific objects, e.g., face \cite {chan2022efficient} and human \cite{hu2023sherf}. Some NeRF-based methods \cite{hong2022eva3d,chen2023sd} extended to generation instead of reconstruction. Other AIGC models \cite{ho2020denoising,wang2022traditional,wang2022stylelight,zhuo2022fast,chen2022text2light,chen2022relighting, yang2023gp,li2023marconet,yang2022vtoonify} focus on the 2D generation or category-specific reconstruction \cite{chen2022structure}, which are different.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2211.11738" label="2211.11738">
-        <attvalues>
-          <attvalue for="0" value="SPARF: Neural Radiance Fields from Sparse and Noisy Poses" />
-          <attvalue for="1" value="  Neural Radiance Field (NeRF) has recently emerged as a powerful&#10;representation to synthesize photorealistic novel views. While showing&#10;impressive performance, it relies on the availability of dense input views with&#10;highly accurate camera poses, thus limiting its application in real-world&#10;scenarios. In this work, we introduce Sparse Pose Adjusting Radiance Field&#10;(SPARF), to address the challenge of novel-view synthesis given only few&#10;wide-baseline input images (as low as 3) with noisy camera poses. Our approach&#10;exploits multi-view geometry constraints in order to jointly learn the NeRF and&#10;refine the camera poses. By relying on pixel matches extracted between the&#10;input views, our multi-view correspondence objective enforces the optimized&#10;scene and camera poses to converge to a global and geometrically accurate&#10;solution. Our depth consistency loss further encourages the reconstructed scene&#10;to be consistent from any viewpoint. Our approach sets a new state of the art&#10;in the sparse-view regime on multiple challenging datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.06912" label="2403.06912">
-        <attvalues>
-          <attvalue for="0" value="DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with&#10;  Global-Local Depth Normalization" />
-          <attvalue for="1" value="  Radiance fields have demonstrated impressive performance in synthesizing&#10;novel views from sparse input views, yet prevailing methods suffer from high&#10;training costs and slow inference speed. This paper introduces DNGaussian, a&#10;depth-regularized framework based on 3D Gaussian radiance fields, offering&#10;real-time and high-quality few-shot novel view synthesis at low costs. Our&#10;motivation stems from the highly efficient representation and surprising&#10;quality of the recent 3D Gaussian Splatting, despite it will encounter a&#10;geometry degradation when input views decrease. In the Gaussian radiance&#10;fields, we find this degradation in scene geometry primarily lined to the&#10;positioning of Gaussian primitives and can be mitigated by depth constraint.&#10;Consequently, we propose a Hard and Soft Depth Regularization to restore&#10;accurate scene geometry under coarse monocular depth supervision while&#10;maintaining a fine-grained color appearance. To further refine detailed&#10;geometry reshaping, we introduce Global-Local Depth Normalization, enhancing&#10;the focus on small local depth changes. Extensive experiments on LLFF, DTU, and&#10;Blender datasets demonstrate that DNGaussian outperforms state-of-the-art&#10;methods, achieving comparable or better results with significantly reduced&#10;memory cost, a $25 \times$ reduction in training time, and over $3000 \times$&#10;faster rendering speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.00109" label="2312.00109">
-        <attvalues>
-          <attvalue for="0" value="Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering" />
-          <attvalue for="1" value="  Neural rendering methods have significantly advanced photo-realistic 3D scene&#10;rendering in various academic and industrial applications. The recent 3D&#10;Gaussian Splatting method has achieved the state-of-the-art rendering quality&#10;and speed combining the benefits of both primitive-based representations and&#10;volumetric representations. However, it often leads to heavily redundant&#10;Gaussians that try to fit every training view, neglecting the underlying scene&#10;geometry. Consequently, the resulting model becomes less robust to significant&#10;view changes, texture-less area and lighting effects. We introduce Scaffold-GS,&#10;which uses anchor points to distribute local 3D Gaussians, and predicts their&#10;attributes on-the-fly based on viewing direction and distance within the view&#10;frustum. Anchor growing and pruning strategies are developed based on the&#10;importance of neural Gaussians to reliably improve the scene coverage. We show&#10;that our method effectively reduces redundant Gaussians while delivering&#10;high-quality rendering. We also demonstrates an enhanced capability to&#10;accommodate scenes with varying levels-of-detail and view-dependent&#10;observations, without sacrificing the rendering speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.15530" label="2403.15530">
-        <attvalues>
-          <attvalue for="0" value="Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian&#10;  Splatting" />
-          <attvalue for="1" value="  3D Gaussian Splatting (3DGS) has demonstrated impressive novel view synthesis&#10;results while advancing real-time rendering performance. However, it relies&#10;heavily on the quality of the initial point cloud, resulting in blurring and&#10;needle-like artifacts in areas with insufficient initializing points. This is&#10;mainly attributed to the point cloud growth condition in 3DGS that only&#10;considers the average gradient magnitude of points from observable views,&#10;thereby failing to grow for large Gaussians that are observable for many&#10;viewpoints while many of them are only covered in the boundaries. To this end,&#10;we propose a novel method, named Pixel-GS, to take into account the number of&#10;pixels covered by the Gaussian in each view during the computation of the&#10;growth condition. We regard the covered pixel numbers as the weights to&#10;dynamically average the gradients from different views, such that the growth of&#10;large Gaussians can be prompted. As a result, points within the areas with&#10;insufficient initializing points can be grown more effectively, leading to a&#10;more accurate and detailed reconstruction. In addition, we propose a simple yet&#10;effective strategy to scale the gradient field according to the distance to the&#10;camera, to suppress the growth of floaters near the camera. Extensive&#10;experiments both qualitatively and quantitatively demonstrate that our method&#10;achieves state-of-the-art rendering quality while maintaining real-time&#10;rendering speed, on the challenging Mip-NeRF 360 and Tanks &amp; Temples datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2311.16493" label="2311.16493">
-        <attvalues>
-          <attvalue for="0" value="Mip-Splatting: Alias-free 3D Gaussian Splatting" />
-          <attvalue for="1" value="  Recently, 3D Gaussian Splatting has demonstrated impressive novel view&#10;synthesis results, reaching high fidelity and efficiency. However, strong&#10;artifacts can be observed when changing the sampling rate, \eg, by changing&#10;focal length or camera distance. We find that the source for this phenomenon&#10;can be attributed to the lack of 3D frequency constraints and the usage of a 2D&#10;dilation filter. To address this problem, we introduce a 3D smoothing filter&#10;which constrains the size of the 3D Gaussian primitives based on the maximal&#10;sampling frequency induced by the input views, eliminating high-frequency&#10;artifacts when zooming in. Moreover, replacing 2D dilation with a 2D Mip&#10;filter, which simulates a 2D box filter, effectively mitigates aliasing and&#10;dilation issues. Our evaluation, including scenarios such a training on&#10;single-scale images and testing on multiple scales, validates the effectiveness&#10;of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.03213" label="2409.03213">
-        <attvalues>
-          <attvalue for="0" value="Optimizing 3D Gaussian Splatting for Sparse Viewpoint Scene&#10;  Reconstruction" />
-          <attvalue for="1" value="  3D Gaussian Splatting (3DGS) has emerged as a promising approach for 3D scene&#10;representation, offering a reduction in computational overhead compared to&#10;Neural Radiance Fields (NeRF). However, 3DGS is susceptible to high-frequency&#10;artifacts and demonstrates suboptimal performance under sparse viewpoint&#10;conditions, thereby limiting its applicability in robotics and computer vision.&#10;To address these limitations, we introduce SVS-GS, a novel framework for Sparse&#10;Viewpoint Scene reconstruction that integrates a 3D Gaussian smoothing filter&#10;to suppress artifacts. Furthermore, our approach incorporates a Depth Gradient&#10;Profile Prior (DGPP) loss with a dynamic depth mask to sharpen edges and 2D&#10;diffusion with Score Distillation Sampling (SDS) loss to enhance geometric&#10;consistency in novel view synthesis. Experimental evaluations on the&#10;MipNeRF-360 and SeaThru-NeRF datasets demonstrate that SVS-GS markedly improves&#10;3D reconstruction from sparse viewpoints, offering a robust and efficient&#10;solution for scene understanding in robotics and computer vision applications.&#10;" />
-          <attvalue for="2" value="&#10;The use of RGB cameras in robotic vision systems for 3D scene reconstruction is essential for acquiring multiple viewpoints, a fundamental requirement for high-quality novel view synthesis (NVS). However, in practical scenarios, obtaining dense multi-view data is often impractical, especially in resource-constrained or complex environments. This limitation necessitates developing methods that can achieve effective scene reconstruction from sparse viewpoints. Traditional Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf,barron2021mip,Barron_2023_ICCV} have shown strong performance in NVS, but their pixel-level ray rendering is computationally intensive and not well-suited for scenarios with sparse input data, requiring substantial resources and processing time.&#10;&#10;In contrast, 3D Gaussian Splatting (3DGS) \cite{kerbl20233d} employs an explicit representation that significantly reduces both training and rendering times while maintaining high-quality outputs. This method initializes a set of 3D Gaussians from point clouds generated by Structure from Motion (SfM) \cite{schonberger2016structure} or via random initialization. It uses adaptive density control to clone and prune these Gaussians, enhancing scene detail representation. Leveraging the smooth, differentiable properties of Gaussian distributions, 3DGS enables rapid rasterization by projecting 3D Gaussians onto 2D image planes, supporting efficient rendering and interpolation \cite{kerbl20233d,tang2023dreamgaussian,yi2024gaussiandreamer}.&#10;&#10;3D Gaussian distributions effectively capture details across multiple scales, and their projection onto a 2D plane simplifies the rasterization process. While this method is capable of efficiently representing complex, large-scale scenes or objects, the absence of size constraints for each 3D Gaussian primitive leads to a loss of detail when reconstructing fine objects, especially upon zooming in. This limitation is particularly evident when dealing with extremely thin lines, where it can result in inaccuracies that hinder the precise capture and reproduction of slender structures and small features, thereby compromising the overall visual realism and detail fidelity of the scene \cite{zhou2024feature,yu2024mip}. Moreover, in practical applications, 3D Gaussian Splatting (3DGS) requires densely sampled multi-view scenes to achieve optimal results \cite{zhang2024pixel,fan2024instantsplat,li2024dngaussian}. However, obtaining such extensive viewpoint data is often impractical in resource-constrained or complex environments. The unconstrained size of primitives in 3DGS and the reliance on dense multi-view image data present significant challenges for practical applications, such as autonomous vehicle navigation.&#10;&#10;3DGS methods are heavily dependent on the density and quality of initial point clouds derived from dense multi-view inputs, which limits their effectiveness in sparse-viewpoint scenarios. To address the inherent limitations of 3DGS, we propose a sparse-view 3DGS framework, termed SVS-GS. To impose size constraints on the 3D Gaussian primitives, we introduce a 3D smoothing filter \cite{yu2024mip}. This filter regulates the diffusion range of Gaussian primitives in both 3D space and their 2D projections, ensuring the preservation of more details during reconstruction, particularly for small and thin structures. In standard 3DGS, the initial 3D Gaussian primitives are derived from point cloud data generated by COLMAP \cite{schonberger2016structure,schoenberger2016mvs}. However, sparse views yield a limited number of initial points, resulting in low point cloud density, which adversely affects the distribution and quality of Gaussian primitives. To enhance the density of these initial 3D Gaussian primitives, we introduce a local adaptive density scaling module. This module dynamically increases the density of Gaussian primitives based on the sparse point clouds, producing a denser set of 3D Gaussian primitives.&#10;&#10;For the optimization of the 3D Gaussian primitives, we employ score distillation sampling (SDS) loss \cite{poole2022dreamfusion} to integrate 3DGS with 2D diffusion, incorporating depth prior information to constrain the positions and sizes of the 3D Gaussian primitives. Additionally, we introduce a dynamic depth mask and Gradient Profile Prior (GPP) loss \cite{sun2010gradient} to enhance the sharpness of edges in the depth maps. SVS-GS effectively addresses gaps in the sparse point cloud data while simultaneously improving the uniformity and spatial coverage of the initial Gaussian primitives, thereby enhancing precision and detail fidelity in 3D scene reconstruction.&#10;&#10;Our main contributions are as follows:&#10;\begin{itemize}&#10; \item Novel Sparse-View Framework: SVS-GS reduces dependency on dense multi-view data by optimizing Gaussian primitive distributions, improving practicality and efficiency.&#10; \item Adaptive Density Scaling: A local adaptive density scaling module generates denser initial 3D Gaussian primitives, addressing the problem of sparse point clouds.&#10; \item Enhanced Optimization Techniques: Integration of SDS loss with 2D diffusion, dynamic depth masks, and depth priors ensures precise control over Gaussian primitives, improving detail reconstruction.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.02837" label="2201.02837">
-        <attvalues>
-          <attvalue for="0" value="Mushrooms Detection, Localization and 3D Pose Estimation using RGB-D&#10;  Sensor for Robotic-picking Applications" />
-          <attvalue for="1" value="  In this paper, we propose mushrooms detection, localization and 3D pose&#10;estimation algorithm using RGB-D data acquired from a low-cost consumer RGB-D&#10;sensor. We use the RGB and depth information for different purposes. From RGB&#10;color, we first extract initial contour locations of the mushrooms and then&#10;provide both the initial contour locations and the original image to active&#10;contour for mushrooms segmentation. These segmented mushrooms are then used as&#10;input to a circular Hough transform for each mushroom detection including its&#10;center and radius. Once each mushroom's center position in the RGB image is&#10;known, we then use the depth information to locate it in 3D space i.e. in world&#10;coordinate system. In case of missing depth information at the detected center&#10;of each mushroom, we estimate from the nearest available depth information&#10;within the radius of each mushroom. We also estimate the 3D pose of each&#10;mushroom using a pre-prepared upright mushroom model. We use a global&#10;registration followed by local refine registration approach for this 3D pose&#10;estimation. From the estimated 3D pose, we use only the rotation part expressed&#10;in quaternion as an orientation of each mushroom. These estimated (X,Y,Z)&#10;positions, diameters and orientations of the mushrooms are used for&#10;robotic-picking applications. We carry out extensive experiments on both 3D&#10;printed and real mushrooms which show that our method has an interesting&#10;performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Harvesting of fruits and vegetables such as grapes, mangoes, apples, kiwifruit, peaches, citrus, cherries, pears and mushrooms is highly labor intensive and poses human risk, for instance, ladder-related injuries that require a significant amount payment for compensation. It is also becoming less sustainable with increasing cost and decreasing availability of a skilled labor force. For instance, the \pounds 120m UK mushroom industry is facing challenges from reduced labour availability due to changes in demographics i.e. fewer younger people are entering the agricultural community whilst global food demand is ever expanding, and changes in national inward migration policies that may impact labour availability. Picking labour currently accounts for about 1/3 of production costs. To meet the increasing labor demand, to lower human risk of injuries, and to decrease the harvesting cost, automated harvesting system is crucial.&#10;&#10;Many researchers and private companies have attempted to develop fully automated robotic system for harvesting of fruits and vegetables, however, no commercialized robotic harvester is available yet i.e. the need to develop robotic approaches to pick fresh fruits and vegetables is global. To develop a robotic harvester, the accuracy of detection, localization and 3D pose (rotation and translation) of fruits and vegetables is very important~\cite{GongAma15}\cite{PetKoi16}. However, there are many challenges to develop accurate detection, localization and 3D pose estimation algorithms such as changing illuminations, severe occlusions, various sizes and textures of fruits and vegetables, highly unstructured scenes, etc. Furthermore, these challenges cause robotic harvesting to be developed for specific data and designed for the task at hand.&#10;&#10;There are some previous works of applying computer vision techniques for agriculture (agrovision) as reviewed in~\cite{KapBar12}\cite{GongAma15}. Some segmentation techniques for fruit detection are discussed in~\cite{SucJam16}, and some works are particularly tuned for a specific type of fruits such as grapes, mangoes, apples, kiwifruit, peaches, etc. Fruit detection, tracking, and 3D reconstruction for crop mapping and yield estimation was developed in~\cite{MooCha10}. These all previous works have focused mainly on 2D images. The recent introduction of cheap RGB-D sensors have opened a new door for computer vision researchers for many real applications. RGB-D sensors combine RGB color information with per-pixel depth information. Though sensors such as the Swiss Ranger SR4000 and Photonic Mixing Device (PMD) Tech products which can provide such data have existed for years, they are very costly. However, new consumer RGB-D sensors such as Kinect, Intel RealSense and Zed stereo camera are very cheap when compared to the aforementioned sensors. The applications and the challenges of Kinect sensor, particularly Kinect version 1 was explained in~\cite{CruzLuc12}. Both Kinect version 1 and 2 send out infrared laser light in order to measure the depth. The Kinect version 1 uses the structured-light-method to measure the depth with Pattern Projection principle. In this case, a known infrared (IR) pattern is projected into the scene and the depth is computed out of its distortion. However, the Kinect version 2 uses the Time-of-Flight (ToF) method and determines the depth by measuring the time the emitted light takes from ToF camera to the object and return. The Kinect version 2 provides more accurate depth information than the older version~\cite{WasStr17}\cite{YanZhaDon15}. For obtaining depth information from the Kinect version 2, measurable distance range is from 0.5m to 4m~\cite{YanZhaDon15}. More importantly the depth accuracy of Kinect version 2 is almost constant over different distances~\cite{YanZhaDon15}. However, it is recommended to pre-heat the device for at least 25 min to obtain reliable results~\cite{WasStr17}. The ZED Stereo Camera, a depth sensor based on passive stereovision, observes objects from 0.5 to 20m. Intel RealSense cameras, both SR300 and D435, use infrared depth-sensing technology. D435 use active stereo infrared technology, which means the device has two infrared cameras as opposed to a single one on the SR300. SR300 is a short-range sensor (hence the model name) using coded-light method like Kinect version 1 whereas D435 is a longer-range sensor with a larger field of view as it uses depth stereo. The measurable distance range for SR300 is 0.2m - 1.5m whereas for D435 is 0.11m - 10m. The maximum range of D435 varies depending on calibration, scene, lighting condition, etc. The emergence of these low-cost RGB-D sensors motivated many computer vision researchers to use them for many applications such as object recognition~\cite{PraAld15}\cite{AndJos15}, object detection~\cite{XiaYun17}, pose estimation~\cite{SchBeh17}\cite{NguKoJeo15}\cite{ChoTagTuz12}, 3D reconstruction~\cite{IzadiKim11}, simultaneous localization and mapping (SLAM)~\cite{EndCre14}, etc.&#10;&#10;However, to the best of our knowledge, only the work in~\cite{Mas13} gave attention to the application of computer vision for mushrooms harvesting. This work focused on a specific mushroom type, brown colour, and developed segmentation algorithm based on K-means. Though a good start, it has many limitations, for instance, the number of clusters needs to be predefined and the colour of the mushrooms to be applied to needs to be known in advance. It also did not include on any work on 3D pose estimation of the mushrooms. Our work takes one step forward by removing any prior knowledge about the mushrooms colour, illumination, etc as well as estimating depth information even if it is missing due to the inherent problems of the RGB-D sensors such as noise and holes. Basically the goal is to discriminate the mushrooms from the rest of the scene, to detect the presence of individual mushrooms, to localise them in space, and to estimate their 3D pose.&#10;&#10;In this work, we propose mushrooms detection, localization and 3D pose estimation algorithm for real-time robotic-picking applications. Accordingly, our contributions can be summarized as follows. &#10;&#10;\begin{enumerate}&#10;\item We segment mushrooms using active contour under varying illumination conditions by providing an automatic initial contour locations.&#10;\item We detect individual mushrooms of varying size by providing the active contour-based segmented results to a circular Hough transform.&#10;\item We localize the mushrooms' centers by estimating from the nearest available depth information in case of missing.&#10;\item We estimate the 3D pose of mushrooms using global registration followed by local refine registration approach.&#10;\item We make extensive experiments on both 3D printed and real mushrooms.&#10;\end{enumerate}&#10;&#10;The paper is organized as follows. The overview of our proposed method is given in section~\ref{sec:ProposedMethod}. In section~\ref{sec:Segmentation}, the segmentation method is described in detail. The mushrooms detection and localization approaches are explained in section~\ref{sec:MushroomsDetection} and section~\ref{sec:MushroomsLocalization}, respectively. In section~\ref{sec:MushroomsPose}, the 3D pose estimation is given. The experimental results are analyzed and compared in section~\ref{sec:ExperimentalResults}. The main conclusions and suggestions for future work are summarized in section~\ref{sec:Conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Robotic Object Detection, Computer Vision, Engineering, Machine Learning, Mathematics, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1610.08120" label="1610.08120">
-        <attvalues>
-          <attvalue for="0" value="Image Segmentation for Fruit Detection and Yield Estimation in Apple&#10;  Orchards" />
-          <attvalue for="1" value="  Ground vehicles equipped with monocular vision systems are a valuable source&#10;of high resolution image data for precision agriculture applications in&#10;orchards. This paper presents an image processing framework for fruit detection&#10;and counting using orchard image data. A general purpose image segmentation&#10;approach is used, including two feature learning algorithms; multi-scale&#10;Multi-Layered Perceptrons (MLP) and Convolutional Neural Networks (CNN). These&#10;networks were extended by including contextual information about how the image&#10;data was captured (metadata), which correlates with some of the appearance&#10;variations and/or class distributions observed in the data. The pixel-wise&#10;fruit segmentation output is processed using the Watershed Segmentation (WS)&#10;and Circular Hough Transform (CHT) algorithms to detect and count individual&#10;fruits. Experiments were conducted in a commercial apple orchard near&#10;Melbourne, Australia. The results show an improvement in fruit segmentation&#10;performance with the inclusion of metadata on the previously benchmarked MLP&#10;network. We extend this work with CNNs, bringing agrovision closer to the&#10;state-of-the-art in computer vision, where although metadata had negligible&#10;influence, the best pixel-wise F1-score of $0.791$ was achieved. The WS&#10;algorithm produced the best apple detection and counting results, with a&#10;detection F1-score of $0.858$. As a final step, image fruit counts were&#10;accumulated over multiple rows at the orchard and compared against the&#10;post-harvest fruit counts that were obtained from a grading and counting&#10;machine. The count estimates using CNN and WS resulted in the best performance&#10;for this dataset, with a squared correlation coefficient of $r^2=0.826$.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.13844" label="2212.13844">
-        <attvalues>
-          <attvalue for="0" value="Evaluating and Improving the Depth Accuracy of Kinect for Windows v2" />
-          <attvalue for="1" value="  Microsoft Kinect sensor has been widely used in many applications since the&#10;launch of its first version. Recently, Microsoft released a new version of&#10;Kinect sensor with improved hardware. However, the accuracy assessment of the&#10;sensor remains to be answered. In this paper, we measure the depth accuracy of&#10;the newly released Kinect v2 depth sensor, and obtain a cone model to&#10;illustrate its accuracy distribution. We then evaluate the variance of the&#10;captured depth values by depth entropy. In addition, we propose a trilateration&#10;method to improve the depth accuracy with multiple Kinects simultaneously. The&#10;experimental results are provided to ascertain the proposed model and method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1507.06821" label="1507.06821">
-        <attvalues>
-          <attvalue for="0" value="Multimodal Deep Learning for Robust RGB-D Object Recognition" />
-          <attvalue for="1" value="  Robust object recognition is a crucial ingredient of many, if not all,&#10;real-world robotics applications. This paper leverages recent progress on&#10;Convolutional Neural Networks (CNNs) and proposes a novel RGB-D architecture&#10;for object recognition. Our architecture is composed of two separate CNN&#10;processing streams - one for each modality - which are consecutively combined&#10;with a late fusion network. We focus on learning with imperfect sensor data, a&#10;typical problem in real-world robotics tasks. For accurate learning, we&#10;introduce a multi-stage training methodology and two crucial ingredients for&#10;handling depth data with CNNs. The first, an effective encoding of depth&#10;information for CNNs that enables learning without the need for large depth&#10;datasets. The second, a data augmentation scheme for robust learning with depth&#10;images by corrupting them with realistic noise patterns. We present&#10;state-of-the-art results on the RGB-D object dataset and show recognition in&#10;challenging RGB-D real-world noisy settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.04887" label="2309.04887">
-        <attvalues>
-          <attvalue for="0" value="SortedAP: Rethinking evaluation metrics for instance segmentation" />
-          <attvalue for="1" value="  Designing metrics for evaluating instance segmentation revolves around&#10;comprehensively considering object detection and segmentation accuracy.&#10;However, other important properties, such as sensitivity, continuity, and&#10;equality, are overlooked in the current study. In this paper, we reveal that&#10;most existing metrics have a limited resolution of segmentation quality. They&#10;are only conditionally sensitive to the change of masks or false predictions.&#10;For certain metrics, the score can change drastically in a narrow range which&#10;could provide a misleading indication of the quality gap between results.&#10;Therefore, we propose a new metric called sortedAP, which strictly decreases&#10;with both object- and pixel-level imperfections and has an uninterrupted&#10;penalization scale over the entire domain. We provide the evaluation toolkit&#10;and experiment code at https://www.github.com/looooongChen/sortedAP.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}. In the field of bioimage computing, segmenting instances of animals~\cite{mazur2020deep}, cells~\cite{edlund2021livecell}, and subcellular structures~\cite{caicedo2019nucleus,glory2007automated} is also common and infrastructural processing for further analysis and study. Instance segmentation not only localizes the object of interest but also delineates the exact boundary, which can be seen as performing object detection and semantic segmentation concurrently.&#10;&#10;Correspondingly, a qualified evaluation metric should consider three fundamental types of imperfections: missed ground truth objects (false negative), falsely predicted objects (false positive), and segmentation inaccuracy. Existing metrics all incorporate the three error types above, but are not discussed with respect to properties, including sensitivity, continuity, and equality.&#10;&#10;Sensitivity. An ideal metric should be sensitive to all occurrences of imperfections of all types. Any additional errors are supposed to lead monotonically to a worse score, not ignored or obscured by the occurrence of other errors. A metric that monotonically decreases with any errors will enable a more accurate comparison.&#10;&#10;Continuity. The penalization scale of a metric should be relatively consistent locally across the score domain. Intuitively, gradually and evenly changing segmentations should correspond to a smoothly changing metric score as well. Abrupt changes are not desired.&#10;&#10;Equality. Without any assumed importance of different objects, all objects should have an equal influence on the metric score. A common case of inequality is that the score is biased towards larger objects. Although larger objects may be prioritized in some applications, as a general metric, the metric should treat all objects equally. Analysis with respect to object size can be easily performed by evaluating different size groups using a metric of equal property.&#10;&#10;Although all metrics discussed in this paper implement a penalization of false positive, false negative, and segmentation inaccuracy, the majority of metrics, even very widely used ones, such as the mean Average Precision (mAP)~\cite{caicedo2019nucleus}, are only conditionally sensitive to errors. This violates the sensitivity property, as some differences in segmentation results are not reflected in the score. For match-based approaches, such as Average Precision (AP)~\cite{caicedo2019nucleus} and Panoptic Quality (PQ)~\cite{kirillov2019panoptic}, the score will change abruptly at the match threshold. There is actually a paradox in choosing thresholds, which is discussed in Section~\ref{sec:deficiency}.&#10;&#10;To address the gap, we propose a new metric called the sorted Average Precision (sortedAP). Unlike mAP~\cite{caicedo2019nucleus}, which queries the AP score at a sequence of fixed intersectionover union (IoU) thresholds, sortedAP detects every exact IoU value at which the AP score drops. This is achieved through our proposed Unique Matching approach and sorting all possible matches according to the IoU values (Section~\ref{sec:sortedAP}). The Unique Matching method explicitly preserves the one-to-one relationship between two sets of instances. This also allows the use of IoU thresholds smaller than 0.5, or under object overlap, in all match-based metrics.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Segmentation Accuracy, Mathematics, Metric Evaluation, Object Detection, Computer Science, Pattern Recognition" />
-        </attvalues>
-      </node>
-      <node id="2003.10152" label="2003.10152">
-        <attvalues>
-          <attvalue for="0" value="SOLOv2: Dynamic and Fast Instance Segmentation" />
-          <attvalue for="1" value="  In this work, we aim at building a simple, direct, and fast instance&#10;segmentation framework with strong performance. We follow the principle of the&#10;SOLO method of Wang et al. &quot;SOLO: segmenting objects by locations&quot;.&#10;Importantly, we take one step further by dynamically learning the mask head of&#10;the object segmenter such that the mask head is conditioned on the location.&#10;Specifically, the mask branch is decoupled into a mask kernel branch and mask&#10;feature branch, which are responsible for learning the convolution kernel and&#10;the convolved features respectively. Moreover, we propose Matrix NMS (non&#10;maximum suppression) to significantly reduce the inference time overhead due to&#10;NMS of masks. Our Matrix NMS performs NMS with parallel matrix operations in&#10;one shot, and yields better results. We demonstrate a simple direct instance&#10;segmentation system, outperforming a few state-of-the-art methods in both speed&#10;and accuracy. A light-weight version of SOLOv2 executes at 31.3 FPS and yields&#10;37.1% AP. Moreover, our state-of-the-art results in object detection (from our&#10;mask byproduct) and panoptic segmentation show the potential to serve as a new&#10;strong baseline for many instance-level recognition tasks besides instance&#10;segmentation. Code is available at: https://git.io/AdelaiDet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.12174" label="1903.12174">
-        <attvalues>
-          <attvalue for="0" value="TensorMask: A Foundation for Dense Object Segmentation" />
-          <attvalue for="1" value="  Sliding-window object detectors that generate bounding-box object predictions&#10;over a dense, regular grid have advanced rapidly and proven popular. In&#10;contrast, modern instance segmentation approaches are dominated by methods that&#10;first detect object bounding boxes, and then crop and segment these regions, as&#10;popularized by Mask R-CNN. In this work, we investigate the paradigm of dense&#10;sliding-window instance segmentation, which is surprisingly under-explored. Our&#10;core observation is that this task is fundamentally different than other dense&#10;prediction tasks such as semantic segmentation or bounding-box object&#10;detection, as the output at every spatial location is itself a geometric&#10;structure with its own spatial dimensions. To formalize this, we treat dense&#10;instance segmentation as a prediction task over 4D tensors and present a&#10;general framework called TensorMask that explicitly captures this geometry and&#10;enables novel operators on 4D tensors. We demonstrate that the tensor view&#10;leads to large gains over baselines that ignore this structure, and leads to&#10;results comparable to Mask R-CNN. These promising results suggest that&#10;TensorMask can serve as a foundation for novel advances in dense mask&#10;prediction and a more complete understanding of the task. Code will be made&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.03551" label="2210.03551">
-        <attvalues>
-          <attvalue for="0" value="Instance Segmentation of Dense and Overlapping Objects via Layering" />
-          <attvalue for="1" value="  Instance segmentation aims to delineate each individual object of interest in&#10;an image. State-of-the-art approaches achieve this goal by either partitioning&#10;semantic segmentations or refining coarse representations of detected objects.&#10;In this work, we propose a novel approach to solve the problem via object&#10;layering, i.e. by distributing crowded, even overlapping objects into different&#10;layers. By grouping spatially separated objects in the same layer, instances&#10;can be effortlessly isolated by extracting connected components in each layer.&#10;In comparison to previous methods, our approach is not affected by complex&#10;object shapes or object overlaps. With minimal post-processing, our method&#10;yields very competitive results on a diverse line of datasets: C. elegans&#10;(BBBC), Overlapping Cervical Cells (OCC) and cultured neuroblastoma cells&#10;(CCDB). The source code is publicly available.&#10;" />
-          <attvalue for="2" value="&#10;&#09;Different from semantic segmentation, which pays no attention to the individual objects, instance segmentation aims not only to associate every pixel of an image with a class label but also to delineate objects of the same class as individuals. This task becomes more challenging, when objects are densely located or even overlapping with each other.&#10;&#10;A prevalent top-down class of instance segmentation methods is detection-based, which firstly localize an instance with a coarse shape representation and then refine the shape in an additional step. As a paradigm, Mask-RCNN~\cite{mrcnn} refines bounding boxes obtained from Region-based Convolutional Neural Networks (R-CNN)~\cite{rcnn,frcnn}. Relying on non-maximum suppression (NMS) to remove duplicate predictions, detection-based methods become less competent in the cases of dense clusters and overlapping objects. A finer polygon representation approach was proposed by~\cite{StarDist} to reduce false suppressions. However, NMS has a methodological flaw for objects that inherently overlap. Moreover, many objects, especially in the biomedical domain, cannot be well-approximated by tractable shape representations, such as bounding boxes and star-convex polygons. &#10;&#10;Free from false suppression and coarse shape approximation, alternative bottom-up approaches obtain instances by grouping pixels~\cite{dcan,ssap,deepWatershed}. DCAN~\cite{dcan} predicts the object boundary explicitly and groups connected pixels that are separated by these boundaries as instances. However, this approach is sensitive to broken boundaries. A few misclassified pixels can lead to erroneous merging of adjacent objects. Graph partition based on pixel-pair affinity~\cite{ssap} and the watershed transform~\cite{deepWatershed} are more robust in terms of grouping, but the pipeline as a whole relies heavily on post-processing, with the learning model only optimized with intermediate results. In addition, grouping-based approaches are inherently incapable of handling overlapping objects, where one pixel may belong to more than one object.&#10;&#10;Recent research~\cite{disEmb,cosEmb,rnnEmb,impemb} introduces pixel embedding for grouping. In these approaches, a deep neural network is trained to map pixels into an embedding space, in which pixel embeddings from the same object are close, while those from different objects, especially adjacent objects, are apart. Then, pixels are grouped in the embedding space with low-level clustering algorithms, such as Mean Shift~\cite{meanShift} and DBSCAN~\cite{DBSCAN}. Although pixel embedding based grouping has proven to be more robust, it still can not avoid the shortcomings of grouping-based methods: the reliance on post-processing, the optimization of intermediate results, and the incapability to segment overlapping objects. &#10;&#10;Our work is partially inspired by the pixel embedding approach. To alleviate the aforementioned limitations, we propose to train a more structured embedding space. Specifically, we restrict one object to &quot;live&quot; in one dimension. Correspondingly, embedding vectors will be one-hot in overlap-free areas and have more than one active digits at locations where an overlap of objects occurs (Figure~\ref{fig:overview}). &#10;&#09;We figuratively call this process of objects being distributed to different dimensions &quot;layering&quot;, and use the terms &quot;dimension&quot; and &quot;layer&quot; interchangeably in the following context.&#10;&#09;Since our training loss penalizes adjacent objects having the same embedding vector, only spatially separated objects can be assigned to the same layer, eliminate crowding and overlapping of objects. Therefore, instances can be effortlessly obtained by finding connected components in each layer.&#10;&#10;The main contributions of our work are as follows: We propose (1) a novel approach to eliminate crowdedness and overlap of instances by layering objects into different output layers; (2) an approach of spontaneous object layering through deep model learning; (3) a concise and effective framework for segmentation of densely distributed objects without data-specific post-processing efforts. &#10;&#10;To our best knowledge, our work is the first pixel embedding based approach that does not require explicit pixel clustering, and is capable of handling object overlap. Competing with several state-of-the-art approaches, our method yields comparable or better results on a diverse line of datasets: C. elegans~\cite{bbbc} (BBBC), overlapping cervical cells~\cite{occ14-B,occ14-A} (OCC) and cultured neuroblastoma cells~\cite{ccdb} (CCDB).&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07057" label="2110.07057">
-        <attvalues>
-          <attvalue for="0" value="High-throughput Phenotyping of Nematode Cysts" />
-          <attvalue for="1" value="  The beet cyst nematode (BCN) Heterodera schachtii is a plant pest responsible&#10;for crop loss on a global scale. Here, we introduce a high-throughput system&#10;based on computer vision that allows quantifying BCN infestation and&#10;characterizing nematode cysts through phenotyping. After recording microscopic&#10;images of soil extracts in a standardized setting, an instance segmentation&#10;algorithm serves to detect nematode cysts in these samples. Going beyond fast&#10;and precise cyst counting, the image-based approach enables quantification of&#10;cyst density and phenotyping of morphological features of cysts under different&#10;conditions, providing the basis for high-throughput applications in agriculture&#10;and plant breeding research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.00868" label="1801.00868">
-        <attvalues>
-          <attvalue for="0" value="Panoptic Segmentation" />
-          <attvalue for="1" value="  We propose and study a task we name panoptic segmentation (PS). Panoptic&#10;segmentation unifies the typically distinct tasks of semantic segmentation&#10;(assign a class label to each pixel) and instance segmentation (detect and&#10;segment each object instance). The proposed task requires generating a coherent&#10;scene segmentation that is rich and complete, an important step toward&#10;real-world vision systems. While early work in computer vision addressed&#10;related image/scene parsing tasks, these are not currently popular, possibly&#10;due to lack of appropriate metrics or associated recognition challenges. To&#10;address this, we propose a novel panoptic quality (PQ) metric that captures&#10;performance for all classes (stuff and things) in an interpretable and unified&#10;manner. Using the proposed metric, we perform a rigorous study of both human&#10;and machine performance for PS on three existing datasets, revealing&#10;interesting insights about the task. The aim of our work is to revive the&#10;interest of the community in a more unified view of image segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.15452" label="2205.15452">
-        <attvalues>
-          <attvalue for="0" value="MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  We present MVMO (Multi-View, Multi-Object dataset): a synthetic dataset of&#10;116,000 scenes containing randomly placed objects of 10 distinct classes and&#10;captured from 25 camera locations in the upper hemisphere. MVMO comprises&#10;photorealistic, path-traced image renders, together with semantic segmentation&#10;ground truth for every view. Unlike existing multi-view datasets, MVMO features&#10;wide baselines between cameras and high density of objects, which lead to large&#10;disparities, heavy occlusions and view-dependent object appearance. Single view&#10;semantic segmentation is hindered by self and inter-object occlusions that&#10;could benefit from additional viewpoints. Therefore, we expect that MVMO will&#10;propel research in multi-view semantic segmentation and cross-view semantic&#10;transfer. We also provide baselines that show that new research is needed in&#10;such fields to exploit the complementary information of multi-view setups.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The task of semantic segmentation~\cite{longFullyConvolutionalNetworks2015} aims at, given an input image, performing pixel-wise classification over a predefined set of categories.&#10;As in many other dense prediction problems, the end-to-end convolutional neural networks (CNN)-based fully supervised approach to this task has become the de facto standard to solve it, leading to robustly performing models~\cite{ronnebergerUNetConvolutionalNetworks2015} at the expense of a large amount of human annotations.&#10;Nevertheless, understanding scenes based on a single 2D input is challenging when applied on&#10;\begin{enumerate*}[label=(\roman*)]&#10; \item scenes with significant inter-object and self-occlusions that hide class-distinctive features&#10; \item scenes covering a wide spatial range, where distant objects can show a small apparent size.&#10;\end{enumerate*}&#10;&#10;In this context, we hypothesize that posing data-driven models that exploit multi-view camera setups that provide complementary information over the imaged scenes could be of potential interest for improving the results obtained by single-camera baselines.&#10;However, so far multi-view semantic segmentation has primarily been approached for close-baseline setups~\cite{RosCVPR16} i.e. those where the distance between cameras (and thus, the resulting disparities) are small, whereas solving the aforementioned obstacles requires wide baselines.&#10;Scenarios that could benefit from this approach are frequent in real life, in domains as diverse as industry (e.g. conveyor belts), surveillance, or traffic management.&#10;&#10;In this paper, we introduce MVMO, the Multi-View Multi-Object dataset, which addresses the current lack of publicly available large-scale datasets of densely annotated wide-baseline multi-view scenes containing multiple objects.&#10;MVMO is a synthetic, path tracing-based set of 116,000 scenes with per-view semantic segmentation annotations of 10 object categories.&#10;Each scene is observed from $25$ camera locations distributed uniformly in the upper hemisphere (see Fig.~\ref{fig_mvmo_samples}). Unlike most existing multi-view image datasets (which are designed to be camera-centric and exhibit very close baselines while sensing their surroundings~\cite{RosCVPR16}), MVMO features wide baselines between many camera pairs as a result of a scene-centric design, and a large amount of objects per scene.&#10;This leads to large disparities, notable occlusions and variable apparent object geometry, size and surface appearances across views.&#10;Therefore, MVMO sets a particularly challenging arrangement that aims at contributing to push research on the fields of multi-view semantic-segmentation and cross-view semantic knowledge transfer.&#10;The experiments presented show that simple baselines fail to be of much help in transferring learned models to novel views, hence suggesting the need for novel research in this direction.&#10;&#10;\minisection{Related work.} Our work relates to a number of previous datasets from various research fields, some of which already leverage wide-baseline multi-view datasets in an attempt to improve upon their respective single-view performances:&#10;In multi-view object detection, \cite{roigConditionalRandomFields2011} introduces a multi object detection dataset with bounding box annotations for pedestrians, cars and buses from 6 calibrated cameras.&#10;Advances on multi-view human pose estimation were possible by leveraging various wide baseline datasets over RGB~\cite{kazemiMultiviewBodyPart2013,yaoMONETMultiviewSemiSupervised2019,ionescuHuman36MLarge2014} and depth~\cite{haque2016viewpoint} images of both groups~\cite{jooPanopticStudioMassively2015} and individuals.&#10;&#10;The field of multi-view semantic segmentation (see Table~\ref{tab_datasets_comparison}) has been addressed from diverse perspectives.&#10;Many early works prior to the irruption of deep learning techniques focused on the binary segmentation of a single static foreground object from a sequence of close-baseline views from a class-agnostic point of view~\cite{campbellAutomatic3DObject2010, leeSilhouetteSegmentationMultiple2011}, often learning sequence-specific models and relying on diverse cues: object-background color distributions, central object fixation or stereo geometry constraints.&#10;More recently, \cite{yaoMultiviewCosegmentationWide2020} used deep self-supervised training to extend the single subject segmentation task to three dynamic scenes in wide-baseline setups.&#10;&#10;Multi-class multi-view semantic segmentation poses harder challenges and calls for larger datasets. Different works leverage the complementary information provided by additional views:&#10;\cite{ladickyJointOptimizationObject2012} extends the Leuven stereo dataset with semantic labels in one of the views to jointly train for segmentation and stereo reconstruction.&#10;A few works focus on cross-view semantic transfer, with an unsupervised transfer of the semantic annotations to new label-free views, e.g. ground to aerial views~\cite{zhaiPredictingGroundLevelScene2017} or among distinct vehicle-mounted cameras~\cite{Coors2019THREEDV} in close-baseline footage from~\cite{dosovitskiy2017carla}. &#10;Both tasks demand datasets comprising two or more 2D RGB views, with annotations in each of them.&#10;SYNTHIA~\cite{RosCVPR16} provides pixel-wise depth and semantic labels for a large synthetic set of scenes captured from a vehicle-mounted 8 RGB camera-rig, thus showing the usual narrow baseline of camera-centric driving setups.&#10;The wide baseline scenario has so far only been tackled by the Human3.6M~\cite{ionescuHuman36MLarge2014} and 3DPeople~\cite{pumarola20193dpeople} datasets.&#10;They both provide body part~\cite{ionescuHuman36MLarge2014,pumarola20193dpeople} or clothing~\cite{pumarola20193dpeople} segmentations, but~\cite{pumarola20193dpeople} has immutable 2D backgrounds, and they are both restricted to single subjects and thus limited in the severity of the occlusions and subject size variation across views.&#10;&#10;Several recent papers~\cite{riemenschneiderLearningWhereClassify2014,maMultiviewDeepLearning2017,dai3DMVJoint3DMultiView2018} leverage the spatial consistencies in temporal sequences of RGB or RGB-D images with small relative baselines among them to address semantic segmentation of either 2D images or their reconstructed 3D representations. The raw sequences of the NYUv2~\cite{silbermanIndoorSegmentationSupport2012}, Camvid, ETHZ RueMonge 2014~\cite{riemenschneiderLearningWhereClassify2014} or the ScanNet~\cite{daiScanNetRichlyAnnotated3D2017} datasets are commonly used to achieve this.&#10;Furthermore, various large scale 3D virtual or reconstructed environments have been released.&#10;Their relevance comes from the fact that, through significant user intervention, parts of the 3D model and associated labels could be projected back to 2D to synthesise semantically annotated multi-view image sets from arbitrary camera locations with different degrees of realism. &#10;The House3D~\cite{wuBuildingGeneralizableAgents2018}, Gibson~\cite{xiaGibsonEnvRealWorld2018} and CARLA~\cite{dosovitskiy2017carla} environments are some relevant examples, although only CARLA, being fully virtual, could yield high object densities via its API.&#10;This was shown in~\cite{panCrossviewSemanticSegmentation2019} for close baseline setups, proposing a multi-view semantic fusion scheme from up to 8 input views onto a new virtual zenithal view.&#10;&#10;In conclusion, MVMO covers the lack of a standardised large scale photo-realistic multi-view dataset with wide-baselines (and hence, large disparities and relevant occlusions) across cameras and comprising semantic segmentation annotations for multiple objects of distinct classes.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Computer Vision, Mathematics, Physics, Multi-View Imaging, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1605.06211" label="1605.06211">
-        <attvalues>
-          <attvalue for="0" value="Fully Convolutional Networks for Semantic Segmentation" />
-          <attvalue for="1" value="  Convolutional networks are powerful visual models that yield hierarchies of&#10;features. We show that convolutional networks by themselves, trained&#10;end-to-end, pixels-to-pixels, improve on the previous best result in semantic&#10;segmentation. Our key insight is to build &quot;fully convolutional&quot; networks that&#10;take input of arbitrary size and produce correspondingly-sized output with&#10;efficient inference and learning. We define and detail the space of fully&#10;convolutional networks, explain their application to spatially dense prediction&#10;tasks, and draw connections to prior models. We adapt contemporary&#10;classification networks (AlexNet, the VGG net, and GoogLeNet) into fully&#10;convolutional networks and transfer their learned representations by&#10;fine-tuning to the segmentation task. We then define a skip architecture that&#10;combines semantic information from a deep, coarse layer with appearance&#10;information from a shallow, fine layer to produce accurate and detailed&#10;segmentations. Our fully convolutional network achieves improved segmentation&#10;of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT&#10;Flow, and PASCAL-Context, while inference takes one tenth of a second for a&#10;typical image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.00104" label="1806.00104">
-        <attvalues>
-          <attvalue for="0" value="MONET: Multiview Semi-supervised Keypoint Detection via Epipolar&#10;  Divergence" />
-          <attvalue for="1" value="  This paper presents MONET -- an end-to-end semi-supervised learning framework&#10;for a keypoint detector using multiview image streams. In particular, we&#10;consider general subjects such as non-human species where attaining a large&#10;scale annotated dataset is challenging. While multiview geometry can be used to&#10;self-supervise the unlabeled data, integrating the geometry into learning a&#10;keypoint detector is challenging due to representation mismatch. We address&#10;this mismatch by formulating a new differentiable representation of the&#10;epipolar constraint called epipolar divergence---a generalized distance from&#10;the epipolar lines to the corresponding keypoint distribution. Epipolar&#10;divergence characterizes when two view keypoint distributions produce zero&#10;reprojection error. We design a twin network that minimizes the epipolar&#10;divergence through stereo rectification that can significantly alleviate&#10;computational complexity and sampling aliasing in training. We demonstrate that&#10;our framework can localize customized keypoints of diverse species, e.g.,&#10;humans, dogs, and monkeys.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.02709" label="1612.02709">
-        <attvalues>
-          <attvalue for="0" value="Predicting Ground-Level Scene Layout from Aerial Imagery" />
-          <attvalue for="1" value="  We introduce a novel strategy for learning to extract semantically meaningful&#10;features from aerial imagery. Instead of manually labeling the aerial imagery,&#10;we propose to predict (noisy) semantic features automatically extracted from&#10;co-located ground imagery. Our network architecture takes an aerial image as&#10;input, extracts features using a convolutional neural network, and then applies&#10;an adaptive transformation to map these features into the ground-level&#10;perspective. We use an end-to-end learning approach to minimize the difference&#10;between the semantic segmentation extracted directly from the ground image and&#10;the semantic segmentation predicted solely based on the aerial image. We show&#10;that a model learned using this strategy, with no additional training, is&#10;already capable of rough semantic labeling of aerial imagery. Furthermore, we&#10;demonstrate that by finetuning this model we can achieve more accurate semantic&#10;segmentation than two baseline initialization strategies. We use our network to&#10;address the task of estimating the geolocation and geoorientation of a ground&#10;image. Finally, we show how features extracted from an aerial image can be used&#10;to hallucinate a plausible ground-level panorama.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.03938" label="1711.03938">
-        <attvalues>
-          <attvalue for="0" value="CARLA: An Open Urban Driving Simulator" />
-          <attvalue for="1" value="  We introduce CARLA, an open-source simulator for autonomous driving research.&#10;CARLA has been developed from the ground up to support development, training,&#10;and validation of autonomous urban driving systems. In addition to open-source&#10;code and protocols, CARLA provides open digital assets (urban layouts,&#10;buildings, vehicles) that were created for this purpose and can be used freely.&#10;The simulation platform supports flexible specification of sensor suites and&#10;environmental conditions. We use CARLA to study the performance of three&#10;approaches to autonomous driving: a classic modular pipeline, an end-to-end&#10;model trained via imitation learning, and an end-to-end model trained via&#10;reinforcement learning. The approaches are evaluated in controlled scenarios of&#10;increasing difficulty, and their performance is examined via metrics provided&#10;by CARLA, illustrating the platform's utility for autonomous driving research.&#10;The supplementary video can be viewed at https://youtu.be/Hp8Dz-Zek2E&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04571" label="1904.04571">
-        <attvalues>
-          <attvalue for="0" value="3DPeople: Modeling the Geometry of Dressed Humans" />
-          <attvalue for="1" value="  Recent advances in 3D human shape estimation build upon parametric&#10;representations that model very well the shape of the naked body, but are not&#10;appropriate to represent the clothing geometry. In this paper, we present an&#10;approach to model dressed humans and predict their geometry from single images.&#10;We contribute in three fundamental aspects of the problem, namely, a new&#10;dataset, a novel shape parameterization algorithm and an end-to-end deep&#10;generative network for predicting shape.&#10;  First, we present 3DPeople, a large-scale synthetic dataset with 2.5 Million&#10;photo-realistic images of 80 subjects performing 70 activities and wearing&#10;diverse outfits. Besides providing textured 3D meshes for clothes and body, we&#10;annotate the dataset with segmentation masks, skeletons, depth, normal maps and&#10;optical flow. All this together makes 3DPeople suitable for a plethora of&#10;tasks.&#10;  We then represent the 3D shapes using 2D geometry images. To build these&#10;images we propose a novel spherical area-preserving parameterization algorithm&#10;based on the optimal mass transportation method. We show this approach to&#10;improve existing spherical maps which tend to shrink the elongated parts of the&#10;full body models such as the arms and legs, making the geometry images&#10;incomplete.&#10;  Finally, we design a multi-resolution deep generative network that, given an&#10;input image of a dressed human, predicts his/her geometry image (and thus the&#10;clothed body shape) in an end-to-end manner. We obtain very promising results&#10;in jointly capturing body pose and clothing shape, both for synthetic&#10;validation and on the wild images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.08866" label="1703.08866">
-        <attvalues>
-          <attvalue for="0" value="Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D&#10;  Cameras" />
-          <attvalue for="1" value="  Visual scene understanding is an important capability that enables robots to&#10;purposefully act in their environment. In this paper, we propose a novel&#10;approach to object-class segmentation from multiple RGB-D views using deep&#10;learning. We train a deep neural network to predict object-class semantics that&#10;is consistent from several view points in a semi-supervised way. At test time,&#10;the semantics predictions of our network can be fused more consistently in&#10;semantic keyframe maps than predictions of a network trained on individual&#10;views. We base our network architecture on a recent single-view deep learning&#10;approach to RGB and depth fusion for semantic object-class segmentation and&#10;enhance it with multi-scale loss minimization. We obtain the camera trajectory&#10;using RGB-D SLAM and warp the predictions of RGB-D images into ground-truth&#10;annotated frames in order to enforce multi-view consistency during training. At&#10;test time, predictions from multiple views are fused into keyframes. We propose&#10;and analyze several methods for enforcing multi-view consistency during&#10;training and testing. We evaluate the benefit of multi-view consistency&#10;training and demonstrate that pooling of deep features and fusion over multiple&#10;views outperforms single-view baselines on the NYUDv2 benchmark for semantic&#10;segmentation. Our end-to-end trained network achieves state-of-the-art&#10;performance on the NYUDv2 dataset in single-view segmentation as well as&#10;multi-view semantic fusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.10409" label="1803.10409">
-        <attvalues>
-          <attvalue for="0" value="3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation" />
-          <attvalue for="1" value="  We present 3DMV, a novel method for 3D semantic scene segmentation of RGB-D&#10;scans in indoor environments using a joint 3D-multi-view prediction network. In&#10;contrast to existing methods that either use geometry or RGB data as input for&#10;this task, we combine both data modalities in a joint, end-to-end network&#10;architecture. Rather than simply projecting color data into a volumetric grid&#10;and operating solely in 3D -- which would result in insufficient detail -- we&#10;first extract feature maps from associated RGB images. These features are then&#10;mapped into the volumetric feature grid of a 3D network using a differentiable&#10;backprojection layer. Since our target is 3D scanning scenarios with possibly&#10;many frames, we use a multi-view pooling approach in order to handle a varying&#10;number of RGB input views. This learned combination of RGB and geometric&#10;features with our joint 2D-3D architecture achieves significantly better&#10;results than existing baselines. For instance, our final result on the ScanNet&#10;3D segmentation benchmark increases from 52.8\% to 75\% accuracy compared to&#10;existing volumetric architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.02209" label="1801.02209">
-        <attvalues>
-          <attvalue for="0" value="Building Generalizable Agents with a Realistic and Rich 3D Environment" />
-          <attvalue for="1" value="  Teaching an agent to navigate in an unseen 3D environment is a challenging&#10;task, even in the event of simulated environments. To generalize to unseen&#10;environments, an agent needs to be robust to low-level variations (e.g. color,&#10;texture, object changes), and also high-level variations (e.g. layout changes&#10;of the environment). To improve overall generalization, all types of variations&#10;in the environment have to be taken under consideration via different level of&#10;data augmentation steps. To this end, we propose House3D, a rich, extensible&#10;and efficient environment that contains 45,622 human-designed 3D scenes of&#10;visually realistic houses, ranging from single-room studios to multi-storied&#10;houses, equipped with a diverse set of fully labeled 3D objects, textures and&#10;scene layouts, based on the SUNCG dataset (Song et.al.). The diversity in&#10;House3D opens the door towards scene-level augmentation, while the label-rich&#10;nature of House3D enables us to inject pixel- &amp; task-level augmentations such&#10;as domain randomization (Toubin et. al.) and multi-task training. Using a&#10;subset of houses in House3D, we show that reinforcement learning agents trained&#10;with an enhancement of different levels of augmentations perform much better in&#10;unseen environments than our baselines with raw RGB input by over 8% in terms&#10;of navigation success rate. House3D is publicly available at&#10;http://github.com/facebookresearch/House3D.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.10654" label="1808.10654">
-        <attvalues>
-          <attvalue for="0" value="Gibson Env: Real-World Perception for Embodied Agents" />
-          <attvalue for="1" value="  Developing visual perception models for active agents and sensorimotor&#10;control are cumbersome to be done in the physical world, as existing algorithms&#10;are too slow to efficiently learn in real-time and robots are fragile and&#10;costly. This has given rise to learning-in-simulation which consequently casts&#10;a question on whether the results transfer to real-world. In this paper, we are&#10;concerned with the problem of developing real-world perception for active&#10;agents, propose Gibson Virtual Environment for this purpose, and showcase&#10;sample perceptual tasks learned therein. Gibson is based on virtualizing real&#10;spaces, rather than using artificially designed ones, and currently includes&#10;over 1400 floor spaces from 572 full buildings. The main characteristics of&#10;Gibson are: I. being from the real-world and reflecting its semantic&#10;complexity, II. having an internal synthesis mechanism, &quot;Goggles&quot;, enabling&#10;deploying the trained models in real-world without needing further domain&#10;adaptation, III. embodiment of agents and making them subject to constraints of&#10;physics and space.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.03560" label="1906.03560">
-        <attvalues>
-          <attvalue for="0" value="Cross-view Semantic Segmentation for Sensing Surroundings" />
-          <attvalue for="1" value="  Sensing surroundings plays a crucial role in human spatial perception, as it&#10;extracts the spatial configuration of objects as well as the free space from&#10;the observations. To facilitate the robot perception with such a surrounding&#10;sensing capability, we introduce a novel visual task called Cross-view Semantic&#10;Segmentation as well as a framework named View Parsing Network (VPN) to address&#10;it. In the cross-view semantic segmentation task, the agent is trained to parse&#10;the first-view observations into a top-down-view semantic map indicating the&#10;spatial location of all the objects at pixel-level. The main issue of this task&#10;is that we lack the real-world annotations of top-down-view data. To mitigate&#10;this, we train the VPN in 3D graphics environment and utilize the domain&#10;adaptation technique to transfer it to handle real-world data. We evaluate our&#10;VPN on both synthetic and real-world agents. The experimental results show that&#10;our model can effectively make use of the information from different views and&#10;multi-modalities to understanding spatial information. Our further experiment&#10;on a LoCoBot robot shows that our model enables the surrounding sensing&#10;capability from 2D image input. Code and demo videos can be found at&#10;\url{https://view-parsing-network.github.io}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.09161" label="2012.09161">
-        <attvalues>
-          <attvalue for="0" value="Learning Continuous Image Representation with Local Implicit Image&#10;  Function" />
-          <attvalue for="1" value="  How to represent an image? While the visual world is presented in a&#10;continuous manner, machines store and see the images in a discrete way with 2D&#10;arrays of pixels. In this paper, we seek to learn a continuous representation&#10;for images. Inspired by the recent progress in 3D reconstruction with implicit&#10;neural representation, we propose Local Implicit Image Function (LIIF), which&#10;takes an image coordinate and the 2D deep features around the coordinate as&#10;inputs, predicts the RGB value at a given coordinate as an output. Since the&#10;coordinates are continuous, LIIF can be presented in arbitrary resolution. To&#10;generate the continuous representation for images, we train an encoder with&#10;LIIF representation via a self-supervised task with super-resolution. The&#10;learned continuous representation can be presented in arbitrary resolution even&#10;extrapolate to x30 higher resolution, where the training tasks are not&#10;provided. We further show that LIIF representation builds a bridge between&#10;discrete and continuous representation in 2D, it naturally supports the&#10;learning tasks with size-varied image ground-truths and significantly&#10;outperforms the method with resizing the ground-truths.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Implicit neural representation. In implicit neural representation, an object is usually represented as a multilayer perceptron (MLP) that maps coordinates to signal. This idea has been widely applied in modeling 3D object shapes~\cite{chen2019learning,michalkiewicz2019implicit,atzmon2020sal,gropp2020implicit}, 3D surfaces of the scene~\cite{sitzmann2019scene,jiang2020local,peng2020convolutional,chabra2020deep} as well as the appearance of the 3D structure~\cite{oechsle2019texture,niemeyer2020differentiable,mildenhall2020nerf}. For example, Mildenhall et al.~\cite{mildenhall2020nerf} propose to perform novel view synthesis by learning an implicit representation for a specific scene using multiple image views. Comparing to explicit 3D representations such as voxel, point cloud, and mesh, the continuous implicit representation can capture the very fine details of the shape with a small number of parameters. Its differentiable property also allows back-propagation through the model for neural rendering~\cite{sitzmann2019scene}. &#10;&#10;Learning implicit function space. Instead of learning an independent implicit neural representation for each object, recent works share a function space for the implicit representations of different objects. Typically, a latent space is defined where each object corresponds to a latent code. The latent code can be obtained by optimization with an auto-decoder~\cite{park2019deepsdf,chen2019learning}. For example, Park et al.~\cite{park2019deepsdf} propose to learn a Signed Distance Function (SDF) for each object shape and different SDFs can be inferred by changing the input latent codes. Recent work from Sitzmann et al.~\cite{sitzmann2020metasdf} also proposes a meta-learning-based method for sharing the function space. Instead of using auto-decoder, our work adopts the auto-encoder architecture~\cite{mescheder2019occupancy,chen2019learning,saito2019pifu,saito2020pifuhd,xu2019disn}, which gives an efficient and effective manner for sharing knowledge between a large variety of samples. For example, Mescheder et al.~\cite{mescheder2019occupancy} propose to estimate a latent code given an image as input, and use an occupancy function conditioning on this latent code to perform 3D reconstruction for the input object.&#10;&#10;Despite the success of implicit neural representation in 3D tasks, its applications in representing images are relatively underexplored. Early works~\cite{stanley2007compositional,mordvintsev2018differentiable} parameterize 2D images with compositional pattern producing networks. Chen et al.~\cite{chen2019learning} explore 2D shape generation from latent space for simple digits. Recently, Sitzmann et al.~\cite{sitzmann2020implicit} observe that previous implicit neural representation parameterized by MLP with ReLU~\cite{nair2010rectified} is incapable of representing fine details of natural images. They replace ReLU with periodic activation&#10;functions (sinusoidal) and demonstrates it can model the natural images in higher quality. However, none of these approaches can represent natural and complex images with high fidelity when sharing the implicit function space, while it is of limited generalization to higher precision if not to share the implicit function space. Related to recent works~\cite{saito2019pifu,genova2020local,chibane2020implicit,peng2020eccv,jiang2020local} on 3D implicit neural representation, LIIF representation is based on local latent codes, which can recover the fine details of natural and complex images. Similar formulations have been recently proposed for 3D reconstruction~\cite{jiang2020local} and super-resolving physics-constrained solution~\cite{jiang2020meshfreeflownet}. Different from these works, LIIF focuses on learning continuous image representation and has image-specific design choices (e.g. cell decoding).&#10;&#10;Image generation and super-resolution. Our work is related to the general image-to-image translation tasks~\cite{zhang2016colorful,pix2pix2017,zhu2017unpaired,hoffman2018cycada,denton2015deep}, where one image is given as input and it is translated to a different domain or format. For example, Isola et al.~\cite{pix2pix2017} propose conditional GANs~\cite{goodfellow2014generative} to perform multiple image translation tasks. Unlike the deconvolution-based approaches, LIIF representation supports performing realistic and high-resolution image generation by independently querying the pixels at different coordinates from the generated implicit representation. While LIIF is useful for general purposes, in this paper, we perform experiments on generating high-resolution images given low-resolution inputs, which is related to the image super-resolution tasks~\cite{chang2004super,timofte2013anchored,tai2017memnet,dong2014learning,ledig2017photo,lim2017enhanced,lai2017deep,zhang2018residual,zhang2020deep}. For example, Lai et al.~\cite{lai2017deep} propose a Laplacian Pyramid Network to progressively reconstruct the image. While related, we stress that this work on learning continuous representation is different from the traditional super-resolution setting. Most previous super-resolution models are designed for up-sampling with a specific scale (or a fixed set of scales), while our goal is to learn a continuous representation that can be presented in arbitrary high resolution. In this respect, our work is more related to MetaSR~\cite{hu2019meta} on magnification-arbitrary super-resolution. Their method generates a convolutional up-sampling layer by its meta-network, while it can perform arbitrary up-sampling in its training scales, it has limited performance on generalizing to the larger-scale synthesis that is out of training distribution. LIIF representation, on the other hand, when trained with tasks from $\times 1$ to $\times 4$, can generate $\times 30$ higher resolution image based on the continuous representation in one forward pass.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Image Representation, Computer Science, Neural Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2303.11328" label="2303.11328">
-        <attvalues>
-          <attvalue for="0" value="Zero-1-to-3: Zero-shot One Image to 3D Object" />
-          <attvalue for="1" value="  We introduce Zero-1-to-3, a framework for changing the camera viewpoint of an&#10;object given just a single RGB image. To perform novel view synthesis in this&#10;under-constrained setting, we capitalize on the geometric priors that&#10;large-scale diffusion models learn about natural images. Our conditional&#10;diffusion model uses a synthetic dataset to learn controls of the relative&#10;camera viewpoint, which allow new images to be generated of the same object&#10;under a specified camera transformation. Even though it is trained on a&#10;synthetic dataset, our model retains a strong zero-shot generalization ability&#10;to out-of-distribution datasets as well as in-the-wild images, including&#10;impressionist paintings. Our viewpoint-conditioned diffusion approach can&#10;further be used for the task of 3D reconstruction from a single image.&#10;Qualitative and quantitative experiments show that our method significantly&#10;outperforms state-of-the-art single-view 3D reconstruction and novel view&#10;synthesis models by leveraging Internet-scale pre-training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;3D generative models.&#10;Recent advancements in generative image architectures combined with large scale image-text datasets~\cite{schuhmann2022laion} have made it possible to synthesize high-fidelity of diverse scenes and objects~\cite{nichol2021glide,ramesh2021zero,saharia2022photorealistic}. In particular, diffusion models have shown to be very effective at learning scalable image generators using a denoising objective~\cite{chen2020wavegrad,sohl2015deep}. However, scaling them to the 3D domain would require large amounts of expensive annotated 3D data. Instead, recent approaches rely on transferring pre-trained large-scale 2D diffusion models to 3D without using any ground truth 3D data. Neural Radiance Fields or NeRFs~\cite{mildenhall2020nerf} have emerged as a powerful representation, thanks to their ability to encode scenes with high fidelity. Typically, NeRF is used for single-scene reconstruction, where many posed images covering the entire scene are provided. The task is then to predict novel views from unobserved angles. DreamFields~\cite{jain2022zero} has shown that NeRF is a more versatile tool that can also be used as the main component in a 3D generative system. Various follow-up works~\cite{poole2022dreamfusion,lin2022magic3d,wang2022score} substitute CLIP for a distillation loss from a 2D diffusion model that is repurposed to generate high-fidelity 3D objects and scenes from text inputs.&#10;&#10;Our work explores an unconventional approach to novel-view synthesis, modeling it as a viewpoint-conditioned image-to-image translation task with diffusion models. The learned model can also be combined with 3D distillation to reconstruct 3D shape from a single image. Prior work~\cite{watson2022novel} adopted a similar pipeline but did not demonstrate zero-shot generalization capability. Concurrent approaches~\cite{deng2022nerdi, melas2023realfusion, xu2022neurallift} proposed similar techniques to perform image-to-3D generation using language-guided priors and textual inversion~\cite{gal2022image}. In comparison, our method learns control of viewpoints through a synthetic dataset and demonstrates zero-shot generalization to in-the-wild images.&#10;&#10;Single-view object reconstruction.&#10;Reconstructing 3D objects from a single view is a highly challenging problem that requires strong priors. One line of work builds priors from relying on collections of 3D primitives represented as meshes~\cite{worchel2022multi,xu2019disn}, voxels~\cite{girdhar2016learning,wu2017marrnet}, or point clouds~\cite{fan2017point,mescheder2019occupancy}, and use image encoders for conditioning. These models are constrained by the variety of the used 3D data collection and show poor generalization capabilities due to the global nature of this type of conditioning. Moreover, they require an additional pose estimation step to ensure alignment between the estimated shape and the input. On the other hand, locally conditioned&#10;models~\cite{saito2019pifu,yu2021pixelnerf,wang2021ibrnet,tucker2020single,revealing}&#10;aim to use local image features directly for scene reconstruction and show greater cross-domain generalization capabilities, though are generally limited to close-by view reconstructions. Recently, MCC~\cite{wu2023multiview} learns a general-purpose representation for 3D reconstruction from RGB-D views and is trained on large-scale dataset of object-centric videos.&#10;&#10;In our work, we demonstrate that rich geometric information can be extracted directly from a pre-trained Stable Diffusion model, alleviating the need for additional depth information.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Deep Learning, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2204.07962" label="2204.07962">
-        <attvalues>
-          <attvalue for="0" value="An Extendable, Efficient and Effective Transformer-based Object Detector" />
-          <attvalue for="1" value="  Transformers have been widely used in numerous vision problems especially for&#10;visual recognition and detection. Detection transformers are the first fully&#10;end-to-end learning systems for object detection, while vision transformers are&#10;the first fully transformer-based architecture for image classification. In&#10;this paper, we integrate Vision and Detection Transformers (ViDT) to construct&#10;an effective and efficient object detector. ViDT introduces a reconfigured&#10;attention module to extend the recent Swin Transformer to be a standalone&#10;object detector, followed by a computationally efficient transformer decoder&#10;that exploits multi-scale features and auxiliary techniques essential to boost&#10;the detection performance without much increase in computational load. In&#10;addition, we extend it to ViDT+ to support joint-task learning for object&#10;detection and instance segmentation. Specifically, we attach an efficient&#10;multi-scale feature fusion layer and utilize two more auxiliary training&#10;losses, IoU-aware loss and token labeling loss. Extensive evaluation results on&#10;the Microsoft COCO benchmark dataset demonstrate that ViDT obtains the best AP&#10;and latency trade-off among existing fully transformer-based object detectors,&#10;and its extended ViDT+ achieves 53.2AP owing to its high scalability for large&#10;models. The source code and trained models are available at&#10;https://github.com/naver-ai/vidt.&#10;" />
-          <attvalue for="2" value="&#10;&#10;O{bject} detection aims to predict both the bounding box and object class for each object of interest in an image.&#10;Recent deep object detectors rely heavily on meticulously designed components, such as anchor generation and non-maximum suppression\,\cite{papageorgiou2000trainable, liu2020deep}.&#10;As a result, the performance of these object detectors depends on specific postprocessing steps, which involve complex pipelines and make fully end-to-end training difficult.&#10;&#10;Motivated by the recent success of Transformers\,\cite{vaswani2017attention} in NLP, numerous models &#10;have been developed for various vision tasks, especially in recognition and detection. &#10;Carion et al.\,\cite{carion2020end} propose the Detection Transformers\,(DETR) to replace the meticulously designed components with a transformer encoder and decoder architecture, which serves as a neck component to bridge a CNN body for feature extraction and a detector head for prediction.&#10;As such, DETR enables end-to-end training of deep object detectors. &#10;On the other hand, Dosovitskiy et al.\,\cite{dosovitskiy2021image} show that a fully-transformer&#10;backbone without any convolutional layers, Vision Transformer (ViT), achieves state-of-the-art results on image classification benchmarks.&#10;DETR and ViT have been shown to learn effective representation models without relying strongly on human inductive biases, \eg, meticulously designed components in object detection\,(DETR), convolutional layers and pooling mechanisms for locality-aware designs\,(ViT). &#10;However, no attempt has been made to synergize DETR and ViT for a better object detection architecture.&#10;In this work, we integrate both approaches to construct a fully transformer-based, end-to-end object detector that achieves state-of-the-art performance without increasing computational load. &#10;&#10;Straightforward integration of DETR and ViT can be achieved by replacing the ResNet backbone (body) of DETR with ViT as shown in Figure \ref{fig:pipelines}(a).&#10;This naive integration, {DETR\,(ViT)}, has two limitations. &#10;First, as the original ViT suffers from the quadratic increase in complexity \wrt image size, this approach does not scale up well. &#10;Furthermore, the attention operation at the transformer encoder and decoder (\ie, the ``neck'' component) adds significant computational overhead to the detector. &#10;Thus, the naive integration of DETR and ViT would cause high latency, as shown in the blue lines of Figure \ref{fig:ap_vs_latency}. &#10;&#10;Recently, Fang et al.\,\cite{fang2021you} propose the YOLOS model &#10;by appending the detection tokens $[\mathtt{DET}]$ to the patch tokens $[\mathtt{PATCH}]$\, (see Figure \ref{fig:pipelines}(b)), where $[\mathtt{DET}]$ tokens are learnable embeddings to specify different objects to detect.&#10;YOLOS is a neck-free architecture and removes the additional computational costs from the neck encoder.&#10;However, YOLOS shows limited performance because it cannot exploit additional optimization techniques on the neck architecture, \eg, multi-scale features and auxiliary loss. &#10;In addition, YOLOS can only accommodate the original transformer due to its architectural limitation, resulting in a quadratic complexity \wrt the input size.\looseness=-1&#10;&#10;In this paper, we propose a novel integration of {\underline{Vi}}sion and {\underline{D}}etection {\underline{T}}ransformers\,({{ViDT}})\,(see Figure \ref{fig:pipelines}(c)).&#10;Our contributions are three-fold.&#10;{First}, ViDT introduces the Reconfigured Attention Module\,(RAM), to facilitate any ViT variant to handle the appended $[\mathtt{DET}]$ and $[\mathtt{PATCH}]$ tokens for object detection. &#10;Thus, we can integrate the Swin Transformer~\cite{liu2021swin} backbone with RAM to be an object detector and obtain high scalability using its local attention mechanism with linear complexity.&#10;{Second}, ViDT adopts a lightweight encoder-free neck architecture to reduce the computational overhead &#10;while still enabling the additional optimization techniques on the neck module. &#10;Note that the neck encoder is unnecessary because RAM directly extracts fine-grained representation for object detection, \ie, $[\mathtt{DET]}$ tokens.&#10;As a result, ViDT achieves better performance than its neck-free counterparts. &#10;{&#10;Finally, we extend the vanilla ViDT to an end-to-end architecture named {ViDT+}, thereby enabling multi-task learning of object detection and instance segmentation. &#10;For effective multi-task learning, ViDT+ equips efficient feature pyramid layers on top of its body for multi-scale feature fusion, and leverages two additional training losses i.e., IoU-aware loss\,\cite{wu2020iou} and token labeling loss\,\cite{jiang2021all}. ViDT+ only adds 1M learnable parameters and barely reduces its inference speed than ViDT, but achieves a significant accuracy gain.&#10;}&#10;&#10;ViDT has {three} architectural advantages over existing approaches. &#10;First, similar to YOLOS, ViDT takes $[\mathtt{DET}]$ tokens as the additional input, maintaining a fixed scale for object detection, but constructs hierarchical representations starting with small-sized image patches for $[\mathtt{PATCH}]$ tokens.&#10;Second, ViDT can use the hierarchical\,(multi-scale) features and additional techniques without a significant computation overhead. Thus, as a fully transformer-based object detector, ViDT facilitates better integration of vision and detection transformers. &#10;{&#10;Third, ViDT accommodates numerous tasks and transformer models. &#10;It can be extended to be an end-to-end architecture for multi-task learning, and easily combined with ViT variants such as CoaT\,\cite{xu2021co} and PVT-v2\,\cite{wang2021pvtv2} other than the Swin Transformer.&#10;}&#10;Extensive experiments on Microsoft COCO~\cite{lin2014microsoft} show that ViDT is highly scalable even for large ViT models, such as Swin-base with 0.1 billion parameters, and achieves the best AP and latency trade-off among existing fully transformer-based detectors. &#10;{&#10;In particular, ViDT+ with Swin-base achieves 53.2AP for object detection, which is $2.6$AP higher than that of the vanilla ViDT.&#10;}&#10;&#10;This work is an extensive version of our ICLR 2022\,\cite{song2022vidt}. Compared to the ICLR 2022 work, this paper includes the following additional contributions: (1) a new joint-learning framework named ViDT+, an extension of the vanilla ViDT, by incorporating three additional components: an efficient pyramid feature fusion module, a unified query representation module, and an IoU-aware and token labeling losses, (2) a detailed computational complexity analysis of the proposed VIDT compared to the YOLOS detector, (3) a performance comparison for object detection and instance segmentation compared with other CNN-based state-of-the-art methods, (4) an ablation study for the three new components of ViDT+ and a complete analysis of all the proposed components, (5) the reconfigured attention module combined with ViT variants such as CoaT and PVT-v2 other than the Swin Transformer.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Computer Science, Transformer Architecture" />
-        </attvalues>
-      </node>
-      <node id="1706.03762" label="1706.03762">
-        <attvalues>
-          <attvalue for="0" value="Attention Is All You Need" />
-          <attvalue for="1" value="  The dominant sequence transduction models are based on complex recurrent or&#10;convolutional neural networks in an encoder-decoder configuration. The best&#10;performing models also connect the encoder and decoder through an attention&#10;mechanism. We propose a new simple network architecture, the Transformer, based&#10;solely on attention mechanisms, dispensing with recurrence and convolutions&#10;entirely. Experiments on two machine translation tasks show these models to be&#10;superior in quality while being more parallelizable and requiring significantly&#10;less time to train. Our model achieves 28.4 BLEU on the WMT 2014&#10;English-to-German translation task, improving over the existing best results,&#10;including ensembles by over 2 BLEU. On the WMT 2014 English-to-French&#10;translation task, our model establishes a new single-model state-of-the-art&#10;BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction&#10;of the training costs of the best models from the literature. We show that the&#10;Transformer generalizes well to other tasks by applying it successfully to&#10;English constituency parsing both with large and limited training data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.12872" label="2005.12872">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Object Detection with Transformers" />
-          <attvalue for="1" value="  We present a new method that views object detection as a direct set&#10;prediction problem. Our approach streamlines the detection pipeline,&#10;effectively removing the need for many hand-designed components like a&#10;non-maximum suppression procedure or anchor generation that explicitly encode&#10;our prior knowledge about the task. The main ingredients of the new framework,&#10;called DEtection TRansformer or DETR, are a set-based global loss that forces&#10;unique predictions via bipartite matching, and a transformer encoder-decoder&#10;architecture. Given a fixed small set of learned object queries, DETR reasons&#10;about the relations of the objects and the global image context to directly&#10;output the final set of predictions in parallel. The new model is conceptually&#10;simple and does not require a specialized library, unlike many other modern&#10;detectors. DETR demonstrates accuracy and run-time performance on par with the&#10;well-established and highly-optimized Faster RCNN baseline on the challenging&#10;COCO object detection dataset. Moreover, DETR can be easily generalized to&#10;produce panoptic segmentation in a unified manner. We show that it&#10;significantly outperforms competitive baselines. Training code and pretrained&#10;models are available at https://github.com/facebookresearch/detr.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.00666" label="2106.00666">
-        <attvalues>
-          <attvalue for="0" value="You Only Look at One Sequence: Rethinking Transformer in Vision through&#10;  Object Detection" />
-          <attvalue for="1" value="  Can Transformer perform 2D object- and region-level recognition from a pure&#10;sequence-to-sequence perspective with minimal knowledge about the 2D spatial&#10;structure? To answer this question, we present You Only Look at One Sequence&#10;(YOLOS), a series of object detection models based on the vanilla Vision&#10;Transformer with the fewest possible modifications, region priors, as well as&#10;inductive biases of the target task. We find that YOLOS pre-trained on the&#10;mid-sized ImageNet-1k dataset only can already achieve quite competitive&#10;performance on the challenging COCO object detection benchmark, e.g.,&#10;YOLOS-Base directly adopted from BERT-Base architecture can obtain 42.0 box AP&#10;on COCO val. We also discuss the impacts as well as limitations of current&#10;pre-train schemes and model scaling strategies for Transformer in vision&#10;through YOLOS. Code and pre-trained models are available at&#10;https://github.com/hustvl/YOLOS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Vision Transformer for Object Detection.&#10;There has been a lot of interest in combining CNNs with forms of self-attention mechanisms~\cite{atten} to improve object detection performance~\cite{Non-local, RelationNet, GCNet}, while recent works trend towards augmenting Transformer with CNNs (or CNN design).&#10;~\cite{ViT-RCNN} propose to use a pre-trained ViT as the feature extractor for a Faster R-CNN~\cite{Faster-R-CNN} object detector.&#10;Despite being effective, they fail to ablate the CNN architectures, region-wise pooling operations~\cite{SPP, FastRCNN, MaskRCNN} as well as hand-crafted components such as dense anchors~\cite{Faster-R-CNN} and NMS.&#10;Inspired by modern CNN architecture, some works~\cite{HaloNet, PVT, Swin, CoaT} introduce the pyramidal feature hierarchy and locality to Vision Transformer design, which largely boost the performance in dense prediction tasks including object detection.&#10;However, these architectures are performance-oriented and cannot reflect the properties of the canonical or vanilla Vision Transformer~\cite{ViT} that directly inherited from~\cite{Transformer}.&#10;Another series of work, the DEtection TRansformer (DETR) families~\cite{DETR, Deformable-DETR}, use a random initialized Transformer to encode \&amp; decode CNN features for object detection, which does not reveal the transferability of a pre-trained Transformer.&#10;&#10;UP-DETR~\cite{Up-detr} is probably the first to study the effects of unsupervised pre-training in the DETR framework, which proposes an ``object detection oriented'' unsupervised pre-training task tailored for Transformer encoder \&amp; decoder in DETR.&#10;In this paper, we argue for the characteristics of a pre-trained vanilla ViT in object detection, which is rare in the existing literature.&#10;&#10;Pre-training and Fine-tuning of Transformer.&#10;The textbook-style usage of Transformer~\cite{Transformer} follows a ``pre-training \&amp; fine-tuning'' paradigm.&#10;In NLP, Transformer-based models are often pre-trained on large corpora and then fine-tuned for different tasks at hand~\cite{BERT, GPT-1}.&#10;In computer vision, ~\cite{ViT} apply Transformer to image recognition at scale using modern vision transfer learning recipe~\cite{BiT}.&#10;They show that a standard Transformer encoder architecture is able to attain excellent results on mid-sized or small image recognition benchmarks (e.g, ImageNet-$1k$~\cite{ImageNet-1k}, CIFAR-$10$/$100$~\cite{CIFAR}, etc.) when pre-trained at sufficient scale (e.g, JFT-$300\mathrm{M}$~\cite{JFT}, ImageNet-$21k$~\cite{ImageNet-21k}).&#10;~\cite{DeiT} achieves competitive Top-$1$ accuracy by training Transformer on ImageNet-$1k$ only, and is also capable of transferring to smaller datasets~\cite{CIFAR, Pets, Flowers}.&#10;However, existing transfer learning literature of Transformer arrest in image-level recognition and does not touch more complex tasks in vision such as object detection, which is also widely used to benchmark CNNs transferability.&#10;&#10;Our work aims to bridge this gap.&#10;We study the performance and properties of ViT on the challenging COCO object detection benchmark~\cite{COCO} when pre-trained on the mid-sized ImageNet-$1k$ dataset~\cite{ImageNet-1k} using different strategies.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2103.14030" label="2103.14030">
-        <attvalues>
-          <attvalue for="0" value="Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" />
-          <attvalue for="1" value="  This paper presents a new vision Transformer, called Swin Transformer, that&#10;capably serves as a general-purpose backbone for computer vision. Challenges in&#10;adapting Transformer from language to vision arise from differences between the&#10;two domains, such as large variations in the scale of visual entities and the&#10;high resolution of pixels in images compared to words in text. To address these&#10;differences, we propose a hierarchical Transformer whose representation is&#10;computed with \textbf{S}hifted \textbf{win}dows. The shifted windowing scheme&#10;brings greater efficiency by limiting self-attention computation to&#10;non-overlapping local windows while also allowing for cross-window connection.&#10;This hierarchical architecture has the flexibility to model at various scales&#10;and has linear computational complexity with respect to image size. These&#10;qualities of Swin Transformer make it compatible with a broad range of vision&#10;tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and&#10;dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP&#10;on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its&#10;performance surpasses the previous state-of-the-art by a large margin of +2.7&#10;box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the&#10;potential of Transformer-based models as vision backbones. The hierarchical&#10;design and the shifted window approach also prove beneficial for all-MLP&#10;architectures. The code and models are publicly available&#10;at~\url{https://github.com/microsoft/Swin-Transformer}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.05992" label="1912.05992">
-        <attvalues>
-          <attvalue for="0" value="IoU-aware Single-stage Object Detector for Accurate Localization" />
-          <attvalue for="1" value="  Due to the simpleness and high efficiency, single-stage object detectors have&#10;been widely applied in many computer vision applications . However, the low&#10;correlation between the classification score and localization accuracy of the&#10;predicted detections has severely hurt the localization accuracy of models. In&#10;this paper, IoU-aware single-stage object detector is proposed to solve this&#10;problem. Specifically, IoU-aware single-stage object detector predicts the IoU&#10;for each detected box. Then the classification score and predicted IoU are&#10;multiplied to compute the final detection confidence, which is more correlated&#10;with the localization accuracy. The detection confidence is then used as the&#10;input of the subsequent NMS and COCO AP computation, which will substantially&#10;improve the localization accuracy of models. Sufficient experiments on COCO and&#10;PASCAL VOC datasets demonstrate the effectiveness of IoU-aware single-stage&#10;object detector on improving model's localization accuracy. Without whistles&#10;and bells, the proposed method can substantially improve AP by $1.7\%\sim1.9\%$&#10;and AP75 by $2.2\%\sim2.5\%$ on COCO \textit{test-dev}. On PASCAL VOC, the&#10;proposed method can substantially improve AP by $2.9\%\sim4.4\%$ and AP80, AP90&#10;by $4.6\%\sim10.2\%$. Code is available here:&#10;{https://github.com/ShengkaiWu/IoU-aware-single-stage-object-detector}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.10858" label="2104.10858">
-        <attvalues>
-          <attvalue for="0" value="All Tokens Matter: Token Labeling for Training Better Vision&#10;  Transformers" />
-          <attvalue for="1" value="  In this paper, we present token labeling -- a new training objective for&#10;training high-performance vision transformers (ViTs). Different from the&#10;standard training objective of ViTs that computes the classification loss on an&#10;additional trainable class token, our proposed one takes advantage of all the&#10;image patch tokens to compute the training loss in a dense manner.&#10;Specifically, token labeling reformulates the image classification problem into&#10;multiple token-level recognition problems and assigns each patch token with an&#10;individual location-specific supervision generated by a machine annotator.&#10;Experiments show that token labeling can clearly and consistently improve the&#10;performance of various ViT models across a wide spectrum. For a vision&#10;transformer with 26M learnable parameters serving as an example, with token&#10;labeling, the model can achieve 84.4% Top-1 accuracy on ImageNet. The result&#10;can be further increased to 86.4% by slightly scaling the model size up to&#10;150M, delivering the minimal-sized model among previous models (250M+) reaching&#10;86%. We also show that token labeling can clearly improve the generalization&#10;capability of the pre-trained models on downstream tasks with dense prediction,&#10;such as semantic segmentation. Our code and all the training details will be&#10;made publicly available at https://github.com/zihangJiang/TokenLabeling.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Transformers~\cite{vaswani2017attention} refer to the models that entirely rely on the self-attention mechanism to build global dependencies, which are originally designed for natural language processing tasks.&#10;Due to their strong capability of capturing spatial information, transformers have also been successfully applied to a variety of vision problems, including low-level vision tasks like image enhancement~\cite{chen2020pre,yang2020learning}, as well as more challenging tasks&#10;such as image classification~\cite{chen2020generative, dosovitskiy2020image}, object &#10;detection~\cite{carion2020end,dai2020up,zheng2020end,zhu2020deformable},&#10;segmentation~\cite{chen2020pre,sun2020rethinking,wang2020end} and image generation~\cite{parmar2018image}. &#10;Some works also extend transformers for video and 3D point cloud processing~\cite{zeng2020learning,zhao2020point,zhou2018end}.&#10;&#10;Vision Transformer (ViT) is one of the earlier attempts that achieved state-of-the-art performance on ImageNet classification, using pure transformers as basic building blocks.&#10;However, ViTs need pretraining on very large datasets, such as ImageNet-22k and JFT-300M,&#10;and huge computation resources to achieve comparable performance to ResNet~\cite{he2016deep} with a similar model size trained on ImageNet.&#10;Later, DeiT~\cite{touvron2020training} manages to tackle the data-inefficiency problem by simply adjusting the network architecture and adding an additional token along with the class token for Knowledge Distillation~\cite{hinton2015distilling, yuan2020revisiting} to improve model performance.&#10;&#10;Some recent works~\cite{chen2021crossvit,han2021transformer,wu2021cvt,yuan2021tokens} also attempt to introduce the local dependency into vision transformers by modifying the patch embedding block or the transformer block or both, leading to significant performance gains.&#10;Moreover, there are also some works~\cite{heo2021rethinking,liu2021swin,wang2021pyramid} adopting a pyramid structure to reduce the overall computation while maintaining the model's ability to capture low-level features.&#10;&#10;Unlike most aforementioned works that design new transformer blocks or transformer architectures, we attempt to improve vision transformers by studying the role of patch tokens that embed rich local information inside image patches.&#10;We show that by slightly tuning the structure of vision transformers and employing the proposed token labeling objective,&#10;we can achieve strong baselines for transformer models at different model size levels.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2104.06399" label="2104.06399">
-        <attvalues>
-          <attvalue for="0" value="Co-Scale Conv-Attentional Image Transformers" />
-          <attvalue for="1" value="  In this paper, we present Co-scale conv-attentional image Transformers&#10;(CoaT), a Transformer-based image classifier equipped with co-scale and&#10;conv-attentional mechanisms. First, the co-scale mechanism maintains the&#10;integrity of Transformers' encoder branches at individual scales, while&#10;allowing representations learned at different scales to effectively communicate&#10;with each other; we design a series of serial and parallel blocks to realize&#10;the co-scale mechanism. Second, we devise a conv-attentional mechanism by&#10;realizing a relative position embedding formulation in the factorized attention&#10;module with an efficient convolution-like implementation. CoaT empowers image&#10;Transformers with enriched multi-scale and contextual modeling capabilities. On&#10;ImageNet, relatively small CoaT models attain superior classification results&#10;compared with similar-sized convolutional neural networks and image/vision&#10;Transformers. The effectiveness of CoaT's backbone is also illustrated on&#10;object detection and instance segmentation, demonstrating its applicability to&#10;downstream computer vision tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.03921" label="2110.03921">
-        <attvalues>
-          <attvalue for="0" value="ViDT: An Efficient and Effective Fully Transformer-based Object Detector" />
-          <attvalue for="1" value="  Transformers are transforming the landscape of computer vision, especially&#10;for recognition tasks. Detection transformers are the first fully end-to-end&#10;learning systems for object detection, while vision transformers are the first&#10;fully transformer-based architecture for image classification. In this paper,&#10;we integrate Vision and Detection Transformers (ViDT) to build an effective and&#10;efficient object detector. ViDT introduces a reconfigured attention module to&#10;extend the recent Swin Transformer to be a standalone object detector, followed&#10;by a computationally efficient transformer decoder that exploits multi-scale&#10;features and auxiliary techniques essential to boost the detection performance&#10;without much increase in computational load. Extensive evaluation results on&#10;the Microsoft COCO benchmark dataset demonstrate that ViDT obtains the best AP&#10;and latency trade-off among existing fully transformer-based object detectors,&#10;and achieves 49.2AP owing to its high scalability for large models. We will&#10;release the code and trained models at https://github.com/naver-ai/vidt&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.01402" label="2010.01402">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Monocular Depth Estimation for Night-time Images using&#10;  Adversarial Domain Feature Adaptation" />
-          <attvalue for="1" value="  In this paper, we look into the problem of estimating per-pixel depth maps&#10;from unconstrained RGB monocular night-time images which is a difficult task&#10;that has not been addressed adequately in the literature. The state-of-the-art&#10;day-time depth estimation methods fail miserably when tested with night-time&#10;images due to a large domain shift between them. The usual photo metric losses&#10;used for training these networks may not work for night-time images due to the&#10;absence of uniform lighting which is commonly present in day-time images,&#10;making it a difficult problem to solve. We propose to solve this problem by&#10;posing it as a domain adaptation problem where a network trained with day-time&#10;images is adapted to work for night-time images. Specifically, an encoder is&#10;trained to generate features from night-time images that are indistinguishable&#10;from those obtained from day-time images by using a PatchGAN-based adversarial&#10;discriminative learning method. Unlike the existing methods that directly adapt&#10;depth prediction (network output), we propose to adapt feature maps obtained&#10;from the encoder network so that a pre-trained day-time depth decoder can be&#10;directly used for predicting depth from these adapted features. Hence, the&#10;resulting method is termed as &quot;Adversarial Domain Feature Adaptation (ADFA)&quot;&#10;and its efficacy is demonstrated through experimentation on the challenging&#10;Oxford night driving dataset. Also, The modular encoder-decoder architecture&#10;for the proposed ADFA method allows us to use the encoder module as a feature&#10;extractor which can be used in many other applications. One such application is&#10;demonstrated where the features obtained from our adapted encoder network are&#10;shown to outperform other state-of-the-art methods in a visual place&#10;recognition problem, thereby, further establishing the usefulness and&#10;effectiveness of the proposed approach.&#10;" />
-          <attvalue for="2" value=" &#10;Estimating depth from RGB images is a challenging problem which finds&#10;applications in a wide range of fields such as augmented reality&#10;\cite{pose_ar}, 3D reconstruction \cite{andreas2011stereo},&#10;self-driving cars \cite{handa2014benchmark}, place recognition&#10;\cite{garg2019look}, etc. The recent success of deep learning methods&#10;has spurred the research in this field leading to the creation of&#10;several new benchmarks that now outperform traditional methods which&#10;rely on handcrafted features and exploit camera geometry and/or camera&#10;motion for depth and pose estimation from monocular or stereo sequence&#10;of images (video). These learning methods can be broadly classified into two categories: supervised and unsupervised. The supervised learning methods \cite{eigen2014depth}&#10;\cite{cao2018estimating} necessitate explicit availability of ground truth information (Laser or&#10;LiDAR range data) which may not always be feasible in many real-world&#10;scenarios. This is overcome by the unsupervised methods&#10;\cite{zhou2017unsupervised} \cite{yin2018geonet}&#10;\cite{babu2018undemon} that harness the spatial and/or temporal&#10;consistency present in image sequences to extract the underlying&#10;geometry to be used as the implicit supervision signal required for&#10;training the models. &#10; Many of these methods were shown to&#10;provide very impressive results on several popular datasets such as&#10;KITTI \cite{geiger2013vision} and Cityscapes&#10;\cite{cordts2016cityscapes} containing only day-time images. In&#10;contrast, there are a very few works that aim to solve the night-time&#10;depth estimation problem, which is comparatively more challenging owing&#10;to factors such as low visibility and non-uniform illumination arising&#10;from multiple (street lights, traffic lights) and possibly, moving light sources (car&#10;headlights). For instance, authors in \cite{im2018robust} exploit&#10;the inherent motion component in burst shot (several successive shots&#10;with varying camera settings, also known as ``auto-bracketing'') to&#10;estimate depth from images taken under low-light condition.&#10;Similarly, Zhu et al. \cite{zhu2019unsupervised} present a deep&#10;learning based method for estimating motion flow, depth and pose from&#10;images obtained from event cameras that return a time-stamped event&#10;tuple whenever a change in pixel intensity is detected. In another&#10;work, Kim et al. \cite{kim2018multispectral} propose a deep network for&#10;estimating depth from thermal images taken during the night time. To&#10;the best of our knowledge, there is no reported work that addresses&#10;the problem of estimating depth and pose directly from a single&#10;ordinary RGB monocular night-time image. The deep learning models&#10;trained on day-time monocular \cite{zhou2017unsupervised} or stereo&#10;images \cite{babu2018undemon} fail miserably on night-time images due&#10;to the inherent large domain shift between these images. The domain&#10;shift refers to the change from day-time conditions (well-lit and&#10;uniform illumination) to night-time conditions comprising low&#10;illumination/visibility with non-uniform illumination caused by&#10;unpredictable appearance and disappearance of multiple point-light&#10;sources (e.g., street lamps or car headlights, etc.).&#10;&#10;One possible solution will be to apply image-to-image translation&#10;methods, such as Cycle-GAN \cite{zhu2017unpaired} or MUNIT&#10;\cite{huang2018multimodal}, to map night-time images to day-time&#10;images and then use a pre-trained day-time depth model to estimate&#10;depth from these translated images. Some of these image translation&#10;techniques have been used in the context of night-time images. For&#10;instance, the authors in \cite{anoosheh2019night} use night-to-day&#10;image translation for solving the place recognition problem required&#10;for localization. Similarly, authors in \cite{atapour2018real}&#10;\cite{zhao2019geometry} explore image translation techniques to generate&#10;synthetic labeled data to reduce the requirement of real-world images&#10;for training depth estimation models. Many of these models trained on&#10;simulated images do not generalize well to natural images due to the&#10;inherent domain shift and hence, employ several domain adaptation&#10;techniques to improve their applicability to real-world situations&#10;\cite{atapour2018real} \cite{zhao2019geometry}&#10;\cite{nath2018adadepth}. These approaches have several limitations.&#10;For instance, many of these methods use two different deep networks -&#10;one for image translation and another for depth estimation, making it&#10;computationally heavy and with possibly, inferior performance due to&#10;the cascading error effect of using two models in a cascade. Since the&#10;image translation module is trained independent of the depth network&#10;module, it may not learn depth-specific attributes required for&#10;preserving structural information during image translation. This may,&#10;in turn, introduce artifacts which might not be understood by the&#10;depth estimation module leading to poor depth prediction for the input&#10;night-time image. Secondly, it is difficult to generate synthetic&#10;night-time images that can capture all the vagaries of real-world&#10;night conditions as one can observe in the Synthia dataset&#10;\cite{ros2016synthia}. Many of the simulated night-time images in this&#10;dataset appear almost like day-time images and using them for&#10;night-time depth prediction may not give desired results. Finally,&#10;these methods have been applied so far to day-time images for depth&#10;estimation. &#10;&#10;In this paper, we propose a PatchGAN-based domain adaptation technique&#10;for estimating depth from monocular night images by using a single&#10;encoder-decoder type deep network model. Specifically, an encoder&#10;network is trained to generate night-time features which are&#10;indistinguishable from those obtained from day-time images. This is achieved by&#10;using an adversarial discriminative learning \cite{tzeng2017adversarial}&#10;that uses day-time encoded features as the reference. These adapted&#10;night features could then be used directly with a decoder network&#10;pre-trained on day-time images for depth estimation. Since the domain&#10;features are adapted through adversarial learning, this method is&#10;termed as ``Adversarial Domain Feature Adaptation (ADFA)''&#10;method to distinguish it from other methods that attempt to adapt&#10;depth predictions directly \cite{atapour2018real}&#10;\cite{zhao2019geometry} \cite{nath2018adadepth}. Patch{GAN} networks&#10;\cite{isola2017image} \cite{vankadari2019unsupervised} have been shown&#10;to provide superior performance compared to conventional GANs by&#10;capturing high frequency local structural information and hence, form&#10;a natural choice of GAN architecture for the proposed method. &#10;&#10;The resulting outcome of our approach is shown qualitatively in Figure~\ref{fig:intro}. We are able to obtain reliable depth maps shown in&#10;Figure~\ref{fig:intro}(c) from monocular night-time images shown in Figure~\ref{fig:intro}(a). This&#10;is also evident from the interpolated ground-truth depth maps obtained from the&#10;LIDAR point clouds as shown in Figure~\ref{fig:intro}(b). The efficacy of the&#10;proposed approach is demonstrated by applying it to the challenging&#10;Oxford night-time driving dataset \cite{maddern20171}. The modular&#10;encoder-decoder architecture provides the flexibility of using the&#10;encoder module as a feature extractor to extract or select useful&#10;features from input images. Such feature extractors are used in&#10;several applications such as pose estimation \cite{godard2018digging},&#10;Visual Place Recognition (VPR)\cite{garg2020delta}\cite{garg2019look},&#10;object detection \cite{zhao2019object} and segmentation&#10;\cite{badrinarayanan2017segnet}. We demonstrate one such application&#10;where the adapted features obtained from our encoder module are shown&#10;to provide superior place recognition accuracy compared to other&#10;state-of-the-art feature representations available in the literature.&#10;&#10;In short, the main contributions made in this paper may be summarized&#10;as follows:&#10;&#10;\begin{itemize} \item We propose a novel PatchGAN-based domain feature&#10; adaptation method for estimating depth from unconstrained&#10; monocular night-time RGB images, which is considered to be more&#10; difficult compared to day-time images. To the best of our&#10; knowledge, this is the first instance where adversarial&#10; discriminative domain feature adaptation is being used for&#10; estimating depth from unconstrained night-time monocular RGB&#10; images and this may act as a stepping-stone for future research&#10; in this field. &#10;&#10;\item We also propose an image translation-based method for&#10; night-time depth estimation by using a combination of an image&#10; translating network (e.g. CycleGAN \cite{zhu2017unpaired}) and a&#10; standard day-time depth estimation network (such as&#10; \cite{godard2018digging}) in cascade. This serves to highlight&#10; the difficulties involved in such methods and hence, provides&#10; a strong motivation in favour of the proposed work.&#10;&#10;\item The usefulness and effectiveness of our method is&#10; further established by demonstrating that the features obtained&#10; using the proposed ADFA method outperform other state-of-the-art&#10; feature representations in a visual place recognition problem.&#10;\end{itemize}&#10;&#10;Rest of this paper is organized as follows. The proposed method is&#10;described in the next section. The experimental evaluation of our approach on various datasets is discussed in Section~\ref{sec:experiment}. The concluding remarks and future scope of this&#10;work is presented in Section~\ref{sec:conc}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1902.07381" label="1902.07381">
-        <attvalues>
-          <attvalue for="0" value="Look No Deeper: Recognizing Places from Opposing Viewpoints under&#10;  Varying Scene Appearance using Single-View Depth Estimation" />
-          <attvalue for="1" value="  Visual place recognition (VPR) - the act of recognizing a familiar visual&#10;place - becomes difficult when there is extreme environmental appearance change&#10;or viewpoint change. Particularly challenging is the scenario where both&#10;phenomena occur simultaneously, such as when returning for the first time along&#10;a road at night that was previously traversed during the day in the opposite&#10;direction. While such problems can be solved with panoramic sensors, humans&#10;solve this problem regularly with limited field of view vision and without&#10;needing to constantly turn around. In this paper, we present a new depth- and&#10;temporal-aware visual place recognition system that solves the opposing&#10;viewpoint, extreme appearance-change visual place recognition problem. Our&#10;system performs sequence-to-single matching by extracting depth-filtered&#10;keypoints using a state-of-the-art depth estimation pipeline, constructing a&#10;keypoint sequence over multiple frames from the reference dataset, and&#10;comparing those keypoints to those in a single query image. We evaluate the&#10;system on a challenging benchmark dataset and show that it consistently&#10;outperforms state-of-the-art techniques. We also develop a range of diagnostic&#10;simulation experiments that characterize the contribution of depth-filtered&#10;keypoint sequences with respect to key domain parameters including degree of&#10;appearance change and camera motion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.2283" label="1406.2283">
-        <attvalues>
-          <attvalue for="0" value="Depth Map Prediction from a Single Image using a Multi-Scale Deep&#10;  Network" />
-          <attvalue for="1" value="  Predicting depth is an essential component in understanding the 3D geometry&#10;of a scene. While for stereo images local correspondence suffices for&#10;estimation, finding depth relations from a single image is less&#10;straightforward, requiring integration of both global and local information&#10;from various cues. Moreover, the task is inherently ambiguous, with a large&#10;source of uncertainty coming from the overall scale. In this paper, we present&#10;a new method that addresses this task by employing two deep network stacks: one&#10;that makes a coarse global prediction based on the entire image, and another&#10;that refines this prediction locally. We also apply a scale-invariant error to&#10;help measure depth relations rather than scale. By leveraging the raw datasets&#10;as large sources of training data, our method achieves state-of-the-art results&#10;on both NYU Depth and KITTI, and matches detailed depth boundaries without the&#10;need for superpixelation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.02305" label="1605.02305">
-        <attvalues>
-          <attvalue for="0" value="Estimating Depth from Monocular Images as Classification Using Deep&#10;  Fully Convolutional Residual Networks" />
-          <attvalue for="1" value="  Depth estimation from single monocular images is a key component of scene&#10;understanding and has benefited largely from deep convolutional neural networks&#10;(CNN) recently. In this article, we take advantage of the recent deep residual&#10;networks and propose a simple yet effective approach to this problem. We&#10;formulate depth estimation as a pixel-wise classification task. Specifically,&#10;we first discretize the continuous depth values into multiple bins and label&#10;the bins according to their depth range. Then we train fully convolutional deep&#10;residual networks to predict the depth label of each pixel. Performing discrete&#10;depth label classification instead of continuous depth value regression allows&#10;us to predict a confidence in the form of probability distribution. We further&#10;apply fully-connected conditional random fields (CRF) as a post processing step&#10;to enforce local smoothness interactions, which improves the results. We&#10;evaluate our approach on both indoor and outdoor datasets and achieve&#10;state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.02276" label="1803.02276">
-        <attvalues>
-          <attvalue for="0" value="GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera&#10;  Pose" />
-          <attvalue for="1" value="  We propose GeoNet, a jointly unsupervised learning framework for monocular&#10;depth, optical flow and ego-motion estimation from videos. The three components&#10;are coupled by the nature of 3D scene geometry, jointly learned by our&#10;framework in an end-to-end manner. Specifically, geometric relationships are&#10;extracted over the predictions of individual modules and then combined as an&#10;image reconstruction loss, reasoning about static and dynamic scene parts&#10;separately. Furthermore, we propose an adaptive geometric consistency loss to&#10;increase robustness towards outliers and non-Lambertian regions, which resolves&#10;occlusions and texture ambiguities effectively. Experimentation on the KITTI&#10;driving dataset reveals that our scheme achieves state-of-the-art results in&#10;all of the three tasks, performing better than previously unsupervised methods&#10;and comparably with supervised ones.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.01685" label="1604.01685">
-        <attvalues>
-          <attvalue for="0" value="The Cityscapes Dataset for Semantic Urban Scene Understanding" />
-          <attvalue for="1" value="  Visual understanding of complex urban street scenes is an enabling factor for&#10;a wide range of applications. Object detection has benefited enormously from&#10;large-scale datasets, especially in the context of deep learning. For semantic&#10;urban scene understanding, however, no current dataset adequately captures the&#10;complexity of real-world urban scenes.&#10;  To address this, we introduce Cityscapes, a benchmark suite and large-scale&#10;dataset to train and test approaches for pixel-level and instance-level&#10;semantic labeling. Cityscapes is comprised of a large, diverse set of stereo&#10;video sequences recorded in streets from 50 different cities. 5000 of these&#10;images have high quality pixel-level annotations; 20000 additional images have&#10;coarse annotations to enable methods that leverage large volumes of&#10;weakly-labeled data. Crucially, our effort exceeds previous attempts in terms&#10;of dataset size, annotation richness, scene variability, and complexity. Our&#10;accompanying empirical study provides an in-depth analysis of the dataset&#10;characteristics, as well as a performance evaluation of several&#10;state-of-the-art approaches based on our benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07702" label="1803.07702">
-        <attvalues>
-          <attvalue for="0" value="Robust Depth Estimation from Auto Bracketed Images" />
-          <attvalue for="1" value="  As demand for advanced photographic applications on hand-held devices grows,&#10;these electronics require the capture of high quality depth. However, under&#10;low-light conditions, most devices still suffer from low imaging quality and&#10;inaccurate depth acquisition. To address the problem, we present a robust depth&#10;estimation method from a short burst shot with varied intensity (i.e., Auto&#10;Bracketing) or strong noise (i.e., High ISO). We introduce a geometric&#10;transformation between flow and depth tailored for burst images, enabling our&#10;learning-based multi-view stereo matching to be performed effectively. We then&#10;describe our depth estimation pipeline that incorporates the geometric&#10;transformation into our residual-flow network. It allows our framework to&#10;produce an accurate depth map even with a bracketed image sequence. We&#10;demonstrate that our method outperforms state-of-the-art methods for various&#10;datasets captured by a smartphone and a DSLR camera. Moreover, we show that the&#10;estimated depth is applicable for image quality enhancement and photographic&#10;editing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.08156" label="1812.08156">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion" />
-          <attvalue for="1" value="  In this work, we propose a novel framework for unsupervised learning for&#10;event cameras that learns motion information from only the event stream. In&#10;particular, we propose an input representation of the events in the form of a&#10;discretized volume that maintains the temporal distribution of the events,&#10;which we pass through a neural network to predict the motion of the events.&#10;This motion is used to attempt to remove any motion blur in the event image. We&#10;then propose a loss function applied to the motion compensated event image that&#10;measures the motion blur in this image. We train two networks with this&#10;framework, one to predict optical flow, and one to predict egomotion and&#10;depths, and evaluate these networks on the Multi Vehicle Stereo Event Camera&#10;dataset, along with qualitative results from a variety of different scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.10593" label="1703.10593">
-        <attvalues>
-          <attvalue for="0" value="Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial&#10;  Networks" />
-          <attvalue for="1" value="  Image-to-image translation is a class of vision and graphics problems where&#10;the goal is to learn the mapping between an input image and an output image&#10;using a training set of aligned image pairs. However, for many tasks, paired&#10;training data will not be available. We present an approach for learning to&#10;translate an image from a source domain $X$ to a target domain $Y$ in the&#10;absence of paired examples. Our goal is to learn a mapping $G: X \rightarrow Y$&#10;such that the distribution of images from $G(X)$ is indistinguishable from the&#10;distribution $Y$ using an adversarial loss. Because this mapping is highly&#10;under-constrained, we couple it with an inverse mapping $F: Y \rightarrow X$&#10;and introduce a cycle consistency loss to push $F(G(X)) \approx X$ (and vice&#10;versa). Qualitative results are presented on several tasks where paired&#10;training data does not exist, including collection style transfer, object&#10;transfiguration, season transfer, photo enhancement, etc. Quantitative&#10;comparisons against several prior methods demonstrate the superiority of our&#10;approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.04732" label="1804.04732">
-        <attvalues>
-          <attvalue for="0" value="Multimodal Unsupervised Image-to-Image Translation" />
-          <attvalue for="1" value="  Unsupervised image-to-image translation is an important and challenging&#10;problem in computer vision. Given an image in the source domain, the goal is to&#10;learn the conditional distribution of corresponding images in the target&#10;domain, without seeing any pairs of corresponding images. While this&#10;conditional distribution is inherently multimodal, existing approaches make an&#10;overly simplified assumption, modeling it as a deterministic one-to-one&#10;mapping. As a result, they fail to generate diverse outputs from a given source&#10;domain image. To address this limitation, we propose a Multimodal Unsupervised&#10;Image-to-image Translation (MUNIT) framework. We assume that the image&#10;representation can be decomposed into a content code that is domain-invariant,&#10;and a style code that captures domain-specific properties. To translate an&#10;image to another domain, we recombine its content code with a random style code&#10;sampled from the style space of the target domain. We analyze the proposed&#10;framework and establish several theoretical results. Extensive experiments with&#10;comparisons to the state-of-the-art approaches further demonstrates the&#10;advantage of the proposed framework. Moreover, our framework allows users to&#10;control the style of translation outputs by providing an example style image.&#10;Code and pretrained models are available at https://github.com/nvlabs/MUNIT&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.09767" label="1809.09767">
-        <attvalues>
-          <attvalue for="0" value="Night-to-Day Image Translation for Retrieval-based Localization" />
-          <attvalue for="1" value="  Visual localization is a key step in many robotics pipelines, allowing the&#10;robot to (approximately) determine its position and orientation in the world.&#10;An efficient and scalable approach to visual localization is to use image&#10;retrieval techniques. These approaches identify the image most similar to a&#10;query photo in a database of geo-tagged images and approximate the query's pose&#10;via the pose of the retrieved database image. However, image retrieval across&#10;drastically different illumination conditions, e.g. day and night, is still a&#10;problem with unsatisfactory results, even in this age of powerful neural&#10;models. This is due to a lack of a suitably diverse dataset with true&#10;correspondences to perform end-to-end learning. A recent class of neural models&#10;allows for realistic translation of images among visual domains with relatively&#10;little training data and, most importantly, without ground-truth pairings. In&#10;this paper, we explore the task of accurately localizing images captured from&#10;two traversals of the same area in both day and night. We propose ToDayGAN - a&#10;modified image-translation model to alter nighttime driving images to a more&#10;useful daytime representation. We then compare the daytime and translated night&#10;images to obtain a pose estimate for the night image using the known 6-DOF&#10;position of the closest day image. Our approach improves localization&#10;performance by over 250% compared the current state-of-the-art, in the context&#10;of standard metrics in multiple categories.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.01870" label="1904.01870">
-        <attvalues>
-          <attvalue for="0" value="Geometry-Aware Symmetric Domain Adaptation for Monocular Depth&#10;  Estimation" />
-          <attvalue for="1" value="  Supervised depth estimation has achieved high accuracy due to the advanced&#10;deep network architectures. Since the groundtruth depth labels are hard to&#10;obtain, recent methods try to learn depth estimation networks in an&#10;unsupervised way by exploring unsupervised cues, which are effective but less&#10;reliable than true labels. An emerging way to resolve this dilemma is to&#10;transfer knowledge from synthetic images with ground truth depth via domain&#10;adaptation techniques. However, these approaches overlook specific geometric&#10;structure of the natural images in the target domain (i.e., real data), which&#10;is important for high-performing depth prediction. Motivated by the&#10;observation, we propose a geometry-aware symmetric domain adaptation framework&#10;(GASDA) to explore the labels in the synthetic data and epipolar geometry in&#10;the real data jointly. Moreover, by training two image style translators and&#10;depth estimators symmetrically in an end-to-end network, our model achieves&#10;better image style transfer and generates high-quality depth maps. The&#10;experimental results demonstrate the effectiveness of our proposed method and&#10;comparable performance against the state-of-the-art. Code will be publicly&#10;available at: https://github.com/sshan-zhao/GASDA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.01599" label="1803.01599">
-        <attvalues>
-          <attvalue for="0" value="AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation" />
-          <attvalue for="1" value="  Supervised deep learning methods have shown promising results for the task of&#10;monocular depth estimation; but acquiring ground truth is costly, and prone to&#10;noise as well as inaccuracies. While synthetic datasets have been used to&#10;circumvent above problems, the resultant models do not generalize well to&#10;natural scenes due to the inherent domain shift. Recent adversarial approaches&#10;for domain adaption have performed well in mitigating the differences between&#10;the source and target domains. But these methods are mostly limited to a&#10;classification setup and do not scale well for fully-convolutional&#10;architectures. In this work, we propose AdaDepth - an unsupervised domain&#10;adaptation strategy for the pixel-wise regression task of monocular depth&#10;estimation. The proposed approach is devoid of above limitations through a)&#10;adversarial learning and b) explicit imposition of content consistency on the&#10;adapted target representation. Our unsupervised approach performs competitively&#10;with other established approaches on depth estimation tasks and achieves&#10;state-of-the-art results in a semi-supervised setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.05464" label="1702.05464">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Discriminative Domain Adaptation" />
-          <attvalue for="1" value="  Adversarial learning methods are a promising approach to training robust deep&#10;networks, and can generate complex samples across diverse domains. They also&#10;can improve recognition despite the presence of domain shift or dataset bias:&#10;several adversarial approaches to unsupervised domain adaptation have recently&#10;been introduced, which reduce the difference between the training and test&#10;domain distributions and thus improve generalization performance. Prior&#10;generative approaches show compelling visualizations, but are not optimal on&#10;discriminative tasks and can be limited to smaller shifts. Prior discriminative&#10;approaches could handle larger domain shifts, but imposed tied weights on the&#10;model and did not exploit a GAN-based loss. We first outline a novel&#10;generalized framework for adversarial adaptation, which subsumes recent&#10;state-of-the-art approaches as special cases, and we use this generalized view&#10;to better relate the prior approaches. We propose a previously unexplored&#10;instance of our general framework which combines discriminative modeling,&#10;untied weight sharing, and a GAN loss, which we call Adversarial Discriminative&#10;Domain Adaptation (ADDA). We show that ADDA is more effective yet considerably&#10;simpler than competing domain-adversarial methods, and demonstrate the promise&#10;of our approach by exceeding state-of-the-art unsupervised adaptation results&#10;on standard cross-domain digit classification tasks and a new more difficult&#10;cross-modality object classification task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07004" label="1611.07004">
-        <attvalues>
-          <attvalue for="0" value="Image-to-Image Translation with Conditional Adversarial Networks" />
-          <attvalue for="1" value="  We investigate conditional adversarial networks as a general-purpose solution&#10;to image-to-image translation problems. These networks not only learn the&#10;mapping from input image to output image, but also learn a loss function to&#10;train this mapping. This makes it possible to apply the same generic approach&#10;to problems that traditionally would require very different loss formulations.&#10;We demonstrate that this approach is effective at synthesizing photos from&#10;label maps, reconstructing objects from edge maps, and colorizing images, among&#10;other tasks. Indeed, since the release of the pix2pix software associated with&#10;this paper, a large number of internet users (many of them artists) have posted&#10;their own experiments with our system, further demonstrating its wide&#10;applicability and ease of adoption without the need for parameter tweaking. As&#10;a community, we no longer hand-engineer our mapping functions, and this work&#10;suggests we can achieve reasonable results without hand-engineering our loss&#10;functions either.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01260" label="1806.01260">
-        <attvalues>
-          <attvalue for="0" value="Digging Into Self-Supervised Monocular Depth Estimation" />
-          <attvalue for="1" value="  Per-pixel ground-truth depth data is challenging to acquire at scale. To&#10;overcome this limitation, self-supervised learning has emerged as a promising&#10;alternative for training models to perform monocular depth estimation. In this&#10;paper, we propose a set of improvements, which together result in both&#10;quantitatively and qualitatively improved depth maps compared to competing&#10;self-supervised methods.&#10;  Research on self-supervised monocular training usually explores increasingly&#10;complex architectures, loss functions, and image formation models, all of which&#10;have recently helped to close the gap with fully-supervised methods. We show&#10;that a surprisingly simple model, and associated design choices, lead to&#10;superior predictions. In particular, we propose (i) a minimum reprojection&#10;loss, designed to robustly handle occlusions, (ii) a full-resolution&#10;multi-scale sampling method that reduces visual artifacts, and (iii) an&#10;auto-masking loss to ignore training pixels that violate camera motion&#10;assumptions. We demonstrate the effectiveness of each component in isolation,&#10;and show high quality, state-of-the-art results on the KITTI benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.05700" label="2006.05700">
-        <attvalues>
-          <attvalue for="0" value="Delta Descriptors: Change-Based Place Representation for Robust Visual&#10;  Localization" />
-          <attvalue for="1" value="  Visual place recognition is challenging because there are so many factors&#10;that can cause the appearance of a place to change, from day-night cycles to&#10;seasonal change to atmospheric conditions. In recent years a large range of&#10;approaches have been developed to address this challenge including deep-learnt&#10;image descriptors, domain translation, and sequential filtering, all with&#10;shortcomings including generality and velocity-sensitivity. In this paper we&#10;propose a novel descriptor derived from tracking changes in any learned global&#10;descriptor over time, dubbed Delta Descriptors. Delta Descriptors mitigate the&#10;offsets induced in the original descriptor matching space in an unsupervised&#10;manner by considering temporal differences across places observed along a&#10;route. Like all other approaches, Delta Descriptors have a shortcoming -&#10;volatility on a frame to frame basis - which can be overcome by combining them&#10;with sequential filtering methods. Using two benchmark datasets, we first&#10;demonstrate the high performance of Delta Descriptors in isolation, before&#10;showing new state-of-the-art performance when combined with sequence-based&#10;matching. We also present results demonstrating the approach working with four&#10;different underlying descriptor types, and two other beneficial properties of&#10;Delta Descriptors in comparison to existing techniques: their increased&#10;inherent robustness to variations in camera motion and a reduced rate of&#10;performance degradation as dimensional reduction is applied. Source code is&#10;made available at https://github.com/oravus/DeltaDescriptors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.05511" label="1807.05511">
-        <attvalues>
-          <attvalue for="0" value="Object Detection with Deep Learning: A Review" />
-          <attvalue for="1" value="  Due to object detection's close relationship with video analysis and image&#10;understanding, it has attracted much research attention in recent years.&#10;Traditional object detection methods are built on handcrafted features and&#10;shallow trainable architectures. Their performance easily stagnates by&#10;constructing complex ensembles which combine multiple low-level image features&#10;with high-level context from object detectors and scene classifiers. With the&#10;rapid development in deep learning, more powerful tools, which are able to&#10;learn semantic, high-level, deeper features, are introduced to address the&#10;problems existing in traditional architectures. These models behave differently&#10;in network architecture, training strategy and optimization function, etc. In&#10;this paper, we provide a review on deep learning based object detection&#10;frameworks. Our review begins with a brief introduction on the history of deep&#10;learning and its representative tool, namely Convolutional Neural Network&#10;(CNN). Then we focus on typical generic object detection architectures along&#10;with some modifications and useful tricks to improve detection performance&#10;further. As distinct specific detection tasks exhibit different&#10;characteristics, we also briefly survey several specific tasks, including&#10;salient object detection, face detection and pedestrian detection. Experimental&#10;analyses are also provided to compare various methods and draw some meaningful&#10;conclusions. Finally, several promising directions and tasks are provided to&#10;serve as guidelines for future work in both object detection and relevant&#10;neural network based learning systems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.00561" label="1511.00561">
-        <attvalues>
-          <attvalue for="0" value="SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image&#10;  Segmentation" />
-          <attvalue for="1" value="  We present a novel and practical deep fully convolutional neural network&#10;architecture for semantic pixel-wise segmentation termed SegNet. This core&#10;trainable segmentation engine consists of an encoder network, a corresponding&#10;decoder network followed by a pixel-wise classification layer. The architecture&#10;of the encoder network is topologically identical to the 13 convolutional&#10;layers in the VGG16 network. The role of the decoder network is to map the low&#10;resolution encoder feature maps to full input resolution feature maps for&#10;pixel-wise classification. The novelty of SegNet lies is in the manner in which&#10;the decoder upsamples its lower resolution input feature map(s). Specifically,&#10;the decoder uses pooling indices computed in the max-pooling step of the&#10;corresponding encoder to perform non-linear upsampling. This eliminates the&#10;need for learning to upsample. The upsampled maps are sparse and are then&#10;convolved with trainable filters to produce dense feature maps. We compare our&#10;proposed architecture with the widely adopted FCN and also with the well known&#10;DeepLab-LargeFOV, DeconvNet architectures. This comparison reveals the memory&#10;versus accuracy trade-off involved in achieving good segmentation performance.&#10;  SegNet was primarily motivated by scene understanding applications. Hence, it&#10;is designed to be efficient both in terms of memory and computational time&#10;during inference. It is also significantly smaller in the number of trainable&#10;parameters than other competing architectures. We also performed a controlled&#10;benchmark of SegNet and other architectures on both road scenes and SUN RGB-D&#10;indoor scene segmentation tasks. We show that SegNet provides good performance&#10;with competitive inference time and more efficient inference memory-wise as&#10;compared to other architectures. We also provide a Caffe implementation of&#10;SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.16606" label="2306.16606">
-        <attvalues>
-          <attvalue for="0" value="EgoCOL: Egocentric Camera pose estimation for Open-world 3D object&#10;  Localization @Ego4D challenge 2023" />
-          <attvalue for="1" value="  We present EgoCOL, an egocentric camera pose estimation method for open-world&#10;3D object localization. Our method leverages sparse camera pose reconstructions&#10;in a two-fold manner, video and scan independently, to estimate the camera pose&#10;of egocentric frames in 3D renders with high recall and precision. We&#10;extensively evaluate our method on the Visual Query (VQ) 3D object localization&#10;Ego4D benchmark. EgoCOL can estimate 62% and 59% more camera poses than the&#10;Ego4D baseline in the Ego4D Visual Queries 3D Localization challenge at CVPR&#10;2023 in the val and test sets, respectively. Our code is publicly available at&#10;https://github.com/BCV-Uniandes/EgoCOL&#10;" />
-          <attvalue for="2" value="&#10;Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}. However, in both augmented reality and robotics applications, we need to understand data captured from a first-person or egocentric point-of-view. Through egocentric data, we see the world from the perspective of an agent interacting with its environment. Thus, egocentric perception requires a constant 3D awareness of the camera wearer's environment and must recognize objects and actions in human context \cite{grauman2022ego4d}. \\&#10;&#10;Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}. However, these methods rely on a 2D perception and fail to relate an egocentric experience to the 3D world. Since egocentric videos record daily life experiences such as who, what, when, and where of an individual, they are ideal for extending the 2D human perception into the 3D world for addressing tasks such as 3D episodic memory \cite{tulving1972episodic}. We define episodic memory by answering the question: when and where I last saw object X?.\\&#10;In this report, we tackle the task of episodic memory with visual queries for 3D localization. As shown in Fig. \ref{fig:VQ3D}, given an egocentric video and a Visual Query (VQ), where the query is a static image of an object, we spatially locate in a 3D scene the last time the VQ was seen in the egocentric video. VQ 3D localization allows us to find an object of interest in a predefined 3D space through an egocentric video input. By working with visual queries, as opposed to categorical queries in which there is a fixed set of categories, VQ 3D localization is an approach for open-world 3D object localization, where no categories are needed for object detection.\\&#10;We present EgoCOL, a new method for camera pose estimation from an egocentric perspective. EgoCOL leverages sparse reconstructions from video and scans to estimate the camera poses with high recall and precision. As shown in Fig. \ref{fig:overview}, we leverage sparse 3D relative camera pose reconstructions to compute transformation functions that allow us to map a large number of frames to the 3D scan coordinate system. In addition, we propose a to evaluate the VQ3D task a 3D detection problem. This approach helps mitigate the tendency of previous metrics to overestimate their performance.\\&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Computer Science, Camera Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2205.01917" label="2205.01917">
-        <attvalues>
-          <attvalue for="0" value="CoCa: Contrastive Captioners are Image-Text Foundation Models" />
-          <attvalue for="1" value="  Exploring large-scale pretrained foundation models is of significant interest&#10;in computer vision because these models can be quickly transferred to many&#10;downstream tasks. This paper presents Contrastive Captioner (CoCa), a&#10;minimalist design to pretrain an image-text encoder-decoder foundation model&#10;jointly with contrastive loss and captioning loss, thereby subsuming model&#10;capabilities from contrastive approaches like CLIP and generative methods like&#10;SimVLM. In contrast to standard encoder-decoder transformers where all decoder&#10;layers attend to encoder outputs, CoCa omits cross-attention in the first half&#10;of decoder layers to encode unimodal text representations, and cascades the&#10;remaining decoder layers which cross-attend to the image encoder for multimodal&#10;image-text representations. We apply a contrastive loss between unimodal image&#10;and text embeddings, in addition to a captioning loss on the multimodal decoder&#10;outputs which predicts text tokens autoregressively. By sharing the same&#10;computational graph, the two training objectives are computed efficiently with&#10;minimal overhead. CoCa is pretrained end-to-end and from scratch on both&#10;web-scale alt-text data and annotated images by treating all labels simply as&#10;text, seamlessly unifying natural language supervision for representation&#10;learning. Empirically, CoCa achieves state-of-the-art performance with&#10;zero-shot transfer or minimal task-specific adaptation on a broad range of&#10;downstream tasks, spanning visual recognition (ImageNet, Kinetics-400/600/700,&#10;Moments-in-Time), crossmodal retrieval (MSCOCO, Flickr30K, MSR-VTT), multimodal&#10;understanding (VQA, SNLI-VE, NLVR2), and image captioning (MSCOCO, NoCaps).&#10;Notably on ImageNet classification, CoCa obtains 86.3% zero-shot top-1&#10;accuracy, 90.6% with a frozen encoder and learned classification head, and new&#10;state-of-the-art 91.0% top-1 accuracy on ImageNet with a finetuned encoder.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09883" label="2111.09883">
-        <attvalues>
-          <attvalue for="0" value="Swin Transformer V2: Scaling Up Capacity and Resolution" />
-          <attvalue for="1" value="  Large-scale NLP models have been shown to significantly improve the&#10;performance on language tasks with no signs of saturation. They also&#10;demonstrate amazing few-shot capabilities like that of human beings. This paper&#10;aims to explore large-scale models in computer vision. We tackle three major&#10;issues in training and application of large vision models, including training&#10;instability, resolution gaps between pre-training and fine-tuning, and hunger&#10;on labelled data. Three main techniques are proposed: 1) a residual-post-norm&#10;method combined with cosine attention to improve training stability; 2) A&#10;log-spaced continuous position bias method to effectively transfer models&#10;pre-trained using low-resolution images to downstream tasks with&#10;high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to&#10;reduce the needs of vast labeled images. Through these techniques, this paper&#10;successfully trained a 3 billion-parameter Swin Transformer V2 model, which is&#10;the largest dense vision model to date, and makes it capable of training with&#10;images of up to 1,536$\times$1,536 resolution. It set new performance records&#10;on 4 representative vision tasks, including ImageNet-V2 image classification,&#10;COCO object detection, ADE20K semantic segmentation, and Kinetics-400 video&#10;action classification. Also note our training is much more efficient than that&#10;in Google's billion-level visual models, which consumes 40 times less labelled&#10;data and 40 times less training time. Code is available at&#10;\url{https://github.com/microsoft/Swin-Transformer}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.04803" label="2106.04803">
-        <attvalues>
-          <attvalue for="0" value="CoAtNet: Marrying Convolution and Attention for All Data Sizes" />
-          <attvalue for="1" value="  Transformers have attracted increasing interests in computer vision, but they&#10;still fall behind state-of-the-art convolutional networks. In this work, we&#10;show that while Transformers tend to have larger model capacity, their&#10;generalization can be worse than convolutional networks due to the lack of the&#10;right inductive bias. To effectively combine the strengths from both&#10;architectures, we present CoAtNets(pronounced &quot;coat&quot; nets), a family of hybrid&#10;models built from two key insights: (1) depthwise Convolution and&#10;self-Attention can be naturally unified via simple relative attention; (2)&#10;vertically stacking convolution layers and attention layers in a principled way&#10;is surprisingly effective in improving generalization, capacity and efficiency.&#10;Experiments show that our CoAtNets achieve state-of-the-art performance under&#10;different resource constraints across various datasets: Without extra data,&#10;CoAtNet achieves 86.0% ImageNet top-1 accuracy; When pre-trained with 13M&#10;images from ImageNet-21K, our CoAtNet achieves 88.56% top-1 accuracy, matching&#10;ViT-huge pre-trained with 300M images from JFT-300M while using 23x less data;&#10;Notably, when we further scale up CoAtNet with JFT-3B, it achieves 90.88% top-1&#10;accuracy on ImageNet, establishing a new state-of-the-art result.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.03857" label="2112.03857">
-        <attvalues>
-          <attvalue for="0" value="Grounded Language-Image Pre-training" />
-          <attvalue for="1" value="  This paper presents a grounded language-image pre-training (GLIP) model for&#10;learning object-level, language-aware, and semantic-rich visual&#10;representations. GLIP unifies object detection and phrase grounding for&#10;pre-training. The unification brings two benefits: 1) it allows GLIP to learn&#10;from both detection and grounding data to improve both tasks and bootstrap a&#10;good grounding model; 2) GLIP can leverage massive image-text pairs by&#10;generating grounding boxes in a self-training fashion, making the learned&#10;representation semantic-rich. In our experiments, we pre-train GLIP on 27M&#10;grounding data, including 3M human-annotated and 24M web-crawled image-text&#10;pairs. The learned representations demonstrate strong zero-shot and few-shot&#10;transferability to various object-level recognition tasks. 1) When directly&#10;evaluated on COCO and LVIS (without seeing any images in COCO during&#10;pre-training), GLIP achieves 49.8 AP and 26.9 AP, respectively, surpassing many&#10;supervised baselines. 2) After fine-tuned on COCO, GLIP achieves 60.8 AP on val&#10;and 61.5 AP on test-dev, surpassing prior SoTA. 3) When transferred to 13&#10;downstream object detection tasks, a 1-shot GLIP rivals with a fully-supervised&#10;Dynamic Head. Code is released at https://github.com/microsoft/GLIP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.08322" label="2106.08322">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Head: Unifying Object Detection Heads with Attentions" />
-          <attvalue for="1" value="  The complex nature of combining localization and classification in object&#10;detection has resulted in the flourished development of methods. Previous works&#10;tried to improve the performance in various object detection heads but failed&#10;to present a unified view. In this paper, we present a novel dynamic head&#10;framework to unify object detection heads with attentions. By coherently&#10;combining multiple self-attention mechanisms between feature levels for&#10;scale-awareness, among spatial locations for spatial-awareness, and within&#10;output channels for task-awareness, the proposed approach significantly&#10;improves the representation ability of object detection heads without any&#10;computational overhead. Further experiments demonstrate that the effectiveness&#10;and efficiency of the proposed dynamic head on the COCO benchmark. With a&#10;standard ResNeXt-101-DCN backbone, we largely improve the performance over&#10;popular object detectors and achieve a new state-of-the-art at 54.0 AP.&#10;Furthermore, with latest transformer backbone and extra data, we can push&#10;current best COCO result to a new record at 60.6 AP. The code will be released&#10;at https://github.com/microsoft/DynamicHead.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Recent studies focus on improving object detectors from various perspectives: scale-awareness, spatial-awareness and task-awareness. Scale-awareness. &#10;Many researches have empathized the importance of scale-awareness in object detection as objects with vastly different scales often co-exist in natural images. Early works have demonstrated the significance of leveraging image pyramid methods \cite{rcnn, snip, sniper} for multi-scale training. Instead of image pyramid, feature pyramid \cite{fpn} was proposed to improve efficiency by concatenating a pyramid of down-sampled convolution features and had become a standard component in modern object detectors. However, features from different levels are usually extracted from different depth of a network, which causes a noticeable semantics gap. To solve this discrepancy, \cite{panet} proposed to enhance the features in lower layers by bottom-up path augmentation from feature pyramid. Later, \cite{libra} improved it by introducing balanced sampling and balanced feature pyramid. Recently, \cite{spec} proposed a pyramid convolution to extract scale and spatial features simultaneously based on a modified 3-D convolution.&#10;&#10;In this work, we present a scale-aware attention in the detection head, which makes the importance of various feature level adaptive to the input. &#10;&#10;Spatial-awareness.&#10;Previous works have tried to improve the spatial-awareness in object detection for better semantic learning. Convolution neural networks were known to be limited in learning spatial transformations existed in images \cite{od-20y}. Some works mitigate this problem by either increasing the model capability (size) \cite{resnet, resnext} or involving expensive data augmentations \cite{aug}, resulting in extremely high computational cost in inference and training. Later, new convolution operators were proposed to improve the learning of spatial transformations. \cite{dilated} proposed to use dilated convolutions to aggregate contextual information from the exponentially expanded receptive&#10;field. \cite{deform} proposed a deformable convolution to sample spatial locations with additional self-learned offsets. \cite{deformv2} reformulated the offset by introducing a learned feature amplitude and further improved its ability.&#10;&#10;In this work, we present a spatial-aware attention in the detection head, which not only applies attention to each spatial location, but also adaptively aggregates multiple feature levels together for learning a more discriminative representation. &#10;&#10;Task-awareness.&#10;Object detection was originated from a two-stage paradigm \cite{edgebox, rcnn}, which first generates object proposals and then classifies the proposals into different classes and background. \cite{fasterrcnn} formalized the modern two-stage framework by introducing Region Proposal Networks (RPN) to formulate both stages into a single convolution network. Later, one-stage object detector \cite{yolo} became popular due to its high efficiency. \cite{retinanet} further improved the architecture by introducing task-specific branches to surpass the accuracy of two-stage detectors while maintaining the speed of previous one-stage detectors. &#10;&#10;Recently, more works have discovered that various representations of objects could potentially improve the performance.&#10;\cite{maskrcnn} first demonstrated that combining bounding box and segmentation mask of objects can further improve the performance. \cite{fcos} proposed to use center representations to solve object detection in a per-pixel prediction fashion. \cite{atss} further improved the performance of center-based method by automatically selecting positive and negative samples according to statistical characteristics of object. Later, \cite{reppoints} formulated object detection as representative key-points to ease the learning. \cite{centernet} further improved the performance by detecting each object as a triplet, rather than a pair of key-points to reduce incorrect predictions. Most recently, \cite{borderdet} proposed to extract border features from the extreme points of each border to enhance the point feature and archived the state-of-the-art performance. &#10;&#10;In this work, we present a task-aware attention in the detection head, which allows attention to be deployed on channels, which can adaptively favor various tasks, for either single-/two-stage detectors, or box-/center-/keypoint-based detectors.&#10;&#10;More importantly, all the above properties are integrated into a unified attention mechanism in our head design. To our best knowledge, it is the first general detection head framework which takes a step towards understanding what role attention plays in the success of object detection head.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2106.09018" label="2106.09018">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Semi-Supervised Object Detection with Soft Teacher" />
-          <attvalue for="1" value="  This paper presents an end-to-end semi-supervised object detection approach,&#10;in contrast to previous more complex multi-stage methods. The end-to-end&#10;training gradually improves pseudo label qualities during the curriculum, and&#10;the more and more accurate pseudo labels in turn benefit object detection&#10;training. We also propose two simple yet effective techniques within this&#10;framework: a soft teacher mechanism where the classification loss of each&#10;unlabeled bounding box is weighed by the classification score produced by the&#10;teacher network; a box jittering approach to select reliable pseudo boxes for&#10;the learning of box regression. On the COCO benchmark, the proposed approach&#10;outperforms previous methods by a large margin under various labeling ratios,&#10;i.e. 1\%, 5\% and 10\%. Moreover, our approach proves to perform also well when&#10;the amount of labeled data is relatively large. For example, it can improve a&#10;40.9 mAP baseline detector trained using the full COCO training set by +3.6&#10;mAP, reaching 44.5 mAP, by leveraging the 123K unlabeled images of COCO. On the&#10;state-of-the-art Swin Transformer based object detector (58.9 mAP on test-dev),&#10;it can still significantly improve the detection accuracy by +1.5 mAP, reaching&#10;60.4 mAP, and improve the instance segmentation accuracy by +1.2 mAP, reaching&#10;52.4 mAP. Further incorporating with the Object365 pre-trained model, the&#10;detection accuracy reaches 61.3 mAP and the instance segmentation accuracy&#10;reaches 53.0 mAP, pushing the new state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.05740" label="2205.05740">
-        <attvalues>
-          <attvalue for="0" value="Surface Representation for Point Clouds" />
-          <attvalue for="1" value="  Most prior work represents the shapes of point clouds by coordinates.&#10;However, it is insufficient to describe the local geometry directly. In this&#10;paper, we present \textbf{RepSurf} (representative surfaces), a novel&#10;representation of point clouds to \textbf{explicitly} depict the very local&#10;structure. We explore two variants of RepSurf, Triangular RepSurf and Umbrella&#10;RepSurf inspired by triangle meshes and umbrella curvature in computer&#10;graphics. We compute the representations of RepSurf by predefined geometric&#10;priors after surface reconstruction. RepSurf can be a plug-and-play module for&#10;most point cloud models thanks to its free collaboration with irregular points.&#10;Based on a simple baseline of PointNet++ (SSG version), Umbrella RepSurf&#10;surpasses the previous state-of-the-art by a large margin for classification,&#10;segmentation and detection on various benchmarks in terms of performance and&#10;efficiency. With an increase of around \textbf{0.008M} number of parameters,&#10;\textbf{0.04G} FLOPs, and \textbf{1.12ms} inference time, our method achieves&#10;\textbf{94.7\%} (+0.5\%) on ModelNet40, and \textbf{84.6\%} (+1.8\%) on&#10;ScanObjectNN for classification, while \textbf{74.3\%} (+0.8\%) mIoU on S3DIS&#10;6-fold, and \textbf{70.0\%} (+1.6\%) mIoU on ScanNet for segmentation. For&#10;detection, previous state-of-the-art detector with our RepSurf obtains&#10;\textbf{71.2\%} (+2.1\%) mAP$\mathit{_{25}}$, \textbf{54.8\%} (+2.0\%)&#10;mAP$\mathit{_{50}}$ on ScanNetV2, and \textbf{64.9\%} (+1.9\%)&#10;mAP$\mathit{_{25}}$, \textbf{47.7\%} (+2.5\%) mAP$\mathit{_{50}}$ on SUN RGB-D.&#10;Our lightweight Triangular RepSurf performs its excellence on these benchmarks&#10;as well. The code is publicly available at&#10;\url{https://github.com/hancyran/RepSurf}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08721" label="2204.08721">
-        <attvalues>
-          <attvalue for="0" value="Multimodal Token Fusion for Vision Transformers" />
-          <attvalue for="1" value="  Many adaptations of transformers have emerged to address the single-modal&#10;vision tasks, where self-attention modules are stacked to handle input sources&#10;like images. Intuitively, feeding multiple modalities of data to vision&#10;transformers could improve the performance, yet the inner-modal attentive&#10;weights may also be diluted, which could thus undermine the final performance.&#10;In this paper, we propose a multimodal token fusion method (TokenFusion),&#10;tailored for transformer-based vision tasks. To effectively fuse multiple&#10;modalities, TokenFusion dynamically detects uninformative tokens and&#10;substitutes these tokens with projected and aggregated inter-modal features.&#10;Residual positional alignment is also adopted to enable explicit utilization of&#10;the inter-modal alignments after fusion. The design of TokenFusion allows the&#10;transformer to learn correlations among multimodal features, while the&#10;single-modal transformer architecture remains largely intact. Extensive&#10;experiments are conducted on a variety of homogeneous and heterogeneous&#10;modalities and demonstrate that TokenFusion surpasses state-of-the-art methods&#10;in three typical vision tasks: multimodal image-to-image translation, RGB-depth&#10;semantic segmentation, and 3D object detection with point cloud and images. Our&#10;code is available at https://github.com/yikaiw/TokenFusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.02251" label="2204.02251">
-        <attvalues>
-          <attvalue for="0" value="RBGNet: Ray-based Grouping for 3D Object Detection" />
-          <attvalue for="1" value="  As a fundamental problem in computer vision, 3D object detection is&#10;experiencing rapid growth. To extract the point-wise features from the&#10;irregularly and sparsely distributed points, previous methods usually take a&#10;feature grouping module to aggregate the point features to an object candidate.&#10;However, these methods have not yet leveraged the surface geometry of&#10;foreground objects to enhance grouping and 3D box generation. In this paper, we&#10;propose the RBGNet framework, a voting-based 3D detector for accurate 3D object&#10;detection from point clouds. In order to learn better representations of object&#10;shape to enhance cluster features for predicting 3D boxes, we propose a&#10;ray-based feature grouping module, which aggregates the point-wise features on&#10;object surfaces using a group of determined rays uniformly emitted from cluster&#10;centers. Considering the fact that foreground points are more meaningful for&#10;box estimation, we design a novel foreground biased sampling strategy in&#10;downsample process to sample more points on object surfaces and further boost&#10;the detection performance. Our model achieves state-of-the-art 3D detection&#10;performance on ScanNet V2 and SUN RGB-D with remarkable performance gains. Code&#10;will be available at https://github.com/Haiyang-W/RBGNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.01509" label="2203.01509">
-        <attvalues>
-          <attvalue for="0" value="SoftGroup for 3D Instance Segmentation on Point Clouds" />
-          <attvalue for="1" value="  Existing state-of-the-art 3D instance segmentation methods perform semantic&#10;segmentation followed by grouping. The hard predictions are made when&#10;performing semantic segmentation such that each point is associated with a&#10;single class. However, the errors stemming from hard decision propagate into&#10;grouping that results in (1) low overlaps between the predicted instance with&#10;the ground truth and (2) substantial false positives. To address the&#10;aforementioned problems, this paper proposes a 3D instance segmentation method&#10;referred to as SoftGroup by performing bottom-up soft grouping followed by&#10;top-down refinement. SoftGroup allows each point to be associated with multiple&#10;classes to mitigate the problems stemming from semantic prediction errors and&#10;suppresses false positive instances by learning to categorize them as&#10;background. Experimental results on different datasets and multiple evaluation&#10;metrics demonstrate the efficacy of SoftGroup. Its performance surpasses the&#10;strongest prior method by a significant margin of +6.2% on the ScanNet v2&#10;hidden test set and +6.8% on S3DIS Area 5 in terms of AP_50. SoftGroup is also&#10;fast, running at 345ms per scan with a single Titan X on ScanNet v2 dataset.&#10;The source code and trained models for both datasets are available at&#10;\url{https://github.com/thangvubk/SoftGroup.git}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00678" label="2104.00678">
-        <attvalues>
-          <attvalue for="0" value="Group-Free 3D Object Detection via Transformers" />
-          <attvalue for="1" value="  Recently, directly detecting 3D objects from 3D point clouds has received&#10;increasing attention. To extract object representation from an irregular point&#10;cloud, existing methods usually take a point grouping step to assign the points&#10;to an object candidate so that a PointNet-like network could be used to derive&#10;object features from the grouped points. However, the inaccurate point&#10;assignments caused by the hand-crafted grouping scheme decrease the performance&#10;of 3D object detection.&#10;  In this paper, we present a simple yet effective method for directly&#10;detecting 3D objects from the 3D point cloud. Instead of grouping local points&#10;to each object candidate, our method computes the feature of an object from all&#10;the points in the point cloud with the help of an attention mechanism in the&#10;Transformers \cite{vaswani2017attention}, where the contribution of each point&#10;is automatically learned in the network training. With an improved attention&#10;stacking scheme, our method fuses object features in different stages and&#10;generates more accurate object detection results. With few bells and whistles,&#10;the proposed method achieves state-of-the-art 3D object detection performance&#10;on two widely used benchmarks, ScanNet V2 and SUN RGB-D. The code and models&#10;are publicly available at \url{https://github.com/zeliu98/Group-Free-3D}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.13586" label="2104.13586">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Skeleton-based Action Recognition" />
-          <attvalue for="1" value="  Human skeleton, as a compact representation of human action, has received&#10;increasing attention in recent years. Many skeleton-based action recognition&#10;methods adopt graph convolutional networks (GCN) to extract features on top of&#10;human skeletons. Despite the positive results shown in previous works,&#10;GCN-based methods are subject to limitations in robustness, interoperability,&#10;and scalability. In this work, we propose PoseC3D, a new approach to&#10;skeleton-based action recognition, which relies on a 3D heatmap stack instead&#10;of a graph sequence as the base representation of human skeletons. Compared to&#10;GCN-based methods, PoseC3D is more effective in learning spatiotemporal&#10;features, more robust against pose estimation noises, and generalizes better in&#10;cross-dataset settings. Also, PoseC3D can handle multiple-person scenarios&#10;without additional computation cost, and its features can be easily integrated&#10;with other modalities at early fusion stages, which provides a great design&#10;space to further boost the performance. On four challenging datasets, PoseC3D&#10;consistently obtains superior performance, when used alone on skeletons and in&#10;combination with the RGB modality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02036" label="2106.02036">
-        <attvalues>
-          <attvalue for="0" value="Anticipative Video Transformer" />
-          <attvalue for="1" value="  We propose Anticipative Video Transformer (AVT), an end-to-end&#10;attention-based video modeling architecture that attends to the previously&#10;observed video in order to anticipate future actions. We train the model&#10;jointly to predict the next action in a video sequence, while also learning&#10;frame feature encoders that are predictive of successive future frames'&#10;features. Compared to existing temporal aggregation strategies, AVT has the&#10;advantage of both maintaining the sequential progression of observed actions&#10;while still capturing long-range dependencies--both critical for the&#10;anticipation task. Through extensive experiments, we show that AVT obtains the&#10;best reported performance on four popular action anticipation benchmarks:&#10;EpicKitchens-55, EpicKitchens-100, EGTEA Gaze+, and 50-Salads; and it wins&#10;first place in the EpicKitchens-100 CVPR'21 challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.09035" label="1905.09035">
-        <attvalues>
-          <attvalue for="0" value="What Would You Expect? Anticipating Egocentric Actions with&#10;  Rolling-Unrolling LSTMs and Modality Attention" />
-          <attvalue for="1" value="  Egocentric action anticipation consists in understanding which objects the&#10;camera wearer will interact with in the near future and which actions they will&#10;perform. We tackle the problem proposing an architecture able to anticipate&#10;actions at multiple temporal scales using two LSTMs to 1) summarize the past,&#10;and 2) formulate predictions about the future. The input video is processed&#10;considering three complimentary modalities: appearance (RGB), motion (optical&#10;flow) and objects (object-based features). Modality-specific predictions are&#10;fused using a novel Modality ATTention (MATT) mechanism which learns to weigh&#10;modalities in an adaptive fashion. Extensive evaluations on two large-scale&#10;benchmark datasets show that our method outperforms prior art by up to +7% on&#10;the challenging EPIC-Kitchens dataset including more than 2500 actions, and&#10;generalizes to EGTEA Gaze+. Our approach is also shown to generalize to the&#10;tasks of early action recognition and action recognition. Our method is ranked&#10;first in the public leaderboard of the EPIC-Kitchens egocentric action&#10;anticipation challenge 2019. Please see our web pages for code and examples:&#10;http://iplab.dmi.unict.it/rulstm - https://github.com/fpv-iplab/rulstm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.08383" label="2201.08383">
-        <attvalues>
-          <attvalue for="0" value="MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient&#10;  Long-Term Video Recognition" />
-          <attvalue for="1" value="  While today's video recognition systems parse snapshots or short clips&#10;accurately, they cannot connect the dots and reason across a longer range of&#10;time yet. Most existing video architectures can only process &lt;5 seconds of a&#10;video without hitting the computation or memory bottlenecks.&#10;  In this paper, we propose a new strategy to overcome this challenge. Instead&#10;of trying to process more frames at once like most existing methods, we propose&#10;to process videos in an online fashion and cache &quot;memory&quot; at each iteration.&#10;Through the memory, the model can reference prior context for long-term&#10;modeling, with only a marginal cost. Based on this idea, we build MeMViT, a&#10;Memory-augmented Multiscale Vision Transformer, that has a temporal support 30x&#10;longer than existing models with only 4.5% more compute; traditional methods&#10;need &gt;3,000% more compute to do the same. On a wide range of settings, the&#10;increased temporal support enabled by MeMViT brings large gains in recognition&#10;accuracy consistently. MeMViT obtains state-of-the-art results on the AVA,&#10;EPIC-Kitchens-100 action classification, and action anticipation datasets. Code&#10;and models are available at https://github.com/facebookresearch/memvit.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07058" label="2110.07058">
-        <attvalues>
-          <attvalue for="0" value="Ego4D: Around the World in 3,000 Hours of Egocentric Video" />
-          <attvalue for="1" value="  We introduce Ego4D, a massive-scale egocentric video dataset and benchmark&#10;suite. It offers 3,670 hours of daily-life activity video spanning hundreds of&#10;scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique&#10;camera wearers from 74 worldwide locations and 9 different countries. The&#10;approach to collection is designed to uphold rigorous privacy and ethics&#10;standards with consenting participants and robust de-identification procedures&#10;where relevant. Ego4D dramatically expands the volume of diverse egocentric&#10;video footage publicly available to the research community. Portions of the&#10;video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo,&#10;and/or synchronized videos from multiple egocentric cameras at the same event.&#10;Furthermore, we present a host of new benchmark challenges centered around&#10;understanding the first-person visual experience in the past (querying an&#10;episodic memory), present (analyzing hand-object manipulation, audio-visual&#10;conversation, and social interactions), and future (forecasting activities). By&#10;publicly sharing this massive annotated dataset and benchmark suite, we aim to&#10;push the frontier of first-person perception. Project page:&#10;https://ego4d-data.org/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.04558" label="1812.04558">
-        <attvalues>
-          <attvalue for="0" value="Grounded Human-Object Interaction Hotspots from Video" />
-          <attvalue for="1" value="  Learning how to interact with objects is an important step towards embodied&#10;visual intelligence, but existing techniques suffer from heavy supervision or&#10;sensing requirements. We propose an approach to learn human-object interaction&#10;&quot;hotspots&quot; directly from video. Rather than treat affordances as a manually&#10;supervised semantic segmentation task, our approach learns about interactions&#10;by watching videos of real human behavior and anticipating afforded actions.&#10;Given a novel image or video, our model infers a spatial hotspot map indicating&#10;how an object would be manipulated in a potential interaction-- even if the&#10;object is currently at rest. Through results with both first and third person&#10;video, we show the value of grounding affordances in real human-object&#10;interactions. Not only are our weakly supervised hotspots competitive with&#10;strongly supervised affordance methods, but they can also anticipate object&#10;interaction for novel object categories.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.08498" label="1908.08498">
-        <attvalues>
-          <attvalue for="0" value="EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action&#10;  Recognition" />
-          <attvalue for="1" value="  We focus on multi-modal fusion for egocentric action recognition, and propose&#10;a novel architecture for multi-modal temporal-binding, i.e. the combination of&#10;modalities within a range of temporal offsets. We train the architecture with&#10;three modalities -- RGB, Flow and Audio -- and combine them with mid-level&#10;fusion alongside sparse temporal sampling of fused representations. In contrast&#10;with previous works, modalities are fused before temporal aggregation, with&#10;shared modality and fusion weights over time. Our proposed architecture is&#10;trained end-to-end, outperforming individual modalities as well as late-fusion&#10;of modalities.&#10;  We demonstrate the importance of audio in egocentric vision, on per-class&#10;basis, for identifying actions as well as interacting objects. Our method&#10;achieves state of the art results on both the seen and unseen test sets of the&#10;largest egocentric dataset: EPIC-Kitchens, on all metrics using the public&#10;leaderboard.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.07905" label="2104.07905">
-        <attvalues>
-          <attvalue for="0" value="Ego-Exo: Transferring Visual Representations from Third-person to&#10;  First-person Videos" />
-          <attvalue for="1" value="  We introduce an approach for pre-training egocentric video models using&#10;large-scale third-person video datasets. Learning from purely egocentric data&#10;is limited by low dataset scale and diversity, while using purely exocentric&#10;(third-person) data introduces a large domain mismatch. Our idea is to discover&#10;latent signals in third-person video that are predictive of key&#10;egocentric-specific properties. Incorporating these signals as knowledge&#10;distillation losses during pre-training results in models that benefit from&#10;both the scale and diversity of third-person video data, as well as&#10;representations that capture salient egocentric properties. Our experiments&#10;show that our Ego-Exo framework can be seamlessly integrated into standard&#10;video models; it outperforms all baselines when fine-tuned for egocentric&#10;activity recognition, achieving state-of-the-art results on Charades-Ego and&#10;EPIC-Kitchens-100.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.09882" label="1904.09882">
-        <attvalues>
-          <attvalue for="0" value="You2Me: Inferring Body Pose in Egocentric Video via First and Second&#10;  Person Interactions" />
-          <attvalue for="1" value="  The body pose of a person wearing a camera is of great interest for&#10;applications in augmented reality, healthcare, and robotics, yet much of the&#10;person's body is out of view for a typical wearable camera. We propose a&#10;learning-based approach to estimate the camera wearer's 3D body pose from&#10;egocentric video sequences. Our key insight is to leverage interactions with&#10;another person---whose body pose we can directly observe---as a signal&#10;inherently linked to the body pose of the first-person subject. We show that&#10;since interactions between individuals often induce a well-ordered series of&#10;back-and-forth responses, it is possible to learn a temporal model of the&#10;interlinked poses even though one party is largely out of view. We demonstrate&#10;our idea on a variety of domains with dyadic interaction and show the&#10;substantial impact on egocentric body pose estimation, which improves the state&#10;of the art. Video results are available at&#10;http://vision.cs.utexas.edu/projects/you2me/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06324" label="1804.06324">
-        <attvalues>
-          <attvalue for="0" value="Dual CNN Models for Unsupervised Monocular Depth Estimation" />
-          <attvalue for="1" value="  The unsupervised depth estimation is the recent trend by utilizing the&#10;binocular stereo images to get rid of depth map ground truth. In unsupervised&#10;depth computation, the disparity images are generated by training the CNN with&#10;an image reconstruction loss. In this paper, a dual CNN based model is&#10;presented for unsupervised depth estimation with 6 losses (DNM6) with&#10;individual CNN for each view to generate the corresponding disparity map. The&#10;proposed dual CNN model is also extended with 12 losses (DNM12) by utilizing&#10;the cross disparities. The presented DNM6 and DNM12 models are experimented&#10;over KITTI driving and Cityscapes urban database and compared with the recent&#10;state-of-the-art result of unsupervised depth estimation. The code is available&#10;at:&#10;https://github.com/ishmav16/Dual-CNN-Models-for-Unsupervised-Monocular-Depth-Estimation.&#10;" />
-          <attvalue for="2" value="&#10;The image based depth estimation of scene is a very active research area in the field of computer vision. &#10;The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc. &#10;The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc. &#10;Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions. These approaches are classified mainly in three categories namely learning-based stereo \cite{zhou2016learning}, \cite{zbontar2016stereo}, supervised single view depth estimation \cite{eigen2014depth}, \cite{liu2016learning}, and unsupervised depth estimation \cite{garg2016unsupervised}, \cite{godard2017unsupervised}. &#10;The stereo image pairs and ground truth disparity data are needed in order to train the learning-based stereo models. In real scenario, creating such data is very difficult. Moreover, these methods generally create the artificial data which can not represent the real challenges appearing in natural images and depth maps. &#10;The supervised single view depth estimation methods also use ground truth depth to train the model. &#10;The main hurdle in supervised approaches is availability and creation of ground truth depth maps which is always not available in real applications.&#10;&#10;The unsupervised depth estimation methods do not need any ground truth depth maps. Basically, they utilize the underlying theory of epipolar constraints \cite{hartley2003multiple}. Recently, Garg et al. used auto-encoder deep CNN to predict the inverse depth map (i.e. disparity) from left image \cite{garg2016unsupervised}. They computed a warp image (i.e. reconstructed left image) from disparity map and right image. Finally, the error between original and reconstructed left image is used as the loss to train the whole setup in unsupervised manner. This approach is further improved by Godard et al. by incorporating the left-right consistency \cite{godard2017unsupervised}. In left-right consistency, basically two depth maps (i.e. left and right) are generated using auto-encoder only from the left input image. The left input image is used with generated right depth map and the right image is used with generated left depth map to reconstruct the right and left images respectively. Zhou et al. \cite{zhou2017unsupervised} utilized the concepts of unsupervised image depth estimation proposed in \cite{eigen2014depth} and \cite{godard2017unsupervised} to tackle the monocular depth and camera motion estimation in unstructured video sequences in unsupervised learning framework. In one of the recent work, the 3D loss such as photometric quality of frame reconstructions is combined with 2D loss such as pixel-wise or gradient-based loss for learning the depth and ego-motion from monocular video in unsupervised manner \cite{mahjourian2018unsupervised}.&#10;&#10;While the unsupervised based methods have gained the attention in recent times, there is still need of discovering better suited unsupervised networks and loss functions. Through this paper, we propose a dual CNN based model for unsupervised monocular image depth estimation by utilizing the 6 losses (DNM6). We also extend the dual CNN model with 12 losses and generate DNM12 architecture to improve the quality of depth maps. The appearance matching loss, disparity smoothness loss and left-right consistency loss are used in this paper. &#10;The rest of the paper is structured by presenting the proposed dual CNN models DNM6 and DNM12 in Section 2, the experimental results and analysis in Section 3, and the concluding remarks in Section 4.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science, Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="1904.01390" label="1904.01390">
-        <attvalues>
-          <attvalue for="0" value="Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal&#10;  Convolutional Neural Networks" />
-          <attvalue for="1" value="  Facial expression recognition in videos is an active area of research in&#10;computer vision. However, fake facial expressions are difficult to be&#10;recognized even by humans. On the other hand, facial micro-expressions&#10;generally represent the actual emotion of a person, as it is a spontaneous&#10;reaction expressed through human face. Despite of a few attempts made for&#10;recognizing micro-expressions, still the problem is far from being a solved&#10;problem, which is depicted by the poor rate of accuracy shown by the&#10;state-of-the-art methods. A few CNN based approaches are found in the&#10;literature to recognize micro-facial expressions from still images. Whereas, a&#10;spontaneous micro-expression video contains multiple frames that have to be&#10;processed together to encode both spatial and temporal information. This paper&#10;proposes two 3D-CNN methods: MicroExpSTCNN and MicroExpFuseNet, for spontaneous&#10;facial micro-expression recognition by exploiting the spatiotemporal&#10;information in CNN framework. The MicroExpSTCNN considers the full spatial&#10;information, whereas the MicroExpFuseNet is based on the 3D-CNN feature fusion&#10;of the eyes and mouth regions. The experiments are performed over CAS(ME)^2 and&#10;SMIC micro-expression databases. The proposed MicroExpSTCNN model outperforms&#10;the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04176" label="1805.04176">
-        <attvalues>
-          <attvalue for="0" value="A Performance Evaluation of Convolutional Neural Networks for Face Anti&#10;  Spoofing" />
-          <attvalue for="1" value="  In the current era, biometric based access control is becoming more popular&#10;due to its simplicity and ease to use by the users. It reduces the manual work&#10;of identity recognition and facilitates the automatic processing. The face is&#10;one of the most important biometric visual information that can be easily&#10;captured without user cooperation in an uncontrolled environment. Precise&#10;detection of spoofed faces should be on the high priority to make face based&#10;identity recognition and access control robust against possible attacks. The&#10;recently evolved Convolutional Neural Network (CNN) based deep learning&#10;technique has proven as one of the excellent method to deal with the visual&#10;information very effectively. The CNN learns the hierarchical features at&#10;intermediate layers automatically from the data. Several CNN based methods such&#10;as Inception and ResNet have shown outstanding performance for image&#10;classification problem. This paper does a performance evaluation of CNNs for&#10;face anti-spoofing. The Inception and ResNet CNN architectures are used in this&#10;study. The results are computed over benchmark MSU Mobile Face Spoofing&#10;Database. The experiments are done by considering the different aspects such as&#10;the depth of the model, random weight initialization vs weight transfer, fine&#10;tuning vs training from scratch and different learning rate. The favorable&#10;results are obtained using these CNN architectures for face anti-spoofing in&#10;different settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.06701" label="1902.06701">
-        <attvalues>
-          <attvalue for="0" value="HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image&#10;  Classification" />
-          <attvalue for="1" value="  Hyperspectral image (HSI) classification is widely used for the analysis of&#10;remotely sensed images. Hyperspectral imagery includes varying bands of images.&#10;Convolutional Neural Network (CNN) is one of the most frequently used deep&#10;learning based methods for visual data processing. The use of CNN for HSI&#10;classification is also visible in recent works. These approaches are mostly&#10;based on 2D CNN. Whereas, the HSI classification performance is highly&#10;dependent on both spatial and spectral information. Very few methods have&#10;utilized the 3D CNN because of increased computational complexity. This letter&#10;proposes a Hybrid Spectral Convolutional Neural Network (HybridSN) for HSI&#10;classification. Basically, the HybridSN is a spectral-spatial 3D-CNN followed&#10;by spatial 2D-CNN. The 3D-CNN facilitates the joint spatial-spectral feature&#10;representation from a stack of spectral bands. The 2D-CNN on top of the 3D-CNN&#10;further learns more abstract level spatial representation. Moreover, the use of&#10;hybrid CNNs reduces the complexity of the model compared to 3D-CNN alone. To&#10;test the performance of this hybrid approach, very rigorous HSI classification&#10;experiments are performed over Indian Pines, Pavia University and Salinas Scene&#10;remote sensing datasets. The results are compared with the state-of-the-art&#10;hand-crafted as well as end-to-end deep learning based methods. A very&#10;satisfactory performance is obtained using the proposed HybridSN for HSI&#10;classification. The source code can be found at&#10;\url{https://github.com/gokriznastic/HybridSN}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.03554" label="1901.03554">
-        <attvalues>
-          <attvalue for="0" value="CSGAN: Cyclic-Synthesized Generative Adversarial Networks for&#10;  Image-to-Image Transformation" />
-          <attvalue for="1" value="  The primary motivation of Image-to-Image Transformation is to convert an&#10;image of one domain to another domain. Most of the research has been focused on&#10;the task of image transformation for a set of pre-defined domains. Very few&#10;works are reported that actually developed a common framework for&#10;image-to-image transformation for different domains. With the introduction of&#10;Generative Adversarial Networks (GANs) as a general framework for the image&#10;generation problem, there is a tremendous growth in the area of image-to-image&#10;transformation. Most of the research focuses over the suitable objective&#10;function for image-to-image transformation. In this paper, we propose a new&#10;Cyclic-Synthesized Generative Adversarial Networks (CSGAN) for image-to-image&#10;transformation. The proposed CSGAN uses a new objective function (loss) called&#10;Cyclic-Synthesized Loss (CS) between the synthesized image of one domain and&#10;cycled image of another domain. The performance of the proposed CSGAN is&#10;evaluated on two benchmark image-to-image transformation datasets, including&#10;CUHK Face dataset and CMP Facades dataset. The results are computed using the&#10;widely used evaluation metrics such as MSE, SSIM, PSNR, and LPIPS. The&#10;experimental results of the proposed CSGAN approach are compared with the&#10;latest state-of-the-art approaches such as GAN, Pix2Pix, DualGAN, CycleGAN and&#10;PS2GAN. The proposed CSGAN technique outperforms all the methods over CUHK&#10;dataset and exhibits the promising and comparable performance over Facades&#10;dataset in terms of both qualitative and quantitative measures. The code is&#10;available at https://github.com/KishanKancharagunta/CSGAN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.02797" label="1810.02797">
-        <attvalues>
-          <attvalue for="0" value="RCCNet: An Efficient Convolutional Neural Network for Histological&#10;  Routine Colon Cancer Nuclei Classification" />
-          <attvalue for="1" value="  Efficient and precise classification of histological cell nuclei is of utmost&#10;importance due to its potential applications in the field of medical image&#10;analysis. It would facilitate the medical practitioners to better understand&#10;and explore various factors for cancer treatment. The classification of&#10;histological cell nuclei is a challenging task due to the cellular&#10;heterogeneity. This paper proposes an efficient Convolutional Neural Network&#10;(CNN) based architecture for classification of histological routine colon&#10;cancer nuclei named as RCCNet. The main objective of this network is to keep&#10;the CNN model as simple as possible. The proposed RCCNet model consists of only&#10;1,512,868 learnable parameters which are significantly less compared to the&#10;popular CNN models such as AlexNet, CIFARVGG, GoogLeNet, and WRN. The&#10;experiments are conducted over publicly available routine colon cancer&#10;histological dataset &quot;CRCHistoPhenotypes&quot;. The results of the proposed RCCNet&#10;model are compared with five state-of-the-art CNN models in terms of the&#10;accuracy, weighted average F1 score and training time. The proposed method has&#10;achieved a classification accuracy of 80.61% and 0.7887 weighted average F1&#10;score. The proposed RCCNet is more efficient and generalized terms of the&#10;training time and data over-fitting, respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.05383" label="1604.05383">
-        <attvalues>
-          <attvalue for="0" value="Learning Dense Correspondence via 3D-guided Cycle Consistency" />
-          <attvalue for="1" value="  Discriminative deep learning approaches have shown impressive results for&#10;problems where human-labeled ground truth is plentiful, but what about tasks&#10;where labels are difficult or impossible to obtain? This paper tackles one such&#10;problem: establishing dense visual correspondence across different object&#10;instances. For this task, although we do not know what the ground-truth is, we&#10;know it should be consistent across instances of that category. We exploit this&#10;consistency as a supervisory signal to train a convolutional neural network to&#10;predict cross-instance correspondences between pairs of images depicting&#10;objects of the same category. For each pair of training images we find an&#10;appropriate 3D CAD model and render two synthetic views to link in with the&#10;pair, establishing a correspondence flow 4-cycle. We use ground-truth&#10;synthetic-to-synthetic correspondences, provided by the rendering engine, to&#10;train a ConvNet to predict synthetic-to-real, real-to-real and&#10;real-to-synthetic correspondences that are cycle-consistent with the&#10;ground-truth. At test time, no CAD models are required. We demonstrate that our&#10;end-to-end trained ConvNet supervised by cycle-consistency outperforms&#10;state-of-the-art pairwise matching methods in correspondence-related tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1510.05970" label="1510.05970">
-        <attvalues>
-          <attvalue for="0" value="Stereo Matching by Training a Convolutional Neural Network to Compare&#10;  Image Patches" />
-          <attvalue for="1" value="  We present a method for extracting depth information from a rectified image&#10;pair. Our approach focuses on the first stage of many stereo algorithms: the&#10;matching cost computation. We approach the problem by learning a similarity&#10;measure on small image patches using a convolutional neural network. Training&#10;is carried out in a supervised manner by constructing a binary classification&#10;data set with examples of similar and dissimilar pairs of patches. We examine&#10;two network architectures for this task: one tuned for speed, the other for&#10;accuracy. The output of the convolutional neural network is used to initialize&#10;the stereo matching cost. A series of post-processing steps follow: cross-based&#10;cost aggregation, semiglobal matching, a left-right consistency check, subpixel&#10;enhancement, a median filter, and a bilateral filter. We evaluate our method on&#10;the KITTI 2012, KITTI 2015, and Middlebury stereo data sets and show that it&#10;outperforms other approaches on all three data sets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1502.07411" label="1502.07411">
-        <attvalues>
-          <attvalue for="0" value="Learning Depth from Single Monocular Images Using Deep Convolutional&#10;  Neural Fields" />
-          <attvalue for="1" value="  In this article, we tackle the problem of depth estimation from single&#10;monocular images. Compared with depth estimation using multiple images such as&#10;stereo depth perception, depth from monocular images is much more challenging.&#10;Prior work typically focuses on exploiting geometric priors or additional&#10;sources of information, most using hand-crafted features. Recently, there is&#10;mounting evidence that features from deep convolutional neural networks (CNN)&#10;set new records for various vision applications. On the other hand, considering&#10;the continuous characteristic of the depth values, depth estimations can be&#10;naturally formulated as a continuous conditional random field (CRF) learning&#10;problem. Therefore, here we present a deep convolutional neural field model for&#10;estimating depths from single monocular images, aiming to jointly explore the&#10;capacity of deep CNN and continuous CRF. In particular, we propose a deep&#10;structured learning scheme which learns the unary and pairwise potentials of&#10;continuous CRF in a unified deep CNN framework. We then further propose an&#10;equally effective model based on fully convolutional networks and a novel&#10;superpixel pooling method, which is $\sim 10$ times faster, to speedup the&#10;patch-wise convolutions in the deep model. With this more efficient model, we&#10;are able to design deeper networks to pursue better performance. Experiments on&#10;both indoor and outdoor scene datasets demonstrate that the proposed method&#10;outperforms state-of-the-art depth estimation approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.04992" label="1603.04992">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised CNN for Single View Depth Estimation: Geometry to the&#10;  Rescue" />
-          <attvalue for="1" value="  A significant weakness of most current deep Convolutional Neural Networks is&#10;the need to train them using vast amounts of manu- ally labelled data. In this&#10;work we propose a unsupervised framework to learn a deep convolutional neural&#10;network for single view depth predic- tion, without requiring a pre-training&#10;stage or annotated ground truth depths. We achieve this by training the network&#10;in a manner analogous to an autoencoder. At training time we consider a pair of&#10;images, source and target, with small, known camera motion between the two such&#10;as a stereo pair. We train the convolutional encoder for the task of predicting&#10;the depth map for the source image. To do so, we explicitly generate an inverse&#10;warp of the target image using the predicted depth and known inter-view&#10;displacement, to reconstruct the source image; the photomet- ric error in the&#10;reconstruction is the reconstruction loss for the encoder. The acquisition of&#10;this training data is considerably simpler than for equivalent systems,&#10;requiring no manual annotation, nor calibration of depth sensor to camera. We&#10;show that our network trained on less than half of the KITTI dataset (without&#10;any further augmentation) gives com- parable performance to that of the state&#10;of art supervised methods for single view depth estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.03677" label="1609.03677">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Monocular Depth Estimation with Left-Right Consistency" />
-          <attvalue for="1" value="  Learning based methods have shown very promising results for the task of&#10;depth estimation in single images. However, most existing approaches treat&#10;depth prediction as a supervised regression problem and as a result, require&#10;vast quantities of corresponding ground truth depth data for training. Just&#10;recording quality depth data in a range of environments is a challenging&#10;problem. In this paper, we innovate beyond existing approaches, replacing the&#10;use of explicit depth data during training with easier-to-obtain binocular&#10;stereo footage.&#10;  We propose a novel training objective that enables our convolutional neural&#10;network to learn to perform single image depth estimation, despite the absence&#10;of ground truth depth data. Exploiting epipolar geometry constraints, we&#10;generate disparity images by training our network with an image reconstruction&#10;loss. We show that solving for image reconstruction alone results in poor&#10;quality depth images. To overcome this problem, we propose a novel training&#10;loss that enforces consistency between the disparities produced relative to&#10;both the left and right images, leading to improved performance and robustness&#10;compared to existing approaches. Our method produces state of the art results&#10;for monocular depth estimation on the KITTI driving dataset, even outperforming&#10;supervised methods that have been trained with ground truth depth.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.07813" label="1704.07813">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Depth and Ego-Motion from Video" />
-          <attvalue for="1" value="  We present an unsupervised learning framework for the task of monocular depth&#10;and camera motion estimation from unstructured video sequences. We achieve this&#10;by simultaneously training depth and camera pose estimation networks using the&#10;task of view synthesis as the supervisory signal. The networks are thus coupled&#10;via the view synthesis objective during training, but can be applied&#10;independently at test time. Empirical evaluation on the KITTI dataset&#10;demonstrates the effectiveness of our approach: 1) monocular depth performing&#10;comparably with supervised methods that use either ground-truth pose or depth&#10;for training, and 2) pose estimation performing favorably with established SLAM&#10;systems under comparable input settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.05522" label="1802.05522">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using&#10;  3D Geometric Constraints" />
-          <attvalue for="1" value="  We present a novel approach for unsupervised learning of depth and ego-motion&#10;from monocular video. Unsupervised learning removes the need for separate&#10;supervisory signals (depth or ego-motion ground truth, or multi-view video).&#10;Prior work in unsupervised depth learning uses pixel-wise or gradient-based&#10;losses, which only consider pixels in small local neighborhoods. Our main&#10;contribution is to explicitly consider the inferred 3D geometry of the scene,&#10;enforcing consistency of the estimated 3D point clouds and ego-motion across&#10;consecutive frames. This is a challenging task and is solved by a novel&#10;(approximate) backpropagation algorithm for aligning 3D structures.&#10;  We combine this novel 3D-based loss with 2D losses based on photometric&#10;quality of frame reconstructions using estimated depth and ego-motion from&#10;adjacent frames. We also incorporate validity masks to avoid penalizing areas&#10;in which no useful information exists.&#10;  We test our algorithm on the KITTI dataset and on a video dataset captured on&#10;an uncalibrated mobile phone camera. Our proposed approach consistently&#10;improves depth estimates on both datasets, and outperforms the state-of-the-art&#10;for both depth and ego-motion. Because we only require a simple video, learning&#10;depth and ego-motion on large and varied datasets becomes possible. We&#10;demonstrate this by training on the low quality uncalibrated video dataset and&#10;evaluating on KITTI, ranking among top performing prior methods which are&#10;trained on KITTI itself.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.09482" label="2110.09482">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Monocular Depth Estimation with Internal Feature Fusion" />
-          <attvalue for="1" value="  Self-supervised learning for depth estimation uses geometry in image&#10;sequences for supervision and shows promising results. Like many computer&#10;vision tasks, depth network performance is determined by the capability to&#10;learn accurate spatial and semantic representations from images. Therefore, it&#10;is natural to exploit semantic segmentation networks for depth estimation. In&#10;this work, based on a well-developed semantic segmentation network HRNet, we&#10;propose a novel depth estimation network DIFFNet, which can make use of&#10;semantic information in down and upsampling procedures. By applying feature&#10;fusion and an attention mechanism, our proposed method outperforms the&#10;state-of-the-art monocular depth estimation methods on the KITTI benchmark. Our&#10;method also demonstrates greater potential on higher resolution training data.&#10;We propose an additional extended evaluation strategy by establishing a test&#10;set of challenging cases, empirically derived from the standard benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2204.02776" label="2204.02776">
-        <attvalues>
-          <attvalue for="0" value="3D face reconstruction with dense landmarks" />
-          <attvalue for="1" value="  Landmarks often play a key role in face analysis, but many aspects of&#10;identity or expression cannot be represented by sparse landmarks alone. Thus,&#10;in order to reconstruct faces more accurately, landmarks are often combined&#10;with additional signals like depth images or techniques like differentiable&#10;rendering. Can we keep things simple by just using more landmarks? In answer,&#10;we present the first method that accurately predicts 10x as many landmarks as&#10;usual, covering the whole head, including the eyes and teeth. This is&#10;accomplished using synthetic training data, which guarantees perfect landmark&#10;annotations. By fitting a morphable model to these dense landmarks, we achieve&#10;state-of-the-art results for monocular 3D face reconstruction in the wild. We&#10;show that dense landmarks are an ideal signal for integrating face shape&#10;information across frames by demonstrating accurate and expressive facial&#10;performance capture in both monocular and multi-view scenarios. This approach&#10;is also highly efficient: we can predict dense landmarks and fit our 3D face&#10;model at over 150FPS on a single CPU thread. Please see our website:&#10;https://microsoft.github.io/DenseLandmarks/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Reconstructing faces in 3D from images is a mature field at the intersectionof vision and graphics.&#10;We focus our literature review on methods that are closer to our own, and&#10;refer the reader to Morales \etal \cite{morales2021survey} for an extensive survey.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Face Analysis, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1704.00675" label="1704.00675">
-        <attvalues>
-          <attvalue for="0" value="The 2017 DAVIS Challenge on Video Object Segmentation" />
-          <attvalue for="1" value="  We present the 2017 DAVIS Challenge on Video Object Segmentation, a public&#10;dataset, benchmark, and competition specifically designed for the task of video&#10;object segmentation. Following the footsteps of other successful initiatives,&#10;such as ILSVRC and PASCAL VOC, which established the avenue of research in the&#10;fields of scene classification and semantic segmentation, the DAVIS Challenge&#10;comprises a dataset, an evaluation methodology, and a public competition with a&#10;dedicated workshop co-located with CVPR 2017. The DAVIS Challenge follows up on&#10;the recent publication of DAVIS (Densely-Annotated VIdeo Segmentation), which&#10;has fostered the development of several novel state-of-the-art video object&#10;segmentation techniques. In this paper we describe the scope of the benchmark,&#10;highlight the main characteristics of the dataset, define the evaluation&#10;metrics of the competition, and present a detailed analysis of the results of&#10;the participants to the challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Vision Competition, Computer Science, Benchmark Dataset, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1905.10711" label="1905.10711">
-        <attvalues>
-          <attvalue for="0" value="DISN: Deep Implicit Surface Network for High-quality Single-view 3D&#10;  Reconstruction" />
-          <attvalue for="1" value="  Reconstructing 3D shapes from single-view images has been a long-standing&#10;research problem. In this paper, we present DISN, a Deep Implicit Surface&#10;Network which can generate a high-quality detail-rich 3D mesh from an 2D image&#10;by predicting the underlying signed distance fields. In addition to utilizing&#10;global image features, DISN predicts the projected location for each 3D point&#10;on the 2D image, and extracts local features from the image feature maps.&#10;Combining global and local features significantly improves the accuracy of the&#10;signed distance field prediction, especially for the detail-rich areas. To the&#10;best of our knowledge, DISN is the first method that constantly captures&#10;details such as holes and thin structures present in 3D shapes from single-view&#10;images. DISN achieves the state-of-the-art single-view reconstruction&#10;performance on a variety of shape categories reconstructed from both synthetic&#10;and real images. Code is available at https://github.com/xharlie/DISN The&#10;supplementary can be found at&#10;https://xharlie.github.io/images/neurips_2019_supp.pdf&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;There have been extensive studies on learning-based single-view 3D reconstruction using various 3D representations including voxels~\cite{Yan2016,Girdhar16b,zhu2017rethinking,marrnet,drcTulsiani17,wu2018,Yang_2018_ECCV}, octrees~\cite{hspHane17,ogn2017,wang2018adaptive}, points~\cite{fan2017point}, and primitives~\cite{zou20173d,niu_cvpr18}. More recently, Sinha et al.~\cite{Sinha2017} propose to generate the surface of an object using geometry images. Tang et al.~\cite{tang2019skeleton} use shape skeletons for surface reconstruction, however, their method requires additional shape primitives dataset. Groueix et al.~\cite{groueix2018} present AtlasNet to generate surfaces of 3D shapes using a set of parametric surface elements. &#10;Wang et al.~\cite{wang2018pixel2mesh} introduce a graph-based network Pix2Mesh to reconstruct 3D manifold shapes from input images whereas Wang et al.~\cite{wang20193dn} present 3DN to reconstruct a 3D shape by deforming a given source mesh. &#10;&#10;Most of the aforementioned methods use explicit 3D representations and often suffer from problems such as limited resolution and fixed mesh topology. Implicit representations provide an alternative representation to overcome these limitations. In our work, we adopt the Signed Distance Functions (SDF) which are among the most popular implicit surface representations. Several deep learning approaches have utilized SDFs recently. Dai et al.~\cite{3depn} use a voxel-based SDF representation for shape inpainting. Nevertheless, 3D CNNs are known to suffer from high memory usage and computation cost. Park et al.~\cite{park2019deepsdf} introduce DeepSDF for shape completion using an auto-decoder structure. However, their network is not feed-forward and requires optimizing the embedding vector during test time which limits the efficiency and capability of the approach. Chen and Zhang~\cite{chen2018learning} use SDFs in deep networks for the task of shape generation. While their method achieves promising results for the generation task, it fails to recover fine-grained details of 3D objects for single-view reconstruction. Mescheder et al.~\cite{Mescheder2019CVPR} learns an implicit representation by predicting the probability of each cell in a volumetric grid being occupied or not, i.e., being inside or outside of a 3D model. By iteratively subdividing each active cell (i.e., cells surrounded by occupied and empty cells) into sub-cells and repeating the prediction for each sub-cell, they alleviate the problem of the limited resolution of volumetric grids. Finally, in concurrent work, Saito et al.~\cite{pifuSHNMKL19} utilize local image features to predict if a 3D point sample is inside or outside the surface of a mesh and demonstrate high quality human reconstruction results. In contrast, our method not only predicts the sign (i.e., being inside or outside) of sampled points but also the distance which is continuous. We compare our method with recent approaches in Section~\ref{sec:exp:main} and demonstrate state-of-the-art results.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Computer Science, Deep Learning Techniques, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2010.02893" label="2010.02893">
-        <attvalues>
-          <attvalue for="0" value="SAFENet: Self-Supervised Monocular Depth Estimation with Semantic-Aware&#10;  Feature Extraction" />
-          <attvalue for="1" value="  Self-supervised monocular depth estimation has emerged as a promising method&#10;because it does not require groundtruth depth maps during training. As an&#10;alternative for the groundtruth depth map, the photometric loss enables to&#10;provide self-supervision on depth prediction by matching the input image&#10;frames. However, the photometric loss causes various problems, resulting in&#10;less accurate depth values compared with supervised approaches. In this paper,&#10;we propose SAFENet that is designed to leverage semantic information to&#10;overcome the limitations of the photometric loss. Our key idea is to exploit&#10;semantic-aware depth features that integrate the semantic and geometric&#10;knowledge. Therefore, we introduce multi-task learning schemes to incorporate&#10;semantic-awareness into the representation of depth features. Experiments on&#10;KITTI dataset demonstrate that our methods compete or even outperform the&#10;state-of-the-art methods. Furthermore, extensive experiments on different&#10;datasets show its better generalization ability and robustness to various&#10;conditions, such as low-light or adverse weather.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science, Semantic Information" />
-        </attvalues>
-      </node>
-      <node id="2409.12193" label="2409.12193">
-        <attvalues>
-          <attvalue for="0" value="Vista3D: Unravel the 3D Darkside of a Single Image" />
-          <attvalue for="1" value="  We embark on the age-old quest: unveiling the hidden dimensions of objects&#10;from mere glimpses of their visible parts. To address this, we present Vista3D,&#10;a framework that realizes swift and consistent 3D generation within a mere 5&#10;minutes. At the heart of Vista3D lies a two-phase approach: the coarse phase&#10;and the fine phase. In the coarse phase, we rapidly generate initial geometry&#10;with Gaussian Splatting from a single image. In the fine phase, we extract a&#10;Signed Distance Function (SDF) directly from learned Gaussian Splatting,&#10;optimizing it with a differentiable isosurface representation. Furthermore, it&#10;elevates the quality of generation by using a disentangled representation with&#10;two independent implicit functions to capture both visible and obscured aspects&#10;of objects. Additionally, it harmonizes gradients from 2D diffusion prior with&#10;3D-aware diffusion priors by angular diffusion prior composition. Through&#10;extensive evaluation, we demonstrate that Vista3D effectively sustains a&#10;balance between the consistency and diversity of the generated 3D objects.&#10;Demos and code will be available at https://github.com/florinshen/Vista3D.&#10;" />
-          <attvalue for="2" value="&#10;Since the earliest times, our ancestors gazed upon the luminous moon, a symbol of mystery and wonder. Its bright facade, an elegant sphere in the cosmos, has always made us think about what remains hidden: the moon's obscure and elusive dark side. This curiosity, as ancient as human history itself, represents our innate desire to uncover the concealed dimensions that exist beyond the visible. &#10;&#10;This quest, once purely philosophical, has now ventured into the realm of practicality, propelled by the advancements in 3D generative model~\cite{dreamfusion, make3d, anything3d, hash3d, consistent3d}. These technologies enable a broad range of applications, especially in gaming and virtual reality, allowing for the creation of rich, detailed environments and objects without extensive modeling.&#10;&#10;Nevertheless, the development of robust large-scale 3D generative models remains a formidable challenge, predominantly due to the limited availability of 3D data. Numerous attempts~\cite{shap-e, point-e, diffTF3D} have been made to train 3D diffusion models on relatively small 3D datasets, condition on textual or visual prompts; Yet, these endeavors often fall short in creating 3D objects with structural integrity and textural consistency.&#10;&#10;This challenge is further compounded in the context of reconstructing 3D objects from single images. In this context, two primary approaches emerge. The first considers the task as a problem of sparse-view reconstruction. However, this often leads to blurred 3D outputs due to the neglect of unseen elements, resulting in excessively blurred 3D objects~\cite{topology3d, pixelnerf} as most views remain unseen. &#10;&#10;On the other hand, the generative approach, which leverages large-scale 2D diffusion models~\cite{dreamfusion, make3d}, introduces its own set of challenges. Efforts to develop 3D-aware 2D diffusion models~\cite{zero123, zero123++, zeronvs, wonder3D, magic123, anything3d, dtc123, dreamcraft3d} involve fine-tuning 2D models with camera transformation modeling on 3D datasets~\cite{objaverse, objaverseXL}. Nevertheless, the prevalence of synthetic objects in these datasets can lead to a compromise in 2D diversity. This often results in the generation of oversimplified geometries and textures.&#10;&#10;In this paper, we present Vista3D, a framework designed for reconstructing the unseen view (or &quot;darkside&quot;) from a single image. Central to Vista3D is a dual-phase strategy: a coarse phase followed by a fine phase.&#10;&#10;In the coarse phase, we leverage 3D Gaussian splatting~\cite{3dgs} to swiftly create basic geometry and textures. To stabilize Gaussian Splatting optimization, we employ a gradient-based Top-K densification strategy, focusing on Gaussian points with the highest gradients. Additionally, we introduce two novel regularization terms targeting the Gaussian scale and transmittance values, significantly enhancing the convergence speed.&#10;&#10;The fine phase then transforms this initial geometry into signed distance fields~(SDF) for further optimization. Here, we employ FlexiCubes~\cite{flexicubes}, an advanced differentiable isosurface technique, to refine the geometry. This refinement aids in learning the signed distance fields (SDFs), deformation, and interpolation weights. The parameters are optimized by ensuring fidelity to the original image and guided by a score function derived from diffusion priors.&#10;&#10;Despite these advancements, a unified representation and supervision across all views, both seen and unseen, prove insufficient for capturing the unique characteristics of different viewpoints and generating diverse, consistent 3D objects. To address this, we enhance the representation by implementing Disentangled Texture Representation, using two angularly disentangled networks for accurate texture prediction. Furthermore, our Angular-based Composition method amalgamates different diffusion priors, adjusting their gradients within specific angular bounds according to their gradient magnitudes. This strategic adjustment assures 3D consistency while promoting diversity in the unseen views. &#10;&#10;Vista3D excels in efficiently generating diverse and consistent 3D objects from a single image within five minutes. Our extensive evaluations demonstrate its ability to maintain a flexible balance between the consistency and diversity of the generated 3D objects.&#10;&#10;We summarize our contribution as follows:&#10;\begin{itemize}&#10;&#10;\item We present Vista3D, a framework for revealing the 3D darkside of single images, efficiently generating diverse 3D objects using 2D priors.&#10;\item We develop a transition from Gaussian Splatting to isosurface 3D representations, refining coarse geometry with a differentiable isosurface method and disentangled texture for textured mesh creation.&#10;\item We propose an angular composition approach for diffusion priors, constraining their gradient magnitudes to achieve diversity on the 3D darkside without sacrificing 3D consistency.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2209.14988" label="2209.14988">
-        <attvalues>
-          <attvalue for="0" value="DreamFusion: Text-to-3D using 2D Diffusion" />
-          <attvalue for="1" value="  Recent breakthroughs in text-to-image synthesis have been driven by diffusion&#10;models trained on billions of image-text pairs. Adapting this approach to 3D&#10;synthesis would require large-scale datasets of labeled 3D data and efficient&#10;architectures for denoising 3D data, neither of which currently exist. In this&#10;work, we circumvent these limitations by using a pretrained 2D text-to-image&#10;diffusion model to perform text-to-3D synthesis. We introduce a loss based on&#10;probability density distillation that enables the use of a 2D diffusion model&#10;as a prior for optimization of a parametric image generator. Using this loss in&#10;a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a&#10;Neural Radiance Field, or NeRF) via gradient descent such that its 2D&#10;renderings from random angles achieve a low loss. The resulting 3D model of the&#10;given text can be viewed from any angle, relit by arbitrary illumination, or&#10;composited into any 3D environment. Our approach requires no 3D training data&#10;and no modifications to the image diffusion model, demonstrating the&#10;effectiveness of pretrained image diffusion models as priors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.02463" label="2305.02463">
-        <attvalues>
-          <attvalue for="0" value="Shap-E: Generating Conditional 3D Implicit Functions" />
-          <attvalue for="1" value="  We present Shap-E, a conditional generative model for 3D assets. Unlike&#10;recent work on 3D generative models which produce a single output&#10;representation, Shap-E directly generates the parameters of implicit functions&#10;that can be rendered as both textured meshes and neural radiance fields. We&#10;train Shap-E in two stages: first, we train an encoder that deterministically&#10;maps 3D assets into the parameters of an implicit function; second, we train a&#10;conditional diffusion model on outputs of the encoder. When trained on a large&#10;dataset of paired 3D and text data, our resulting models are capable of&#10;generating complex and diverse 3D assets in a matter of seconds. When compared&#10;to Point-E, an explicit generative model over point clouds, Shap-E converges&#10;faster and reaches comparable or better sample quality despite modeling a&#10;higher-dimensional, multi-representation output space. We release model&#10;weights, inference code, and samples at https://github.com/openai/shap-e.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.08051" label="2212.08051">
-        <attvalues>
-          <attvalue for="0" value="Objaverse: A Universe of Annotated 3D Objects" />
-          <attvalue for="1" value="  Massive data corpora like WebText, Wikipedia, Conceptual Captions,&#10;WebImageText, and LAION have propelled recent dramatic progress in AI. Large&#10;neural models trained on such datasets produce impressive results and top many&#10;of today's benchmarks. A notable omission within this family of large-scale&#10;datasets is 3D data. Despite considerable interest and potential applications&#10;in 3D vision, datasets of high-fidelity 3D models continue to be mid-sized with&#10;limited diversity of object categories. Addressing this gap, we present&#10;Objaverse 1.0, a large dataset of objects with 800K+ (and growing) 3D models&#10;with descriptive captions, tags, and animations. Objaverse improves upon&#10;present day 3D repositories in terms of scale, number of categories, and in the&#10;visual diversity of instances within a category. We demonstrate the large&#10;potential of Objaverse via four diverse applications: training generative 3D&#10;models, improving tail category segmentation on the LVIS benchmark, training&#10;open-vocabulary object-navigation models for Embodied AI, and creating a new&#10;benchmark for robustness analysis of vision models. Objaverse can open new&#10;directions for research and enable new applications across the field of AI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.05371" label="2308.05371">
-        <attvalues>
-          <attvalue for="0" value="Flexible Isosurface Extraction for Gradient-Based Mesh Optimization" />
-          <attvalue for="1" value="  This work considers gradient-based mesh optimization, where we iteratively&#10;optimize for a 3D surface mesh by representing it as the isosurface of a scalar&#10;field, an increasingly common paradigm in applications including&#10;photogrammetry, generative modeling, and inverse physics. Existing&#10;implementations adapt classic isosurface extraction algorithms like Marching&#10;Cubes or Dual Contouring; these techniques were designed to extract meshes from&#10;fixed, known fields, and in the optimization setting they lack the degrees of&#10;freedom to represent high-quality feature-preserving meshes, or suffer from&#10;numerical instabilities. We introduce FlexiCubes, an isosurface representation&#10;specifically designed for optimizing an unknown mesh with respect to geometric,&#10;visual, or even physical objectives. Our main insight is to introduce&#10;additional carefully-chosen parameters into the representation, which allow&#10;local flexible adjustments to the extracted mesh geometry and connectivity.&#10;These parameters are updated along with the underlying scalar field via&#10;automatic differentiation when optimizing for a downstream task. We base our&#10;extraction scheme on Dual Marching Cubes for improved topological properties,&#10;and present extensions to optionally generate tetrahedral and&#10;hierarchically-adaptive meshes. Extensive experiments validate FlexiCubes on&#10;both synthetic benchmarks and real-world applications, showing that it offers&#10;significant improvements in mesh quality and geometric fidelity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.03459" label="1909.03459">
-        <attvalues>
-          <attvalue for="0" value="Blind Geometric Distortion Correction on Images Through Deep Learning" />
-          <attvalue for="1" value="  We propose the first general framework to automatically correct different&#10;types of geometric distortion in a single input image. Our proposed method&#10;employs convolutional neural networks (CNNs) trained by using a large synthetic&#10;distortion dataset to predict the displacement field between distorted images&#10;and corrected images. A model fitting method uses the CNN output to estimate&#10;the distortion parameters, achieving a more accurate prediction. The final&#10;corrected image is generated based on the predicted flow using an efficient,&#10;high-quality resampling method. Experimental results demonstrate that our&#10;algorithm outperforms traditional correction methods, and allows for&#10;interesting applications such as distortion transfer, distortion exaggeration,&#10;and co-occurring distortion correction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Geometric distortion correction. For camera lens distortions, pre-calibration techniques have been proposed for correction with known distortion parameters~\cite{tardif2009calibration,duane1971close,tsai1987versatile,heikkila1997four,zhang1999flexible}. However, they are unsuitable for zoom lenses, and the calibration process is usually tedious. On the other hand, auto-calibration methods do not require special calibration patterns and automatically extracts camera parameters from multi-view images~\cite{fitzgibbon2001simultaneous,hartley2007parameter,kukelova2011minimal,ramalingam2010generic,henrique2013radial}. But for many application scenarios, multiple images with different views are unavailable. To address these limitations, automatic distortion correction from a single image has gained more research interest recently. Fitzgibbon~\cite{fitzgibbon2001simultaneous} proposes a division model to approximate the radial distortion curve with higher accuracy and fewer parameters. Wang \etal~\cite{wang2009simple} studied the geometry property of straight lines under the division model and proposed to estimate the distortion parameters through arc fitting. Since plumb line methods rely on robust line detection, Aleman-Flores \etal~\cite{aleman2014automatic} used an improved Hough Transform to improve the robustness while Bukhari and Dailey~\cite{bukhari2013automatic} proposed a sampling method that robustly chooses the circular arcs and determines distortion parameters that are insensitive to outliers. &#10;&#10;For other distortions, such as rotation and perspective, most of the image correction methods rely on the detection of low-level features such as vanishing point, repeated textures, and co-planar circles~\cite{gallagher2005using,lee2012automatic, chaudhury2015automatic, santana2017automatic}. Recently, Zhai \etal~\cite{zhai2016detecting} proposed to use deep convolutional neural networks to estimate the horizon line by aggregating the global image context with the clue of the vanishing point. Workman \etal~\cite{workman2016horizon} goes further and directly estimates the horizon line in the single image. Unlike these specialized methods, our approach is generalizable for multi-type distortion correction using a single network.&#10;&#10;Deformation estimation. There has been recent work on automatic detection of geometric deformation or variations in a single image. Dekel \etal~\cite{dekel2015revealing} use a non-local variations algorithm to automatically detect and correct small deformations between repeating structures from a single image. Wadhwa \etal~\cite{wadhwa2015deviation} fit parametric models to compute the geometric deviations and exaggerate the departure from ideal geometries. Estimating deformations has also been studied in the context of texture images \cite{kim2012symmetry, hays2006discovering, park2009deformed}. None of these techniques are learning-based and are mostly for specialized domains.&#10;&#10;Neural networks for pixel-wise prediction. Recently, convolutional neural networks have been used in many pixel-wise prediction tasks from a single image, such as semantic segmentation~\cite{long2015fully}, depth estimation~\cite{eigen2014depth} and motion prediction~\cite{walker2015dense}. One of the main problems for dense prediction is how to combine multi-scale contextual reasoning with the full-resolution output. Long \etal~\cite{long2015fully} proposed fully convolutional networks which popularized CNNs for dense predictions without fully connected layers. Some methods focus on dilated or atrous convolution~\cite{yu2015multi, chen2018deeplab} which supports exponential expansion the receptive field and systematically aggregate multi-scale contextual information without losing resolution. Another strategy is to use the encoder-decoder architecture~\cite{noh2015learning, badrinarayanan2015segnet, ronneberger2015u}. The encoder gradually reduces the spatial dimension to increase the receptive field of the neuron, while the decoder maps the low-resolution feature maps to full input resolution maps. Noh \etal~\cite{noh2015learning} developed deconvolution and unpooling layers for the decoder part. Badrinarayanan \etal~\cite{badrinarayanan2015segnet} used pooling indices to connect the encoder and the corresponding decoder, making the architecture more memory efficient. Another popular network is U-net~\cite{ronneberger2015u}, which uses the skip connections to combine the contracting paths with the upsampled feature maps. Our networks use an encoder-decoder architecture with residual connection design and achieve more accurate results.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Convolutional Neural Networks, Computer Vision Applications, Mathematics, Physics, Image Processing, Computer Science, Image Distortion Correction" />
-        </attvalues>
-      </node>
-      <node id="2009.02857" label="2009.02857">
-        <attvalues>
-          <attvalue for="0" value="3D Room Layout Estimation Beyond the Manhattan World Assumption" />
-          <attvalue for="1" value="  Predicting 3D room layout from single image is a challenging task with many&#10;applications. In this paper, we propose a new training and post-processing&#10;method for 3D room layout estimation, built on a recent state-of-the-art 3D&#10;room layout estimation model. Experimental results show our method outperforms&#10;state-of-the-art approaches by a large margin in predicting visible room&#10;layout. Our method has obtained the 3rd place in 2020 Holistic Scene Structures&#10;for 3D Vision Workshop.&#10;" />
-          <attvalue for="2" value="&#10;Last decade saw growing attention for recovering 3D room layout from a single image, which would benefit many applications with architectonics, design, virtual and augmented reality. 3D room layout can be viewed as a composition of orientation, corner position, and wall boundaries.&#10;&#10;Various works \cite{delage2006dynamic,hedau2009recovering,lee2017roomnet,zhang2014panocontext} have been developed for room layout estimation.&#10;Recent methods train deep neural networks to detect room corners and boundaries.&#10;Specifically, Xu \etal \cite{xu2017pano2cad} estimated&#10;the room layout and the pose of objects by detecting surface normal orientations.&#10;Yang and Zhang \cite{yang2016efficient} predicted the depth from single image to infer 3D room model.&#10;Yang \etal \cite{yang2019dula} performed semantic segmentation of floor plan segmentation.&#10;Although these methods can predict objects in the room, they lack to represent sharp edges and flat wall planes.&#10;&#10;Most common and the best performing method was to predict room boundaries and corners&#10;with Manhattan world assumption \cite{coughlan1999manhattan},&#10;where all wall planes are aligned with three perpendicular axes.&#10;Fernandez-Labrador \etal \cite{fernandez2020corners} and Zou \etal \cite{zou2018layoutnet} predicted the probability map of boundaries and corners as 2D image, while Sun \etal \cite{SunHSC19} predicted as 1D vector. These approaches well predict fine room layouts, but project every objects to wall texture.&#10;However, the Manhattan world assumption limits the potential application&#10;of predicting real world room layout since numerous rooms with non-rectangular shapes exist.&#10;Moreover, these works tried to predict hidden points while hidden points generate numerous layout candidates as in Fig. \ref{fig:hidden}, where all candidates are plausible.&#10;&#10;In this work, we propose a novel training and post-processing method&#10;to predict the visible layout without the Manhattan world assumption.&#10;As shown in Fig. \ref{fig:qualitative},&#10;our method performs better in accuracy than HorizonNet \cite{SunHSC19},&#10;which is the current state-of-the-art method.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Image Processing, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1703.06241" label="1703.06241">
-        <attvalues>
-          <attvalue for="0" value="RoomNet: End-to-End Room Layout Estimation" />
-          <attvalue for="1" value="  This paper focuses on the task of room layout estimation from a monocular RGB&#10;image. Prior works break the problem into two sub-tasks: semantic segmentation&#10;of floor, walls, ceiling to produce layout hypotheses, followed by an iterative&#10;optimization step to rank these hypotheses. In contrast, we adopt a more direct&#10;formulation of this problem as one of estimating an ordered set of room layout&#10;keypoints. The room layout and the corresponding segmentation is completely&#10;specified given the locations of these ordered keypoints. We predict the&#10;locations of the room layout keypoints using RoomNet, an end-to-end trainable&#10;encoder-decoder network. On the challenging benchmark datasets Hedau and LSUN,&#10;we achieve state-of-the-art performance along with 200x to 600x speedup&#10;compared to the most recent work. Additionally, we present optional extensions&#10;to the RoomNet architecture such as including recurrent computations and memory&#10;units to refine the keypoint locations under the same parametric capacity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.09270" label="1609.09270">
-        <attvalues>
-          <attvalue for="0" value="Pano2CAD: Room Layout From A Single Panorama Image" />
-          <attvalue for="1" value="  This paper presents a method of estimating the geometry of a room and the 3D&#10;pose of objects from a single 360-degree panorama image. Assuming Manhattan&#10;World geometry, we formulate the task as a Bayesian inference problem in which&#10;we estimate positions and orientations of walls and objects. The method&#10;combines surface normal estimation, 2D object detection and 3D object pose&#10;estimation. Quantitative results are presented on a dataset of synthetically&#10;generated 3D rooms containing objects, as well as on a subset of hand-labeled&#10;images from the public SUN360 dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.11977" label="1811.11977">
-        <attvalues>
-          <attvalue for="0" value="DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a&#10;  Single RGB Panorama" />
-          <attvalue for="1" value="  We present a deep learning framework, called DuLa-Net, to predict&#10;Manhattan-world 3D room layouts from a single RGB panorama. To achieve better&#10;prediction accuracy, our method leverages two projections of the panorama at&#10;once, namely the equirectangular panorama-view and the perspective&#10;ceiling-view, that each contains different clues about the room layouts. Our&#10;network architecture consists of two encoder-decoder branches for analyzing&#10;each of the two views. In addition, a novel feature fusion structure is&#10;proposed to connect the two branches, which are then jointly trained to predict&#10;the 2D floor plans and layout heights. To learn more complex room layouts, we&#10;introduce the Realtor360 dataset that contains panoramas of Manhattan-world&#10;room layouts with different numbers of corners. Experimental results show that&#10;our work outperforms recent state-of-the-art in prediction accuracy and&#10;performance, especially in the rooms with non-cuboid layouts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.08094" label="1903.08094">
-        <attvalues>
-          <attvalue for="0" value="Corners for Layout: End-to-End Layout Recovery from 360 Images" />
-          <attvalue for="1" value="  The problem of 3D layout recovery in indoor scenes has been a core research&#10;topic for over a decade. However, there are still several major challenges that&#10;remain unsolved. Among the most relevant ones, a major part of the&#10;state-of-the-art methods make implicit or explicit assumptions on the scenes --&#10;e.g. box-shaped or Manhattan layouts. Also, current methods are computationally&#10;expensive and not suitable for real-time applications like robot navigation and&#10;AR/VR. In this work we present CFL (Corners for Layout), the first end-to-end&#10;model for 3D layout recovery on 360 images. Our experimental results show that&#10;we outperform the state of the art relaxing assumptions about the scene and at&#10;a lower cost. We also show that our model generalizes better to camera position&#10;variations than conventional approaches by using EquiConvs, a type of&#10;convolution applied directly on the sphere projection and hence invariant to&#10;the equirectangular distortions.&#10;  CFL Webpage: https://cfernandezlab.github.io/CFL/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08999" label="1803.08999">
-        <attvalues>
-          <attvalue for="0" value="LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image" />
-          <attvalue for="1" value="  We propose an algorithm to predict room layout from a single image that&#10;generalizes across panoramas and perspective images, cuboid layouts and more&#10;general layouts (e.g. L-shape room). Our method operates directly on the&#10;panoramic image, rather than decomposing into perspective images as do recent&#10;works. Our network architecture is similar to that of RoomNet, but we show&#10;improvements due to aligning the image based on vanishing points, predicting&#10;multiple layout elements (corners, boundaries, size and translation), and&#10;fitting a constrained Manhattan layout to the resulting predictions. Our method&#10;compares well in speed and accuracy to other existing work on panoramas,&#10;achieves among the best accuracy for perspective images, and can handle both&#10;cuboid-shaped and more general Manhattan layouts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.03861" label="1901.03861">
-        <attvalues>
-          <attvalue for="0" value="HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch&#10;  Data Augmentation" />
-          <attvalue for="1" value="  We present a new approach to the problem of estimating the 3D room layout&#10;from a single panoramic image. We represent room layout as three 1D vectors&#10;that encode, at each image column, the boundary positions of floor-wall and&#10;ceiling-wall, and the existence of wall-wall boundary. The proposed network,&#10;HorizonNet, trained for predicting 1D layout, outperforms previous&#10;state-of-the-art approaches. The designed post-processing procedure for&#10;recovering 3D room layouts from 1D predictions can automatically infer the room&#10;shape with low computation cost - it takes less than 20ms for a panorama image&#10;while prior works might need dozens of seconds. We also propose Pano Stretch&#10;Data Augmentation, which can diversify panorama data and be applied to other&#10;panorama-related learning tasks. Due to the limited data available for&#10;non-cuboid layout, we relabel 65 general layout from the current dataset for&#10;finetuning. Our approach shows good performance on general layouts by&#10;qualitative results and cross-validation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.12547" label="2306.12547">
-        <attvalues>
-          <attvalue for="0" value="DGC-GNN: Leveraging Geometry and Color Cues for Visual Descriptor-Free&#10;  2D-3D Matching" />
-          <attvalue for="1" value="  Matching 2D keypoints in an image to a sparse 3D point cloud of the scene&#10;without requiring visual descriptors has garnered increased interest due to its&#10;low memory requirements, inherent privacy preservation, and reduced need for&#10;expensive 3D model maintenance compared to visual descriptor-based methods.&#10;However, existing algorithms often compromise on performance, resulting in a&#10;significant deterioration compared to their descriptor-based counterparts. In&#10;this paper, we introduce DGC-GNN, a novel algorithm that employs a&#10;global-to-local Graph Neural Network (GNN) that progressively exploits&#10;geometric and color cues to represent keypoints, thereby improving matching&#10;accuracy. Our procedure encodes both Euclidean and angular relations at a&#10;coarse level, forming the geometric embedding to guide the point matching. We&#10;evaluate DGC-GNN on both indoor and outdoor datasets, demonstrating that it not&#10;only doubles the accuracy of the state-of-the-art visual descriptor-free&#10;algorithm but also substantially narrows the performance gap between&#10;descriptor-based and descriptor-free methods.&#10;" />
-          <attvalue for="2" value="&#10;Establishing 2D-3D matches plays a crucial role in various computer vision applications, including visual localization~\cite{irschara2009structure, sattler2012improving, sattler2018benchmarking, sarlin2019coarse, wang2021continual, sarlin2020superglue}, 3D reconstruction~\cite{snavely2006photo, cui2015global, schoenberger2016sfm, lindenberger2021pixel}, and Simultaneous Localization and Mapping (SLAM)~\cite{eade2006scalable, mur2015orb, mur2017orb}. &#10;Traditional methods for establishing point-to-point matches involve extracting keypoints and descriptors from a query image, then matching the 2D and 3D descriptors using exhaustive search.&#10;To circumvent the computationally expensive matching process, some approaches~\cite{irschara2009structure, sarlin2019coarse} narrow the search space by employing image retrieval methods~\cite{nister2006scalable, Arandjelovic16} first to identify the most similar images in the database, and then perform descriptor-based image matching~\cite{lowe2004distinctive, detone2018superpoint,dusmanu2019d2, sarlin2020superglue, shi2022clustergnn} between the query and retrieved images.&#10;The 2D-3D correspondences are subsequently established by connecting the 2D-2D image matches with the prebuilt 2D-3D correspondences in the database.&#10;Another approach~\cite{sattler2011fast} is to build 2D-to-3D matches by searching through all point descriptors with an efficient vocabulary-based method.&#10;Sattler et al.~\cite{sattler2012improving, sattler2016efficient} further explore the combination of both 2D-3D and 3D-2D search as an active correspondence search step for a faster and more efficient matching process.&#10;&#10;While descriptor-based algorithms achieve state-of-the-art accuracy, they store and maintain high-dimensional visual descriptors for each point in potentially large 3D point clouds. &#10;The stored model often requires orders of magnitude more storage than the point cloud and images alone~\cite{zhou2022geometry}.&#10;These methods are susceptible to privacy attacks~\cite{dosovitskiy2016inverting, dosovitskiy2016generating, chelani2021privacy, pan2023privacy} and necessitate computationally expensive model maintenance and descriptor update procedures~\cite{zhou2022geometry} when incorporating new descriptors or points into the model.&#10;Several approaches have been proposed to address these limitations. &#10;Yang et al.~\cite{Yang_2022_CVPR} employ learned point selection to sample a subset of the point cloud for scene compression. Other methods~\cite{li2020hierarchical, brachmann2021dsacstar} directly learn a function that maps 2D pixels to 3D coordinates without explicitly storing the 3D scene. Additionally, \cite{ng2022ninjadesc} introduces an adversarial learning framework to develop content-concealing descriptors that prevent privacy leakage.&#10;&#10;Recently, researchers~\cite{campbell2020solving, liu2020learning} have begun exploring deep learning techniques for cross-domain direct 2D-3D matching and pose estimation without visual descriptors, showcasing the potential of descriptor-free matching through differentiable geometric optimization.&#10;The recently proposed GoMatch~\cite{zhou2022geometry} represents significant progress in descriptor-free 2D-3D keypoints matching, achieving reasonable matching performance on a variety of real-world datasets~\cite{li2018megadepth, shotton2013scene, kendall2015posenet}.&#10;GoMatch first identifies keypoints in the query image, which, along with the 3D points from the model, are converted to bearing vectors in the camera coordinate system.&#10;The algorithm employs an attention mechanism~\cite{sarlin2020superglue, vaswani2017attention} to establish reliable 2D-3D correspondences effectively.&#10;While GoMatch attains reasonable accuracy, its performance still significantly lags behind its descriptor-based counterparts~\cite{sarlin2020superglue, sarlin2019coarse, sattler2016efficient}.&#10;Additionally, it relies on geometric cues only from the points and their local neighbors, rendering it incapable of distinguishing geometrically similar structures.&#10;&#10;These observations lead us to two critical questions: &#10;(1) Is geometry the only information we can utilize? &#10;(2) How can we leverage the geometric information derived from the points for matching? &#10;In practice, humans identify correspondences between objects by considering global structures and local geometric cues. &#10;For example, when matching an image to a point cloud as in Fig.~\ref{teaser}, we first locate the building based on its unique structure and then identify the local structure of the roof for matching. &#10;Besides geometric cues, the visual context, such as the color information at each point, also provides constraints for 2D-3D matching. &#10;Importantly, this color information still preserves privacy, as the RGB data from sparse keypoints is insufficient to reconstruct the scene.&#10;&#10;Building upon these observations and the groundwork set by GoMatch, we propose a novel graph-based pipeline, named DGC-GNN, which leverages geometric and color cues in a global-to-local manner for descriptor-free 2D-3D matching. &#10;DGC-GNN encodes position and RGB information for each point and extracts a global distance-angular embedding to guide local point matching. &#10;Taking inspiration from~\cite{shi2022clustergnn}, we employ a cluster-based transformer to constrain information flow within local clusters. &#10;We observe, from real-world datasets, that DGC-GNN leads to substantial improvements in the number of correct matches and the accuracy of pose estimation. Notably, it doubles the accuracy of GoMatch, thereby reducing the gap between descriptor-based and descriptor-free methods. In summary, our paper makes the following contributions:&#10;&#10;\begin{itemize}[leftmargin=0.5cm,noitemsep,topsep=0pt]&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Geometric Computing, Computer Science, Graph Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="2203.12979" label="2203.12979">
-        <attvalues>
-          <attvalue for="0" value="Is Geometry Enough for Matching in Visual Localization?" />
-          <attvalue for="1" value="  In this paper, we propose to go beyond the well-established approach to&#10;vision-based localization that relies on visual descriptor matching between a&#10;query image and a 3D point cloud. While matching keypoints via visual&#10;descriptors makes localization highly accurate, it has significant storage&#10;demands, raises privacy concerns and requires update to the descriptors in the&#10;long-term. To elegantly address those practical challenges for large-scale&#10;localization, we present GoMatch, an alternative to visual-based matching that&#10;solely relies on geometric information for matching image keypoints to maps,&#10;represented as sets of bearing vectors. Our novel bearing vectors&#10;representation of 3D points, significantly relieves the cross-modal challenge&#10;in geometric-based matching that prevented prior work to tackle localization in&#10;a realistic environment. With additional careful architecture design, GoMatch&#10;improves over prior geometric-based matching work with a reduction of&#10;(10.67m,95.7deg) and (1.43m, 34.7deg) in average median pose errors on&#10;Cambridge Landmarks and 7-Scenes, while requiring as little as 1.5/1.7% of&#10;storage capacity in comparison to the best visual-based matching methods. This&#10;confirms its potential and feasibility for real-world localization and opens&#10;the door to future efforts in advancing city-scale visual localization methods&#10;that do not require storing visual descriptors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.02753" label="1506.02753">
-        <attvalues>
-          <attvalue for="0" value="Inverting Visual Representations with Convolutional Networks" />
-          <attvalue for="1" value="  Feature representations, both hand-designed and learned ones, are often hard&#10;to analyze and interpret, even when they are extracted from visual data. We&#10;propose a new approach to study image representations by inverting them with an&#10;up-convolutional neural network. We apply the method to shallow representations&#10;(HOG, SIFT, LBP), as well as to deep networks. For shallow representations our&#10;approach provides significantly better reconstructions than existing methods,&#10;revealing that there is surprisingly rich information contained in these&#10;features. Inverting a deep network trained on ImageNet provides several&#10;insights into the properties of the feature representation learned by the&#10;network. Most strikingly, the colors and the rough contours of an image can be&#10;reconstructed from activations in higher network layers and even from the&#10;predicted class probabilities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.06216" label="1909.06216">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Scene Coordinate Classification and Regression for Visual&#10;  Localization" />
-          <attvalue for="1" value="  Visual localization is critical to many applications in computer vision and&#10;robotics. To address single-image RGB localization, state-of-the-art&#10;feature-based methods match local descriptors between a query image and a&#10;pre-built 3D model. Recently, deep neural networks have been exploited to&#10;regress the mapping between raw pixels and 3D coordinates in the scene, and&#10;thus the matching is implicitly performed by the forward pass through the&#10;network. However, in a large and ambiguous environment, learning such a&#10;regression task directly can be difficult for a single network. In this work,&#10;we present a new hierarchical scene coordinate network to predict pixel scene&#10;coordinates in a coarse-to-fine manner from a single RGB image. The network&#10;consists of a series of output layers, each of them conditioned on the previous&#10;ones. The final output layer predicts the 3D coordinates and the others produce&#10;progressively finer discrete location labels. The proposed method outperforms&#10;the baseline regression-only network and allows us to train compact models&#10;which scale robustly to large environments. It sets a new state-of-the-art for&#10;single-image RGB localization performance on the 7-Scenes, 12-Scenes, Cambridge&#10;Landmarks datasets, and three combined scenes. Moreover, for large-scale&#10;outdoor localization on the Aachen Day-Night dataset, we present a hybrid&#10;approach which outperforms existing scene coordinate regression methods, and&#10;reduces significantly the performance gap w.r.t. explicit feature matching&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.12785" label="2112.12785">
-        <attvalues>
-          <attvalue for="0" value="NinjaDesc: Content-Concealing Visual Descriptors via Adversarial&#10;  Learning" />
-          <attvalue for="1" value="  In the light of recent analyses on privacy-concerning scene revelation from&#10;visual descriptors, we develop descriptors that conceal the input image&#10;content. In particular, we propose an adversarial learning framework for&#10;training visual descriptors that prevent image reconstruction, while&#10;maintaining the matching accuracy. We let a feature encoding network and image&#10;reconstruction network compete with each other, such that the feature encoder&#10;tries to impede the image reconstruction with its generated descriptors, while&#10;the reconstructor tries to recover the input image from the descriptors. The&#10;experimental results demonstrate that the visual descriptors obtained with our&#10;method significantly deteriorate the image reconstruction quality with minimal&#10;impact on correspondence matching and camera localization performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.14628" label="2007.14628">
-        <attvalues>
-          <attvalue for="0" value="Solving the Blind Perspective-n-Point Problem End-To-End With Robust&#10;  Differentiable Geometric Optimization" />
-          <attvalue for="1" value="  Blind Perspective-n-Point (PnP) is the problem of estimating the position and&#10;orientation of a camera relative to a scene, given 2D image points and 3D scene&#10;points, without prior knowledge of the 2D-3D correspondences. Solving for pose&#10;and correspondences simultaneously is extremely challenging since the search&#10;space is very large. Fortunately it is a coupled problem: the pose can be found&#10;easily given the correspondences and vice versa. Existing approaches assume&#10;that noisy correspondences are provided, that a good pose prior is available,&#10;or that the problem size is small. We instead propose the first fully&#10;end-to-end trainable network for solving the blind PnP problem efficiently and&#10;globally, that is, without the need for pose priors. We make use of recent&#10;results in differentiating optimization problems to incorporate geometric model&#10;fitting into an end-to-end learning framework, including Sinkhorn, RANSAC and&#10;PnP algorithms. Our proposed approach significantly outperforms other methods&#10;on synthetic and real data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.00607" label="1804.00607">
-        <attvalues>
-          <attvalue for="0" value="MegaDepth: Learning Single-View Depth Prediction from Internet Photos" />
-          <attvalue for="1" value="  Single-view depth prediction is a fundamental problem in computer vision.&#10;Recently, deep learning methods have led to significant progress, but such&#10;methods are limited by the available training data. Current datasets based on&#10;3D sensors have key limitations, including indoor-only images (NYU), small&#10;numbers of training examples (Make3D), and sparse sampling (KITTI). We propose&#10;to use multi-view Internet photo collections, a virtually unlimited data&#10;source, to generate training data via modern structure-from-motion and&#10;multi-view stereo (MVS) methods, and present a large depth dataset called&#10;MegaDepth based on this idea. Data derived from MVS comes with its own&#10;challenges, including noise and unreconstructable objects. We address these&#10;challenges with new data cleaning methods, as well as automatically augmenting&#10;our data with ordinal depth relations generated using semantic segmentation. We&#10;validate the use of large amounts of Internet data by showing that models&#10;trained on MegaDepth exhibit strong generalization-not only to novel scenes,&#10;but also to other diverse datasets including Make3D, KITTI, and DIW, even when&#10;no images from those datasets are seen during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.11700" label="2204.11700">
-        <attvalues>
-          <attvalue for="0" value="ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for&#10;  Efficient Feature Matching" />
-          <attvalue for="1" value="  Graph Neural Networks (GNNs) with attention have been successfully applied&#10;for learning visual feature matching. However, current methods learn with&#10;complete graphs, resulting in a quadratic complexity in the number of features.&#10;Motivated by a prior observation that self- and cross- attention matrices&#10;converge to a sparse representation, we propose ClusterGNN, an attentional GNN&#10;architecture which operates on clusters for learning the feature matching task.&#10;Using a progressive clustering module we adaptively divide keypoints into&#10;different subgraphs to reduce redundant connectivity, and employ a&#10;coarse-to-fine paradigm for mitigating miss-classification within images. Our&#10;approach yields a 59.7% reduction in runtime and 58.4% reduction in memory&#10;consumption for dense detection, compared to current state-of-the-art GNN-based&#10;matching, while achieving a competitive performance on various computer vision&#10;tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.04815" label="2302.04815">
-        <attvalues>
-          <attvalue for="0" value="To Perceive or Not to Perceive: Lightweight Stacked Hourglass Network" />
-          <attvalue for="1" value="  Human pose estimation (HPE) is a classical task in computer vision that&#10;focuses on representing the orientation of a person by identifying the&#10;positions of their joints. We design a lighterversion of the stacked hourglass&#10;network with minimal loss in performance of the model. The lightweight&#10;2-stacked hourglass has a reduced number of channels with depthwise separable&#10;convolutions, residual connections with concatenation, and residual connections&#10;between the necks of the hourglasses. The final model has a marginal drop in&#10;performance with 79% reduction in the number of parameters and a similar drop&#10;in MAdds&#10;" />
-          <attvalue for="2" value="&#10;Human pose estimation (HPE) is a classical task in computer vision that focuses on representing the orientation of a person by identifying the positions of their joints. HPE can be used to understand and analyze geometric and motion-related information of humans. The stacked-hourglass architecture presented by Newell et al. in \cite{hourglass} is one of the first compelling deep learning-based approaches to HPE, as classical approaches dominated HPE literature prior to it. In this work, repeated bottom-up and top-down processing is utilized to capture information from various scales and intermediate supervision is introduced to iteratively refine predictions at each stage. This led to a significant boost in accuracy as compared to state-of-the-art approaches at the time. &#10;&#10;However, HPE is meant to be a real-time application as it is often used as a precursor to another module. Thus, focus on computational efficiency is vital in this context. In this study, we implement architectural and non-architectural modifications on the stacked hourglass network to obtain a model that is both accurate and computationally efficient.&#10;&#10;In what follows, we provide a brief description of the baseline model. The original architecture is made up of multiple stacked hourglass units, each of which is composed of four downsampling and upsampling levels. At each level, downsampling is achieved through a residual block and a max pooling operation, while upsampling is achieved with a residual block and naive nearest neighbor interpolation. This process ensures that the model captures both local and global information, which is important to coherently understand the full body for an accurate final pose estimate. After each max pooling operation, the network branches off to apply more convolutions through another residual block at the pre-pooling resolution, the result of which is added as a skip connection to the corresponding upsampled feature map in the second half of the hourglass. The output of the model is a heatmap for each joint that models the probability of a joint's presence at each pixel. Intermediate heatmaps after each hourglass are predicted upon which a loss is applied. In addition, these predictions are projected to a larger number of channels and act as the input to the subsequent hourglass, along with the input of the current hourglass and its feature map outputs. The source code and trained models are available at \href{&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Computer Science, Human Pose Estimation, Neural Network Optimization" />
-        </attvalues>
-      </node>
-      <node id="2201.06775" label="2201.06775">
-        <attvalues>
-          <attvalue for="0" value="Deformable One-Dimensional Object Detection for Routing and Manipulation" />
-          <attvalue for="1" value="  Many methods exist to model and track deformable one-dimensional objects&#10;(e.g., cables, ropes, and threads) across a stream of video frames. However,&#10;these methods depend on the existence of some initial conditions. To the best&#10;of our knowledge, the topic of detection methods that can extract those initial&#10;conditions in non-trivial situations has hardly been addressed. The lack of&#10;detection methods limits the use of the tracking methods in real-world&#10;applications and is a bottleneck for fully autonomous applications that work&#10;with these objects.&#10;  This paper proposes an approach for detecting deformable one-dimensional&#10;objects which can handle crossings and occlusions. It can be used for tasks&#10;such as routing and manipulation and automatically provides the initialization&#10;required by the tracking methods. Our algorithm takes an image containing a&#10;deformable object and outputs a chain of fixed-length cylindrical segments&#10;connected with passive spherical joints. The chain follows the natural behavior&#10;of the deformable object and fills the gaps and occlusions in the original&#10;image. Our tests and experiments have shown that the method can correctly&#10;detect deformable one-dimensional objects in various complex conditions.&#10;" />
-          <attvalue for="2" value=" &#10;&#10;M{anipulating} non-rigid objects using robots has long been the subject of research in various contexts~\cite{frobt.2020.00082}. A specific class of non-rigid objects is called Deformable One-dimensional Objects (DOOs) or Deformable Linear Objects (DLOs) and includes objects such as ropes, cables, threads, sutures, and wires. &#10;&#10;In order to achieve full autonomy in applications working with DOOs, one of the essential and challenging parts is perception. Many applications require complete knowledge of the object's initial conditions, even in the occluded parts. Moreover, routing and manipulation applications also require a representation model of the DOO that allows simulation and the computation of its dynamics.&#10;&#10;There are many methods proposed in the medical imaging community that can find the DOOs (known as tubular structures in that context) in the image frame~\cite{KRISSIAN2000130, 28493, Wang2021, Noble2011, 32226}. While these methods are perfected for low signal-to-noise images, and some can even work with self-crossings, they mainly only provide the region in the image containing the DOO and are considered segmentation methods.&#10;&#10;Various algorithms have been proposed to track a DOO across the video frames, even in the presence of occlusions and self-crossings. These methods may devise different tools such as registration methods (e.g., Coherent Point Drift~\cite{5432191}), learning, dynamics models, and simulation to predict and correct the prediction across the consecutive frames~\cite{5980431, 15711, wang2020tracking, rastegarpanah, 6630714, 8560497, af66d77e53304e6bbb64049bb46193cb, 8206058}. While some of these methods can initialize the DOO in the first frame using trivial conditions (e.g., a straight rope in camera view), the other methods require even a simple DOO configuration to be provided to them a priori. Most tracking methods will fail to initialize in even slightly complex initial conditions. &#10;&#10;The initial conditions are not generally provided in real-world applications, and pure segmentation is insufficient. For example, an aerial manipulator working with electrical wires at the top of a utility pole needs to detect the desired wire, including its occluded parts and crossings, and requires a model of the detected wire to perform any task on it.&#10;&#10;The authors of \cite{8972568} proposed a method that trains on rendered images and fine-tunes on real images to detect and track the state of a rope. The method requires tens of thousands of rendered and real images of the same rope for training and needs re-training and tuning for a new DOO. This approach may be acceptable for an industrial application focusing on DOOs of the exact same characteristics, but it may not be practical for many other applications.&#10;&#10;This paper proposes a method to detect the initial conditions of a deformable one-dimensional object. The method fills the occluded parts and works with self-crossings. The output is a single object represented as a discretized model useful for routing, manipulation, and simulation. The detected object can be used to initialize other existing tracking methods to be used in applications such as manipulating DOOs into desired shapes and knots~\cite{9044123, 8403315, 0278364906064819, 4359263}.&#10;&#10;Sections~\ref{sec:problem} and~\ref{sec:method} define the problem and present the proposed method for detection of DOOs; Section~\ref{sec:tests} describes our implementation and shows the experiments and results. Finally, Section~\ref{sec:conclusion} discusses how the proposed method can be further improved in the future.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Object Detection, Computer Science, Robotics, Deformable Objects" />
-        </attvalues>
-      </node>
-      <node id="1911.06283" label="1911.06283">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Learning of State Estimation for Manipulating Deformable&#10;  Linear Objects" />
-          <attvalue for="1" value="  We demonstrate model-based, visual robot manipulation of linear deformable&#10;objects. Our approach is based on a state-space representation of the physical&#10;system that the robot aims to control. This choice has multiple advantages,&#10;including the ease of incorporating physics priors in the dynamics model and&#10;perception model, and the ease of planning manipulation actions. In addition,&#10;physical states can naturally represent object instances of different&#10;appearances. Therefore, dynamics in the state space can be learned in one&#10;setting and directly used in other visually different settings. This is in&#10;contrast to dynamics learned in pixel space or latent space, where&#10;generalization to visual differences are not guaranteed. Challenges in taking&#10;the state-space approach are the estimation of the high-dimensional state of a&#10;deformable object from raw images, where annotations are very expensive on real&#10;data, and finding a dynamics model that is both accurate, generalizable, and&#10;efficient to compute. We are the first to demonstrate self-supervised training&#10;of rope state estimation on real images, without requiring expensive&#10;annotations. This is achieved by our novel self-supervising learning objective,&#10;which is generalizable across a wide range of visual appearances. With&#10;estimated rope states, we train a fast and differentiable neural network&#10;dynamics model that encodes the physics of mass-spring systems. Our method has&#10;a higher accuracy in predicting future states compared to models that do not&#10;involve explicit state estimation and do not use any physics prior, while only&#10;using 3\% of training data. We also show that our approach achieves more&#10;efficient manipulation, both in simulation and on a real robot, when used&#10;within a model predictive controller.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08870" label="2204.08870">
-        <attvalues>
-          <attvalue for="0" value="OpenGlue: Open Source Graph Neural Net Based Pipeline for Image Matching" />
-          <attvalue for="1" value="  We present OpenGlue: a free open-source framework for image matching, that&#10;uses a Graph Neural Network-based matcher inspired by SuperGlue&#10;\cite{sarlin20superglue}. We show that including additional geometrical&#10;information, such as local feature scale, orientation, and affine geometry,&#10;when available (e.g. for SIFT features), significantly improves the performance&#10;of the OpenGlue matcher. We study the influence of the various attention&#10;mechanisms on accuracy and speed. We also present a simple architectural&#10;improvement by combining local descriptors with context-aware descriptors. The&#10;code and pretrained OpenGlue models for the different local features are&#10;publicly available.&#10;" />
-          <attvalue for="2" value="&#10;Image matching is a fundamental problem in multiple-view geometry \cite{Zitovaimageregistration} which attempts to overlay two or more images of one scene or object. Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}. The most widely adopted pipeline for image matching~\cite{Pritchett1998, mishkin2021learning} consists of four consecutive steps: feature detection, feature description, feature matching and filtering, geometric transformation estimation. &#10;\par&#10;The SuperGlue \cite{sarlin20superglue} approach to image matching fits into the standard image matching pipeline replacing classical Nearest Neighbor-based methods. It tackles keypoints matching by formulating it as an end-to-end classification task solved by an Attention-based \cite{DBLP:conf/nips/VaswaniSPUJGKP17} Graph Neural Network (AGNN).&#10;\par&#10;We propose OpenGlue, an open-source framework with an AGNN at the core based on SuperGlue image matcher. Our solution offers several alterations to the original architecture of SuperGlue, whose effect we study in this paper. In contrast to the original method, OpenGlue makes use of an additional geometric information, which can be extracted from many local feature detectors and descriptors, e.g. SIFT~\cite{DBLP:journals/ijcv/Lowe04} -- like orientation, scale and so on. In addition, we study the effect of using Non-Maximum Suppression (NMS) over detected keypoints when applying OpenGlue matcher. We show that applying NMS both at training and testing improve the results of matching and downstream tasks. &#10;\par&#10;Also, we study various efficient attention mechanisms that help speed up the training and inference while also saving memory immensely. We mainly focus on linear~\cite{katharopoulos-et-al-2020} and FAVOR+~\cite{choromanski2020rethinking} versions of attention and show how they contribute to the faster computation time and memory usage. &#10;&#10;Our contribution can be summed up as follows:&#10;\begin{itemize}&#10; \item Presented an open-source framework with flexible components that can be adapted to various needs. The implemented pipeline supports a four-stage approach to image matching and provides an opportunity to substitute or customize each step. For example, at the first stage of local feature extraction user can choose one of the several extractors or add a custom one following a standard showcased interface. We include classical local feature extractors together with learnable methods and provide options for customizing their architecture.&#10; \item Improved SuperGlue results for SIFT-extracted features by adding non-maximum suppression over detected keypoints.&#10; \item Showed the increase in matcher performance by adding keypoint geometrical information into positional encoding.&#10; \item Tested the impact of different attention mechanisms.&#10; \item Combined local and global descriptors in the final stage before matching to include residual information and improve the matching results.&#10;\end{itemize}&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Frameworks, Mathematics, Artificial Intelligence, Computer Science, Image Matching Techniques, Graph Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="1707.09092" label="1707.09092">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions" />
-          <attvalue for="1" value="  Visual localization enables autonomous vehicles to navigate in their&#10;surroundings and augmented reality applications to link virtual to real worlds.&#10;Practical visual localization approaches need to be robust to a wide variety of&#10;viewing condition, including day-night changes, as well as weather and seasonal&#10;variations, while providing highly accurate 6 degree-of-freedom (6DOF) camera&#10;pose estimates. In this paper, we introduce the first benchmark datasets&#10;specifically designed for analyzing the impact of such factors on visual&#10;localization. Using carefully created ground truth poses for query images taken&#10;under a wide variety of conditions, we evaluate the impact of various factors&#10;on 6DOF camera pose estimation accuracy through extensive experiments with&#10;state-of-the-art localization approaches. Based on our results, we draw&#10;conclusions about the difficulty of different conditions, showing that&#10;long-term localization is far from solved, and propose promising avenues for&#10;future work, including sequence-based localization approaches and the need for&#10;better local features. Our benchmark is available at visuallocalization.net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.00338" label="1907.00338">
-        <attvalues>
-          <attvalue for="0" value="Large-scale, real-time visual-inertial localization revisited" />
-          <attvalue for="1" value="  The overarching goals in image-based localization are scale, robustness and&#10;speed. In recent years, approaches based on local features and sparse 3D&#10;point-cloud models have both dominated the benchmarks and seen successful&#10;realworld deployment. They enable applications ranging from robot navigation,&#10;autonomous driving, virtual and augmented reality to device geo-localization.&#10;Recently end-to-end learned localization approaches have been proposed which&#10;show promising results on small scale datasets. However the positioning&#10;accuracy, scalability, latency and compute &amp; storage requirements of these&#10;approaches remain open challenges. We aim to deploy localization at&#10;global-scale where one thus relies on methods using local features and sparse&#10;3D models. Our approach spans from offline model building to real-time&#10;client-side pose fusion. The system compresses appearance and geometry of the&#10;scene for efficient model storage and lookup leading to scalability beyond what&#10;what has been previously demonstrated. It allows for low-latency localization&#10;queries and efficient fusion run in real-time on mobile platforms by combining&#10;server-side localization with real-time visual-inertial-based camera pose&#10;tracking. In order to further improve efficiency we leverage a combination of&#10;priors, nearest neighbor search, geometric match culling and a cascaded pose&#10;candidate refinement step. This combination outperforms previous approaches&#10;when working with large scale models and allows deployment at unprecedented&#10;scale. We demonstrate the effectiveness of our approach on a proof-of-concept&#10;system localizing 2.5 million images against models from four cities in&#10;different regions on the world achieving query latencies in the 200ms range.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1502.00956" label="1502.00956">
-        <attvalues>
-          <attvalue for="0" value="ORB-SLAM: a Versatile and Accurate Monocular SLAM System" />
-          <attvalue for="1" value="  This paper presents ORB-SLAM, a feature-based monocular SLAM system that&#10;operates in real time, in small and large, indoor and outdoor environments. The&#10;system is robust to severe motion clutter, allows wide baseline loop closing&#10;and relocalization, and includes full automatic initialization. Building on&#10;excellent algorithms of recent years, we designed from scratch a novel system&#10;that uses the same features for all SLAM tasks: tracking, mapping,&#10;relocalization, and loop closing. A survival of the fittest strategy that&#10;selects the points and keyframes of the reconstruction leads to excellent&#10;robustness and generates a compact and trackable map that only grows if the&#10;scene content changes, allowing lifelong operation. We present an exhaustive&#10;evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves&#10;unprecedented performance with respect to other state-of-the-art monocular SLAM&#10;approaches. For the benefit of the community, we make the source code public.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.07410" label="1707.07410">
-        <attvalues>
-          <attvalue for="0" value="Toward Geometric Deep SLAM" />
-          <attvalue for="1" value="  We present a point tracking system powered by two deep convolutional neural&#10;networks. The first network, MagicPoint, operates on single images and extracts&#10;salient 2D points. The extracted points are &quot;SLAM-ready&quot; because they are by&#10;design isolated and well-distributed throughout the image. We compare this&#10;network against classical point detectors and discover a significant&#10;performance gap in the presence of image noise. As transformation estimation is&#10;more simple when the detected points are geometrically stable, we designed a&#10;second network, MagicWarp, which operates on pairs of point images (outputs of&#10;MagicPoint), and estimates the homography that relates the inputs. This&#10;transformation engine differs from traditional approaches because it does not&#10;use local point descriptors, only point locations. Both networks are trained&#10;with simple synthetic data, alleviating the requirement of expensive external&#10;camera ground truthing and advanced graphics rendering pipelines. The system is&#10;fast and lean, easily running 30+ FPS on a single CPU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.07247" label="1511.07247">
-        <attvalues>
-          <attvalue for="0" value="NetVLAD: CNN architecture for weakly supervised place recognition" />
-          <attvalue for="1" value="  We tackle the problem of large scale visual place recognition, where the task&#10;is to quickly and accurately recognize the location of a given query&#10;photograph. We present the following three principal contributions. First, we&#10;develop a convolutional neural network (CNN) architecture that is trainable in&#10;an end-to-end manner directly for the place recognition task. The main&#10;component of this architecture, NetVLAD, is a new generalized VLAD layer,&#10;inspired by the &quot;Vector of Locally Aggregated Descriptors&quot; image representation&#10;commonly used in image retrieval. The layer is readily pluggable into any CNN&#10;architecture and amenable to training via backpropagation. Second, we develop a&#10;training procedure, based on a new weakly supervised ranking loss, to learn&#10;parameters of the architecture in an end-to-end manner from images depicting&#10;the same places over time downloaded from Google Street View Time Machine.&#10;Finally, we show that the proposed architecture significantly outperforms&#10;non-learnt image representations and off-the-shelf CNN descriptors on two&#10;challenging place recognition benchmarks, and improves over current&#10;state-of-the-art compact image representations on standard image retrieval&#10;benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.06321" label="1612.06321">
-        <attvalues>
-          <attvalue for="0" value="Large-Scale Image Retrieval with Attentive Deep Local Features" />
-          <attvalue for="1" value="  We propose an attentive local feature descriptor suitable for large-scale&#10;image retrieval, referred to as DELF (DEep Local Feature). The new feature is&#10;based on convolutional neural networks, which are trained only with image-level&#10;annotations on a landmark image dataset. To identify semantically useful local&#10;features for image retrieval, we also propose an attention mechanism for&#10;keypoint selection, which shares most network layers with the descriptor. This&#10;framework can be used for image retrieval as a drop-in replacement for other&#10;keypoint detectors and descriptors, enabling more accurate feature matching and&#10;geometric verification. Our system produces reliable confidence scores to&#10;reject false positives---in particular, it is robust against queries that have&#10;no correct match in the database. To evaluate the proposed descriptor, we&#10;introduce a new large-scale dataset, referred to as Google-Landmarks dataset,&#10;which involves challenges in both database and query such as background&#10;clutter, partial occlusion, multiple landmarks, objects in variable scales,&#10;etc. We show that DELF outperforms the state-of-the-art global and local&#10;descriptors in the large-scale setting by significant margins. Code and dataset&#10;can be found at the project webpage:&#10;https://github.com/tensorflow/models/tree/master/research/delf .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.16236" label="2006.16236">
-        <attvalues>
-          <attvalue for="0" value="Transformers are RNNs: Fast Autoregressive Transformers with Linear&#10;  Attention" />
-          <attvalue for="1" value="  Transformers achieve remarkable performance in several tasks but due to their&#10;quadratic complexity, with respect to the input's length, they are&#10;prohibitively slow for very long sequences. To address this limitation, we&#10;express the self-attention as a linear dot-product of kernel feature maps and&#10;make use of the associativity property of matrix products to reduce the&#10;complexity from $\mathcal{O}\left(N^2\right)$ to $\mathcal{O}\left(N\right)$,&#10;where $N$ is the sequence length. We show that this formulation permits an&#10;iterative implementation that dramatically accelerates autoregressive&#10;transformers and reveals their relationship to recurrent neural networks. Our&#10;linear transformers achieve similar performance to vanilla transformers and&#10;they are up to 4000x faster on autoregressive prediction of very long&#10;sequences.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.14794" label="2009.14794">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Attention with Performers" />
-          <attvalue for="1" value="  We introduce Performers, Transformer architectures which can estimate regular&#10;(softmax) full-rank-attention Transformers with provable accuracy, but using&#10;only linear (as opposed to quadratic) space and time complexity, without&#10;relying on any priors such as sparsity or low-rankness. To approximate softmax&#10;attention-kernels, Performers use a novel Fast Attention Via positive&#10;Orthogonal Random features approach (FAVOR+), which may be of independent&#10;interest for scalable kernel methods. FAVOR+ can be also used to efficiently&#10;model kernelizable attention mechanisms beyond softmax. This representational&#10;power is crucial to accurately compare softmax with other kernels for the first&#10;time on large-scale tasks, beyond the reach of regular Transformers, and&#10;investigate optimal attention-kernels. Performers are linear architectures&#10;fully compatible with regular Transformers and with strong theoretical&#10;guarantees: unbiased or nearly-unbiased estimation of the attention matrix,&#10;uniform convergence and low estimation variance. We tested Performers on a rich&#10;set of tasks stretching from pixel-prediction through text models to protein&#10;sequence modeling. We demonstrate competitive results with other examined&#10;efficient sparse and dense attention methods, showcasing effectiveness of the&#10;novel attention-learning paradigm leveraged by Performers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.05883" label="2211.05883">
-        <attvalues>
-          <attvalue for="0" value="Open-Set Automatic Target Recognition" />
-          <attvalue for="1" value="  Automatic Target Recognition (ATR) is a category of computer vision&#10;algorithms which attempts to recognize targets on data obtained from different&#10;sensors. ATR algorithms are extensively used in real-world scenarios such as&#10;military and surveillance applications. Existing ATR algorithms are developed&#10;for traditional closed-set methods where training and testing have the same&#10;class distribution. Thus, these algorithms have not been robust to unknown&#10;classes not seen during the training phase, limiting their utility in&#10;real-world applications. To this end, we propose an Open-set Automatic Target&#10;Recognition framework where we enable open-set recognition capability for ATR&#10;algorithms. In addition, we introduce a plugin Category-aware Binary Classifier&#10;(CBC) module to effectively tackle unknown classes seen during inference. The&#10;proposed CBC module can be easily integrated with any existing ATR algorithms&#10;and can be trained in an end-to-end manner. Experimental results show that the&#10;proposed approach outperforms many open-set methods on the DSIAC and CIFAR-10&#10;datasets. To the best of our knowledge, this is the first work to address the&#10;open-set classification problem for ATR algorithms. Source code is available&#10;at: https://github.com/bardisafa/Open-set-ATR.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Automatic Target Recognition (ATR) algorithms process information acquired from multiple sensors (i.e. visible and infrared) to recognize the targets appearing in the scene \cite{dudgeon1993overview, rogers1995neural, Patel_ATRICIP2010, Patel_ATR_SRC}. These ATR algorithms are capable of detecting targets at different scales and ranges (Range:1000m - 5000m) \cite{vs2022meta}, which are often not recognized by the naked eye. Further, these algorithms can effectively remove the human intervention from the process of target acquisition and recognition \cite{bhanu1986automatic}, making them a successful automatic recognition system. Thus, ATR systems are extensively used in different commercial and military applications \cite{bhanu1986automatic}. An ATR algorithm consists of two major components; detection and classification. The detection component generally involves a computationally simple region proposal pipeline where target proposals are generated from motion or by eliminating surrounding clutter \cite{kazemi2019deep}. The classification component involves feature extraction and pattern recognition for classifying target categories. &#10;&#10;The rise in Deep Neural Networks (DNNs) \cite{krizhevsky2017imagenet} has drastically improved the performance of computer vision tasks such as image classification, image segmentation and object detection \cite{krizhevsky2017imagenet, ren2015faster, felzenszwalb2004efficient}. Further, the increase in computational resources and datasets have enabled these deep networks to be deployable in real-world applications. Recently, there has been significant interest in improving ATR algorithms using DNNs \cite{kazemi2019deep, vs2022meta} where the DNN frameworks are employed to improve detection and classification components of the ATR algorithms \cite{ren2015faster, he2016deep}. Unlike the traditional ATR methods, these deep learning frameworks eliminate the need for problem-specific discriminative features designed by humans. One of the major challenges in existing DNN frameworks is that they are based on the closed-world assumption; the model assumes only the classes seen during training will appear in the real-world. This assumption limits the utility of existing DNN-based ATR systems in actual applications. Therefore, it is important to develop a DNN-based ATR algorithm that detects unknown classes not seen during training.&#10;&#10;Open-set recognition is a problem of handling `unknown' classes that are not seen during training, whereas traditional closed-set classifiers assume that only known classes appear during testing \cite{geng2020recent, vaze2021open}. Existing open-set recognition methods are generally classified into generative and discriminative methods. G-OpenMax \cite{ge2017generative} is a generative method which trains a model with synthesized unknown data. However, these methods cannot be applied to natural images other than hand-written characters due to the difficulty of generative modelling. OpenMax \cite{bendale2016towards} is a discriminative method which trains a model with softmax cross-entropy classification loss for closed-set samples and unknown classes are detected by applying a threshold on predicted probabilities or logit scores \cite{bendale2016towards}. Outlier (also called anomaly or novelty) detection can be incorporated into the concept of open-set classification as an unknown detector. &#10;Some of the generic methods for outlier detection are one-class extensions of discriminative models such as SVM \cite{manevitz2001one} or forests \cite{liu2008isolation} and subspace methods \cite{ringberg2007sensitivity}. However, outlier detectors are not open-set classifiers by themselves because they have no discriminative power within known classes and it cannot be used to build a general purpose open-set classifier. Therefore, extending discriminative open-set recognition methods is a promising approach towards solving the open-set ATR problem. &#10;&#10;In this work, we employ ResNet 18 \cite{he2016deep} as the base network for ATR closed-set classification. Our experiments have shown that for open-set classification, employing OpenMax \cite{bendale2016towards} or SoftMax \cite{Neal_2018_ECCV} for classifying targets in short ranges (0m-1000m) works reasonably well. However, the open-set classification performance drops as the input data range increases (see in Table \ref{open-acc}). This is due to the fact that as the range increases the target size decreases and the network finds it difficult even to learn discriminative known class features resulting in poor performance. To overcome this issue, we propose a novel plugin Category-aware Binary Classifier (CBC) module which effectively tackles the unknown categories seen during inference at various ranges. In CBC, for each category, a binary classifier is trained using the samples of the corresponding category as known samples. The remaining samples from other categories are treated as unknown samples. In this way, each binary classifier learns category-aware decision boundaries, which decide whether a given sample belongs to that category or not. In the testing phase, the closed-set classifier identifies the nearest known class and the corresponding binary classifier's score is leveraged to decide whether it is a known or unknown sample. Hence, combining multi-class and category-ware decision boundaries effectively classifies known and unknown samples at various ranges making it more suitable for ATR (see Fig. 1). This paper makes the following contributions:&#10;\begin{itemize}[topsep=0pt,noitemsep,leftmargin=*] &#10; \item To the best of our knowledge, this is the first work to consider an open-set recognition problem for automatic target recognition. &#10; \item We propose a plugin Category-aware Binary Classifier (CBC) module to effectively tackle unknown classes seen during inference at various ranges.&#10; \item We consider the DSIAC and CIFAR-10 benchmark datasets for experimental analysis and show that the proposed method outperforms many open-set methods.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Automatic Target Recognition, Mathematics, Artificial Intelligence, Computer Science, Open-Set Classification" />
-        </attvalues>
-      </node>
-      <node id="2110.03143" label="2110.03143">
-        <attvalues>
-          <attvalue for="0" value="Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using&#10;  Meta-Learning" />
-          <attvalue for="1" value="  Object detectors trained on large-scale RGB datasets are being extensively&#10;employed in real-world applications. However, these RGB-trained models suffer a&#10;performance drop under adverse illumination and lighting conditions. Infrared&#10;(IR) cameras are robust under such conditions and can be helpful in real-world&#10;applications. Though thermal cameras are widely used for military applications&#10;and increasingly for commercial applications, there is a lack of robust&#10;algorithms to robustly exploit the thermal imagery due to the limited&#10;availability of labeled thermal data. In this work, we aim to enhance the&#10;object detection performance in the thermal domain by leveraging the labeled&#10;visible domain data in an Unsupervised Domain Adaptation (UDA) setting. We&#10;propose an algorithm agnostic meta-learning framework to improve existing UDA&#10;methods instead of proposing a new UDA strategy. We achieve this by&#10;meta-learning the initial condition of the detector, which facilitates the&#10;adaptation process with fine updates without overfitting or getting stuck at&#10;local optima. However, meta-learning the initial condition for the detection&#10;scenario is computationally heavy due to long and intractable computation&#10;graphs. Therefore, we propose an online meta-learning paradigm which performs&#10;online updates resulting in a short and tractable computation graph. To this&#10;end, we demonstrate the superiority of our method over many baselines in the&#10;UDA setting, producing a state-of-the-art thermal detector for the KAIST and&#10;DSIAC datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.08581" label="1811.08581">
-        <attvalues>
-          <attvalue for="0" value="Recent Advances in Open Set Recognition: A Survey" />
-          <attvalue for="1" value="  In real-world recognition/classification tasks, limited by various objective&#10;factors, it is usually difficult to collect training samples to exhaust all&#10;classes when training a recognizer or classifier. A more realistic scenario is&#10;open set recognition (OSR), where incomplete knowledge of the world exists at&#10;training time, and unknown classes can be submitted to an algorithm during&#10;testing, requiring the classifiers to not only accurately classify the seen&#10;classes, but also effectively deal with the unseen ones. This paper provides a&#10;comprehensive survey of existing open set recognition techniques covering&#10;various aspects ranging from related definitions, representations of models,&#10;datasets, evaluation criteria, and algorithm comparisons. Furthermore, we&#10;briefly analyze the relationships between OSR and its related tasks including&#10;zero-shot, one-shot (few-shot) recognition/learning techniques, classification&#10;with reject option, and so forth. Additionally, we also overview the open world&#10;recognition which can be seen as a natural extension of OSR. Importantly, we&#10;highlight the limitations of existing approaches and point out some promising&#10;subsequent research directions in this field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.07418" label="1707.07418">
-        <attvalues>
-          <attvalue for="0" value="Generative OpenMax for Multi-Class Open Set Classification" />
-          <attvalue for="1" value="  We present a conceptually new and flexible method for multi-class open set&#10;classification. Unlike previous methods where unknown classes are inferred with&#10;respect to the feature or decision distance to the known classes, our approach&#10;is able to provide explicit modelling and decision score for unknown classes.&#10;The proposed method, called Gener- ative OpenMax (G-OpenMax), extends OpenMax&#10;by employing generative adversarial networks (GANs) for novel category image&#10;synthesis. We validate the proposed method on two datasets of handwritten&#10;digits and characters, resulting in superior results over previous deep&#10;learning based method OpenMax Moreover, G-OpenMax provides a way to visualize&#10;samples representing the unknown classes from open space. Our simple and&#10;effective approach could serve as a new direction to tackle the challenging&#10;multi-class open set classification problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.06233" label="1511.06233">
-        <attvalues>
-          <attvalue for="0" value="Towards Open Set Deep Networks" />
-          <attvalue for="1" value="  Deep networks have produced significant gains for various visual recognition&#10;problems, leading to high impact academic and commercial applications. Recent&#10;work in deep networks highlighted that it is easy to generate images that&#10;humans would never classify as a particular object class, yet networks classify&#10;such images high confidence as that given class - deep network are easily&#10;fooled with images humans do not consider meaningful. The closed set nature of&#10;deep networks forces them to choose from one of the known classes leading to&#10;such artifacts. Recognition in the real world is open set, i.e. the recognition&#10;system should reject unknown/unseen classes at test time. We present a&#10;methodology to adapt deep networks for open set recognition, by introducing a&#10;new model layer, OpenMax, which estimates the probability of an input being&#10;from an unknown class. A key element of estimating the unknown probability is&#10;adapting Meta-Recognition concepts to the activation patterns in the&#10;penultimate layer of the network. OpenMax allows rejection of &quot;fooling&quot; and&#10;unrelated open set images presented to the system; OpenMax greatly reduces the&#10;number of obvious errors made by a deep network. We prove that the OpenMax&#10;concept provides bounded open space risk, thereby formally providing an open&#10;set recognition solution. We evaluate the resulting open set deep networks&#10;using pre-trained networks from the Caffe Model-zoo on ImageNet 2012 validation&#10;data, and thousands of fooling and open set images. The proposed OpenMax model&#10;significantly outperforms open set recognition accuracy of basic deep networks&#10;as well as deep networks with thresholding of SoftMax probabilities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.03385" label="1512.03385">
-        <attvalues>
-          <attvalue for="0" value="Deep Residual Learning for Image Recognition" />
-          <attvalue for="1" value="  Deeper neural networks are more difficult to train. We present a residual&#10;learning framework to ease the training of networks that are substantially&#10;deeper than those used previously. We explicitly reformulate the layers as&#10;learning residual functions with reference to the layer inputs, instead of&#10;learning unreferenced functions. We provide comprehensive empirical evidence&#10;showing that these residual networks are easier to optimize, and can gain&#10;accuracy from considerably increased depth. On the ImageNet dataset we evaluate&#10;residual nets with a depth of up to 152 layers---8x deeper than VGG nets but&#10;still having lower complexity. An ensemble of these residual nets achieves&#10;3.57% error on the ImageNet test set. This result won the 1st place on the&#10;ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100&#10;and 1000 layers.&#10;  The depth of representations is of central importance for many visual&#10;recognition tasks. Solely due to our extremely deep representations, we obtain&#10;a 28% relative improvement on the COCO object detection dataset. Deep residual&#10;nets are foundations of our submissions to ILSVRC &amp; COCO 2015 competitions,&#10;where we also won the 1st places on the tasks of ImageNet detection, ImageNet&#10;localization, COCO detection, and COCO segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.08562" label="2409.08562">
-        <attvalues>
-          <attvalue for="0" value="CSS: Overcoming Pose and Scene Challenges in Crowd-Sourced 3D Gaussian&#10;  Splatting" />
-          <attvalue for="1" value="  We introduce Crowd-Sourced Splatting (CSS), a novel 3D Gaussian Splatting&#10;(3DGS) pipeline designed to overcome the challenges of pose-free scene&#10;reconstruction using crowd-sourced imagery. The dream of reconstructing&#10;historically significant but inaccessible scenes from collections of&#10;photographs has long captivated researchers. However, traditional 3D techniques&#10;struggle with missing camera poses, limited viewpoints, and inconsistent&#10;lighting. CSS addresses these challenges through robust geometric priors and&#10;advanced illumination modeling, enabling high-quality novel view synthesis&#10;under complex, real-world conditions. Our method demonstrates clear&#10;improvements over existing approaches, paving the way for more accurate and&#10;flexible applications in AR, VR, and large-scale 3D reconstruction.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Reconstructing historically significant or inaccessible scenes from existing crowdsourced or archival photographs \cite{doi:10.1177/14780771231168224,22d9af9531054c88a8a46c194f6b3712,Magnani_Douglass_Schroder_Reeves_Braun_2020} has long been a key objective in fields such as virtual reality (VR), augmented reality (AR), and autonomous driving \cite{survey}. While 3D Gaussian Splatting (3DGS) \cite{3dgs} has made significant strides in achieving high-fidelity, real-time rendering through its differentiability, its primary focus is on optimized visual representation rather than the reconstruction of scenes from diverse, unstructured image sources \cite{10630672}. In contrast, crowdsourcing has emerged as a transformative approach for data aggregation in 3D visual computation, significantly lowering the cost and time required for data collection compared to traditional methods \cite{10.1145/3569090}. By harnessing the diversity and widespread availability of crowdsourced imagery, researchers can achieve more comprehensive and varied datasets, laying the groundwork for more detailed and nuanced 3D reconstructions \cite{csnerf}.&#10;&#10;Crowdsourced imagery presents unique challenges that complicate the construction of 3DGS models. The primary issues include the lack of precise camera poses \cite{colmapfree}, sparse and limited viewpoints \cite{10446844}, and inconsistent lighting conditions across images \cite{wildinnerf}. These challenges are further compounded by the absence of positional priors, and the temporal and spatial variations in lighting \cite{9413356} caused by the asynchronous nature of crowdsourced data. Such inconsistencies disrupt traditional methods like COLMAP \cite{colmap1,colmap2}, which rely on accurate Structure from Motion (SfM) \cite{DBLP:journals/corr/abs-2312-07504,fan2024instantsplat,10446550}, and they particularly affect the ability of novel view synthesis methods to maintain consistent color and texture across perspectives \cite{Gao_2024_CVPR}. As a result, synthesizing accurate and visually coherent new viewpoints from crowdsourced data remains a significant challenge \cite{DBLP:journals/jair/MogadalaKK21}. Addressing these issues requires more robust approaches, such as leveraging large visual models \cite{mast3r}, which can better generalize across diverse and noisy inputs to enhance pose estimation \cite{Wang_2024_CVPR} and illumination consistency \cite{10480646}.&#10;&#10;To address these challenges, we introduce Crowd-Sourced Splatting (CSS), a novel pose-free 3DGS generation pipeline designed for crowd-sourced imagery. The key innovations of CSS include: (1) A robust initialization mechanism utilizing expert models and extensive 2D geometric priors to overcome the lack of precise camera poses and inconsistent imaging conditions in crowdsourced data. (2) An advanced illumination model employing high-order spherical harmonics to harmonize varying lighting conditions and perspectives, ensuring consistent and high-quality 3DGS under complex crowdsourced scenarios. (3) The development of CSScenes, a comprehensive dataset sourced from internet-based crowd imagery, providing benchmarks across diverse indoor and outdoor environments.&#10;&#10;These innovations mark a major advancement in 3D visual computation, directly addressing the key challenges of crowdsourced imagery. By eliminating the need for precise camera poses and incorporating advanced illumination modeling, CSS generates high-quality 3DGS models even in complex and varied conditions. The introduction of CSScenes strengthens the practical impact of our approach, providing a valuable benchmark for future research and development in this field.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Crowd-Sourced Data Processing, Computer Vision Techniques, Computer Vision, Mathematics, Physics, Geometric Computing, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2406.16289" label="2406.16289">
-        <attvalues>
-          <attvalue for="0" value="Crowd-Sourced NeRF: Collecting Data from Production Vehicles for 3D&#10;  Street View Reconstruction" />
-          <attvalue for="1" value="  Recently, Neural Radiance Fields (NeRF) achieved impressive results in novel&#10;view synthesis. Block-NeRF showed the capability of leveraging NeRF to build&#10;large city-scale models. For large-scale modeling, a mass of image data is&#10;necessary. Collecting images from specially designed data-collection vehicles&#10;can not support large-scale applications. How to acquire massive high-quality&#10;data remains an opening problem. Noting that the automotive industry has a huge&#10;amount of image data, crowd-sourcing is a convenient way for large-scale data&#10;collection. In this paper, we present a crowd-sourced framework, which utilizes&#10;substantial data captured by production vehicles to reconstruct the scene with&#10;the NeRF model. This approach solves the key problem of large-scale&#10;reconstruction, that is where the data comes from and how to use them. Firstly,&#10;the crowd-sourced massive data is filtered to remove redundancy and keep a&#10;balanced distribution in terms of time and space. Then a structure-from-motion&#10;module is performed to refine camera poses. Finally, images, as well as poses,&#10;are used to train the NeRF model in a certain block. We highlight that we&#10;present a comprehensive framework that integrates multiple modules, including&#10;data selection, sparse 3D reconstruction, sequence appearance embedding, depth&#10;supervision of ground surface, and occlusion completion. The complete system is&#10;capable of effectively processing and reconstructing high-quality 3D scenes&#10;from crowd-sourced data. Extensive quantitative and qualitative experiments&#10;were conducted to validate the performance of our system. Moreover, we proposed&#10;an application, named first-view navigation, which leveraged the NeRF model to&#10;generate 3D street view and guide the driver with a synthesized video.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.07504" label="2312.07504">
-        <attvalues>
-          <attvalue for="0" value="COLMAP-Free 3D Gaussian Splatting" />
-          <attvalue for="1" value="  While neural rendering has led to impressive advances in scene reconstruction&#10;and novel view synthesis, it relies heavily on accurately pre-computed camera&#10;poses. To relax this constraint, multiple efforts have been made to train&#10;Neural Radiance Fields (NeRFs) without pre-processed camera poses. However, the&#10;implicit representations of NeRFs provide extra challenges to optimize the 3D&#10;structure and camera poses at the same time. On the other hand, the recently&#10;proposed 3D Gaussian Splatting provides new opportunities given its explicit&#10;point cloud representations. This paper leverages both the explicit geometric&#10;representation and the continuity of the input video stream to perform novel&#10;view synthesis without any SfM preprocessing. We process the input frames in a&#10;sequential manner and progressively grow the 3D Gaussians set by taking one&#10;input frame at a time, without the need to pre-compute the camera poses. Our&#10;method significantly improves over previous approaches in view synthesis and&#10;camera pose estimation under large motion changes. Our project page is&#10;https://oasisyang.github.io/colmap-free-3dgs&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2311.17123" label="2311.17123">
-        <attvalues>
-          <attvalue for="0" value="ConTex-Human: Free-View Rendering of Human from a Single Image with&#10;  Texture-Consistent Synthesis" />
-          <attvalue for="1" value="  In this work, we propose a method to address the challenge of rendering a 3D&#10;human from a single image in a free-view manner. Some existing approaches could&#10;achieve this by using generalizable pixel-aligned implicit fields to&#10;reconstruct a textured mesh of a human or by employing a 2D diffusion model as&#10;guidance with the Score Distillation Sampling (SDS) method, to lift the 2D&#10;image into 3D space. However, a generalizable implicit field often results in&#10;an over-smooth texture field, while the SDS method tends to lead to a&#10;texture-inconsistent novel view with the input image. In this paper, we&#10;introduce a texture-consistent back view synthesis module that could transfer&#10;the reference image content to the back view through depth and text-guided&#10;attention injection. Moreover, to alleviate the color distortion that occurs in&#10;the side region, we propose a visibility-aware patch consistency regularization&#10;for texture mapping and refinement combined with the synthesized back view&#10;texture. With the above techniques, we could achieve high-fidelity and&#10;texture-consistent human rendering from a single image. Experiments conducted&#10;on both real and synthetic data demonstrate the effectiveness of our method and&#10;show that our approach outperforms previous baseline methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.09358" label="1907.09358">
-        <attvalues>
-          <attvalue for="0" value="Trends in Integration of Vision and Language Research: A Survey of&#10;  Tasks, Datasets, and Methods" />
-          <attvalue for="1" value="  Interest in Artificial Intelligence (AI) and its applications has seen&#10;unprecedented growth in the last few years. This success can be partly&#10;attributed to the advancements made in the sub-fields of AI such as machine&#10;learning, computer vision, and natural language processing. Much of the growth&#10;in these fields has been made possible with deep learning, a sub-area of&#10;machine learning that uses artificial neural networks. This has created&#10;significant interest in the integration of vision and language. In this survey,&#10;we focus on ten prominent tasks that integrate language and vision by&#10;discussing their problem formulation, methods, existing datasets, evaluation&#10;measures, and compare the results obtained with corresponding state-of-the-art&#10;methods. Our efforts go beyond earlier surveys which are either task-specific&#10;or concentrate only on one type of visual content, i.e., image or video.&#10;Furthermore, we also provide some potential future directions in this field of&#10;research with an anticipation that this survey stimulates innovative thoughts&#10;and ideas to address the existing challenges and build new applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.09756" label="2406.09756">
-        <attvalues>
-          <attvalue for="0" value="Grounding Image Matching in 3D with MASt3R" />
-          <attvalue for="1" value="  Image Matching is a core component of all best-performing algorithms and&#10;pipelines in 3D vision. Yet despite matching being fundamentally a 3D problem,&#10;intrinsically linked to camera pose and scene geometry, it is typically treated&#10;as a 2D problem. This makes sense as the goal of matching is to establish&#10;correspondences between 2D pixel fields, but also seems like a potentially&#10;hazardous choice. In this work, we take a different stance and propose to cast&#10;matching as a 3D task with DUSt3R, a recent and powerful 3D reconstruction&#10;framework based on Transformers. Based on pointmaps regression, this method&#10;displayed impressive robustness in matching views with extreme viewpoint&#10;changes, yet with limited accuracy. We aim here to improve the matching&#10;capabilities of such an approach while preserving its robustness. We thus&#10;propose to augment the DUSt3R network with a new head that outputs dense local&#10;features, trained with an additional matching loss. We further address the&#10;issue of quadratic complexity of dense matching, which becomes prohibitively&#10;slow for downstream applications if not carefully treated. We introduce a fast&#10;reciprocal matching scheme that not only accelerates matching by orders of&#10;magnitude, but also comes with theoretical guarantees and, lastly, yields&#10;improved results. Extensive experiments show that our approach, coined MASt3R,&#10;significantly outperforms the state of the art on multiple matching tasks. In&#10;particular, it beats the best published methods by 30% (absolute improvement)&#10;in VCRE AUC on the extremely challenging Map-free localization dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\mypar{Keypoint-based matching} has been a cornerstone of computer vision. &#10;Matching is carried out in three distinct stages: keypoint detection, locally invariant description and nearest-neighbor search in descriptor space.&#10;Departing from the former handcrafted methods like SIFT~\cite{sift,rublee11orb}, modern approaches have been shifting towards learning-based data-driven schemes for detecting keypoints~\cite{laguna19keynet,mishkin18repeatability,verdie15tilde,zhang17discriminative}, describing them~\cite{balntas17hpatches,he18local,tian19sosnet, germain20s2dnet} or both at the same time~\cite{ma21survey,revaud19r2d2,wang21p2net,bhowmik20reinforced,superpoint,aslfeatCVPR20}. &#10;Overall, keypoint-based approaches are predominant in many benchmarks~\cite{balntas17hpatches, schoenberger17,ltlv,jin19}, underscoring their enduring value in tasks requiring high precision and speed~\cite{csurka18,schoenberger17}. One notable issue, however, is they reduce matching to a local problem, \ie discarding its holistic nature.&#10;SuperGlue and similar approaches~\cite{superglue, lightglue} thus propose to perform global reasoning in the last pairing step leveraging stronger priors to guide matching, yet leaving the detection and description local. While successful, it is still limited by the local nature of keypoints and their inability to remain invariant to strong viewpoint changes.&#10;&#10;\mypar{Dense matching.}&#10;In contrast to keypoint-based approaches, semi-dense~\cite{sun2021loftr, jiang2021cotr, bokman22, aspanformer22,tang2022quadtree, pats} and dense approaches~\cite{edstedt2023dkm,roma,dgcnet18, glunet, truong21,pmatch,pdcnetp,dfm21} offer a different paradigm for establishing image correspondences, considering all &#10;possible pixel associations. &#10;Very reminiscent of optical flow approaches~\cite{raft, flowformer, flowformerpp, videoflow, dong2023rethinking, gma21}, they are usually employing coarse-to-fine schemes to decrease computational complexity. Overall, these methods aim to consider matching from a global perspective, at the cost of increased computational resources. &#10;Dense matching has proven effective in scenarios where detailed spatial relationships and textures are critical for understanding scene geometry, leading to top performance on many benchmarks~\cite{imc22,imc23,mapfree,wxbs,sun2021loftr,superglue} that are especially challenging for keypoints due to extreme changes in viewpoint or illumination. &#10;These approaches still cast matching as a 2D problem, which limits their usage for visual localization.&#10;&#10;\mypar{Camera Pose estimation}&#10;techniques vary widely, but the most successful strategies, for speed, accuracy and robustness trade-off, are fundamentally based on pixel matching~\cite{colmap, hloc, vsfm}. &#10;The constant improvement of matching methods has fostered the introduction of more challenging camera pose estimation benchmarks, such as Aachen Day-Night, InLoc, CO3D or Map-free~\cite{inloc,aachen,co3d,mapfree}, all featuring strong viewpoint and/or illumination changes.&#10;The most challenging of them is undoubtedly Map-free~\cite{mapfree}, a localization dataset for which a single reference image is provided but no map, with viewpoint changes up to 180$^\circ$.&#10;&#10;\mypar{Grounding matching in 3D}&#10;thus becomes a crucial necessity in these challenging conditions where classical 2D-based matching utterly falls short.&#10;Leveraging priors about the physical properties of the scene in order to improve accuracy or robustness has been widely explored in the past,&#10;but most previous works settle for leveraging epipolar constraints for semi-supervised learning of correspondences without any fundamental change~\cite{yang19dataadaptive,patch2pix,yao19monet,caps_epipolar_2020,epipolar_vit_2020,bhalgat_epipolar_2023,kloepfer_scenes_2024,inductive_bias_epipolar_2022}. &#10;Toft \etal~\cite{mono_depth_helps20}, on its part, propose to improve keypoint descriptors by rectifying images with perspective transformations obtained from an off-the-shelf monocular depth predictor. &#10;Recently, diffusion for pose~\cite{posediffusion} or rays~\cite{raydiffusion}, although not matching approaches strictly speaking, show promising performance by incorporating 3D geometric constraints into their pose estimation formulation. Finally, the recent~\duster~\cite{dust3r} explore the possibility of recovering correspondences from the a-priori harder task of 3D reconstruction from uncalibrated images.&#10;Despite not being trained explicitly for matching, this approach yields promising results, topping the Map-free leaderboard~\cite{mapfree}.&#10;Our contribution is to pursue this idea, by regressing local features and explicitly training them for pairwise matching.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2312.14132" label="2312.14132">
-        <attvalues>
-          <attvalue for="0" value="DUSt3R: Geometric 3D Vision Made Easy" />
-          <attvalue for="1" value="  Multi-view stereo reconstruction (MVS) in the wild requires to first estimate&#10;the camera parameters e.g. intrinsic and extrinsic parameters. These are&#10;usually tedious and cumbersome to obtain, yet they are mandatory to triangulate&#10;corresponding pixels in 3D space, which is the core of all best performing MVS&#10;algorithms. In this work, we take an opposite stance and introduce DUSt3R, a&#10;radically novel paradigm for Dense and Unconstrained Stereo 3D Reconstruction&#10;of arbitrary image collections, i.e. operating without prior information about&#10;camera calibration nor viewpoint poses. We cast the pairwise reconstruction&#10;problem as a regression of pointmaps, relaxing the hard constraints of usual&#10;projective camera models. We show that this formulation smoothly unifies the&#10;monocular and binocular reconstruction cases. In the case where more than two&#10;images are provided, we further propose a simple yet effective global alignment&#10;strategy that expresses all pairwise pointmaps in a common reference frame. We&#10;base our network architecture on standard Transformer encoders and decoders,&#10;allowing us to leverage powerful pretrained models. Our formulation directly&#10;provides a 3D model of the scene as well as depth information, but&#10;interestingly, we can seamlessly recover from it, pixel matches, relative and&#10;absolute camera. Exhaustive experiments on all these tasks showcase that the&#10;proposed DUSt3R can unify various 3D vision tasks and set new SoTAs on&#10;monocular/multi-view depth estimation as well as relative pose estimation. In&#10;summary, DUSt3R makes many geometric 3D vision tasks easy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.01949" label="2208.01949">
-        <attvalues>
-          <attvalue for="0" value="Negative Frames Matter in Egocentric Visual Query 2D Localization" />
-          <attvalue for="1" value="  The recently released Ego4D dataset and benchmark significantly scales and&#10;diversifies the first-person visual perception data. In Ego4D, the Visual&#10;Queries 2D Localization task aims to retrieve objects appeared in the past from&#10;the recording in the first-person view. This task requires a system to&#10;spatially and temporally localize the most recent appearance of a given object&#10;query, where query is registered by a single tight visual crop of the object in&#10;a different scene.&#10;  Our study is based on the three-stage baseline introduced in the Episodic&#10;Memory benchmark. The baseline solves the problem by detection and tracking:&#10;detect the similar objects in all the frames, then run a tracker from the most&#10;confident detection result. In the VQ2D challenge, we identified two&#10;limitations of the current baseline. (1) The training configuration has&#10;redundant computation. Although the training set has millions of instances,&#10;most of them are repetitive and the number of unique object is only around&#10;14.6k. The repeated gradient computation of the same object lead to an&#10;inefficient training; (2) The false positive rate is high on background frames.&#10;This is due to the distribution gap between training and evaluation. During&#10;training, the model is only able to see the clean, stable, and labeled frames,&#10;but the egocentric videos also have noisy, blurry, or unlabeled background&#10;frames. To this end, we developed a more efficient and effective solution.&#10;Concretely, we bring the training loop from ~15 days to less than 24 hours, and&#10;we achieve 0.17% spatial-temporal AP, which is 31% higher than the baseline.&#10;Our solution got the first ranking on the public leaderboard. Our code is&#10;publicly available at https://github.com/facebookresearch/vq2d_cvpr.&#10;" />
-          <attvalue for="2" value="&#10;The task of Visual Queries 2D Localization in egocentric videos aims to retrieve objects from episodic memory. This task can be described as `when was the last time that I saw X', where X is a query represented by a visual crop. In a real-world application, X can be the user's wallet -- after registering the wallet image as an image crop, the user can easily retrieve the wallet's location by a visual query system. &#10;&#10;As a formal definition, given a visual crop $v$ of a query object $o$ and a query frame (time) $q$, the system should provide a `response track' $r$ that follows $o$ before frame $q$. The response track $r$ is a temporally contiguous set of bounding boxes surrounding the object $o$. $r = \{r_s,r_{s+1},\cdots,r_{e-1},r_e\}$, where $s$ and $e$ are the first and last frames that the object is visible, respectively, and $r_i$ represents a bounding box $(x, y, w, h)$ in frame $i$.&#10;&#10;The predictions are mainly evaluated by spatio-temporal Average-Precision (stAP)~\cite{gkioxari2015finding}. It measures how closely the predicted bounding box set matches with the ground-truth response track in the spatio-temporal extent. &#10;&#10;The VQ2D baseline is a three-step detection + tracking solution. We summarize the process in Fig.~\ref{fig:baseline}. Firstly, an object detector is trained using frame-crop pairs. The model inputs are a query object crop and a video frame, and the output is the object in the frame that matches the crop. When the training of the detection model is converged, we apply the detector to all the frames in the video, then create a similarity curve, where each point scores the similarity of the top-1 prediction in the frame to the query object. From the curve we can detect the peak and locate the bounding box in the peak frame. Finally, the response object track is generated by running a tracker in forward and backward directions from the peak bounding box. &#10;&#10;Our solution is developed from the baseline with the following improvements: &#10;&#10;(1) Accelerated training and evaluation. The original baseline requires around 3600 hours to finish the training loops for the object detection model in step 1, and it takes nearly 400 hours for a full evaluation in step 2 and 3. We explore a new set of training parameters aiming at training efficiency. Additionally, we refactor training and evaluation code to enable parallel computing. This results in optimized training and evaluation process, reducing execution time to 24 hours and 12 hours, respectively. This means we have improved the speed of the research time cycle to $1500\%$ in training and $3200\%$ in testing. The improvement significantly shortens the experiment loop and can give fast feedback to new research ideas for us and the community.&#10;&#10;(2) Error type analysis. We also study the error in detection and tracking results. The model produces false positives on negative frames, which is not reflected in the default detection evaluation. This is because the detection evaluation metric in step 1 disregards background frames and leads to a discrepancy in the final retrieval evaluation (stAP). The detection metric only requires the model to have a high recall but this VQ2D task also requires a high-precision system. We visualize different types of false-positive errors in negative frames in the video and show that simply training our model with extra negative frames can bring a performance gain in the final evaluation. We wish this exposition to the community can lead to more investigations in this area.&#10;&#10;(3) A more precise implementation. Our model can surpass the state-of-the-art by $0.02$ in stAP25, by $3.8\%$ in success rate, and by $4.1\%$ in recovery ratio on the validation set. On the test set, we achieve $0.17\%$ spatial-temporal AP, which is $31\%$ higher than the baseline. The main reason for such large improvement is that we optimize the backbone detector with egocentric data: we identified that the pre-trained model can increase its capability to represent Ego4D data by backbone optimization, and training with negative frames which do not include query objects can reduce false positives of the detector. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Egocentric Perception, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1411.6031" label="1411.6031">
-        <attvalues>
-          <attvalue for="0" value="Finding Action Tubes" />
-          <attvalue for="1" value="  We address the problem of action detection in videos. Driven by the latest&#10;progress in object detection from 2D images, we build action models using rich&#10;feature hierarchies derived from shape and kinematic cues. We incorporate&#10;appearance and motion in two ways. First, starting from image region proposals&#10;we select those that are motion salient and thus are more likely to contain the&#10;action. This leads to a significant reduction in the number of regions being&#10;processed and allows for faster computations. Second, we extract&#10;spatio-temporal feature representations to build strong classifiers using&#10;Convolutional Neural Networks. We link our predictions to produce detections&#10;consistent in time, which we call action tubes. We show that our approach&#10;outperforms other techniques in the task of action detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.04310" label="2111.04310">
-        <attvalues>
-          <attvalue for="0" value="Residual-Guided Learning Representation for Self-Supervised Monocular&#10;  Depth Estimation" />
-          <attvalue for="1" value="  Photometric consistency loss is one of the representative objective functions&#10;commonly used for self-supervised monocular depth estimation. However, this&#10;loss often causes unstable depth predictions in textureless or occluded regions&#10;due to incorrect guidance. Recent self-supervised learning approaches tackle&#10;this issue by utilizing feature representations explicitly learned from&#10;auto-encoders, expecting better discriminability than the input image. Despite&#10;the use of auto-encoded features, we observe that the method does not embed&#10;features as discriminative as auto-encoded features. In this paper, we propose&#10;residual guidance loss that enables the depth estimation network to embed the&#10;discriminative feature by transferring the discriminability of auto-encoded&#10;features. We conducted experiments on the KITTI benchmark and verified our&#10;method's superiority and orthogonality on other state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;Accurate depth estimation is essential for many computer vision applications, such as autonomous driving and robotics navigation. &#10;In these days of explosively increasing raw data, self-supervised monocular depth estimation has emerged as a promising approach. &#10;However, conventional self-supervised monocular depth estimation methods \cite{zhou2017unsupervised, yang2017unsupervised, yin2018geonet} often adopt self-supervision losses, such as a photometric consistency loss and a feature consistency loss despite the underlying side effects. &#10;Specifically, the photometric consistency loss tends to match nearby wrong pixels of similar appearance, while the feature consistency loss often results in the local minimum problem, causing degenerative discriminability.&#10;&#10;A recent self-supervised depth estimation method \cite{shu2020feature} proposes a feature-metric loss that utilizes the discriminability of auto-encoded feature representations, expecting explicitly trained auto-encoder to have better capacity on encoding more discriminative information in textureless or occluded regions.&#10;Despite the reasonable feature enhancement, we observed that the auto-encoded feature and the original depth feature of the depth estimation network derive different loss landscapes, especially in the erroneous regions, meaning that the discriminability of the depth feature is not sufficiently improved at the level of the auto-encoded feature.&#10;&#10;In this paper, we focus on addressing the inaccurate depth predictions on the erroneous regions by sufficiently transferring the auto-encoded feature's discriminability to the depth feature. &#10;We propose a novel residual guidance loss with a self-supervision framework that minimizes the difference between residual depths derived by the auto-encoded feature and the depth feature.&#10;Our method is based on the insight that harmonizing the residual depths of auto-encoded features and depth features leads to the assimilation of their loss landscapes.&#10;We adopt the Gauss-Newton refinement approach \cite{yu2020fast} to derive residual depths from the auto-encoded feature and depth feature. We evaluate to verify the effectiveness and show that our method outperforms other state-of-the-art methods on the KITTI 2015 benchmark.&#10;&#10;In summary, our major contributions are summarized as follows;&#10;(a) We provide a mathematical explanation that residual depths guide the discriminability information, &#10;(b) Residual-guidance loss is proposed to match residual depths between features generated by auto-encoder and depth network.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2007.10603" label="2007.10603">
-        <attvalues>
-          <attvalue for="0" value="Feature-metric Loss for Self-supervised Learning of Depth and Egomotion" />
-          <attvalue for="1" value="  Photometric loss is widely used for self-supervised depth and egomotion&#10;estimation. However, the loss landscapes induced by photometric differences are&#10;often problematic for optimization, caused by plateau landscapes for pixels in&#10;textureless regions or multiple local minima for less discriminative pixels. In&#10;this work, feature-metric loss is proposed and defined on feature&#10;representation, where the feature representation is also learned in a&#10;self-supervised manner and regularized by both first-order and second-order&#10;derivatives to constrain the loss landscapes to form proper convergence basins.&#10;Comprehensive experiments and detailed analysis via visualization demonstrate&#10;the effectiveness of the proposed feature-metric loss. In particular, our&#10;method improves state-of-the-art methods on KITTI from 0.885 to 0.925 measured&#10;by $\delta_1$ for depth estimation, and significantly outperforms previous&#10;method for visual odometry.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;In this section, we review related works of self-supervised learning for two tasks, \ie, monocular depth and egomotion estimation, as well as visual representation learning.&#10;&#10;Monocular depth and egomotion estimation: &#10;SfMLearner is a pioneering work~\cite{sfmlearner} for this task, where geometry estimation from DepthNet and PoseNet is supervised by photometric loss. To tackle moving objects that break the assumption of static scenes, optical flow is estimated to compensate these moving pixels~\cite{GeoNet,epc,epc++,dfnet}, segmentation masks provided by pre-trained segmentation models are also to handle potential moving objects separately~\cite{struct2depth,signet,learnk}.&#10;&#10;More geometric priors are also used to strengthen the self-supervised learning. Depth-normal consistency loss is proposed as as extra constraint~\cite{lego,dnc}. 3D consistency between point clouds backprojected from adjacent views is considered in~\cite{vid2deep,glnet,sc-sfmlearner}. In addition, binocular videos are used for training to solve both scale ambiguity and scene dynamics~\cite{undeepvo,monodepth2,epc,epc++}, where only inference can be carried on monocular video.&#10;&#10;In contrast to all above methods where focuses are on the geometry parts of the task, deep feature reconstruction~\cite{dfr} proposed to use deep features from pre-trained models to define reconstruction loss. Our method shares the same spirit, but takes a step further to explicitly learn deep features from the geometry problem under the same self-supervised learning framework.&#10;&#10;Visual representation learning: &#10;It is of great interest of self-supervised visual representation learning for downstream tasks. Without explicitly provided labels, the losses are defined by manipulating the data itself in different ways, which could be reconstructing input data~\cite{stacked,denoise,afl,avb}, predicting spatial transformations~\cite{sp1,sp2,sp3,sp4}, coloring grayscale input images~\cite{colorization1,colorization2,colorization3,colorization4} etc. Our work belongs to reconstruct the input through an auto-encoder network. Different from previous works mainly aiming for learning better features for recognition tasks, our method is designed to learn better features for the geometry task.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2003.13017" label="2003.13017">
-        <attvalues>
-          <attvalue for="0" value="Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation&#10;  and Gauss-Newton Refinement" />
-          <attvalue for="1" value="  Almost all previous deep learning-based multi-view stereo (MVS) approaches&#10;focus on improving reconstruction quality. Besides quality, efficiency is also&#10;a desirable feature for MVS in real scenarios. Towards this end, this paper&#10;presents a Fast-MVSNet, a novel sparse-to-dense coarse-to-fine framework, for&#10;fast and accurate depth estimation in MVS. Specifically, in our Fast-MVSNet, we&#10;first construct a sparse cost volume for learning a sparse and high-resolution&#10;depth map. Then we leverage a small-scale convolutional neural network to&#10;encode the depth dependencies for pixels within a local region to densify the&#10;sparse high-resolution depth map. At last, a simple but efficient Gauss-Newton&#10;layer is proposed to further optimize the depth map. On one hand, the&#10;high-resolution depth map, the data-adaptive propagation method and the&#10;Gauss-Newton layer jointly guarantee the effectiveness of our method. On the&#10;other hand, all modules in our Fast-MVSNet are lightweight and thus guarantee&#10;the efficiency of our approach. Besides, our approach is also memory-friendly&#10;because of the sparse depth representation. Extensive experimental results show&#10;that our method is 5$\times$ and 14$\times$ faster than Point-MVSNet and&#10;R-MVSNet, respectively, while achieving comparable or even better results on&#10;the challenging Tanks and Temples dataset as well as the DTU dataset. Code is&#10;available at https://github.com/svip-lab/FastMVSNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.10563" label="2109.10563">
-        <attvalues>
-          <attvalue for="0" value="Improving 360 Monocular Depth Estimation via Non-local Dense Prediction&#10;  Transformer and Joint Supervised and Self-supervised Learning" />
-          <attvalue for="1" value="  Due to difficulties in acquiring ground truth depth of equirectangular (360)&#10;images, the quality and quantity of equirectangular depth data today is&#10;insufficient to represent the various scenes in the world. Therefore, 360 depth&#10;estimation studies, which relied solely on supervised learning, are destined to&#10;produce unsatisfactory results. Although self-supervised learning methods&#10;focusing on equirectangular images (EIs) are introduced, they often have&#10;incorrect or non-unique solutions, causing unstable performance. In this paper,&#10;we propose 360 monocular depth estimation methods which improve on the areas&#10;that limited previous studies. First, we introduce a self-supervised 360 depth&#10;learning method that only utilizes gravity-aligned videos, which has the&#10;potential to eliminate the needs for depth data during the training procedure.&#10;Second, we propose a joint learning scheme realized by combining supervised and&#10;self-supervised learning. The weakness of each learning is compensated, thus&#10;leading to more accurate depth estimation. Third, we propose a non-local fusion&#10;block, which can further retain the global information encoded by vision&#10;transformer when reconstructing the depths. With the proposed methods, we&#10;successfully apply the transformer to 360 depth estimations, to the best of our&#10;knowledge, which has not been tried before. On several benchmarks, our approach&#10;achieves significant improvements over previous works and establishes a state&#10;of the art.&#10;" />
-          <attvalue for="2" value="&#10;Recently, research interest in processing equirectangular (360$^\circ$) images has increased as virtual reality enters the limelight. &#10;Equirectangular images (EIs) have advantages over traditional rectilinear images (RIs) in that they enable a \ang{360} field of view. This benefit, however, complicates the acquisition of ground truth depths. Aside from the technical difficulties associated with \ang{360} depth scanners, one practical difficulty is that sensors would be visible from the \ang{360} RGB cameras, leading to partially obscured images \cite{Lowcost, omnidepth}. Moreover, to acquire diverse and realistic synthesized data, numerous things should be set exquisitely, which often requires professional designers and tools \cite{structure3d}. Due to such problems, the quality and quantity of equirectangular depth data today is insufficient to represent fully the various scenes in the world. &#10; Therefore, learning 360$^\circ$ depths in a supervised manner is destined to produce unsatisfactory results because the performance of supervised learning is highly dependent on the dataset.&#10;To overcome the lack of data, learning 360$^\circ$ depths in a self-supervised manner has been attempted. However, previous methods require either calibrated stereo EI pairs \cite{EBS,svsyn,360sdnet} or conversion to cubemap projection \cite{360self}, both of which have limitations with regard to further improvements. Moreover, self-supervised learning often delivers incorrect or non-unique solutions (e.g., light reflected object), which cause unstable performance.&#10;&#10;In this paper, we propose 360$^\circ$ monocular depth estimation methods which improve on the areas that limited previous studies. First, we propose a self-supervised method for the learning of depth that only utilizes gravity-aligned video sequences, which has the potential to eliminate the needs of depth data during the training procedure. Similar to prior work \cite{svsyn}, we utilize the relationships between consecutive scenes but improve it through consistency between depths.&#10; Second, we propose a joint learning scheme realized by combining supervised and self-supervised learning. &#10; Despite the limitations of each learning scheme, all previous works on the 360$^\circ$ depth estimation, to the best of our knowledge, have relied solely on either supervised or self-supervised learning. &#10; We show that the joint learning improves the unstable performance of self-supervised learning as well as the incorrect prediction of supervised learning caused by data scarcity, as visualized in Figure \ref{fig:intro}. Third, we propose a non-local fusion block which improves on the areas missed by vision transformers for dense prediction. Through non-local operations, global information encoded by a transformer can be further retained when reconstructing the depths. Under a challenging environment for a transformer (i.e., lack of a large-scale dataset), we were able to train the vision transformer successfully using the features learned from depth of RIs. To the best of our knowledge, this is the first work applying transformers successfully to 360$^\circ$ depth estimation. Our approaches achieve significant improvements over previous works on several benchmarks, thus establishing a state of the art. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1908.00222" label="1908.00222">
-        <attvalues>
-          <attvalue for="0" value="Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling" />
-          <attvalue for="1" value="  Recently, there has been growing interest in developing learning-based&#10;methods to detect and utilize salient semi-global or global structures, such as&#10;junctions, lines, planes, cuboids, smooth surfaces, and all types of&#10;symmetries, for 3D scene modeling and understanding. However, the ground truth&#10;annotations are often obtained via human labor, which is particularly&#10;challenging and inefficient for such tasks due to the large number of 3D&#10;structure instances (e.g., line segments) and other factors such as viewpoints&#10;and occlusions. In this paper, we present a new synthetic dataset,&#10;Structured3D, with the aim of providing large-scale photo-realistic images with&#10;rich 3D structure annotations for a wide spectrum of structured 3D modeling&#10;tasks. We take advantage of the availability of professional interior designs&#10;and automatically extract 3D structures from them. We generate high-quality&#10;images with an industry-leading rendering engine. We use our synthetic dataset&#10;in combination with real images to train deep networks for room layout&#10;estimation and demonstrate improved performance on benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02523" label="2106.02523">
-        <attvalues>
-          <attvalue for="0" value="Hallucination In Object Detection -- A Study In Visual Part Verification" />
-          <attvalue for="1" value="  We show that object detectors can hallucinate and detect missing objects;&#10;potentially even accurately localized at their expected, but non-existing,&#10;position. This is particularly problematic for applications that rely on visual&#10;part verification: detecting if an object part is present or absent. We show&#10;how popular object detectors hallucinate objects in a visual part verification&#10;task and introduce the first visual part verification dataset: DelftBikes,&#10;which has 10,000 bike photographs, with 22 densely annotated parts per image,&#10;where some parts may be missing. We explicitly annotated an extra object state&#10;label for each part to reflect if a part is missing or intact. We propose to&#10;evaluate visual part verification by relying on recall and compare popular&#10;object detectors on DelftBikes.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Automatically localizing and detecting an object in an image is one of the most important applications of computer vision. It is therefore paramount to be aware that deep object detectors can hallucinate non-existent objects, and they may even detect those missing objects at their expected location in the image, see~Fig.~\ref{fig:fig1}. Detecting non-existing objects is particularly detrimental to applications of automatic visual part verification or visual verification: determining the presence or absence of an object. Examples of visual verification include infrastructure verification in map making, missing instrument detection after surgery, part inspections in machine manufacturing etc. This paper shows how popular deep detectors hallucinate objects in a case study on a novel, specifically created visual object part verification dataset: DelftBikes.&#10;&#10;Visual verification as automatic visual inspection is typically used for manufacturing systems with applications such as checking pharmaceutical blister package~\cite{rosandich1997automated}, components on PCBs~\cite{garcia2009automated, koniar2014virtual}, solder joint~\cite{kim1999visual}, parts of railway tracks~\cite{resendiz2013automated}, rail bolts~\cite{marino2007real}, aeronautic components~\cite{ben2019automatic,san2017automatic}, objects~\cite{baerveldt2001vision}, and parts under motion~\cite{sim2006recognition}. In this paper, we do not focus on a particular application. Instead, we evaluate generic deep object detectors which potentially can be used in several visual inspection applications. &#10;&#10;There are important differences between visual verification and object detection. An object detector should not detect the same object multiple times. For visual verification, however, the goal is to determine if an object is present or absent, and thus having an existing object detected multiple times is not a problem, as long as the object is detected at least once. This makes recall more important than precision. Moreover, there are differences in how much costs a mistake has. The cost for an existing object that is not detected (false negative) is that a human needs to check the detection. The cost for a missing object that is falsely hallucinated as being present (false positive) is that this object is a wrongly judged as intact and thus may cause accidents in road infrastructure, or may cause incomplete objects to be sent to a customer. The costs for hallucinating missing objects is higher than missing an existing object. These aspects motivate us to not use the evaluation measure of object detection. Object detectors are typically evaluated with mean Average Precision (mAP) and because detections of non-existent objects at lower confidence levels does not significantly impact mAP, the problem of object hallucination has largely been ignored. Here, we propose to evaluate visual verification not with precision but with a cost-weighted variant of recall.&#10;&#10;Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}. Here, we focus on the visual verification task, its evaluation measure, a novel dataset, and a comparison of popular existing detectors. Investigating context is future work.&#10;&#10;Existing object detection datasets such as PASCAL VOC~\cite{pascal-voc-2012}, MS-COCO\cite{lin2014microsoft}, Imagenet-det~\cite{russakovsky2015imagenet}, and Open Image~\cite{DBLP:journals/corr/abs-1811-00982} have no annotated object parts. Pascal-Parts~\cite{chen2014detect} and GoCaRD~\cite{stappen2020gocard} include part labels, yet lack information if a part is missing and where, as is required to evaluate visual verification. Thus, we collected a novel visual verification dataset: DelftBikes where we explicitly annotate all part locations and part states as missing, intact, damaged, or occluded.&#10;&#10;We have the following contributions:&#10;&#10;1. We demonstrate hallucination in object detection for 3 popular object detectors.&#10;&#10;2. A dataset of 10k images with 22 densely annotated parts specifically collected and labeled for visual verification. &#10;&#10;3. An evaluation criteria for visual verification.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Visual Verification, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2008.05787" label="2008.05787">
-        <attvalues>
-          <attvalue for="0" value="Shift Equivariance in Object Detection" />
-          <attvalue for="1" value="  Robustness to small image translations is a highly desirable property for&#10;object detectors. However, recent works have shown that CNN-based classifiers&#10;are not shift invariant. It is unclear to what extent this could impact object&#10;detection, mainly because of the architectural differences between the two and&#10;the dimensionality of the prediction space of modern detectors. To assess shift&#10;equivariance of object detection models end-to-end, in this paper we propose an&#10;evaluation metric, built upon a greedy search of the lower and upper bounds of&#10;the mean average precision on a shifted image set. Our new metric shows that&#10;modern object detection architectures, no matter if one-stage or two-stage,&#10;anchor-based or anchor-free, are sensitive to even one pixel shift to the input&#10;images. Furthermore, we investigate several possible solutions to this problem,&#10;both taken from the literature and newly proposed, quantifying the&#10;effectiveness of each one with the suggested metric. Our results indicate that&#10;none of these methods can provide full shift equivariance. Measuring and&#10;analyzing the extent of shift variance of different models and the&#10;contributions of possible factors, is a first step towards being able to devise&#10;methods that mitigate or even leverage such variabilities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;{\bfseries Network robustness}&#10;Convolutional neural networks are reported to have brittle robustness to perturbations. Adversarial attacks~\cite{goodfellow2014explaining,kang2019testing}, which are intentionally designed noises to fool the network to make wrong predictions, are used to investigate network failures. Recent work has found that models are also brittle to less extreme perturbations that are not adversarially constructed, such as image quality distortions, i.e. blur, noise, contrast and compression~\cite{dodge2016understanding}. &#10;CNNs are reported to not be robust to scale variations~\cite{singh2018analysis} and highly sensitive to pose perturbations~\cite{Alcorn_2019_CVPR}. Networks fail to even small rotations and translations~\cite{azulay2019deep,engstrom2018rotation,zhang2019making}. &#10;Imperceptible variations and natural transformations across consecutive video frames would induce instability for networks as well~\cite{azulay2019deep,gu2019using}.&#10;&#10;There are also works on the robustness of object detectors. Image corruptions such as pixel noise, blur, varying weather conditions lead to significant performance drops in object detection models~\cite{michaelis2019benchmarking,von2019simulating}. Object detectors are not robust against scale variations~\cite{singh2018analysis}. Replacing image sub-regions with another sub-image is shown to have non-local impact on object detection, which may occasionally make other objects undetected or misclassified~\cite{rosenfeld2018elephant}.&#10;&#10;{\bfseries Network robustness evaluation} The robustness is often evaluated for classification networks in terms of classification label swap or mean absolute score changes. For example attack success rates (changes of top-1 prediction) across several distortion sizes are used to measure robustness with a broader threat model and diverse differentiable attacks \cite{kang2019testing}. Azulay et al. quantify both top-1 and mean average score changes of the classifier to small translations or re-scaling of the input images \cite{azulay2019deep}.&#10;\cite{hendrycks2018benchmarking} quantifies the classifiers mean and relative error rate on corruptions with different severity level to show the networks robustness against a variety of common corruptions.&#10;To measure robustness against geometric transformations, Manitest defines the invariance as the minimal geodesic distance between the identity transformation and a transformation in Lie group $\tau$ which is sufficient to change the predicted label of the classifier~\cite{fawzi2015manitest}.&#10;In our paper, we focus on evaluating shift equivariance of object detectors, considering both classification and localization performance. &#10;&#10;{\bfseries Approaches to improve shift invariance} One approach to improve the robustness of the model is to apply data-augmentation such as random cropping to make the network learn to be invariant to shifts~\cite{hendrycks2019augmix}. However,~\cite{azulay2019deep} has shown that data augmentation teaches the network to be invariant to translations but only for images that are visually similar to typical images seen during training, i.e. images that obey the photographer’s bias.&#10;Another way to mitigate shift variance is to apply anti-aliasing operations by blurring the representations before downsampling~\cite{zhang2019making}, as often done in signal processing. Although~\cite{zhang2019making} observed increased accuracy in ImageNet classification,~\cite{azulay2019deep} has shown that blurring would degrade model performance, especially in datasets that obey the photographer’s bias.&#10;GaussNet CNN architecture, which represents convolutional kernels with&#10;an orthogonal Gauss-Hermite basis whose basis coefficients are learned in convolution layers without a sub-sampling layer, leads to fully translation invariant representations that keeps the number of parameters in kernels across layers constant~\cite{sundaramoorthi2019translation}. &#10;In our paper, we also investigate several solutions to alleviate translation variance, focusing the analysis on object detection models.&#10; " />
-        </attvalues>
-      </node>
-      <node id="2003.07064" label="2003.07064">
-        <attvalues>
-          <attvalue for="0" value="On Translation Invariance in CNNs: Convolutional Layers can Exploit&#10;  Absolute Spatial Location" />
-          <attvalue for="1" value="  In this paper we challenge the common assumption that convolutional layers in&#10;modern CNNs are translation invariant. We show that CNNs can and will exploit&#10;the absolute spatial location by learning filters that respond exclusively to&#10;particular absolute locations by exploiting image boundary effects. Because&#10;modern CNNs filters have a huge receptive field, these boundary effects operate&#10;even far from the image boundary, allowing the network to exploit absolute&#10;spatial location all over the image. We give a simple solution to remove&#10;spatial location encoding which improves translation invariance and thus gives&#10;a stronger visual inductive bias which particularly benefits small data sets.&#10;We broadly demonstrate these benefits on several architectures and various&#10;applications such as image classification, patch matching, and two video&#10;classification datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.05471" label="1711.05471">
-        <attvalues>
-          <attvalue for="0" value="Exploring the Bounds of the Utility of Context for Object Detection" />
-          <attvalue for="1" value="  The recurring context in which objects appear holds valuable information that&#10;can be employed to predict their existence. This intuitive observation indeed&#10;led many researchers to endow appearance-based detectors with explicit&#10;reasoning about context. The underlying thesis suggests that stronger&#10;contextual relations would facilitate greater improvements in detection&#10;capacity. In practice, however, the observed improvement in many cases is&#10;modest at best, and often only marginal. In this work we seek to improve our&#10;understanding of this phenomenon, in part by pursuing an opposite approach.&#10;Instead of attempting to improve detection scores by employing context, we&#10;treat the utility of context as an optimization problem: to what extent can&#10;detection scores be improved by considering context or any other kind of&#10;additional information? With this approach we explore the bounds on improvement&#10;by using contextual relations between objects and provide a tool for&#10;identifying the most helpful ones. We show that simple co-occurrence relations&#10;can often provide large gains, while in other cases a significant improvement&#10;is simply impossible or impractical with either co-occurrence or more precise&#10;spatial relations. To better understand these results we then analyze the&#10;ability of context to handle different types of false detections, revealing&#10;that tested contextual information cannot ameliorate localization errors,&#10;severely limiting its gains. These and additional insights further our&#10;understanding on where and why utilization of context for object detection&#10;succeeds and fails.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.00119" label="1807.00119">
-        <attvalues>
-          <attvalue for="0" value="Structure Inference Net: Object Detection Using Scene-Level Context and&#10;  Instance-Level Relationships" />
-          <attvalue for="1" value="  Context is important for accurate visual recognition. In this work we propose&#10;an object detection algorithm that not only considers object visual appearance,&#10;but also makes use of two kinds of context including scene contextual&#10;information and object relationships within a single image. Therefore, object&#10;detection is regarded as both a cognition problem and a reasoning problem when&#10;leveraging these structured information. Specifically, this paper formulates&#10;object detection as a problem of graph structure inference, where given an&#10;image the objects are treated as nodes in a graph and relationships between the&#10;objects are modeled as edges in such graph. To this end, we present a so-called&#10;Structure Inference Network (SIN), a detector that incorporates into a typical&#10;detection framework (e.g. Faster R-CNN) with a graphical model which aims to&#10;infer object state. Comprehensive experiments on PASCAL VOC and MS COCO&#10;datasets indicate that scene context and object relationships truly improve the&#10;performance of object detection with more desirable and reasonable outputs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.02863" label="1708.02863">
-        <attvalues>
-          <attvalue for="0" value="CoupleNet: Coupling Global Structure with Local Parts for Object&#10;  Detection" />
-          <attvalue for="1" value="  The region-based Convolutional Neural Network (CNN) detectors such as Faster&#10;R-CNN or R-FCN have already shown promising results for object detection by&#10;combining the region proposal subnetwork and the classification subnetwork&#10;together. Although R-FCN has achieved higher detection speed while keeping the&#10;detection performance, the global structure information is ignored by the&#10;position-sensitive score maps. To fully explore the local and global&#10;properties, in this paper, we propose a novel fully convolutional network,&#10;named as CoupleNet, to couple the global structure with local parts for object&#10;detection. Specifically, the object proposals obtained by the Region Proposal&#10;Network (RPN) are fed into the the coupling module which consists of two&#10;branches. One branch adopts the position-sensitive RoI (PSRoI) pooling to&#10;capture the local part information of the object, while the other employs the&#10;RoI pooling to encode the global and context information. Next, we design&#10;different coupling strategies and normalization ways to make full use of the&#10;complementary advantages between the global and local branches. Extensive&#10;experiments demonstrate the effectiveness of our approach. We achieve&#10;state-of-the-art results on all three challenging datasets, i.e. a mAP of 82.7%&#10;on VOC07, 80.4% on VOC12, and 34.4% on COCO. Codes will be made publicly&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.03152" label="2001.03152">
-        <attvalues>
-          <attvalue for="0" value="Don't Judge an Object by Its Context: Learning to Overcome Contextual&#10;  Bias" />
-          <attvalue for="1" value="  Existing models often leverage co-occurrences between objects and their&#10;context to improve recognition accuracy. However, strongly relying on context&#10;risks a model's generalizability, especially when typical co-occurrence&#10;patterns are absent. This work focuses on addressing such contextual biases to&#10;improve the robustness of the learnt feature representations. Our goal is to&#10;accurately recognize a category in the absence of its context, without&#10;compromising on performance when it co-occurs with context. Our key idea is to&#10;decorrelate feature representations of a category from its co-occurring&#10;context. We achieve this by learning a feature subspace that explicitly&#10;represents categories occurring in the absence of context along side a joint&#10;feature subspace that represents both categories and context. Our very simple&#10;yet effective method is extensible to two multi-label tasks -- object and&#10;attribute classification. On 4 challenging datasets, we demonstrate the&#10;effectiveness of our method in reducing contextual bias.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.00982" label="1811.00982">
-        <attvalues>
-          <attvalue for="0" value="The Open Images Dataset V4: Unified image classification, object&#10;  detection, and visual relationship detection at scale" />
-          <attvalue for="1" value="  We present Open Images V4, a dataset of 9.2M images with unified annotations&#10;for image classification, object detection and visual relationship detection.&#10;The images have a Creative Commons Attribution license that allows to share and&#10;adapt the material, and they have been collected from Flickr without a&#10;predefined list of class names or tags, leading to natural class statistics and&#10;avoiding an initial design bias. Open Images V4 offers large scale across&#10;several dimensions: 30.1M image-level labels for 19.8k concepts, 15.4M bounding&#10;boxes for 600 object classes, and 375k visual relationship annotations&#10;involving 57 classes. For object detection in particular, we provide 15x more&#10;bounding boxes than the next largest datasets (15.4M boxes on 1.9M images). The&#10;images often show complex scenes with several objects (8 annotated objects per&#10;image on average). We annotated visual relationships between them, which&#10;support visual relationship detection, an emerging task that requires&#10;structured reasoning. We provide in-depth comprehensive statistics about the&#10;dataset, we validate the quality of the annotations, we study how the&#10;performance of several modern models evolves with increasing amounts of&#10;training data, and we demonstrate two applications made possible by having&#10;unified annotations of multiple types coexisting in the same images. We hope&#10;that the scale, quality, and variety of Open Images V4 will foster further&#10;research and innovation even beyond the areas of image classification, object&#10;detection, and visual relationship detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.2031" label="1406.2031">
-        <attvalues>
-          <attvalue for="0" value="Detect What You Can: Detecting and Representing Objects using Holistic&#10;  Models and Body Parts" />
-          <attvalue for="1" value="  Detecting objects becomes difficult when we need to deal with large shape&#10;deformation, occlusion and low resolution. We propose a novel approach to i)&#10;handle large deformations and partial occlusions in animals (as examples of&#10;highly deformable objects), ii) describe them in terms of body parts, and iii)&#10;detect them when their body parts are hard to detect (e.g., animals depicted at&#10;low resolution). We represent the holistic object and body parts separately and&#10;use a fully connected model to arrange templates for the holistic object and&#10;body parts. Our model automatically decouples the holistic object or body parts&#10;from the model when they are hard to detect. This enables us to represent a&#10;large number of holistic object and body part combinations to better deal with&#10;different &quot;detectability&quot; patterns caused by deformations, occlusion and/or low&#10;resolution.&#10;  We apply our method to the six animal categories in the PASCAL VOC dataset&#10;and show that our method significantly improves state-of-the-art (by 4.1% AP)&#10;and provides a richer representation for objects. During training we use&#10;annotations for body parts (e.g., head, torso, etc), making use of a new&#10;dataset of fully annotated object parts for PASCAL VOC 2010, which provides a&#10;mask for each part.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.13520" label="2407.13520">
-        <attvalues>
-          <attvalue for="0" value="EaDeblur-GS: Event assisted 3D Deblur Reconstruction with Gaussian&#10;  Splatting" />
-          <attvalue for="1" value="  3D deblurring reconstruction techniques have recently seen significant&#10;advancements with the development of Neural Radiance Fields (NeRF) and 3D&#10;Gaussian Splatting (3DGS). Although these techniques can recover relatively&#10;clear 3D reconstructions from blurry image inputs, they still face limitations&#10;in handling severe blurring and complex camera motion. To address these issues,&#10;we propose Event-assisted 3D Deblur Reconstruction with Gaussian Splatting&#10;(EaDeblur-GS), which integrates event camera data to enhance the robustness of&#10;3DGS against motion blur. By employing an Adaptive Deviation Estimator (ADE)&#10;network to estimate Gaussian center deviations and using novel loss functions,&#10;EaDeblur-GS achieves sharp 3D reconstructions in real-time, demonstrating&#10;performance comparable to state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;Reconstructing 3D scenes and objects from images has long been a research hotspot in computer vision and computer graphics. &#10;The advent of Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf} has brought revolutionary advancements in photo-realistic novel view synthesis.&#10;Building on this progress, the recent introduction of 3D Gaussian Splatting (3DGS) \cite{kerbl20233d} has further enhanced 3D scene representation using Gaussian ellipsoids, achieving high-quality 2D rendering and faster training and rendering speeds.&#10;However, in real-world applications, factors like camera shake and shutter speed often lead to image blurriness and inaccurate camera poses estimation, challenging the clear neural volumetric representation. &#10;&#10;Several methods address blurriness in NeRFs and 3DGS.&#10;NeRF's deblurring technology has undergone relatively early development, with Deblur-NeRF \cite{ma2022deblur} being the first framework to tackle this issue, which utilized an analysis-by-synthesis approach to recover sharp NeRFs from blurry inputs.&#10;MP-NeRF\cite{wang2024mp} further enhances this by introducing a multi-branch fusion network and prior-based learnable weights to handle extremely blurry or unevenly blurred images.&#10;But the NeRF-based methods always consume extensive training time and rendering time.&#10;Hence, some methods based on 3DGS has been developed because of its advantages in rendering and training speed.&#10;For instance, Wenbo Chen et al.~\cite{Chen_deblurgs2024} proposed Deblur-GS, which models the problem of motion blur as a joint optimization involving camera trajectory and time sampling.&#10;B. Lee etal.~\cite{lee2024deblurring} assign the corrections on the rotation and scaling matrix of 3D gaussians by using a small MLP, enhancing the clarity of scenes reconstructed from blurred images.&#10;Nonetheless, these methods can only achieve clear 3D reconstruction results with mildly blurred input images.&#10;Consequently, additional data sources like event cameras have been introduced into 3D deblurring reconstruction.&#10;Event cameras, a bio-inspired sensor, offers high temporal resolution and has advantages in motion deblur.&#10;For NeRFs, EventNeRF \cite{rudnev2023eventnerf} used event integration and color simulation for colored 3D representations, while Qi et al. \cite{qi2023e2nerf} combined blurred images with event streams using innovative loss functions and the Event Double Integral (EDI) approach.&#10;For 3DGS, Yu proposed EvaGaussian\cite{yu2024evagaussians}, optimizing both scene reconstruction and camera trajectory estimation to achieve unparalleled detail and fidelity in real-time 3D scene synthesis.&#10;&#10;Despite achieving excellent performance in 3D deblurring reconstruction, the aforementioned methods still have limitations. &#10;For instance, RGB single-modality deblurring 3DGS and NeRF are often effective only in mildly blurred or simple camera motion scenarios.&#10;When input images are severely blurred, these methods usually fail to reconstruct 3D objects and can only produce relatively clear 2D renderings from certain angles.&#10;On the other hand, techniques using event cameras for NeRF 3D reconstruction are limited by NeRF's training and rendering speed. Additionally, methods that incorporate event data into 3DGS deblurring reconstruction face challenges, such as inaccurate camera motion trajectory estimation.&#10;&#10;To overcome these limitations, we propose a novel integration of event streams with 3DGS, namely Event-assisted 3D Deblur Reconstruction with Gaussian Splatting(EaDeblur-GS), aiming to guide the learning of better 3D Gaussian representations and tackle issues arising from blurry inputs. &#10;&#10;Our contributions are summarized as follows:&#10;\begin{itemize}&#10;\item We propose Event-assisted 3D Deblur Gaussian Splatting (EaDeblur-GS), which incorporates blurry RGB images and event streams into Gaussian Splatting to recover sharp 3D representations.&#10;\item We introduce a novel Adaptive Deviation Estimator (ADE) network to simulate the shaking motion during exposure accurately by estimating the deviations of Gaussians.&#10;\item We comprehensively evaluate the proposed method and compare it with several baselines, demonstrating that our EaDeblur-GS achieves advanced performance while enabling real-time sharp image rendering.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Image Deblurring, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2111.14292" label="2111.14292">
-        <attvalues>
-          <attvalue for="0" value="Deblur-NeRF: Neural Radiance Fields from Blurry Images" />
-          <attvalue for="1" value="  Neural Radiance Field (NeRF) has gained considerable attention recently for&#10;3D scene reconstruction and novel view synthesis due to its remarkable&#10;synthesis quality. However, image blurriness caused by defocus or motion, which&#10;often occurs when capturing scenes in the wild, significantly degrades its&#10;reconstruction quality. To address this problem, We propose Deblur-NeRF, the&#10;first method that can recover a sharp NeRF from blurry input. We adopt an&#10;analysis-by-synthesis approach that reconstructs blurry views by simulating the&#10;blurring process, thus making NeRF robust to blurry inputs. The core of this&#10;simulation is a novel Deformable Sparse Kernel (DSK) module that models&#10;spatially-varying blur kernels by deforming a canonical sparse kernel at each&#10;spatial location. The ray origin of each kernel point is jointly optimized,&#10;inspired by the physical blurring process. This module is parameterized as an&#10;MLP that has the ability to be generalized to various blur types. Jointly&#10;optimizing the NeRF and the DSK module allows us to restore a sharp NeRF. We&#10;demonstrate that our method can be used on both camera motion blur and defocus&#10;blur: the two most common types of blur in real scenes. Evaluation results on&#10;both synthetic and real-world data show that our method outperforms several&#10;baselines. The synthetic and real datasets along with the source code is&#10;publicly available at https://limacv.github.io/deblurnerf/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.00834" label="2401.00834">
-        <attvalues>
-          <attvalue for="0" value="Deblurring 3D Gaussian Splatting" />
-          <attvalue for="1" value="  Recent studies in Radiance Fields have paved the robust way for novel view&#10;synthesis with their photorealistic rendering quality. Nevertheless, they&#10;usually employ neural networks and volumetric rendering, which are costly to&#10;train and impede their broad use in various real-time applications due to the&#10;lengthy rendering time. Lately 3D Gaussians splatting-based approach has been&#10;proposed to model the 3D scene, and it achieves remarkable visual quality while&#10;rendering the images in real-time. However, it suffers from severe degradation&#10;in the rendering quality if the training images are blurry. Blurriness commonly&#10;occurs due to the lens defocusing, object motion, and camera shake, and it&#10;inevitably intervenes in clean image acquisition. Several previous studies have&#10;attempted to render clean and sharp images from blurry input images using&#10;neural fields. The majority of those works, however, are designed only for&#10;volumetric rendering-based neural radiance fields and are not straightforwardly&#10;applicable to rasterization-based 3D Gaussian splatting methods. Thus, we&#10;propose a novel real-time deblurring framework, Deblurring 3D Gaussian&#10;Splatting, using a small Multi-Layer Perceptron (MLP) that manipulates the&#10;covariance of each 3D Gaussian to model the scene blurriness. While Deblurring&#10;3D Gaussian Splatting can still enjoy real-time rendering, it can reconstruct&#10;fine and sharp details from blurry images. A variety of experiments have been&#10;conducted on the benchmark, and the results have revealed the effectiveness of&#10;our approach for deblurring. Qualitative results are available at&#10;https://benhenryl.github.io/Deblurring-3D-Gaussian-Splatting/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2206.11896" label="2206.11896">
-        <attvalues>
-          <attvalue for="0" value="EventNeRF: Neural Radiance Fields from a Single Colour Event Camera" />
-          <attvalue for="1" value="  Asynchronously operating event cameras find many applications due to their&#10;high dynamic range, vanishingly low motion blur, low latency and low data&#10;bandwidth. The field saw remarkable progress during the last few years, and&#10;existing event-based 3D reconstruction approaches recover sparse point clouds&#10;of the scene. However, such sparsity is a limiting factor in many cases,&#10;especially in computer vision and graphics, that has not been addressed&#10;satisfactorily so far. Accordingly, this paper proposes the first approach for&#10;3D-consistent, dense and photorealistic novel view synthesis using just a&#10;single colour event stream as input. At its core is a neural radiance field&#10;trained entirely in a self-supervised manner from events while preserving the&#10;original resolution of the colour event channels. Next, our ray sampling&#10;strategy is tailored to events and allows for data-efficient training. At test,&#10;our method produces results in the RGB space at unprecedented quality. We&#10;evaluate our method qualitatively and numerically on several challenging&#10;synthetic and real scenes and show that it produces significantly denser and&#10;more visually appealing renderings than the existing methods. We also&#10;demonstrate robustness in challenging scenarios with fast motion and under low&#10;lighting conditions. We release the newly recorded dataset and our source code&#10;to facilitate the research field, see https://4dqv.mpi-inf.mpg.de/EventNeRF.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Novel View Synthesis from Event Streams.&#10;The properties of event cameras (i.e., no motion blur, high dynamic range, low latency and ultra-low power consumption) motivated their usage in computer vision&#10;~\cite{Tulyakov21CVPR,Wang20Restoration,zhang2022unifying,Gehrig22,Gehrig19ijcv}.&#10;While many of the developed techniques address problems from a 2D-vision perspective~\cite{Gallego22}, event cameras have also been increasingly used for 3D-vision problems.&#10;This includes methods designed for dedicated objects such as the human body~\cite{EventCap2020,zou2021eventhpe} and the human hands~\cite{rudnev2021eventhands,Nehvi2021}, as well as methods for general scenes~\cite{Baudron20,Wang2022EvAC3D,Cui22,Uddin22,VidalSLAM,Kim16,Rebecq18}.&#10;EventCap~\cite{EventCap2020} is a method for the 3D reconstruction of a human body in motion.&#10;Here, the tracked mesh can be used to render novel views of the performer.&#10;However, the method requires grayscale frames and a 3D body scan as inputs. &#10;&#10;Thus, novel views of the actor do not contain new details observed in the input event streams (instead, the initial 3D scan is deforming).&#10;The analysis-by-synthesis method of Nevhi~et al.~\cite{Nehvi2021} operates on events only and supports arbitrary non-rigid motions of the human hands.&#10;Rudnev~et al.~\cite{rudnev2021eventhands} regresses sparse 3D hand keypoints and drives a MANO~\cite{MANO:SIGGRAPHASIA:2017} hand model from that.&#10;Even though the final results of both Nevhi~et al.~\cite{Nehvi2021} or Rudnev~et al.~\cite{Nehvi2021} can be in form of a mesh, none of them is designed for the task of photorealistic novel view synthesis.&#10;&#10;Methods for the 3D reconstruction of general scenes have recently seen remarkable progress.&#10;They can be classified into ones that aim to estimate the underlying geometry only~\cite{Baudron20,Wang2022EvAC3D,Cui22,Uddin22} and others that also estimate the camera trajectory~\cite{VidalSLAM,Kim16,Rebecq18,Zhou21tro,Zuo22}. For the former, several methods have been proposed including ones that use a single event camera~\cite{Baudron20,Wang2022EvAC3D}, two event cameras~\cite{Uddin22} and other input data modalities such as LiDAR~\cite{Cui22}.&#10;In the latter, methods follow either a SLAM~\cite{Zhou18eccv,Rebecq18,VidalSLAM,Kim16} or an odometry based formulation~\cite{Zhou21tro,Zuo22,Hadviger21,Kueng16,Rebecq17EVO} and target large-scale scenes. &#10;In addition to reconstructing the 3D scene to some extent, these methods also estimate the position of a moving camera scanning the scene.&#10;To this end, some methods use as input a single event camera~\cite{Rebecq18,Kim16,Rebecq17EVO}, a pair of event cameras~\cite{Zhou18eccv,Zhou21tro,Hadviger21} and some use a mix of input data modalities such as depth~\cite{Zuo22}, intensity images~\cite{Kueng16} and IMUs~\cite{VidalSLAM}.&#10;While SLAM aims for denser 3D reconstruction of the underlying scene than odometry methods, both approach classes produce only sparse reconstructions (usually edges and corners).&#10;This is far from allowing 3D-consistent dense and photorealistic rendering of the examined scene.&#10;In stark contrast to these works, we learn---for the first time---an implicit 3D scene representation from event streams that enables dense photorealistic novel view synthesis in the RGB space.&#10;&#10;3D Scene Representation Learning&#10;is a long-standing problem in computer vision and computer graphics~\cite{Gortler96Lumigraph,Debevec96Modeling,Mildenhall19,sitzmann2019srns,Lombardi:2019,mildenhall2020nerf}.&#10;Some of the existing methods produce meshes~\cite{Thies18Face2Face,DeepBlending2018,kair2022sft} and multi-plane images (MPIs)~\cite{Mildenhall19,zhou2018stereo,li2020crowdsampling}.&#10;Such representations can be learnt from 2D images~\cite{Thies19DNR,zhou2018stereo,li2020crowdsampling},&#10;but they suffer from several limitations, including the inability to capture fine geometrical details (meshes) and being mostly bounded to 2.5D novel view synthesis (MPIs).&#10;In contrast, the recently introduced implicit coordinate-based representations learn to map the 3D spatial coordinates of a static scene to a continuous function of the local scene properties~\cite{Mescheder2019,Park2019,sitzmann2019srns,mildenhall2020nerf,Lombardi:2019}.&#10;Here, Neural Radiance Fields, or NeRF~\cite{mildenhall2020nerf}, models a scene through MLP as a continuous function of the scene radiance and volume density learnt from a set of 2D RGB images.&#10;At test time, NeRF accepts arbitrary 3D camera pose and viewing direction to render novel views.&#10;Due to its simplicity and high accuracy of the generated novel views,&#10;multiple follow-up works adopted NeRF for many applications such as reenactment~\cite{Gafni_2021_CVPR,liu2021neural,zheng2022imavatar}, scene appearance and light editing~\cite{martinbrualla2020nerfw,boss2021nerd,nerv2021}, 3D shape modelling~\cite{hong2021headnerf,corona2022lisa,chanmonteiro2020piGAN,Schwarz20} and others~\cite{yang2021objectnerf,Niemeyer2020GIRAFFE,Tewari2022NeuRendSTAR}.&#10;All these methods, however, assume input data captured by a traditional RGB camera.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2405.20224" label="2405.20224">
-        <attvalues>
-          <attvalue for="0" value="EvaGaussians: Event Stream Assisted Gaussian Splatting from Blurry&#10;  Images" />
-          <attvalue for="1" value="  3D Gaussian Splatting (3D-GS) has demonstrated exceptional capabilities in 3D&#10;scene reconstruction and novel view synthesis. However, its training heavily&#10;depends on high-quality, sharp images and accurate camera poses. Fulfilling&#10;these requirements can be challenging in non-ideal real-world scenarios, where&#10;motion-blurred images are commonly encountered in high-speed moving cameras or&#10;low-light environments that require long exposure times. To address these&#10;challenges, we introduce Event Stream Assisted Gaussian Splatting&#10;(EvaGaussians), a novel approach that integrates event streams captured by an&#10;event camera to assist in reconstructing high-quality 3D-GS from blurry images.&#10;Capitalizing on the high temporal resolution and dynamic range offered by the&#10;event camera, we leverage the event streams to explicitly model the formation&#10;process of motion-blurred images and guide the deblurring reconstruction of&#10;3D-GS. By jointly optimizing the 3D-GS parameters and recovering camera motion&#10;trajectories during the exposure time, our method can robustly facilitate the&#10;acquisition of high-fidelity novel views with intricate texture details. We&#10;comprehensively evaluated our method and compared it with previous&#10;state-of-the-art deblurring rendering methods. Both qualitative and&#10;quantitative comparisons demonstrate that our method surpasses existing&#10;techniques in restoring fine details from blurry images and producing&#10;high-fidelity novel views.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.09836" label="2401.09836">
-        <attvalues>
-          <attvalue for="0" value="Exploring Latent Cross-Channel Embedding for Accurate 3D Human Pose&#10;  Reconstruction in a Diffusion Framework" />
-          <attvalue for="1" value="  Monocular 3D human pose estimation poses significant challenges due to the&#10;inherent depth ambiguities that arise during the reprojection process from 2D&#10;to 3D. Conventional approaches that rely on estimating an over-fit projection&#10;matrix struggle to effectively address these challenges and often result in&#10;noisy outputs. Recent advancements in diffusion models have shown promise in&#10;incorporating structural priors to address reprojection ambiguities. However,&#10;there is still ample room for improvement as these methods often overlook the&#10;exploration of correlation between the 2D and 3D joint-level features. In this&#10;study, we propose a novel cross-channel embedding framework that aims to fully&#10;explore the correlation between joint-level features of 3D coordinates and&#10;their 2D projections. In addition, we introduce a context guidance mechanism to&#10;facilitate the propagation of joint graph attention across latent channels&#10;during the iterative diffusion process. To evaluate the effectiveness of our&#10;proposed method, we conduct experiments on two benchmark datasets, namely&#10;Human3.6M and MPI-INF-3DHP. Our results demonstrate a significant improvement&#10;in terms of reconstruction accuracy compared to state-of-the-art methods. The&#10;code for our method will be made available online for further reference.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The 3D Human Pose Estimation (HPE) technique is essential, as it encompasses wide-ranging applications in security surveillance, film making as well as emerging trends such as dance generation~\cite{au2022choreograph} and pedestrian behaviour prediction~\cite{papaioannidis2023fast}. Its primary objective is to predict the 3D locations of human body parts based on 2D observations and then reconstruct human motion. &#10;Despite the considerable progress on multi-view 3D HPE~\cite{jiang2022dual,zhou2022quickpose,pavlakos2019expressive,gong2022posetriplet}, monocular pose estimation remains challenging because of the depth ambiguity caused by uncertain 2D-to-3D projection and occlusion of body parts. &#10;&#10;To deal with the depth ambiguity, researchers mainly rely on a two-stage pipeline, i.e., utilize a 2D pose detector to obtain 2D poses, and then regress a 2D-to-3D lifting network which learns one specific view's projection implicitly~\cite{xu2021graph,zhao2022graformer,pavllo20193d,zhang2022mixste,du2023flowpose}. &#10;Specifically, Xu et al.~\cite{xu2021graph} propose to utilize Graph Convolutional Network (GCN) as both the encoder and decoder within the stacked hourglass network architecture for the purpose of exploring the humanoid kinematic structure. &#10;Following \cite{xu2021graph}, Zhao et al.~\cite{zhao2022graformer} propose to involve the attention mechanism into the vanilla GCN blocks. Therefore the attention-involved GCN block is able to model more implicit relations. On the other hand, many works focus on incorporating 2D motion sequences to extract spatial-temporal information from them. Pavllo et al~\cite{pavllo20193d} predict the centre frame's 3D pose from a 2D motion sequence by a dilate Convolutional Neural Network (CNN). Zhang et al.~\cite{zhang2022mixste} propose a Transformer-based model for sequence-to-sequence lifting which efficiently models skeletal structure and kinematic correlation. Du et al.~\cite{du2023flowpose} introduce Normalizing Flow to mine the feasible motion distribution across spatial and temporal domains. Despite multi-frame estimation frameworks~\cite{pavllo20193d,zhang2022mixste,du2023flowpose} leveraging temporal information to effectively address ambiguity, those frameworks rely on future frames that hinder real-time performance. In contrast, this paper focuses on a more challenging task, i.e., monocular single-frame 3D HPE.&#10;&#10;Recently, Denoising Diffusion Probabilistic Models~\cite{song2021denoising,ho2020denoising,gong2023diffpose,choi2022diffupose} (DDPMs, a.k.a., diffusion models) have garnered the interest of researchers owing to their high-quality noise-to-image generation. Some \cite{gong2023diffpose,choi2022diffupose} propose to regard the depth ambiguity as the noise and utilize the diffusion model to enhance 2D-to-3D lifting results. In particular, Choi et al.~\cite{choi2022diffupose} suggest replacing the standard DDPM's U-Net structure with a GCN. They input noisy 3D joints, initialized with the Gaussian distribution and condition the 3D noise on 2D observation. Similarly, Gong et al.~\cite{gong2023diffpose} also adopt a GCN structure~\cite{zhao2022graformer} and accelerate the diffusion process using DDIM~\cite{song2021denoising}. They assume the noisy 3D input is sampled from an indeterminate pose distribution.&#10;Inspired by ~\cite{gong2023diffpose, choi2022diffupose}, we propose a monocular single-frame 3D HPE framework based on diffusion modelling. Unlike the previous works, we model the correlation between both 2D and 3D uncertainty, while also incorporating the 2D-to-3D context attention. &#10;To be more specific, the contributions of this paper are three-fold: 1) We develop a novel monocular single-frame 3D HPE framework driven by the diffusion model that works as a post-processing to refine the lifting results. 2) Inside the framework, we first propose the Cross-Channel Embedding (CCE) module that explores the correlation between both 2D and 3D distributions. We further propose the Context Attention Guidance (CAG) module to encourage efficient cross-joint attention propagation. &#10;3) The proposed framework achieves state-of-the-art performance on two widely-used benchmarks~\cite{ionescu2013human3,mehta2017monocular} demonstrating our superiority.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Diffusion Models, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="2207.07386" label="2207.07386">
-        <attvalues>
-          <attvalue for="0" value="ChoreoGraph: Music-conditioned Automatic Dance Choreography over a Style&#10;  and Tempo Consistent Dynamic Graph" />
-          <attvalue for="1" value="  To generate dance that temporally and aesthetically matches the music is a&#10;challenging problem, as the following factors need to be considered. First, the&#10;aesthetic styles and messages conveyed by the motion and music should be&#10;consistent. Second, the beats of the generated motion should be locally aligned&#10;to the musical features. And finally, basic choreomusical rules should be&#10;observed, and the motion generated should be diverse. To address these&#10;challenges, we propose ChoreoGraph, which choreographs high-quality dance&#10;motion for a given piece of music over a Dynamic Graph. A data-driven learning&#10;strategy is proposed to evaluate the aesthetic style and rhythmic connections&#10;between music and motion in a progressively learned cross-modality embedding&#10;space. The motion sequences will be beats-aligned based on the music segments&#10;and then incorporated as nodes of a Dynamic Motion Graph. Compatibility factors&#10;such as the style and tempo consistency, motion context connection, action&#10;completeness, and transition smoothness are comprehensively evaluated to&#10;determine the node transition in the graph. We demonstrate that our&#10;repertoire-based framework can generate motions with aesthetic consistency and&#10;robustly extensible in diversity. Both quantitative and qualitative experiment&#10;results show that our proposed model outperforms other baseline models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.07381" label="2207.07381">
-        <attvalues>
-          <attvalue for="0" value="A Dual-Masked Auto-Encoder for Robust Motion Capture with&#10;  Spatial-Temporal Skeletal Token Completion" />
-          <attvalue for="1" value="  Multi-person motion capture can be challenging due to ambiguities caused by&#10;severe occlusion, fast body movement, and complex interactions. Existing&#10;frameworks build on 2D pose estimations and triangulate to 3D coordinates via&#10;reasoning the appearance, trajectory, and geometric consistencies among&#10;multi-camera observations. However, 2D joint detection is usually incomplete&#10;and with wrong identity assignments due to limited observation angle, which&#10;leads to noisy 3D triangulation results. To overcome this issue, we propose to&#10;explore the short-range autoregressive characteristics of skeletal motion using&#10;transformer. First, we propose an adaptive, identity-aware triangulation module&#10;to reconstruct 3D joints and identify the missing joints for each identity. To&#10;generate complete 3D skeletal motion, we then propose a Dual-Masked&#10;Auto-Encoder (D-MAE) which encodes the joint status with both&#10;skeletal-structural and temporal position encoding for trajectory completion.&#10;D-MAE's flexible masking and encoding mechanism enable arbitrary skeleton&#10;definitions to be conveniently deployed under the same framework. In order to&#10;demonstrate the proposed model's capability in dealing with severe data loss&#10;scenarios, we contribute a high-accuracy and challenging motion capture dataset&#10;of multi-person interactions with severe occlusion. Evaluations on both&#10;benchmark and our new dataset demonstrate the efficiency of our proposed model,&#10;as well as its advantage against the other state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.15625" label="2203.15625">
-        <attvalues>
-          <attvalue for="0" value="PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and&#10;  Hallucination under Self-supervision" />
-          <attvalue for="1" value="  Existing self-supervised 3D human pose estimation schemes have largely relied&#10;on weak supervisions like consistency loss to guide the learning, which,&#10;inevitably, leads to inferior results in real-world scenarios with unseen&#10;poses. In this paper, we propose a novel self-supervised approach that allows&#10;us to explicitly generate 2D-3D pose pairs for augmenting supervision, through&#10;a self-enhancing dual-loop learning framework. This is made possible via&#10;introducing a reinforcement-learning-based imitator, which is learned jointly&#10;with a pose estimator alongside a pose hallucinator; the three components form&#10;two loops during the training process, complementing and strengthening one&#10;another. Specifically, the pose estimator transforms an input 2D pose sequence&#10;to a low-fidelity 3D output, which is then enhanced by the imitator that&#10;enforces physical constraints. The refined 3D poses are subsequently fed to the&#10;hallucinator for producing even more diverse data, which are, in turn,&#10;strengthened by the imitator and further utilized to train the pose estimator.&#10;Such a co-evolution scheme, in practice, enables training a pose estimator on&#10;self-generated motion data without relying on any given 3D data. Extensive&#10;experiments across various benchmarks demonstrate that our approach yields&#10;encouraging results significantly outperforming the state of the art and, in&#10;some cases, even on par with results of fully-supervised methods. Notably, it&#10;achieves 89.1% 3D PCK on MPI-INF-3DHP under self-supervised cross-dataset&#10;evaluation setup, improving upon the previous best self-supervised methods by&#10;8.6%. Code can be found at: https://github.com/Garfield-kh/PoseTriplet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;3D pose estimation&#10;3D pose estimation have been wildly explored under fully supervised, semi-supervised, self-supervised.&#10;Various approaches have been explored under fully supervised setting~\cite{martinez2017simple,mehta2017vnect,sun2018integral, nie2019spm,Yang2020Distill,pavllo2019videopose3d,kocabas2020vibe,Yang2021Pose,zhang2021bmp,wang2021mvp,WangJueICCV19}.&#10;Through offering impressive results, those approaches highly rely on accurate motion capture data, which are hard to collect.&#10;To address high cost of data collection, &#10;semi-supervised methods~\cite{zhou2017towards, li2019boosting, mitra2020multiview} are proposed to utilize the information from unlabeled data. &#10;Besides semi-supervised approach, augmentation based methods~\cite{Li_2020_CVPR, gong2021poseaug} are proposed to enlarge the data amount through evolution strategy~\cite{Li_2020_CVPR} or learnable approach~\cite{gong2021poseaug}.&#10;&#10;Different from the above schemes, self-supervised methods, with multi-view data, explore the intrinsic supervision for model training, without requiring ground truth 3D pose~\cite{kocabas2019epipolar, wandt2021canonpose,iqbal2020weakly }. &#10;For instance, Kocabas~et al.~\cite{kocabas2019epipolar} utilize the epipolar geometry to generate pseudo label, ~\cite{wandt2021canonpose, iqbal2020weakly} utilize the 3D pose consistency across different views. Though being effective, those approaches require synchronized multiple cameras, which are not usual in real scenarios.&#10;Other methods~\cite{drover2018can, chen2019unsupervised, yu2021towards, hu2021unsupervised} explore the more challenge single view setting.&#10;For example, Drover~et al.~\cite{drover2018can} utilize the prior that a random projection of a plausible 3D pose estimation will be plausible in 2D pose distribution through adversary training.&#10;Chen~et al.~\cite{chen2019unsupervised} improves this idea by adding cycle consistency. Yu~et al.~\cite{yu2021towards} further introduces the scale steps for 2D poses to resolve the ambiguity issue.&#10;Zhang~et al.~\cite{zhang2020inference} applies self-supervised learning on test data to adapt model to new scenarios.&#10;&#10;Our method belongs to the self-supervised approaches {under single view setting}.&#10;Different from previous self-supervised approaches which implement weak supervision signal through consistency~\cite{chen2019unsupervised} or adversary~\cite{drover2018can, yu2021towards}, our method directly uses the strong supervision signal from self-generated data, results in more accurate and stable model performance.&#10;The pseudo label strategy~\cite{li2019boosting} under semi-supervised category is close to our approach. However, our approach does not require ground truth data for model pretraining, and our method introduces physical plausibility refinement and diversity enhancement to achieve better performance, which are absent in~\cite{li2019boosting}. &#10;&#10;Physics-based pose estimation&#10;The above methods are all Kinematics based. Though providing impressive results, they do not consider physical constrains, thus suffering physical implausible artifacts (\eg, foot skating and ground penetration).&#10;To ensure physical plausibility, recent works explore physical constraints.&#10;Rempe~et al.~\cite{rempe2020contact} introduces physical law to the foot contact and human dynamic,&#10;while its iterative optimization is high time costly (\eg, 30 minutes for 2s clip).&#10;Later, ~\cite{shimada2020physcap, shimada2021neural, xie2021physics} propose differentiable physical constrain to reduce the time cost.&#10;But they only consider foot contact, making them less effective in scenarios with other important contact (\eg, lay down, sitting with chair). &#10;&#10;Different from optimization based approaches, physics simulation based methods use physics simulators to provide realistic physical constrains.&#10;DeepMimic~\cite{peng2018deepmimic} tries to imitate various motion from reference mocap data in physics engine via reinforcement learning.&#10;SFV~\cite{peng2018sfv} proposes to refine the low fidelity motion data from video-based pose estimation through imitation learning.&#10;However, their adopted imitation learning requires days of training for just one clip.&#10;Later, SimPoe~\cite{yuan2021simpoe} addresses this issue by introducing RFC~\cite{yuan2020residual} to effectively reduce the time consumption by training one policy for all motion clips.&#10;Our method is built upon SimPoe~\cite{yuan2021simpoe} for better generalization and low time cost. However, different from those methods only using physical constraints for post processing, our method propose to involve it in the learning loop. As such, no mocap data for pose estimation training and imitation learning is required.&#10;&#10;Motion synthesis&#10;Motion synthesis includes non-learning based and learning based approaches.&#10;In non-learning based approaches,&#10;the motion graph method~\cite{kovar2008motion} first builds transition edges between different motion points based on their similarity, and then generates new motion data through traversing the graphs. Motion matching~\cite{buttner2015motion} searches proper future frames in motion data based on motion states in real time.&#10;In learning based approaches,&#10;motion perdition based methods~\cite{martinez2017human, zhang2020we, li2018convolutional, gui2018adversarial, barsoum2018hp, yuan2020dlow, pavllo2018quaternet} aim to predict future poses conditioned on previous poses. Action generation~\cite{wang2020learning, butepage2017deep, battan2021glocalnet} aims to generate pose sequence conditioned on action labels.&#10;Motion completion~\cite{hernandez2019human, kaufmann2020convolutional, holden2016deep, duan2021ssmc, harvey2018recurrent, harvey2020robust} generates realistic transitions between key frames, which most relevant to our work in their aims. The pose hallucination in our framework also aims to generate novel motion sequence, where motion graph and motion match methods are not applicable due to tight restriction in their generated data. &#10;We therefore choose motion completion considering it can generate longer sequence with continuously input key frames.&#10; " />
-        </attvalues>
-      </node>
-      <node id="2103.16385" label="2103.16385">
-        <attvalues>
-          <attvalue for="0" value="Graph Stacked Hourglass Networks for 3D Human Pose Estimation" />
-          <attvalue for="1" value="  In this paper, we propose a novel graph convolutional network architecture,&#10;Graph Stacked Hourglass Networks, for 2D-to-3D human pose estimation tasks. The&#10;proposed architecture consists of repeated encoder-decoder, in which&#10;graph-structured features are processed across three different scales of human&#10;skeletal representations. This multi-scale architecture enables the model to&#10;learn both local and global feature representations, which are critical for 3D&#10;human pose estimation. We also introduce a multi-level feature learning&#10;approach using different-depth intermediate features and show the performance&#10;improvements that result from exploiting multi-scale, multi-level feature&#10;representations. Extensive experiments are conducted to validate our approach,&#10;and the results show that our model outperforms the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.11742" label="1811.11742">
-        <attvalues>
-          <attvalue for="0" value="3D human pose estimation in video with temporal convolutions and&#10;  semi-supervised training" />
-          <attvalue for="1" value="  In this work, we demonstrate that 3D poses in video can be effectively&#10;estimated with a fully convolutional model based on dilated temporal&#10;convolutions over 2D keypoints. We also introduce back-projection, a simple and&#10;effective semi-supervised training method that leverages unlabeled video data.&#10;We start with predicted 2D keypoints for unlabeled video, then estimate 3D&#10;poses and finally back-project to the input 2D keypoints. In the supervised&#10;setting, our fully-convolutional model outperforms the previous best result&#10;from the literature by 6 mm mean per-joint position error on Human3.6M,&#10;corresponding to an error reduction of 11%, and the model also shows&#10;significant improvements on HumanEva-I. Moreover, experiments with&#10;back-projection show that it comfortably outperforms previous state-of-the-art&#10;results in semi-supervised settings where labeled data is scarce. Code and&#10;models are available at https://github.com/facebookresearch/VideoPose3D&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.00859" label="2203.00859">
-        <attvalues>
-          <attvalue for="0" value="MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose&#10;  Estimation in Video" />
-          <attvalue for="1" value="  Recent transformer-based solutions have been introduced to estimate 3D human&#10;pose from 2D keypoint sequence by considering body joints among all frames&#10;globally to learn spatio-temporal correlation. We observe that the motions of&#10;different joints differ significantly. However, the previous methods cannot&#10;efficiently model the solid inter-frame correspondence of each joint, leading&#10;to insufficient learning of spatial-temporal correlation. We propose MixSTE&#10;(Mixed Spatio-Temporal Encoder), which has a temporal transformer block to&#10;separately model the temporal motion of each joint and a spatial transformer&#10;block to learn inter-joint spatial correlation. These two blocks are utilized&#10;alternately to obtain better spatio-temporal feature encoding. In addition, the&#10;network output is extended from the central frame to entire frames of the input&#10;video, thereby improving the coherence between the input and output sequences.&#10;Extensive experiments are conducted on three benchmarks (Human3.6M,&#10;MPI-INF-3DHP, and HumanEva). The results show that our model outperforms the&#10;state-of-the-art approach by 10.9% P-MPJPE and 7.6% MPJPE. The code is&#10;available at https://github.com/JinluZhang1126/MixSTE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.02502" label="2010.02502">
-        <attvalues>
-          <attvalue for="0" value="Denoising Diffusion Implicit Models" />
-          <attvalue for="1" value="  Denoising diffusion probabilistic models (DDPMs) have achieved high quality&#10;image generation without adversarial training, yet they require simulating a&#10;Markov chain for many steps to produce a sample. To accelerate sampling, we&#10;present denoising diffusion implicit models (DDIMs), a more efficient class of&#10;iterative implicit probabilistic models with the same training procedure as&#10;DDPMs. In DDPMs, the generative process is defined as the reverse of a&#10;Markovian diffusion process. We construct a class of non-Markovian diffusion&#10;processes that lead to the same training objective, but whose reverse process&#10;can be much faster to sample from. We empirically demonstrate that DDIMs can&#10;produce high quality samples $10 \times$ to $50 \times$ faster in terms of&#10;wall-clock time compared to DDPMs, allow us to trade off computation for sample&#10;quality, and can perform semantically meaningful image interpolation directly&#10;in the latent space.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.11239" label="2006.11239">
-        <attvalues>
-          <attvalue for="0" value="Denoising Diffusion Probabilistic Models" />
-          <attvalue for="1" value="  We present high quality image synthesis results using diffusion probabilistic&#10;models, a class of latent variable models inspired by considerations from&#10;nonequilibrium thermodynamics. Our best results are obtained by training on a&#10;weighted variational bound designed according to a novel connection between&#10;diffusion probabilistic models and denoising score matching with Langevin&#10;dynamics, and our models naturally admit a progressive lossy decompression&#10;scheme that can be interpreted as a generalization of autoregressive decoding.&#10;On the unconditional CIFAR10 dataset, we obtain an Inception score of 9.46 and&#10;a state-of-the-art FID score of 3.17. On 256x256 LSUN, we obtain sample quality&#10;similar to ProgressiveGAN. Our implementation is available at&#10;https://github.com/hojonathanho/diffusion&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.16940" label="2211.16940">
-        <attvalues>
-          <attvalue for="0" value="DiffPose: Toward More Reliable 3D Pose Estimation" />
-          <attvalue for="1" value="  Monocular 3D human pose estimation is quite challenging due to the inherent&#10;ambiguity and occlusion, which often lead to high uncertainty and&#10;indeterminacy. On the other hand, diffusion models have recently emerged as an&#10;effective tool for generating high-quality images from noise. Inspired by their&#10;capability, we explore a novel pose estimation framework (DiffPose) that&#10;formulates 3D pose estimation as a reverse diffusion process. We incorporate&#10;novel designs into our DiffPose to facilitate the diffusion process for 3D pose&#10;estimation: a pose-specific initialization of pose uncertainty distributions, a&#10;Gaussian Mixture Model-based forward diffusion process, and a&#10;context-conditioned reverse diffusion process. Our proposed DiffPose&#10;significantly outperforms existing methods on the widely used pose estimation&#10;benchmarks Human3.6M and MPI-INF-3DHP. Project page:&#10;https://gongjia0208.github.io/Diffpose/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10; 3D Human Pose Estimation.&#10;Existing monocular 3D pose estimation methods can roughly be categorized into two groups: frame-based methods and video-based ones.&#10;Frame-based methods predict the 3D pose from a single RGB image. Some works \citeMain{pavlakos2017coarse, sun2018integral, fan2021motion, fan2020adaptive, park20163d, foo2023system} use Convolutional Neural Networks (CNNs) to output a human pose from the RGB image, while many works \citeMain{martinez2017simple,zhao2019semantic,zhao2022graformer,xu2021graph} first detect the 2D pose and then use it to regress the 3D pose.&#10;On the other hand, video-based methods tend to exploit temporal dependencies between frames in the video clip.&#10;Most video-based methods \citeMain{pavllo20193d,chen2021anatomy,shan2021improving,cai2019exploiting,ci2019optimizing,zheng20213d,shan2022p,hu2021conditional,xu2020deep,wang2020motion,foo2023unified} extract 2D pose sequences from the input video clip via a 2D pose detector, and focus on distilling the crucial spatial-temporal information from these 2D pose sequences for 3D pose estimation.&#10;To encode spatial-temporal information, existing works explore CNN-based frameworks with temporal convolutions \citeMain{pavllo20193d,chen2021anatomy}, GCNs \citeMain{cai2019exploiting,ci2019optimizing}, or Transformers \citeMain{zheng20213d,shan2022p}. &#10;Notably, several works \citeMain{li2022mhformer,sharma2019monocular,li2019generating} aim to alleviate the uncertainty and indeterminacy in 3D pose estimation by designing models that can generate multiple hypothesis solutions from a single input.&#10;Different from all the aforementioned works, DiffPose is formulated as a distribution-to-distribution transformation process, where we train a diffusion model to smoothly denoise from the indeterminate pose distribution to a pose distribution with low indeterminacy.&#10;By framing the 3D pose estimation procedure as a reverse diffusion process, DiffPose can naturally handle the indeterminacy and uncertainty for 3D pose estimation.&#10;&#10;Denoising Diffusion Probabilistic Models (DDPMs).&#10;DDPMs (called diffusion models for short) have emerged as an effective approach to learn a data distribution that is straightforward to sample from. &#10;Introduced by Sohl-Dickstein et al. \citeMain{sohl2015deep} for image generation, DDPMs have been further simplified and accelerated \citeMain{ho2020denoising,song2021denoising}, &#10;{and enhanced \citeMain{nachmani2021non,Zhao_2023_arxiv_watermark_dm,austin2021structured,nichol2021improved} in recent years.}&#10;Previous works have explored applying diffusion models to various generation tasks, including image inpainting \citeMain{lugmayr2022repaint} and text generation \citeMain{li2022diffusion}.&#10;Here, we explore using diffusion models to tackle 3D pose estimation with our DiffPose framework.&#10;Unlike these generation tasks \citeMain{lugmayr2022repaint,li2022diffusion} that often start the generation process from random noise, our pose estimation process starts from an estimated 2D pose with uncertainty and indeterminacy in 3D space, where the uncertainty distribution differs for each pose and can also be irregular and difficult to characterize.&#10;We also design a GCN-based architecture as our diffusion model $g$, and condition it on spatial-temporal context information to aid the reverse diffusion process and obtain accurate 3D poses.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2212.02796" label="2212.02796">
-        <attvalues>
-          <attvalue for="0" value="DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion&#10;  Probabilistic Model" />
-          <attvalue for="1" value="  Thanks to the development of 2D keypoint detectors, monocular 3D human pose&#10;estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable&#10;improvements. Still, monocular 3D HPE is a challenging problem due to the&#10;inherent depth ambiguities and occlusions. To handle this problem, many&#10;previous works exploit temporal information to mitigate such difficulties.&#10;However, there are many real-world applications where frame sequences are not&#10;accessible. This paper focuses on reconstructing a 3D pose from a single 2D&#10;keypoint detection. Rather than exploiting temporal information, we alleviate&#10;the depth ambiguity by generating multiple 3D pose candidates which can be&#10;mapped to an identical 2D keypoint. We build a novel diffusion-based framework&#10;to effectively sample diverse 3D poses from an off-the-shelf 2D detector. By&#10;considering the correlation between human joints by replacing the conventional&#10;denoising U-Net with graph convolutional network, our approach accomplishes&#10;further performance improvements. We evaluate our method on the widely adopted&#10;Human3.6M and HumanEva-I datasets. Comprehensive experiments are conducted to&#10;prove the efficacy of the proposed method, and they confirm that our model&#10;outperforms state-of-the-art multi-hypothesis 3D HPE methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.09813" label="1611.09813">
-        <attvalues>
-          <attvalue for="0" value="Monocular 3D Human Pose Estimation In The Wild Using Improved CNN&#10;  Supervision" />
-          <attvalue for="1" value="  We propose a CNN-based approach for 3D human body pose estimation from single&#10;RGB images that addresses the issue of limited generalizability of models&#10;trained solely on the starkly limited publicly available 3D pose data. Using&#10;only the existing 3D pose data and 2D pose data, we show state-of-the-art&#10;performance on established benchmarks through transfer of learned features,&#10;while also generalizing to in-the-wild scenes. We further introduce a new&#10;training set for human body pose estimation from monocular images of real&#10;humans that has the ground truth captured with a multi-camera marker-less&#10;motion capture system. It complements existing corpora with greater diversity&#10;in pose, human appearance, clothing, occlusion, and viewpoints, and enables an&#10;increased scope of augmentation. We also contribute a new benchmark that covers&#10;outdoor and indoor scenes, and demonstrate that our 3D pose dataset shows&#10;better in-the-wild performance than existing annotated data, which is further&#10;improved in conjunction with transfer learning from 2D pose data. All in all,&#10;we argue that the use of transfer learning of representations in tandem with&#10;algorithmic and data contributions is crucial for general 3D body pose&#10;estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.13257" label="2012.13257">
-        <attvalues>
-          <attvalue for="0" value="Interpolating Points on a Non-Uniform Grid using a Mixture of Gaussians" />
-          <attvalue for="1" value="  In this work, we propose an approach to perform non-uniform image&#10;interpolation based on a Gaussian Mixture Model. Traditional image&#10;interpolation methods, like nearest neighbor, bilinear, Hamming, Lanczos, etc.&#10;assume that the coordinates you want to interpolate from, are positioned on a&#10;uniform grid. However, it is not always the case in practice and we develop an&#10;interpolation method that is able to generate an image from arbitrarily&#10;positioned pixel values. We do this by representing each known pixel as a 2D&#10;normal distribution and considering each output image pixel as a sample from&#10;the mixture of all the known ones. Apart from the ability to reconstruct an&#10;image from arbitrarily positioned set of pixels, this also allows us to&#10;differentiate through the interpolation procedure, which might be helpful for&#10;downstream applications. Our optimized CUDA kernel and the source code to&#10;reproduce the benchmarks is located at&#10;https://github.com/universome/non-uniform-interpolation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Imagine that we have access to some image in a functional form.&#10;I.e. the image is represented as a function $f: \bm{p} \mapsto \bm{c}$ which takes a pixel coordinate $\bm{p} = (x,y) \in \R^2$ as an input and produces its corresponding RGB value $\bm{c} = (r, g, b) \in \R^3$.&#10;Such representations arise, for example, in differentiable rendering pipelines \cite{SynSin, SoftRasterizer} or implicit representations of images \cite{siren, fourier_inr, inr_gan, cips}.&#10;&#10;Now imagine, that we want to generate a high-resolution raster image from this functional representation $f(\bm p)$.&#10;This means that we need to evaluate $f(\bm p)$ in every coordinate location of $H \times W$ grid to generate a $H \times W$ sized image.&#10;If evaluating $f(\bm p)$ is costly then it is a tedious procedure.&#10;What can we do?&#10;&#10;One approach would be to speed up the inference for $f(\bm p)$.&#10;Another one is to generate a low-resolution version of an image and then upsample it with one of the existing methods.&#10;Such upsampling methods assume that the points you are trying to upsample from, are positioned on a uniform grid, i.e. they have a fixed equal horizontal and vertical spacing between each other, as depicted on figure \ref{fig:uniform-interpolation}.&#10;However, in practice there sometimes occur situations when your points are positioned on a non-uniform grid, like on image \ref{fig:non-uniform-interpolation}, limiting the applicability of the existing tools.&#10;&#10;To alleviate the issue, we propose a novel interpolation method that makes it possible to reconstruct an image from a subset of points that are arbitrarily scattered across the image.&#10;We achieve this by representing each known color $c^{i}$ at location $(x^{(i)}, y^{(i)})$ as a 2D normal distribution $\mathcal{N}(\bm \mu^{(i)}, \sigma^2 I)$ for $\bm\mu^{(i)} = (x^{(i)}, y^{(i)})$ and some predefined variance $\sigma^2$.&#10;&#10;To summarize, our contributions are the following:&#10;\begin{itemize}&#10; \item We propose a novel interpolation technique which is based on representing the known points as a GMM model and inferring the value for the unknown ones as an expectation.&#10; \item We develop an optimized CUDA kernel for both the forward and backward passes of the proposed interpolation procedure.&#10; \item We conduct the experiments on ImageNet dataset and show that our proposed interpolation technique outperforms in several scenarios 6 other standard interpolation methods based on the reconstruction quality.&#10;\end{itemize}&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Gaussian Mixture Models, Mathematics, Computer Science, Image Interpolation" />
-        </attvalues>
-      </node>
-      <node id="1912.08804" label="1912.08804">
-        <attvalues>
-          <attvalue for="0" value="SynSin: End-to-end View Synthesis from a Single Image" />
-          <attvalue for="1" value="  Single image view synthesis allows for the generation of new views of a scene&#10;given a single input image. This is challenging, as it requires comprehensively&#10;understanding the 3D scene from a single image. As a result, current methods&#10;typically use multiple images, train on ground-truth depth, or are limited to&#10;synthetic data. We propose a novel end-to-end model for this task; it is&#10;trained on real images without any ground-truth 3D information. To this end, we&#10;introduce a novel differentiable point cloud renderer that is used to transform&#10;a latent 3D point cloud of features into the target view. The projected&#10;features are decoded by our refinement network to inpaint missing regions and&#10;generate a realistic output image. The 3D component inside of our generative&#10;model allows for interpretable manipulation of the latent feature space at test&#10;time, e.g. we can animate trajectories from a single image. Unlike prior work,&#10;we can generate high resolution images and generalise to other input&#10;resolutions. We outperform baselines and prior work on the Matterport, Replica,&#10;and RealEstate10K datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.09661" label="2006.09661">
-        <attvalues>
-          <attvalue for="0" value="Implicit Neural Representations with Periodic Activation Functions" />
-          <attvalue for="1" value="  Implicitly defined, continuous, differentiable signal representations&#10;parameterized by neural networks have emerged as a powerful paradigm, offering&#10;many possible benefits over conventional representations. However, current&#10;network architectures for such implicit neural representations are incapable of&#10;modeling signals with fine detail, and fail to represent a signal's spatial and&#10;temporal derivatives, despite the fact that these are essential to many&#10;physical signals defined implicitly as the solution to partial differential&#10;equations. We propose to leverage periodic activation functions for implicit&#10;neural representations and demonstrate that these networks, dubbed sinusoidal&#10;representation networks or Sirens, are ideally suited for representing complex&#10;natural signals and their derivatives. We analyze Siren activation statistics&#10;to propose a principled initialization scheme and demonstrate the&#10;representation of images, wavefields, video, sound, and their derivatives.&#10;Further, we show how Sirens can be leveraged to solve challenging boundary&#10;value problems, such as particular Eikonal equations (yielding signed distance&#10;functions), the Poisson equation, and the Helmholtz and wave equations. Lastly,&#10;we combine Sirens with hypernetworks to learn priors over the space of Siren&#10;functions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.05355" label="2006.05355">
-        <attvalues>
-          <attvalue for="0" value="A Hybrid Framework for Matching Printing Design Files to Product Photos" />
-          <attvalue for="1" value="  We propose a real-time image matching framework, which is hybrid in the sense&#10;that it uses both hand-crafted features and deep features obtained from a&#10;well-tuned deep convolutional network. The matching problem, which we&#10;concentrate on, is specific to a certain application, that is, printing design&#10;to product photo matching. Printing designs are any kind of template image&#10;files, created using a design tool, thus are perfect image signals. However,&#10;photographs of a printed product suffer many unwanted effects, such as&#10;uncontrolled shooting angle, uncontrolled illumination, occlusions, printing&#10;deficiencies in color, camera noise, optic blur, et cetera. For this purpose,&#10;we create an image set that includes printing design and corresponding product&#10;photo pairs with collaboration of an actual printing facility. Using this image&#10;set, we benchmark various hand-crafted and deep features for matching&#10;performance and propose a framework in which deep learning is utilized with&#10;highest contribution, but without disabling real-time operation using an&#10;ordinary desktop computer.&#10;" />
-          <attvalue for="2" value="&#10;Image matching is a broad title that covers or partially relates to various topics among a number of different computer vision problems, namely image-based localization, multi-view 3D reconstruction, structure-from-motion, image retrieval, tracking, just to name a few. This title may refer to finding a transformed version of an image \cite{Dharani13}, or may allude to a different version of the problem, such as finding an image with a similar semantic context \cite{Liu2007}. Regardless of the problem definition, image matching boils down to a simple statement: {finding a similarity model between (at least) two images, which would satisfy the pairings for a given image set}.&#10;&#10;In recent years, a number of algorithms have been proposed under this title, which can be mainly split into two principle categories. The first category consists of approaches that utilize hand-crafted representations. Among these methods, the bag-of-words (BoW) algorithm \cite{Sivic2003} proved to be very successful, irrespective of the type of the hand-crafted feature used, and is still the state-of-the-art approach due to its flexibility, compactness and speed. &#10;&#10;However, as a part of the growing wave of interest on deep-learning-based methods, a second category of approaches recently focus on image matching using convolutional neural networks (CNN) \cite{Wang15}. The strength of these methods comes from the abstract features that emerge at the deeper layers of CNNs \cite{Yosinski2015}. The earlier approaches of this category \cite{Simonyan15,Szegedy15,Krizhevsky12,Sermanet13} performs particularly good at problems like image category classification, object detection and/or localization, mainly because of their capability to convolve abstract features into image categories or object definitions. There are also attempts with promising results, which aim at transferring pre-trained and well-tuned CNNs into image retrieval frameworks \cite{Babenko14,Chandrasekhar16}. Nonetheless, these network structures are not well-suited to match a given image to its pair, since they use fully connected layers that lead to a classification layer (such as soft-max). This final layer is used to classify the extracted abstract features into object categories. Therefore, their structure is not designed with the purpose of finding pairs. &#10;&#10;Very recently a new CNN structure, namely the Siamese network (SN), has been proposed specifically for the problem of image matching \cite{Melekhov16}. SNs utilize labeled training image pairs to learn an image-level feature representation so that similar images are mapped close to each other in the feature space, where dissimilar image pairs are mapped far from each other. This approach has also been successfully applied to similar problems that require an image-to-image matching, such as face recognition \cite{Taigman14} or aerial-to-ground image matching \cite{Lin15}. SNs improve matching performance dramatically, however they come up with two main drawbacks. Firstly, they necessitate the creation of a large-scale image set, because in order to span the entire space of possible transformations from the original image to the image to be matched, a massive number of image pairs are required. Such an image set collection and annotation effort is extremely expensive and usually, industrially impracticable. Secondly, these networks make a separate full forward deep CNN run for each candidate image in the image set, thus are slow even with dedicated hardware, such as a GPU.&#10;&#10;Deep learning is a powerful tool. In less than a decade, nearly all vision problems shifted to CNN domain. Nevertheless, it still comes with a price. As the layers of a CNN get deeper, the hardware requirements for real-time operation become more and more expensive. We still don't have a mobile solution, which may replace the high-cost and power-hungry GPUs that allow real-time deep learning operations. And when it comes to the problem of image matching, our best solution yet, namely the Siamese networks architecture, require massive training sets and forward-run for all possible candidate images. In conclusion we still need ingenious solutions for real-time, operation-specific and affordable image matching frameworks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Image Matching Techniques, Computer Vision Systems, Deep Learning Applications, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1506.06579" label="1506.06579">
-        <attvalues>
-          <attvalue for="0" value="Understanding Neural Networks Through Deep Visualization" />
-          <attvalue for="1" value="  Recent years have produced great advances in training large, deep neural&#10;networks (DNNs), including notable successes in training convolutional neural&#10;networks (convnets) to recognize natural images. However, our understanding of&#10;how these models work, especially what computations they perform at&#10;intermediate layers, has lagged behind. Progress in the field will be further&#10;accelerated by the development of better tools for visualizing and interpreting&#10;neural nets. We introduce two such tools here. The first is a tool that&#10;visualizes the activations produced on each layer of a trained convnet as it&#10;processes an image or video (e.g. a live webcam stream). We have found that&#10;looking at live activations that change in response to user input helps build&#10;valuable intuitions about how convnets work. The second tool enables&#10;visualizing features at each layer of a DNN via regularized optimization in&#10;image space. Because previous versions of this idea produced less recognizable&#10;images, here we introduce several new regularization methods that combine to&#10;produce qualitatively clearer, more interpretable visualizations. Both tools&#10;are open source and work on a pre-trained convnet with minimal setup.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1409.1556" label="1409.1556">
-        <attvalues>
-          <attvalue for="0" value="Very Deep Convolutional Networks for Large-Scale Image Recognition" />
-          <attvalue for="1" value="  In this work we investigate the effect of the convolutional network depth on&#10;its accuracy in the large-scale image recognition setting. Our main&#10;contribution is a thorough evaluation of networks of increasing depth using an&#10;architecture with very small (3x3) convolution filters, which shows that a&#10;significant improvement on the prior-art configurations can be achieved by&#10;pushing the depth to 16-19 weight layers. These findings were the basis of our&#10;ImageNet Challenge 2014 submission, where our team secured the first and the&#10;second places in the localisation and classification tracks respectively. We&#10;also show that our representations generalise well to other datasets, where&#10;they achieve state-of-the-art results. We have made our two best-performing&#10;ConvNet models publicly available to facilitate further research on the use of&#10;deep visual representations in computer vision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1312.6229" label="1312.6229">
-        <attvalues>
-          <attvalue for="0" value="OverFeat: Integrated Recognition, Localization and Detection using&#10;  Convolutional Networks" />
-          <attvalue for="1" value="  We present an integrated framework for using Convolutional Networks for&#10;classification, localization and detection. We show how a multiscale and&#10;sliding window approach can be efficiently implemented within a ConvNet. We&#10;also introduce a novel deep learning approach to localization by learning to&#10;predict object boundaries. Bounding boxes are then accumulated rather than&#10;suppressed in order to increase detection confidence. We show that different&#10;tasks can be learned simultaneously using a single shared network. This&#10;integrated framework is the winner of the localization task of the ImageNet&#10;Large Scale Visual Recognition Challenge 2013 (ILSVRC2013) and obtained very&#10;competitive results for the detection and classifications tasks. In&#10;post-competition work, we establish a new state of the art for the detection&#10;task. Finally, we release a feature extractor from our best model called&#10;OverFeat.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1508.02496" label="1508.02496">
-        <attvalues>
-          <attvalue for="0" value="A Practical Guide to CNNs and Fisher Vectors for Image Instance&#10;  Retrieval" />
-          <attvalue for="1" value="  With deep learning becoming the dominant approach in computer vision, the use&#10;of representations extracted from Convolutional Neural Nets (CNNs) is quickly&#10;gaining ground on Fisher Vectors (FVs) as favoured state-of-the-art global&#10;image descriptors for image instance retrieval. While the good performance of&#10;CNNs for image classification are unambiguously recognised, which of the two&#10;has the upper hand in the image retrieval context is not entirely clear yet. In&#10;this work, we propose a comprehensive study that systematically evaluates FVs&#10;and CNNs for image retrieval. The first part compares the performances of FVs&#10;and CNNs on multiple publicly available data sets. We investigate a number of&#10;details specific to each method. For FVs, we compare sparse descriptors based&#10;on interest point detectors with dense single-scale and multi-scale variants.&#10;For CNNs, we focus on understanding the impact of depth, architecture and&#10;training data on retrieval results. Our study shows that no descriptor is&#10;systematically better than the other and that performance gains can usually be&#10;obtained by using both types together. The second part of the study focuses on&#10;the impact of geometrical transformations such as rotations and scale changes.&#10;FVs based on interest point detectors are intrinsically resilient to such&#10;transformations while CNNs do not have a built-in mechanism to ensure such&#10;invariance. We show that performance of CNNs can quickly degrade in presence of&#10;rotations while they are far less affected by changes in scale. We then propose&#10;a number of ways to incorporate the required invariances in the CNN pipeline.&#10;Overall, our work is intended as a reference guide offering practically useful&#10;and simply implementable guidelines to anyone looking for state-of-the-art&#10;global descriptors best suited to their specific image instance retrieval&#10;problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.12217" label="2112.12217">
-        <attvalues>
-          <attvalue for="0" value="Person Detection in Collaborative Group Learning Environments Using&#10;  Multiple Representations" />
-          <attvalue for="1" value="  We introduce the problem of detecting a group of students from classroom&#10;videos. The problem requires the detection of students from different angles&#10;and the separation of the group from other groups in long videos (one to one&#10;and a half hours). We use multiple image representations to solve the problem.&#10;We use FM components to separate each group from background groups, AM-FM&#10;components for detecting the back-of-the-head, and YOLO for face detection. We&#10;use classroom videos from four different groups to validate our approach. Our&#10;use of multiple representations is shown to be significantly more accurate than&#10;the use of YOLO alone.&#10;" />
-          <attvalue for="2" value="&#10;We study the problem of person detection in &#10; collaborative learning environments' videos.&#10;The problem has some unique challenges associated&#10; with detecting students sitting around a table.&#10;&#10;We present an example of the collaborative learning environment in Figure \ref{fig:AOLMESample}. &#10;The students' faces are imaged from different angles. They are at different distances from the camera. In many cases, the faces are not visible. Furthermore, there are other groups in the background. &#10;For the purposes of this paper, &#10; we are only interested in detecting students &#10; that are sitting around the table&#10; that is closest to the camera.&#10;All other students are not to be included in the analysis.&#10;&#10;Currently, human detection methods are dominated by neural network methods.&#10;As an example, in \cite{nikouei2018real}, &#10; the authors used a lightweight Convolutional Neural Network (L-CNN) to detect &#10; humans in surveillance video frames. &#10;In another example, in \cite{zhang2017joint}, the authors &#10; used a multi-stream multitask deep network for joint human detection and head poses estimation in RGB-D videos.&#10;&#10;We also provide a summary of prior research on classroom videos.&#10;In \cite{shi2016human}, we considered using K-NN classifiers with AM-FM representations&#10; for person detection.&#10;In \cite{shi2018robust}, the combination of color and FM representations was considered&#10; for face detection.&#10;In \cite{shi2018robust}, back-of-the-head detection was performed using AM-FM representations.&#10;In \cite{shi2018dynamic}, the method in \cite{shi2018robust} was extended to detect where&#10; the students were looking.&#10;The importance of FM representations for face detection was further documented&#10; in \cite{tapia2020importance}.&#10;In \cite{shi2021talking}, the authors used head detection to detect talking activities.&#10;In \cite{teeparthi2021fast}, the authors developed methods for hand detection.&#10;In \cite{sanchez2021bilingual}, the authors considered the use of YOLO \cite{redmon2018yolov3} for head&#10; detection to build a bilingual speech recognition system.&#10;Fast video face detection was recently described in \cite{tran2021facial}.&#10;&#10;The current paper extends prior methods &#10; through the combination of YOLO with&#10; AM-FM representations.&#10;Firstly, YOLO is used to process RGB images for face detection.&#10;Secondly, FM images, characterized by higher instantaneous frequencies, are used&#10; with LeNet5 to remove non-group faces that were falsely detected by YOLO.&#10;Thirdly, LeNet5 is used to remove false positives from the back-of-the-head classifier. &#10;&#10;We summarize the rest of the paper in three additional sections.&#10;The proposed method is summarized in section\ref{sec:method}. &#10;The results are given in section\ref{sec:results}.&#10;Concluding remarks are given in section\ref{sec:conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Video Object Detection, Group Separation Algorithms, Image Representation Techniques, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1805.00330" label="1805.00330">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Human Detection as an Edge Service Enabled by a Lightweight&#10;  CNN" />
-          <attvalue for="1" value="  Edge computing allows more computing tasks to take place on the decentralized&#10;nodes at the edge of networks. Today many delay sensitive, mission-critical&#10;applications can leverage these edge devices to reduce the time delay or even&#10;to enable real time, online decision making thanks to their onsite presence.&#10;Human objects detection, behavior recognition and prediction in smart&#10;surveillance fall into that category, where a transition of a huge volume of&#10;video streaming data can take valuable time and place heavy pressure on&#10;communication networks. It is widely recognized that video processing and&#10;object detection are computing intensive and too expensive to be handled by&#10;resource limited edge devices. Inspired by the depthwise separable convolution&#10;and Single Shot Multi-Box Detector (SSD), a lightweight Convolutional Neural&#10;Network (LCNN) is introduced in this paper. By narrowing down the classifier's&#10;searching space to focus on human objects in surveillance video frames, the&#10;proposed LCNN algorithm is able to detect pedestrians with an affordable&#10;computation workload to an edge device. A prototype has been implemented on an&#10;edge node (Raspberry PI 3) using openCV libraries, and satisfactory performance&#10;is achieved using real world surveillance video streams. The experimental study&#10;has validated the design of LCNN and shown it is a promising approach to&#10;computing intensive applications at the edge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.07093" label="2203.07093">
-        <attvalues>
-          <attvalue for="0" value="Human Attention Detection Using AM-FM Representations" />
-          <attvalue for="1" value="  Human activity detection from digital videos presents many challenges to the&#10;computer vision and image processing communities. Recently, many methods have&#10;been developed to detect human activities with varying degree of success. Yet,&#10;the general human activity detection problem remains very challenging,&#10;especially when the methods need to work 'in the wild' (e.g., without having&#10;precise control over the imaging geometry). The thesis explores phase-based&#10;solutions for (i) detecting faces, (ii) back of the heads, (iii) joint&#10;detection of faces and back of the heads, and (iv) whether the head is looking&#10;to the left or the right, using standard video cameras without any control on&#10;the imaging geometry. The proposed phase-based approach is based on the&#10;development of simple and robust methods that rely on the use of Amplitude&#10;Modulation- Frequency Modulation (AM-FM) models. The approach is validated&#10;using video frames extracted from the Advancing Out-of-school Learning in&#10;Mathematics and Engineering (AOLME) project. The dataset consisted of 13,265&#10;images from ten students looking at the camera, and 6,122 images from five&#10;students looking away from the camera. For the students facing the camera, the&#10;method was able to correctly classify 97.1% of them looking to the left and&#10;95.9% of them looking to the right. For the students facing the back of the&#10;camera, the method was able to correctly classify 87.6% of them looking to the&#10;left and 93.3% of them looking to the right. The results indicate that AM-FM&#10;based methods hold great promise for analyzing human activity videos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07646" label="2110.07646">
-        <attvalues>
-          <attvalue for="0" value="Talking Detection In Collaborative Learning Environments" />
-          <attvalue for="1" value="  We study the problem of detecting talking activities in collaborative&#10;learning videos. Our approach uses head detection and projections of the&#10;log-magnitude of optical flow vectors to reduce the problem to a simple&#10;classification of small projection images without the need for training&#10;complex, 3-D activity classification systems. The small projection images are&#10;then easily classified using a simple majority vote of standard classifiers.&#10;For talking detection, our proposed approach is shown to significantly&#10;outperform single activity systems. We have an overall accuracy of 59% compared&#10;to 42% for Temporal Segment Network (TSN) and 45% for Convolutional 3D (C3D).&#10;In addition, our method is able to detect multiple talking instances from&#10;multiple speakers, while also detecting the speakers themselves.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07070" label="2110.07070">
-        <attvalues>
-          <attvalue for="0" value="Fast Hand Detection in Collaborative Learning Environments" />
-          <attvalue for="1" value="  Long-term object detection requires the integration of frame-based results&#10;over several seconds. For non-deformable objects, long-term detection is often&#10;addressed using object detection followed by video tracking. Unfortunately,&#10;tracking is inapplicable to objects that undergo dramatic changes in appearance&#10;from frame to frame. As a related example, we study hand detection over long&#10;video recordings in collaborative learning environments. More specifically, we&#10;develop long-term hand detection methods that can deal with partial occlusions&#10;and dramatic changes in appearance.&#10;  Our approach integrates object-detection, followed by time projections,&#10;clustering, and small region removal to provide effective hand detection over&#10;long videos. The hand detector achieved average precision (AP) of 72% at 0.5&#10;intersection over union (IoU). The detection results were improved to 81% by&#10;using our optimized approach for data augmentation. The method runs at 4.7x the&#10;real-time with AP of 81% at 0.5 intersection over the union. Our method reduced&#10;the number of false-positive hand detections by 80% by improving IoU ratios&#10;from 0.2 to 0.5. The overall hand detection system runs at 4x real-time.&#10;" />
-          <attvalue for="2" value="&#10;We study the problem of developing&#10; a robust method for detecting student&#10; hands in collaborative&#10; learning environment&#10; \cite{celedon2013interdisciplinary}.&#10;Here, we define a collaborative learning environment&#10; as a small group of students working together&#10; in a single table as shown in Fig. \ref{fig:problem_many_hands}.&#10;Our goal is to recognize&#10; writing and typing&#10; activities over the&#10; detected hand regions.&#10;We will then use the&#10; writing and typing&#10; activities to assess&#10; student participation.&#10;&#10;For robust detection,&#10; we require that our hand detection&#10; results are consistent throughout the video,&#10; implying that we need to deal with occlusions.&#10;Furthermore, we need to reject&#10; hands that belong to students that &#10; belong to other groups, as opposed&#10; to the collaborative group&#10; that is closer to the camera&#10; (see Fig. \ref{fig:problem_many_hands}).&#10;Since our ultimate goal is to apply&#10; our methods to about 1,000 hours of&#10; digital videos, we also require&#10; that our methods are fast.&#10;&#10;We also recognize the dynamic aspects&#10; of the hand detection problem.&#10;First, it is clear that we need to associate&#10; hands with different people and&#10; that there is a need to deal with &#10; the fact that hands can disappear from&#10; view due to occlusion (see Fig. \ref{fig:problem_many_hands}).&#10;Second, we note that the same hands assume&#10; very different appearances throughout the video&#10; and that there is a need to associate&#10; their variations with a single instance.&#10;&#10;We summarize some earlier research on the same problem&#10; in the M.Sc. thesis by C.J. Darsey \cite{darsey2018hand}.&#10;In her thesis, the author studied the problem of accurate&#10; hand segmentation over a limited dataset. &#10;The dataset consisted of $15$ video clips of a &#10; maximum duration of $99$ seconds.&#10;While the methods were successful over a limited video dataset,&#10; it is important to note that we are dramatically&#10; extending this prior research to long-term detection&#10; of hand regions over long video segments.&#10;Thus, unlike \cite{darsey2018hand}, the current paper&#10; also deals with occlusion, rejecting hands outside&#10; the group, and associating hand regions with different&#10; students.&#10;We also have an earlier attempt to detect hands&#10; using deep learning in \cite{jacoby2018context}.&#10;The current paper dramatically extends this prior&#10; research that was focused on very short video datasets&#10; without considering occlusion, appearance issues, and associating hands with different people.&#10;We also note that head detection and person recognition has been studied in&#10; \cite{shi3}, \cite{shi2}, \cite{Tran2021} and \cite{shi2021}.&#10;Human activity classification over cropped regions was&#10; studied in \cite{eilar2016distributed1}, \cite{shi2021b} and \cite{VJ2021}.&#10; In addtion, we note that speech recognition using speaker geometry is studied in \cite{Luis2021}.&#10;&#10;The current paper uses transfer learning from&#10; deep learning methods to provide initial hand&#10; detection results.&#10;For this initial step, we tested several&#10; well-known methods.&#10;We tested Faster R-CNN \cite{ren2016faster}, &#10; YOLO \cite{redmon2016you}, &#10; and SSD \cite{liu2016ssd}.&#10;We then decided to adopt Faster R-CNN as our baseline&#10; model due to the fact that it is more widely supported&#10; within human activity recognition systems.&#10;We then build our system by post-processing the results&#10; from Faster R-CNN.&#10;More specifically, we project the results over short&#10; video segments to address occlusion and then develop&#10; a clustering approach and small area removal to&#10; identify the students within the &#10; current collaborative group, which are&#10; not addressed by traditional hand tracking methods (e.g., \cite{HandTracking}).&#10;Our approach yields significant improvements over&#10; the standard use of Faster R-CNN.&#10;&#10;The rest of the paper is organized into three additional&#10; sections.&#10;We summarize the methodology in &#10; section\ref{sec:methodology}. &#10;We then present results in section\ref{sec:results}&#10; and provide concluding remarks in &#10; section\ref{sec:conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.13463" label="2112.13463">
-        <attvalues>
-          <attvalue for="0" value="Bilingual Speech Recognition by Estimating Speaker Geometry from Video&#10;  Data" />
-          <attvalue for="1" value="  Speech recognition is very challenging in student learning environments that&#10;are characterized by significant cross-talk and background noise. To address&#10;this problem, we present a bilingual speech recognition system that uses an&#10;interactive video analysis system to estimate the 3D speaker geometry for&#10;realistic audio simulations. We demonstrate the use of our system in generating&#10;a complex audio dataset that contains significant cross-talk and background&#10;noise that approximate real-life classroom recordings. We then test our&#10;proposed system with real-life recordings.&#10;  In terms of the distance of the speakers from the microphone, our interactive&#10;video analysis system obtained a better average error rate of 10.83% compared&#10;to 33.12% for a baseline approach. Our proposed system gave an accuracy of&#10;27.92% that is 1.5% better than Google Speech-to-text on the same dataset. In&#10;terms of 9 important keywords, our approach gave an average sensitivity of 38%&#10;compared to 24% for Google Speech-to-text, while both methods maintained high&#10;average specificity of 90% and 92%.&#10;  On average, sensitivity improved from 24% to 38% for our proposed approach.&#10;On the other hand, specificity remained high for both methods (90% to 92%).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.02767" label="1804.02767">
-        <attvalues>
-          <attvalue for="0" value="YOLOv3: An Incremental Improvement" />
-          <attvalue for="1" value="  We present some updates to YOLO! We made a bunch of little design changes to&#10;make it better. We also trained this new network that's pretty swell. It's a&#10;little bigger than last time but more accurate. It's still fast though, don't&#10;worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but&#10;three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3&#10;is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5&#10;mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always,&#10;all the code is online at https://pjreddie.com/yolo/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.13269" label="2110.13269">
-        <attvalues>
-          <attvalue for="0" value="Facial Recognition in Collaborative Learning Videos" />
-          <attvalue for="1" value="  Face recognition in collaborative learning videos presents many challenges.&#10;In collaborative learning videos, students sit around a typical table at&#10;different positions to the recording camera, come and go, move around, get&#10;partially or fully occluded. Furthermore, the videos tend to be very long,&#10;requiring the development of fast and accurate methods. We develop a dynamic&#10;system of recognizing participants in collaborative learning systems. We&#10;address occlusion and recognition failures by using past information about the&#10;face detection history. We address the need for detecting faces from different&#10;poses and the need for speed by associating each participant with a collection&#10;of prototype faces computed through sampling or K-means clustering. Our results&#10;show that the proposed system is proven to be very fast and accurate. We also&#10;compare our system against a baseline system that uses InsightFace [2] and the&#10;original training video segments. We achieved an average accuracy of 86.2%&#10;compared to 70.8% for the baseline system. On average, our recognition rate was&#10;28.1 times faster than the baseline system.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.03783" label="1810.03783">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Online Video Object Segmentation with Motion Property&#10;  Understanding" />
-          <attvalue for="1" value="  Unsupervised video object segmentation aims to automatically segment moving&#10;objects over an unconstrained video without any user annotation. So far, only&#10;few unsupervised online methods have been reported in literature and their&#10;performance is still far from satisfactory, because the complementary&#10;information from future frames cannot be processed under online setting. To&#10;solve this challenging problem, in this paper, we propose a novel Unsupervised&#10;Online Video Object Segmentation (UOVOS) framework by construing the motion&#10;property to mean moving in concurrence with a generic object for segmented&#10;regions. By incorporating salient motion detection and object proposal, a&#10;pixel-wise fusion strategy is developed to effectively remove detection noise&#10;such as dynamic background and stationary objects. Furthermore, by leveraging&#10;the obtained segmentation from immediately preceding frames, a forward&#10;propagation algorithm is employed to deal with unreliable motion detection and&#10;object proposals. Experimental results on several benchmark datasets&#10;demonstrate the efficacy of the proposed method. Compared to the&#10;state-of-the-art unsupervised online segmentation algorithms, the proposed&#10;method achieves an absolute gain of 6.2%. Moreover, our method achieves better&#10;performance than the best unsupervised offline algorithm on the DAVIS-2016&#10;benchmark dataset. Our code is available on the project website:&#10;https://github.com/visiontao/uovos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2307.04392" label="2307.04392">
-        <attvalues>
-          <attvalue for="0" value="FODVid: Flow-guided Object Discovery in Videos" />
-          <attvalue for="1" value="  Segmentation of objects in a video is challenging due to the nuances such as&#10;motion blurring, parallax, occlusions, changes in illumination, etc. Instead of&#10;addressing these nuances separately, we focus on building a generalizable&#10;solution that avoids overfitting to the individual intricacies. Such a solution&#10;would also help us save enormous resources involved in human annotation of&#10;video corpora. To solve Video Object Segmentation (VOS) in an unsupervised&#10;setting, we propose a new pipeline (FODVid) based on the idea of guiding&#10;segmentation outputs using flow-guided graph-cut and temporal consistency.&#10;Basically, we design a segmentation model incorporating intra-frame appearance&#10;and flow similarities, and inter-frame temporal continuation of the objects&#10;under consideration. We perform an extensive experimental analysis of our&#10;straightforward methodology on the standard DAVIS16 video benchmark. Though&#10;simple, our approach produces results comparable (within a range of ~2 mIoU) to&#10;the existing top approaches in unsupervised VOS. The simplicity and&#10;effectiveness of our technique opens up new avenues for research in the video&#10;domain.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object segmentation, in its various forms, is a widely studied problem in computer vision~\cite{long2015fully}. The classic task finds critical applications across multiple domains, such as autonomous driving, augmented reality, human-computer interaction, video summarization etc. The deep neural networks employed for this purpose are typically trained on large annotated datasets created through enormous human efforts that take several months of focused work. Moreover, training a segmentation model on one such dataset does not guarantee transfer-ability to real-world data since dataset-specific considerations in model design overfit the model to a particular use case. These issues call attention to developing generalizable solutions that can work with minimal human supervision.&#10;&#10;To alleviate some of the problems with supervised segmentation, methods that work on weaker forms of human supervision were proposed. These methods function with weak supervision provided through scribbles~\cite{lin2016scribblesup, tang2018normalized, tang2018regularized, vernaza2017learning, xu2015learning} or clicks~\cite{bearman2016s} or image-level tags~\cite{papandreou2015weakly, tang2018regularized, xu2015learning} or even bounds~\cite{dai2015boxsup}. Further, semi-supervised segmentation techniques~\cite{dai2015boxsup, hong2015decoupled, hung2018adversarial, papandreou2015weakly, pathak2015constrained} were proposed that attempt segmentation in a setting where only a fraction of the image datasets are human-labelled. Nonetheless, both weak-supervised and semi-supervised techniques still rely on bulky human supervision directly or indirectly. Therefore, in the present work, we focus on segmenting objects in a video without relying on any form of external supervision.&#10;&#10;The aim of Video Object Segmentation (VOS) is to localize the most salient object(s) in a given video frame~\cite{perazzi2016benchmark}. In the literature~\cite{li2013video, ochs2013segmentation, perazzi2016benchmark, xie2022segmenting, ye2022deformable}, VOS is generally re-framed as a foreground-background separation problem. We propose an end-to-end pipeline as follows -- for a given video frame, we encode its frames and their optical flows in RGB format using a self-supervised ViT like DINO~\cite{caron2021emerging}. The image and flow features are then used in a linear combination to form a similarity based adjacency matrix between the frame patches. By performing graph-cut on this adjacency matrix, we obtain a preliminary set of segmentation masks for all the frames in a video. These masks are then used as pseudo-ground truths to train a segmentation network. We devise a loss schedule that alternates between graph-cut mask of current frame and a nearby frame to enforce temporal consistency while training the segmentation network(refer \ref{sec:temporal_warping}).&#10;&#10;We summarize the contributions of this work below:&#10;&#10;\begin{enumerate}[nosep]&#10;&#10;\item We present a new pipeline (FODVid) for unsupervised Video Object Segmentation (VOS), utilising both appearance and motion information contained in videos.&#10;&#10;\item We demonstrate the importance of perceptual motion cues for object discovery. In particular, we employ the Gestalt principle, ``things that move together, belong together&quot; to enrich frame patch similarity.&#10;&#10;\item Our methodology is simple to implement and produces results comparable to existing top unsupervised VOS approaches. We achieve an mIoU score of $78.71$ on the standard DAVIS16 benchmark. Additionally, the proposed temporal refinement provides an improvement of as much as $+9.88 mIoU$ on certain video sequences.&#10;&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1604.05144" label="1604.05144">
-        <attvalues>
-          <attvalue for="0" value="ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Large-scale data is of crucial importance for learning semantic segmentation&#10;models, but annotating per-pixel masks is a tedious and inefficient procedure.&#10;We note that for the topic of interactive image segmentation, scribbles are&#10;very widely used in academic research and commercial software, and are&#10;recognized as one of the most user-friendly ways of interacting. In this paper,&#10;we propose to use scribbles to annotate images, and develop an algorithm to&#10;train convolutional networks for semantic segmentation supervised by scribbles.&#10;Our algorithm is based on a graphical model that jointly propagates information&#10;from scribbles to unmarked pixels and learns network parameters. We present&#10;competitive object semantic segmentation results on the PASCAL VOC dataset by&#10;using scribbles as annotations. Scribbles are also favored for annotating stuff&#10;(e.g., water, sky, grass) that has no well-defined shape, and our method shows&#10;excellent results on the PASCAL-CONTEXT dataset thanks to extra inexpensive&#10;scribble annotations. Our scribble annotations on PASCAL VOC are available at&#10;http://research.microsoft.com/en-us/um/people/jifdai/downloads/scribble_sup&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.01640" label="1503.01640">
-        <attvalues>
-          <attvalue for="0" value="BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks&#10;  for Semantic Segmentation" />
-          <attvalue for="1" value="  Recent leading approaches to semantic segmentation rely on deep convolutional&#10;networks trained with human-annotated, pixel-level segmentation masks. Such&#10;pixel-accurate supervision demands expensive labeling effort and limits the&#10;performance of deep networks that usually benefit from more training data. In&#10;this paper, we propose a method that achieves competitive accuracy but only&#10;requires easily obtained bounding box annotations. The basic idea is to iterate&#10;between automatically generating region proposals and training convolutional&#10;networks. These two steps gradually recover segmentation masks for improving&#10;the networks, and vise versa. Our method, called BoxSup, produces competitive&#10;results supervised by boxes only, on par with strong baselines fully supervised&#10;by masks under the same setting. By leveraging a large amount of bounding&#10;boxes, BoxSup further unleashes the power of deep convolutional networks and&#10;yields state-of-the-art results on PASCAL VOC 2012 and PASCAL-CONTEXT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14294" label="2104.14294">
-        <attvalues>
-          <attvalue for="0" value="Emerging Properties in Self-Supervised Vision Transformers" />
-          <attvalue for="1" value="  In this paper, we question if self-supervised learning provides new&#10;properties to Vision Transformer (ViT) that stand out compared to convolutional&#10;networks (convnets). Beyond the fact that adapting self-supervised methods to&#10;this architecture works particularly well, we make the following observations:&#10;first, self-supervised ViT features contain explicit information about the&#10;semantic segmentation of an image, which does not emerge as clearly with&#10;supervised ViTs, nor with convnets. Second, these features are also excellent&#10;k-NN classifiers, reaching 78.3% top-1 on ImageNet with a small ViT. Our study&#10;also underlines the importance of momentum encoder, multi-crop training, and&#10;the use of small patches with ViTs. We implement our findings into a simple&#10;self-supervised method, called DINO, which we interpret as a form of&#10;self-distillation with no labels. We show the synergy between DINO and ViTs by&#10;achieving 80.1% top-1 on ImageNet in linear evaluation with ViT-Base.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.14581" label="2203.14581">
-        <attvalues>
-          <attvalue for="0" value="S2-Net: Self-supervision Guided Feature Representation Learning for&#10;  Cross-Modality Images" />
-          <attvalue for="1" value="  Combining the respective advantages of cross-modality images can compensate&#10;for the lack of information in the single modality, which has attracted&#10;increasing attention of researchers into multi-modal image matching tasks.&#10;Meanwhile, due to the great appearance differences between cross-modality image&#10;pairs, it often fails to make the feature representations of correspondences as&#10;close as possible. In this letter, we design a cross-modality feature&#10;representation learning network, S2-Net, which is based on the recently&#10;successful detect-and-describe pipeline, originally proposed for visible images&#10;but adapted to work with cross-modality image pairs. To solve the consequent&#10;problem of optimization difficulties, we introduce self-supervised learning&#10;with a well-designed loss function to guide the training without discarding the&#10;original advantages. This novel strategy simulates image pairs in the same&#10;modality, which is also a useful guide for the training of cross-modality&#10;images. Notably, it does not require additional data but significantly improves&#10;the performance and is even workable for all methods of the detect-and-describe&#10;pipeline. Extensive experiments are conducted to evaluate the performance of&#10;the strategy we proposed, compared to both handcrafted and deep learning-based&#10;methods. Results show that our elegant formulation of combined optimization of&#10;supervised and self-supervised learning outperforms state-of-the-arts on&#10;RoadScene and RGB-NIR datasets.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Establishing the local correspondences between two images, as a primary task, is the premise of various visual applications, including target recognition, visual navigation, image stitching, 3D reconstruction and visual localization\cite{ma2021image,ohata2020automatic}. In particular, local feature representation is a popular approach to correspondence estimation. Recently, increasing attention has been focused on the cross-modality image feature representation learning and matching, since images of different modalities have their own advantages and contain different perceptual information, so it is meaningful to fuse the superiority of cross-modality images \cite{ma2019infrared}. &#10;&#10;The conventional matching methods are based on the handcrafted local feature descriptors \cite{ma2022robust,lowe2004distinctive,rublee2011orb,li2019rift} to make the representation of two matched features as similar as possible and as discriminant as possible from that of unmatched ones. Over the recent years, the deep learning-based methods have achieved significant progress in general visual tasks, and have also been introduced into the field of image matching \cite{yi2016lift,ono2018lf}. The current approaches are mostly based on a two-stage pipeline that first completes the extraction of keypoints and then encodes the patches centered on the keypoints into descriptors, thus referred to as the detect-then-describe methods. In the field of cross-modality image matching, the detect-then-describe methods have been widely used with a manual detector to detect and an adapted deep learning network to perform description \cite{jiang2021review,zhang2019registration,aguilera2017cross}. &#10;&#10;Despite this apparent success, it is an inevitable disadvantage of this paradigm that the global spatial information is discarded during the description process, which happens to be essential for cross-modality images. In contrast to it, the detect-and-describe framework for visible images uses a network to simultaneously perform feature point extraction and descriptor construction \cite{dusmanu2019d2,revaud2019r2d2}. This approach postpones the detection process without missing high-level information of images. Additionally, the detection stage is tightly coupled with the description so as to detect pixels with locally unique descriptors that are better for matching. Undoubtedly, it is promising to introduce the framework into cross-modality image matching, however, challenges come up due to the huge heterogeneity. To be specific, it is difficult to optimize the model for cross-modality images with extreme geometric and radiometric variances. &#10;&#10;Self-supervised learning (SSL), which helps the model obtain easy invariance with augmented data, is one of the most popular technics in natural language processing and computer vision\cite{chen2020simple,sermanet2017time}. As for local feature representation learning, the well-known Superpoint \cite{detone2018superpoint} proposed a novel Homographic Adaptation procedure, which is a form of self-supervision, to tackle the ill-posed problem of keypoint extraction. Nevertheless, the SSL technics have not been introduced into cross-modality scenario, while current methods are devoted to obtaining supervised signals from labeled data instead. Since the learning becomes harder for cross-modality images due to the serious radiometric variances, it is desirable to introduce SSL into this task. In fact, among the challenges faced by cross-modality descriptors, excluding inter-modal invariance, other necessities including geometric invariance as well as robustness to noise and grayscale variations can be well-addressed by SSL. The excellent data utilization efficiency and generalization ability of SSL enables its low-cost access to large amounts of training data, which is desperately required for cross-modality feature representation learning tasks.&#10;&#10;In this work, we explore the possibility of using SSL, based on the recent success of the detect-and-describe methods, but adapted to work with cross-modality image pairs. Although the cross-modality images are heterogeneous and quite different in appearance, they still have some similar semantic information, such as the shape, structure, and topological relationship. The detect-and-describe methods retains the global spatial information which is rather crucial for our task. As for the optimization problem, we provide an effective solution for the application of the detect-and-describe methods to the cross-modality domain. More precisely, we propose our novel architecture of joint training with supervised and self-supervised learning, termed S2-Net, which takes full advantage of SSL to improve matching performance without extra labeled data, as illustrated in Fig. \ref{fig:1}. Self-supervision simulates the feature representation learning of images in the same modalities. Since the task of training image representations of the same modality is relatively easier compared to different modalities, self-supervision plays a guiding role in the training process. So in addition to supervised learning (SL), we add SSL to the optimization process, simultaneously training representations of images in the same and different modalities. Also, we design a loss function that combines both supervised and self-supervised learning and optimally balances the guidance of the two optimization methods. To the best of our knowledge, S2-Net is the first algorithm that introduces the SSL technique into cross-modality feature representation, and sufficient experiments have demonstrated the great effectiveness of our work. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Multi-Modal Image Matching, Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Deep Learning Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1805.09662" label="1805.09662">
-        <attvalues>
-          <attvalue for="0" value="LF-Net: Learning Local Features from Images" />
-          <attvalue for="1" value="  We present a novel deep architecture and a training strategy to learn a local&#10;feature pipeline from scratch, using collections of images without the need for&#10;human supervision. To do so we exploit depth and relative camera pose cues to&#10;create a virtual target that the network should achieve on one image, provided&#10;the outputs of the network for the other image. While this process is&#10;inherently non-differentiable, we show that we can optimize the network in a&#10;two-branch setup by confining it to one branch, while preserving&#10;differentiability in the other. We train our method on both indoor and outdoor&#10;datasets, with depth data from 3D sensors for the former, and depth estimates&#10;from an off-the-shelf Structure-from-Motion solution for the latter. Our models&#10;outperform the state of the art on sparse feature matching on both datasets,&#10;while running at 60+ fps for QVGA images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.05709" label="2002.05709">
-        <attvalues>
-          <attvalue for="0" value="A Simple Framework for Contrastive Learning of Visual Representations" />
-          <attvalue for="1" value="  This paper presents SimCLR: a simple framework for contrastive learning of&#10;visual representations. We simplify recently proposed contrastive&#10;self-supervised learning algorithms without requiring specialized architectures&#10;or a memory bank. In order to understand what enables the contrastive&#10;prediction tasks to learn useful representations, we systematically study the&#10;major components of our framework. We show that (1) composition of data&#10;augmentations plays a critical role in defining effective predictive tasks, (2)&#10;introducing a learnable nonlinear transformation between the representation and&#10;the contrastive loss substantially improves the quality of the learned&#10;representations, and (3) contrastive learning benefits from larger batch sizes&#10;and more training steps compared to supervised learning. By combining these&#10;findings, we are able to considerably outperform previous methods for&#10;self-supervised and semi-supervised learning on ImageNet. A linear classifier&#10;trained on self-supervised representations learned by SimCLR achieves 76.5%&#10;top-1 accuracy, which is a 7% relative improvement over previous&#10;state-of-the-art, matching the performance of a supervised ResNet-50. When&#10;fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy,&#10;outperforming AlexNet with 100X fewer labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.00966" label="1807.00966">
-        <attvalues>
-          <attvalue for="0" value="Supervision-by-Registration: An Unsupervised Approach to Improve the&#10;  Precision of Facial Landmark Detectors" />
-          <attvalue for="1" value="  In this paper, we present supervision-by-registration, an unsupervised&#10;approach to improve the precision of facial landmark detectors on both images&#10;and video. Our key observation is that the detections of the same landmark in&#10;adjacent frames should be coherent with registration, i.e., optical flow.&#10;Interestingly, the coherency of optical flow is a source of supervision that&#10;does not require manual labeling, and can be leveraged during detector&#10;training. For example, we can enforce in the training loss function that a&#10;detected landmark at frame$_{t-1}$ followed by optical flow tracking from&#10;frame$_{t-1}$ to frame$_t$ should coincide with the location of the detection&#10;at frame$_t$. Essentially, supervision-by-registration augments the training&#10;loss function with a registration loss, thus training the detector to have&#10;output that is not only close to the annotations in labeled images, but also&#10;consistent with registration on large amounts of unlabeled videos. End-to-end&#10;training with the registration loss is made possible by a differentiable&#10;Lucas-Kanade operation, which computes optical flow registration in the forward&#10;pass, and back-propagates gradients that encourage temporal coherency in the&#10;detector. The output of our method is a more precise image-based facial&#10;landmark detector, which can be applied to single images or video. With&#10;supervision-by-registration, we demonstrate (1) improvements in facial landmark&#10;detection on both images (300W, ALFW) and video (300VW, Youtube-Celebrities),&#10;and (2) significant reduction of jittering in video detections.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Optical Flow Registration, Facial Landmark Detection, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2007.01549" label="2007.01549">
-        <attvalues>
-          <attvalue for="0" value="PointTrack++ for Effective Online Multi-Object Tracking and Segmentation" />
-          <attvalue for="1" value="  Multiple-object tracking and segmentation (MOTS) is a novel computer vision&#10;task that aims to jointly perform multiple object tracking (MOT) and instance&#10;segmentation. In this work, we present PointTrack++, an effective on-line&#10;framework for MOTS, which remarkably extends our recently proposed PointTrack&#10;framework. To begin with, PointTrack adopts an efficient one-stage framework&#10;for instance segmentation, and learns instance embeddings by converting compact&#10;image representations to un-ordered 2D point cloud. Compared with PointTrack,&#10;our proposed PointTrack++ offers three major improvements. Firstly, in the&#10;instance segmentation stage, we adopt a semantic segmentation decoder trained&#10;with focal loss to improve the instance selection quality. Secondly, to further&#10;boost the segmentation performance, we propose a data augmentation strategy by&#10;copy-and-paste instances into training images. Finally, we introduce a better&#10;training strategy in the instance association stage to improve the&#10;distinguishability of learned instance embeddings. The resulting framework&#10;achieves the state-of-the-art performance on the 5th BMTT MOTChallenge.&#10;" />
-          <attvalue for="2" value="&#10;Multi-object tracking (MOT) is an essential task in computer vision with broad applications such as robotics and video surveillance. It is widely noticed that object detection and association become challenging in crowded scenes where bounding boxes (bboxes) of different objects might overlap heavily. Recently, multi-object tracking and segmentation (MOTS) \cite{voigtlaender2019mots} extends MOT by jointly considering instance segmentation and tracking. &#10;In addition to bbox annotations, MOTS provides pixel-wise segmentation labels. As segments precisely delineate the visible object boundaries and separate adjacent objects naturally, MOTS not only enables pixel-level analysis but more importantly encourages to learn more discriminative embeddings for instance association based on segments rather than bboxes.&#10;&#10;Nevertheless, learning instance embeddings from segments have rarely been explored by current MOTS methods. TRCNN \cite{voigtlaender2019mots} extends Mask-RCNN to jointly process consecutive frames using 3D convolutions and adopts ROI Align to extract instance embeddings in bbox proposals. To focus on the segment area, Porzi et al. \cite{porzi2019learning} introduce mask pooling rather than ROI Align for instance feature extraction. However, vanilla 2D or 3D convolutions are harmful for learning discriminative instance embeddings due to inherent large receptive fields. Deep convolutional features not only mix up the foreground area and the background area but also mix up the foreground area of the interested instance and its adjacent instances. Therefore, though current MOTS methods adopt advanced segmentation backbones to extract image features, they fail to learn discriminative instance embeddings which are essential for robust instance association, resulting in limited performances.&#10;&#10;In our previous work, we propose a simple yet highly effective method named PointTrack \cite{xu2020Segment} to learn instance embeddings on segments. As bbox-proposal based instance segmentation methods always miss bboxes when instances multiple bboxes are heavily overlapped, PointTrack adopts a proposal-free instance segmentation network \cite{Neven_2019_CVPR} following the encoder-decoder architecture for efficient instance segmentation. Afterward, for each instance, PointTrack regards raw 2D image pixels as un-ordered 2D point clouds and learns instance embeddings on segments in a point cloud processing manner. As the instance embeddings are learned from raw pixels based on predicted segments, the instance segmentation stage and the instance embedding extraction stage are completely decoupled. In this way, different from previous works \cite{voigtlaender2019mots} which requires consecutive frames as inputs, PointTrack enables a more flexible training strategy since both image and video level segmentation labels can be used. Built on PointTrack, in this paper, we propose PointTrack++ which improves PointTrack by three modifications. Firstly, based on the observation that the poor seed map prediction results in many false-positives and false-negatives, we replace the seed map branch with the semantic segmentation branch and regard the semantic class confidence as the seed score for pixel selection in inference. Secondly, to create more crowded scenes for instance segmentation training, we introduce the Copy-and-Paste strategy by copying instances with similar lightness to cover instances in training images. Lastly, based on the intuition that larger intra-track-id discrepancy which is beneficial for learning the foreground embeddings is harmful for learning the environment embeddings and the position embeddings, we propose a multi-stage training for learning more discriminative instance embeddings. The resulting framework PointTrack++ ranks first on the official KITTI MOTS leader-board and is the winning solution for 5th BMTT MOTChallenge.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Tracking, Instance Segmentation, Image Processing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1902.03604" label="1902.03604">
-        <attvalues>
-          <attvalue for="0" value="MOTS: Multi-Object Tracking and Segmentation" />
-          <attvalue for="1" value="  This paper extends the popular task of multi-object tracking to multi-object&#10;tracking and segmentation (MOTS). Towards this goal, we create dense&#10;pixel-level annotations for two existing tracking datasets using a&#10;semi-automatic annotation procedure. Our new annotations comprise 65,213 pixel&#10;masks for 977 distinct objects (cars and pedestrians) in 10,870 video frames.&#10;For evaluation, we extend existing multi-object tracking metrics to this new&#10;task. Moreover, we propose a new baseline method which jointly addresses&#10;detection, tracking, and segmentation with a single convolutional network. We&#10;demonstrate the value of our datasets by achieving improvements in performance&#10;when training on MOTS annotations. We believe that our datasets, metrics and&#10;baseline will become a valuable resource towards developing multi-object&#10;tracking approaches that go beyond 2D bounding boxes. We make our annotations,&#10;code, and models available at https://www.vision.rwth-aachen.de/page/mots.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.02096" label="1912.02096">
-        <attvalues>
-          <attvalue for="0" value="Learning Multi-Object Tracking and Segmentation from Automatic&#10;  Annotations" />
-          <attvalue for="1" value="  In this work we contribute a novel pipeline to automatically generate&#10;training data, and to improve over state-of-the-art multi-object tracking and&#10;segmentation (MOTS) methods. Our proposed track mining algorithm turns raw&#10;street-level videos into high-fidelity MOTS training data, is scalable and&#10;overcomes the need of expensive and time-consuming manual annotation&#10;approaches. We leverage state-of-the-art instance segmentation results in&#10;combination with optical flow predictions, also trained on automatically&#10;harvested training data. Our second major contribution is MOTSNet - a deep&#10;learning, tracking-by-detection architecture for MOTS - deploying a novel&#10;mask-pooling layer for improved object association over time. Training MOTSNet&#10;with our automatically extracted data leads to significantly improved sMOTSA&#10;scores on the novel KITTI MOTS dataset (+1.9%/+7.5% on cars/pedestrians), and&#10;MOTSNet improves by +4.1% over previously best methods on the MOTSChallenge&#10;dataset. Our most impressive finding is that we can improve over previous&#10;best-performing works, even in complete absence of manually annotated MOTS&#10;training data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.01550" label="2007.01550">
-        <attvalues>
-          <attvalue for="0" value="Segment as Points for Efficient Online Multi-Object Tracking and&#10;  Segmentation" />
-          <attvalue for="1" value="  Current multi-object tracking and segmentation (MOTS) methods follow the&#10;tracking-by-detection paradigm and adopt convolutions for feature extraction.&#10;However, as affected by the inherent receptive field, convolution based feature&#10;extraction inevitably mixes up the foreground features and the background&#10;features, resulting in ambiguities in the subsequent instance association. In&#10;this paper, we propose a highly effective method for learning instance&#10;embeddings based on segments by converting the compact image representation to&#10;un-ordered 2D point cloud representation. Our method generates a new&#10;tracking-by-points paradigm where discriminative instance embeddings are&#10;learned from randomly selected points rather than images. Furthermore, multiple&#10;informative data modalities are converted into point-wise representations to&#10;enrich point-wise features. The resulting online MOTS framework, named&#10;PointTrack, surpasses all the state-of-the-art methods including 3D tracking&#10;methods by large margins (5.4% higher MOTSA and 18 times faster over&#10;MOTSFusion) with the near real-time speed (22 FPS). Evaluations across three&#10;datasets demonstrate both the effectiveness and efficiency of our method.&#10;Moreover, based on the observation that current MOTS datasets lack crowded&#10;scenes, we build a more challenging MOTS dataset named APOLLO MOTS with higher&#10;instance density. Both APOLLO MOTS and our codes are publicly available at&#10;https://github.com/detectRecog/PointTrack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11109" label="1906.11109">
-        <attvalues>
-          <attvalue for="0" value="Instance Segmentation by Jointly Optimizing Spatial Embeddings and&#10;  Clustering Bandwidth" />
-          <attvalue for="1" value="  Current state-of-the-art instance segmentation methods are not suited for&#10;real-time applications like autonomous driving, which require fast execution&#10;times at high accuracy. Although the currently dominant proposal-based methods&#10;have high accuracy, they are slow and generate masks at a fixed and low&#10;resolution. Proposal-free methods, by contrast, can generate masks at high&#10;resolution and are often faster, but fail to reach the same accuracy as the&#10;proposal-based methods. In this work we propose a new clustering loss function&#10;for proposal-free instance segmentation. The loss function pulls the spatial&#10;embeddings of pixels belonging to the same instance together and jointly learns&#10;an instance-specific clustering bandwidth, maximizing the&#10;intersection-over-union of the resulting instance mask. When combined with a&#10;fast architecture, the network can perform instance segmentation in real-time&#10;while maintaining a high accuracy. We evaluate our method on the challenging&#10;Cityscapes benchmark and achieve top results (5\% improvement over Mask R-CNN)&#10;at more than 10 fps on 2MP images. Code will be available at&#10;https://github.com/davyneven/SpatialEmbeddings .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;The current best performing instance segmentation methods are proposal-based, and rely on the Faster R-CNN~\cite{ren2015faster} object detection framework, which is the current leader in most object detection benchmarks. Previous instance segmentation approaches relied on their detection output to get object proposals, which they then refine into instance masks~\cite{dai2016instance,li2016fully,pinheiro2015learning,pinheiro2016learning}. Mask R-CNN~\cite{he2017mask} and its derivative PANet~\cite{liu2018path} refine and simplify this pipeline by augmenting the Faster R-CNN network with a branch for predicting an object mask. Although they are the best-scoring methods on popular benchmarks, such as COCO, their instance masks are generated at a low resolution (32x32 pixels) and in practice are not often used in real-time applications.&#10;&#10;Another branch of instance segmentation methods rely on dense-prediction, segmentation networks to generate instance masks at input resolution. Most of these methods~\cite{fathi2017semantic,newell2017associative,kong2018recurrent,de2017semantic,novotny2018semi} are based on an embedding loss function, which forces the feature vectors of pixels belonging to the same object to be similar to each other and sufficiently dissimilar from feature vectors of pixels belonging to other objects. &#10;Recently, works~\cite{novotny2018semi,liu2018intriguing} have shown that the spatial-invariant nature of Fully Convolutional Networks is not ideal for embedding methods and propose to either incorporate coordinate maps~\cite{liu2018intriguing} or use so-called semi-convolutions~\cite{novotny2018semi} to alleviate this problem. Nevertheless, at the current time these methods still fail to achieve the same performance as the proposal-based ones.&#10;&#10;In light of this, a more promising and simple method is proposed by Kendall et al.~\cite{kendall2017multi}, inspired by~\cite{liang2015proposal}, in which they propose to assign pixels to objects by pointing to its object's center. This way, they avoid the aforementioned problem of spatial-invariance by learning position-relative offset vectors. Our method is based on the same concept, but integrates the post-processing clustering step directly into the loss function and optimizes the intersection-over-union of each object's mask directly. Related to our method is the very recent work of Novotny et al.~\cite{novotny2018semi}. Although similar in concepts, they use a different loss function and still apply a detection-first principle.&#10;&#10;Also inspired by~\cite{kendall2017multi} is Box2Pix, a work proposed by Uhrig et al.~\cite{uhrig2018box2pix}, where they first predict bounding boxes based on a single-shot detection method, and then associate pixels by pointing to object centers, which can afterwards be efficiently clustered. Its focus lays on real-time instance segmentation and shows promising results on the Cityscapes dataset. Our method also shows real-time performance on the Cityscapes dataset, but at a much higher accuracy.&#10;&#10;Our loss relaxation by learning an optimal clustering margin shows some similarites with \cite{novotny2017learning, kendall2017uncertainties}, where they integrate the aleatoric uncertainty into the loss function. In contrast to these works, we directly use the learned margin at test time.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2312.07100" label="2312.07100">
-        <attvalues>
-          <attvalue for="0" value="Lightweight high-resolution Subject Matting in the Real World" />
-          <attvalue for="1" value="  Existing saliency object detection (SOD) methods struggle to satisfy fast&#10;inference and accurate results simultaneously in high resolution scenes. They&#10;are limited by the quality of public datasets and efficient network modules for&#10;high-resolution images. To alleviate these issues, we propose to construct a&#10;saliency object matting dataset HRSOM and a lightweight network PSUNet.&#10;Considering efficient inference of mobile depolyment framework, we design a&#10;symmetric pixel shuffle module and a lightweight module TRSU. Compared to 13&#10;SOD methods, the proposed PSUNet has the best objective performance on the&#10;high-resolution benchmark dataset. Evaluation results of objective assessment&#10;are superior compared to U$^2$Net that has 10 times of parameter amount of our&#10;network. On Snapdragon 8 Gen 2 Mobile Platform, inference a single&#10;640$\times$640 image only takes 113ms. And on the subjective assessment,&#10;evaluation results are better than the industry benchmark IOS16 (Lift subject&#10;from background).&#10;" />
-          <attvalue for="2" value="&#10;SOD is a technique for automatic identification and localization of the most salient objects.&#10;SOD algorithms can be classified into traditional methods and deep learning methods.&#10;Traditional methods mainly use low-level features such as color, brightness, orientation, and contrast of an image to compute saliency maps, but they often fail to deal with complex background and multiple objects \cite{yang2013saliency,zhu2014saliency,srivatsa2015salient}.&#10;Deep learning methods utilize CNNs to learn high-level semantic features to improve performance of SOD.&#10;One of the challenges of SOD is to extract saliency maps with clear boundaries from high-resolution images while controlling the computational cost. &#10;Practical SOD algorithms need to balance between speed and accuracy, in order to adapt to various application requirements and hardware conditions.&#10;In this regard, Wu et al.\cite{wu2021mobilesal}, Lee et al.\cite{lee2021tracer}, Wang et al.\cite{wang2021raining} designed lightweight structures to accelerate inference.&#10;Liu et al.\cite{Liu2019PoolSal}, Wu et al.\cite{Wu_2022}, Qin et al.\cite{Qin_2020_PR, qin2022} used multi-scale feature fusion to improve the quality of saliency maps.&#10;Qin et al.\cite{qin2021boundary} used coarse-to-fine framework to obtain finer results. &#10;Xie et al.\cite{xie2022pyramid}, Kim et al.\cite{kim2022revisiting} designed high resolution modules to obtain accurate high resolution saliency maps.&#10;&#10;High-resolution SOD datasets \cite{7780454, xie2022pyramid, zeng2019highresolution} also contribute to the advancement of SOD algorithms.&#10;They contain images of better quality and labels of higher accuracy, compared to low-resolution SOD datasets. &#10;Most existing SOD methods can achieve good performances on public datasets, but they are unable to apply on real scenes. &#10;Firstly, most SOD datasets are low-resolution, which hinders the algorithm to process high-resolution images. Secondly, binary labels define SOD as a binary segmentation task of foreground and background. And label edges of some datasets have jaggedness, leading to difficulty for the algorithm to derive fine results. &#10;Lastly, saliency labels have large deviation from subjective perception, which is not friendly for learning. &#10;&#10;Our contributions are summarised as following.&#10;We propose a lightweight SOD network PSUNet, which is designed with full considerations of mobile adaptation to achieve a good balance between high resolution and efficiency.&#10;We propose a symmetric pixel shuffle module to effectively enhance high-resolution SOD results, at a very low cost of computation and parameter amount increment.&#10;We construct a saliency object matting dataset HRSOM consisting of 13w+ high-quality and high-resolution images, based on almost all available saliency detection datasets and self-collected datasets.&#10;We design a set of training methods and subjective and objective measures for real-world datasets, enabling the algorithm to robustly cope with open-world problems.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Computer Vision Applications, Mathematics, Artificial Intelligence, Saliency Object Detection, Computer Science, Deep Learning Networks" />
-        </attvalues>
-      </node>
-      <node id="1506.07363" label="1506.07363">
-        <attvalues>
-          <attvalue for="0" value="Salient Object Detection via Objectness Measure" />
-          <attvalue for="1" value="  Salient object detection has become an important task in many image&#10;processing applications. The existing approaches exploit background prior and&#10;contrast prior to attain state of the art results. In this paper, instead of&#10;using background cues, we estimate the foreground regions in an image using&#10;objectness proposals and utilize it to obtain smooth and accurate saliency&#10;maps. We propose a novel saliency measure called `foreground connectivity'&#10;which determines how tightly a pixel or a region is connected to the estimated&#10;foreground. We use the values assigned by this measure as foreground weights&#10;and integrate these in an optimization framework to obtain the final saliency&#10;maps. We extensively evaluate the proposed approach on two benchmark databases&#10;and demonstrate that the results obtained are better than the existing state of&#10;the art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.13095" label="2012.13095">
-        <attvalues>
-          <attvalue for="0" value="MobileSal: Extremely Efficient RGB-D Salient Object Detection" />
-          <attvalue for="1" value="  The high computational cost of neural networks has prevented recent successes&#10;in RGB-D salient object detection (SOD) from benefiting real-world&#10;applications. Hence, this paper introduces a novel network, MobileSal, which&#10;focuses on efficient RGB-D SOD using mobile networks for deep feature&#10;extraction. However, mobile networks are less powerful in feature&#10;representation than cumbersome networks. To this end, we observe that the depth&#10;information of color images can strengthen the feature representation related&#10;to SOD if leveraged properly. Therefore, we propose an implicit depth&#10;restoration (IDR) technique to strengthen the mobile networks' feature&#10;representation capability for RGB-D SOD. IDR is only adopted in the training&#10;phase and is omitted during testing, so it is computationally free. Besides, we&#10;propose compact pyramid refinement (CPR) for efficient multi-level feature&#10;aggregation to derive salient objects with clear boundaries. With IDR and CPR&#10;incorporated, MobileSal performs favorably against state-of-the-art methods on&#10;six challenging RGB-D SOD datasets with much faster speed (450fps for the input&#10;size of 320 $\times$ 320) and fewer parameters (6.5M). The code is released at&#10;https://mmcheng.net/mobilesal.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.07380" label="2112.07380">
-        <attvalues>
-          <attvalue for="0" value="TRACER: Extreme Attention Guided Salient Object Tracing Network" />
-          <attvalue for="1" value="  Existing studies on salient object detection (SOD) focus on extracting&#10;distinct objects with edge information and aggregating multi-level features to&#10;improve SOD performance. To achieve satisfactory performance, the methods&#10;employ refined edge information and low multi-level discrepancy. However, both&#10;performance gain and computational efficiency cannot be attained, which has&#10;motivated us to study the inefficiencies in existing encoder-decoder structures&#10;to avoid this trade-off. We propose TRACER, which detects salient objects with&#10;explicit edges by incorporating attention guided tracing modules. We employ a&#10;masked edge attention module at the end of the first encoder using a fast&#10;Fourier transform to propagate the refined edge information to the downstream&#10;feature extraction. In the multi-level aggregation phase, the union attention&#10;module identifies the complementary channel and important spatial information.&#10;To improve the decoder performance and computational efficiency, we minimize&#10;the decoder block usage with object attention module. This module extracts&#10;undetected objects and edge information from refined channels and spatial&#10;representations. Subsequently, we propose an adaptive pixel intensity loss&#10;function to deal with the relatively important pixels unlike conventional loss&#10;functions which treat all pixels equally. A comparison with 13 existing methods&#10;reveals that TRACER achieves state-of-the-art performance on five benchmark&#10;datasets. We have released TRACER at https://github.com/Karel911/TRACER.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.07877" label="2104.07877">
-        <attvalues>
-          <attvalue for="0" value="A De-raining semantic segmentation network for real-time foreground&#10;  segmentation" />
-          <attvalue for="1" value="  Few researches have been proposed specifically for real-time semantic&#10;segmentation in rainy environments. However, the demand in this area is huge&#10;and it is challenging for lightweight networks. Therefore, this paper proposes&#10;a lightweight network which is specially designed for the foreground&#10;segmentation in rainy environments, named De-raining Semantic Segmentation&#10;Network (DRSNet). By analyzing the characteristics of raindrops, the&#10;MultiScaleSE Block is targetedly designed to encode the input image, it uses&#10;multi-scale dilated convolutions to increase the receptive field, and SE&#10;attention mechanism to learn the weights of each channels. In order to combine&#10;semantic information between different encoder and decoder layers, it is&#10;proposed to use Asymmetric Skip, that is, the higher semantic layer of encoder&#10;employs bilinear interpolation and the output passes through pointwise&#10;convolution, then added element-wise to the lower semantic layer of decoder.&#10;According to the control experiments, the performances of MultiScaleSE Block&#10;and Asymmetric Skip compared with SEResNet18 and Symmetric Skip respectively&#10;are improved to a certain degree on the Foreground Accuracy index. The&#10;parameters and the floating point of operations (FLOPs) of DRSNet is only 0.54M&#10;and 0.20GFLOPs separately. The state-of-the-art results and real-time&#10;performances are achieved on both the UESTC all-day Scenery add rain&#10;(UAS-add-rain) and the Baidu People Segmentation add rain (BPS-add-rain)&#10;benchmarks with the input sizes of 192*128, 384*256 and 768*512. The speed of&#10;DRSNet exceeds all the networks within 1GFLOPs, and Foreground Accuracy index&#10;is also the best among the similar magnitude networks on both benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.09569" label="1904.09569">
-        <attvalues>
-          <attvalue for="0" value="A Simple Pooling-Based Design for Real-Time Salient Object Detection" />
-          <attvalue for="1" value="  We solve the problem of salient object detection by investigating how to&#10;expand the role of pooling in convolutional neural networks. Based on the&#10;U-shape architecture, we first build a global guidance module (GGM) upon the&#10;bottom-up pathway, aiming at providing layers at different feature levels the&#10;location information of potential salient objects. We further design a feature&#10;aggregation module (FAM) to make the coarse-level semantic information well&#10;fused with the fine-level features from the top-down pathway. By adding FAMs&#10;after the fusion operations in the top-down pathway, coarse-level features from&#10;the GGM can be seamlessly merged with features at various scales. These two&#10;pooling-based modules allow the high-level semantic features to be&#10;progressively refined, yielding detail enriched saliency maps. Experiment&#10;results show that our proposed approach can more accurately locate the salient&#10;objects with sharpened details and hence substantially improve the performance&#10;compared to the previous state-of-the-arts. Our approach is fast as well and&#10;can run at a speed of more than 30 FPS when processing a $300 \times 400$&#10;image. Code can be found at http://mmcheng.net/poolnet/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.12011" label="2106.12011">
-        <attvalues>
-          <attvalue for="0" value="P2T: Pyramid Pooling Transformer for Scene Understanding" />
-          <attvalue for="1" value="  Recently, the vision transformer has achieved great success by pushing the&#10;state-of-the-art of various vision tasks. One of the most challenging problems&#10;in the vision transformer is that the large sequence length of image tokens&#10;leads to high computational cost (quadratic complexity). A popular solution to&#10;this problem is to use a single pooling operation to reduce the sequence&#10;length. This paper considers how to improve existing vision transformers, where&#10;the pooled feature extracted by a single pooling operation seems less powerful.&#10;To this end, we note that pyramid pooling has been demonstrated to be effective&#10;in various vision tasks owing to its powerful ability in context abstraction.&#10;However, pyramid pooling has not been explored in backbone network design. To&#10;bridge this gap, we propose to adapt pyramid pooling to Multi-Head&#10;Self-Attention (MHSA) in the vision transformer, simultaneously reducing the&#10;sequence length and capturing powerful contextual features. Plugged with our&#10;pooling-based MHSA, we build a universal vision transformer backbone, dubbed&#10;Pyramid Pooling Transformer (P2T). Extensive experiments demonstrate that, when&#10;applied P2T as the backbone network, it shows substantial superiority in&#10;various vision tasks such as image classification, semantic segmentation,&#10;object detection, and instance segmentation, compared to previous CNN- and&#10;transformer-based networks. The code will be released at&#10;https://github.com/yuhuan-wu/P2T.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.04704" label="2101.04704">
-        <attvalues>
-          <attvalue for="0" value="Boundary-Aware Segmentation Network for Mobile and Web Applications" />
-          <attvalue for="1" value="  Although deep models have greatly improved the accuracy and robustness of&#10;image segmentation, obtaining segmentation results with highly accurate&#10;boundaries and fine structures is still a challenging problem. In this paper,&#10;we propose a simple yet powerful Boundary-Aware Segmentation Network (BASNet),&#10;which comprises a predict-refine architecture and a hybrid loss, for highly&#10;accurate image segmentation. The predict-refine architecture consists of a&#10;densely supervised encoder-decoder network and a residual refinement module,&#10;which are respectively used to predict and refine a segmentation probability&#10;map. The hybrid loss is a combination of the binary cross entropy, structural&#10;similarity and intersection-over-union losses, which guide the network to learn&#10;three-level (ie, pixel-, patch- and map- level) hierarchy representations. We&#10;evaluate our BASNet on two reverse tasks including salient object segmentation,&#10;camouflaged object segmentation, showing that it achieves very competitive&#10;performance with sharp segmentation boundaries. Importantly, BASNet runs at&#10;over 70 fps on a single GPU which benefits many potential real applications.&#10;Based on BASNet, we further developed two (close to) commercial applications:&#10;AR COPY &amp; PASTE, in which BASNet is integrated with augmented reality for&#10;&quot;COPYING&quot; and &quot;PASTING&quot; real-world objects, and OBJECT CUT, which is a&#10;web-based tool for automatic object background removal. Both applications have&#10;already drawn huge amount of attention and have important real-world impacts.&#10;The code and two applications will be publicly available at:&#10;https://github.com/NathanUA/BASNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.05041" label="2204.05041">
-        <attvalues>
-          <attvalue for="0" value="Pyramid Grafting Network for One-Stage High Resolution Saliency&#10;  Detection" />
-          <attvalue for="1" value="  Recent salient object detection (SOD) methods based on deep neural network&#10;have achieved remarkable performance. However, most of existing SOD models&#10;designed for low-resolution input perform poorly on high-resolution images due&#10;to the contradiction between the sampling depth and the receptive field size.&#10;Aiming at resolving this contradiction, we propose a novel one-stage framework&#10;called Pyramid Grafting Network (PGNet), using transformer and CNN backbone to&#10;extract features from different resolution images independently and then graft&#10;the features from transformer branch to CNN branch. An attention-based&#10;Cross-Model Grafting Module (CMGM) is proposed to enable CNN branch to combine&#10;broken detailed information more holistically, guided by different source&#10;feature during decoding process. Moreover, we design an Attention Guided Loss&#10;(AGL) to explicitly supervise the attention matrix generated by CMGM to help&#10;the network better interact with the attention from different models. We&#10;contribute a new Ultra-High-Resolution Saliency Detection dataset UHRSD,&#10;containing 5,920 images at 4K-8K resolutions. To our knowledge, it is the&#10;largest dataset in both quantity and resolution for high-resolution SOD task,&#10;which can be used for training and testing in future research. Sufficient&#10;experiments on UHRSD and widely-used SOD datasets demonstrate that our method&#10;achieves superior performance compared to the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.09475" label="2209.09475">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Image Pyramid Structure for High Resolution Salient Object&#10;  Detection" />
-          <attvalue for="1" value="  Salient object detection (SOD) has been in the spotlight recently, yet has&#10;been studied less for high-resolution (HR) images. Unfortunately, HR images and&#10;their pixel-level annotations are certainly more labor-intensive and&#10;time-consuming compared to low-resolution (LR) images and annotations.&#10;Therefore, we propose an image pyramid-based SOD framework, Inverse Saliency&#10;Pyramid Reconstruction Network (InSPyReNet), for HR prediction without any of&#10;HR datasets. We design InSPyReNet to produce a strict image pyramid structure&#10;of saliency map, which enables to ensemble multiple results with pyramid-based&#10;image blending. For HR prediction, we design a pyramid blending method which&#10;synthesizes two different image pyramids from a pair of LR and HR scale from&#10;the same image to overcome effective receptive field (ERF) discrepancy. Our&#10;extensive evaluations on public LR and HR SOD benchmarks demonstrate that&#10;InSPyReNet surpasses the State-of-the-Art (SotA) methods on various SOD metrics&#10;and boundary accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.11075" label="2207.11075">
-        <attvalues>
-          <attvalue for="0" value="RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos" />
-          <attvalue for="1" value="  Obtaining the ground truth labels from a video is challenging since the&#10;manual annotation of pixel-wise flow labels is prohibitively expensive and&#10;laborious. Besides, existing approaches try to adapt the trained model on&#10;synthetic datasets to authentic videos, which inevitably suffers from domain&#10;discrepancy and hinders the performance for real-world applications. To solve&#10;these problems, we propose RealFlow, an Expectation-Maximization based&#10;framework that can create large-scale optical flow datasets directly from any&#10;unlabeled realistic videos. Specifically, we first estimate optical flow&#10;between a pair of video frames, and then synthesize a new image from this pair&#10;based on the predicted flow. Thus the new image pairs and their corresponding&#10;flows can be regarded as a new training set. Besides, we design a Realistic&#10;Image Pair Rendering (RIPR) module that adopts softmax splatting and&#10;bi-directional hole filling techniques to alleviate the artifacts of the image&#10;synthesis. In the E-step, RIPR renders new images to create a large quantity of&#10;training data. In the M-step, we utilize the generated training data to train&#10;an optical flow network, which can be used to estimate optical flows in the&#10;next E-step. During the iterative learning steps, the capability of the flow&#10;network is gradually improved, so is the accuracy of the flow, as well as the&#10;quality of the synthesized dataset. Experimental results show that RealFlow&#10;outperforms previous dataset generation methods by a considerably large margin.&#10;Moreover, based on the generated dataset, our approach achieves&#10;state-of-the-art performance on two standard benchmarks compared with both&#10;supervised and unsupervised optical flow methods. Our code and dataset are&#10;available at https://github.com/megvii-research/RealFlow&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Optical Flow Estimation, Computer Science, Dataset Generation Methods" />
-        </attvalues>
-      </node>
-      <node id="2404.09161" label="2404.09161">
-        <attvalues>
-          <attvalue for="0" value="Coreset Selection for Object Detection" />
-          <attvalue for="1" value="  Coreset selection is a method for selecting a small, representative subset of&#10;an entire dataset. It has been primarily researched in image classification,&#10;assuming there is only one object per image. However, coreset selection for&#10;object detection is more challenging as an image can contain multiple objects.&#10;As a result, much research has yet to be done on this topic. Therefore, we&#10;introduce a new approach, Coreset Selection for Object Detection (CSOD). CSOD&#10;generates imagewise and classwise representative feature vectors for multiple&#10;objects of the same class within each image. Subsequently, we adopt submodular&#10;optimization for considering both representativeness and diversity and utilize&#10;the representative vectors in the submodular optimization process to select a&#10;subset. When we evaluated CSOD on the Pascal VOC dataset, CSOD outperformed&#10;random selection by +6.4%p in AP$_{50}$ when selecting 200 images.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In today's data-driven era, managing the sheer volume and variety of data presents a crucial challenge, particularly in areas like computer vision and deep learning, which deal with a tremendous amount of data~\cite{radford2021learning,codetr2022,cheng2021mask2former}.&#10;With the advent of technologies such as autonomous vehicles and smart surveillance systems, accurate and efficient recognition of such image data has become paramount. One key strategy in managing these massive datasets involves `coreset selection,' a method aimed at identifying a smaller, representative subset of the original dataset. This subset is then used to streamline complex computations and enhance processing efficiency.&#10;&#10;However, as illustrated in Figure \ref{fig:overview}, traditional coreset selection methods are unrealistic because they were developed under the naïve assumption of a single object per image, a condition that real-world images do not often meet~\cite{braverman2019coresets,coleman2019selection,guo2022deepcore}. Real-world images typically contain the natural variability, such as multiple objects of various categories, sizes, and locations.&#10;&#10;This implies that we should develop methods that consider the natural variability. As Figure \ref{fig:overview} illustrates, if we evaluate the suitability of an image on an object-by-object basis, considering one object as suitable for training does not necessarily imply that the others are also suitable. In other words, the&#10;decision of a core image should be based on all objects in an image.&#10;However, traditional methods, designed to solve only the single-image-single-object pairing, fail to consider this, and therefore struggle in realistic conditions.&#10;&#10;{In this paper, we address a significant limitation in the field of coreset selection which has traditionally operated under the assumption of a single object per image. We introduce a realistic approach tailored to the more complex, yet common, scenario where images inherently contain multiple objects. This shift from single-object to multi-object consideration is a central advancement of our work.}&#10;CSOD not only recognizes the presence of numerous objects within each image but also tackles the compounded uncertainties by considering objects' spatial information such as size and location. To validate our method, we implemented and conducted experiments in object detection, a representative task of situations where multiple objects may reside within a single image.&#10;&#10;Our method, CSOD, is built upon a unique concept: the `imagewise-classwise vector.' To select the most representative images among compounded uncertainties, we need a way to summarize the information of each image effectively. The imagewise-classwise vector serves this purpose by averaging the features of objects of the same class within an image. This comprehensive representation allows for informed decision-making when addressing the complexities of multi-object images.&#10;&#10;We employ a greedy approach to select individual data points sequentially by class order, thereby constructing the coreset step by step. Although this method considers only one class at each selection step, it guarantees that the most pertinent selections for each class are made, enhancing the representativeness and diversity of the coreset. Furthermore, to ensure that the selected subset informatively represents the entire dataset, we introduce a mathematical tool known as a `submodular function,' as delineated by \cite{krause2014submodular}. The function aids in selecting the most informative subset based on the imagewise-classwise average features for each category. &#10;&#10;Our empirical evaluations, particularly in scenarios involving the detection of multiple objects, demonstrate the effectiveness of CSOD. For instance, when selecting 200 images from the Pascal VOC dataset~\cite{everingham2007pascal}, our method achieved an impressive improvement of +6.4\% point in AP$_{50}$ compared to random selection. Moreover, we also evaluated it on the BDD100k~\cite{yu2020bdd100k} and MS COCO2017~\cite{lin2014microsoft} datasets and confirmed that our method outperforms random selection. These significant achievements emphasize the efficacy and innovativeness of CSOD in addressing the challenges of coreset selection in multi-object image data.&#10;&#10;{In summary, CSOD represents a pivotal extension of existing coreset selection frameworks to encompass scenarios with multiple objects per image. This approach addresses a gap that traditional methods, which assumed only one object per image, did not cover.}&#10;While our focus is on multi-objects of the same class, &#10;we acknowledge the potential for future expansions of our method to accommodate images featuring various categories of objects. &#10;With our unique problem recognition and solution, &#10;we aim to shift the paradigm of coreset selection towards more realistic scenarios, specifically image datasets containing multiple objects.&#10;This research transcends mere technical advancement, marking a pivotal shift in processing complex real-world datasets and opening new horizons for coreset selection, thereby addressing a significant challenge of the big data era.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Optimization, Machine Learning, Computer Vision, Mathematics, Object Detection, Computer Science, Statistics, Coreset Selection" />
-        </attvalues>
-      </node>
-      <node id="2103.00020" label="2103.00020">
-        <attvalues>
-          <attvalue for="0" value="Learning Transferable Visual Models From Natural Language Supervision" />
-          <attvalue for="1" value="  State-of-the-art computer vision systems are trained to predict a fixed set&#10;of predetermined object categories. This restricted form of supervision limits&#10;their generality and usability since additional labeled data is needed to&#10;specify any other visual concept. Learning directly from raw text about images&#10;is a promising alternative which leverages a much broader source of&#10;supervision. We demonstrate that the simple pre-training task of predicting&#10;which caption goes with which image is an efficient and scalable way to learn&#10;SOTA image representations from scratch on a dataset of 400 million (image,&#10;text) pairs collected from the internet. After pre-training, natural language&#10;is used to reference learned visual concepts (or describe new ones) enabling&#10;zero-shot transfer of the model to downstream tasks. We study the performance&#10;of this approach by benchmarking on over 30 different existing computer vision&#10;datasets, spanning tasks such as OCR, action recognition in videos,&#10;geo-localization, and many types of fine-grained object classification. The&#10;model transfers non-trivially to most tasks and is often competitive with a&#10;fully supervised baseline without the need for any dataset specific training.&#10;For instance, we match the accuracy of the original ResNet-50 on ImageNet&#10;zero-shot without needing to use any of the 1.28 million training examples it&#10;was trained on. We release our code and pre-trained model weights at&#10;https://github.com/OpenAI/CLIP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.12860" label="2211.12860">
-        <attvalues>
-          <attvalue for="0" value="DETRs with Collaborative Hybrid Assignments Training" />
-          <attvalue for="1" value="  In this paper, we provide the observation that too few queries assigned as&#10;positive samples in DETR with one-to-one set matching leads to sparse&#10;supervision on the encoder's output which considerably hurt the discriminative&#10;feature learning of the encoder and vice visa for attention learning in the&#10;decoder. To alleviate this, we present a novel collaborative hybrid assignments&#10;training scheme, namely $\mathcal{C}$o-DETR, to learn more efficient and&#10;effective DETR-based detectors from versatile label assignment manners. This&#10;new training scheme can easily enhance the encoder's learning ability in&#10;end-to-end detectors by training the multiple parallel auxiliary heads&#10;supervised by one-to-many label assignments such as ATSS and Faster RCNN. In&#10;addition, we conduct extra customized positive queries by extracting the&#10;positive coordinates from these auxiliary heads to improve the training&#10;efficiency of positive samples in the decoder. In inference, these auxiliary&#10;heads are discarded and thus our method introduces no additional parameters and&#10;computational cost to the original detector while requiring no hand-crafted&#10;non-maximum suppression (NMS). We conduct extensive experiments to evaluate the&#10;effectiveness of the proposed approach on DETR variants, including DAB-DETR,&#10;Deformable-DETR, and DINO-Deformable-DETR. The state-of-the-art&#10;DINO-Deformable-DETR with Swin-L can be improved from 58.5% to 59.5% AP on COCO&#10;val. Surprisingly, incorporated with ViT-L backbone, we achieve 66.0% AP on&#10;COCO test-dev and 67.9% AP on LVIS val, outperforming previous methods by clear&#10;margins with much fewer model sizes. Codes are available at&#10;\url{https://github.com/Sense-X/Co-DETR}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.01527" label="2112.01527">
-        <attvalues>
-          <attvalue for="0" value="Masked-attention Mask Transformer for Universal Image Segmentation" />
-          <attvalue for="1" value="  Image segmentation is about grouping pixels with different semantics, e.g.,&#10;category or instance membership, where each choice of semantics defines a task.&#10;While only the semantics of each task differ, current research focuses on&#10;designing specialized architectures for each task. We present Masked-attention&#10;Mask Transformer (Mask2Former), a new architecture capable of addressing any&#10;image segmentation task (panoptic, instance or semantic). Its key components&#10;include masked attention, which extracts localized features by constraining&#10;cross-attention within predicted mask regions. In addition to reducing the&#10;research effort by at least three times, it outperforms the best specialized&#10;architectures by a significant margin on four popular datasets. Most notably,&#10;Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on&#10;COCO), instance segmentation (50.1 AP on COCO) and semantic segmentation (57.7&#10;mIoU on ADE20K).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.04351" label="1903.04351">
-        <attvalues>
-          <attvalue for="0" value="Coresets for Ordered Weighted Clustering" />
-          <attvalue for="1" value="  We design coresets for Ordered k-Median, a generalization of classical&#10;clustering problems such as k-Median and k-Center, that offers a more flexible&#10;data analysis, like easily combining multiple objectives (e.g., to increase&#10;fairness or for Pareto optimization). Its objective function is defined via the&#10;Ordered Weighted Averaging (OWA) paradigm of Yager (1988), where data points&#10;are weighted according to a predefined weight vector, but in order of their&#10;contribution to the objective (distance from the centers).&#10;  A powerful data-reduction technique, called a coreset, is to summarize a&#10;point set $X$ in $\mathbb{R}^d$ into a small (weighted) point set $X'$, such&#10;that for every set of $k$ potential centers, the objective value of the coreset&#10;$X'$ approximates that of $X$ within factor $1\pm \epsilon$. When there are&#10;multiple objectives (weights), the above standard coreset might have limited&#10;usefulness, whereas in a \emph{simultaneous} coreset, which was introduced&#10;recently by Bachem and Lucic and Lattanzi (2018), the above approximation holds&#10;for all weights (in addition to all centers). Our main result is a construction&#10;of a simultaneous coreset of size $O_{\epsilon, d}(k^2 \log^2 |X|)$ for Ordered&#10;k-Median.&#10;  To validate the efficacy of our coreset construction we ran experiments on a&#10;real geographical data set. We find that our algorithm produces a small&#10;coreset, which translates to a massive speedup of clustering computations,&#10;while maintaining high accuracy for a range of weights.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11829" label="1906.11829">
-        <attvalues>
-          <attvalue for="0" value="Selection via Proxy: Efficient Data Selection for Deep Learning" />
-          <attvalue for="1" value="  Data selection methods, such as active learning and core-set selection, are&#10;useful tools for machine learning on large datasets. However, they can be&#10;prohibitively expensive to apply in deep learning because they depend on&#10;feature representations that need to be learned. In this work, we show that we&#10;can greatly improve the computational efficiency by using a small proxy model&#10;to perform data selection (e.g., selecting data points to label for active&#10;learning). By removing hidden layers from the target model, using smaller&#10;architectures, and training for fewer epochs, we create proxies that are an&#10;order of magnitude faster to train. Although these small proxy models have&#10;higher error rates, we find that they empirically provide useful signals for&#10;data selection. We evaluate this &quot;selection via proxy&quot; (SVP) approach on&#10;several data selection tasks across five datasets: CIFAR10, CIFAR100, ImageNet,&#10;Amazon Review Polarity, and Amazon Review Full. For active learning, applying&#10;SVP can give an order of magnitude improvement in data selection runtime (i.e.,&#10;the time it takes to repeatedly train and select points) without significantly&#10;increasing the final error (often within 0.1%). For core-set selection on&#10;CIFAR10, proxies that are over 10x faster to train than their larger, more&#10;accurate targets can remove up to 50% of the data without harming the final&#10;accuracy of the target, leading to a 1.6x end-to-end training time improvement.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08499" label="2204.08499">
-        <attvalues>
-          <attvalue for="0" value="DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning" />
-          <attvalue for="1" value="  Coreset selection, which aims to select a subset of the most informative&#10;training samples, is a long-standing learning problem that can benefit many&#10;downstream tasks such as data-efficient learning, continual learning, neural&#10;architecture search, active learning, etc. However, many existing coreset&#10;selection methods are not designed for deep learning, which may have high&#10;complexity and poor generalization performance. In addition, the recently&#10;proposed methods are evaluated on models, datasets, and settings of different&#10;complexities. To advance the research of coreset selection in deep learning, we&#10;contribute a comprehensive code library, namely DeepCore, and provide an&#10;empirical study on popular coreset selection methods on CIFAR10 and ImageNet&#10;datasets. Extensive experiments on CIFAR10 and ImageNet datasets verify that,&#10;although various methods have advantages in certain experiment settings, random&#10;selection is still a strong baseline.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04687" label="1805.04687">
-        <attvalues>
-          <attvalue for="0" value="BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning" />
-          <attvalue for="1" value="  Datasets drive vision progress, yet existing driving datasets are&#10;impoverished in terms of visual content and supported tasks to study multitask&#10;learning for autonomous driving. Researchers are usually constrained to study a&#10;small set of problems on one dataset, while real-world computer vision&#10;applications require performing tasks of various complexities. We construct&#10;BDD100K, the largest driving video dataset with 100K videos and 10 tasks to&#10;evaluate the exciting progress of image recognition algorithms on autonomous&#10;driving. The dataset possesses geographic, environmental, and weather&#10;diversity, which is useful for training models that are less likely to be&#10;surprised by new conditions. Based on this diverse dataset, we build a&#10;benchmark for heterogeneous multitask learning and study how to solve the tasks&#10;together. Our experiments show that special training strategies are needed for&#10;existing models to perform such heterogeneous tasks. BDD100K opens the door for&#10;future studies in this important venue.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00556" label="2104.00556">
-        <attvalues>
-          <attvalue for="0" value="Deep Two-View Structure-from-Motion Revisited" />
-          <attvalue for="1" value="  Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction&#10;and visual SLAM. Existing deep learning-based approaches formulate the problem&#10;by either recovering absolute pose scales from two consecutive frames or&#10;predicting a depth map from a single image, both of which are ill-posed&#10;problems. In contrast, we propose to revisit the problem of deep two-view SfM&#10;by leveraging the well-posedness of the classic pipeline. Our method consists&#10;of 1) an optical flow estimation network that predicts dense correspondences&#10;between two frames; 2) a normalized pose estimation module that computes&#10;relative camera poses from the 2D optical flow correspondences, and 3) a&#10;scale-invariant depth estimation network that leverages epipolar geometry to&#10;reduce the search space, refine the dense correspondences, and estimate&#10;relative depth maps. Extensive experiments show that our method outperforms all&#10;state-of-the-art two-view SfM methods by a clear margin on KITTI depth, KITTI&#10;VO, MVS, Scenes11, and SUN3D datasets in both relative pose and depth&#10;estimation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Two-view structure-from-motion (SfM) is the problem of estimating the camera motion and scene geometry from two image frames of a monocular sequence.&#10;As the foundation of both 3D reconstruction and visual simultaneous localization and mapping (vSLAM), this important problem finds its way into a wide range of applications, including autonomous driving, augmented/virtual reality, and robotics.&#10;&#10;Classic approaches to two-view SfM follow a standard pipeline of first matching features/edges between the two images, then inferring motion and geometry from those matches~\cite{davison2007pami:monoslam,klein2007ismar:ptam,mur2015orbslam,forster2014icra:svo,engel14eccv:lsdslam,engel2016dso,zhong2017self,zhong2020displacement}.&#10;When imaging conditions are well-behaved (constant lighting, diffuse and rigid surfaces, and non-repeating visual texture), the matching process is well-posed.&#10;And, once the matches have been found, the motion and geometry can be recovered.&#10;&#10;For decades, researchers who work in this area have generally required at least two views, and their methods have recovered only relative camera motion and relative scene geometry (that is, shape up to an unknown scale factor).&#10;Without a priori knowledge of scale or recognizable objects in the scene, it is impossible to recover scene geometry from a single view~\cite{hassner2006cvprw,zhong18eccvmono,xie2019iccv:pix2vox,li2019bmvc:svosr}. Similarly, it is impossible to infer absolute scale from two views of a scene~\cite{hartley2003multiple}.&#10;&#10;With the rise of deep learning, a number of researchers have recently explored neural network-based solutions to two-view SfM.&#10;Most of these methods fall into one of two categories.&#10;In the first category, which we shall call Type $\mathbf{I}$, the problem is treated as a joint optimization task of monocular depth and pose regression~\cite{sfmlearner,yin2018geonet,Ranjan_2019_CVPR}.&#10;Two networks are used: one to estimate the up-to-scale depth from a single image, and the other one to predict the up-to-scale camera pose from two input images. Both networks act independently during inference.&#10;In the second category, denoted Type $\mathbf{II}$, the scaled camera pose and the scaled depth are inferred from the image pair, and are iteratively refined via multi-view geometry~\cite{ummenhofer2017demon,tang2018ba,teed2018deepv2d}. &#10;While the power of deep learning allows both Type $\mathbf{I}$ and Type $\mathbf{II}$ solutions to achieve compelling results,&#10;we note that their formulations attempt to solve one of the ill-posed~\cite{bertero1988illposed} problems mentioned above.&#10;&#10;In this paper, we revisit the use of deep learning for two-view SfM.&#10;Our framework follows the classic SfM pipeline that features are matched between image frames to yield relative camera poses, from which relative depths are then estimated. &#10;By combining the strengths of deep learning within a classic pipeline, we are able to avoid ill-posedness, which allows our approach to achieve state-of-the-art results on several benchmarks.&#10;&#10;A comparison between our approach and existing pipelines is shown in Fig.~\ref{fig:compare_method_type}.&#10;Our method operates by first estimating dense matching points between two frames using a deep optical flow network~\cite{zhong2019unsupervised,zhong2020nipsflow}, from which a set of highly reliable matches are sampled in order to compute the relative camera pose via a GPU-accelerated classic five-point algorithm~\cite{li2006five} with RANSAC~\cite{fischler1981random}. Since these relative camera poses have scale ambiguity, the estimated depth suffers from scale ambiguity as well. Therefore, in order to supervise the estimated scale-ambiguous depth with the (scaled) ground truth depth, we propose a scale-invariant depth estimation network combined with scale-specific losses to estimate the final relative depth maps. Since the search space of the depth estimation network is reduced to epipolar lines thanks to the camera poses, it yields higher accuracy than directly triangulating the optical flows with the estimated camera poses. &#10;We demonstrate the effectiveness of our framework by achieving state-of-the-art accuracy in both pose and depth estimation on KITTI depth, KITTI VO, MVS, Scenes11, and SUN3D datasets. &#10;&#10;Our main contributions are summarized as:&#10;\begin{compactenum}[1)]&#10; \item We revisit the use of deep learning in SfM, and propose a new deep two-view SfM framework that avoids ill-posedness. Our framework combines the best of deep learning and classical geometry.&#10; \item We propose a scale-invariant depth estimation module to handle the mismatched scales between ground truth depth and the estimated depth.&#10; \item Our method outperforms all previous methods on various benchmarks for both relative pose estimation and depth estimation under the two-view SfM setting.&#10;\end{compactenum}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1607.02565" label="1607.02565">
-        <attvalues>
-          <attvalue for="0" value="Direct Sparse Odometry" />
-          <attvalue for="1" value="  We propose a novel direct sparse visual odometry formulation. It combines a&#10;fully direct probabilistic model (minimizing a photometric error) with&#10;consistent, joint optimization of all model parameters, including geometry --&#10;represented as inverse depth in a reference frame -- and camera motion. This is&#10;achieved in real time by omitting the smoothness prior used in other direct&#10;methods and instead sampling pixels evenly throughout the images. Since our&#10;method does not depend on keypoint detectors or descriptors, it can naturally&#10;sample pixels from across all image regions that have intensity gradient,&#10;including edges or smooth intensity variations on mostly white walls. The&#10;proposed model integrates a full photometric calibration, accounting for&#10;exposure time, lens vignetting, and non-linear response functions. We&#10;thoroughly evaluate our method on three different datasets comprising several&#10;hours of video. The experiments show that the presented approach significantly&#10;outperforms state-of-the-art direct and indirect methods in a variety of&#10;real-world settings, both in terms of tracking accuracy and robustness.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.00930" label="1709.00930">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Learning for Stereo Matching with Self-Improving Ability" />
-          <attvalue for="1" value="  Exiting deep-learning based dense stereo matching methods often rely on&#10;ground-truth disparity maps as the training signals, which are however not&#10;always available in many situations. In this paper, we design a simple&#10;convolutional neural network architecture that is able to learn to compute&#10;dense disparity maps directly from the stereo inputs. Training is performed in&#10;an end-to-end fashion without the need of ground-truth disparity maps. The idea&#10;is to use image warping error (instead of disparity-map residuals) as the loss&#10;function to drive the learning process, aiming to find a depth-map that&#10;minimizes the warping error. While this is a simple concept well-known in&#10;stereo matching, to make it work in a deep-learning framework, many non-trivial&#10;challenges must be overcome, and in this work we provide effective solutions.&#10;Our network is self-adaptive to different unseen imageries as well as to&#10;different camera settings. Experiments on KITTI and Middlebury stereo benchmark&#10;datasets show that our method outperforms many state-of-the-art stereo matching&#10;methods with a margin, and at the same time significantly faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.00899" label="2012.00899">
-        <attvalues>
-          <attvalue for="0" value="Displacement-Invariant Cost Computation for Efficient Stereo Matching" />
-          <attvalue for="1" value="  Although deep learning-based methods have dominated stereo matching&#10;leaderboards by yielding unprecedented disparity accuracy, their inference time&#10;is typically slow, on the order of seconds for a pair of 540p images. The main&#10;reason is that the leading methods employ time-consuming 3D convolutions&#10;applied to a 4D feature volume. A common way to speed up the computation is to&#10;downsample the feature volume, but this loses high-frequency details. To&#10;overcome these challenges, we propose a \emph{displacement-invariant cost&#10;computation module} to compute the matching costs without needing a 4D feature&#10;volume. Rather, costs are computed by applying the same 2D convolution network&#10;on each disparity-shifted feature map pair independently. Unlike previous 2D&#10;convolution-based methods that simply perform context mapping between inputs&#10;and disparity maps, our proposed approach learns to match features between the&#10;two images. We also propose an entropy-based refinement strategy to refine the&#10;computed disparity map, which further improves speed by avoiding the need to&#10;compute a second disparity map on the right image. Extensive experiments on&#10;standard datasets (SceneFlow, KITTI, ETH3D, and Middlebury) demonstrate that&#10;our method achieves competitive accuracy with much less inference time. On&#10;typical image sizes, our method processes over 100 FPS on a desktop GPU, making&#10;our method suitable for time-critical applications such as autonomous driving.&#10;We also show that our approach generalizes well to unseen datasets,&#10;outperforming 4D-volumetric methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.08690" label="1808.08690">
-        <attvalues>
-          <attvalue for="0" value="Stereo Computation for a Single Mixture Image" />
-          <attvalue for="1" value="  This paper proposes an original problem of \emph{stereo computation from a&#10;single mixture image}-- a challenging problem that had not been researched&#10;before. The goal is to separate (\ie, unmix) a single mixture image into two&#10;constitute image layers, such that the two layers form a left-right stereo&#10;image pair, from which a valid disparity map can be recovered. This is a&#10;severely illposed problem, from one input image one effectively aims to recover&#10;three (\ie, left image, right image and a disparity map). In this work we give&#10;a novel deep-learning based solution, by jointly solving the two subtasks of&#10;image layer separation as well as stereo matching. Training our deep net is a&#10;simple task, as it does not need to have disparity maps. Extensive experiments&#10;demonstrate the efficacy of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.11153" label="1901.11153">
-        <attvalues>
-          <attvalue for="0" value="Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view&#10;  Images" />
-          <attvalue for="1" value="  Recovering the 3D representation of an object from single-view or multi-view&#10;RGB images by deep neural networks has attracted increasing attention in the&#10;past few years. Several mainstream works (e.g., 3D-R2N2) use recurrent neural&#10;networks (RNNs) to fuse multiple feature maps extracted from input images&#10;sequentially. However, when given the same set of input images with different&#10;orders, RNN-based approaches are unable to produce consistent reconstruction&#10;results. Moreover, due to long-term memory loss, RNNs cannot fully exploit&#10;input images to refine reconstruction results. To solve these problems, we&#10;propose a novel framework for single-view and multi-view 3D reconstruction,&#10;named Pix2Vox. By using a well-designed encoder-decoder, it generates a coarse&#10;3D volume from each input image. Then, a context-aware fusion module is&#10;introduced to adaptively select high-quality reconstructions for each part&#10;(e.g., table legs) from different coarse 3D volumes to obtain a fused 3D&#10;volume. Finally, a refiner further refines the fused 3D volume to generate the&#10;final output. Experimental results on the ShapeNet and Pix3D benchmarks&#10;indicate that the proposed Pix2Vox outperforms state-of-the-arts by a large&#10;margin. Furthermore, the proposed method is 24 times faster than 3D-R2N2 in&#10;terms of backward inference time. The experiments on ShapeNet unseen 3D&#10;categories have shown the superior generalization abilities of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.11921" label="1811.11921">
-        <attvalues>
-          <attvalue for="0" value="Single-view Object Shape Reconstruction Using Deep Shape Prior and&#10;  Silhouette" />
-          <attvalue for="1" value="  3D shape reconstruction from a single image is a highly ill-posed problem.&#10;Modern deep learning based systems try to solve this problem by learning an&#10;end-to-end mapping from image to shape via a deep network. In this paper, we&#10;aim to solve this problem via an online optimization framework inspired by&#10;traditional methods. Our framework employs a deep autoencoder to learn a set of&#10;latent codes of 3D object shapes, which are fitted by a probabilistic shape&#10;prior using Gaussian Mixture Model (GMM). At inference, the shape and pose are&#10;jointly optimized guided by both image cues and deep shape prior without&#10;relying on an initialization from any trained deep nets. Surprisingly, our&#10;method achieves comparable performance to state-of-the-art methods even without&#10;training an end-to-end network, which shows a promising step in this direction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;3D understanding of object shapes from images is considered an important step in scene perception. While mapping an environment using Structure from Motion \cite{tomasi1992shape,agarwal2009building} and SLAM \cite{davison2007monoslam,newcombe2011dtam} facilitates localization and navigation, a higher level of understanding about objects in terms of their shape and relative position with respect to the rest of the background is essential for their manipulation. Initial works, to localize objects while estimating its pose from an image, are limited to the case where a pre-scanned object structure is available \cite{rosenhahn2007three,gall2007clustered,schmaltz2007occlusion}.&#10;&#10;Soon it is realized that the valid 3D shapes of objects belonging to a specific category are highly correlated and dimensionality reduction emerges as a prominent tool to model object shapes. &#10;Works like \cite{vicente2014reconstructing} reconstruct traditional image segmentation datasets like PASCAL VOC \cite{everingham2010pascal} by extending the ideas of non-rigid Structure from Motion \cite{bregler2000recovering,brand2001morphable,dai2014simple,garg2013dense}.&#10;Methods like \cite{kar2015category} propose to use these learned category specific object shape manifolds to reconstruct shape of the object from single image by fitting the reconstructed shape to a single image silhouette and refine it with simple image based methods like shape from shading \cite{horn1989obtaining}. &#10;&#10;While the methods mentioned above attempt to learn linear object shape manifolds from 2D key-point correspondences or object segmentation annotations, increasing availability of class specific 3D CAD models and object scans allow researchers to learn complex manifolds for shapes within an object category directly from data. For example, Gaussian Process Latent Variable Model or Kernel/simple Principal Component Analysis is employed to learn a compact latent space of object shapes in \cite{dambreville2008framework,prisacariu2012simultaneous,dame2013dense,engelmann2016joint}. &#10;Taking advantage of modern deep learning techniques, deep auto-encoders \cite{hinton2006reducing,vincent2008extracting}, VAEs \cite{kingma2013auto}, and GANs \cite{goodfellow2014generative,wu2016learning} can be better alternatives to model object shapes than traditional dimensionality reduction tools. &#10;&#10;Recently, with the advance of deep learning, there are many end-to-end systems proposed to solve the object shape reconstruction problem from a single RGB image directly. Different supervision has been explored. Direct 3D supervision is dominant in the early works \cite{choy20163d,girdhar2016learning,fan2017point,wu2017marrnet,groueix2018atlasnet}. Weaker supervision that uses geometry projection (i.e.\ depth consistency, silhouette consistency) is later employed in \cite{tulsiani2017multi,tulsiani2018multi,insafutdinov18pointclouds,lin2018learning,li2018efficient,zhu2017rethinking,wiles2017silnet,henderson2018learning,yan2016perspective} as one step moving toward training the network using real images. However, all multiple views of an object are required to train these networks, and they still rely on a one-shot prediction from the network at inference time.&#10;&#10;Most related to our work are CodeSLAM \cite{bloesch2018codeslam}, MarrNet \cite{wu2017marrnet}, and Zhu \etal \cite{zhu2018object}, which also use a low-dimensional latent space learnt by a deep network to represent the geometry information. For the scene-level geometry reconstruction, CodeSLAM uses nearby views to form a photometric loss, which is minimized by searching in the learned latent space of depth maps. MarrNet \cite{wu20183d} and Zhu \etal \cite{zhu2018object} enforce the 2.5D (silhouette, depth or normal) - 3D shape constraint and photometric loss respectively to search in the latent space of object shapes. Our work differs from theirs in twofold. Firstly, a common disadvantage of these works, which we address in our work, is that they do not consider a probabilistic prior in the latent space explicitly when they perform optimization, which we show is essential for the optimization at inference time through our ablation study. Secondly, we do not need to train a deep network to initialize optimization variables (i.e.\ shape latent code and pose)&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1805.09806" label="1805.09806">
-        <attvalues>
-          <attvalue for="0" value="Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera&#10;  Motion, Optical Flow and Motion Segmentation" />
-          <attvalue for="1" value="  We address the unsupervised learning of several interconnected problems in&#10;low-level vision: single view depth prediction, camera motion estimation,&#10;optical flow, and segmentation of a video into the static scene and moving&#10;regions. Our key insight is that these four fundamental vision problems are&#10;coupled through geometric constraints. Consequently, learning to solve them&#10;together simplifies the problem because the solutions can reinforce each other.&#10;We go beyond previous work by exploiting geometry more explicitly and&#10;segmenting the scene into static and moving regions. To that end, we introduce&#10;Competitive Collaboration, a framework that facilitates the coordinated&#10;training of multiple specialized neural networks to solve complex problems.&#10;Competitive Collaboration works much like expectation-maximization, but with&#10;neural networks that act as both competitors to explain pixels that correspond&#10;to static or moving regions, and as collaborators through a moderator that&#10;assigns pixels to be either static or independently moving. Our novel method&#10;integrates all these problems in a common framework and simultaneously reasons&#10;about the segmentation of the scene into moving objects and the static&#10;background, the camera motion, depth of the static scene structure, and the&#10;optical flow of moving objects. Our model is trained without any supervision&#10;and achieves state-of-the-art performance among joint unsupervised methods on&#10;all sub-problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.02401" label="1612.02401">
-        <attvalues>
-          <attvalue for="0" value="DeMoN: Depth and Motion Network for Learning Monocular Stereo" />
-          <attvalue for="1" value="  In this paper we formulate structure from motion as a learning problem. We&#10;train a convolutional network end-to-end to compute depth and camera motion&#10;from successive, unconstrained image pairs. The architecture is composed of&#10;multiple stacked encoder-decoder networks, the core part being an iterative&#10;network that is able to improve its own predictions. The network estimates not&#10;only depth and motion, but additionally surface normals, optical flow between&#10;the images and confidence of the matching. A crucial component of the approach&#10;is a training loss based on spatial relative differences. Compared to&#10;traditional two-frame structure from motion methods, results are more accurate&#10;and more robust. In contrast to the popular depth-from-single-image networks,&#10;DeMoN learns the concept of matching and, thus, better generalizes to&#10;structures not seen during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.04807" label="1806.04807">
-        <attvalues>
-          <attvalue for="0" value="BA-Net: Dense Bundle Adjustment Network" />
-          <attvalue for="1" value="  This paper introduces a network architecture to solve the&#10;structure-from-motion (SfM) problem via feature-metric bundle adjustment (BA),&#10;which explicitly enforces multi-view geometry constraints in the form of&#10;feature-metric error. The whole pipeline is differentiable so that the network&#10;can learn suitable features that make the BA problem more tractable.&#10;Furthermore, this work introduces a novel depth parameterization to recover&#10;dense per-pixel depth. The network first generates several basis depth maps&#10;according to the input image and optimizes the final depth as a linear&#10;combination of these basis depth maps via feature-metric BA. The basis depth&#10;maps generator is also learned via end-to-end training. The whole system nicely&#10;combines domain knowledge (i.e. hard-coded multi-view geometry constraints) and&#10;deep learning (i.e. feature learning and basis depth maps learning) to address&#10;the challenging dense SfM problem. Experiments on large scale real data prove&#10;the success of the proposed method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.04605" label="1812.04605">
-        <attvalues>
-          <attvalue for="0" value="DeepV2D: Video to Depth with Differentiable Structure from Motion" />
-          <attvalue for="1" value="  We propose DeepV2D, an end-to-end deep learning architecture for predicting&#10;depth from video. DeepV2D combines the representation ability of neural&#10;networks with the geometric principles governing image formation. We compose a&#10;collection of classical geometric algorithms, which are converted into&#10;trainable modules and combined into an end-to-end differentiable architecture.&#10;DeepV2D interleaves two stages: motion estimation and depth estimation. During&#10;inference, motion and depth estimation are alternated and converge to accurate&#10;depth. Code is available https://github.com/princeton-vl/DeepV2D.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03848" label="1904.03848">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes" />
-          <attvalue for="1" value="  Unsupervised deep learning for optical flow computation has achieved&#10;promising results. Most existing deep-net based methods rely on image&#10;brightness consistency and local smoothness constraint to train the networks.&#10;Their performance degrades at regions where repetitive textures or occlusions&#10;occur. In this paper, we propose Deep Epipolar Flow, an unsupervised optical&#10;flow method which incorporates global geometric constraints into network&#10;learning. In particular, we investigate multiple ways of enforcing the epipolar&#10;constraint in flow estimation. To alleviate a &quot;chicken-and-egg&quot; type of problem&#10;encountered in dynamic scenes where multiple motions may be present, we propose&#10;a low-rank constraint as well as a union-of-subspaces constraint for training.&#10;Experimental results on various benchmarking datasets show that our method&#10;achieves competitive performance compared with supervised methods and&#10;outperforms state-of-the-art unsupervised deep-learning methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.14851" label="2010.14851">
-        <attvalues>
-          <attvalue for="0" value="Displacement-Invariant Matching Cost Learning for Accurate Optical Flow&#10;  Estimation" />
-          <attvalue for="1" value="  Learning matching costs has been shown to be critical to the success of the&#10;state-of-the-art deep stereo matching methods, in which 3D convolutions are&#10;applied on a 4D feature volume to learn a 3D cost volume. However, this&#10;mechanism has never been employed for the optical flow task. This is mainly due&#10;to the significantly increased search dimension in the case of optical flow&#10;computation, ie, a straightforward extension would require dense 4D&#10;convolutions in order to process a 5D feature volume, which is computationally&#10;prohibitive. This paper proposes a novel solution that is able to bypass the&#10;requirement of building a 5D feature volume while still allowing the network to&#10;learn suitable matching costs from data. Our key innovation is to decouple the&#10;connection between 2D displacements and learn the matching costs at each 2D&#10;displacement hypothesis independently, ie, displacement-invariant cost&#10;learning. Specifically, we apply the same 2D convolution-based matching net&#10;independently on each 2D displacement hypothesis to learn a 4D cost volume.&#10;Moreover, we propose a displacement-aware projection layer to scale the learned&#10;cost volume, which reconsiders the correlation between different displacement&#10;candidates and mitigates the multi-modal problem in the learned cost volume.&#10;The cost volume is then projected to optical flow estimation through a 2D&#10;soft-argmin layer. Extensive experiments show that our approach achieves&#10;state-of-the-art accuracy on various datasets, and outperforms all published&#10;optical flow methods on the Sintel benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.05423" label="2103.05423">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning Based 3D Segmentation: A Survey" />
-          <attvalue for="1" value="  3D segmentation is a fundamental and challenging problem in computer vision&#10;with applications in autonomous driving and robotics. It has received&#10;significant attention from the computer vision, graphics and machine learning&#10;communities. Conventional methods for 3D segmentation, based on hand-crafted&#10;features and machine learning classifiers, lack generalization ability. Driven&#10;by their success in 2D computer vision, deep learning techniques have recently&#10;become the tool of choice for 3D segmentation tasks. This has led to an influx&#10;of many methods in the literature that have been evaluated on different&#10;benchmark datasets. Whereas survey papers on RGB-D and point cloud segmentation&#10;exist, there is a lack of a recent in-depth survey that covers all 3D data&#10;modalities and application domains. This paper fills the gap and&#10;comprehensively surveys the recent progress in deep learning-based 3D&#10;segmentation techniques. We cover over 220 works from the last six years,&#10;analyze their strengths and limitations, and discuss their competitive results&#10;on benchmark datasets. The survey provides a summary of the most commonly used&#10;pipelines and finally highlights promising research directions for the future.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Segmentation of 3D scenes is a fundamental and challenging problem in computer vision as well as computer graphics. The objective of 3D segmentation is to build computational techniques that predict the fine-grained labels of objects in a 3D scene for a wide range of applications,&#10;such as autonomous driving, mobile robots, industrial control, and augmented and virtual reality. As illustrated in Figure~\ref{dataandsegmentation}, 3D segmentation can be divided into semantic, instance and part segmentation. {Semantic segmentation aims to predict object class labels such as tables and chairs. Instance segmentation additionally distinguishes between different instances of the same class labels, e.g., chair one and two. Part segmentation aims to further decompose instances into various components, such as the same chair's armrests, legs, and backrests.} Semantic, instance, and part segmentation tasks show a progressive relationship in terms of semantic levels. Despite the difference in specific goals of three segmentation tasks, these tasks share a broader, unifying goal: they aim to segment 3D data into semantically meaningful regions based on certain criteria-whether it's by class,instance, part.&#10;&#10;Compared to conventional single view 2D segmentation, 3D segmentation gives a more comprehensive understanding of a scene since 3D data (e.g., RGB-D, point cloud, voxel, mesh, 3D video) contain richer geometric, shape, and scale information with less background noise. Moreover, the representation of 3D data, for example, in the form of projected images, has more semantic information.&#10;&#10;Recently, deep learning techniques have dominated many research areas, including computer vision and natural language processing. Motivated by its success in learning powerful features, deep learning for 3D segmentation has also attracted growing interest from the research community over the past decade. However, 3D deep learning methods still face many unsolved challenges. For example, the irregularity of point clouds makes it difficult to exploit local features, and converting them to high-resolution voxels has a substantial computational burden.&#10;&#10;This paper comprehensively surveys recent progress in deep learning methods for 3D segmentation. It focuses on analyzing commonly used building blocks, convolutional kernels and complete architectures, pointing out the pros and cons in each case. The survey covers over 180 representative articles published in the last six years. Although some notable 3D segmentation surveys have been released including RGB-D semantic segmentation~\cite{fooladgar2020survey}, remote sensing imagery segmentation~\cite{yuan2021review}, point clouds segmentation~\cite{xie2020linking},~\cite{guo2020deep},~\cite{liu2019deep},~\cite{bello2020deep},~\cite{naseer2018indoor},~\cite{ioannidou2017deep}, these surveys do not comprehensively cover all 3D data types and typical application domains. Most importantly, these surveys do not focus on 3D segmentation but give a general survey of deep learning from point clouds~\cite{guo2020deep},~\cite{liu2019deep},~\cite{bello2020deep},~\cite{naseer2018indoor},~\cite{ioannidou2017deep}.&#10;Given the differences in domain knowledge required for semantic, instance, and part segmentation tasks in 3D segmentation, this paper reviews the deep learning techniques for each of these three segmentation tasks separately.&#10;Figure~\ref{overview} shows the overview of the whole article. The contributions of this paper are summarized as follows:&#10;\begin{itemize}&#10;&#10;\item To the best of our knowledge, this is the first survey paper to comprehensively cover deep learning methods on 3D segmentation {in computer vision}, covering the most common 3D data representations, including RGB-D, projected images, voxels, point clouds, meshes, and 3D videos.&#10;&#10;\item This survey provides an in-depth analysis of the relative advantages and disadvantages of different 3D data segmentation methods. Unlike existing reviews, this survey focuses on deep learning methods designed specifically for 3D segmentation and discusses typical segmentation pipelines.&#10;&#10;\item Finally, this survey provides comprehensive comparisons of existing methods on several public benchmark 3D datasets, draws interesting conclusions, and identifies promising future research directions.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1908.08854" label="1908.08854">
-        <attvalues>
-          <attvalue for="0" value="Linking Points With Labels in 3D: A Review of Point Cloud Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  3D Point Cloud Semantic Segmentation (PCSS) is attracting increasing&#10;interest, due to its applicability in remote sensing, computer vision and&#10;robotics, and due to the new possibilities offered by deep learning techniques.&#10;In order to provide a needed up-to-date review of recent developments in PCSS,&#10;this article summarizes existing studies on this topic. Firstly, we outline the&#10;acquisition and evolution of the 3D point cloud from the perspective of remote&#10;sensing and computer vision, as well as the published benchmarks for PCSS&#10;studies. Then, traditional and advanced techniques used for Point Cloud&#10;Segmentation (PCS) and PCSS are reviewed and compared. Finally, important&#10;issues and open questions in PCSS studies are discussed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.12033" label="1912.12033">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning for 3D Point Clouds: A Survey" />
-          <attvalue for="1" value="  Point cloud learning has lately attracted increasing attention due to its&#10;wide applications in many areas, such as computer vision, autonomous driving,&#10;and robotics. As a dominating technique in AI, deep learning has been&#10;successfully used to solve various 2D vision problems. However, deep learning&#10;on point clouds is still in its infancy due to the unique challenges faced by&#10;the processing of point clouds with deep neural networks. Recently, deep&#10;learning on point clouds has become even thriving, with numerous methods being&#10;proposed to address different problems in this area. To stimulate future&#10;research, this paper presents a comprehensive review of recent progress in deep&#10;learning methods for point clouds. It covers three major tasks, including 3D&#10;shape classification, 3D object detection and tracking, and 3D point cloud&#10;segmentation. It also presents comparative results on several publicly&#10;available datasets, together with insightful observations and inspiring future&#10;research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.03352" label="1803.03352">
-        <attvalues>
-          <attvalue for="0" value="Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey" />
-          <attvalue for="1" value="  With the availability of low-cost and compact 2.5/3D visual sensing devices,&#10;computer vision community is experiencing a growing interest in visual scene&#10;understanding of indoor environments. This survey paper provides a&#10;comprehensive background to this research topic. We begin with a historical&#10;perspective, followed by popular 3D data representations and a comparative&#10;analysis of available datasets. Before delving into the application specific&#10;details, this survey provides a succinct introduction to the core technologies&#10;that are the underlying methods extensively used in the literature. Afterwards,&#10;we review the developed techniques according to a taxonomy based on the scene&#10;understanding tasks. This covers holistic indoor scene understanding as well as&#10;subtasks such as scene classification, object detection, pose estimation,&#10;semantic segmentation, 3D reconstruction, saliency detection, physics-based&#10;reasoning and affordance prediction. Later on, we summarize the performance&#10;metrics used for evaluation in different tasks and a quantitative comparison&#10;among the recent state-of-the-art techniques. We conclude this review with the&#10;current challenges and an outlook towards the open research problems requiring&#10;further investigation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.01828" label="2306.01828">
-        <attvalues>
-          <attvalue for="0" value="Unifying (Machine) Vision via Counterfactual World Modeling" />
-          <attvalue for="1" value="  Leading approaches in machine vision employ different architectures for&#10;different tasks, trained on costly task-specific labeled datasets. This&#10;complexity has held back progress in areas, such as robotics, where robust&#10;task-general perception remains a bottleneck. In contrast, &quot;foundation models&quot;&#10;of natural language have shown how large pre-trained neural networks can&#10;provide zero-shot solutions to a broad spectrum of apparently distinct tasks.&#10;Here we introduce Counterfactual World Modeling (CWM), a framework for&#10;constructing a visual foundation model: a unified, unsupervised network that&#10;can be prompted to perform a wide variety of visual computations. CWM has two&#10;key components, which resolve the core issues that have hindered application of&#10;the foundation model concept to vision. The first is structured masking, a&#10;generalization of masked prediction methods that encourages a prediction model&#10;to capture the low-dimensional structure in visual data. The model thereby&#10;factors the key physical components of a scene and exposes an interface to them&#10;via small sets of visual tokens. This in turn enables CWM's second main idea --&#10;counterfactual prompting -- the observation that many apparently distinct&#10;visual representations can be computed, in a zero-shot manner, by comparing the&#10;prediction model's output on real inputs versus slightly modified&#10;(&quot;counterfactual&quot;) inputs. We show that CWM generates high-quality readouts on&#10;real-world images and videos for a diversity of tasks, including estimation of&#10;keypoints, optical flow, occlusions, object segments, and relative depth. Taken&#10;together, our results show that CWM is a promising path to unifying the&#10;manifold strands of machine vision in a conceptually simple foundation.&#10;" />
-          <attvalue for="2" value="&#10;Within their first years of life, humans acquire an unparalleled ability to understand visual scenes and make predictions about them: what objects are present, how they are arranged in space, where they will (or could) move, and how they might interact.&#10;These abilities in turn support a panoply of complex behaviors, such as object manipulation and tool use, long-range physical prediction and planning, spatial navigation, language grounding, and social interaction. &#10;&#10;Despite great recent progress in modeling high-level cognitive functions, such as language understanding and generation, even today's large-scale AI algorithms fall far short of human-like visual scene-understanding.&#10;This has held back artificial systems, such as robots, that would profitably use such abilities for model-predictive control and planning.&#10;Importantly, because they arise so early in human development -- and are shared in part by other animals, like non-human primates -- visual scene understanding and world-modeling abilities almost certainly do not require language learning or supervision.&#10;This suggests that bridging the machine-human gap in vision (or pre-linguistic sensorimotor abilities more broadly) may require more powerful and efficient new forms of unsupervised learning.&#10;&#10;One appealing approach, inspired by vision science and visual systems neuroscience, is to break the problem of scene understanding down into sub-tasks that can be solved with special-purpose modules.&#10;The human visual system extracts a wide range of low- and mid-level representations of scenes, including (among many others) contours and border ownership~\cite{zhou2000coding}, optical flow and self-induced motion~\cite{beauchemin1995computation}, perceptual groups and figure-ground segmentations~\cite{lamme1995neurophysiology, treisman1982perceptual}, a ``2.5D sketch'' of depth and surface normal vectors~\cite{marr2010vision}, and estimates of 3D object shape~\cite{todd2004visual}.&#10;Individual computer vision algorithms successfully model some of these ``intermediate'' computations, but most appear to be well below human level and fail to capture more complex visual behaviors~\cite{bear2021physion}, raising deep questions about this agenda: &#10;which intermediates are necessary and sufficient for holistic scene understanding?&#10;How can better algorithms be found without hard-to-obtain labeled data?&#10;Are improvements in one module useful for improving others?&#10;Do the many intermediates need to be assembled carefully into a unified architecture, or can a generic procedure discover automatically how to make use of them?&#10;&#10;A promising alternative may dispense with these challenges.&#10;Recently, the ``foundation model'' approach~\cite{bommasani2021opportunities} has driven rapid progress in natural language generation and understanding tasks, without breaking them down into pieces that require special methods.&#10;At a high level, this approach has two key components:&#10;\begin{itemize}&#10; \item A pretrained model, typically a large, relatively unspecialized neural network, trained to solve a ``masked token prediction'' task on a large real-world dataset; and, &#10; \item A generic task interface, a simple process for translating any task within a wide domain into a zero- (or few-)shot input to the pretrained model. &#10;\end{itemize}&#10;&#10;In the case of language, simple and powerful versions of both of these ideas have been discovered. &#10;Effective large language models, like GPT-n~\cite{radford2019language}, are trained on large corpuses of web text to take in a context string and predict the likely next word. &#10;The next-word prediction task is straightforward to formulate (if not to solve) and remains unsurpassed, despite attempts to identify more powerful training methods.&#10;Moreover, the form of this training immediately yields a universal interface: &#10;a well-trained model prompted with a natural language description of a task -- that is, provided with this description as its context string -- must provide an attempted solution, its prediction of what text would naturally follow the prompt.&#10;Thus language foundation models have gained widespread success from their remarkable simplicity, compared to the wiring together of specialized subsystems that predominates in computer vision.&#10;&#10;But applying the foundation model framework to vision has not been as straightforward.&#10;Both key ingredients are harder to formulate and successfully implement: &#10;masked prediction in vision, in the guise of future prediction of video frames~\cite{voleti2022masked, yan2022temporally} (or meaningful image-derived latents~\cite{nair2022r3m}) from previous context frames (or latents), has remained a largely unsolved problem~\cite{bear2021physion};&#10;and it is not at all clear how to elicit most of the ``intermediate'' computations above through an interface to a single vision model.&#10;For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments?&#10;Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}.&#10;They do not explain, therefore, how to derive a task-general interface from a single unsupervised training process, which we take to be the kernel of the foundation model concept.&#10;&#10;Here, we propose to resolve these issues with an approach we call Counterfactual World Modeling (CWM). &#10;The CWM framework has two basic concepts, which build on the two fundamental components of foundation models:&#10;\begin{enumerate}&#10; \item [Pretraining via structured masked prediction.] The recent Masked Autoencoder models~\cite{he2022masked} directly translate the masked prediction task from text to visual data (images and video); however, these models do not appear to learn representations as general as language models.&#10; In stark contrast to the language domain, we find that simple but nontrivial variations of the masks used to train vision models produce highly distinct learned representations. &#10; In particular, we show that a temporally-factored masking structure teaches models to separate appearance information from dynamics information. &#10; The resulting prediction model encodes powerful implicit information about the physical structure of scenes, objects, and their dynamics.&#10; \item [Generic task interface via counterfactual prompting.] Different visual representations do not, at first blush, appear to have much in common: state-of-the-art computer vision uses quite distinct methods to extract, e.g., keypoints, optical flow, and object segments, with few obvious relationships between them. Yet we show that many such concepts arise from the above prediction model via counterfactual prompting: by systematically modifying visual inputs in simple ways and inspecting the perturbed model responses, core computer vision concepts can be extracted in a zero-shot manner. For example, extracting optical flow is equivalent to asking for a prediction of where a hypothetical visual marker would move from one frame to the next. Moreover, we show that these counterfactuals can be mathematically formalized as derivatives of the underlying prediction model itself, which provides a way both to identify ``natural'' counterfactual prompts and to implement them efficiently with autograd methods. Figure \ref{fig:unified} illustrates the use of a Counterfactual World Model, $\Psi$, in both generating counterfactual simulations and predicting properties of the true visual input.&#10;\end{enumerate}&#10;In this work we show how basic application of Counterfactual World Modeling can construct a variety of visual concepts, including keypoints, optical flow and occlusion, object segmentation, and monocular depth. &#10;Several of these constructions involve ``bootstrapping,'' using one extracted representation to help extract another.&#10;Bootstrapping is simple in the CWM framework because it is effectively an application of the chain rule; for example, we illustrate below how object segmentation can be considered the derivative of a model that estimates optical flow, which in turn is the derivative of a temporally-factored masked prediction model.&#10;&#10;Our results illustrate how a high-performing, ``true'' foundation model can be formalized and trained on unlabeled visual data.&#10;The CWM approach thus synthesizes a plethora of classical computer vision concepts within the modern foundation model framework.&#10;Because CWM does not require supervision or specialized architectures for different sub-tasks, it offers a practical solution for many visual problems and -- for the first time -- a unified computational theory of how these abilities could arise in a biological system.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Visual Computing, Physics, Foundation Models, Computer Science, Machine Vision, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2106.08261" label="2106.08261">
-        <attvalues>
-          <attvalue for="0" value="Physion: Evaluating Physical Prediction from Vision in Humans and&#10;  Machines" />
-          <attvalue for="1" value="  While current vision algorithms excel at many challenging tasks, it is&#10;unclear how well they understand the physical dynamics of real-world&#10;environments. Here we introduce Physion, a dataset and benchmark for rigorously&#10;evaluating the ability to predict how physical scenarios will evolve over time.&#10;Our dataset features realistic simulations of a wide range of physical&#10;phenomena, including rigid and soft-body collisions, stable multi-object&#10;configurations, rolling, sliding, and projectile motion, thus providing a more&#10;comprehensive challenge than previous benchmarks. We used Physion to benchmark&#10;a suite of models varying in their architecture, learning objective,&#10;input-output structure, and training data. In parallel, we obtained precise&#10;measurements of human prediction behavior on the same set of scenarios,&#10;allowing us to directly evaluate how well any model could approximate human&#10;behavior. We found that vision algorithms that learn object-centric&#10;representations generally outperform those that do not, yet still fall far&#10;short of human performance. On the other hand, graph neural networks with&#10;direct access to physical state information both perform substantially better&#10;and make predictions that are more similar to those made by humans. These&#10;results suggest that extracting physical representations of scenes is the main&#10;bottleneck to achieving human-level and human-like physical understanding in&#10;vision algorithms. We have publicly released all data and code to facilitate&#10;the use of Physion to benchmark additional models in a fully reproducible&#10;manner, enabling systematic evaluation of progress towards vision algorithms&#10;that understand physical environments as robustly as people do.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.07258" label="2108.07258">
-        <attvalues>
-          <attvalue for="0" value="On the Opportunities and Risks of Foundation Models" />
-          <attvalue for="1" value="  AI is undergoing a paradigm shift with the rise of models (e.g., BERT,&#10;DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a&#10;wide range of downstream tasks. We call these models foundation models to&#10;underscore their critically central yet incomplete character. This report&#10;provides a thorough account of the opportunities and risks of foundation&#10;models, ranging from their capabilities (e.g., language, vision, robotics,&#10;reasoning, human interaction) and technical principles(e.g., model&#10;architectures, training procedures, data, systems, security, evaluation,&#10;theory) to their applications (e.g., law, healthcare, education) and societal&#10;impact (e.g., inequity, misuse, economic and environmental impact, legal and&#10;ethical considerations). Though foundation models are based on standard deep&#10;learning and transfer learning, their scale results in new emergent&#10;capabilities,and their effectiveness across so many tasks incentivizes&#10;homogenization. Homogenization provides powerful leverage but demands caution,&#10;as the defects of the foundation model are inherited by all the adapted models&#10;downstream. Despite the impending widespread deployment of foundation models,&#10;we currently lack a clear understanding of how they work, when they fail, and&#10;what they are even capable of due to their emergent properties. To tackle these&#10;questions, we believe much of the critical research on foundation models will&#10;require deep interdisciplinary collaboration commensurate with their&#10;fundamentally sociotechnical nature.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.12601" label="2203.12601">
-        <attvalues>
-          <attvalue for="0" value="R3M: A Universal Visual Representation for Robot Manipulation" />
-          <attvalue for="1" value="  We study how visual representations pre-trained on diverse human video data&#10;can enable data-efficient learning of downstream robotic manipulation tasks.&#10;Concretely, we pre-train a visual representation using the Ego4D human video&#10;dataset using a combination of time-contrastive learning, video-language&#10;alignment, and an L1 penalty to encourage sparse and compact representations.&#10;The resulting representation, R3M, can be used as a frozen perception module&#10;for downstream policy learning. Across a suite of 12 simulated robot&#10;manipulation tasks, we find that R3M improves task success by over 20% compared&#10;to training from scratch and by over 10% compared to state-of-the-art visual&#10;representations like CLIP and MoCo. Furthermore, R3M enables a Franka Emika&#10;Panda arm to learn a range of manipulation tasks in a real, cluttered apartment&#10;given just 20 demonstrations. Code and pre-trained models are available at&#10;https://tinyurl.com/robotr3m.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.08916" label="2206.08916">
-        <attvalues>
-          <attvalue for="0" value="Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks" />
-          <attvalue for="1" value="  We propose Unified-IO, a model that performs a large variety of AI tasks&#10;spanning classical computer vision tasks, including pose estimation, object&#10;detection, depth estimation and image generation, vision-and-language tasks&#10;such as region captioning and referring expression, to natural language&#10;processing tasks such as question answering and paraphrasing. Developing a&#10;single unified model for such a large variety of tasks poses unique challenges&#10;due to the heterogeneous inputs and outputs pertaining to each task, including&#10;RGB images, per-pixel maps, binary masks, bounding boxes, and language. We&#10;achieve this unification by homogenizing every supported input and output into&#10;a sequence of discrete vocabulary tokens. This common representation across all&#10;tasks allows us to train a single transformer-based architecture, jointly on&#10;over 90 diverse datasets in the vision and language fields. Unified-IO is the&#10;first model capable of performing all 7 tasks on the GRIT benchmark and&#10;produces strong results across 16 diverse benchmarks like NYUv2-Depth,&#10;ImageNet, VQA2.0, OK-VQA, Swig, VizWizGround, BoolQ, and SciTail, with no&#10;task-specific fine-tuning. Code and demos for Unified-IO are available at:&#10;https://unified-io.allenai.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.07669" label="2206.07669">
-        <attvalues>
-          <attvalue for="0" value="A Unified Sequence Interface for Vision Tasks" />
-          <attvalue for="1" value="  While language tasks are naturally expressed in a single, unified, modeling&#10;framework, i.e., generating sequences of tokens, this has not been the case in&#10;computer vision. As a result, there is a proliferation of distinct&#10;architectures and loss functions for different vision tasks. In this work we&#10;show that a diverse set of &quot;core&quot; computer vision tasks can also be unified if&#10;formulated in terms of a shared pixel-to-sequence interface. We focus on four&#10;tasks, namely, object detection, instance segmentation, keypoint detection, and&#10;image captioning, all with diverse types of outputs, e.g., bounding boxes or&#10;dense masks. Despite that, by formulating the output of each task as a sequence&#10;of discrete tokens with a unified interface, we show that one can train a&#10;neural network with a single model architecture and loss function on all these&#10;tasks, with no task-specific customization. To solve a specific task, we use a&#10;short prompt as task description, and the sequence output adapts to the prompt&#10;so it can produce task-specific output. We show that such a model can achieve&#10;competitive performance compared to well-established task-specific models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.04671" label="2303.04671">
-        <attvalues>
-          <attvalue for="0" value="Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation&#10;  Models" />
-          <attvalue for="1" value="  ChatGPT is attracting a cross-field interest as it provides a language&#10;interface with remarkable conversational competency and reasoning capabilities&#10;across many domains. However, since ChatGPT is trained with languages, it is&#10;currently not capable of processing or generating images from the visual world.&#10;At the same time, Visual Foundation Models, such as Visual Transformers or&#10;Stable Diffusion, although showing great visual understanding and generation&#10;capabilities, they are only experts on specific tasks with one-round fixed&#10;inputs and outputs. To this end, We build a system called \textbf{Visual&#10;ChatGPT}, incorporating different Visual Foundation Models, to enable the user&#10;to interact with ChatGPT by 1) sending and receiving not only languages but&#10;also images 2) providing complex visual questions or visual editing&#10;instructions that require the collaboration of multiple AI models with&#10;multi-steps. 3) providing feedback and asking for corrected results. We design&#10;a series of prompts to inject the visual model information into ChatGPT,&#10;considering models of multiple inputs/outputs and models that require visual&#10;feedback. Experiments show that Visual ChatGPT opens the door to investigating&#10;the visual roles of ChatGPT with the help of Visual Foundation Models. Our&#10;system is publicly available at&#10;\url{https://github.com/microsoft/visual-chatgpt}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.09808" label="2211.09808">
-        <attvalues>
-          <attvalue for="0" value="Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and&#10;  Vision-Language Tasks" />
-          <attvalue for="1" value="  Despite the remarkable success of foundation models, their task-specific&#10;fine-tuning paradigm makes them inconsistent with the goal of general&#10;perception modeling. The key to eliminating this inconsistency is to use&#10;generalist models for general task modeling. However, existing attempts at&#10;generalist models are inadequate in both versatility and performance. In this&#10;paper, we propose Uni-Perceiver v2, which is the first generalist model capable&#10;of handling major large-scale vision and vision-language tasks with competitive&#10;performance. Specifically, images are encoded as general region proposals,&#10;while texts are encoded via a Transformer-based language model. The encoded&#10;representations are transformed by a task-agnostic decoder. Different tasks are&#10;formulated as a unified maximum likelihood estimation problem. We further&#10;propose an improved optimizer to ensure stable multi-task learning with an&#10;unmixed sampling strategy, which is helpful for tasks requiring large&#10;batch-size training. After being jointly trained on various tasks,&#10;Uni-Perceiver v2 is capable of directly handling downstream tasks without any&#10;task-specific adaptation. Results show that Uni-Perceiver v2 outperforms all&#10;existing generalist models in both versatility and performance. Meanwhile,&#10;compared with the commonly-recognized strong baselines that require&#10;tasks-specific fine-tuning, Uni-Perceiver v2 achieves competitive performance&#10;on a broad range of vision and vision-language tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.02499" label="2212.02499">
-        <attvalues>
-          <attvalue for="0" value="Images Speak in Images: A Generalist Painter for In-Context Visual&#10;  Learning" />
-          <attvalue for="1" value="  In-context learning, as a new paradigm in NLP, allows the model to rapidly&#10;adapt to various tasks with only a handful of prompts and examples. But in&#10;computer vision, the difficulties for in-context learning lie in that tasks&#10;vary significantly in the output representations, thus it is unclear how to&#10;define the general-purpose task prompts that the vision model can understand&#10;and transfer to out-of-domain tasks. In this work, we present Painter, a&#10;generalist model which addresses these obstacles with an &quot;image&quot;-centric&#10;solution, that is, to redefine the output of core vision tasks as images, and&#10;specify task prompts as also images. With this idea, our training process is&#10;extremely simple, which performs standard masked image modeling on the stitch&#10;of input and output image pairs. This makes the model capable of performing&#10;tasks conditioned on visible image patches. Thus, during inference, we can&#10;adopt a pair of input and output images from the same task as the input&#10;condition, to indicate which task to perform. Without bells and whistles, our&#10;generalist Painter can achieve competitive performance compared to&#10;well-established task-specific models, on seven representative vision tasks&#10;ranging from high-level visual understanding to low-level image processing. In&#10;addition, Painter significantly outperforms recent generalist models on several&#10;challenging tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.06377" label="2111.06377">
-        <attvalues>
-          <attvalue for="0" value="Masked Autoencoders Are Scalable Vision Learners" />
-          <attvalue for="1" value="  This paper shows that masked autoencoders (MAE) are scalable self-supervised&#10;learners for computer vision. Our MAE approach is simple: we mask random&#10;patches of the input image and reconstruct the missing pixels. It is based on&#10;two core designs. First, we develop an asymmetric encoder-decoder architecture,&#10;with an encoder that operates only on the visible subset of patches (without&#10;mask tokens), along with a lightweight decoder that reconstructs the original&#10;image from the latent representation and mask tokens. Second, we find that&#10;masking a high proportion of the input image, e.g., 75%, yields a nontrivial&#10;and meaningful self-supervisory task. Coupling these two designs enables us to&#10;train large models efficiently and effectively: we accelerate training (by 3x&#10;or more) and improve accuracy. Our scalable approach allows for learning&#10;high-capacity models that generalize well: e.g., a vanilla ViT-Huge model&#10;achieves the best accuracy (87.8%) among methods that use only ImageNet-1K&#10;data. Transfer performance in downstream tasks outperforms supervised&#10;pre-training and shows promising scaling behavior.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.00803" label="2110.00803">
-        <attvalues>
-          <attvalue for="0" value="Welsch Based Multiview Disparity Estimation" />
-          <attvalue for="1" value="  In this work, we explore disparity estimation from a high number of views. We&#10;experimentally identify occlusions as a key challenge for disparity estimation&#10;for applications with high numbers of views. In particular, occlusions can&#10;actually result in a degradation in accuracy as more views are added to a&#10;dataset. We propose the use of a Welsch loss function for the data term in a&#10;global variational framework for disparity estimation. We also propose a&#10;disciplined warping strategy and a progressive inclusion of views strategy that&#10;can reduce the need for coarse to fine strategies that discard high spatial&#10;frequency components from the early iterations. Experimental results&#10;demonstrate that the proposed approach produces superior and/or more robust&#10;estimates than other conventional variational approaches.&#10;" />
-          <attvalue for="2" value="&#10;Disparity estimation has been studied extensively in the literature and many different approaches have been taken for this problem. Most existing approaches focus on stereo pairs and excellent results have been achieved in this context \cite{Scharstein2001} \cite{yang2019hierarchical} \cite{song2020edgestereo} \cite{Xu2020}. However, the intuitive assumption that adding more views of a given scene will make disparity estimation more accurate does not actually hold. In scenes with occlusions, it turns out that as more views are added, with larger baselines, the increased areas of occlusion pose problems for most disparity estimation algorithms \cite{Kang2001}. This is unfortunate, since larger baselines allow for greater precision in the estimated disparity \cite{okutomi1993multiple}. &#10;&#10;Current hand-crafted methods of multiview disparity estimation with large numbers of views generally attempt to deal with this by using explicit occlusion based reasoning. This is done to reduce the influence of occluded areas on the estimated disparity. These techniques often use heuristics such as confidence based measures \cite{Strecha2002} or specific visibility reasoning \cite{Kang2001} \cite{Zhu2015} \cite{Hu2012}. These are often based on preliminary estimates of the disparity field themselves, so that early errors can potentially be reinforced. Other approaches treat occluded areas simply as outliers \cite{Bailer2012} \cite{10.1007/978-3-540-88682-2_58} \cite{Jeon2015} and reduce their effect on the solution. This is conceptually similar to the use of robust cost functions in optical flow, reducing the influence of outliers on the solution without explicitly identifying them. Common robust cost functions include the Huber norm \cite{shulman1989regularization} and similar $L^1$-like norms \cite{brox2004high} \cite{volz2011modeling} the Geman and McClure norm, the Lorentzian norm \cite{Black1996}, the Tukey norm \cite{odobez1995robust} and the Welsch loss function \cite{Holland1977}.&#10;&#10;In this paper we experimentally show that occlusions are a key problem for extending stereo disparity to a large number of views. We also propose a novel method of dealing with these occlusions in a multiview context using a Welsch loss function based data term. In particular, we propose an automatic selection process for Welsch loss parameter $\sigma_d$, a progressive approach to including the multiple views and a warping strategy that uses a disciplined multiple hypotheses method for upsampling the disparity field. We evaluate the proposed method using a synthetic dataset with 31 views of the same scene as well as a 4D Light Field Benchmark training dataset \cite{Honauer2016}.&#10;&#10;This paper is organised as follows. The proposed Welsch-$L^1$ multiview disparity algorithm is detailed in Section \ref{sec:algorithm}. The progressive inclusion of views is detailed in Section \ref{sec:ProgViews}. The warping strategy which uses a disciplined method for upsampling disparity fields is discussed in Section \ref{sssec:WarpStrat}.&#10;The performance of the proposed algorithm is evaluated in Section \ref{sec:Eval}. Conclusions are provided in Section \ref{sec:Conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Disparity Estimation, Mathematics, Physics, Computer Science, Variational Frameworks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1912.06704" label="1912.06704">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Deep Stereo Matching on High-resolution Images" />
-          <attvalue for="1" value="  We explore the problem of real-time stereo matching on high-res imagery. Many&#10;state-of-the-art (SOTA) methods struggle to process high-res imagery because of&#10;memory constraints or speed limitations. To address this issue, we propose an&#10;end-to-end framework that searches for correspondences incrementally over a&#10;coarse-to-fine hierarchy. Because high-res stereo datasets are relatively rare,&#10;we introduce a dataset with high-res stereo pairs for both training and&#10;evaluation. Our approach achieved SOTA performance on Middlebury-v3 and&#10;KITTI-15 while running significantly faster than its competitors. The&#10;hierarchical design also naturally allows for anytime on-demand reports of&#10;disparity by capping intermediate coarse results, allowing us to accurately&#10;predict disparity for near-range structures with low latency (30ms). We&#10;demonstrate that the performance-vs-speed trade-off afforded by on-demand&#10;hierarchies may address sensing needs for time-critical applications such as&#10;autonomous driving.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.01700" label="1903.01700">
-        <attvalues>
-          <attvalue for="0" value="EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching&#10;  and Edge Detection" />
-          <attvalue for="1" value="  Recently, leveraging on the development of end-to-end convolutional neural&#10;networks (CNNs), deep stereo matching networks have achieved remarkable&#10;performance far exceeding traditional approaches. However, state-of-the-art&#10;stereo frameworks still have difficulties at finding correct correspondences in&#10;texture-less regions, detailed structures, small objects and near boundaries,&#10;which could be alleviated by geometric clues such as edge contours and&#10;corresponding constraints. To improve the quality of disparity estimates in&#10;these challenging areas, we propose an effective multi-task learning network,&#10;EdgeStereo, composed of a disparity estimation branch and an edge detection&#10;branch, which enables end-to-end predictions of both disparity map and edge&#10;map. To effectively incorporate edge cues, we propose the edge-aware smoothness&#10;loss and edge feature embedding for inter-task interactions. It is demonstrated&#10;that based on our unified model, edge detection task and stereo matching task&#10;can promote each other. In addition, we design a compact module called residual&#10;pyramid to replace the commonly-used multi-stage cascaded structures or 3-D&#10;convolution based regularization modules in current stereo matching networks.&#10;By the time of the paper submission, EdgeStereo achieves state-of-art&#10;performance on the FlyingThings3D dataset, KITTI 2012 and KITTI 2015 stereo&#10;benchmarks, outperforming other published stereo matching methods by a&#10;noteworthy margin. EdgeStereo also achieves comparable generalization&#10;performance for disparity estimation because of the incorporation of edge cues.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.02695" label="1810.02695">
-        <attvalues>
-          <attvalue for="0" value="Learning Depth with Convolutional Spatial Propagation Network" />
-          <attvalue for="1" value="  Depth prediction is one of the fundamental problems in computer vision. In&#10;this paper, we propose a simple yet effective convolutional spatial propagation&#10;network (CSPN) to learn the affinity matrix for various depth estimation tasks.&#10;Specifically, it is an efficient linear propagation model, in which the&#10;propagation is performed with a manner of recurrent convolutional operation,&#10;and the affinity among neighboring pixels is learned through a deep&#10;convolutional neural network (CNN). We can append this module to any output&#10;from a state-of-the-art (SOTA) depth estimation networks to improve their&#10;performances. In practice, we further extend CSPN in two aspects: 1) take&#10;sparse depth map as additional input, which is useful for the task of depth&#10;completion; 2) similar to commonly used 3D convolution operation in CNNs, we&#10;propose 3D CSPN to handle features with one additional dimension, which is&#10;effective in the task of stereo matching using 3D cost volume. For the tasks of&#10;sparse to dense, a.k.a depth completion. We experimented the proposed CPSN&#10;conjunct algorithms over the popular NYU v2 and KITTI datasets, where we show&#10;that our proposed algorithms not only produce high quality (e.g., 30% more&#10;reduction in depth error), but also run faster (e.g., 2 to 5x faster) than&#10;previous SOTA spatial propagation network. We also evaluated our stereo&#10;matching algorithm on the Scene Flow and KITTI Stereo datasets, and rank 1st on&#10;both the KITTI Stereo 2012 and 2015 benchmarks, which demonstrates the&#10;effectiveness of the proposed module. The code of CSPN proposed in this work&#10;will be released at https://github.com/XinJCheng/CSPN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Depth estimation has been a center problem for computer vision and robotics for a long time. Due to space limitation, here we summarize those works in several most relevant aspects.&#10;&#10;Single view depth estimation via CNN and CRF. We first review single view depth estimation since methods in this area motivated our design. Deep neural networks (DCN) provide strong feature representation for various vision tasks. Depending on DCN, numerous algorithms are developed, including supervised methods~\cite{wang2015designing,eigen2015predicting,laina2016deeper,li2017two}\cite{luo2018single},\cite{jiao2018look},\cite{Fu_2018_CVPR}&#10;semi-supervised methods~\cite{kuznietsov2017semi} or unsupervised methods~\cite{godard2016unsupervised,zhou2017unsupervised,yang2018aaai,yang2018lego,luo2018every,Jiang_2018_ECCV}.&#10;Others tried to improve the estimated details further by appending a conditional random field (CRF)~\cite{DBLP:conf/cvpr/WangSLCPY15,Liu_2015_CVPR,li2015depth,xu2017multi,xu2018structured,Heo_2018_ECCV} and multi-task correlation with joint training~\cite{peng2016depth,jiao2018look,Qi_2018_CVPR,Xu_2018_CVPR}. &#10;However, the affinity for measuring the coherence of neighboring pixels is commonly manual-designed based on color similarity or intervening contour~\cite{shi2000normalized} with RBF kernel~\cite{peng2016depth,Heo_2018_ECCV}. &#10;In our case, CSPN learn data dependent affinity that has demonstrated to be more robust in practice. &#10;&#10;Depth Enhancement.&#10;Traditionally, depth output can be also efficiently enhanced with explicitly designed affinity through image filtering~\cite{barron2016fast,matsuo2015depth}, or data-driven ones through total variation (TV)~\cite{ferstl2013image,ferstl2015variational} and learning to diffuse~\cite{liu2016learning} by incorporating more priors into diffusion partial differential equations (PDEs).&#10;However, due to the lack of an effective learning strategy, they are limited for large-scale complex visual enhancement.&#10;&#10;Recently, deep learning based enhancement yields impressive results on super resolution of both images~\cite{dong2014learning,yang2014color} and depths~\cite{song2016deep,hui2016depth,kwon2015data,riegler2016atgv}. The network takes low-resolution inputs and output the high-resolution results, and is trained end-to-end where the mapping between input and output is implicitly learned.&#10;However, these methods are trained and experimented only with perfect correspondent ground-truth low-resolution and high-resolution depth maps and often a black-box model. In our scenario, neither the input nor ground truth depth are non-perfect, \eg depths from a low cost LiDAR or a network, thus an explicit diffusion process to guide the enhancement such as SPN is necessary.&#10;&#10;Learning affinity for spatial diffusion.&#10;Learning affinity matrix with deep CNN for diffusion or spatial propagation receives high interests in recent years due to its theoretical supports and guarantees~\cite{weickert1998anisotropic}.&#10;Maire \etal~\cite{maire2016affinity} trained a deep CNN to directly predict the entities of an affinity matrix, which demonstrated good performance on image segmentation. However, the affinity is followed by an independent non-differentiable solver of spectral embedding, it can not be supervised end-to-end for the prediction task. Bertasius \etal~\cite{bertasius2016convolutional} introduced a random walk network that optimizes the objectives of pixel-wise affinity for semantic segmentation. Nevertheless, their affinity matrix needs additional supervision from ground-truth sparse pixel pairs, which limits the potential connections between pixels. Chen \etal~\cite{chen2016semantic} try to explicit model an edge map for domain transform to improve the performance. &#10;&#10;The most related work is SPN~\cite{liu2017learning}, where the learning of a large affinity matrix for diffusion is converted to learning a local linear spatial propagation, yielding a simple yet effective approach for output enhancement. However, as mentioned in~\ref{sec:intro}, depth enhancement commonly needs local context, especially when sparse depth samples are available for depth completion. It might not be necessary to update a pixel by scanning the whole image. In our experiments, the proposed CSPN is more efficient and provides much better results.&#10;&#10;Depth completion with sparse samples.&#10; Different from depth enhancement, the provided depths are usually from low-cost LiDAR, yielding a map with valid depth in only few hundreds of pixels, as illustrated in gref{fig:example}(a).&#10;Most recently, Ma \etal~\cite{Ma2017SparseToDense,ma2018self} propose to treat the sparse depth map as an additional input to a ResNet~\cite{laina2016deeper} based depth predictor, producing superior results than the depth output from CNN with sole image input. However, the output results are still blurry, and do not satisfy our requirements of depth as discussed in \ref{sec:intro}. In CSPN for depth completion, we embed the sampled depth samples in the diffusion process, where all the requirements we proposed are held properly.&#10;&#10;Some other works directly convert 3D points from LiDAR to dense ones without image input~\cite{Zimmermann2017Learning,Ladicky_2017_ICCV,uhrig2017sparsity}, In these methods, the density of valid depth must be high enough to reveal the scene structure.&#10;Other works~\cite{Zhang_2018_CVPR} complete depth with an RGB image in a manner of in-painting, where dense depth values are partially available. These approaches are dealing with a different setup from ours.&#10;&#10;Stereo depth estimation.&#10;Stereo depth estimation has long been a central problem in computer vision. Traditionally, Scharstein and Szeliski~\cite{scharstein2002taxonomy} provide a taxonomy of stereo algorithms including: matching cost calculations, matching cost aggregation, disparity calculation and disparity refinement\cite{hirschmuller2008stereo,heise2013pm,wang2008stereoscopic}. &#10;&#10;CNNs were first introduced to stereo matching by Zbontar and LeCun~\cite{zbontar2016stereo} to replace the computation of the matching cost. Their method showed that by using CNNs, the matching could be more robust, and achieved SOTA results over KITTI Stereo benchmarks. However, the networks are still shallow, and it needs post-processing for refinement. &#10;Following~\cite{zbontar2016stereo}, several methods were proposed to increase computational efficiency~\cite{feng2017efficient,luo2016efficient}, or matching cost accuracy~\cite{shaked2017improved} with stronger network and confidence predictions. &#10;Later, some works focused on post-process by incorporating high-level knowledge from objects such as Displets~\cite{guney2015displets}.&#10;&#10;This inspires the study of stereo matching networks to develop a fully learnable architecture without manually designed processing. FlowNet~\cite{ilg2017flownet} are designed to find 2D optical flow by inserting two corresponding frames, which can be easily extended to stereo matching by limiting the searching within the epipolar line. PWCNet~\cite{sun2018pwc} follows a similar idea while having cost volumes calculated using a pyramid warping strategy within a local region with size of $d \times d$. However, for stereo estimation, one may only consider a limited range for disparity matching based on the epipolar constraint.&#10;Therefore, to better model per-pixel disparity matching, GCNet~\cite{kendall2017end} proposes to generate a 3D cost volume of size $d \times h \times w \times c$ by densely comparing the feature at pixel (i, j) from the reference image to all possible matching pixels within the epipolar line at the target image. The network can estimate the best matching disparity through a $soft-argmin$ operation. PSMNet~\cite{chang2018pyramid}, embracing the experience of semantic segmentation studies, which additionally exploits scale space through pyramid spatial pooling and hourglass networks to capture global image context, yielding better results than GCNet. &#10;As can be seen, both GCNet and PSMNet are benefited from exploring a new dimension, \ie disparity value space and scale space respectively, which motivates us to extend CSPN to 3D. Built upon PSMNet, 3D CSPN considers modeling the relationship with diffusion along their proposed new dimension, and produces more robust results.&#10;&#10;Spatial pyramid for hierarchical context. As we also explore scale space for a dense prediction model, we would like to review spatial pyramid pooling (SPP)~\cite{he2014spatial} to provide more insight for our proposed model. Liu \etal~\cite{liu2015parsenet} first propose SPP to increase the empirical receptive field of a fully convolutional network. Such an idea is demonstrated to be very effective in both semantic segmentation~\cite{chen2017rethinking,chen2016deeplab,zhao2017pyramid}, and depth estimation~\cite{chang2018pyramid}.&#10;Here, in our perspective, the parameters for SPP form a scale space that is manually set and experimentally determined based on certain dataset by previous works~\cite{chen2017rethinking}. However, our CSPN with 3D convolution can learn the affinity for fusing the proposed scale space, which softly discovers the proper scale of context for the network. Most recently, Xu \etal~\cite{xu2018structured} also propose to learn attention guided multi-scale CRF for depth estimation, which shares similar spirits with ours.&#10;We show in our experiments, such a strategy effectively improves the depth estimation results over PSMNet. In the near future, we will extend this idea to semantic segmentation to validate its generalization capability.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science, Neural Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2006.04902" label="2006.04902">
-        <attvalues>
-          <attvalue for="0" value="What Matters in Unsupervised Optical Flow" />
-          <attvalue for="1" value="  We systematically compare and analyze a set of key components in unsupervised&#10;optical flow to identify which photometric loss, occlusion handling, and&#10;smoothness regularization is most effective. Alongside this investigation we&#10;construct a number of novel improvements to unsupervised flow models, such as&#10;cost volume normalization, stopping the gradient at the occlusion mask,&#10;encouraging smoothness before upsampling the flow field, and continual&#10;self-supervision with image resizing. By combining the results of our&#10;investigation with our improved model components, we are able to present a new&#10;unsupervised flow technique that significantly outperforms the previous&#10;unsupervised state-of-the-art and performs on par with supervised FlowNet2 on&#10;the KITTI 2015 dataset, while also being significantly simpler than related&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The motion between an object and a viewer causes apparent movement of brightness patterns in the image~\cite{Gibson1950}. Optical flow techniques attempt to invert this relationship to recover a motion estimate~\cite{lucas1981iterative}. Classical methods infer optical flow for a pair of images by minimizing a loss function that measures photometric consistency and smoothness~\cite{Horn1981,Brox04,Sun2010}. Recent approaches reframe optical flow estimation as a learning problem in which a CNN-based model regresses from a pair of images to a flow field~\cite{FlowNet,Flownet2}. Some models incorporate ideas from earlier methods, such as cost volumes and coarse-to-fine warping~\cite{spynet2017,Sun2018PWCNet,yang2019volumetric}. These supervised approaches require representative training data with accurate optical flow labels. Though such data can be generated for rigid objects with known geometry~\cite{Geiger2012CVPR,KITTI2015}, recovering this ground truth flow for arbitrary scenes is laborious, and requires approaches as unusual as manually painting scenes with textured fluorescent paint and imaging it under ultraviolet light~\cite{Baker2011}. Since such approaches scale poorly, supervised methods have mainly relied on synthetic data for training, and often for evaluation \cite{Barron94performanceof,ButlerECCV2012,FlowNet}. Synthesizing ``good'' training data (such that learned models generalize to real images) is itself a hard research problem, requiring careful consideration of scene content, camera motion, lens distortion, and sensor degradation~\cite{Mayer2018}.&#10;&#10;Unsupervised approaches circumvent the need for labels by optimizing photometric consistency with some regularization~\cite{ren2017unsupervised,jjyu2016unsupflow}, similar to the classical optimization-based methods mentioned above. Where traditional methods solve an optimization problem for each image pair, unsupervised learning jointly optimizes an objective across all pairs in a dataset and learns a function that regresses a flow field from images. This approach has two advantages: 1) inference is fast because optimization is only performed during training, and 2) by jointly optimizing across the whole train set, information is shared across image pairs which can potentially improve performance. This unsupervised approach was extended to use edge-aware smoothness~\cite{wang2018occlusion}, a bi-directional Census loss~\cite{meister2018unflow}, different forms of occlusion estimation~\cite{Janai2018ECCV,meister2018unflow,wang2018occlusion}, self-supervision~\cite{DDFlow,SelFlow}, and estimation from multiple frames~\cite{Janai2018ECCV,SelFlow}. Other extensions introduced geometric reasoning through epipolar constraints~\cite{Zhong2019UnsupervisedDE} or by co-training optical flow with depth and ego-motion models from monocular~\cite{ranjan2019cvpr,yin2018geonet,zou2018dfnet} or stereo input~\cite{wang2018unos}.&#10;&#10;These works have pushed the state of the art and generated a range of ideas for unsupervised optical flow. But since each of them evaluates a different combination of ideas, it is unclear how individual ideas compare to each other and which ideas combine well together. For example, the methods OAFlow~\cite{wang2018occlusion} and DDFlow~\cite{DDFlow} use different photometric losses and different ways to mask occlusions, and OAFlow uses an edge-aware smoothness loss while DDFlow regularizes learning through self-supervision. DDFlow performs better than OAFlow, but does this mean that every component of DDFlow is better than every component of OAFlow?&#10;The ablation studies often presented in these papers show that each novel contribution of each work does indeed improve the performance of each individual model, but they do not provide a guarantee that each such contribution will always improve performance when added to any other model.&#10;Our work addresses this problem by systematically comparing and combining photometric losses (L1, Charbonnier~\cite{Sun2010}, Census~\cite{DDFlow,meister2018unflow,Zhong2019UnsupervisedDE,zou2018dfnet}, and structural similarity~\cite{ranjan2019cvpr,wang2018unos,yin2018geonet}), different methods for occlusion estimation~\cite{Brox04,wang2018occlusion}, first order and second order edge-aware smoothness~\cite{tomasi1998bilateral}, and self-supervision~\cite{DDFlow}. Our work also improves cost volume computation, occlusion estimation, smoothness, and self-supervision and integrates all components into an state of the art framework for unsupervised optical, while being simpler than many proposed methods to form a solid base for future work.&#10;&#10;" />
-          <attvalue for="4" value="Optical Flow Analysis, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2011.04862" label="2011.04862">
-        <attvalues>
-          <attvalue for="0" value="On Efficient and Robust Metrics for RANSAC Hypotheses and 3D Rigid&#10;  Registration" />
-          <attvalue for="1" value="  This paper focuses on developing efficient and robust evaluation metrics for&#10;RANSAC hypotheses to achieve accurate 3D rigid registration. Estimating&#10;six-degree-of-freedom (6-DoF) pose from feature correspondences remains a&#10;popular approach to 3D rigid registration, where random sample consensus&#10;(RANSAC) is a de-facto choice to this problem. However, existing metrics for&#10;RANSAC hypotheses are either time-consuming or sensitive to common nuisances,&#10;parameter variations, and different application scenarios, resulting in&#10;performance deterioration in overall registration accuracy and speed. We&#10;alleviate this problem by first analyzing the contributions of inliers and&#10;outliers, and then proposing several efficient and robust metrics with&#10;different designing motivations for RANSAC hypotheses. Comparative experiments&#10;on four standard datasets with different nuisances and application scenarios&#10;verify that the proposed metrics can significantly improve the registration&#10;performance and are more robust than several state-of-the-art competitors,&#10;making them good gifts to practical applications. This work also draws an&#10;interesting conclusion, i.e., not all inliers are equal while all outliers&#10;should be equal, which may shed new light on this research problem.&#10;" />
-          <attvalue for="2" value="&#10;R{igid} registration of 3D point clouds is an active research area in computer vision with numerous applications such as 3D reconstruction~\cite{yang2018aligning,guo2014accurate}, 3D object recognition~\cite{guo20143d}, and localization~\cite{tateno20162}. The essential problem is estimating a six-degree-of-freedom (6-DoF) pose to transform the source point cloud to the coordinate system of the target point cloud. Methods based on point-to-point feature correspondences, due to their robustness to occlusion, are widely employed in various registration scenarios~\cite{mian2005automatic,guo20143d,lei2017fast}. However, initial correspondences generated by matching local geometric descriptors are usually contaminated by heavy outliers owing to nuisances such as noise, varying data resolutions, clutter, and occlusion~\cite{Yang2020corr_group_eval}. Therefore, 6-DoF pose estimation from correspondences with low inlier ratios remains a challenging problem.&#10;&#10;Random sample consensus (RANSAC)~\cite{fischler1981random} appears to be a de-facto solution to this problem. It iteratively generates 6-DoF pose hypotheses from correspondences and finds the best hypothesis as the final solution based on some hypothesis evaluation metrics. Although there have been a number of RANSAC variants~\cite{yang2016fast,yang2017multi,guo2015integrated}, they still follow the fundamental pipeline as RANSAC. Given the assumption that correct hypotheses are generated during the sampling process, the optimal solution can be found if with reasonable hypothesis evaluation metrics. Existing metrics for RANSAC hypotheses can be divided into two categories: correspondences-based and point-cloud-based. Metrics in the former category, e.g., inlier count~\cite{fischler1981random,rusu20113d} and Huber loss~\cite{rusu2009fast}, are known to be quite efficient because only sparse correspondence data are leveraged. However, they need very careful parameter tuning and are sensitive to many nuisances as will be verified Sect.~\ref{sec:exp}. Metrics in the latter category, e.g., the number of overlapped points~\cite{Quan2018Local} and point cloud distance~\cite{yang2016fast}, consider the whole point cloud for hypothesis evaluation. It has been demonstrated that these metrics can effectively improve the registration performance on some benchmarks. Nonetheless, they are particularly time-consuming because dense point clouds are treated as the input and nearest neighbor search is required. Therefore, existing hypothesis evaluation metrics for RANSAC in the context of 3D rigid registration fail to achieve a good balance between efficiency and robustness.&#10;\\\\Related work This work focuses on the problem of defining efficient and robust hypothesis evaluation metrics for RANSAC to achieve accurate and efficient 3D rigid registration. We find that this problem has been more or less overlooked in the literature.&#10;&#10;For RANSAC estimators applied to the 3D rigid registration problem, the classical hypothesis evaluation metric, i.e., the inlier count, is arguably the most commonly employed one~\cite{mian2005automatic,rusu20113d}. In the recent decade, some other metrics have been proposed. Rusu et al.~\cite{rusu2009fast} proposed using the Huber loss as the evaluation metric for the sample consensus initial alignment (SAC-IA) estimator, i.e., a variant of RANSAC. Both inlier count and Huber loss only leverage sparse correspondence data. Yang et al.~\cite{yang2016fast} proposed the first point-cloud-based metric for RANSAC hypotheses named the point cloud distance, which is the mean of the distances of each point in the source point cloud to the target point cloud. Later, Quan et al.~\cite{Quan2018Local} also leveraged both point clouds during hypothesis evaluation but instead calculated the number of qualified points in the source point cloud that are close to the target point cloud.&#10;&#10;Unfortunately, existing correspondences-based metrics are sensitive to a number of nuisances and parameter changes; point-cloud-based metrics are shown to be more robust but particularly time-consuming. In addition, we will show that different hypothesis evaluation metrics will result in significant registration performance variation. Although for RANSAC estimators applied to image registration, some metrics different from the classical inlier count metric have bee proposed~\cite{barath2018graph,lebeda2012fixing}, they fail to comprehensively analyze the contributions of inliers and outliers.&#10;\\\\Contributions By reviewing existing related works, we find that the evaluation metric definition problem for RANSAC hypotheses has been overlooked and existing metrics fail to strike a good balance between robustness and efficiency. Consequently, we propose several continuous functions as hypothesis evaluation metrics, which evaluate the contributions of inliers and outliers in different ways. To show the impact of different hypothesis evaluation metrics on 3D registration performance and the effectiveness of the proposed metrics, comprehensive experiments on four datasets with comparison to all existing hypothesis evaluation metrics (to the best of our knowledge) have been conducted. The results suggest that our metrics can significantly improve the registration performance and are efficient as well as robust to common nuisances. In a nutshell, this paper has the following contributions.&#10;\begin{itemize}&#10;&#09;\item This paper proposes several metrics for RANSAC hypotheses, which evaluate the contributions of inliers and outliers with different motivations. The technique details are simple, while the performance boosting is impressive.&#10;&#09;\item In-depth experiments on four datasets with different application scenarios and nuisances have been conducted, to on one hand demonstrate the significance of hypothesis evaluation to the eventual 3D rigid registration performance and on the other show the overall superiority (i.e., fast, robust, and insensitive to parameter changes) of the proposed metrics. &#10;&#09;\item We draw an interesting conclusion that {not all inliers are equal while all outliers should be equal}, which may shed new light on this research problem.&#10;\end{itemize}&#10;Paper organization The remainder of this paper is structured as follows. Sect.~\ref{sec:mtd} introduces the technique details of the proposed metrics. Sect.~\ref{sec:exp} presents the experiments to validate the effectiveness of the proposed metrics with necessary explanations. Finally, Sect.~\ref{sec:conc} concludes the paper and presents potential future research directions.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, RANSAC Evaluation Metrics, Computer Science, Computer Vision Algorithms, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1907.02890" label="1907.02890">
-        <attvalues>
-          <attvalue for="0" value="A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid&#10;  Data Matching" />
-          <attvalue for="1" value="  Seeking consistent point-to-point correspondences between 3D rigid data&#10;(point clouds, meshes, or depth maps) is a fundamental problem in 3D computer&#10;vision. While a number of correspondence selection methods have been proposed&#10;in recent years, their advantages and shortcomings remain unclear regarding&#10;different applications and perturbations. To fill this gap, this paper gives a&#10;comprehensive evaluation of nine state-of-the-art 3D correspondence grouping&#10;methods. A good correspondence grouping algorithm is expected to retrieve as&#10;many as inliers from initial feature matches, giving a rise in both precision&#10;and recall as well as facilitating accurate transformation estimation. Toward&#10;this rule, we deploy experiments on three benchmarks with different application&#10;contexts including shape retrieval, 3D object recognition, and point cloud&#10;registration together with various perturbations such as noise, point density&#10;variation, clutter, occlusion, partial overlap, different scales of initial&#10;correspondences, and different combinations of keypoint detectors and&#10;descriptors. The rich variety of application scenarios and nuisances result in&#10;different spatial distributions and inlier ratios of initial feature&#10;correspondences, thus enabling a thorough evaluation. Based on the outcomes, we&#10;give a summary of the traits, merits, and demerits of evaluated approaches and&#10;indicate some potential future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.00984" label="1706.00984">
-        <attvalues>
-          <attvalue for="0" value="Graph-Cut RANSAC" />
-          <attvalue for="1" value="  A novel method for robust estimation, called Graph-Cut RANSAC, GC-RANSAC in&#10;short, is introduced. To separate inliers and outliers, it runs the graph-cut&#10;algorithm in the local optimization (LO) step which is applied when a&#10;so-far-the-best model is found. The proposed LO step is conceptually simple,&#10;easy to implement, globally optimal and efficient. GC-RANSAC is shown&#10;experimentally, both on synthesized tests and real image pairs, to be more&#10;geometrically accurate than state-of-the-art methods on a range of problems,&#10;e.g. line fitting, homography, affine transformation, fundamental and essential&#10;matrix estimation. It runs in real-time for many problems at a speed&#10;approximately equal to that of the less accurate alternatives (in milliseconds&#10;on standard CPU).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.06884" label="1909.06884">
-        <attvalues>
-          <attvalue for="0" value="Performance Evaluation of Learned 3D Features" />
-          <attvalue for="1" value="  Matching surfaces is a challenging 3D Computer Vision problem typically&#10;addressed by local features. Although a variety of 3D feature detectors and&#10;descriptors has been proposed in literature, they have seldom been proposed&#10;together and it is yet not clear how to identify the most effective&#10;detector-descriptor pair for a specific application. A promising solution is to&#10;leverage machine learning to learn the optimal 3D detector for any given 3D&#10;descriptor [15]. In this paper, we report a performance evaluation of the&#10;detector-descriptor pairs obtained by learning a paired 3D detector for the&#10;most popular 3D descriptors. In particular, we address experimental settings&#10;dealing with object recognition and surface registration.&#10;" />
-          <attvalue for="2" value="&#10;Surface matching is an ubiquitous task in 3D Computer Vision, where it helps to tackle major applications such as object recognition and surface registration.&#10;Nowadays, most surface matching methods follow a local paradigm based on establishing correspondences between 3D patches referred to as features. The typical feature-matching pipeline consists of three steps: detection, description and matching.&#10;&#10;Although over the last decades many 3D detectors and descriptors have been proposed in literature, it is yet unclear how to effectively combine these proposals to create an effective pipeline. Indeed, unlike the related field of local image features, methods to either detect or describe 3D features have been designed and proposed separately, alongside with specific application settings and related datasets. This is also vouched by the main performance evaluation papers in the field, which address either repeatability of 3D detectors designed to highlight geometrically salient surface patches \cite{tombari2013performance} or distinctiveness and robustness of popular 3D descriptors \cite{guo2016comprehensive}. &#10;&#10;More recently, however, \cite{salti2015learning} and \cite{tonioni2018learning} have proposed &#10;a machine learning approach that allows for learning an optimal 3D keypoint detector for any given 3D descriptor so as to maximize the end-to-end performance of the overall feature-matching pipeline. &#10;The authors show that this approach provides effective pipelines across diverse applications and datasets. Moreover, their object recognition experiments show that, with the considered descriptors (SHOT \cite{Tombari:2010:USH:1927006.1927035}, Spin Image (SI) \cite{johnson1999using}, FPFH \cite{rusu2010fast}), learning to detect specific keypoints leads to better performance than relying on existing general-purpose handcrafted detectors (ISS \cite{zhong2009intrinsic}, Harris3D \cite{sipiran2011harris}, NARF \cite{steder2010narf}). &#10;&#10;By enabling an optimal detector to be learned for any descriptor, \cite{tonioni2018learning} sets forth a novel paradigm to maximize affinity between 3D detectors and descriptors. This opens up the question of which learned detector-descriptor pair may turn out most effective in the main application areas. This paper tries to answer this question by proposing an experimental evaluation of learned 3D pipelines. In particular, we address object recognition and surface registration, and compare the performance attained by learning a paired feature detector for the most popular handcrafted 3D descriptors (SHOT \cite{Tombari:2010:USH:1927006.1927035}, SI \cite{johnson1999using}, FPFH \cite{rusu2010fast}, USC\cite{tombari2010unique}, RoPS \cite{guo2013rotational}) as well as for a recently proposed descriptor based on deep learning (CGF-32 \cite{khoury2017learning}). &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Object Recognition, Mathematics, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1304.3192" label="1304.3192">
-        <attvalues>
-          <attvalue for="0" value="Rotational Projection Statistics for 3D Local Surface Description and&#10;  Object Recognition" />
-          <attvalue for="1" value="  Recognizing 3D objects in the presence of noise, varying mesh resolution,&#10;occlusion and clutter is a very challenging task. This paper presents a novel&#10;method named Rotational Projection Statistics (RoPS). It has three major&#10;modules: Local Reference Frame (LRF) definition, RoPS feature description and&#10;3D object recognition. We propose a novel technique to define the LRF by&#10;calculating the scatter matrix of all points lying on the local surface. RoPS&#10;feature descriptors are obtained by rotationally projecting the neighboring&#10;points of a feature point onto 2D planes and calculating a set of statistics&#10;(including low-order central moments and entropy) of the distribution of these&#10;projected points. Using the proposed LRF and RoPS descriptor, we present a&#10;hierarchical 3D object recognition algorithm. The performance of the proposed&#10;LRF, RoPS descriptor and object recognition algorithm was rigorously tested on&#10;a number of popular and publicly available datasets. Our proposed techniques&#10;exhibited superior performance compared to existing techniques. We also showed&#10;that our method is robust with respect to noise and varying mesh resolution.&#10;Our RoPS based algorithm achieved recognition rates of 100%, 98.9%, 95.4% and&#10;96.0% respectively when tested on the Bologna, UWA, Queen's and Ca' Foscari&#10;Venezia Datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.05056" label="1709.05056">
-        <attvalues>
-          <attvalue for="0" value="Learning Compact Geometric Features" />
-          <attvalue for="1" value="  We present an approach to learning features that represent the local geometry&#10;around a point in an unstructured point cloud. Such features play a central&#10;role in geometric registration, which supports diverse applications in robotics&#10;and 3D vision. Current state-of-the-art local features for unstructured point&#10;clouds have been manually crafted and none combines the desirable properties of&#10;precision, compactness, and robustness. We show that features with these&#10;properties can be learned from data, by optimizing deep networks that map&#10;high-dimensional histograms into low-dimensional Euclidean spaces. The&#10;presented approach yields a family of features, parameterized by dimension,&#10;that are both more compact and more accurate than existing descriptors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.06305" label="2409.06305">
-        <attvalues>
-          <attvalue for="0" value="High-Performance Few-Shot Segmentation with Foundation Models: An&#10;  Empirical Study" />
-          <attvalue for="1" value="  Existing few-shot segmentation (FSS) methods mainly focus on designing novel&#10;support-query matching and self-matching mechanisms to exploit implicit&#10;knowledge in pre-trained backbones. However, the performance of these methods&#10;is often constrained by models pre-trained on classification tasks. The&#10;exploration of what types of pre-trained models can provide more beneficial&#10;implicit knowledge for FSS remains limited. In this paper, inspired by the&#10;representation consistency of foundational computer vision models, we develop a&#10;FSS framework based on foundation models. To be specific, we propose a simple&#10;approach to extract implicit knowledge from foundation models to construct&#10;coarse correspondence and introduce a lightweight decoder to refine coarse&#10;correspondence for fine-grained segmentation. We systematically summarize the&#10;performance of various foundation models on FSS and discover that the implicit&#10;knowledge within some of these models is more beneficial for FSS than models&#10;pre-trained on classification tasks. Extensive experiments on two widely used&#10;datasets demonstrate the effectiveness of our approach in leveraging the&#10;implicit knowledge of foundation models. Notably, the combination of DINOv2 and&#10;DFN exceeds previous state-of-the-art methods by 17.5% on COCO-20i. Code is&#10;available at https://github.com/DUT-CSJ/FoundationFSS.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Significant progress in fully-supervised semantic segmentation is driven by large-scale pixel-level labeled datasets. It takes more than an hour to obtain pixel-level annotation of an image~\cite{cordts2016cityscapes}, which makes the labor cost of large-scale pixel-level annotated datasets expensive. And once the datasets are labeled pixel by pixel, it is difficult to add new categories to them. In response to the above challenge, few-shot segmentation (FSS), which aims to segment the corresponding object with unseen categories of the query image using only a few support image-mask pairs, has been proposed~\cite{shaban2017one}.&#10;&#10;Based on different strategies for mining query-support information, existing FSS methods can be divided into two technical approaches: support-query matching mechanism~\cite{min2021hypercorrelation, hong2022cost, moon2023msi, peng2023hierarchical, shuai2023pgmanet, yang2023mianet} and self-support matching mechanism~\cite{fan2022ssp, wang2023focus}. &#10;The former mechanism matches support features with query features by designing novel prototypical learning or dense correlation modules. The latter mechanism obtains a coarse segmentation map through support-query matching and uses a self-support module to refine the segmentation map. The continuous emergence of novel matching methods improves the performance of FSS. However, existing FSS methods mainly focus on designing matching modules to mine the implicit knowledge in the frozen pre-trained backbone, neglecting the exploration on which types of pre-trained backbone's implicit knowledge are more beneficial for FSS. &#10;In this paper, we aim to find a combination of pre-trained backbones that is more advantageous for the FSS task.&#10;&#10;Recently, various foundation models with powerful transfer learning and zero-shot capabilities have emerged. MAE~\cite{he2022mae} uses an intra-image self-supervised training paradigm and shows significant performance improvement when fine-tuned on downstream tasks. DINO~\cite{caron2021dinov1} and DINOv2~\cite{oquab2023dinov2} learn semantically invariant features with a discriminative self-supervised learning paradigm. Contrastive language-image pre-training (CLIP)~\cite{radford2021clip} can provide well-aligned textual and visual embeddings. The internal representations of text-to-image diffusion models~\cite{rombach2022sd} also demonstrate transfer capability to downstream tasks.&#10;Many methods transfer the aforementioned foundation models to their respective downstream tasks, {e.g.}, correspondence estimation and FSS, demonstrating superior performance.&#10;DIFT~\cite{tang2023dift} extracts diffusion features to establish correspondences between images. &#10;SD-DINO~\cite{zhang2024sddino} proposes to fuse the features of Stable Diffusion (SD)~\cite{rombach2022sd} and DINOv2~\cite{oquab2023dinov2} for semantic and dense correspondence.&#10;UniFSS~\cite{chang2024unifss} utilizes CLIP to build a universal vision-language framework to accomplish seven FSS tasks.&#10;While the above prior works show that foundation models can be used for correspondence estimation and FSS, the performance of various foundation models in FSS has not been thoroughly analyzed. Which foundation models' implicit knowledge is more beneficial for FSS should be explored.&#10;&#10;To address the challenges above, we propose a simple framework that extracts implicit knowledge from foundation models to construct coarse correspondence and refine the coarse correspondence for fine-grained segmentation with a lightweight decoder. We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}. We explore the benefits of foundation models for FSS from a different perspective than previous work, which may inspire researchers to address FSS from additional perspectives.&#10;&#10;Our contributions are summarized as follows:&#10;\begin{itemize}&#10; \item We address FSS from a new perspective, focusing on which knowledge from pre-trained models benefits FSS, rather than designing new matching algorithms. We analyze the potential of several vision and vision-language foundation models for FSS.&#10; \item We propose a simple framework that leverages implicit knowledge from foundation models to establish coarse correspondences and a lightweight decoder to refine them for fine-grained segmentation.&#10; \item Experiments show that our method achieves a new state-of-the-art on PASCAL-5$^i$~\cite{shaban2017one} and COCO-20$^i$~\cite{lin2014microsoft} under both mask&#10; FSS and class-aware mask FSS settings. Ablation studies indicate that the combination of DINOv2 and DFN achieves the best performance, surpassing previous state-of-the-art methods by 17.5\% on COCO-20$^i$ in terms of mIoU. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Foundation Models, Computer Science, Few-Shot Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1709.03410" label="1709.03410">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Learning for Semantic Segmentation" />
-          <attvalue for="1" value="  Low-shot learning methods for image classification support learning from&#10;sparse data. We extend these techniques to support dense semantic image&#10;segmentation. Specifically, we train a network that, given a small set of&#10;annotated images, produces parameters for a Fully Convolutional Network (FCN).&#10;We use this FCN to perform dense pixel-level prediction on a test image for the&#10;new semantic class. Our architecture shows a 25% relative meanIoU improvement&#10;compared to the best baseline methods for one-shot segmentation on unseen&#10;classes in the PASCAL VOC 2012 dataset and is at least 3 times faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.01538" label="2104.01538">
-        <attvalues>
-          <attvalue for="0" value="Hypercorrelation Squeeze for Few-Shot Segmentation" />
-          <attvalue for="1" value="  Few-shot semantic segmentation aims at learning to segment a target object&#10;from a query image using only a few annotated support images of the target&#10;class. This challenging task requires to understand diverse levels of visual&#10;cues and analyze fine-grained correspondence relations between the query and&#10;the support images. To address the problem, we propose Hypercorrelation Squeeze&#10;Networks (HSNet) that leverages multi-level feature correlation and efficient&#10;4D convolutions. It extracts diverse features from different levels of&#10;intermediate convolutional layers and constructs a collection of 4D correlation&#10;tensors, i.e., hypercorrelations. Using efficient center-pivot 4D convolutions&#10;in a pyramidal architecture, the method gradually squeezes high-level semantic&#10;and low-level geometric cues of the hypercorrelation into precise segmentation&#10;masks in coarse-to-fine manner. The significant performance improvements on&#10;standard few-shot segmentation benchmarks of PASCAL-5i, COCO-20i, and FSS-1000&#10;verify the efficacy of the proposed method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.11549" label="2207.11549">
-        <attvalues>
-          <attvalue for="0" value="Self-Support Few-Shot Semantic Segmentation" />
-          <attvalue for="1" value="  Existing few-shot segmentation methods have achieved great progress based on&#10;the support-query matching framework. But they still heavily suffer from the&#10;limited coverage of intra-class variations from the few-shot supports provided.&#10;Motivated by the simple Gestalt principle that pixels belonging to the same&#10;object are more similar than those to different objects of same class, we&#10;propose a novel self-support matching strategy to alleviate this problem, which&#10;uses query prototypes to match query features, where the query prototypes are&#10;collected from high-confidence query predictions. This strategy can effectively&#10;capture the consistent underlying characteristics of the query objects, and&#10;thus fittingly match query features. We also propose an adaptive self-support&#10;background prototype generation module and self-support loss to further&#10;facilitate the self-support matching procedure. Our self-support network&#10;substantially improves the prototype quality, benefits more improvement from&#10;stronger backbones and more supports, and achieves SOTA on multiple datasets.&#10;Codes are at \url{https://github.com/fanq15/SSP}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.07193" label="2304.07193">
-        <attvalues>
-          <attvalue for="0" value="DINOv2: Learning Robust Visual Features without Supervision" />
-          <attvalue for="1" value="  The recent breakthroughs in natural language processing for model pretraining&#10;on large quantities of data have opened the way for similar foundation models&#10;in computer vision. These models could greatly simplify the use of images in&#10;any system by producing all-purpose visual features, i.e., features that work&#10;across image distributions and tasks without finetuning. This work shows that&#10;existing pretraining methods, especially self-supervised methods, can produce&#10;such features if trained on enough curated data from diverse sources. We&#10;revisit existing approaches and combine different techniques to scale our&#10;pretraining in terms of data and model size. Most of the technical&#10;contributions aim at accelerating and stabilizing the training at scale. In&#10;terms of data, we propose an automatic pipeline to build a dedicated, diverse,&#10;and curated image dataset instead of uncurated data, as typically done in the&#10;self-supervised literature. In terms of models, we train a ViT model&#10;(Dosovitskiy et al., 2020) with 1B parameters and distill it into a series of&#10;smaller models that surpass the best available all-purpose features, OpenCLIP&#10;(Ilharco et al., 2021) on most of the benchmarks at image and pixel levels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.10752" label="2112.10752">
-        <attvalues>
-          <attvalue for="0" value="High-Resolution Image Synthesis with Latent Diffusion Models" />
-          <attvalue for="1" value="  By decomposing the image formation process into a sequential application of&#10;denoising autoencoders, diffusion models (DMs) achieve state-of-the-art&#10;synthesis results on image data and beyond. Additionally, their formulation&#10;allows for a guiding mechanism to control the image generation process without&#10;retraining. However, since these models typically operate directly in pixel&#10;space, optimization of powerful DMs often consumes hundreds of GPU days and&#10;inference is expensive due to sequential evaluations. To enable DM training on&#10;limited computational resources while retaining their quality and flexibility,&#10;we apply them in the latent space of powerful pretrained autoencoders. In&#10;contrast to previous work, training diffusion models on such a representation&#10;allows for the first time to reach a near-optimal point between complexity&#10;reduction and detail preservation, greatly boosting visual fidelity. By&#10;introducing cross-attention layers into the model architecture, we turn&#10;diffusion models into powerful and flexible generators for general conditioning&#10;inputs such as text or bounding boxes and high-resolution synthesis becomes&#10;possible in a convolutional manner. Our latent diffusion models (LDMs) achieve&#10;a new state of the art for image inpainting and highly competitive performance&#10;on various tasks, including unconditional image generation, semantic scene&#10;synthesis, and super-resolution, while significantly reducing computational&#10;requirements compared to pixel-based DMs. Code is available at&#10;https://github.com/CompVis/latent-diffusion .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.15347" label="2305.15347">
-        <attvalues>
-          <attvalue for="0" value="A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot&#10;  Semantic Correspondence" />
-          <attvalue for="1" value="  Text-to-image diffusion models have made significant advances in generating&#10;and editing high-quality images. As a result, numerous approaches have explored&#10;the ability of diffusion model features to understand and process single images&#10;for downstream tasks, e.g., classification, semantic segmentation, and&#10;stylization. However, significantly less is known about what these features&#10;reveal across multiple, different images and objects. In this work, we exploit&#10;Stable Diffusion (SD) features for semantic and dense correspondence and&#10;discover that with simple post-processing, SD features can perform&#10;quantitatively similar to SOTA representations. Interestingly, the qualitative&#10;analysis reveals that SD features have very different properties compared to&#10;existing representation learning features, such as the recently released&#10;DINOv2: while DINOv2 provides sparse but accurate matches, SD features provide&#10;high-quality spatial information but sometimes inaccurate semantic matches. We&#10;demonstrate that a simple fusion of these two features works surprisingly well,&#10;and a zero-shot evaluation using nearest neighbors on these fused features&#10;provides a significant performance gain over state-of-the-art methods on&#10;benchmark datasets, e.g., SPair-71k, PF-Pascal, and TSS. We also show that&#10;these correspondences can enable interesting applications such as instance&#10;swapping in two images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\myparagraph{Feature descriptor for dense correspondence.}&#10;Deep models \cite{D2Net, LFNet, R2D2, Superglue, DISK, LIFT} learn effective features for dense correspondence that can be robust to photometric and geometric changes such as rotation, scaling, or perspective transformation. &#10;However, most of the methods focus on the image matching task for outdoor scenes to account for rigid transformation.&#10;Amir \etal~\cite{dino-vit-features} demonstrate that features extracted from self-supervised Vision Transformer (DINOv1 \cite{caron2021emerging}) serve as effective dense visual descriptors with &#10;localized semantic information by applying them to a variety of applications, \eg, (part) co-segmentation and semantic correspondences.&#10;Recently, DINOv2 \cite{oquab2023dinov2} introduces a scaled-up version of DINOv1 \cite{caron2021emerging} by increasing the model size and combining a large quantity of curated data.&#10;DINOv2 shows strong generalization to downstream tasks such as classification, segmentation, and monocular depth, but has not been extensively studied on correspondence tasks. Our work shows that DINOv2 also leads to significantly better correspondence results than DINOv1, but those DINO features in general produce sparse and noisy correspondence field.&#10;&#10;\myparagraph{Semantic correspondence.}&#10;Semantic correspondence~\cite{liu2010sift} aims to estimate dense correspondence between objects that belong to the same object class with different appearance, viewpoint, or non-rigid deformation. &#10;Conventional methods consist of three steps~\cite{truong2022probabilistic}: feature extraction, cost volume construction, and displacement field \cite{GLU-Net,GOCor,truong2021learning,truong2021warp} or parameterized transformation regression~\cite{PARN,kim2018recurrent,rocco2017convolutional,rocco2018end,seo2018attentive}.&#10;Those methods, however, cannot effectively determine dense correspondence for complex articulated deformation due to their smooth displacement fields or locally varying parameterized (affine) transformations.&#10;Motivated by a classical congealing method \cite{learned2005data}, a couple of recent methods introduce to learn to align multiple objects in the same class by using pretrained DINOv1 feature \cite{gupta2023asic,NeuralCongealing} or GAN-generated data \cite{Gangealing}.&#10;The methods show the possibility that knowledge learned for different tasks can also be utilized for the dense correspondence task.&#10;However, those models are not effective in handling severe topological changes probably due to their strong rigidity assumption in the object alignment.&#10;In contrasts, we discover that the Stable Diffusion features for the image generation task exhibit great capability for the zero-shot dense correspondence task as well, where challenging articulate, non-rigid deformation exists.&#10;&#10;\myparagraph{Diffusion models.}&#10;Diffusion probabilistic models (diffusion models) \cite{sohl2015deep} and denoising diffusion probabilistic models \cite{ho2020denoising,nichol2021improved} have demonstrated state-of-the-art performance for image generation task \cite{dhariwal2021diffusion,nichol2021glide,rombach2022high,saharia2022palette,saharia2022photorealistic,song2020improved}.&#10;In addition, the diffusion models have been applied to numerous vision tasks, \eg, image segmentation \cite{amit2021segdiff,baranchuklabel,chen2022generalist,jiang2018difnet,tan2022semantic,wolleb2022diffusion}, object detection \cite{chen2022diffusiondet}, and monocular depth estimation \cite{duan2023diffusiondepth,saxena2023monocular}.&#10;Recently, much attention has been made to analyze what pre-trained diffusion models understand and extract its knowledge for single-image tasks, \eg, panoptic segmentation~\cite{odise}, semantic segmentation and depth~\cite{vpd}.&#10;In this paper, we unveil the potential of diffusion features for zero-shot dense correspondence between images.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2407.11503" label="2407.11503">
-        <attvalues>
-          <attvalue for="0" value="Beyond Mask: Rethinking Guidance Types in Few-shot Segmentation" />
-          <attvalue for="1" value="  Existing few-shot segmentation (FSS) methods mainly focus on prototype&#10;feature generation and the query-support matching mechanism. As a crucial&#10;prompt for generating prototype features, the pair of image-mask types in the&#10;support set has become the default setting. However, various types such as&#10;image, text, box, and mask all can provide valuable information regarding the&#10;objects in context, class, localization, and shape appearance. Existing work&#10;focuses on specific combinations of guidance, leading FSS into different&#10;research branches. Rethinking guidance types in FSS is expected to explore the&#10;efficient joint representation of the coupling between the support set and&#10;query set, giving rise to research trends in the weakly or strongly annotated&#10;guidance to meet the customized requirements of practical users. In this work,&#10;we provide the generalized FSS with seven guidance paradigms and develop a&#10;universal vision-language framework (UniFSS) to integrate prompts from text,&#10;mask, box, and image. Leveraging the advantages of large-scale pre-training&#10;vision-language models in textual and visual embeddings, UniFSS proposes&#10;high-level spatial correction and embedding interactive units to overcome the&#10;semantic ambiguity drawbacks typically encountered by pure visual matching&#10;methods when facing intra-class appearance diversities. Extensive experiments&#10;show that UniFSS significantly outperforms the state-of-the-art methods.&#10;Notably, the weakly annotated class-aware box paradigm even surpasses the&#10;finely annotated mask paradigm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.07143" label="2212.07143">
-        <attvalues>
-          <attvalue for="0" value="Reproducible scaling laws for contrastive language-image learning" />
-          <attvalue for="1" value="  Scaling up neural networks has led to remarkable performance across a wide&#10;range of tasks. Moreover, performance often follows reliable scaling laws as a&#10;function of training set size, model size, and compute, which offers valuable&#10;guidance as large-scale experiments are becoming increasingly expensive.&#10;However, previous work on scaling laws has primarily used private data \&amp;&#10;models or focused on uni-modal language or vision learning. To address these&#10;limitations, we investigate scaling laws for contrastive language-image&#10;pre-training (CLIP) with the public LAION dataset and the open-source OpenCLIP&#10;repository. Our large-scale experiments involve models trained on up to two&#10;billion image-text pairs and identify power law scaling for multiple downstream&#10;tasks including zero-shot classification, retrieval, linear probing, and&#10;end-to-end fine-tuning. We find that the training distribution plays a key role&#10;in scaling laws as the OpenAI and OpenCLIP models exhibit different scaling&#10;behavior despite identical model architectures and similar training recipes. We&#10;open-source our evaluation workflow and all models, including the largest&#10;public CLIP models, to ensure reproducibility and make scaling laws research&#10;more accessible. Source code and instructions to reproduce this study will be&#10;available at https://github.com/LAION-AI/scaling-laws-openclip&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.08402" label="2210.08402">
-        <attvalues>
-          <attvalue for="0" value="LAION-5B: An open large-scale dataset for training next generation&#10;  image-text models" />
-          <attvalue for="1" value="  Groundbreaking language-vision architectures like CLIP and DALL-E proved the&#10;utility of training on large amounts of noisy image-text data, without relying&#10;on expensive accurate labels used in standard vision unimodal supervised&#10;learning. The resulting models showed capabilities of strong text-guided image&#10;generation and transfer to downstream tasks, while performing remarkably at&#10;zero-shot classification with noteworthy out-of-distribution robustness. Since&#10;then, large-scale language-vision models like ALIGN, BASIC, GLIDE, Flamingo and&#10;Imagen made further improvements. Studying the training and capabilities of&#10;such models requires datasets containing billions of image-text pairs. Until&#10;now, no datasets of this size have been made openly available for the broader&#10;research community. To address this problem and democratize research on&#10;large-scale multi-modal models, we present LAION-5B - a dataset consisting of&#10;5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English&#10;language. We show successful replication and fine-tuning of foundational models&#10;like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further&#10;experiments enabled with an openly available dataset of this scale.&#10;Additionally we provide several nearest neighbor indices, an improved&#10;web-interface for dataset exploration and subset generation, and detection&#10;scores for watermark, NSFW, and toxic content detection. Announcement page&#10;https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.15343" label="2303.15343">
-        <attvalues>
-          <attvalue for="0" value="Sigmoid Loss for Language Image Pre-Training" />
-          <attvalue for="1" value="  We propose a simple pairwise Sigmoid loss for Language-Image Pre-training&#10;(SigLIP). Unlike standard contrastive learning with softmax normalization, the&#10;sigmoid loss operates solely on image-text pairs and does not require a global&#10;view of the pairwise similarities for normalization. The sigmoid loss&#10;simultaneously allows further scaling up the batch size, while also performing&#10;better at smaller batch sizes. Combined with Locked-image Tuning, with only&#10;four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet&#10;zero-shot accuracy in two days. The disentanglement of the batch size from the&#10;loss further allows us to study the impact of examples vs pairs and negative to&#10;positive ratio. Finally, we push the batch size to the extreme, up to one&#10;million, and find that the benefits of growing batch size quickly diminish,&#10;with a more reasonable batch size of 32k being sufficient. We release our&#10;models at https://github.com/google-research/big_vision and hope our research&#10;motivates further explorations in improving the quality and efficiency of&#10;language-image pre-training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.17425" label="2309.17425">
-        <attvalues>
-          <attvalue for="0" value="Data Filtering Networks" />
-          <attvalue for="1" value="  Large training sets have become a cornerstone of machine learning and are the&#10;foundation for recent advances in language modeling and multimodal learning.&#10;While data curation for pre-training is often still ad-hoc, one common paradigm&#10;is to first collect a massive pool of data from the Web and then filter this&#10;candidate pool down to an actual training set via various heuristics. In this&#10;work, we study the problem of learning a data filtering network (DFN) for this&#10;second step of filtering a large uncurated dataset. Our key finding is that the&#10;quality of a network for filtering is distinct from its performance on&#10;downstream tasks: for instance, a model that performs well on ImageNet can&#10;yield worse training sets than a model with low ImageNet accuracy that is&#10;trained on a small amount of high-quality data. Based on our insights, we&#10;construct new data filtering networks that induce state-of-the-art image-text&#10;datasets. Specifically, our best performing dataset DFN-5B enables us to train&#10;state-of-the-art CLIP models for their compute budgets: among other&#10;improvements on a variety of tasks, a ViT-H trained on our dataset achieves&#10;84.4% zero-shot transfer accuracy on ImageNet, out-performing models trained on&#10;other datasets such as LAION-2B, DataComp-1B, or OpenAI's WIT. In order to&#10;facilitate further research in dataset design, we also release a new 2 billion&#10;example dataset DFN-2B and show that high performance data filtering networks&#10;can be trained from scratch using only publicly available data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.06468" label="2004.06468">
-        <attvalues>
-          <attvalue for="0" value="Self6D: Self-Supervised Monocular 6D Object Pose Estimation" />
-          <attvalue for="1" value="  6D object pose estimation is a fundamental problem in computer vision.&#10;Convolutional Neural Networks (CNNs) have recently proven to be capable of&#10;predicting reliable 6D pose estimates even from monocular images. Nonetheless,&#10;CNNs are identified as being extremely data-driven, and acquiring adequate&#10;annotations is oftentimes very time-consuming and labor intensive. To overcome&#10;this shortcoming, we propose the idea of monocular 6D pose estimation by means&#10;of self-supervised learning, removing the need for real annotations. After&#10;training our proposed network fully supervised with synthetic RGB data, we&#10;leverage recent advances in neural rendering to further self-supervise the&#10;model on unannotated real RGB-D data, seeking for a visually and geometrically&#10;optimal alignment. Extensive evaluations demonstrate that our proposed&#10;self-supervision is able to significantly enhance the model's original&#10;performance, outperforming all other methods relying on synthetic data or&#10;employing elaborate techniques from the domain adaptation realm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;We first introduce recent work in monocular 6D pose estimation. Afterwards, we discuss important methods from neural rendering as they form a core part of our (as well as other) self-supervised learning frameworks. We then outline other successful approaches grounded on self-supervised learning. Lastly, we take a brief look at domain adaptation in the field of 6D pose, since our method can be considered an implicit formulation to close the synthetic-to-real domain gap.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Object Pose Estimation, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2204.07262" label="2204.07262">
-        <attvalues>
-          <attvalue for="0" value="Imposing Consistency for Optical Flow Estimation" />
-          <attvalue for="1" value="  Imposing consistency through proxy tasks has been shown to enhance&#10;data-driven learning and enable self-supervision in various tasks. This paper&#10;introduces novel and effective consistency strategies for optical flow&#10;estimation, a problem where labels from real-world data are very challenging to&#10;derive. More specifically, we propose occlusion consistency and zero forcing in&#10;the forms of self-supervised learning and transformation consistency in the&#10;form of semi-supervised learning. We apply these consistency techniques in a&#10;way that the network model learns to describe pixel-level motions better while&#10;requiring no additional annotations. We demonstrate that our consistency&#10;strategies applied to a strong baseline network model using the original&#10;datasets and labels provide further improvements, attaining the&#10;state-of-the-art results on the KITTI-2015 scene flow benchmark in the&#10;non-stereo category. Our method achieves the best foreground accuracy (4.33% in&#10;Fl-all) over both the stereo and non-stereo categories, even though using only&#10;monocular image inputs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Optical Flow:&#10;Classic solutions have been studied for decades \cite{horn1981determining, brox2004high}, and recent advancements have been made with deep learning methods \cite{dosovitskiy2015flownet, ilg2017flownet, ranjan2017optical, sun2018pwc, zhao2020maskflownet, teed2020raft}. &#10;RAFT~\cite{teed2020raft} demonstrates notable improvement by extracting per-pixel features from the corresponding image pair $(I_{t}, I_{t+1})$, building multi-scale 4-dimensional correlation volumes for all pixel pairs, and iteratively adjust the flow estimates through a refinement module with gated recurrent units (GRUs) \cite{cho2014learning} with repeated lookups in the correlation volume. &#10;The loss is computed between the ground truth optical flow $f(I_t, I_{t+1})$ and the predicted optical flow $\tilde{f^i}(I_t,I_{t+1})$ in each iteration $i$ with $\ell_1$ norm &#10;\begin{equation} &#10;\mathcal{L}_{RAFT} = \sum_{i=1}^N \gamma^{N-i} \| f(I_t, I_{t+1}) - \tilde{f}^{i}(I_t, I_{t+1}) \| _{1},&#10;\end{equation}&#10;where $N$ is the number of GRU iterations and $\gamma$ is a decay factor ($\gamma$ $&lt;$ 1). The final predicted flow is then $\tilde{f}(I_t, I_{t+1}) = \tilde{f}^{N}(I_t, I_{t+1})$, the prediction after all iterations.&#10;\\&#10;Methods for Occlusion Handling: &#10;UnFlow \cite{meister2017unflow} identifies occlusions with the forward-backward constraint assumption \cite{sundaram2010dense} and excludes the occlusion area during training. For the forward-backward constraint, a bidirectional optical flow is required, and the errors could accumulate and propagate, partially due to the discretization of continuous values in the estimates. Self-supervised learning has also been introduced in recent works for optical flow estimation. SelFlow~\cite{liu2019selflow}, as an example, performs flow estimation for non-occluded regions and uses these predictions to estimate flows in occluded regions. However, it requires four optical flow inferences (forward/backward$\times$occlusion/non-occlusion pairs) and significantly increases computational and memory costs to obtain occlusion maps and non-occlusion/occlusion flows. Maskflownet \cite{zhao2020maskflownet} proposes a learnable occlusion mask, which is applied to the next image frame $I_{t+1}$ when calculating the correlation between the features of $I_{t}$ and $I_{t+1}$. Recent studies \cite{hur2019iterative, jonschkowski2020matters} also propose predicting the occlusion mask with an additional channel, and we adopt this approach.&#10;&#10;Another solution is to integrate contextual information. Recently, RAFT~\cite{teed2020raft} presented a context sub-network to incorporate neighborhood pixels' information. By assuming the pixels in an object or segment to have a similar flow, it refines the estimated flow fields in occlusion areas. However, as shown in Fig.~\ref{exp:example_image} (RAFT results), the matched parts can be incorrectly updated in case of severe occlusions. We analyze contextual information in more detail in the following section.&#10;&#10;In contrast to previous algorithms, our method generates occlusion itself and enforces the network to predict the occlusion areas without multiple inferences.&#10;&#10;\\&#10; Contextual Information: Using context to regularize estimations within an image segment can improve optical flow as \cite{teed2020raft} intended with the context sub-network. However, such a regularization needs to be imposed while keeping the degree of occlusion in mind. Figure~\ref{method:occlusion} shows an example. In the case of minor occlusions, most pixels (in green) in a contextual segment (car) are likely to be estimated correctly. Here, the context sub-network may provide adequate support over the refinement iterations. On the other hand, in the case of major occlusions, the dominating portion of the occlusion region (in red) can be biased towards incorrect context, creating possibly significant deterioration in the correspondence estimation. RAFT estimation in Figure~\ref{exp:example_image} gives a real example of this problem occurring under a major occlusion. To tackle this problem, we propose the occlusion consistency strategy, as described in Section \ref{subsec:self}. &#10;\\&#10;Self-Supervised Learning: By defining pretext tasks for unlabeled data and then using them to pretrain models, self-supervision allows making the best use of the unlabeled data and enhancing the performance of the downstream tasks~\cite{gidaris2018unsupervised, zhai2019s4l}. In \cite{zhai2019s4l}, the image is rotated by a random angle, and this angle is predicted. With this auxiliary task of rotation estimation, the network makes room for performance improvement in the original task. However, the use of this auxiliary task is reported to underperform in supervised settings while it performs better in semi-supervised and self-supervised settings \cite{gidaris2018unsupervised, zhai2019s4l}.&#10;\\&#10; Semi-Supervised Learning: Data augmentation with consistency regularization has been popular in semi-supervised learning \cite{laine2016temporal, tarvainen2017mean, oliver2018realistic} where a set of predefined transformations are applied to the original labeled data and the outputs of the perturbed inputs are enforced to agree with the outputs of the original data~\cite{laine2016temporal}. The loss is defined as the mismatch between the outputs for the original and perturbed inputs. It is shown that consistency regularization improves robustness by smoothing the underlying data manifold \cite{oliver2018realistic}. The consistency regularization loss and the supervised loss is often aggregated. Similar ideas are also applied localization problems, and demonstrated better performance \cite{jeong2019consistency, jeong2021interpolation}. In our work, we extend this promising concept to optical flow estimation.&#10;&#10;There have also been studies on semi-supervised optical flow estimation to reduce dependency on the labeled data. In \cite{lai2017semi}, an adversarial learning setup is used where the discriminator learns whether an optical flow is real (by comparison with the ground truth) or generated with a model. In the process of minimizing the discriminator loss, the generator with unlabeled data pairs is trained. In \cite{yan2020optical}, clean images are generated from foggy images, and foggy images are generated from clean images. A model is trained with interchangeable samples among clean and foggy images. These algorithms require additional networks to translate images into flow estimates. In our proposal, we do not require any separate network as a part of our training framework as we derive equivariance-based consistency losses simply by comparing the original pairs with the transformation pairs.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Physics, Optical Flow Estimation, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2406.11570" label="2406.11570">
-        <attvalues>
-          <attvalue for="0" value="Projecting Radiance Fields to Mesh Surfaces" />
-          <attvalue for="1" value="  Radiance fields produce high fidelity images with high rendering speed, but&#10;are difficult to manipulate. We effectively perform avatar texture transfer&#10;across different appearances by combining benefits from radiance fields and&#10;mesh surfaces. We represent the source as a radiance field using 3D Gaussian&#10;Splatter, then project the Gaussians on the target mesh. Our pipeline consists&#10;of Source Preconditioning, Target Vectorization and Texture Projection. The&#10;projection completes in 1.12s in a pure CPU compute, compared to baselines&#10;techniques of Per Face Texture Projection and Ray Casting (31s, 4.1min). This&#10;method lowers the computational requirements, which makes it applicable to a&#10;broader range of devices from low-end mobiles to high end computers.&#10;" />
-          <attvalue for="2" value="&#10;Texture transfer between differently shaped avatars is an important rendering problem. Work on texture transfer includes mesh methods such as using adaptive triangulations of meshes, which completes 11 texture transfers in 38 minutes \cite{schmidt2023surface}; or mapping pixels from the UV parameterization of a surface to the 3D avatar which was used to transfer clothing onto avatars \cite{mir2020learning}; or deep learning methods that perform direct image-to-image translation \cite{shysheya2019textured}.&#10;However, these methods take a long time to process and render the textures.&#10;&#10;To speed up the texture transfer process, radiance field rendering methods using 3D Gaussians (3DG) can be harnessed. 3DG can be used to preserve the continuous properties of the original texture, and avoid unnecessary computation in empty spaces \cite{kerbl20233d}. Most importantly, such a technique preserves rendering quality.&#10;\cite{kerbl20233d} used 3DG as volumetric representation for real-time scene rendering.&#10;\cite{wang20243d} generated 3DG Textures for volumetric texturing in animations. &#10;&#10;In this work, we present a method for avatar texture transfer. The texture transfer is mediated through a Gaussian splattering-based pipeline. &#10;We represent our source radiance field as a 3D Gaussian Splatter, and incorporated local space projection \cite{10.1145/3588028.3603653} to cache and speed up locality projection. We then directly project the Gaussians on the target texture, blending them for colour and lighting. Our technique efficiently completes realistic avatar texture transfer across different appearances in 1.12s.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Physics, Image Processing, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2003.02050" label="2003.02050">
-        <attvalues>
-          <attvalue for="0" value="Learning to Transfer Texture from Clothing Images to 3D Humans" />
-          <attvalue for="1" value="  In this paper, we present a simple yet effective method to automatically&#10;transfer textures of clothing images (front and back) to 3D garments worn on&#10;top SMPL, in real time. We first automatically compute training pairs of images&#10;with aligned 3D garments using a custom non-rigid 3D to 2D registration method,&#10;which is accurate but slow. Using these pairs, we learn a mapping from pixels&#10;to the 3D garment surface. Our idea is to learn dense correspondences from&#10;garment image silhouettes to a 2D-UV map of a 3D garment surface using shape&#10;information alone, completely ignoring texture, which allows us to generalize&#10;to the wide range of web images. Several experiments demonstrate that our model&#10;is more accurate than widely used baselines such as thin-plate-spline warping&#10;and image-to-image translation networks while being orders of magnitude faster.&#10;Our model opens the door for applications such as virtual try-on, and allows&#10;for generation of 3D humans with varied textures which is necessary for&#10;learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.08776" label="1905.08776">
-        <attvalues>
-          <attvalue for="0" value="Textured Neural Avatars" />
-          <attvalue for="1" value="  We present a system for learning full-body neural avatars, i.e. deep networks&#10;that produce full-body renderings of a person for varying body pose and camera&#10;position. Our system takes the middle path between the classical graphics&#10;pipeline and the recent deep learning approaches that generate images of humans&#10;using image-to-image translation. In particular, our system estimates an&#10;explicit two-dimensional texture map of the model surface. At the same time, it&#10;abstains from explicit shape modeling in 3D. Instead, at test time, the system&#10;uses a fully-convolutional network to directly map the configuration of body&#10;feature points w.r.t. the camera to the 2D texture coordinates of individual&#10;pixels in the image frame. We show that such a system is capable of learning to&#10;generate realistic renderings while being trained on videos annotated with 3D&#10;poses and foreground masks. We also demonstrate that maintaining an explicit&#10;texture representation helps our system to achieve better generalization&#10;compared to systems that use direct image-to-image translation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.19441" label="2402.19441">
-        <attvalues>
-          <attvalue for="0" value="3D Gaussian Model for Animation and Texturing" />
-          <attvalue for="1" value="  3D Gaussian Splatting has made a marked impact on neural rendering by&#10;achieving impressive fidelity and performance. Despite this achievement,&#10;however, it is not readily applicable to developing interactive applications.&#10;Real-time applications like XR apps and games require functions such as&#10;animation, UV-mapping, and model editing simultaneously manipulated through the&#10;usage of a 3D model. We propose a modeling that is analogous to typical 3D&#10;models, which we call 3D Gaussian Model (3DGM); it provides a manipulatable&#10;proxy for novel animation and texture transfer. By binding the 3D Gaussians in&#10;texture space and re-projecting them back to world space through implicit shell&#10;mapping, we show how our 3D modeling can serve as a valid rendering methodology&#10;for interactive applications. It is further noted that recently, 3D mesh&#10;reconstruction works have been able to produce high-quality mesh for rendering.&#10;Our work, on the other hand, only requires an approximated geometry for&#10;rendering an object in high fidelity. Applicationwise, we will show that our&#10;proxy-based 3DGM is capable of driving novel animation without animated&#10;training data and texture transferring via UV mapping of the 3D Gaussians. We&#10;believe the result indicates the potential of our work for enabling interactive&#10;applications for 3D Gaussian Splatting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.04123" label="2011.04123">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning based Monocular Depth Prediction: Datasets, Methods and&#10;  Applications" />
-          <attvalue for="1" value="  Estimating depth from RGB images can facilitate many computer vision tasks,&#10;such as indoor localization, height estimation, and simultaneous localization&#10;and mapping (SLAM). Recently, monocular depth estimation has obtained great&#10;progress owing to the rapid development of deep learning techniques. They&#10;surpass traditional machine learning-based methods by a large margin in terms&#10;of accuracy and speed. Despite the rapid progress in this topic, there are&#10;lacking of a comprehensive review, which is needed to summarize the current&#10;progress and provide the future directions. In this survey, we first introduce&#10;the datasets for depth estimation, and then give a comprehensive introduction&#10;of the methods from three perspectives: supervised learning-based methods,&#10;unsupervised learning-based methods, and sparse samples guidance-based methods.&#10;In addition, downstream applications that benefit from the progress have also&#10;been illustrated. Finally, we point out the future directions and conclude the&#10;paper.&#10;" />
-          <attvalue for="2" value="&#10;&#09;Depth estimation from monocular images is an important research topic in the photometric computer vision community. It aims to produce pixel-wise depth maps from images at certain view perspectives. Such depth information helps better understand 3D scenes and also facilitates many computer vision tasks such as indoor localization \cite{li20203d}, height estimation \cite{amirkolaee2019height}, simultaneous localization and mapping (SLAM) \cite{tateno2017cnn}, visual odometry \cite{yang2018deep}, classification \cite{he2017estimated}, etc. Usually, the depth information is obtained through commercial depth sensors such as various LiDAR devices and Kinects. However, apart from the high cost and requirement of operational skill, they also confront the disadvantages of low resolution and short perception distance, which limit extensive applications. Inferring depth map from monocular images attracts increasing interests due to the wide availability of RGB images. However, it is a challenging ill-posed problem as it is inherently scale ambiguous, which means that an infinite number of possible depth maps can be associated with an image. &#10;&#10;Depth cues in monocular images have been exploited as hints for depth estimation. Guo et al. leverage the hazing generation principle for depth estimation \cite{guo2014adaptive}. They simulate a hazed image by adding a haze veil on the input image and use a haze removal algorithm to estimate the depth map. Imaging principle are exploited for depth estimation in \cite{tang2015depth}, where scene at the focus distance are clear while the others are blur. They first predict depth at the edge positions and recover the full depth map with depth matting optimization. Occlusion reasoning and vanish lines and points are leveraged to generate depth map indirectly in \cite{hoiem2007recovering,tsai2006block-based}. Though these methods have been proven to be effective in their experiments, they are not widely applied since the depth cues are not always presented in normal images.&#10;&#10;Another direction for depth estimation is to utilize machine learning techniques. They can be categorized into two groups: non-parametric methods and learning-based methods. Non-parametric methods exploit vast available RGBD datasets and infer the target depth from the multiple reference depth images. The reference depths are determined according to the image similarity of RGB domain between targeted image and the rest images. Konrad et al. \cite{konrad2013learning} search $K$ candidate depth image based on the histograms of oriented features. They generate the initial depth with median operator and refine it with cross-bilateral filtering. GIST feature and optical flow feature are exploited to find $K$ similar images and produce the initial depth map through a warping procedure based on SIFT flow \cite{karsch2014depth}. A global optimization is conducted to smooth and interpolate the warped initial depth. Herrera et al. \cite{herrera2014learning} retrieve candidate depth images using LBP feature and the preliminary depth is produced through weighted combination of depth candidates and refined with the joint bilateral filtering to enforce the depth consistency. Structure information is believed to be highly related with depth information in \cite{herrera2018automatic}. Besides, the candidate depth images are found based on structure similarity in colour domain. The prior depth map is obtained through the weighted average fusion with retrieved candidates. The final depth map is optimized with segmentation guided filtering. Kong et al. \cite{kong2015intrinsic} also retrieve $K$ candidate video clips with GIST features and generate the preliminary depth map through sift flow warping. They finally enhance the depth boundary and contours using albedo and shading as well as the regularizer of output structure from motion. Choi et al. \cite{choi2015depth} retrieve $K$ RGB-D image pairs and generate $K$ depth gradient samples by constructing the correspondences between retrieved images. The initial depth map is generated through Poisson equations. The refined depth map is obtained using edge-aware median filter. Xu et al. \cite{xu2019depth} exploit the label transfer in gradient domain. They fused the candidate depth gradients through confidence analysis to generate the initial gradient depth map and refine it based on image boundaries. Finally, the depth map is constructed through Poisson function and smoothed with edge-aware filter. Non-parametric methods is of low efficiency since they have to search candidates images from large image databases.&#10;&#10;Learning-based methods directly estimate the depth value from various features and usually incorporate with graph-based models. Saxena et al. \cite{saxena2005learning} directly predict the depth values from multi-scale local feature and global feature, and depth relations between different locations are modelled with Markov random field. They further refine the approach with regions under the assumption that each superpixel corresponds to the single value. Liu et al. \cite{liu2010single} incorporate semantic information with local pixel appearance for supervised depth prediction. Ladický et al. \cite{ladick2014pulling} also predict the depth with semantic and visual features and they simplify the classifiers by introducing the constraints of perspective geometry. Cao et al. \cite{cao2010a} utilize an iterative framework for depth estimation and image segmentation. They first generate an initial image segmentation results and then utilize a multi-scale Markov random field to infer depth. Given the initial depth map, they refine the segmentation results based on a graph-based merging algorithm and the final depth is refined with the new segmentation map. Zhuo et al. \cite{zhuo2015indoor} estimate the depth from the local and global structure of the scenes and fuse the feature in a continuous conditional random field (CRF) for the depth prediction. Liu et al. \cite{liu2014discrete} cast depth prediction as a discrete-continuous problem. Depth in the same superpixels are treated as continuous while depth between superpixels are believed to discrete. &#10;&#10;Inspired by the powerful performance of deep learning on image classification, object detection, and semantic segmentation, many researchers attempt to tackle monocular depth prediction with deep learning techniques. Various network architecture based on convolutional neural network (CNN) \cite{liu2010single}, recurrent neural network (RNN) \cite{wang2019recurrent} and generative adversarial network (GAN) \cite{almalioglu2019ganvo:} have been proposed to exploit RGB images. Superior performances have been achieved compared with traditional methods. However, comprehensive survey is lacking to summarize the current progress and provide some hints to future directions. To bridge the gap, in this paper, we overview the monocular depth prediction approaches based on deep learning from four aspects: benchmark datasets, supervised methods, unsupervised methods, and methods with sparse depth points guidance. We also discuss the future direction and trends. &#10;&#10;The rest paper is organized as follows. Section \ref{sec:dataset} introduces the widely used indoor and outdoor benchmarks and the associated evaluation metrics in monocular depth estimation. Section \ref{sec:sup} and Section \ref{sec:unsup} review the supervised and unsupervised methods for depth prediction respectively. We also review recent works in depth prediction with sparse points in section\ref{sec:sparse}. Current challenges and promising directions are discussed in Section \ref{sec:fut}. Finally, Section \ref{sec:con} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1704.03489" label="1704.03489">
-        <attvalues>
-          <attvalue for="0" value="CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction" />
-          <attvalue for="1" value="  Given the recent advances in depth prediction from Convolutional Neural&#10;Networks (CNNs), this paper investigates how predicted depth maps from a deep&#10;neural network can be deployed for accurate and dense monocular reconstruction.&#10;We propose a method where CNN-predicted dense depth maps are naturally fused&#10;together with depth measurements obtained from direct monocular SLAM. Our&#10;fusion scheme privileges depth prediction in image locations where monocular&#10;SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa.&#10;We demonstrate the use of depth prediction for estimating the absolute scale of&#10;the reconstruction, hence overcoming one of the major limitations of monocular&#10;SLAM. Finally, we propose a framework to efficiently fuse semantic labels,&#10;obtained from a single frame, with dense SLAM, yielding semantically coherent&#10;scene reconstruction from a single view. Evaluation results on two benchmark&#10;datasets show the robustness and accuracy of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.02570" label="1807.02570">
-        <attvalues>
-          <attvalue for="0" value="Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for&#10;  Monocular Direct Sparse Odometry" />
-          <attvalue for="1" value="  Monocular visual odometry approaches that purely rely on geometric cues are&#10;prone to scale drift and require sufficient motion parallax in successive&#10;frames for motion estimation and 3D reconstruction. In this paper, we propose&#10;to leverage deep monocular depth prediction to overcome limitations of&#10;geometry-based monocular visual odometry. To this end, we incorporate deep&#10;depth predictions into Direct Sparse Odometry (DSO) as direct virtual stereo&#10;measurements. For depth prediction, we design a novel deep network that refines&#10;predicted depth from a single image in a two-stage process. We train our&#10;network in a semi-supervised way on photoconsistency in stereo images and on&#10;consistency with accurate sparse depth reconstructions from Stereo DSO. Our&#10;deep predictions excel state-of-the-art approaches for monocular depth on the&#10;KITTI benchmark. Moreover, our Deep Virtual Stereo Odometry clearly exceeds&#10;previous monocular and deep learning based methods in accuracy. It even&#10;achieves comparable performance to the state-of-the-art stereo methods, while&#10;only relying on a single camera.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.07077" label="1709.07077">
-        <attvalues>
-          <attvalue for="0" value="Estimated Depth Map Helps Image Classification" />
-          <attvalue for="1" value="  We consider image classification with estimated depth. This problem falls&#10;into the domain of transfer learning, since we are using a model trained on a&#10;set of depth images to generate depth maps (additional features) for use in&#10;another classification problem using another disjoint set of images. It's&#10;challenging as no direct depth information is provided. Though depth estimation&#10;has been well studied, none have attempted to aid image classification with&#10;estimated depth. Therefore, we present a way of transferring domain knowledge&#10;on depth estimation to a separate image classification task over a disjoint set&#10;of train, and test data. We build a RGBD dataset based on RGB dataset and do&#10;image classification on it. Then evaluation the performance of neural networks&#10;on the RGBD dataset compared to the RGB dataset. From our experiments, the&#10;benefit is significant with shallow and deep networks. It improves ResNet-20 by&#10;0.55% and ResNet-56 by 0.53%. Our code and dataset are available publicly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.05786" label="1809.05786">
-        <attvalues>
-          <attvalue for="0" value="GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation&#10;  with Generative Adversarial Networks" />
-          <attvalue for="1" value="  In the last decade, supervised deep learning approaches have been extensively&#10;employed in visual odometry (VO) applications, which is not feasible in&#10;environments where labelled data is not abundant. On the other hand,&#10;unsupervised deep learning approaches for localization and mapping in unknown&#10;environments from unlabelled data have received comparatively less attention in&#10;VO research. In this study, we propose a generative unsupervised learning&#10;framework that predicts 6-DoF pose camera motion and monocular depth map of the&#10;scene from unlabelled RGB image sequences, using deep convolutional Generative&#10;Adversarial Networks (GANs). We create a supervisory signal by warping view&#10;sequences and assigning the re-projection minimization to the objective loss&#10;function that is adopted in multi-view pose estimation and single-view depth&#10;generation network. Detailed quantitative and qualitative evaluations of the&#10;proposed framework on the KITTI and Cityscapes datasets show that the proposed&#10;method outperforms both existing traditional and unsupervised deep VO methods&#10;providing better results for both pose estimation and depth recovery.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.00515" label="2007.00515">
-        <attvalues>
-          <attvalue for="0" value="Learning unbiased zero-shot semantic segmentation networks via&#10;  transductive transfer" />
-          <attvalue for="1" value="  Semantic segmentation, which aims to acquire a detailed understanding of&#10;images, is an essential issue in computer vision. However, in practical&#10;scenarios, new categories that are different from the categories in training&#10;usually appear. Since it is impractical to collect labeled data for all&#10;categories, how to conduct zero-shot learning in semantic segmentation&#10;establishes an important problem. Although the attribute embedding of&#10;categories can promote effective knowledge transfer across different&#10;categories, the prediction of segmentation network reveals obvious bias to seen&#10;categories. In this paper, we propose an easy-to-implement transductive&#10;approach to alleviate the prediction bias in zero-shot semantic segmentation.&#10;Our method assumes that both the source images with full pixel-level labels and&#10;unlabeled target images are available during training. To be specific, the&#10;source images are used to learn the relationship between visual images and&#10;semantic embeddings, while the target images are used to alleviate the&#10;prediction bias towards seen categories. We conduct comprehensive experiments&#10;on diverse split s of the PASCAL dataset. The experimental results clearly&#10;demonstrate the effectiveness of our method.&#10;" />
-          <attvalue for="2" value="&#10;&#10;W{ith} the rapid development of deep learning, semantic segmentation has achieved great advances over the recent years. To train a good semantic segmentation model, we need a large amount of images with full pixel-level labels. However, it is impractical to collect labeled data for all categories~\cite{DBLP:conf/cvpr/ArdeshirCS15,DBLP:conf/cvpr/LiuH15,DBLP:conf/cvpr/BanicaS15}. In practical scenarios, new categories that are different from the categories in training usually appear. In this case, semantic segmentation neural networks struggle to make correct predictions for them.&#10;&#10;Zero-Shot Learning (ZSL) is mainly studied in the context of image recognition. It aims to recognize unseen (target) categories by transferring knowledge from seen (source) categories~\cite{DBLP:conf/cvpr/KodirovXG17,DBLP:conf/cvpr/XianLSA18,DBLP:conf/cvpr/PaulKM19}. For zero-shot semantic segmentation, the previous methods propose to incorporate semantic embeddings of categories to semantic segmentation neural networks~\cite{DBLP:conf/cvpr/XianC0SA19}.&#10;&#10;However, as shown in Fig. \ref{intro}, the existing zero-shot semantic segmentation approaches reveal strong bias towards seen categories in Generalized ZSL(GZSL)~\cite{DBLP:conf/cvpr/SongSYLS18}. Since the relationship between visual images and semantic embeddings is entirely captured over the source data, the visual instances will show obvious bias to source categories. Hence, the semantic segmentation model tends to map the pixels of unseen images to the semantic embeddings of source categories and obtains poor generalization performance~\cite{DBLP:journals/pami/ChenPKMY18}. In this paper, we propose an easy-to-implement transductive approach to alleviate the prediction bias in zero-shot semantic segmentation~\cite{DBLP:conf/cvpr/LongSD15}. Our method assumes that both the source images with full pixel-level labels and unlabeled target images are available during training. It should be noted that our method is also useful for the conventional ZSL setting since it can regularize the segmentation model to make more reasonable predictions for target images, which produces gradients to make the semantic segmentation neural network better fit them~\cite{DBLP:journals/corr/abs-1906-00817}.&#10;&#10;Overall, the main contribution of this work is three-fold:&#10;\begin{itemize}&#10;\item We propose to introduce unlabeled target images in zero-shot semantic segmentation to alleviate the prediction bias towards source categories. To the best of our knowledge, this is the first work to conduct transductive learning in zero-shot semantic segmentation.&#10;\item We propose to alleviate the prediction bias through explicitly projecting the pixels of target images to other points in the semantic embedding space. Our proposed method is both effective and easy-to-implement.&#10;\item We conduct comprehensive experiments on diverse splits of the PASCAL dataset. The experimental results clearly demonstrate the effectiveness of our method.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Zero-Shot Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1704.08345" label="1704.08345">
-        <attvalues>
-          <attvalue for="0" value="Semantic Autoencoder for Zero-Shot Learning" />
-          <attvalue for="1" value="  Existing zero-shot learning (ZSL) models typically learn a projection&#10;function from a feature space to a semantic embedding space (e.g.~attribute&#10;space). However, such a projection function is only concerned with predicting&#10;the training seen class semantic representation (e.g.~attribute prediction) or&#10;classification. When applied to test data, which in the context of ZSL contains&#10;different (unseen) classes without training data, a ZSL model typically suffers&#10;from the project domain shift problem. In this work, we present a novel&#10;solution to ZSL based on learning a Semantic AutoEncoder (SAE). Taking the&#10;encoder-decoder paradigm, an encoder aims to project a visual feature vector&#10;into the semantic space as in the existing ZSL models. However, the decoder&#10;exerts an additional constraint, that is, the projection/code must be able to&#10;reconstruct the original visual feature. We show that with this additional&#10;reconstruction constraint, the learned projection function from the seen&#10;classes is able to generalise better to the new unseen classes. Importantly,&#10;the encoder and decoder are linear and symmetric which enable us to develop an&#10;extremely efficient learning algorithm. Extensive experiments on six benchmark&#10;datasets demonstrate that the proposed SAE outperforms significantly the&#10;existing ZSL models with the additional benefit of lower computational cost.&#10;Furthermore, when the SAE is applied to supervised clustering problem, it also&#10;beats the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.00981" label="1712.00981">
-        <attvalues>
-          <attvalue for="0" value="Feature Generating Networks for Zero-Shot Learning" />
-          <attvalue for="1" value="  Suffering from the extreme training data imbalance between seen and unseen&#10;classes, most of existing state-of-the-art approaches fail to achieve&#10;satisfactory results for the challenging generalized zero-shot learning task.&#10;To circumvent the need for labeled examples of unseen classes, we propose a&#10;novel generative adversarial network (GAN) that synthesizes CNN features&#10;conditioned on class-level semantic information, offering a shortcut directly&#10;from a semantic descriptor of a class to a class-conditional feature&#10;distribution. Our proposed approach, pairing a Wasserstein GAN with a&#10;classification loss, is able to generate sufficiently discriminative CNN&#10;features to train softmax classifiers or any multimodal embedding method. Our&#10;experimental results demonstrate a significant boost in accuracy over the state&#10;of the art on five challenging datasets -- CUB, FLO, SUN, AWA and ImageNet --&#10;in both the zero-shot learning and generalized zero-shot learning settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.07659" label="1904.07659">
-        <attvalues>
-          <attvalue for="0" value="Semantically Aligned Bias Reducing Zero Shot Learning" />
-          <attvalue for="1" value="  Zero shot learning (ZSL) aims to recognize unseen classes by exploiting&#10;semantic relationships between seen and unseen classes. Two major problems&#10;faced by ZSL algorithms are the hubness problem and the bias towards the seen&#10;classes. Existing ZSL methods focus on only one of these problems in the&#10;conventional and generalized ZSL setting. In this work, we propose a novel&#10;approach, Semantically Aligned Bias Reducing (SABR) ZSL, which focuses on&#10;solving both the problems. It overcomes the hubness problem by learning a&#10;latent space that preserves the semantic relationship between the labels while&#10;encoding the discriminating information about the classes. Further, we also&#10;propose ways to reduce the bias of the seen classes through a simple&#10;cross-validation process in the inductive setting and a novel weak transfer&#10;constraint in the transductive setting. Extensive experiments on three&#10;benchmark datasets suggest that the proposed model significantly outperforms&#10;existing state-of-the-art algorithms by ~1.5-9% in the conventional ZSL setting&#10;and by ~2-14% in the generalized ZSL for both the inductive and transductive&#10;settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.11320" label="1803.11320">
-        <attvalues>
-          <attvalue for="0" value="Transductive Unbiased Embedding for Zero-Shot Learning" />
-          <attvalue for="1" value="  Most existing Zero-Shot Learning (ZSL) methods have the strong bias problem,&#10;in which instances of unseen (target) classes tend to be categorized as one of&#10;the seen (source) classes. So they yield poor performance after being deployed&#10;in the generalized ZSL settings. In this paper, we propose a straightforward&#10;yet effective method named Quasi-Fully Supervised Learning (QFSL) to alleviate&#10;the bias problem. Our method follows the way of transductive learning, which&#10;assumes that both the labeled source images and unlabeled target images are&#10;available for training. In the semantic embedding space, the labeled source&#10;images are mapped to several fixed points specified by the source categories,&#10;and the unlabeled target images are forced to be mapped to other points&#10;specified by the target categories. Experiments conducted on AwA2, CUB and SUN&#10;datasets demonstrate that our method outperforms existing state-of-the-art&#10;approaches by a huge margin of 9.3~24.5% following generalized ZSL settings,&#10;and by a large margin of 0.2~16.2% following conventional ZSL settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.00915" label="1606.00915">
-        <attvalues>
-          <attvalue for="0" value="DeepLab: Semantic Image Segmentation with Deep Convolutional Nets,&#10;  Atrous Convolution, and Fully Connected CRFs" />
-          <attvalue for="1" value="  In this work we address the task of semantic image segmentation with Deep&#10;Learning and make three main contributions that are experimentally shown to&#10;have substantial practical merit. First, we highlight convolution with&#10;upsampled filters, or 'atrous convolution', as a powerful tool in dense&#10;prediction tasks. Atrous convolution allows us to explicitly control the&#10;resolution at which feature responses are computed within Deep Convolutional&#10;Neural Networks. It also allows us to effectively enlarge the field of view of&#10;filters to incorporate larger context without increasing the number of&#10;parameters or the amount of computation. Second, we propose atrous spatial&#10;pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP&#10;probes an incoming convolutional feature layer with filters at multiple&#10;sampling rates and effective fields-of-views, thus capturing objects as well as&#10;image context at multiple scales. Third, we improve the localization of object&#10;boundaries by combining methods from DCNNs and probabilistic graphical models.&#10;The commonly deployed combination of max-pooling and downsampling in DCNNs&#10;achieves invariance but has a toll on localization accuracy. We overcome this&#10;by combining the responses at the final DCNN layer with a fully connected&#10;Conditional Random Field (CRF), which is shown both qualitatively and&#10;quantitatively to improve localization performance. Our proposed &quot;DeepLab&quot;&#10;system sets the new state-of-art at the PASCAL VOC-2012 semantic image&#10;segmentation task, reaching 79.7% mIOU in the test set, and advances the&#10;results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and&#10;Cityscapes. All of our code is made publicly available online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.00817" label="1906.00817">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation models are limited in their ability to scale to large&#10;numbers of object classes. In this paper, we introduce the new task of&#10;zero-shot semantic segmentation: learning pixel-wise classifiers for never-seen&#10;object categories with zero training examples. To this end, we present a novel&#10;architecture, ZS3Net, combining a deep visual segmentation model with an&#10;approach to generate visual representations from semantic word embeddings. By&#10;this way, ZS3Net addresses pixel classification tasks where both seen and&#10;unseen categories are faced at test time (so called &quot;generalized&quot; zero-shot&#10;classification). Performance is further improved by a self-training step that&#10;relies on automatic pseudo-labeling of pixels from unseen classes. On the two&#10;standard segmentation datasets, Pascal-VOC and Pascal-Context, we propose&#10;zero-shot benchmarks and set competitive baselines. For complex scenes as ones&#10;in the Pascal-Context dataset, we extend our approach by using a graph-context&#10;encoding to fully leverage spatial context priors coming from class-wise&#10;segmentation maps.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.01837" label="2007.01837">
-        <attvalues>
-          <attvalue for="0" value="LOOC: Localize Overlapping Objects with Count Supervision" />
-          <attvalue for="1" value="  Acquiring count annotations generally requires less human effort than&#10;point-level and bounding box annotations. Thus, we propose the novel problem&#10;setup of localizing objects in dense scenes under this weaker supervision. We&#10;propose LOOC, a method to Localize Overlapping Objects with Count supervision.&#10;We train LOOC by alternating between two stages. In the first stage, LOOC&#10;learns to generate pseudo point-level annotations in a semi-supervised manner.&#10;In the second stage, LOOC uses a fully-supervised localization method that&#10;trains on these pseudo labels. The localization method is used to progressively&#10;improve the quality of the pseudo labels. We conducted experiments on popular&#10;counting datasets. For localization, LOOC achieves a strong new baseline in the&#10;novel problem setup where only count supervision is available. For counting,&#10;LOOC outperforms current state-of-the-art methods that only use count as their&#10;supervision. Code is available at: https://github.com/ElementAI/looc.&#10;" />
-          <attvalue for="2" value="&#10;&#10;We consider the task where a model has to predict the location of each object in an image. This task is important for applications such as public safety, crowd monitoring, and traffic management. Typically, bounding boxes~\cite{ren2015faster, redmon2016you} or point-level annotations~\cite{laradji2018blobs, laradji2019instance, li2018csrnet, lempitsky2010learning, zhang2016single, onoro2016towards, cholakkal2019object} are provided during training. However, we consider the more challenging problem setup where only count-level annotations are available. These labels are cheaper to acquire than point-level annotations, but they make the localization task significantly more difficult for the model. In dense scenes, the model has to identify which objects in the image correspond to the object count. These objects can heavily overlap, can vary widely in scale, shape, and appearance. Current methods~\cite{Gao2018cwsl} partially address this problem setup but only for datasets where objects are salient and rarely overlap. These methods do not work for dense scene datasets as they are designed to work with training images that have few objects. Thus, we address a novel problem setup of learning to localize objects for dense scenes under count supervision.&#10;&#10;Acquiring object count labels in images requires much less human effort than annotating the location of each object. For training images with $4$ or less objects, the annotator can obtain the object count much faster than with point annotations through subitizing~\cite{chattopadhyay2016counting}. For videos, the annotator can obtain the object count quickly across image frames as the count changes much less frequently than the object locations in the video. In some cases, object counts can be obtained with no effort compared to point-level annotations. These cases include keeping count of products on retail stock shelves, and keeping count of a crowd of people at events where the ticket system registers their actual count. In both cases, identifying object locations is important for safety and logistics.&#10;&#10;Many methods exist that can perform object localization but they need to be trained on point-level annotations~\cite{lempitsky2010learning, laradji2018blobs, li2018csrnet, laradji2019instance} or image-level~\cite{laradji2019masks}. They fall under two main categories: density-based and segmentation-based localization. Density-based methods~\cite{lempitsky2010learning, li2018csrnet} transform the point-level annotations into a density map using a Gaussian kernel. Then, they train using a least-squares objective to predict the density map. However, these methods do not provide individual locations of the objects. On the other hand, segmentation-based methods such as LC-FCN~\cite{laradji2018blobs} train using a loss function that encourages the output to contain a single blob per object. For our framework, we use the individual object locations obtained by LC-FCN to help in generating the pseudo point-level annotations.&#10;&#10;Glance~\cite{chattopadhyay2016counting} is a state-of-the-art method for object counting when only count supervision is provided. This method is an ImageNet~\cite{deng2009imagenet} pre-trained model such as ResNet50~\cite{he2016deep} with a regression layer as its output layer. Unfortunately, Glance is not designed to localize the objects of interest in the image. In this work, we propose a novel approach that uses count supervision to localize objects in dense scenes. Further, we show that our method achieves better count results than Glance.&#10;&#10;Most weakly supervised localization methods fall under multiple-instance learning (MIL)~\cite{MIL}. In this setup, each image corresponds to a bag of object proposals. Each bag is labeled based on whether an object class exists. \cite{Li2016WeaklySO} present a two-step approach. First, they use a mask-out strategy to filter the noisy object proposals; then, they use a Faster RCNN~\cite{ren2015faster} for detection using bags of instances. \cite{tang2017multiple} use a refinement learning strategy to improve on the quality of the proposals. C-MIL~\cite{Wan2019CMILCM} introduces a continuation optimization method to avoid getting stuck in a local minima. C-WSL~\cite{Gao2018cwsl} is the most relevant to our work as they use count information to obtain the highest scoring proposals. However, it differs from our setup in that it relies on a classification network that is not designed for dense scenes.&#10;&#10;We propose LOOC which can learn to Localize Overlapping Objects with Count supervision. It trains by alternating between two stages. In the first stage, LOOC learns to generate pseudo point-level annotations in a semi-supervised learning manner. In the second stage, LOOC uses a fully-supervised localization method that trains on these pseudo labels. &#10;&#10;As in Figure~\ref{fig:PN}, the pseudo labels are the centroids of the highest scoring proposals generated using a standard proposal method such as selective search~\cite{uijlings2013selective}. This set of scores is the combination of the proposal objectness and the probability heat-map obtained from the trained localization method. The proposals that have low scores are considered unlabeled. The localization method uses the pseudo labels and ignores the regions that are unlabeled. The goal for the localization method is to infer the object probabilities in these unlabeled regions. These probabilities are used to re-score the proposals to generate the pseudo labels in the next round~\ref{fig:looc}. At test time, only the localization method is kept, which can be directly used to predict the locations and count of the objects of interest.&#10;&#10;Since no direct relevant work exists for this particular setup, we compare our methods against Glance~\cite{chattopadhyay2016counting} and the fully supervised LCFCN~\cite{laradji2018blobs}. We benchmark our methods against various counting datasets such as Trancos~\cite{guerrero2015Trancos}, Penguins~\cite{arteta2016counting}, UCSD~\cite{chan2008privacy}, and Mall~\cite{chen2012feature}. We observed that LOOC achieves a strong new baseline in the novel problem setup where only count supervision is available with respect to localization. Further, we observed that LOOC outperforms current state-of-the-art methods that only use count as their supervision. &#10;&#10;We summarize our contributions as follows: we (1) present LOOC, a novel framework that can count and locate objects with count-level supervision for dense scenes; (2) propose a semi-supervised learning scheme where pseudo labels are inferred for unlabeled regions in the image, and (3) show that LOOC achieves better count accuracy than Glance with the addition that it locates objects efficiently. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Weak Supervision, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Object Localization" />
-        </attvalues>
-      </node>
-      <node id="1807.09856" label="1807.09856">
-        <attvalues>
-          <attvalue for="0" value="Where are the Blobs: Counting by Localization with Point Supervision" />
-          <attvalue for="1" value="  Object counting is an important task in computer vision due to its growing&#10;demand in applications such as surveillance, traffic monitoring, and counting&#10;everyday objects. State-of-the-art methods use regression-based optimization&#10;where they explicitly learn to count the objects of interest. These often&#10;perform better than detection-based methods that need to learn the more&#10;difficult task of predicting the location, size, and shape of each object.&#10;However, we propose a detection-based method that does not need to estimate the&#10;size and shape of the objects and that outperforms regression-based methods.&#10;Our contributions are three-fold: (1) we propose a novel loss function that&#10;encourages the network to output a single blob per object instance using&#10;point-level annotations only; (2) we design two methods for splitting large&#10;predicted blobs between object instances; and (3) we show that our method&#10;achieves new state-of-the-art results on several challenging datasets including&#10;the Pascal VOC and the Penguins dataset. Our method even outperforms those that&#10;use stronger supervision such as depth features, multi-point annotations, and&#10;bounding-box labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.05282" label="1711.05282">
-        <attvalues>
-          <attvalue for="0" value="C-WSL: Count-guided Weakly Supervised Localization" />
-          <attvalue for="1" value="  We introduce count-guided weakly supervised localization (C-WSL), an approach&#10;that uses per-class object count as a new form of supervision to improve weakly&#10;supervised localization (WSL). C-WSL uses a simple count-based region selection&#10;algorithm to select high-quality regions, each of which covers a single object&#10;instance during training, and improves existing WSL methods by training with&#10;the selected regions. To demonstrate the effectiveness of C-WSL, we integrate&#10;it into two WSL architectures and conduct extensive experiments on VOC2007 and&#10;VOC2012. Experimental results show that C-WSL leads to large improvements in&#10;WSL and that the proposed approach significantly outperforms the&#10;state-of-the-art methods. The results of annotation experiments on VOC2007&#10;suggest that a modest extra time is needed to obtain per-class object counts&#10;compared to labeling only object categories in an image. Furthermore, we reduce&#10;the annotation time by more than $2\times$ and $38\times$ compared to&#10;center-click and bounding-box annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.03505" label="1604.03505">
-        <attvalues>
-          <attvalue for="0" value="Counting Everyday Objects in Everyday Scenes" />
-          <attvalue for="1" value="  We are interested in counting the number of instances of object classes in&#10;natural, everyday images. Previous counting approaches tackle the problem in&#10;restricted domains such as counting pedestrians in surveillance videos. Counts&#10;can also be estimated from outputs of other vision tasks like object detection.&#10;In this work, we build dedicated models for counting designed to tackle the&#10;large variance in counts, appearances, and scales of objects found in natural&#10;scenes. Our approach is inspired by the phenomenon of subitizing - the ability&#10;of humans to make quick assessments of counts given a perceptual signal, for&#10;small count values. Given a natural scene, we employ a divide and conquer&#10;strategy while incorporating context across the scene to adapt the subitizing&#10;idea to counting. Our approach offers consistent improvements over numerous&#10;baseline approaches for counting on the PASCAL VOC 2007 and COCO datasets.&#10;Subsequently, we study how counting can be used to improve object detection. We&#10;then show a proof of concept application of our counting methods to the task of&#10;Visual Question Answering, by studying the `how many?' questions in the VQA and&#10;COCO-QA datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.01430" label="1907.01430">
-        <attvalues>
-          <attvalue for="0" value="Where are the Masks: Instance Segmentation with Image-level Supervision" />
-          <attvalue for="1" value="  A major obstacle in instance segmentation is that existing methods often need&#10;many per-pixel labels in order to be effective. These labels require large&#10;human effort and for certain applications, such labels are not readily&#10;available. To address this limitation, we propose a novel framework that can&#10;effectively train with image-level labels, which are significantly cheaper to&#10;acquire. For instance, one can do an internet search for the term &quot;car&quot; and&#10;obtain many images where a car is present with minimal effort. Our framework&#10;consists of two stages: (1) train a classifier to generate pseudo masks for the&#10;objects of interest; (2) train a fully supervised Mask R-CNN on these pseudo&#10;masks. Our two main contribution are proposing a pipeline that is simple to&#10;implement and is amenable to different segmentation methods; and achieves new&#10;state-of-the-art results for this problem setup. Our results are based on&#10;evaluating our method on PASCAL VOC 2012, a standard dataset for weakly&#10;supervised methods, where we demonstrate major performance gains compared to&#10;existing methods with respect to mean average precision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05647" label="1904.05647">
-        <attvalues>
-          <attvalue for="0" value="C-MIL: Continuation Multiple Instance Learning for Weakly Supervised&#10;  Object Detection" />
-          <attvalue for="1" value="  Weakly supervised object detection (WSOD) is a challenging task when provided&#10;with image category supervision but required to simultaneously learn object&#10;locations and object detectors. Many WSOD approaches adopt multiple instance&#10;learning (MIL) and have non-convex loss functions which are prone to get stuck&#10;into local minima (falsely localize object parts) while missing full object&#10;extent during training. In this paper, we introduce a continuation optimization&#10;method into MIL and thereby creating continuation multiple instance learning&#10;(C-MIL), with the intention of alleviating the non-convexity problem in a&#10;systematic way. We partition instances into spatially related and class related&#10;subsets, and approximate the original loss function with a series of smoothed&#10;loss functions defined within the subsets. Optimizing smoothed loss functions&#10;prevents the training procedure falling prematurely into local minima and&#10;facilitates the discovery of Stable Semantic Extremal Regions (SSERs) which&#10;indicate full object extent. On the PASCAL VOC 2007 and 2012 datasets, C-MIL&#10;improves the state-of-the-art of weakly supervised object detection and weakly&#10;supervised object localization with large margins.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.05553" label="2203.05553">
-        <attvalues>
-          <attvalue for="0" value="Transfer of Representations to Video Label Propagation: Implementation&#10;  Factors Matter" />
-          <attvalue for="1" value="  This work studies feature representations for dense label propagation in&#10;video, with a focus on recently proposed methods that learn video&#10;correspondence using self-supervised signals such as colorization or temporal&#10;cycle consistency. In the literature, these methods have been evaluated with an&#10;array of inconsistent settings, making it difficult to discern trends or&#10;compare performance fairly. Starting with a unified formulation of the label&#10;propagation algorithm that encompasses most existing variations, we&#10;systematically study the impact of important implementation factors in feature&#10;extraction and label propagation. Along the way, we report the accuracies of&#10;properly tuned supervised and unsupervised still image baselines, which are&#10;higher than those found in previous works. We also demonstrate that augmenting&#10;video-based correspondence cues with still-image-based ones can further improve&#10;performance. We then attempt a fair comparison of recent video-based methods on&#10;the DAVIS benchmark, showing convergence of best methods to performance levels&#10;near our strong ImageNet baseline, despite the usage of a variety of&#10;specialized video-based losses and training particulars. Additional comparisons&#10;on JHMDB and VIP datasets confirm the similar performance of current methods.&#10;We hope that this study will help to improve evaluation practices and better&#10;inform future research directions in temporal correspondence.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Dense label propagation in video is a challenging task related to tracking and correspondence. Given a labeling in an initial frame, e.g., a segmentation mask or a set of keypoints, the goal is to propagate these labels forward through subsequent video frames (Figure \ref{fig:teaser}). Popular benchmarks for this task include DAVIS~\cite{pont20172017}, JHMDB~\cite{jhuang2013towards}, and VIP~\cite{zhou2018adaptive}. Obtaining the best accuracy on such benchmarks requires fully supervised training from large amounts of dense pixel annotations, often with an additional step of online adaptation to each test sequence~\cite{bao2018cnn,caelles2017one,luiten2018premvos,DMMNet19}. However, since large-scale pixel-level annotations are difficult to obtain, a number of recent works have focused instead on {self-supervised} approaches for learning temporal correspondence, where a feature representation pre-trained on a different dataset and pretext task is transferred to the desired label propagation benchmark with the help of a custom-designed propagation algorithm \cite{vondrick2018tracking,wang2019learning,lai2019self,li2019joint,lai2020mast,jabri2020walk,xu2021rethinking,araslanov2021dense}. &#10;Popular self-supervised pretext tasks include frame-to-frame colorization \cite{vondrick2018tracking, lai2020mast,lai2019self,li2019joint} and patch tracking with cycle consistency~\cite{wang2019learning,jabri2020walk}, while the propagation step relies on some variant of $k$-nearest-neighbors to label a location based on the most similar locations in previous frames, with the similarity computed from the pre-trained feature representation. &#10;&#10;For understandable reasons, published papers on self-supervised label propagation tend to emphasize their pretext tasks and losses, and to downplay such mundane implementation factors as resolution of input video frames and output feature maps, the layer of the network from which the feature map is taken, the number of context frames in which nearest neighbors are found, and other hand-crafted details of the propagation algorithm. &#10;However, changing these factors can have at least as much influence on the performance of a method as the underlying feature representation.&#10;To underscore this point, Figure \ref{fig:imagenet-history} shows the performance of a supervised ImageNet-pretrained baseline on DAVIS as reported by three recent papers~\cite{wang2019learning,li2019joint,jabri2020walk}. While each of these papers uses the exact same ImageNet-pretrained network, the reported performance varies dramatically due to differences in feature extraction and label propagation procedures (see caption and footnote for details). In our work, we show that the baseline accuracy can be improved even further (Figure \ref{fig:imagenet-history}, right) by fine-tuning the late layers following stride changes, but still {on the ImageNet classification task alone} (see Section \ref{sec:implementation-facts:feat-extract} for details). We argue that such attention to optimizing the baseline is justified because researchers tend to spend a lot of effort on getting every last bit of performance out of their own method -- often by tuning factors unrelated to their core technical contributions -- while neglecting to expend a similar amount of effort on the baseline. Establishing the strongest possible baseline is essential for conducting a clear-eyed evaluation of existing work. Beyond sometimes relying on suboptimal baseline numbers, papers in this area have compared their numbers to those of previously published methods employing a range of inconsistent settings, which makes it difficult to understand the causes of performance differences or to assess how much progress has really been made.&#10;In the spirit of comparative studies that have taken a critical look at areas such as metric learning~\cite{musgrave2020metric}, self-supervised representation learning~\cite{kolesnikov2019revisiting}, and tracking~\cite{wang2021different}, we conduct a study to clarify performance differences between recent self-supervised video correspondence works by comparing several feature representations trained on still images and video. &#10;&#10;We begin in Sec. \ref{sec:propagation} by presenting our unified label propagation algorithm to systematize the variations used in most existing works. Sec. \ref{section:approaches} summarizes the feature representations and video correspondence methods included in our study. Sec. \ref{sec:implementation-facts} evaluates the effect of several key implementation factors on our main benchmark, the DAVIS dataset. Here, we report the accuracies of properly tuned supervised and unsupervised still image baselines, which are higher than those found in previous works. Next, in Sec. \ref{sec:davis-comp:uvc-studies}, we investigate whether it is possible to further improve the performance of video-based correspondence models by augmenting their training with still image cues. We introduce a hybrid method relying on signals from both UVC~\cite{li2019joint} video-based learning and a self-supervised image objective. This method, which we call UVC+, outperforms both UVC and still-image-based methods, pointing in a promising future research direction. Next, Sec. \ref{sec:davis-comp} attempts a fair comparison of recent top-performing methods on the DAVIS dataset in light of our findings. Here, we find that several of the most recent methods, including our UVC+, CRW~\cite{jabri2020walk}, and VFS~\cite{xu2021rethinking}, achieve very similar levels of performance, close to our strong ImageNet baseline, despite relying on different combinations of cues and training particulars. Lastly, Sec. \ref{sec:vip-jhmdb} presents similar evaluations of existing methods on the JHMDB and VIP datasets. Finally, Sec. \ref{sec:conclusion} closes the paper with recommendations for future best practices. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Video Label Propagation, Feature Representation Learning, Temporal Correspondence Analysis" />
-        </attvalues>
-      </node>
-      <node id="1808.00661" label="1808.00661">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Temporal Encoding Network for Video Instance-level Human&#10;  Parsing" />
-          <attvalue for="1" value="  Beyond the existing single-person and multiple-person human parsing tasks in&#10;static images, this paper makes the first attempt to investigate a more&#10;realistic video instance-level human parsing that simultaneously segments out&#10;each person instance and parses each instance into more fine-grained parts&#10;(e.g., head, leg, dress). We introduce a novel Adaptive Temporal Encoding&#10;Network (ATEN) that alternatively performs temporal encoding among key frames&#10;and flow-guided feature propagation from other consecutive frames between two&#10;key frames. Specifically, ATEN first incorporates a Parsing-RCNN to produce the&#10;instance-level parsing result for each key frame, which integrates both the&#10;global human parsing and instance-level human segmentation into a unified&#10;model. To balance between accuracy and efficiency, the flow-guided feature&#10;propagation is used to directly parse consecutive frames according to their&#10;identified temporal consistency with key frames. On the other hand, ATEN&#10;leverages the convolution gated recurrent units (convGRU) to exploit temporal&#10;changes over a series of key frames, which are further used to facilitate the&#10;frame-level instance-level parsing. By alternatively performing direct feature&#10;propagation between consistent frames and temporal encoding network among key&#10;frames, our ATEN achieves a good balance between frame-level accuracy and time&#10;efficiency, which is a common crucial problem in video object segmentation&#10;research. To demonstrate the superiority of our ATEN, extensive experiments are&#10;conducted on the most popular video segmentation benchmark (DAVIS) and a newly&#10;collected Video Instance-level Parsing (VIP) dataset, which is the first video&#10;instance-level human parsing dataset comprised of 404 sequences and over 20k&#10;frames with instance-level and pixel-wise annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.09453" label="1803.09453">
-        <attvalues>
-          <attvalue for="0" value="CNN in MRF: Video Object Segmentation via Inference in A CNN-Based&#10;  Higher-Order Spatio-Temporal MRF" />
-          <attvalue for="1" value="  This paper addresses the problem of video object segmentation, where the&#10;initial object mask is given in the first frame of an input video. We propose a&#10;novel spatio-temporal Markov Random Field (MRF) model defined over pixels to&#10;handle this problem. Unlike conventional MRF models, the spatial dependencies&#10;among pixels in our model are encoded by a Convolutional Neural Network (CNN).&#10;Specifically, for a given object, the probability of a labeling to a set of&#10;spatially neighboring pixels can be predicted by a CNN trained for this&#10;specific object. As a result, higher-order, richer dependencies among pixels in&#10;the set can be implicitly modeled by the CNN. With temporal dependencies&#10;established by optical flow, the resulting MRF model combines both spatial and&#10;temporal cues for tackling video object segmentation. However, performing&#10;inference in the MRF model is very difficult due to the very high-order&#10;dependencies. To this end, we propose a novel CNN-embedded algorithm to perform&#10;approximate inference in the MRF. This algorithm proceeds by alternating&#10;between a temporal fusion step and a feed-forward CNN step. When initialized&#10;with an appearance-based one-shot segmentation CNN, our model outperforms the&#10;winning entries of the DAVIS 2017 Challenge, without resorting to model&#10;ensembling or any dedicated detectors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.05198" label="1611.05198">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Video Object Segmentation" />
-          <attvalue for="1" value="  This paper tackles the task of semi-supervised video object segmentation,&#10;i.e., the separation of an object from the background in a video, given the&#10;mask of the first frame. We present One-Shot Video Object Segmentation (OSVOS),&#10;based on a fully-convolutional neural network architecture that is able to&#10;successively transfer generic semantic information, learned on ImageNet, to the&#10;task of foreground segmentation, and finally to learning the appearance of a&#10;single annotated object of the test sequence (hence one-shot). Although all&#10;frames are processed independently, the results are temporally coherent and&#10;stable. We perform experiments on two annotated video segmentation databases,&#10;which show that OSVOS is fast and improves the state of the art by a&#10;significant margin (79.8% vs 68.0%).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.09190" label="1807.09190">
-        <attvalues>
-          <attvalue for="0" value="PReMVOS: Proposal-generation, Refinement and Merging for Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  We address semi-supervised video object segmentation, the task of&#10;automatically generating accurate and consistent pixel masks for objects in a&#10;video sequence, given the first-frame ground truth annotations. Towards this&#10;goal, we present the PReMVOS algorithm (Proposal-generation, Refinement and&#10;Merging for Video Object Segmentation). Our method separates this problem into&#10;two steps, first generating a set of accurate object segmentation mask&#10;proposals for each video frame and then selecting and merging these proposals&#10;into accurate and temporally consistent pixel-wise object tracks over a video&#10;sequence in a way which is designed to specifically tackle the difficult&#10;challenges involved with segmenting multiple objects across a video sequence.&#10;Our approach surpasses all previous state-of-the-art results on the DAVIS 2017&#10;video object segmentation benchmark with a J &amp; F mean score of 71.6 on the&#10;test-dev dataset, and achieves first place in both the DAVIS 2018 Video Object&#10;Segmentation Challenge and the YouTube-VOS 1st Large-scale Video Object&#10;Segmentation Challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.09594" label="1806.09594">
-        <attvalues>
-          <attvalue for="0" value="Tracking Emerges by Colorizing Videos" />
-          <attvalue for="1" value="  We use large amounts of unlabeled video to learn models for visual tracking&#10;without manual human supervision. We leverage the natural temporal coherency of&#10;color to create a model that learns to colorize gray-scale videos by copying&#10;colors from a reference frame. Quantitative and qualitative experiments suggest&#10;that this task causes the model to automatically learn to track visual regions.&#10;Although the model is trained without any ground-truth labels, our method&#10;learns to track well enough to outperform the latest methods based on optical&#10;flow. Moreover, our results suggest that failures to track are correlated with&#10;failures to colorize, indicating that advancing video colorization may further&#10;improve self-supervised visual tracking.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.07593" label="1903.07593">
-        <attvalues>
-          <attvalue for="0" value="Learning Correspondence from the Cycle-Consistency of Time" />
-          <attvalue for="1" value="  We introduce a self-supervised method for learning visual correspondence from&#10;unlabeled video. The main idea is to use cycle-consistency in time as free&#10;supervisory signal for learning visual representations from scratch. At&#10;training time, our model learns a feature map representation to be useful for&#10;performing cycle-consistent tracking. At test time, we use the acquired&#10;representation to find nearest neighbors across space and time. We demonstrate&#10;the generalizability of the representation -- without finetuning -- across a&#10;range of visual correspondence tasks, including video object segmentation,&#10;keypoint tracking, and optical flow. Our approach outperforms previous&#10;self-supervised methods and performs competitively with strongly supervised&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.00875" label="1905.00875">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Learning for Video Correspondence Flow" />
-          <attvalue for="1" value="  The objective of this paper is self-supervised learning of feature embeddings&#10;that are suitable for matching correspondences along the videos, which we term&#10;correspondence flow. By leveraging the natural spatial-temporal coherence in&#10;videos, we propose to train a ``pointer'' that reconstructs a target frame by&#10;copying pixels from a reference frame.&#10;  We make the following contributions: First, we introduce a simple information&#10;bottleneck that forces the model to learn robust features for correspondence&#10;matching, and prevent it from learning trivial solutions, \eg matching based on&#10;low-level colour information. Second, to tackle the challenges from tracker&#10;drifting, due to complex object deformations, illumination changes and&#10;occlusions, we propose to train a recursive model over long temporal windows&#10;with scheduled sampling and cycle consistency. Third, we achieve&#10;state-of-the-art performance on DAVIS 2017 video segmentation and JHMDB&#10;keypoint tracking tasks, outperforming all previous self-supervised learning&#10;approaches by a significant margin. Fourth, in order to shed light on the&#10;potential of self-supervised learning on the task of video correspondence flow,&#10;we probe the upper bound by training on additional data, \ie more diverse&#10;videos, further demonstrating significant improvements on video segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.11895" label="1909.11895">
-        <attvalues>
-          <attvalue for="0" value="Joint-task Self-supervised Learning for Temporal Correspondence" />
-          <attvalue for="1" value="  This paper proposes to learn reliable dense correspondence from videos in a&#10;self-supervised manner. Our learning process integrates two highly related&#10;tasks: tracking large image regions \emph{and} establishing fine-grained&#10;pixel-level associations between consecutive video frames. We exploit the&#10;synergy between both tasks through a shared inter-frame affinity matrix, which&#10;simultaneously models transitions between video frames at both the region- and&#10;pixel-levels. While region-level localization helps reduce ambiguities in&#10;fine-grained matching by narrowing down search regions; fine-grained matching&#10;provides bottom-up features to facilitate region-level localization. Our method&#10;outperforms the state-of-the-art self-supervised methods on a variety of visual&#10;correspondence tasks, including video-object and part-segmentation propagation,&#10;keypoint tracking, and object tracking. Our self-supervised method even&#10;surpasses the fully-supervised affinity feature representation obtained from a&#10;ResNet-18 pre-trained on the ImageNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.07793" label="2002.07793">
-        <attvalues>
-          <attvalue for="0" value="MAST: A Memory-Augmented Self-supervised Tracker" />
-          <attvalue for="1" value="  Recent interest in self-supervised dense tracking has yielded rapid progress,&#10;but performance still remains far from supervised methods. We propose a dense&#10;tracking model trained on videos without any annotations that surpasses&#10;previous self-supervised methods on existing benchmarks by a significant margin&#10;(+15%), and achieves performance comparable to supervised methods. In this&#10;paper, we first reassess the traditional choices used for self-supervised&#10;training and reconstruction loss by conducting thorough experiments that&#10;finally elucidate the optimal choices. Second, we further improve on existing&#10;methods by augmenting our architecture with a crucial memory component. Third,&#10;we benchmark on large-scale semi-supervised video object segmentation(aka.&#10;dense tracking), and propose a new metric: generalizability. Our first two&#10;contributions yield a self-supervised network that for the first time is&#10;competitive with supervised methods on standard evaluation metrics of dense&#10;tracking. When measuring generalizability, we show self-supervised approaches&#10;are actually superior to the majority of supervised methods. We believe this&#10;new generalizability metric can better capture the real-world use-cases for&#10;dense tracking, and will spur new interest in this research direction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.14613" label="2006.14613">
-        <attvalues>
-          <attvalue for="0" value="Space-Time Correspondence as a Contrastive Random Walk" />
-          <attvalue for="1" value="  This paper proposes a simple self-supervised approach for learning a&#10;representation for visual correspondence from raw video. We cast correspondence&#10;as prediction of links in a space-time graph constructed from video. In this&#10;graph, the nodes are patches sampled from each frame, and nodes adjacent in&#10;time can share a directed edge. We learn a representation in which pairwise&#10;similarity defines transition probability of a random walk, so that long-range&#10;correspondence is computed as a walk along the graph. We optimize the&#10;representation to place high probability along paths of similarity. Targets for&#10;learning are formed without supervision, by cycle-consistency: the objective is&#10;to maximize the likelihood of returning to the initial node when walking along&#10;a graph constructed from a palindrome of frames. Thus, a single path-level&#10;constraint implicitly supervises chains of intermediate comparisons. When used&#10;as a similarity metric without adaptation, the learned representation&#10;outperforms the self-supervised state-of-the-art on label propagation tasks&#10;involving objects, semantic parts, and pose. Moreover, we demonstrate that a&#10;technique we call edge dropout, as well as self-supervised adaptation at&#10;test-time, further improve transfer for object-centric correspondence.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.17263" label="2103.17263">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Self-supervised Correspondence Learning: A Video Frame-level&#10;  Similarity Perspective" />
-          <attvalue for="1" value="  Learning a good representation for space-time correspondence is the key for&#10;various computer vision tasks, including tracking object bounding boxes and&#10;performing video object pixel segmentation. To learn generalizable&#10;representation for correspondence in large-scale, a variety of self-supervised&#10;pretext tasks are proposed to explicitly perform object-level or patch-level&#10;similarity learning. Instead of following the previous literature, we propose&#10;to learn correspondence using Video Frame-level Similarity (VFS) learning, i.e,&#10;simply learning from comparing video frames. Our work is inspired by the recent&#10;success in image-level contrastive learning and similarity learning for visual&#10;recognition. Our hypothesis is that if the representation is good for&#10;recognition, it requires the convolutional features to find correspondence&#10;between similar objects or parts. Our experiments show surprising results that&#10;VFS surpasses state-of-the-art self-supervised approaches for both OTB visual&#10;object tracking and DAVIS video object segmentation. We perform detailed&#10;analysis on what matters in VFS and reveals new properties on image and frame&#10;level similarity learning. Project page with code is available at&#10;https://jerryxu.net/VFS&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.06265" label="2111.06265">
-        <attvalues>
-          <attvalue for="0" value="Dense Unsupervised Learning for Video Segmentation" />
-          <attvalue for="1" value="  We present a novel approach to unsupervised learning for video object&#10;segmentation (VOS). Unlike previous work, our formulation allows to learn dense&#10;feature representations directly in a fully convolutional regime. We rely on&#10;uniform grid sampling to extract a set of anchors and train our model to&#10;disambiguate between them on both inter- and intra-video levels. However, a&#10;naive scheme to train such a model results in a degenerate solution. We propose&#10;to prevent this with a simple regularisation scheme, accommodating the&#10;equivariance property of the segmentation task to similarity transformations.&#10;Our training objective admits efficient implementation and exhibits fast&#10;training convergence. On established VOS benchmarks, our approach exceeds the&#10;segmentation accuracy of previous work despite using significantly less&#10;training data and compute power.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.08505" label="2003.08505">
-        <attvalues>
-          <attvalue for="0" value="A Metric Learning Reality Check" />
-          <attvalue for="1" value="  Deep metric learning papers from the past four years have consistently&#10;claimed great advances in accuracy, often more than doubling the performance of&#10;decade-old methods. In this paper, we take a closer look at the field to see if&#10;this is actually true. We find flaws in the experimental methodology of&#10;numerous metric learning papers, and show that the actual improvements over&#10;time have been marginal at best.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.09005" label="1901.09005">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Self-Supervised Visual Representation Learning" />
-          <attvalue for="1" value="  Unsupervised visual representation learning remains a largely unsolved&#10;problem in computer vision research. Among a big body of recently proposed&#10;approaches for unsupervised learning of visual representations, a class of&#10;self-supervised techniques achieves superior performance on many challenging&#10;benchmarks. A large number of the pretext tasks for self-supervised learning&#10;have been studied, but other important aspects, such as the choice of&#10;convolutional neural networks (CNN), has not received equal attention.&#10;Therefore, we revisit numerous previously proposed self-supervised models,&#10;conduct a thorough large scale study and, as a result, uncover multiple crucial&#10;insights. We challenge a number of common practices in selfsupervised visual&#10;representation learning and observe that standard recipes for CNN design do not&#10;always translate to self-supervised representation learning. As part of our&#10;study, we drastically boost the performance of previously proposed techniques&#10;and outperform previously published state-of-the-art results by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.02156" label="2107.02156">
-        <attvalues>
-          <attvalue for="0" value="Do Different Tracking Tasks Require Different Appearance Models?" />
-          <attvalue for="1" value="  Tracking objects of interest in a video is one of the most popular and widely&#10;applicable problems in computer vision. However, with the years, a Cambrian&#10;explosion of use cases and benchmarks has fragmented the problem in a multitude&#10;of different experimental setups. As a consequence, the literature has&#10;fragmented too, and now novel approaches proposed by the community are usually&#10;specialised to fit only one specific setup. To understand to what extent this&#10;specialisation is necessary, in this work we present UniTrack, a solution to&#10;address five different tasks within the same framework. UniTrack consists of a&#10;single and task-agnostic appearance model, which can be learned in a supervised&#10;or self-supervised fashion, and multiple ``heads'' that address individual&#10;tasks and do not require training. We show how most tracking tasks can be&#10;solved within this framework, and that the same appearance model can be&#10;successfully used to obtain results that are competitive against specialised&#10;methods for most of the tasks considered. The framework also allows us to&#10;analyse appearance models obtained with the most recent self-supervised&#10;methods, thus extending their evaluation and comparison to a larger variety of&#10;important problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;To the best of our knowledge, sharing the appearance model across multiple tracking tasks has not been extensively studied in the computer vision literature, and especially not in the context of SSL representations.&#10;Some existing methods do share a common backbone architecture across tasks.&#10;For instance, STEm-Seg~\cite{stem} addresses VIS~\cite{ytbvis} and MOTS; while TraDeS~\cite{trades} addresses MOT, MOTS and VIS.&#10;However, both methods need to be trained separately and on different datasets for every task.&#10;Conversely, we reuse the same representation across five tasks.&#10;A promising direction for future work would be to use UniTrack to train a shared representation in a multi-task fashion.&#10;Only a few relevant works do adopt a multi-task approach~\cite{siammask,ocean+,d3s}, and they usually consider SOT and VOS tasks only. &#10;In general, despite the multi-task direction being surely interesting, it requires the availability of large-scale datasets with annotations in multiple formats, and costly training.&#10;These are two of the main reasons for which we believe that having a framework that allows to achieve competitive performance on multiple tasks with previously-trained models is a worthwhile endeavour.&#10;&#10;Self-supervised model evaluation. &#10;Given the difference between the pretext tasks used to train self-supervised models and the downstream tasks used to evaluate them, the comparison between self-supervised approaches has always been a delicate matter.&#10;Existing evaluation strategies typically require additional training once a general-purpose representation has been obtained.&#10;One strategy keeps the representation fixed, and then trains additional task-specific heads with very limited capacity (\eg a linear classifier~\cite{sslbenchmark,simclrv1,mocov1} or a regression head for object detection~\cite{sslbenchmark}).&#10;A second strategy, instead, leverages SSL to obtain particularly effective initializations, and then proceeds to fine-tune such initialized models on the downstream task of interest.&#10;A wider range of tasks can be tested using this setup, such as semantic segmentation~\cite{ssltransfer,sslbenchmark} and surface normal estimation~\cite{sslbenchmark,ssltrack}.&#10;In contrast, UniTrack provides a simpler way to evaluate SSL models, one that does not require additional training or fine-tuning.&#10;Also, this work is the first to extend SSL evaluation to a set of diverse video tasks.&#10;We believe this contribution will allow the study of self-supervised learning methods with a broader scope of applicability.&#10;Our work is also related to a line of self-supervised learning methods ~\cite{CRW,colorize,uvc,mast} that learn their representations in a task-agnostic fashion, and then test it on propagation tasks (SOT and VOS).&#10;The design of UniTrack is inspired by their task-agnostic philosophy, while significantly extending their scope to a new set of tasks.&#10;" />
-        </attvalues>
-      </node>
-      <node id="2311.04246" label="2311.04246">
-        <attvalues>
-          <attvalue for="0" value="ADFactory: An Effective Framework for Generalizing Optical Flow with&#10;  Nerf" />
-          <attvalue for="1" value="  A significant challenge facing current optical flow methods is the difficulty&#10;in generalizing them well to the real world. This is mainly due to the high&#10;cost of hand-crafted datasets, and existing self-supervised methods are limited&#10;by indirect loss and occlusions, resulting in fuzzy outcomes. To address this&#10;challenge, we introduce a novel optical flow training framework: automatic data&#10;factory (ADF). ADF only requires RGB images as input to effectively train the&#10;optical flow network on the target data domain. Specifically, we use advanced&#10;Nerf technology to reconstruct scenes from photo groups collected by a&#10;monocular camera, and then calculate optical flow labels between camera pose&#10;pairs based on the rendering results. To eliminate erroneous labels caused by&#10;defects in the scene reconstructed by Nerf, we screened the generated labels&#10;from multiple aspects, such as optical flow matching accuracy, radiation field&#10;confidence, and depth consistency. The filtered labels can be directly used for&#10;network supervision. Experimentally, the generalization ability of ADF on KITTI&#10;surpasses existing self-supervised optical flow and monocular scene flow&#10;algorithms. In addition, ADF achieves impressive results in real-world&#10;zero-point generalization evaluations and surpasses most supervised methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Optical flow aims to estimate the motion of each pixel on the image plane between two consecutive frames. It is one of the oldest problems in computer vision and has essential applications in autonomous driving\cite{capito2020optical,wang2021end,wang2021learning}, video understanding\cite{fan2018end}, and human action recognition\cite{lertniphonphan2011human,arshad2022human}. In recent years, deep learning methods\cite{sun2018pwc,teed2020raft} have become the mainstream solution for optical flow problem and have achieved performance far superior to traditional handcrafted methods \cite{beauchemin1995computation} in benchmark tests\cite{Menze2015ISA,butler2012naturalistic}.&#10;&#10;Compared to traditional methods, deep optical flow methods can learn robust matching features and contextual features from large-scale data, as well as infer the optical flow of occluded parts\cite{jiang2021learning}. However, acquiring real-world optical flow datasets is challenging. It requires additional sensors such as LiDAR, GPS, IMU, and much manual annotations\cite{Menze2015ISA}, which has been one of the significant limitations of applying optical flow methods\cite{xu2022gmflow}. Although recent works in synthetic datasets\cite{mayer2016large,butler2012naturalistic,mehl2023spring} and self-supervised learning\cite{jonschkowski2020matters,kong2022mdflow} have alleviated this limitation to some extent, they are still subject to constraints in domain transferability, lighting variations, and occlusions.&#10;&#10;In this era where data is the gold mine, massive training datasets are a necessary factor for the success of powerful models such as ChatGPT\cite{brown2020language} and SAM\cite{kirillov2023segment}. &#10;However, how to quickly and cost-effectively mine high-quality training data from the `gold mines' and fully tap into the enormous potential of deep networks remains an unexplored challenge.&#10;In this paper, we propose a novel automated data factory (ADF) to address this issue. ADF utilizes only photos taken by a single camera as input, and does not require any manual intervention or ground-truth labels. It can quickly generate almost infinite datasets at a meagre cost, making it a valuable asset for various geometric matching tasks. Specifically, ADF mainly consists of two parts: data generation and data filtering.&#10;&#10;Data generation: We build a data generator based on the latest radiation rendering technology (Nerf)\cite{barron2023zip}. Firstly, based on Nerf, a high-resolution scene is reconstructed from photos taken by a monocular camera. Then, by rendering a depth map and randomly generating new camera pose pairs, we can calculate the corresponding optical flow results between poses. In addition, to alleviate the problem that Nerf can only reconstruct static scenes with high quality, we also added randomly shaped foreground floaters based on the Bezier curve.&#10;&#10;Data filtering:&#10;In Nerf reconstruction, it is inevitable to encounter areas of reconstruction failure, resulting in further generation of erroneous training data. Therefore, constructing a comprehensive data filtering mechanism is essential.&#10;Before us, the only evaluation metric used was ambient occlusion (AO) \cite{truong2023sparf,tosi2023nerf}. However, AO can only be used to check whether floating objects are between the viewpoint and the target depth, which cannot be widely used for Nerf methods and downstream tasks.&#10;In this paper, we have designed practical evaluation metrics such as reconstruction stability, depth consistency, and optical flow visual consistency based on the characteristics of the Nerf method. Experiments have shown that the combination of these metrics improves the overall performance of the optical flow method by at least 20\%.&#10;&#10;The filtered optical flow results can be directly used to train complex geometric matching tasks\cite{sun2021loftr,xu2022gmflow,liu2022camliflow,ling2023learning}. As shown in Fig.\ref{fig_top}, the optical flow model trained on ADF-generated data achieves stunning real-world generalization effects.&#10;We believe that ADF is an important step in making optical flow methods widely applicable in real-world scenarios and a stepping stone to building matching large models. In this paper, we only used about 300 collected scenarios, which has made the ADF-supervised optical flow methods far superior in zero-shot generalization to self-supervised and supervised models trained on existing datasets \cite{mayer2016large,butler2012naturalistic,Menze2015ISA}.&#10;&#10;Our contributions can be summarized as follows:&#10;&#10;\begin{itemize}&#10;&#09;\item{A new mode for self-supervised learning of optical flow algorithms based on Nerf.}&#10;&#09;\item{A fully automatic optical flow label generation pipeline ADF, which only requires images taken by a monocular camera as input to generate almost infinite datasets.}&#10;&#09;\item{We collected and created an optical flow dataset ADF58 containing over 300 scenes and 58800 images. Experimental results demonstrate that ADF58 achieves superior zero-shot generalization capability in optical flow tasks compared to existing datasets.}&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Computer Vision, Optical Flow Methods, Mathematics, Physics, Computer Science, Self-Supervised Learning, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2006.00168" label="2006.00168">
-        <attvalues>
-          <attvalue for="0" value="Optical Flow based Visual Potential Field for Autonomous Driving" />
-          <attvalue for="1" value="  Monocular vision-based navigation for automated driving is a challenging task&#10;due to the lack of enough information to compute temporal relationships among&#10;objects on the road. Optical flow is an option to obtain temporal information&#10;from monocular camera images and has been used widely with the purpose of&#10;identifying objects and their relative motion. This work proposes to generate&#10;an artificial potential field, i.e. visual potential field, from a sequence of&#10;images using sparse optical flow, which is used together with a gradient&#10;tracking sliding mode controller to navigate the vehicle to destination without&#10;collision with obstacles. The angular reference for the vehicle is computed&#10;online. This work considers that the vehicle does not require to have a priori&#10;information from the map or obstacles to navigate successfully. The proposed&#10;technique is tested both in synthetic and real images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.08862" label="2104.08862">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Interactive Prediction and Planning with Optical Flow&#10;  Distillation for Autonomous Driving" />
-          <attvalue for="1" value="  With the recent advancement of deep learning technology, data-driven&#10;approaches for autonomous car prediction and planning have achieved&#10;extraordinary performance. Nevertheless, most of these approaches follow a&#10;non-interactive prediction and planning paradigm, hypothesizing that a&#10;vehicle's behaviors do not affect others. The approaches based on such a&#10;non-interactive philosophy typically perform acceptably in sparse traffic&#10;scenarios but can easily fail in dense traffic scenarios. Therefore, we propose&#10;an end-to-end interactive neural motion planner (INMP) for autonomous driving&#10;in this paper. Given a set of past surrounding-view images and a high&#10;definition map, our INMP first generates a feature map in bird's-eye-view&#10;space, which is then processed to detect other agents and perform interactive&#10;prediction and planning jointly. Also, we adopt an optical flow distillation&#10;paradigm, which can effectively improve the network performance while still&#10;maintaining its real-time inference speed. Extensive experiments on the&#10;nuScenes dataset and in the closed-loop Carla simulation environment&#10;demonstrate the effectiveness and efficiency of our INMP for the detection,&#10;prediction, and planning tasks. Our project page is at&#10;sites.google.com/view/inmp-ofd.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.12861" label="2104.12861">
-        <attvalues>
-          <attvalue for="0" value="Learning Interpretable End-to-End Vision-Based Motion Planning for&#10;  Autonomous Driving with Optical Flow Distillation" />
-          <attvalue for="1" value="  Recently, deep-learning based approaches have achieved impressive performance&#10;for autonomous driving. However, end-to-end vision-based methods typically have&#10;limited interpretability, making the behaviors of the deep networks difficult&#10;to explain. Hence, their potential applications could be limited in practice.&#10;To address this problem, we propose an interpretable end-to-end vision-based&#10;motion planning approach for autonomous driving, referred to as IVMP. Given a&#10;set of past surrounding-view images, our IVMP first predicts future egocentric&#10;semantic maps in bird's-eye-view space, which are then employed to plan&#10;trajectories for self-driving vehicles. The predicted future semantic maps not&#10;only provide useful interpretable information, but also allow our motion&#10;planning module to handle objects with low probability, thus improving the&#10;safety of autonomous driving. Moreover, we also develop an optical flow&#10;distillation paradigm, which can effectively enhance the network while still&#10;maintaining its real-time performance. Extensive experiments on the nuScenes&#10;dataset and closed-loop simulation show that our IVMP significantly outperforms&#10;the state-of-the-art approaches in imitating human drivers with a much higher&#10;success rate. Our project page is available at&#10;https://sites.google.com/view/ivmp.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.00413" label="1804.00413">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Learning of Motion Representation for Video Understanding" />
-          <attvalue for="1" value="  Despite the recent success of end-to-end learned representations,&#10;hand-crafted optical flow features are still widely used in video analysis&#10;tasks. To fill this gap, we propose TVNet, a novel end-to-end trainable neural&#10;network, to learn optical-flow-like features from data. TVNet subsumes a&#10;specific optical flow solver, the TV-L1 method, and is initialized by unfolding&#10;its optimization iterations as neural layers. TVNet can therefore be used&#10;directly without any extra learning. Moreover, it can be naturally concatenated&#10;with other task-specific networks to formulate an end-to-end architecture, thus&#10;making our method more efficient than current multi-stage approaches by&#10;avoiding the need to pre-compute and store features on disk. Finally, the&#10;parameters of the TVNet can be further fine-tuned by end-to-end training. This&#10;enables TVNet to learn richer and task-specific patterns beyond exact optical&#10;flow. Extensive experiments on two action recognition benchmarks verify the&#10;effectiveness of the proposed approach. Our TVNet achieves better accuracies&#10;than all compared methods, while being competitive with the fastest counterpart&#10;in terms of features extraction time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.02409" label="2104.02409">
-        <attvalues>
-          <attvalue for="0" value="Learning to Estimate Hidden Motions with Global Motion Aggregation" />
-          <attvalue for="1" value="  Occlusions pose a significant challenge to optical flow algorithms that rely&#10;on local evidences. We consider an occluded point to be one that is imaged in&#10;the first frame but not in the next, a slight overloading of the standard&#10;definition since it also includes points that move out-of-frame. Estimating the&#10;motion of these points is extremely difficult, particularly in the two-frame&#10;setting. Previous work relies on CNNs to learn occlusions, without much&#10;success, or requires multiple frames to reason about occlusions using temporal&#10;smoothness. In this paper, we argue that the occlusion problem can be better&#10;solved in the two-frame case by modelling image self-similarities. We introduce&#10;a global motion aggregation module, a transformer-based approach to find&#10;long-range dependencies between pixels in the first image, and perform global&#10;aggregation on the corresponding motion features. We demonstrate that the&#10;optical flow estimates in the occluded regions can be significantly improved&#10;without damaging the performance in non-occluded regions. This approach obtains&#10;new state-of-the-art results on the challenging Sintel dataset, improving the&#10;average end-point error by 13.6% on Sintel Final and 13.7% on Sintel Clean. At&#10;the time of submission, our method ranks first on these benchmarks among all&#10;published and unpublished approaches. Code is available at&#10;https://github.com/zacjiang/GMA&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Occlusions in optical flow.&#10;&#10;Occlusion poses a key challenge in optical flow estimation due to its violation of the brightness&#10;constancy constraint \cite{horn}. Most traditional optical flow algorithms treat occlusions as outliers&#10;and so develop and optimise robust objective functions. In continuous optimisation for optical flow, &#10;Brox \etal \cite{broxl1} used the $L^1$ norm due to its robustness to outliers caused by occlusions&#10;or large brightness variations. Zach \etal \cite{tvl1} added total variation regularisation&#10;and proposed an efficient numerical scheme to optimise the energy functional. &#10;This formulation was later improved by Wedel \etal \cite{wedel2009improved}. Later work introduced&#10;additional robust optimisation terms, including the Charbonnier potential \cite{LKmeetsHS} and the Lorentzian&#10;potential \cite{lorentzian}.&#10;&#10;More recently, discrete optimisation approaches, especially Markov Random Fields (MRFs) \cite{boykov},&#10;have been used to estimate optical flow. These algorithms \cite{discreteflow, fullflow, dcflow} first estimate&#10;the forward and backward flows separately using a robust, truncated data term. They then conduct a &#10;forward--backward consistency check to determine the occluded regions. Lastly, as a post-processing step,&#10;they use interpolation methods \cite{epicflow} to fill in the optical flow of the occluded regions.&#10;&#10;Other work incorporates occlusion estimation as a joint objective together with optical flow&#10;estimation. Alvarez \etal \cite{alvarez} use&#10;forward--backward consistency as an optimisation objective, thus estimating time-symmetric&#10;optical flow. In addition to forward--backward consistency, MirrorFlow \cite{mirrorflow} &#10;incorporates occlusion--disocclusion symmetry in the energy function and achieves&#10;performance improvements. Since occlusions are caused by 3D motions, other works&#10;\cite{sun2010layered, sun2014local} explicitly model &#10;local depth relationships into layers and reason about occlusions. &#10;&#10;Contrary to the above approaches, we do not overload the loss function with explicit occlusion reasoning.&#10;Instead, we adopt a learning approach, similar to other supervised deep optical flow learning&#10;approaches \cite{pwcnet,hd3,liteflownet,irr,vcn,scopeflow,maskflownet,raft,scv}. Rather than estimating&#10;an occlusion map explicitly, our goal is to improve the optical flow accuracy at occluded regions. &#10;We take an implicit approach to globally aggregate motion features, which provides extra information&#10;to correctly predict flow at occluded regions. Our approach can be thought of as a non-local interpolation&#10;approach \cite{sun2010secrets}, in contrast to local interpolation approaches \cite{epicflow}. &#10;In the deep learning literature, the occlusion problem&#10;has been addressed in an unsupervised learning setting \cite{occaware,multiframe,selflow}, &#10;however, existing supervised learning approaches all rely on convolutions&#10;to interpolate in occluded regions, which are prone to failure for&#10;more significant occlusions. &#10;&#10;Self-attention and transformers.&#10;Our design principle is inspired by the recent successes of the transformer literature \cite{transformer}. &#10;The transformer architecture was first successful in natural language processing (NLP), &#10;due to its ability to model long-range dependencies and its scalability for GPU parallel processing. &#10;Among various modules in the transformer achitecture, self-attention is the key design feature that make transformers&#10;work. Recently, researchers have introduced the transformer and related attention ideas to the vision &#10;community, mostly in high-level tasks such as image classification \cite{standalone, vit} &#10;and semantic segmentation \cite{dual, nonlocal, crisscross}. &#10;To the best of our knowledge, we are the first to use the idea of attention&#10;to solve the optical flow problem. Different from many existing works in the transformer&#10;literature, we do not use self-attention in our work. &#10;Self-attention refers to the query, key and value vectors coming from the same features. In our case, &#10;query and key vectors come from the context features modelling the appearance of the image while &#10;value vectors come from the motion features, which is an encoding of the correlation volume. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2111.13680" label="2111.13680">
-        <attvalues>
-          <attvalue for="0" value="GMFlow: Learning Optical Flow via Global Matching" />
-          <attvalue for="1" value="  Learning-based optical flow estimation has been dominated with the pipeline&#10;of cost volume with convolutions for flow regression, which is inherently&#10;limited to local correlations and thus is hard to address the long-standing&#10;challenge of large displacements. To alleviate this, the state-of-the-art&#10;framework RAFT gradually improves its prediction quality by using a large&#10;number of iterative refinements, achieving remarkable performance but&#10;introducing linearly increasing inference time. To enable both high accuracy&#10;and efficiency, we completely revamp the dominant flow regression pipeline by&#10;reformulating optical flow as a global matching problem, which identifies the&#10;correspondences by directly comparing feature similarities. Specifically, we&#10;propose a GMFlow framework, which consists of three main components: a&#10;customized Transformer for feature enhancement, a correlation and softmax layer&#10;for global feature matching, and a self-attention layer for flow propagation.&#10;We further introduce a refinement step that reuses GMFlow at higher feature&#10;resolution for residual flow prediction. Our new framework outperforms&#10;31-refinements RAFT on the challenging Sintel benchmark, while using only one&#10;refinement and running faster, suggesting a new paradigm for accurate and&#10;efficient optical flow estimation. Code is available at&#10;https://github.com/haofeixu/gmflow.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.02134" label="1512.02134">
-        <attvalues>
-          <attvalue for="0" value="A Large Dataset to Train Convolutional Networks for Disparity, Optical&#10;  Flow, and Scene Flow Estimation" />
-          <attvalue for="1" value="  Recent work has shown that optical flow estimation can be formulated as a&#10;supervised learning task and can be successfully solved with convolutional&#10;networks. Training of the so-called FlowNet was enabled by a large&#10;synthetically generated dataset. The present paper extends the concept of&#10;optical flow estimation via convolutional networks to disparity and scene flow&#10;estimation. To this end, we propose three synthetic stereo video datasets with&#10;sufficient realism, variation, and size to successfully train large networks.&#10;Our datasets are the first large-scale datasets to enable training and&#10;evaluating scene flow methods. Besides the datasets, we present a convolutional&#10;network for real-time disparity estimation that provides state-of-the-art&#10;results. By combining a flow and disparity estimation network and training it&#10;jointly, we demonstrate the first scene flow estimation with a convolutional&#10;network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.01943" label="2303.01943">
-        <attvalues>
-          <attvalue for="0" value="Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene&#10;  Flow, Optical Flow and Stereo" />
-          <attvalue for="1" value="  While recent methods for motion and stereo estimation recover an&#10;unprecedented amount of details, such highly detailed structures are neither&#10;adequately reflected in the data of existing benchmarks nor their evaluation&#10;methodology. Hence, we introduce Spring $-$ a large, high-resolution,&#10;high-detail, computer-generated benchmark for scene flow, optical flow, and&#10;stereo. Based on rendered scenes from the open-source Blender movie &quot;Spring&quot;,&#10;it provides photo-realistic HD datasets with state-of-the-art visual effects&#10;and ground truth training data. Furthermore, we provide a website to upload,&#10;analyze and compare results. Using a novel evaluation methodology based on a&#10;super-resolved UHD ground truth, our Spring benchmark can assess the quality of&#10;fine structures and provides further detailed performance statistics on&#10;different image regions. Regarding the number of ground truth frames, Spring is&#10;60$\times$ larger than the only scene flow benchmark, KITTI 2015, and&#10;15$\times$ larger than the well-established MPI Sintel optical flow benchmark.&#10;Initial results for recent methods on our benchmark show that estimating fine&#10;details is indeed challenging, as their accuracy leaves significant room for&#10;improvement. The Spring benchmark and the corresponding datasets are available&#10;at http://spring-benchmark.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In the literature there exists a large number of datasets and benchmarks for dense matching covering scene flow, optical flow and stereo. An overview over the state of the art is given in \ref{eval:tab:overview}.&#10;Datasets and benchmarks can roughly be divided along two orthogonal axes: The scene axis and the data axis.&#10;On one edge of the scene axis, there are datasets focusing on automotive scenes such as KITTI~\cite{Geiger2012_KITTI,Menze2015_KITTI}, Virtual KITTI~\cite{Gaidon2016_VKITTI,Cabon2020_VKITTI2}, HD1K~\cite{Kondermann2016_HD1K}, VIPER~\cite{Richter2017_VIPER} and Driving~\cite{Mayer2016_FTH}.&#10;On the other edge are datasets that target general scenes such as Middlebury~\cite{Baker2007_MiddleburyFlow,Scharstein2014_MiddleburyStereo}, MPI Sintel~\cite{Butler2012_Sintel}, ETH3D~\cite{Schoeps2017_ETH3D}, FlyingThings3D~\cite{Mayer2016_FTH}, Monkaa~\cite{Mayer2016_FTH} and FlyingChairs~\cite{Dosovitskiy2015_FlowNet} with the special case of Human OF \cite{Ranjan2020_HumanMotion} that addresses human body motion.&#10;Along the orthogonal data axis, datasets and benchmarks can be divided into real-world data~\cite{Geiger2012_KITTI,Menze2015_KITTI,Kondermann2016_HD1K,Scharstein2014_MiddleburyStereo,Schoeps2017_ETH3D} and synthetic data~\cite{Butler2012_Sintel,Richter2017_VIPER,Mayer2016_FTH,Dosovitskiy2015_FlowNet,Gaidon2016_VKITTI,Ranjan2020_HumanMotion,Sun2021_AutoFlow}.&#10;In the context of optical flow, Sintel showed that synthetic benchmarks can validly approximate the statistics of natural images and motion, while avoiding ground truth measurements in the real world.&#10;This motivates the use of synthetic data for general scenes in our Spring dataset. &#10;&#10;\paragraphnew{Resolution.}&#10;Most of the benchmarks and datasets are limited to images with QHD resolution (0.5 Mpix) or below~\cite{Geiger2012_KITTI, Menze2015_KITTI, Butler2012_Sintel, Mayer2016_FTH, Dosovitskiy2015_FlowNet, Schoeps2017_ETH3D, Ranjan2020_HumanMotion, Sun2021_AutoFlow, Greff2022_Kubric}.&#10;Only in the case of automotive optical flow, full HD resolution (2-3 Mpix) has been considered~\cite{Kondermann2016_HD1K, Richter2017_VIPER}.&#10;Moreover, first attempts with image sizes beyond HD (5.6 Mpix) have been made for real-world stereo, but only for a small number of images~\cite{Scharstein2014_MiddleburyStereo}.&#10;With respect to general optical flow and scene flow, Spring is the first dataset to consider HD resolution and above for input images and ground truth, respectively.&#10;&#10;\paragraphnew{Dataset size.}&#10;Considering the number of input and ground truth images, most popular benchmarks have at most 1600 frames.&#10;This is either due to the use of real-world footage~\cite{Geiger2012_KITTI,Scharstein2014_MiddleburyStereo,Menze2015_KITTI,Kondermann2016_HD1K, Schoeps2017_ETH3D} or their creation in the pre-deep-learning era~\cite{Geiger2012_KITTI,Butler2012_Sintel,Menze2015_KITTI}. &#10;The only exception is the automotive optical flow benchmark VIPER~\cite{Richter2017_VIPER}, if we leave aside several large training datasets without benchmark functionality such as FlyingChairs~\cite{Dosovitskiy2015_FlowNet}, Virtual KITTI~\cite{Gaidon2016_VKITTI,Cabon2020_VKITTI2}, FlyingThings3D with Monkaa and Driving~\cite{Mayer2016_FTH}, Human OF~\cite{Ranjan2020_HumanMotion}, AutoFlow \cite{Sun2021_AutoFlow} and the dataset generator Kubric~ \cite{Greff2022_Kubric}.&#10;With respect to the number of frames for stereo and scene flow, however, Spring is the first general-scene benchmark with several thousand samples for training and testing, closing &#10;a previously existing gap in the available training data for deep-learning matching methods for those tasks.&#10;&#10;\paragraphnew{Benchmark evaluation.}&#10;Regarding focused evaluations&#10;in particularly important regions, current benchmarks mainly focus on occlusions (unmatched regions)~\cite{Baker2011_MiddleburyFlow,Butler2012_Sintel, Menze2015_KITTI,Scharstein2014_MiddleburyStereo,Schoeps2017_ETH3D}, discontinuities~\cite{Baker2007_MiddleburyFlow,Butler2012_Sintel}, large displacements~\cite{Butler2012_Sintel} and non-rigid areas~\cite{Menze2015_KITTI}.&#10;However, recent stereo~\cite{Li2022_CREStereo} and optical flow methods~\cite{Jahedi2022_MSRAFT,Jahedi2022_MSRAFT_RVC} achieve highly detailed results due to cascaded recurrent neural networks that process images at larger resolutions and omit a significant upsampling of the final results (e.g.\ in contrast to~\cite{Teed2020_RAFT,Jiang2021_GMA,Lipson2021_RAFTStereo}).&#10;This raises the question how well such high-quality methods can estimate fine-scale details such as grass or hair.&#10;In this context, Spring not only provides high-resolution images that contain small-scale details at pixel level, but also provides a novel evaluation methodology that allows to measure the accuracy in the presence of thin structures.&#10;Therefore, the Spring benchmark provides focused evaluations for regions with unmatched, high-detail, non-rigid,&#10;sky and large-displacement pixels.&#10;&#10;\paragraphnew{Scene flow benchmarking.}&#10;Finally, with only 400 frames, KITTI 2015~\cite{Menze2015_KITTI} is the only benchmark that, besides optical flow and stereo, also allows the evaluation of scene flow. Moreover, while there are a few datasets and challenges for scene flow from RGB-D \cite{Sturm2012_RGBD,Shao2018_RGBD,Lv2018_RGBD} and LiDAR \cite{Sun2020_Waymo} data, KITTI 2015 is the only benchmark for image-based scene flow, \ie scene flow only from stereo image pairs. As a consequence, recent image-based scene flow methods perform well on KITTI 2015 but are likely not robust under other types of data.&#10;As shown for optical flow and stereo in context of the Robust Vision Challenge \cite{rvc}, improving the generalization across benchmarks is essential to increase applicability and robustness~\cite{Schmalfuss2022}.&#10;Hence, having more specifically tailored benchmarks at hand would not only be beneficial for optical flow and stereo, but in particular for image-based scene flow, where&#10;they are indispensable to further advance research. &#10;While with FlyingThings3D, Monkaa and Driving \cite{Mayer2016_FTH} as well as VKITTI~2 \cite{Cabon2020_VKITTI2} there are a few datasets available that could also be used for benchmarking, these datasets provide the ground truth for all frames which encourages overfitting. In contrast, Spring offers the {full benchmark functionality}, \ie hidden ground truth for the test set, an evaluation protocol, cheating prevention and a website to compare results.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2211.06018" label="2211.06018">
-        <attvalues>
-          <attvalue for="0" value="MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge&#10;  Distillation" />
-          <attvalue for="1" value="  Recent works have shown that optical flow can be learned by deep networks&#10;from unlabelled image pairs based on brightness constancy assumption and&#10;smoothness prior. Current approaches additionally impose an augmentation&#10;regularization term for continual self-supervision, which has been proved to be&#10;effective on difficult matching regions. However, this method also amplify the&#10;inevitable mismatch in unsupervised setting, blocking the learning process&#10;towards optimal solution. To break the dilemma, we propose a novel mutual&#10;distillation framework to transfer reliable knowledge back and forth between&#10;the teacher and student networks for alternate improvement. Concretely, taking&#10;estimation of off-the-shelf unsupervised approach as pseudo labels, our insight&#10;locates at defining a confidence selection mechanism to extract relative good&#10;matches, and then add diverse data augmentation for distilling adequate and&#10;reliable knowledge from teacher to student. Thanks to the decouple nature of&#10;our method, we can choose a stronger student architecture for sufficient&#10;learning. Finally, better student prediction is adopted to transfer knowledge&#10;back to the efficient teacher without additional costs in real deployment.&#10;Rather than formulating it as a supervised task, we find that introducing an&#10;extra unsupervised term for multi-target learning achieves best final results.&#10;Extensive experiments show that our approach, termed MDFlow, achieves&#10;state-of-the-art real-time accuracy and generalization ability on challenging&#10;benchmarks. Code is available at https://github.com/ltkong218/MDFlow.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.14165" label="2005.14165">
-        <attvalues>
-          <attvalue for="0" value="Language Models are Few-Shot Learners" />
-          <attvalue for="1" value="  Recent work has demonstrated substantial gains on many NLP tasks and&#10;benchmarks by pre-training on a large corpus of text followed by fine-tuning on&#10;a specific task. While typically task-agnostic in architecture, this method&#10;still requires task-specific fine-tuning datasets of thousands or tens of&#10;thousands of examples. By contrast, humans can generally perform a new language&#10;task from only a few examples or from simple instructions - something which&#10;current NLP systems still largely struggle to do. Here we show that scaling up&#10;language models greatly improves task-agnostic, few-shot performance, sometimes&#10;even reaching competitiveness with prior state-of-the-art fine-tuning&#10;approaches. Specifically, we train GPT-3, an autoregressive language model with&#10;175 billion parameters, 10x more than any previous non-sparse language model,&#10;and test its performance in the few-shot setting. For all tasks, GPT-3 is&#10;applied without any gradient updates or fine-tuning, with tasks and few-shot&#10;demonstrations specified purely via text interaction with the model. GPT-3&#10;achieves strong performance on many NLP datasets, including translation,&#10;question-answering, and cloze tasks, as well as several tasks that require&#10;on-the-fly reasoning or domain adaptation, such as unscrambling words, using a&#10;novel word in a sentence, or performing 3-digit arithmetic. At the same time,&#10;we also identify some datasets where GPT-3's few-shot learning still struggles,&#10;as well as some datasets where GPT-3 faces methodological issues related to&#10;training on large web corpora. Finally, we find that GPT-3 can generate samples&#10;of news articles which human evaluators have difficulty distinguishing from&#10;articles written by humans. We discuss broader societal impacts of this finding&#10;and of GPT-3 in general.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.02643" label="2304.02643">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything" />
-          <attvalue for="1" value="  We introduce the Segment Anything (SA) project: a new task, model, and&#10;dataset for image segmentation. Using our efficient model in a data collection&#10;loop, we built the largest segmentation dataset to date (by far), with over 1&#10;billion masks on 11M licensed and privacy respecting images. The model is&#10;designed and trained to be promptable, so it can transfer zero-shot to new&#10;image distributions and tasks. We evaluate its capabilities on numerous tasks&#10;and find that its zero-shot performance is impressive -- often competitive with&#10;or even superior to prior fully supervised results. We are releasing the&#10;Segment Anything Model (SAM) and corresponding dataset (SA-1B) of 1B masks and&#10;11M images at https://segment-anything.com to foster research into foundation&#10;models for computer vision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.06706" label="2304.06706">
-        <attvalues>
-          <attvalue for="0" value="Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields" />
-          <attvalue for="1" value="  Neural Radiance Field training can be accelerated through the use of&#10;grid-based representations in NeRF's learned mapping from spatial coordinates&#10;to colors and volumetric density. However, these grid-based approaches lack an&#10;explicit understanding of scale and therefore often introduce aliasing, usually&#10;in the form of jaggies or missing scene content. Anti-aliasing has previously&#10;been addressed by mip-NeRF 360, which reasons about sub-volumes along a cone&#10;rather than points along a ray, but this approach is not natively compatible&#10;with current grid-based techniques. We show how ideas from rendering and signal&#10;processing can be used to construct a technique that combines mip-NeRF 360 and&#10;grid-based models such as Instant NGP to yield error rates that are 8% - 77%&#10;lower than either prior technique, and that trains 24x faster than mip-NeRF&#10;360.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.17603" label="2303.17603">
-        <attvalues>
-          <attvalue for="0" value="NeRF-Supervised Deep Stereo" />
-          <attvalue for="1" value="  We introduce a novel framework for training deep stereo networks effortlessly&#10;and without any ground-truth. By leveraging state-of-the-art neural rendering&#10;solutions, we generate stereo training data from image sequences collected with&#10;a single handheld camera. On top of them, a NeRF-supervised training procedure&#10;is carried out, from which we exploit rendered stereo triplets to compensate&#10;for occlusions and depth maps as proxy labels. This results in stereo networks&#10;capable of predicting sharp and detailed disparity maps. Experimental results&#10;show that models trained under this regime yield a 30-40% improvement over&#10;existing self-supervised methods on the challenging Middlebury dataset, filling&#10;the gap to supervised models and, most times, outperforming them at zero-shot&#10;generalization.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00680" label="2104.00680">
-        <attvalues>
-          <attvalue for="0" value="LoFTR: Detector-Free Local Feature Matching with Transformers" />
-          <attvalue for="1" value="  We present a novel method for local image feature matching. Instead of&#10;performing image feature detection, description, and matching sequentially, we&#10;propose to first establish pixel-wise dense matches at a coarse level and later&#10;refine the good matches at a fine level. In contrast to dense methods that use&#10;a cost volume to search correspondences, we use self and cross attention layers&#10;in Transformer to obtain feature descriptors that are conditioned on both&#10;images. The global receptive field provided by Transformer enables our method&#10;to produce dense matches in low-texture areas, where feature detectors usually&#10;struggle to produce repeatable interest points. The experiments on indoor and&#10;outdoor datasets show that LoFTR outperforms state-of-the-art methods by a&#10;large margin. LoFTR also ranks first on two public benchmarks of visual&#10;localization among the published methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.10502" label="2111.10502">
-        <attvalues>
-          <attvalue for="0" value="CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and&#10;  Scene Flow Estimation" />
-          <attvalue for="1" value="  In this paper, we study the problem of jointly estimating the optical flow&#10;and scene flow from synchronized 2D and 3D data. Previous methods either employ&#10;a complex pipeline that splits the joint task into independent stages, or fuse&#10;2D and 3D information in an &quot;early-fusion&quot; or &quot;late-fusion&quot; manner. Such&#10;one-size-fits-all approaches suffer from a dilemma of failing to fully utilize&#10;the characteristic of each modality or to maximize the inter-modality&#10;complementarity. To address the problem, we propose a novel end-to-end&#10;framework, called CamLiFlow. It consists of 2D and 3D branches with multiple&#10;bidirectional connections between them in specific layers. Different from&#10;previous work, we apply a point-based 3D branch to better extract the geometric&#10;features and design a symmetric learnable operator to fuse dense image features&#10;and sparse point features. Experiments show that CamLiFlow achieves better&#10;performance with fewer parameters. Our method ranks 1st on the KITTI Scene Flow&#10;benchmark, outperforming the previous art with 1/7 parameters. Code is&#10;available at https://github.com/MCG-NJU/CamLiFlow.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Optical Flow. Optical flow estimation aims to predict dense 2D motion for each pixel from a pair of frames. Traditional methods \cite{horn1981determining, black1996robust, brox2009large, weinzaepfel2013deepflow, brox2004warping, bruhn2005lucas} often formulate optical flow as an energy minimization problem. FlowNet \cite{dosovitskiy2015flownet} is the first end-to-end trainable CNN for optical flow estimation, which adopts an encoder-decoder architecture. FlowNet2 \cite{ilg2017flownet2} stacks several FlowNets into a larger one. PWC-Net \cite{sun2018pwc} and some other methods \cite{ranjan2017spynet, hur2019iterative, hui2018liteflownet, yang2019volumetric} apply iterative refinement using coarse-to-fine pyramids. RAFT \cite{teed2020raft} constructs 4D cost volumes for all pairs of pixels and updates the flow iteratively. Although achieving state-of-the-art performance, RAFT runs much slower than PWC-Net. Hence, our two branches are built upon the PWC architecture to achieve a better trade-off between accuracy and speed.&#10;&#10;Scene Flow from RGB-D Frames. RGB-D scene flow is the problem of estimating dense 3D motion for each pixel from a pair of stereo or RGB-D frames. Like optical flow, traditional methods \cite{menze2015osf, quiroga2014dense, jaimez2015primal, jaimez2015motion} explore variational optimization and discrete optimization and treat scene flow as an energy minimization problem. Recent methods \cite{behl2017isf, ma2019drisf, yang2020opticalexp, yang2021rigidmask} divide scene flow estimation into multiple subtasks and build a modular network with one or more submodules for each subtask. Although achieving remarkable progress, their submodules are independent of each other, which can not exploit the complementary characteristics of different modalities. RAFT-3D \cite{teed2021raft3d} concatenate images and depth maps to RGB-D frames at an early stage, followed by a unified 2D network which iteratively updates a dense field of pixel-wise SE3 motion. However, this kind of ``early fusion'' makes it hard for 2D CNNs to take advantage of the rich 3D structural information.&#10;&#10;Scene Flow from Point Clouds. Recently, researchers start to study scene flow estimation in 3D point clouds (e.g. from LiDAR) \cite{liu2019flownet3d, wang2020flownet3d++, gu2019hplflownet, wu2019pointpwc, puy2020flot, liu2019meteornet, wei2021pvraft, kittenplon2021flowstep3d}. Based on \cite{qi2017pointnet++}, FlowNet3D \cite{liu2019flownet3d} uses a flow embedding layer to represent the motion of points. FlowNet3D++ \cite{wang2020flownet3d++} achieves better performance by adding geometric constraints. Inspired by Bilateral Convolutional Layers, HPLFlowNet \cite{gu2019hplflownet} projects the points onto a permutohedral lattice. PointPWC-Net \cite{wu2019pointpwc} introduces a learnable cost volume for point clouds and estimates scene flow in a coarse-to-fine fashion. However, these methods do not exploit color features provided by images. As we demonstrate in our experiments, fusing point clouds with images can bring significant improvements.&#10;&#10;Camera-LiDAR Fusion. Cameras and LiDARs have complementary characteristics, facilitating many computer vision tasks, such as depth estimation \cite{ma2018sparse, you2019pseudo,feng2021advancing}, scene flow estimation \cite{battrawy2019lidarflow, rishav2020deeplidarflow}, 3D object detection \cite{qi2018frustum, liang2018continuous, xu2018pointfusion, vora2020pointpainting, chen2017multi}, etc. Some researchers \cite{you2019pseudo, battrawy2019lidarflow, qi2018frustum, vora2020pointpainting} build a modular network and perform result-level fusion, while the others \cite{ma2018sparse, feng2021advancing, rishav2020deeplidarflow, liang2018continuous, xu2018pointfusion} explore feature-level fusion schemes including early-fusion and late-fusion. Instead, we propose a multi-stage and bidirectional fusion pipeline, which not only fully utilizes the characteristic of each modality, but maximizes the inter-modality complementarity as well.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1911.01223" label="1911.01223">
-        <attvalues>
-          <attvalue for="0" value="Superpixel-Based Background Recovery from Multiple Images" />
-          <attvalue for="1" value="  In this paper, we propose an intuitive method to recover background from&#10;multiple images. The implementation consists of three stages: model&#10;initialization, model update, and background output. We consider the pixels&#10;whose values change little in all input images as background seeds. Images are&#10;then segmented into superpixels with simple linear iterative clustering. When&#10;the number of pixels labelled as background in a superpixel is bigger than a&#10;predefined threshold, we label the superpixel as background to initialize the&#10;background candidate masks. Background candidate images are obtained from input&#10;raw images with the masks. Combining all candidate images, a background image&#10;is produced. The background candidate masks, candidate images, and the&#10;background image are then updated alternately until convergence. Finally,&#10;ghosting artifacts is removed with the k-nearest neighbour method. An&#10;experiment on an outdoor dataset demonstrates that the proposed algorithm can&#10;achieve promising results.&#10;" />
-          <attvalue for="2" value="&#10;In most popular tourism places, it is usually difficult for tourists to take a photo without other tourists walking around on camera. An optional choice for tourists is to recover a desired background scenery image from a series of images taken in burst mode. Background recovery is a basic task of background modelling \cite{BOUWMANS20173}.&#10;&#10;So far, many background modelling methods have been proposed. Some methods presume that clean background images without foreground objects need to exist in the dataset \cite{BOUWMANS20173}. However, the assumption is not satisfied in the application. Therefore, in this work, we aim to recover a background scenery image from a series of images containing both background and foreground objects.&#10;&#10;Depending on data abstraction, background modelling methods are classified into three categories \cite{BOUWMANS20173}: pixel-level \cite{TemporalMedian,784637,s120912279,NovelRobustStatistica,Automaticocclusion}, region-level \cite{spatiotemporalcontinuities,PatchBasedBackground,Backgroundrecovery,SuperpixelsMotion,SuperpixelsSimilarity}, and hybrid methods \cite{Extractingessence}.&#10;&#10;Among pixel-level methods, Huang et al. \cite{TemporalMedian} have proposed a speed-up temporal median filter for background modelling. But it still requires that all background pixels exist in more than 50\% of frames. Gaussian mixture models (GMMs) \cite{784637} and kennel density estimation (KDE) methods \cite{s120912279} are often used to estimate the true probability density function of pixels value. But GMMs are difficult to determine the number of Gaussian distributions, while KDE is sensitive to a predefined band-width parameter. Both of them tend to misclassify foreground pixels, if a foreground pixel occurs more frequently than a background pixel at the same position.&#10;Wang and Suter \cite{NovelRobustStatistica} chose the pixel value which varies little along the time-line as a background pixel, since the value of a background pixel remains fixed. Nevertheless, it fails when a foreground object moves slowly or stays stationary in several frames.&#10;Herley et al. \cite{Automaticocclusion} assume that if a pixel has same value in any two images, it is a background pixel, which is a strong assumption.&#10;&#10;Among region-level methods, Varadarajan et al. \cite{spatiotemporalcontinuities} employed change detection on a certain number of the beginning frames, categorizing pixels into background, weak foreground, and strong foreground according to two predefined thresholds. Each categorized pixel is modified recursively according to its 8-connected neighbourhood to remove outliers. Afterwards, spatial continuity and temporal continuity metrics are used to remove ghosting artifacts. However, the spatial continuity metric cannot handle complex situations and the temporal continuity metric cannot deal with temporarily static foreground objects. &#10;&#10;Depending on the type of regions, region-level methods can be further categorized into patch-based methods and superpixel-based methods. A patch-based method proposed by Colombari and Fusiello \cite{PatchBasedBackground} can cope with heavy clutter.&#10;Shrotre and Karam \cite{Backgroundrecovery} recover background by measuring a 5-D (L, a, b, color channel, and gradients in horizontal and vertical directions) patch-based distance metric. However, it may fail to recover a background patch if a foreground object stays stationary in several frames. In addition, all patch-based methods may result in blocking artifacts and seams \cite{RandomWalk}.&#10;&#10;Unlike patches split without specific meanings, superpixels \cite{superpixelsstateoftheart} are visually meaningful entities. &#10;Matrix decomposition into low-rank and sparse components is an effective method for background subtraction \cite{guyon2012robust}, but the computational cost is very high. Javed et al. \cite{SuperpixelMatrix} presented a superpixel-based background subtraction method to improve matrix decomposition performance.&#10;Lim and Han \cite{SuperpixelsMotion} proposed to estimate motion models based on superpixels, and compute foreground and background probabilities for each pixel.&#10;Fang et al. \cite{SuperpixelsSimilarity} found pixel value changes are more similar in a superpixel than their ordinary neighbours. This neighbourhood information is useful for background subtraction.&#10;&#10;In this work, we propose a novel and intuitive hybrid method based on superpixels. It can overcome the drawbacks caused by patch-based methods mentioned above. The remainder of this paper is organized as follows. Section 2 describes the algorithm's details. The results on a real outdoor dataset are presented in Section 3. Section 4 concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Image Processing, Computer Science, Background Recovery, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1612.01601" label="1612.01601">
-        <attvalues>
-          <attvalue for="0" value="Superpixels: An Evaluation of the State-of-the-Art" />
-          <attvalue for="1" value="  Superpixels group perceptually similar pixels to create visually meaningful&#10;entities while heavily reducing the number of primitives for subsequent&#10;processing steps. As of these properties, superpixel algorithms have received&#10;much attention since their naming in 2003. By today, publicly available&#10;superpixel algorithms have turned into standard tools in low-level vision. As&#10;such, and due to their quick adoption in a wide range of applications,&#10;appropriate benchmarks are crucial for algorithm selection and comparison.&#10;Until now, the rapidly growing number of algorithms as well as varying&#10;experimental setups hindered the development of a unifying benchmark. We&#10;present a comprehensive evaluation of 28 state-of-the-art superpixel algorithms&#10;utilizing a benchmark focussing on fair comparison and designed to provide new&#10;insights relevant for applications. To this end, we explicitly discuss&#10;parameter optimization and the importance of strictly enforcing connectivity.&#10;Furthermore, by extending well-known metrics, we are able to summarize&#10;algorithm performance independent of the number of generated superpixels,&#10;thereby overcoming a major limitation of available benchmarks. Furthermore, we&#10;discuss runtime, robustness against noise, blur and affine transformations,&#10;implementation details as well as aspects of visual quality. Finally, we&#10;present an overall ranking of superpixel algorithms which redefines the&#10;state-of-the-art and enables researchers to easily select appropriate&#10;algorithms and the corresponding implementations which themselves are made&#10;publicly available as part of our benchmark at&#10;davidstutz.de/projects/superpixel-benchmark/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.11038" label="2403.11038">
-        <attvalues>
-          <attvalue for="0" value="Texture Edge detection by Patch consensus (TEP)" />
-          <attvalue for="1" value="  We propose Texture Edge detection using Patch consensus (TEP) which is a&#10;training-free method to detect the boundary of texture. We propose a new simple&#10;way to identify the texture edge location, using the consensus of segmented&#10;local patch information. While on the boundary, even using local patch&#10;information, the distinction between textures are typically not clear, but&#10;using neighbor consensus give a clear idea of the boundary. We utilize local&#10;patch, and its response against neighboring regions, to emphasize the&#10;similarities and the differences across different textures. The step of&#10;segmentation of response further emphasizes the edge location, and the&#10;neighborhood voting gives consensus and stabilize the edge detection. We&#10;analyze texture as a stationary process to give insight into the patch width&#10;parameter verses the quality of edge detection. We derive the necessary&#10;condition for textures to be distinguished, and analyze the patch width with&#10;respect to the scale of textures. Various experiments are presented to validate&#10;the proposed model.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Texture has been explored for decades \cite{van1985texture} and fruitful results are established for different type of textures: &#10;Markov random field \cite{raad2018survey} is widely used for texture synthesis. Its generative feature fits well with the randomness of certain types of textures (wood surface, sand); Lattice based method \cite{liu2004computational} is powerful in modeling highly symmetric and periodic texture, when the texton is relatively well defined (wall paper, honeycomb); Frequency/Wavelet analysis \cite{livens1997wavelets, unser1995texture} utilize spacial filters to vectorize textures and plays an important role in image compression \cite{lewis1992image} and texture classification and segmentation \cite{wang1990texture, jain1991unsupervised}. We refer to \cite{tuceryan1993texture,raad2018survey} for a comprehensive review of classical models. &#10;Textures remains to be a challenging topic, since there is no general nor precise definition of textures, and the boundaries of different textures are especially difficult to recognize. &#10;&#10;We explore texture edge detection and segmentation. &#10;The classical Canny edge detection \cite{canny1986computational} detects edge locations by thinning the mask function, which is computed by applying thresholds to the magnitude of $|\nabla U_0|$. &#10;One of the most well-known variational segmentation models is the Mumford-Shah functional \cite{mumford1989optimal}, &#10;\begin{align}&#10;&#09;E_{MS}(U, \Gamma) = \alpha\int_{\Omega\backslash\Gamma}|\nabla U|^2 dx + \beta &#10; \mathcal{H}^1(\Gamma) &#10; + \int_{\Omega\backslash\Gamma}(U-U_0)^2 dx.&#10;\end{align}&#10;Here, $\alpha$ and $\beta$ are positive parameters, $\Omega \subset \mathbb{R}$ is a bounded image domain, $U_0:\Omega\to\mathbb{R}$ is the given image, and $ \mathcal{H}^1(\Gamma)$ denotes one-dimensional Hausdorff measure of the object boundary $\Gamma$. Chan and Vese \cite{active} proposed using level set method, and it gives very effective piece-wise constant segmentation.&#10;For textured image segmentation, some texture descriptors, such as Gabor filter, can be used with these models. &#10;Gabor filter \cite{mehrotra1992gabor} can detect localized frequency response in varied orientations and scales. &#10;Figure \ref{fig:intro} illustrates challenges of texture edge detection.&#10;For a comprehensive review of classical texture segmentation models, we refer to \cite{ilea2011image}.&#10;In real images, there are many different types of textures and it is typically difficult to find a proper filter bank that is suitable for all types of images. &#10;More recent network based approaches with data-adaptive property is capable of accomplishing high level tasks as semantic segmentation, e.g., \cite{bertasius2015deepedge, 9093290, xie15hed, he2019bi, liu2017richer}, assuming the network is well-trained. &#10;&#10;In this paper, we propose filter-free and training-free approach utilizing local patches response to capture the similarities and the differences between textures. Non-local filter \cite{nonlocal} for image denoising averages pixel intensity values over similar image patches effectively. &#10;In \cite{efros1999texture}, the authors extend this to texture synthesis algorithm, where the pixels are selected over similar image patches to regenerate textures. &#10;One of the difficulties in applying such nonlocal filter for texture edge detection is that near the boundary of texture, there may not be similar patches to give the clear boundary, different from denoising non textures edges. &#10;&#10;We use local patch information utilizing similar responses within one texture, and different responses against different textures, and propose a Texture Edge detection method by Patch consensus (TEP). &#10;We propose a consensus based edge detection, which utilize the fact that away from the boundary often gives a clear idea about where the boundary of the texture should be located. We analyze the statistical condition for the texture edge to be detected by patch-wise similarity, and explore the relation between the patch width and the performance of the proposed method. &#10;The contributions of this paper are as follows:&#10;\begin{itemize}&#10;\item{We propose a simple training-free filter-free Texture Edge detection method using patch consensus (TEP). }&#10;\item{We statistically analyze when the texture can be separated by the patch consensus. }&#10;\item{Numerical results are presented to validate the proposed model. }&#10;\end{itemize}&#10;&#10;The paper is outlined as follows: In Section \ref{sec:method}, the details of texture edge detection with Patch consensus (TEP) is presented. Statistical analysis of the proposed model is provided in Section \ref{sec:analysis}. In Section \ref{sec:Numerical}, we present the algorithms and numerical implementation details, and in Section \ref{sec:experiments} various experiments with comparisons and applications are presented. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Texture Edge Detection, Computer Science, Computer Vision Algorithms, Image Segmentation Techniques, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1707.07184" label="1707.07184">
-        <attvalues>
-          <attvalue for="0" value="A survey of exemplar-based texture synthesis" />
-          <attvalue for="1" value="  Exemplar-based texture synthesis is the process of generating, from an input&#10;sample, new texture images of arbitrary size and which are perceptually&#10;equivalent to the sample. The two main approaches are statistics-based methods&#10;and patch re-arrangement methods. In the first class, a texture is&#10;characterized by a statistical signature; then, a random sampling conditioned&#10;to this signature produces genuinely different texture images. The second class&#10;boils down to a clever &quot;copy-paste&quot; procedure, which stitches together large&#10;regions of the sample. Hybrid methods try to combine ideas from both approaches&#10;to avoid their hurdles. The recent approaches using convolutional neural&#10;networks fit to this classification, some being statistical and others&#10;performing patch re-arrangement in the feature space. They produce impressive&#10;synthesis on various kinds of textures. Nevertheless, we found that most real&#10;textures are organized at multiple scales, with global structures revealed at&#10;coarse scales and highly varying details at finer ones. Thus, when confronted&#10;with large natural images of textures the results of state-of-the-art methods&#10;degrade rapidly, and the problem of modeling them remains wide open.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1412.1123" label="1412.1123">
-        <attvalues>
-          <attvalue for="0" value="DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour&#10;  Detection" />
-          <attvalue for="1" value="  Contour detection has been a fundamental component in many image segmentation&#10;and object detection systems. Most previous work utilizes low-level features&#10;such as texture or saliency to detect contours and then use them as cues for a&#10;higher-level task such as object detection. However, we claim that recognizing&#10;objects and predicting contours are two mutually related tasks. Contrary to&#10;traditional approaches, we show that we can invert the commonly established&#10;pipeline: instead of detecting contours with low-level cues for a higher-level&#10;recognition task, we exploit object-related features as high-level cues for&#10;contour detection.&#10;  We achieve this goal by means of a multi-scale deep network that consists of&#10;five convolutional layers and a bifurcated fully-connected sub-network. The&#10;section from the input layer to the fifth convolutional layer is fixed and&#10;directly lifted from a pre-trained network optimized over a large-scale object&#10;classification task. This section of the network is applied to four different&#10;scales of the image input. These four parallel and identical streams are then&#10;attached to a bifurcated sub-network consisting of two independently-trained&#10;branches. One branch learns to predict the contour likelihood (with a&#10;classification objective) whereas the other branch is trained to learn the&#10;fraction of human labelers agreeing about the contour presence at a given point&#10;(with a regression criterion).&#10;  We show that without any feature engineering our multi-scale deep learning&#10;approach achieves state-of-the-art results in contour detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.11326" label="2309.11326">
-        <attvalues>
-          <attvalue for="0" value="How to turn your camera into a perfect pinhole model" />
-          <attvalue for="1" value="  Camera calibration is a first and fundamental step in various computer vision&#10;applications. Despite being an active field of research, Zhang's method remains&#10;widely used for camera calibration due to its implementation in popular&#10;toolboxes. However, this method initially assumes a pinhole model with&#10;oversimplified distortion models. In this work, we propose a novel approach&#10;that involves a pre-processing step to remove distortions from images by means&#10;of Gaussian processes. Our method does not need to assume any distortion model&#10;and can be applied to severely warped images, even in the case of multiple&#10;distortion sources, e.g., a fisheye image of a curved mirror reflection. The&#10;Gaussian processes capture all distortions and camera imperfections, resulting&#10;in virtual images as though taken by an ideal pinhole camera with square&#10;pixels. Furthermore, this ideal GP-camera only needs one image of a square grid&#10;calibration pattern. This model allows for a serious upgrade of many algorithms&#10;and applications that are designed in a pure projective geometry setting but&#10;with a performance that is very sensitive to nonlinear lens distortions. We&#10;demonstrate the effectiveness of our method by simplifying Zhang's calibration&#10;method, reducing the number of parameters and getting rid of the distortion&#10;parameters and iterative optimization. We validate by means of synthetic data&#10;and real world images. The contributions of this work include the construction&#10;of a virtual ideal pinhole camera using Gaussian processes, a simplified&#10;calibration method and lens distortion removal.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Camera calibration is a vital first step in numerous computer vision applications, ranging from photogrammetry \cite{li2022review} and depth estimation \cite{mertan2022single} to robotics \cite{khan2019vision} and SLAM \cite{smith2006real,devernay2001straight}. As such, it is still a very active field of research, resulting in a myriad of calibration techniques. \cite{beardsley1992camera,caprile1990using,raza2019artificial,zheng2022image,puig2012calibration}. In this work, we also use the term camera for systems such as multi-camera systems or catadioptric systems \cite{galan2019design} which also include a mirror.&#10;&#10;Several attempts have been made to unify the calibration procedures for different camera types and camera systems \cite{Ramalingam2017,Hartley2004,liao2023deep}. However, the most popular method in practice today is still Zhang's method \cite{zhang2000flexible}, which is the basis for the camera calibration toolboxes of both MATLAB and OpenCV. This method assumes a pinhole model with additional lens distortions. The resulting calibration is a compromise between all intrinsic and extrinsic values, including the distortion model parameters. This approach has several drawbacks. First, the pinhole assumption is not applicable to non-central cameras. Second, the calibration is the result of a converging optimization process in which one parameter is adjusted in favour of another one to obtain a better optimum, without actual justification. Third, the proposed distortion models for radial and tangential distortion are in some cases oversimplifications, for instance when the distortion is not perfectly radially symmetric or when the centre of distortion is not at the principle point of the camera. &#10;&#10;In this work, we propose a new approach in which we first perform a pre-processing step on the images to remove all distortions. Next, the undistorted images serve as input for a simplified version of Zhang's method for perfect pinhole cameras. By distortions, we mean all deformations resulting from lenses, camera hardware imperfections, faults in the calibration board and even noise. To capture these, we rely on Gaussian processes \cite{Rasmussen2006}. They are a non-parametric Bayesian regression technique that are very well suited to handle sparse noisy datasets.&#10;&#10;The proposed method applies to a variety of 2D-cameras. For any such camera, we train a Gaussian process on the relationship from pixel coordinates of the corners detected in an image of a square grid pattern (e.g., a checkerboard) to a perfectly spread square lattice of virtual 2D points. Only one image of the calibration board is needed for this training. This lattice can be seen as the non-linear projection of the checkerboard corners in the original camera image to a virtual image plane, consisting of virtual pixels. &#10;The Gaussian process captures all possible distortions. &#10;&#10;All future images can now be mapped to the same virtual image plane. As all distortions are removed, we are left with virtual images as though they were taken by an ideal pinhole camera. This method does not need to assume any distortion model and can be applied to severely warped images, even in the case of multiple distortion sources, e.g., a fisheye image of a curved mirror reflection.&#10;&#10;The process of first taking images by the given camera followed by the Gaussian process mapping to this virtual image plane can be considered as acquiring images by a virtual camera, called the GP-camera, &#10;replacing the original camera. We will validate that the imaging by this GP-camera corresponds to a perspectivity (central projective transformation) from the 3D scene to the virtual image plane. In other words, we prove that the GP camera is a pinhole camera. &#10;&#10;The main benefit of obtaining an ideal pinhole camera is that a lot of well-studied algorithms and applications can be employed on its images. These include pose estimation, depth estimation, epipolar geometry, shape from motion, 3D scene reconstruction, optical flow, externally calibrating multiple cameras and other 3D sensors, etc. Many of these assume a central projection. For a treatise on these topics, we refer to the industry standard book \cite{Hartley2004}. Our model allows for a serious upgrade of these algorithms and applications that are designed in a pure projective geometry setting. Their performance is very sensitive to non-linear lens distortions. In particular, the quality of calibration techniques that lean on sphere images is drastically improved when rectified images with square pixels are available \cite{Sun2015,Penne2018,Penne2019}. &#10;&#10;In \cite{Ranganathan2012}, Gaussian processes are also used to model lens distortions. However, they serve as a surrogate model for the function that captures the lens distortion. As such, they are still part of the iterative camera calibration process. Lens distortion based on one checkerboard pattern is proposed in \cite{wu2015lens}. However, they implement the Levenberg–Marquardt algorithm to find an optimal set of parameter values for their distortion models. Gaussian processes are non-parametric and as such do not depend on this. A deep learning variant of this can be found in \cite{zhang2022learning}.&#10;&#10;The contributions of this works are:&#10;\begin{itemize}&#10; \item We explain how to construct a virtual ideal pinhole camera out of a variety of non-pinhole cameras using Gaussian processes.&#10; \item We show that our calibrated GP-camera using a simplified version of Zhang's method leads to more accurate measurements compared to the calibrated original camera using the general Zhang's method with iteration.&#10; \item We show how our method can be used to remove heavy distortions in images.&#10;\end{itemize}&#10;&#10;The rest of the paper is structured as follows: Section \ref{Methods} provides some theoretical background of Gaussian processes. It explains the construction and operation of a virtual GP-camera, and describes how we will validate this pinhole model. In Section \ref{Results} we show the results and compare our method to the MATLAB implementation of Zhang's method. We discuss these results in Section \ref{Discussion}. Finally, we formulate our conclusions in Section \ref{Conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Camera Calibration, Computer Science, Image Distortion Correction, Geometry" />
-        </attvalues>
-      </node>
-      <node id="2104.06456" label="2104.06456">
-        <attvalues>
-          <attvalue for="0" value="Single Image Depth Estimation: An Overview" />
-          <attvalue for="1" value="  We review solutions to the problem of depth estimation, arguably the most&#10;important subtask in scene understanding. We focus on the single image depth&#10;estimation problem. Due to its properties, the single image depth estimation&#10;problem is currently best tackled with machine learning methods, most&#10;successfully with convolutional neural networks. We provide an overview of the&#10;field by examining key works. We examine non-deep learning approaches that&#10;mostly predate deep learning and utilize hand-crafted features and assumptions,&#10;and more recent works that mostly use deep learning techniques. The single&#10;image depth estimation problem is tackled first in a supervised fashion with&#10;absolute or relative depth information acquired from human or sensor-labeled&#10;data, or in an unsupervised way using unlabelled stereo images or video&#10;datasets. We also study multitask approaches that combine the depth estimation&#10;problem with related tasks such as semantic segmentation and surface normal&#10;estimation. Finally, we discuss investigations into the mechanisms, principles,&#10;and failure cases of contemporary solutions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.10559" label="2303.10559">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning for Camera Calibration and Beyond: A Survey" />
-          <attvalue for="1" value="  Camera calibration involves estimating camera parameters to infer geometric&#10;features from captured sequences, which is crucial for computer vision and&#10;robotics. However, conventional calibration is laborious and requires dedicated&#10;collection. Recent efforts show that learning-based solutions have the&#10;potential to be used in place of the repeatability works of manual&#10;calibrations. Among these solutions, various learning strategies, networks,&#10;geometric priors, and datasets have been investigated. In this paper, we&#10;provide a comprehensive survey of learning-based camera calibration techniques,&#10;by analyzing their strengths and limitations. Our main calibration categories&#10;include the standard pinhole camera model, distortion camera model, cross-view&#10;model, and cross-sensor model, following the research trend and extended&#10;applications. As there is no benchmark in this community, we collect a holistic&#10;calibration dataset that can serve as a public platform to evaluate the&#10;generalization of existing methods. It comprises both synthetic and real-world&#10;data, with images and videos captured by different cameras in diverse scenes.&#10;Toward the end of this paper, we discuss the challenges and provide further&#10;research directions. To our knowledge, this is the first survey for the&#10;learning-based camera calibration (spanned 8 years). The summarized methods,&#10;datasets, and benchmarks are available and will be regularly updated at&#10;https://github.com/KangLiao929/Awesome-Deep-Camera-Calibration.&#10;" />
-          <attvalue for="2" value="&#10;&#10;C{amera} calibration is a fundamental and indispensable field in computer vision and it has a long research history \cite{duane1971close, maybank1992theory, weng1992camera, zhang2000flexible, qi2023minimal}, tracing back to around 60 years ago\cite{brown1966decentering}. The first step for many vision and robotics tasks is to calibrate the intrinsic (image sensor and distortion parameters) and/or extrinsic (rotation and translation) camera parameters, ranging from computational photography, and multi-view geometry, to 3D reconstruction. In terms of the task type, there are different techniques to calibrate the standard pinhole camera, fisheye lens camera, stereo camera, light field camera, event camera, and LiDAR-camera system, etc. Figure~\ref{fig:teaser} shows the popular calibration objectives, models, and extended applications in camera calibration.&#10;&#10;Traditional methods for camera calibration generally depend on hand-crafted features and model assumptions. These methods can be broadly divided into three categories. The most prevalent one involves using a known calibration target (e.g., a checkerboard) as it is deliberately moved in the 3D scene \cite{zhang1999flexible, gasparini2009plane, shah1994simple}. Then, the camera captures the target from different viewpoints and the checkerboard corners are detected for calculating the camera parameters. However, such a procedure requires cumbersome manual interactions and it cannot achieve automatic calibration ``in the wild''. To pursue better flexibility, the second category of camera calibration, i.e., the geometric-prior-based calibration has been largely studied \cite{barreto2005geometric, Carroll, Bukhari, Miguel}. To be specific, the geometric structures are leveraged to model the 3D-2D correspondence in the scene, such as lines and vanishing points. However, this type of method heavily relies on structured man-made scenes containing rich geometric priors, leading to poor performance when applied to general environments. The third category is self-calibration\cite{faugeras1992camera, fraser1997digital, hartley1994self}. Such a solution takes a sequence of images as inputs and estimates the camera parameters using multi-view geometry. The accuracy of self-calibration, however, is constrained by the limits of the feature detectors, which can be influenced by diverse lighting conditions and textures. The above parametric models are based on the physical interpretation of camera geometry. While they are user-friendly, they tend to be tailored for specific camera models and may not offer optimal accuracy. Instead, non-parametric models~\cite{camposeco2015non, schops2020having, pan2022camera} link each image pixel to its 3D observation ray, eliminating the constraints of parametric models.&#10;&#10;Since there are many standard techniques for calibrating cameras in an industry/laboratory implementation\cite{opencv, matlab}, this process is usually ignored in recent developments. However, calibrating single and wild images remains challenging, especially when images are collected from websites and unknown camera models. This challenge motivates the researchers to investigate a new paradigm.&#10;&#10;Recently, deep learning has brought new inspirations to camera calibration and its applications. Learning-based methods achieve state-of-the-art performances on various tasks with higher efficiency. Particularly, diverse deep neural networks have been developed, such as convolutional neural networks (CNNs), generative adversarial networks (GANs), vision transformers (ViTs), and diffusion models, of which the high-level semantic features show more powerful representation capability compared with the hand-crafted features. Moreover, diverse learning strategies have been exploited to boost the geometric perception of networks. Learning-based methods offer a flexible and end-to-end camera calibration solution, without manual interventions or calibration targets, which sets them apart from traditional methods. Furthermore, some of these methods achieve camera model-free and label-free calibration, showing promising and meaningful applications.&#10;&#10;With the rapid increase in the number of learning-based camera calibration methods, it has become increasingly challenging to keep up with new advances. Consequently, there is an urgent need to analyze existing works and foster a community dedicated to this field. Previously, some surveys, e.g., \cite{salvi2002comparative, hughes2008review, fan2022wide} only focused on a specific task/camera in camera calibration or one type of approach. For instance, Salvi et al. \cite{salvi2002comparative} reviewed the traditional camera calibration methods in terms of the algorithms. Hughes et al. \cite{hughes2008review} provided a detailed review for calibrating fisheye cameras with traditional solutions. While Fan et al. \cite{fan2022wide} discussed both the traditional methods and deep learning methods, their survey only considers calibrating the wide-angle cameras. In addition, due to the few amount of reviewed learning-based methods (around 10 papers), the readers are difficult to picture the development trend of general camera calibration in Fan et al. \cite{fan2022wide}. &#10;&#10;In this paper, we provide a comprehensive and in-depth overview of recent advances in learning-based camera calibration, covering over 200 related papers. We also discuss potential directions for further improvements and examine various types of cameras and targets. To facilitate future research on different topics, we categorize the current solutions according to calibration objectives and applications. In addition to fundamental parameters such as focal length, rotation, and translation, we also provide detailed reviews for correcting image distortion (radial distortion and rolling shutter distortion), estimating cross-view mapping, calibrating camera-LiDAR systems, and other applications. Such a trend follows the development of cameras and market demands for virtual reality, autonomous driving, neural rendering, etc. &#10;&#10;To our best knowledge, this is the first survey of the learning-based camera calibration and its extended applications, it has the following unique contributions. (1) Our work mainly follows recent advances in deep learning-based camera calibration. In-depth analysis and discussion in various aspects are offered, including publications, network architecture, loss functions, datasets, evaluation metrics, learning strategies, implementation platforms, etc. The detailed information of each literature is listed in Table \ref{table:methods:standard:distortion} and Table \ref{table:methods:cross-view:cross-sensor}. (2) Despite the calibration algorithm, we comprehensively review the classical camera models and their extended models. In particular, we summarize the redesigned calibration objectives in deep learning since some traditional calibration objectives are verified to be hard to learn by neural networks. (3) We collect a dataset containing images and videos captured by different cameras in different environments, which can serve as a platform to evaluate the generalization of existing methods. (4) We discuss the open challenges in learning-based camera calibration and propose future directions to guide further research in this field. (5) An open-source repository is created that provides a taxonomy of all reviewed works and benchmarks. The repository will be updated regularly in.&#10;&#10;In the following sections, we discuss and analyze various aspects of learning-based camera calibration. The remainder of this paper is organized as follows. In Section~\ref{sec2}, we provide the concrete learning paradigms and learning strategies of the learning-based camera calibration. Subsequently, we introduce and discuss the specific methods based on the standard camera model, distortion model, cross-view model, and cross-sensor model in Section~\ref{sec:pure}, Section~\ref{sec:distortion}, Section~\ref{sec:projection}, and Section~\ref{sec:hybrid}, respectively (see Figure~\ref{fig:taxonomy}). The collected benchmark for calibration methods is depicted in Section~\ref{sec:evaluation}. Finally, we conclude the learning-based camera calibration and suggest the future directions of this community in Section~\ref{sec:Future}.&#10; &#09;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.00158" label="2202.00158">
-        <attvalues>
-          <attvalue for="0" value="Learning-Based Framework for Camera Calibration with Distortion&#10;  Correction and High Precision Feature Detection" />
-          <attvalue for="1" value="  Camera calibration is a crucial technique which significantly influences the&#10;performance of many robotic systems. Robustness and high precision have always&#10;been the pursuit of diverse calibration methods. State-of-the-art calibration&#10;techniques based on classical Zhang's method, however, still suffer from&#10;environmental noise, radial lens distortion and sub-optimal parameter&#10;estimation. Therefore, in this paper, we propose a hybrid camera calibration&#10;framework which combines learning-based approaches with traditional methods to&#10;handle these bottlenecks. In particular, this framework leverages&#10;learning-based approaches to perform efficient distortion correction and robust&#10;chessboard corner coordinate encoding. For sub-pixel accuracy of corner&#10;detection, a specially-designed coordinate decoding algorithm with embed&#10;outlier rejection mechanism is proposed. To avoid sub-optimal estimation&#10;results, we improve the traditional parameter estimation by RANSAC algorithm&#10;and achieve stable results. Compared with two widely-used camera calibration&#10;toolboxes, experiment results on both real and synthetic datasets manifest the&#10;better robustness and higher precision of the proposed framework. The massive&#10;synthetic dataset is the basis of our framework's decent performance and will&#10;be publicly available along with the code at&#10;https://github.com/Easonyesheng/CCS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.08040" label="2101.08040">
-        <attvalues>
-          <attvalue for="0" value="1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for&#10;  Tracking" />
-          <attvalue for="1" value="  We extend the classical tracking-by-detection paradigm to this&#10;tracking-any-object task. Solid detection results are first extracted from TAO&#10;dataset. Some state-of-the-art techniques like \textbf{BA}lanced-\textbf{G}roup&#10;\textbf{S}oftmax (\textbf{BAGS}\cite{li2020overcoming}) and&#10;DetectoRS\cite{qiao2020detectors} are integrated during detection. Then we&#10;learned appearance features to represent any object by training feature&#10;learning networks. We ensemble several models for improving detection and&#10;feature representation. Simple linking strategies with most similar appearance&#10;features and tracklet-level post association module are finally applied to&#10;generate final tracking results. Our method is submitted as \textbf{AOA} on the&#10;challenge website. Code is available at&#10;https://github.com/feiaxyt/Winner_ECCV20_TAO.&#10;" />
-          <attvalue for="2" value="&#10;Tracking Any Object (TAO)\cite{2020TAO} dataset is the first work to bring more sophisticated categories in an open world to the multiple-objects-tracking (MOT) community. Besides, despite the whole consecutive frames are given, they are only annotated at 1 FPS, which makes some popular MOT frameworks not work. Since there are over 1200 categories in it, detecting and tracking on such a long-tailed dataset is super challenging. In this paper, we will describe our method used for TAO and our thoughts about tracking on such a hard setting.&#10;&#10;We follow the classical tracking-by-detection framework to build TAO pipeline. Since better detection leads to powerful tracking results, especially for this long-tailed task, we carefully built our fundamental detection model with Mask-RCNN. More details can be found in 2.1.&#10;In the tasks of multiple objects tracking on pedestrians or vehicles (like KITTI, MOT-16, MOT-17, etc.), tracking is usually done by combining appearance and motion features since it also tracks on real-time videos. But as for TAO, we tested some mainstream tracking approaches within real-time and low-frame-rate ones. We chose to track with anything-appearance feature on 1 FPS. More details can be found in 2.2 and 2.3.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Feature Learning, Machine Learning, Engineering, Computer Vision, Tracking Algorithms, Mathematics, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2005.10356" label="2005.10356">
-        <attvalues>
-          <attvalue for="0" value="TAO: A Large-Scale Benchmark for Tracking Any Object" />
-          <attvalue for="1" value="  For many years, multi-object tracking benchmarks have focused on a handful of&#10;categories. Motivated primarily by surveillance and self-driving applications,&#10;these datasets provide tracks for people, vehicles, and animals, ignoring the&#10;vast majority of objects in the world. By contrast, in the related field of&#10;object detection, the introduction of large-scale, diverse datasets (e.g.,&#10;COCO) have fostered significant progress in developing highly robust solutions.&#10;To bridge this gap, we introduce a similarly diverse dataset for Tracking Any&#10;Object (TAO). It consists of 2,907 high resolution videos, captured in diverse&#10;environments, which are half a minute long on average. Importantly, we adopt a&#10;bottom-up approach for discovering a large vocabulary of 833 categories, an&#10;order of magnitude more than prior tracking benchmarks. To this end, we ask&#10;annotators to label objects that move at any point in the video, and give names&#10;to them post factum. Our vocabulary is both significantly larger and&#10;qualitatively different from existing tracking datasets. To ensure scalability&#10;of annotation, we employ a federated approach that focuses manual effort on&#10;labeling tracks for those relevant objects in a video (e.g., those that move).&#10;We perform an extensive evaluation of state-of-the-art trackers and make a&#10;number of important discoveries regarding large-vocabulary tracking in an&#10;open-world. In particular, we show that existing single- and multi-object&#10;trackers struggle when applied to this scenario in the wild, and that&#10;detection-based, multi-object trackers are in fact competitive with&#10;user-initialized ones. We hope that our dataset and analysis will boost further&#10;progress in the tracking community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.12904" label="2310.12904">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Object Localization in the Era of Self-Supervised ViTs: A&#10;  Survey" />
-          <attvalue for="1" value="  The recent enthusiasm for open-world vision systems show the high interest of&#10;the community to perform perception tasks outside of the closed-vocabulary&#10;benchmark setups which have been so popular until now. Being able to discover&#10;objects in images/videos without knowing in advance what objects populate the&#10;dataset is an exciting prospect. But how to find objects without knowing&#10;anything about them? Recent works show that it is possible to perform&#10;class-agnostic unsupervised object localization by exploiting self-supervised&#10;pre-trained features. We propose here a survey of unsupervised object&#10;localization methods that discover objects in images without requiring any&#10;manual annotation in the era of self-supervised ViTs. We gather links of&#10;discussed methods in the repository&#10;https://github.com/valeoai/Awesome-Unsupervised-Object-Localization.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object localization in 2D images is a key task for many perception systems, e.g., autonomous robots and cars, augmented reality headsets, visual search engines, etc. Depending on the application, the task can take different forms such as object detection \cite{ren2015fasterrcnn,carion2020detr} and instance segmentation \cite{he2017mask, chen2018deeplab}.&#10;Achieving good results in these tasks has traditionally hinged on one critical factor: access to extensive, meticulously annotated datasets \cite{coco2014,pascal-voc-2007} to build and train deep neural networks.&#10;&#10;However, this paradigm comes with inherent limitations.&#10;The first obvious one is the high cost and tediousness involved in acquiring these datasets. The second limitation stems from the finite and pre-defined nature of the set of object classes which significantly narrows the scope of what supervised models can perceive and identify.&#10;This becomes particularly problematic in contexts where the ability to recognize and respond to unknown or unconventional objects is crucial.&#10;For example, in autonomous driving applications, where the road can present a multitude of unpredictable elements, the need to apprehend whatever comes into view is paramount.&#10;&#10;The high interest around the recent Segment Anything (SAM) \cite{kirillov2023segment} model shows the desire for the community to segment any object in an image in a class-agnostic fashion. Although the results obtained with the fully-supervised SAM model &#10;--- trained on 11M images carefully annotated by humans with 1B masks --- are exciting, &#10;recent works have shown that impressive object localization can be obtained without human in the loop. By avoiding to rely on human-made annotation, we can hope to obtain systems which (1) would not suffer human biases, (2) could generalize to new objects / domains.&#10;Moreover, SAM necessitates prompts like points, boxes, or coarse masks to indicate the object to be segmented. &#10;As opposed to SAM, unsupervised object localization methods are able to find the objects of interest without prompts. &#10;&#10;Indeed, a solution to discover objects of interest without relying on annotations is to perform unsupervised class-agnostic object localization.&#10;This challenging task consists in localizing objects in images with no human supervision. Such problem has recently gained a lot of attention as shown by the evolution of the number of papers written about `unsupervised object localization' (see \ref{fig:hist-publi}).&#10;Moreover, as shown in \ref{fig:res-in-time}, recent methods (e.g., MOST \cite{rambhatla2023most} and MOVE \cite{bielski2022move}) have achieved impressive results without any annotations, predicting accurate object boxes in over 75\% of the images in the VOC07 \cite{pascal-voc-2007} dataset. &#10;&#10;The recent progress in unsupervised localization tasks owes its success to two critical factors.&#10;First, Vision Transformer (ViT) models \cite{dosovitskiy2021vit} provide global correlations between patches when Convolutional Neural Networks (CNNs) only correlate pixels in a local receptive field.&#10;Second, self-supervised representation learning \cite{caron2021dino, chen2021mocov3, he2022mae} has improved and scaled to massive datasets for feature learning. These techniques can now extract local and global semantically meaningful features from the weak signal provided by pretext tasks.&#10;With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}. &#10;&#10;In this survey, we propose to review unsupervised object localization methods in the era of self-supervised ViTs.&#10;We thoroughly present and detail all recent methods addressing this topic and comprehensively organize them for the community. These methods are summarized in \ref{tab:method_overview}.&#10;To our knowledge, this is the first survey on such topic and we would like to point the reader to related surveys on image classification with limited annotations~\cite{schmarje2021survey_classification}, weakly supervised object localization~\cite{zhang2022weakly_supervised_survey,shao2022weakly_supervised_survey}, object localization on natural scenes \cite{choudhuri2018survey_localization_natural}, object instance segmentation \cite{sharma2022survey_segmentation}, object segmentation \cite{wang2022review_segmentation}, and object detection \cite{amjoud2023review_detection,shehzadi2023detection_transformers}.&#10;&#10;The survey is organized as follows \OS{(state-of-the-art results are gathered in \ref{sec:sota})}:&#10;\begin{itemize}&#10; \item \ref{sec:background}: We comprehensively present the different tasks used to evaluate object localization capabilities, as shown in \ref{fig:unsup-tasks}, as well as the corresponding metrics and typical datasets used to evaluate the methods.&#10; \item \ref{sec:ssonly}: We review modern solutions that &#10; exploit &#10; ViT self-supervised features to produce localization masks in a training-free way.&#10; \item \ref{sec:training}: We detail different self-training strategies employed to enhance the quality of coarse localization mined in the self-supervised features. &#10; \item \ref{sec:discussion}: We discuss the success and failures of presented methods and open to different means to obtain object localization information without manual annotation.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science, Object Localization, Unsupervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2304.05387" label="2304.05387">
-        <attvalues>
-          <attvalue for="0" value="MOST: Multiple Object localization with Self-supervised Transformers for&#10;  object discovery" />
-          <attvalue for="1" value="  We tackle the challenging task of unsupervised object localization in this&#10;work. Recently, transformers trained with self-supervised learning have been&#10;shown to exhibit object localization properties without being trained for this&#10;task. In this work, we present Multiple Object localization with&#10;Self-supervised Transformers (MOST) that uses features of transformers trained&#10;using self-supervised learning to localize multiple objects in real world&#10;images. MOST analyzes the similarity maps of the features using box counting; a&#10;fractal analysis tool to identify tokens lying on foreground patches. The&#10;identified tokens are then clustered together, and tokens of each cluster are&#10;used to generate bounding boxes on foreground regions. Unlike recent&#10;state-of-the-art object localization methods, MOST can localize multiple&#10;objects per image and outperforms SOTA algorithms on several object&#10;localization and discovery benchmarks on PASCAL-VOC 07, 12 and COCO20k&#10;datasets. Additionally, we show that MOST can be used for self-supervised&#10;pre-training of object detectors, and yields consistent improvements on fully,&#10;semi-supervised object detection and unsupervised region proposal generation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;\underline{Unsupervised Object Localization and Discovery}:&#10;Object category discovery can be broadly segregated into image-based~\cite{HsuLK16,HsuLK18,HsuLSOK19,han19learning,han20automatically,singh-cvpr2019} and region-based methods~\cite{VoBCHLPP19,Vo20rOSD,Vo21LOD,LeeG11,5540237,hal-01110036,hal-01153017,LOST,wang2022tokencut,doersch2014context,Rambhatla2021ThePO}. Region-based methods start by generating object proposals and later group them into coherent semantic groups. Image-based approaches on the other hand, assume the localization task to be solved and focus solely on the semantic grouping. Our current method is closer to the former. Vo \etal,~\cite{VoBCHLPP19,Vo20rOSD,Vo21LOD} localize regions in images by constructing an inter-image similarity graph between regions and partitioning it using optimization or ranking. Due to the quadratic nature of the graph, these methods cannot scale to large datasets beyond tens of thousands of images. Our current work does not compute inter-image similarity between regions and scales linearly with number of images. Lee \etal,~\cite{LeeG11} propose object graphs that incorporates knowledge about a few known categories to facilitate the discovery of novel categories. MOST doesn't assume any prior knowledge and has the ability to discover objects from scratch. Lee \etal,~\cite{5540237} extend object graphs to a curriculum based discovery pipeline, that learns to discover easy objects first and progressively proceeds to discover the harder ones. Along similar lines, Rambhatla \etal,~\cite{Rambhatla2021ThePO} propose a large scale discovery pipeline, similar to ~\cite{LeeG11} that leverages prior knowledge about a few categories. Authors of~\cite{Rambhatla2021ThePO} use an RPN~\cite{NIPS2015_5638} trained on known categories as the localization method. In contrast to that, MOST localizes objects in images using features of a transformer~\cite{attisall} trained with DINO~\cite{caron2021emerging}.&#10;&#10;\underline{Object localization using self-supervised networks}:&#10;Recently, CNNs~\cite{He2016DeepRL} and Transformers~\cite{attisall} trained in a self-supervised fashion, have been shown to exhibit object localization/segmentation properties~\cite{caron2021emerging,Dwibedi_2021_ICCV}. This property is of particular interest as it has the potential to facilitate research on unsupervised localization, detection and segmentation. \cite{LOST} is a simple method, based on the observation that the key features of the last self attention layer of a transformer, trained using DINO, has certain affinity. They construct a map of inverse degree to extract bounding boxes on objects in an unsupervised fashion. This method is shown to outperform recent state-of-the-art methods by a significant margin. \cite{wang2022tokencut} propose an alternate method for localizing objects using self-supervised transformers, based on normalized cut~\cite{Shi1997NormalizedCA}. &#10;TokenCut~\cite{wang2022tokencut} construct an undirected graph based on token feature similarities and use normalized cut to cluster foreground and background patches. Spectral clustering is used to solve the graph-cut and they show that the eigen vector corresponding to the second smallest eigenvalue provides a good cutting solution. TokenCut outperforms LOST on unsupervised object discovery. In addition to discovery,~\cite{wang2022tokencut} also demonstrate impressive results on unsupervised saliency detection and weakly supervised object localization. Kyriazi \etal,~\cite{melas-kyriazi22deep} propose deep spectral methods, that performs normalized cut~\cite{Shi1997NormalizedCA} but using an affinity matrix computed using semantic and color features. Since this method is very similar to TokenCut and achieves lower performance, we only compare with the latter in this work.&#10;&#10;However, one limitation of LOST and TokenCut is that they can localize only one object per image. Our proposed method, MOST can automatically localize multiple objects per image and outperforms LOST and TokenCut on standard discovery benchmarks.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2210.07920" label="2210.07920">
-        <attvalues>
-          <attvalue for="0" value="MOVE: Unsupervised Movable Object Segmentation and Detection" />
-          <attvalue for="1" value="  We introduce MOVE, a novel method to segment objects without any form of&#10;supervision. MOVE exploits the fact that foreground objects can be shifted&#10;locally relative to their initial position and result in realistic&#10;(undistorted) new images. This property allows us to train a segmentation model&#10;on a dataset of images without annotation and to achieve state of the art&#10;(SotA) performance on several evaluation datasets for unsupervised salient&#10;object detection and segmentation. In unsupervised single object discovery,&#10;MOVE gives an average CorLoc improvement of 7.2% over the SotA, and in&#10;unsupervised class-agnostic object detection it gives a relative AP improvement&#10;of 53% on average. Our approach is built on top of self-supervised features&#10;(e.g. from DINO or MAE), an inpainting network (based on the Masked&#10;AutoEncoder) and adversarial training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.08127" label="2105.08127">
-        <attvalues>
-          <attvalue for="0" value="Finding an Unsupervised Image Segmenter in Each of Your Deep Generative&#10;  Models" />
-          <attvalue for="1" value="  Recent research has shown that numerous human-interpretable directions exist&#10;in the latent space of GANs. In this paper, we develop an automatic procedure&#10;for finding directions that lead to foreground-background image separation, and&#10;we use these directions to train an image segmentation model without human&#10;supervision. Our method is generator-agnostic, producing strong segmentation&#10;results with a wide range of different GAN architectures. Furthermore, by&#10;leveraging GANs pretrained on large datasets such as ImageNet, we are able to&#10;segment images from a range of domains without further training or finetuning.&#10;Evaluating our method on image segmentation benchmarks, we compare favorably to&#10;prior work while using neither human supervision nor access to the training&#10;data. Broadly, our results demonstrate that automatically extracting&#10;foreground-background structure from pretrained deep generative models can&#10;serve as a remarkably effective substitute for human supervision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.06397" label="1707.06397">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Object Discovery and Co-Localization by Deep Descriptor&#10;  Transforming" />
-          <attvalue for="1" value="  Reusable model design becomes desirable with the rapid expansion of computer&#10;vision and machine learning applications. In this paper, we focus on the&#10;reusability of pre-trained deep convolutional models. Specifically, different&#10;from treating pre-trained models as feature extractors, we reveal more&#10;treasures beneath convolutional layers, i.e., the convolutional activations&#10;could act as a detector for the common object in the image co-localization&#10;problem. We propose a simple yet effective method, termed Deep Descriptor&#10;Transforming (DDT), for evaluating the correlations of descriptors and then&#10;obtaining the category-consistent regions, which can accurately locate the&#10;common object in a set of unlabeled images, i.e., unsupervised object&#10;discovery. Empirical studies validate the effectiveness of the proposed DDT&#10;method. On benchmark image co-localization datasets, DDT consistently&#10;outperforms existing state-of-the-art methods by a large margin. Moreover, DDT&#10;also demonstrates good generalization ability for unseen categories and&#10;robustness for dealing with noisy data. Beyond those, DDT can be also employed&#10;for harvesting web images into valid external data sources for improving&#10;performance of both image recognition and object detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03148" label="1904.03148">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Image Matching and Object Discovery as Optimization" />
-          <attvalue for="1" value="  Learning with complete or partial supervision is powerful but relies on&#10;ever-growing human annotation efforts. As a way to mitigate this serious&#10;problem, as well as to serve specific applications, unsupervised learning has&#10;emerged as an important field of research. In computer vision, unsupervised&#10;learning comes in various guises. We focus here on the unsupervised discovery&#10;and matching of object categories among images in a collection, following the&#10;work of Cho et al. 2015. We show that the original approach can be reformulated&#10;and solved as a proper optimization problem. Experiments on several benchmarks&#10;establish the merit of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.02662" label="2007.02662">
-        <attvalues>
-          <attvalue for="0" value="Toward unsupervised, multi-object discovery in large-scale image&#10;  collections" />
-          <attvalue for="1" value="  This paper addresses the problem of discovering the objects present in a&#10;collection of images without any supervision. We build on the optimization&#10;approach of Vo et al. (CVPR'19) with several key novelties: (1) We propose a&#10;novel saliency-based region proposal algorithm that achieves significantly&#10;higher overlap with ground-truth objects than other competitive methods. This&#10;procedure leverages off-the-shelf CNN features trained on classification tasks&#10;without any bounding box information, but is otherwise unsupervised. (2) We&#10;exploit the inherent hierarchical structure of proposals as an effective&#10;regularizer for the approach to object discovery of Vo et al., boosting its&#10;performance to significantly improve over the state of the art on several&#10;standard benchmarks. (3) We adopt a two-stage strategy to select promising&#10;proposals using small random sets of images before using the whole image&#10;collection to discover the objects it depicts, allowing us to tackle, for the&#10;first time (to the best of our knowledge), the discovery of multiple objects in&#10;each one of the pictures making up datasets with up to 20,000 images, an over&#10;five-fold increase compared to existing methods, and a first step toward true&#10;large-scale unsupervised image interpretation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.06650" label="2106.06650">
-        <attvalues>
-          <attvalue for="0" value="Large-Scale Unsupervised Object Discovery" />
-          <attvalue for="1" value="  Existing approaches to unsupervised object discovery (UOD) do not scale up to&#10;large datasets without approximations that compromise their performance. We&#10;propose a novel formulation of UOD as a ranking problem, amenable to the&#10;arsenal of distributed methods available for eigenvalue problems and link&#10;analysis. Through the use of self-supervised features, we also demonstrate the&#10;first effective fully unsupervised pipeline for UOD. Extensive experiments on&#10;COCO and OpenImages show that, in the single-object discovery setting where a&#10;single prominent object is sought in each image, the proposed LOD (Large-scale&#10;Object Discovery) approach is on par with, or better than the state of the art&#10;for medium-scale datasets (up to 120K images), and over 37% better than the&#10;only other algorithms capable of scaling up to 1.7M images. In the multi-object&#10;discovery setting where multiple objects are sought in each image, the proposed&#10;LOD is over 14% better in average precision (AP) than all other methods for&#10;datasets ranging from 20K to 1.7M images. Using self-supervised features, we&#10;also show that the proposed method obtains state-of-the-art UOD performance on&#10;OpenImages. Our code is publicly available at https://github.com/huyvvo/LOD.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.08721" label="2002.08721">
-        <attvalues>
-          <attvalue for="0" value="A survey on Semi-, Self- and Unsupervised Learning for Image&#10;  Classification" />
-          <attvalue for="1" value="  While deep learning strategies achieve outstanding results in computer vision&#10;tasks, one issue remains: The current strategies rely heavily on a huge amount&#10;of labeled data. In many real-world problems, it is not feasible to create such&#10;an amount of labeled training data. Therefore, it is common to incorporate&#10;unlabeled data into the training process to reach equal results with fewer&#10;labels. Due to a lot of concurrent research, it is difficult to keep track of&#10;recent developments. In this survey, we provide an overview of often used ideas&#10;and methods in image classification with fewer labels. We compare 34 methods in&#10;detail based on their performance and their commonly used ideas rather than a&#10;fine-grained taxonomy. In our analysis, we identify three major trends that&#10;lead to future research opportunities. 1. State-of-the-art methods are&#10;scaleable to real-world applications in theory but issues like class imbalance,&#10;robustness, or fuzzy labels are not considered. 2. The degree of supervision&#10;which is needed to achieve comparable results to the usage of all labels is&#10;decreasing and therefore methods need to be extended to settings with a&#10;variable number of classes. 3. All methods share some common ideas but we&#10;identify clusters of methods that do not share many ideas. We show that&#10;combining ideas from different clusters can lead to better performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.07918" label="2104.07918">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Object Localization and Detection: A Survey" />
-          <attvalue for="1" value="  As an emerging and challenging problem in the computer vision community,&#10;weakly supervised object localization and detection plays an important role for&#10;developing new generation computer vision systems and has received significant&#10;attention in the past decade. As methods have been proposed, a comprehensive&#10;survey of these topics is of great importance. In this work, we review (1)&#10;classic models, (2) approaches with feature representations from off-the-shelf&#10;deep networks, (3) approaches solely based on deep learning, and (4) publicly&#10;available datasets and standard evaluation metrics that are widely used in this&#10;field. We also discuss the key challenges in this field, development history of&#10;this field, advantages/disadvantages of the methods in each category, the&#10;relationships between methods in different categories, applications of the&#10;weakly supervised object localization and detection methods, and potential&#10;future directions to further promote the development of this research field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.07499" label="2301.07499">
-        <attvalues>
-          <attvalue for="0" value="A Comprehensive Review of Modern Object Segmentation Approaches" />
-          <attvalue for="1" value="  Image segmentation is the task of associating pixels in an image with their&#10;respective object class labels. It has a wide range of applications in many&#10;industries including healthcare, transportation, robotics, fashion, home&#10;improvement, and tourism. Many deep learning-based approaches have been&#10;developed for image-level object recognition and pixel-level scene&#10;understanding-with the latter requiring a much denser annotation of scenes with&#10;a large set of objects. Extensions of image segmentation tasks include 3D and&#10;video segmentation, where units of voxels, point clouds, and video frames are&#10;classified into different objects. We use &quot;Object Segmentation&quot; to refer to the&#10;union of these segmentation tasks. In this monograph, we investigate both&#10;traditional and modern object segmentation approaches, comparing their&#10;strengths, weaknesses, and utilities. We examine in detail the wide range of&#10;deep learning-based segmentation techniques developed in recent years, provide&#10;a review of the widely used datasets and evaluation metrics, and discuss&#10;potential future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.04670" label="2306.04670">
-        <attvalues>
-          <attvalue for="0" value="Object Detection with Transformers: A Review" />
-          <attvalue for="1" value="  The astounding performance of transformers in natural language processing&#10;(NLP) has motivated researchers to explore their applications in computer&#10;vision tasks. DEtection TRansformer (DETR) introduces transformers to object&#10;detection tasks by reframing detection as a set prediction problem.&#10;Consequently, eliminating the need for proposal generation and post-processing&#10;steps. Initially, despite competitive performance, DETR suffered from slow&#10;training convergence and ineffective detection of smaller objects. However,&#10;numerous improvements are proposed to address these issues, leading to&#10;substantial improvements in DETR and enabling it to exhibit state-of-the-art&#10;performance. To our knowledge, this is the first paper to provide a&#10;comprehensive review of 21 recently proposed advancements in the original DETR&#10;model. We dive into both the foundational modules of DETR and its recent&#10;enhancements, such as modifications to the backbone structure, query design&#10;strategies, and refinements to attention mechanisms. Moreover, we conduct a&#10;comparative analysis across various detection transformers, evaluating their&#10;performance and network architectures. We hope that this study will ignite&#10;further interest among researchers in addressing the existing challenges and&#10;exploring the application of transformers in the object detection domain.&#10;Readers interested in the ongoing developments in detection transformers can&#10;refer to our website at:&#10;https://github.com/mindgarage-shan/trans_object_detection_survey&#10;" />
-          <attvalue for="2" value="&#10;Object detection is one of the fundamental tasks in computer vision that involves locating and classifying objects within an image \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68}. Over the years, convolutional neural networks (CNNs) have been the primary backbone for object detection models \cite{fasterrcnn3}. However, the recent success of transformers in natural language processing (NLP) has led researchers to explore their potential in computer vision as well \cite{ViT23}. The transformer architecture \cite{att75} has been shown to be effective in capturing long-range dependencies in sequential data \cite{att75}, making it an attractive candidate for object detection tasks.&#10;&#10;In 2020, Carion et al. proposed a novel object detection framework called DEtection TRansformer (DETR) \cite {detr34}, which replaces the traditional region proposal-based methods with a fully end-to-end trainable architecture that uses a transformer encoder-decoder network. The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}.&#10;&#10;Since the introduction of DETR, several modifications and improvements have been proposed to overcome its limitations, such as slow training convergence and performance drops for small objects. Figure~\ref{fig:papers} shows the literature overview on the Detection Transformer and its modifications to improve performance and training convergence. Deformable-DETR \cite{Deformable54} modifies the attention modules to process the image feature maps by considering the attention mechanism as the main reason for slow training convergence. UP-DETR \cite{updetr23} proposes a few modifications to Pre-train the DETR similar to the pretraining of transformers in natural language processing. Efficient-DETR \cite{efficientDE} based on original DETR and Deformable-DETR examines the randomly initialized object probabilities, including reference points and object queries, which is one of the reasons for multiple training iterations. SMCA-DETR \cite{smca23} introduces a Spatially-Modulated Co-attention module that replaces the existing co-attention mechanism in DETR to overcome the slow training convergence of DETR. TSP-DETR \cite{Reth78} deals with the cross-attention and the instability of bipartite matching to overcome the slow training convergence of DETR. Conditional-DETR \cite{CondDE} presents a conditional cross-attention mechanism to solve the training convergence issue of DETR. WB-DETR \cite{WBdetr4} considers CNN backbone for feature extraction as an extra component and presents a transformer encoder-decoder network without a backbone. PnP-DETR \cite{pnp6} proposes a PnP sampling module to reduce spatial redundancy and make the transformer network computationally more efficient. Dynamic-DETR \cite{DynamicDE} introduces dynamic attention in the encoder-decoder network to improve training convergence. YOLOS-DETR \cite{yolos6} presents the transferability and versatility of the Transformer from image recognition to detection in the sequence aspect using the least information about the spatial design of the input and improves performance. Anchor-DETR \cite{Anchor-detr} proposes object queries as anchor points that are extensively used in CNN-based object detectors. Sparse-DETR \cite{sparsedetr} reduces the computational cost by filtering encoder tokens with learnable cross-attention maps. D$^2$ETR \cite{decoderonly60} uses the fine-fused feature maps in the decoder from the backbone network with a novel cross-scale attention module. FP-DETR \cite{fpdetr} reformulates the pretraining and fine-tuning stages for detection transformers. CF-DETR \cite{cfdetr} refines the predicted locations by utilizing local information, as incorrect bounding box location reduces performance on small objects. DN-DETR \cite{dn42} uses noised object queries as additional decoder input to reduce the instability of the bipartite-matching mechanism in DETR, which causes the slow convergence problem. AdaMixer \cite{adamixer7} considers the encoder an extra network between the backbone and decoder that limits the performance and slower the training convergence because of its design complexity. It proposes a 3D sampling process and a few other modifications in the decoder. REGO-DETR \cite{rego2} proposes an RoI-based method for detection refinement to improve the attention mechanism in the detection transformer. DINO \cite{dino23} considers positive and negative noised object queries to make training convergence faster and to enhance the performance on small objects. &#10;&#10;Due to the rapid progress of transformer-based detection methods, keeping track of new advancements is becoming increasingly challenging. Thus, a review of ongoing progress is necessary and would be helpful for the researchers in the field. This paper provides a detailed overview of recent advancements in detection transformers. Table~\ref{tab:overview} shows the overview of Detection Transformer (DETR) modifications to improve performance and training convergence. &#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.12285" label="2406.12285">
-        <attvalues>
-          <attvalue for="0" value="DASSF: Dynamic-Attention Scale-Sequence Fusion for Aerial Object&#10;  Detection" />
-          <attvalue for="1" value="  The detection of small objects in aerial images is a fundamental task in the&#10;field of computer vision. Moving objects in aerial photography have problems&#10;such as different shapes and sizes, dense overlap, occlusion by the background,&#10;and object blur, however, the original YOLO algorithm has low overall detection&#10;accuracy due to its weak ability to perceive targets of different scales. In&#10;order to improve the detection accuracy of densely overlapping small targets&#10;and fuzzy targets, this paper proposes a dynamic-attention scale-sequence&#10;fusion algorithm (DASSF) for small target detection in aerial images. First, we&#10;propose a dynamic scale sequence feature fusion (DSSFF) module that improves&#10;the up-sampling mechanism and reduces computational load. Secondly, a x-small&#10;object detection head is specially added to enhance the detection capability of&#10;small targets. Finally, in order to improve the expressive ability of targets&#10;of different types and sizes, we use the dynamic head (DyHead). The model we&#10;proposed solves the problem of small target detection in aerial images and can&#10;be applied to multiple different versions of the YOLO algorithm, which is&#10;universal. Experimental results show that when the DASSF method is applied to&#10;YOLOv8, compared to YOLOv8n, on the VisDrone-2019 and DIOR datasets, the model&#10;shows an increase of 9.2% and 2.4% in the mean average precision (mAP),&#10;respectively, and outperforms the current mainstream methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object detection is now widely applied across various fields, such as smart transportation~\cite{jiao2023dsam}, medical diagnosis~\cite{ni2018multiple}, industrial manufacturing~\cite{zhao2018surface}, and re-identification~\cite{shen2023git}. &#10;With the ongoing advancements in UAV technology and the maturation of remote sensing technology, aerial image target detection has emerged as a significant focus due to its vast potential. However, compared to traditional target detection scenarios, aerial images present unique challenges, including a wide range of target scales, small object sizes, diverse angle variations, complex backgrounds, and susceptibility to solar radiation and atmospheric transparency. These factors significantly complicate the accurate detection and identification of small targets in aerial images. Therefore, further research is needed to develop innovative methods and technologies to enhance the accuracy and efficiency of small target detection in these contexts.&#10;&#10;According to the MS COCO dataset~\cite{lin2014microsoft} definition in the field of object detection, small objects are those smaller than 32 × 32 pixels. CFIL~\cite{weng2023cross} introduces a frequency-domain feature extraction module~\cite{weng2024enhancing} and frequency domain feature interaction to enhance salient features. MFC~\cite{qiao2022novel} proposes a frequency-domain filtering module to achieve dense target feature enhancement. LR-FPN~\cite{li2024lr} improves remote sensing target detection by enhancing low-level positional information and fine-grained context interaction, leading to superior performance in agricultural and urban planning applications. PBSL~\cite{shen2023pbsl} introduces a multimodal alignment approach to highlight relevant features and suppress irrelevant information. To address the issues of blurred boundaries and significant scale variations in high-resolution aerial images, Wang et al.~\cite{wang2023improved} made enhancements to the aerial image target detection task based on the YOLOX-X algorithm. By incorporating data augmentation, introducing a path aggregation network (PANet), and adding x-small detection heads, they significantly improved the overall detection capability of the network.&#10;&#10;While certain results have been achieved in aerial scene target detection tasks using these methods, there are still some problems, such as insufficient extraction of features of different scales by the neck network, information loss during the transmission process of small target features, and the use of too many cascade networks or attention mechanisms resulting in high computational overhead. In order to solve these problems, we present a YOLO model with dynamic-attention scale-sequence fusion (DASSF) to significantly improve detection accuracy. The main contributions of this paper can be summarized as follows:&#10;&#10;\begin{itemize}&#10; \item [{$\bullet$}]We design a dynamic scale sequence feature fusion (DSSFF) module to accurately and efficiently extract global high-level semantic information in images of different scales, which not only reduces the model complexity but also can accurately detect small objects through point sampling.&#10; \item[{$\bullet$}]By adding a x-small detection head with a resolution of 160 × 160, we can more&#10;exactly identify the categories of small targets and more precisely locate the coor-&#10;dinates of small targets during the detection stage, thereby improving the detection&#10;accuracy of models for small targets.&#10; \item [{$\bullet$}]We use a dynamic head (DyHead) that combines scale, spatial, and task attention to improve the ability to integrate full-scale object features during the detection phase. It improves the detection of occluded targets commonly found in drone images and the blurring of targets affected by weather in remote sensing images.&#10; \item [{$\bullet$}]The DASSF method we finally propose not only significantly enhances the detection accuracy of objects at different scales and categories in aerial images, but also can be flexibly applied to various YOLO series algorithms.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Image Processing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2107.05475" label="2107.05475">
-        <attvalues>
-          <attvalue for="0" value="GiT: Graph Interactive Transformer for Vehicle Re-identification" />
-          <attvalue for="1" value="  Transformers are more and more popular in computer vision, which treat an&#10;image as a sequence of patches and learn robust global features from the&#10;sequence. However, pure transformers are not entirely suitable for vehicle&#10;re-identification because vehicle re-identification requires both robust global&#10;features and discriminative local features. For that, a graph interactive&#10;transformer (GiT) is proposed in this paper. In the macro view, a list of GiT&#10;blocks are stacked to build a vehicle re-identification model, in where graphs&#10;are to extract discriminative local features within patches and transformers&#10;are to extract robust global features among patches. In the micro view, graphs&#10;and transformers are in an interactive status, bringing effective cooperation&#10;between local and global features. Specifically, one current graph is embedded&#10;after the former level's graph and transformer, while the current transform is&#10;embedded after the current graph and the former level's transformer. In&#10;addition to the interaction between graphs and transforms, the graph is a&#10;newly-designed local correction graph, which learns discriminative local&#10;features within a patch by exploring nodes' relationships. Extensive&#10;experiments on three large-scale vehicle re-identification datasets demonstrate&#10;that our GiT method is superior to state-of-the-art vehicle re-identification&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.01614" label="2404.01614">
-        <attvalues>
-          <attvalue for="0" value="LR-FPN: Enhancing Remote Sensing Object Detection with Location Refined&#10;  Feature Pyramid Network" />
-          <attvalue for="1" value="  Remote sensing target detection aims to identify and locate critical targets&#10;within remote sensing images, finding extensive applications in agriculture and&#10;urban planning. Feature pyramid networks (FPNs) are commonly used to extract&#10;multi-scale features. However, existing FPNs often overlook extracting&#10;low-level positional information and fine-grained context interaction. To&#10;address this, we propose a novel location refined feature pyramid network&#10;(LR-FPN) to enhance the extraction of shallow positional information and&#10;facilitate fine-grained context interaction. The LR-FPN consists of two primary&#10;modules: the shallow position information extraction module (SPIEM) and the&#10;contextual interaction module (CIM). Specifically, SPIEM first maximizes the&#10;retention of solid location information of the target by simultaneously&#10;extracting positional and saliency information from the low-level feature map.&#10;Subsequently, CIM injects this robust location information into different&#10;layers of the original FPN through spatial and channel interaction, explicitly&#10;enhancing the object area. Moreover, in spatial interaction, we introduce a&#10;simple local and non-local interaction strategy to learn and retain the&#10;saliency information of the object. Lastly, the LR-FPN can be readily&#10;integrated into common object detection frameworks to improve performance&#10;significantly. Extensive experiments on two large-scale remote sensing datasets&#10;(i.e., DOTAV1.0 and HRSC2016) demonstrate that the proposed LR-FPN is superior&#10;to state-of-the-art object detection approaches. Our code and models will be&#10;publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.12315" label="2111.12315">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Texture Recognition using PDV Hashing and Dictionary Learning on&#10;  Multi-scale Volume Local Binary Pattern" />
-          <attvalue for="1" value="  Spatial-temporal local binary pattern (STLBP) has been widely used in dynamic&#10;texture recognition. STLBP often encounters the high-dimension problem as its&#10;dimension increases exponentially, so that STLBP could only utilize a small&#10;neighborhood. To tackle this problem, we propose a method for dynamic texture&#10;recognition using PDV hashing and dictionary learning on multi-scale volume&#10;local binary pattern (PHD-MVLBP). Instead of forming very high-dimensional LBP&#10;histogram features, it first uses hash functions to map the pixel difference&#10;vectors (PDVs) to binary vectors, then forms a dictionary using the derived&#10;binary vector, and encodes them using the derived dictionary. In such a way,&#10;the PDVs are mapped to feature vectors of the size of dictionary, instead of&#10;LBP histograms of very high dimension. Such an encoding scheme could extract&#10;the discriminant information from videos in a much larger neighborhood&#10;effectively. The experimental results on two widely-used dynamic textures&#10;datasets, DynTex++ and UCLA, show the superiority performance of the proposed&#10;approach over the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Dynamic textures (DTs) refer to sequences of the image that consists of repeated patterns related to time and space. DT has been widely used in various applications such as video retrieval \cite{RelevanceFeedback}, fire detection \cite{5208233}, and micro-expression analysis \cite{7851001}. Compared to static textures, DT classification poses increased challenges because their appearance, organization and motion information are not static but time-varying \cite{8600380}. Consequently, a descriptive feature representation is a key to the success of dynamic texture recognition.&#10;&#10;Many methods have been developed for dynamic texture recognition, \eg, geometric properties computation \cite{QUAN201785}, local spatio-temporal filtering \cite{Ramirez2015}, dictionary learning~\cite{7780409}, deep convolutional neural network~\cite{QI20161230, shervin2017dynamic}, and various spatial-temporal local binary patterns~\cite{4160945, MBSIFTOP, LPQTOP,ren2013dynamic, ren2015chi, ren2015learning}. Among these, STLBP is most widely used in DT recognition. Zhao et al. developed Volume Local Binary Pattern (VLBP) \cite{4160945}, which combines the motion and appearance features together, instead of analyzing each frame individually. However, the feature dimension of VLBP increases exponentially with the number of neighbors, which prevents VLBP from utilizing the information in a large neighborhood. To reduce the feature dimension, LBP-TOP \cite{4160945} was developed to extract LBP features in three orthogonal planes. Inspired by LBP-TOP, various improved STLBPs have been developed, \eg, MBSIF-TOP~\cite{MBSIFTOP}, LPQ-TOP~\cite{LPQTOP}, and ASF-TOP~\cite{Hong2018}.&#10;&#10;LBP-TOP and its variants partially address the high-dimension problem of VLBP, but the resulting dimension may be still high, \eg, the dimension of LBP-TOP is $3\times2^{P^2-1}$ for a VLBP neighborhood of size $P\times P \times P$. In addition, as LBP features are extracted independently from the three orthogonal planes, the correlation information among the three planes is lost, which leads to possible performance degradation. &#10;&#10;To address these challenges, we propose a method using PDV hashing and dictionary learning on multi-scale VLBP (PHD-MVLBP). Firstly, the pixel difference vector (PDV) of neighbors w.r.t. the center pixel is mapped to a binary vector using hash functions. Both the hash functions and binary vectors are jointly optimized so that the resulting binary vectors are evenly distributed. Human knowledge on image/video such as the uniformity of LBP codes~\cite{ren2013noise} is embedded in the optimization function. Then, clustering is performed on the binary codes to construct a dictionary, and a histogram feature for each video clip is derived using the learned codebook. Multi-scale histogram features are extracted for neighborhoods of different sizes. In such a way, the learned feature representations aggregate the discriminant information using all neighbors in the LBP neighborhoods, at different scales, from all videos in the dataset.&#10;&#10;The proposed PHD-MVLBP is compared with the state-of-the-art methods on DynTex++ and UCLA datasets for dynamic texture recognition. It consistently outperforms all the compared methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Dynamic Texture Recognition, Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Feature Extraction Methods, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1511.00423" label="1511.00423">
-        <attvalues>
-          <attvalue for="0" value="Towards Reading Hidden Emotions: A comparative Study of Spontaneous&#10;  Micro-expression Spotting and Recognition Methods" />
-          <attvalue for="1" value="  Micro-expressions (MEs) are rapid, involuntary facial expressions which&#10;reveal emotions that people do not intend to show. Studying MEs is valuable as&#10;recognizing them has many important applications, particularly in forensic&#10;science and psychotherapy. However, analyzing spontaneous MEs is very&#10;challenging due to their short duration and low intensity. Automatic ME&#10;analysis includes two tasks: ME spotting and ME recognition. For ME spotting,&#10;previous studies have focused on posed rather than spontaneous videos. For ME&#10;recognition, the performance of previous studies is low. To address these&#10;challenges, we make the following contributions: (i)We propose the first method&#10;for spotting spontaneous MEs in long videos (by exploiting feature difference&#10;contrast). This method is training free and works on arbitrary unseen videos.&#10;(ii)We present an advanced ME recognition framework, which outperforms previous&#10;work by a large margin on two challenging spontaneous ME databases (SMIC and&#10;CASMEII). (iii)We propose the first automatic ME analysis system (MESR), which&#10;can spot and recognize MEs from spontaneous video data. Finally, we show our&#10;method outperforms humans in the ME recognition task by a large margin, and&#10;achieves comparable performance to humans at the very challenging task of&#10;spotting and then recognizing spontaneous MEs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1502.00303" label="1502.00303">
-        <attvalues>
-          <attvalue for="0" value="Dynamic texture and scene classification by transferring deep image&#10;  features" />
-          <attvalue for="1" value="  Dynamic texture and scene classification are two fundamental problems in&#10;understanding natural video content. Extracting robust and effective features&#10;is a crucial step towards solving these problems. However the existing&#10;approaches suffer from the sensitivity to either varying illumination, or&#10;viewpoint changing, or even camera motion, and/or the lack of spatial&#10;information. Inspired by the success of deep structures in image&#10;classification, we attempt to leverage a deep structure to extract feature for&#10;dynamic texture and scene classification. To tackle with the challenges in&#10;training a deep structure, we propose to transfer some prior knowledge from&#10;image domain to video domain. To be specific, we propose to apply a&#10;well-trained Convolutional Neural Network (ConvNet) as a mid-level feature&#10;extractor to extract features from each frame, and then form a representation&#10;of a video by concatenating the first and the second order statistics over the&#10;mid-level features. We term this two-level feature extraction scheme as a&#10;Transferred ConvNet Feature (TCoF). Moreover we explore two different&#10;implementations of the TCoF scheme, i.e., the \textit{spatial} TCoF and the&#10;\textit{temporal} TCoF, in which the mean-removed frames and the difference&#10;between two adjacent frames are used as the inputs of the ConvNet,&#10;respectively. We evaluate systematically the proposed spatial TCoF and the&#10;temporal TCoF schemes on three benchmark data sets, including DynTex, YUPENN,&#10;and Maryland, and demonstrate that the proposed approach yields superior&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.07235" label="2107.07235">
-        <attvalues>
-          <attvalue for="0" value="Deep Automatic Natural Image Matting" />
-          <attvalue for="1" value="  Automatic image matting (AIM) refers to estimating the soft foreground from&#10;an arbitrary natural image without any auxiliary input like trimap, which is&#10;useful for image editing. Prior methods try to learn semantic features to aid&#10;the matting process while being limited to images with salient opaque&#10;foregrounds such as humans and animals. In this paper, we investigate the&#10;difficulties when extending them to natural images with salient&#10;transparent/meticulous foregrounds or non-salient foregrounds. To address the&#10;problem, a novel end-to-end matting network is proposed, which can predict a&#10;generalized trimap for any image of the above types as a unified semantic&#10;representation. Simultaneously, the learned semantic features guide the matting&#10;network to focus on the transition areas via an attention mechanism. We also&#10;construct a test set AIM-500 that contains 500 diverse natural images covering&#10;all types along with manually labeled alpha mattes, making it feasible to&#10;benchmark the generalization ability of AIM models. Results of the experiments&#10;demonstrate that our network trained on available composite matting datasets&#10;outperforms existing methods both objectively and subjectively. The source code&#10;and dataset are available at https://github.com/JizhiziLi/AIM.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Natural image matting refers to estimating a soft foreground from a natural image, which is a fundamental process for many applications, e.g., film post-production and image editing~\cite{chen2013knn,zhang2020empowering}. Since image matting is a highly ill-posed problem, previous methods usually adopt auxiliary user input, e.g. trimap~\cite{sun2004poisson,cai2019disentangled}, scribble~\cite{levin2007closed}, or background image~\cite{backgroundmatting} as constraints. While traditional methods estimate the alpha value by sampling neighboring pixels~\cite{wang2007optimized} or defining affinity metrics for alpha propagation~\cite{levin2008spectral}, deep learning-based approaches solve it by learning discriminative representations from a large amount of labeled data and predicting alpha matte directly~\cite{lu2019indices,li2020natural}.&#10;&#10;However, extra manual effort is required while generating such auxiliary inputs, which makes these methods impractical in automatic industrial applications. To address the limitations, automatic image matting (AIM) has attracted increasing attention recently~\cite{zhang2019late}, which refers to automatically extracting the soft foreground from an arbitrary natural image. Prior AIM methods~\cite{Qiao_2020_CVPR,gfm} solve this problem by learning semantic features from the image to aid the matting process but is limited to images with salient opaque foregrounds, e.g., human~\cite{shen2016deep,chen2018semantic}, animal~\cite{gfm}. It is difficult to extend them to images with salient transparent/meticulous foregrounds or non-salient foregrounds due to their limited semantic representation ability. &#10;&#10;To address this issue, we dig into the problem of AIM by investigating the difficulties of extending it to all types of natural images. First, we propose to divide natural images into three types according to the characteristics of foreground alpha matte. As shown in Figure~\ref{fig:introduction}, the three types are: 1) SO (Salient Opaque): images that have salient foregrounds with opaque interiors, e.g. human, animal; 2) STM (Salient Transparent/Meticulous): images that have salient foregrounds with transparent or meticulous interiors, e.g. glass, plastic bag; and 3) NS (Non-Salient): images with non-salient foregrounds, e.g. smoke, grid, raindrop. Some examples are shown in Figure~\ref{fig:introduction}. Then, we systematically analyze the ability of baseline matting models for each type of image in terms of understanding global semantics and local matting details. We find that existing methods usually learn implicit semantic features or use an explicit semantic representation that is defined for a specific type of image, i.e., SO. Consequently, they are inefficient to handle different types of images with various characteristics in foreground alpha mattes, e.g., salient opaque/transparent and non-salient foregrounds.&#10;&#10;In this paper, we make the first attempt to address the problem by devising a novel automatic end-to-end matting network for all types of natural images. First, we define a simple but effective unified semantic representation for the above three types by generalizing the traditional trimap according to the characteristics of foreground alpha matte. Then, we build our model upon the recently proposed effective GFM model \cite{gfm} with customized designs. Specifically, 1) we use the generalized trimap as the semantic representation in the semantic decoder to adapt it for all types of images; 2) we exploit the effective SE attention \cite{hu2018squeeze} in the semantic decoder to learn better semantic features to handle different characteristics of foreground alpha mattes; 3) we improve the interaction between the semantic decoder and matting decoder by devising a spatial attention module, which guides the matting decoder to focus on the details in the transition areas. These customized designs prove to be effective for AIM rather than trivial tinkering, as shown in Figure~\ref{fig:introduction}.&#10;&#10;Besides, there is not a test bed for evaluating AIM models on different types of natural images. Previous methods use composition images by pasting different foregrounds on background images from COCO dataset~\cite{lin2014microsoft} for evaluation, which may introduce composition artifacts and are not representative for natural images as mentioned in~\cite{gfm}. Some recent works collect natural test images with manually labeled alpha mattes, however, they are limited to specific types of images, such as portrait images~\cite{MODNet} or animal images~\cite{gfm}, which are not suitable for comprehensive evaluation of matting models for AIM. To fill this gap, we establish a benchmark AIM-500 by collecting 500 diverse natural images covering all three types and many categories and manually label their alpha mattes.&#10;&#10;The main contributions of this paper are threefold: 1) we make the first attempt to investigate the difficulties of automatic natural image matting for all types of natural images; 2) we propose a new matting network with customized designs upon a reference architecture that are effective for AIM on different types of images; and 3) we establish the first natural images matting benchmark AIM-500 by collecting 500 natural images covering all three types and manually labeling their alpha mattes, which can serve as a test bed to facilitate future research on AIM.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Image Matting Techniques, Computer Vision Applications, Mathematics, Image Processing, Computer Science, Deep Learning Networks" />
-        </attvalues>
-      </node>
-      <node id="2011.08612" label="2011.08612">
-        <attvalues>
-          <attvalue for="0" value="Empowering Things with Intelligence: A Survey of the Progress,&#10;  Challenges, and Opportunities in Artificial Intelligence of Things" />
-          <attvalue for="1" value="  In the Internet of Things (IoT) era, billions of sensors and devices collect&#10;and process data from the environment, transmit them to cloud centers, and&#10;receive feedback via the internet for connectivity and perception. However,&#10;transmitting massive amounts of heterogeneous data, perceiving complex&#10;environments from these data, and then making smart decisions in a timely&#10;manner are difficult. Artificial intelligence (AI), especially deep learning,&#10;is now a proven success in various areas including computer vision, speech&#10;recognition, and natural language processing. AI introduced into the IoT&#10;heralds the era of artificial intelligence of things (AIoT). This paper&#10;presents a comprehensive survey on AIoT to show how AI can empower the IoT to&#10;make it faster, smarter, greener, and safer. Specifically, we briefly present&#10;the AIoT architecture in the context of cloud computing, fog computing, and&#10;edge computing. Then, we present progress in AI research for IoT from four&#10;perspectives: perceiving, learning, reasoning, and behaving. Next, we summarize&#10;some promising applications of AIoT that are likely to profoundly reshape our&#10;world. Finally, we highlight the challenges facing AIoT and some potential&#10;research opportunities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.04686" label="1909.04686">
-        <attvalues>
-          <attvalue for="0" value="Disentangled Image Matting" />
-          <attvalue for="1" value="  Most previous image matting methods require a roughly-specificed trimap as&#10;input, and estimate fractional alpha values for all pixels that are in the&#10;unknown region of the trimap. In this paper, we argue that directly estimating&#10;the alpha matte from a coarse trimap is a major limitation of previous methods,&#10;as this practice tries to address two difficult and inherently different&#10;problems at the same time: identifying true blending pixels inside the trimap&#10;region, and estimate accurate alpha values for them. We propose AdaMatting, a&#10;new end-to-end matting framework that disentangles this problem into two&#10;sub-tasks: trimap adaptation and alpha estimation. Trimap adaptation is a&#10;pixel-wise classification problem that infers the global structure of the input&#10;image by identifying definite foreground, background, and semi-transparent&#10;image regions. Alpha estimation is a regression problem that calculates the&#10;opacity value of each blended pixel. Our method separately handles these two&#10;sub-tasks within a single deep convolutional neural network (CNN). Extensive&#10;experiments show that AdaMatting has additional structure awareness and trimap&#10;fault-tolerance. Our method achieves the state-of-the-art performance on Adobe&#10;Composition-1k dataset both qualitatively and quantitatively. It is also the&#10;current best-performing method on the alphamatting.com online evaluation for&#10;all commonly-used metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Natural Image Matting&#10;Natural image matting is essentially the per-pixel opacity estimation of the foreground region. The typical input to natural matting algorithms is in the form of scribbles \cite{wang2005iterative} or trimaps \cite{chuang2001bayesian}, which help reduce the solution space of this ill-posed problem.&#10;&#10;Existing traditional methods can be categorized into color sampling based and alpha propagation based methods. Color sampling based methods \cite{chuang2001bayesian,gastal2010shared,he2011global,feng2016cluster} collect a set of known foreground and background samples to find candidate colors for a given pixel's foreground and background. According to the local smoothness assumption on the image statistics, these sampling colors are supposed to be ``close&quot; to the true foreground and background colors. Once the foreground color and background color are determined, we can calculate the corresponding alpha value base on Eq. \ref{eq:matting}. Following this assumption, various sampling-based methods are proposed, including Bayesian matting \cite{chuang2001bayesian}, shared sampling matting \cite{gastal2010shared}, global sampling matting \cite{he2011global}, and sparse coding matting \cite{feng2016cluster}.&#10;&#10;Compared to sampling-based methods, propagation-based approaches \cite{sun2004poisson,grady2005random,chen2013knn,aksoy2017designing,levin2008closed} avoid matte discontinuities which sampling-based approaches may suffer from. These methods utilize the affinities of neighboring pixels to propagate alpha values from the known regions into unknown ones. A popular approach among these is the closed-form matting \cite{levin2008closed}, which finds globally optimal alpha matte by solving a sparse linear system of equations. Other propagation-based approaches include Poisson matting \cite{sun2004poisson}, random walk matting \cite{grady2005random}, KNN matting \cite{chen2013knn} and information-flow matting \cite{aksoy2017designing}. &#10;&#10;Recently, deep learning has shown impressive performance on various computer vision tasks including image matting. Cho et al. \cite{cho2016natural} proposed an end-to-end architecture named DCNN that utilizes the results of closed-form matting \cite{levin2008closed} and KNN matting \cite{chen2013knn} to predict better alpha mattes. Shen et al. \cite{shen2016deep} proposed a fully automatic matting system for portrait photos based on end-to-end CNNs. Lutz et al. utilized \cite{lutz2018alphagan} the power of adversarial learning to extract alpha mattes which led to visually pleasing compositions. Wang et al. \cite{wang2018deep} showed a semantic-level pairwise similarity for propagation based matting can be learned via deep learning mechanism.&#10;&#10;Trimap Generation&#10;To the best of our knowledge, there are no existing work for trimap adaptation (i.e. generating the accurate optimal trimaps). The most related topic is automatic trimap generation. Automatic trimap generation has been an important part for traditional matting methods. Wang et al. \cite{wang2007automatic} used depth information acquired by a time-of-flight range scanner to obtain trimap. Some other algorithms \cite{cho2017automatic,hsieh2013automatic} rely on the binary segmentation to obtain the coarse trimaps. \cite{singh2013automatic} used the RGB image feature maps together with morphological dilation to automatically generate trimap, and refined trimap using region growing mechanism. \cite{al2015novel} first introduced the Gestalt laws to the matting problem, making more robust trimap generation possible. More recently, \cite{chen2018semantic, shen2016deep} utilized neural networks to generate trimaps, greatly improving the matting performances.&#10;&#10;Multi-task Learning&#10;Multi-task learning is a sub-field of machine learning, in which multiple learning tasks are solved within a single model simultaneously. Compared with training separate models for each task, multi-task learning improves learning efficiency and prediction accuracy for each task by utilizing their inter-relation. In computer vision, there are various exemplars of using multi-task learning, e.g. joint object detection and semantic segmentation \cite{hariharan2014simultaneous}, simultaneous depth estimation and scene parsing \cite{xu2018pad}, and universal network for handle low, middle and high-level vision tasks \cite{kokkinos2017ubernet}. Recently, Kendall et al. \cite{kendall2017multi} proposed a general way of combining multiple loss functions to simultaneously learn multiple objectives using homoscedastic task uncertainty. By dynamically adjusting the weights for each objective, their model could obtain superior performance compared to separately trained models.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1908.00672" label="1908.00672">
-        <attvalues>
-          <attvalue for="0" value="Indices Matter: Learning to Index for Deep Image Matting" />
-          <attvalue for="1" value="  We show that existing upsampling operators can be unified with the notion of&#10;the index function. This notion is inspired by an observation in the decoding&#10;process of deep image matting where indices-guided unpooling can recover&#10;boundary details much better than other upsampling operators such as bilinear&#10;interpolation. By looking at the indices as a function of the feature map, we&#10;introduce the concept of learning to index, and present a novel index-guided&#10;encoder-decoder framework where indices are self-learned adaptively from data&#10;and are used to guide the pooling and upsampling operators, without the need of&#10;supervision. At the core of this framework is a flexible network module, termed&#10;IndexNet, which dynamically predicts indices given an input. Due to its&#10;flexibility, IndexNet can be used as a plug-in applying to any off-the-shelf&#10;convolutional networks that have coupled downsampling and upsampling stages.&#10;  We demonstrate the effectiveness of IndexNet on the task of natural image&#10;matting where the quality of learned indices can be visually observed from&#10;predicted alpha mattes. Results on the Composition-1k matting dataset show that&#10;our model built on MobileNetv2 exhibits at least $16.1\%$ improvement over the&#10;seminal VGG-16 based deep matting baseline, with less training data and lower&#10;model capacity. Code and models has been made available at:&#10;https://tinyurl.com/IndexNetV1&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.04069" label="2001.04069">
-        <attvalues>
-          <attvalue for="0" value="Natural Image Matting via Guided Contextual Attention" />
-          <attvalue for="1" value="  Over the last few years, deep learning based approaches have achieved&#10;outstanding improvements in natural image matting. Many of these methods can&#10;generate visually plausible alpha estimations, but typically yield blurry&#10;structures or textures in the semitransparent area. This is due to the local&#10;ambiguity of transparent objects. One possible solution is to leverage the&#10;far-surrounding information to estimate the local opacity. Traditional&#10;affinity-based methods often suffer from the high computational complexity,&#10;which are not suitable for high resolution alpha estimation. Inspired by&#10;affinity-based method and the successes of contextual attention in inpainting,&#10;we develop a novel end-to-end approach for natural image matting with a guided&#10;contextual attention module, which is specifically designed for image matting.&#10;Guided contextual attention module directly propagates high-level opacity&#10;information globally based on the learned low-level affinity. The proposed&#10;method can mimic information flow of affinity-based methods and utilize rich&#10;features learned by deep neural networks simultaneously. Experiment results on&#10;Composition-1k testing set and alphamatting.com benchmark dataset demonstrate&#10;that our method outperforms state-of-the-art approaches in natural image&#10;matting. Code and models are available at&#10;https://github.com/Yaoyi-Li/GCA-Matting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.01507" label="1709.01507">
-        <attvalues>
-          <attvalue for="0" value="Squeeze-and-Excitation Networks" />
-          <attvalue for="1" value="  The central building block of convolutional neural networks (CNNs) is the&#10;convolution operator, which enables networks to construct informative features&#10;by fusing both spatial and channel-wise information within local receptive&#10;fields at each layer. A broad range of prior research has investigated the&#10;spatial component of this relationship, seeking to strengthen the&#10;representational power of a CNN by enhancing the quality of spatial encodings&#10;throughout its feature hierarchy. In this work, we focus instead on the channel&#10;relationship and propose a novel architectural unit, which we term the&#10;&quot;Squeeze-and-Excitation&quot; (SE) block, that adaptively recalibrates channel-wise&#10;feature responses by explicitly modelling interdependencies between channels.&#10;We show that these blocks can be stacked together to form SENet architectures&#10;that generalise extremely effectively across different datasets. We further&#10;demonstrate that SE blocks bring significant improvements in performance for&#10;existing state-of-the-art CNNs at slight additional computational cost.&#10;Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017&#10;classification submission which won first place and reduced the top-5 error to&#10;2.251%, surpassing the winning entry of 2016 by a relative improvement of ~25%.&#10;Models and code are available at https://github.com/hujie-frank/SENet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.06868" label="2102.06868">
-        <attvalues>
-          <attvalue for="0" value="Fast, Accurate Barcode Detection in Ultra High-Resolution Images" />
-          <attvalue for="1" value="  Object detection in Ultra High-Resolution (UHR) images has long been a&#10;challenging problem in computer vision due to the varying scales of the&#10;targeted objects. When it comes to barcode detection, resizing UHR input images&#10;to smaller sizes often leads to the loss of pertinent information, while&#10;processing them directly is highly inefficient and computationally expensive.&#10;In this paper, we propose using semantic segmentation to achieve a fast and&#10;accurate detection of barcodes of various scales in UHR images. Our pipeline&#10;involves a modified Region Proposal Network (RPN) on images of size greater&#10;than 10k$\times$10k and a newly proposed Y-Net segmentation network, followed&#10;by a post-processing workflow for fitting a bounding box around each segmented&#10;barcode mask. The end-to-end system has a latency of 16 milliseconds, which is&#10;$2.5\times$ faster than YOLOv4 and $5.9\times$ faster than Mask R-CNN. In terms&#10;of accuracy, our method outperforms YOLOv4 and Mask R-CNN by a $mAP$ of 5.5%&#10;and 47.1% respectively, on a synthetic dataset. We have made available the&#10;generated synthetic barcode dataset and its code at&#10;http://www.github.com/viplabB/SBD/.&#10;" />
-          <attvalue for="2" value="&#10;Barcodes are digital signs often made of adjacent and alternating black and white smaller rectangles that have become an intrinsic part of human society. In administration, for example, they are used to encode, save, and retrieve various users' information. At grocery stores, they are used to track sales and inventories. More interestingly in e-commerce, they are used to track and speed up processing time in warehouses and fulfillment centers.&#10;&#10;In classical signal processing, filters used for detection are image-specific since input images are not all necessarily acquired with the same illumination, brightness, angle, or camera. Consequently, adaptive image processing algorithms are required, which can impact detection accuracy \cite{Namane2017}. In addition, because classical signal processing methods often run on Central Processing Units, they tend to be much slower compared with deep learning implementations that are easily optimized on Graphics Processing Units (GPUs).&#10;&#10;Over the years, a number of methods have been proposed to detect barcodes using classical signal processing \cite{Namane2017,Hock2004,Katona2013,Soros2013,Creusot2016}, but nearly all of them take too long to process Ultra High-Resolution (UHR) images. More specifically, \cite{ Creusot2016} used parallel segment detectors which improved on their previous work \cite{Creusot20155} of finding imaginary perpendicular lines in Hough space with maximal stable extremal regions to detect barcodes. Katona et al. \cite{ Katona2013} used morphological manipulation for barcode detection, but this method did not generalize well as different barcode types have varying detection performances. Similarly, \cite{Gallo2011} proposed using x and y derivative differences, but varying input images yields different outputs, and using such operation on UHR images often become highly inefficient. &#10;&#10;With neural networks, though there has been much improvement in barcode detection tasks, few of them have addressed the fast and accurate detection problem in UHR images. Zamberletti et al. \cite{Zamberletti2010} paved the way for using neural networks to detect barcodes by investigating Hough spaces. This was followed by \cite{Hansen2017} which adapted the You Only Looked Once (YOLO) detector to find barcodes in Low Resolution (LR) images, but the YOLO algorithm is known to perform poorly with long shaped objects such as code 39 barcodes. Instance segmentation methods such as Mask R-CNN \cite{He2017} perform better on images of size $1024\times1024$ pixels but on smaller size images, the outputted Region of Interests (RoI) do not align well with long, 1D-barcode structures. This is because it typically predicts masks on $28\times28$ pixels irrespective of object size, and thereby generates &quot;wiggly&quot; artifacts on some barcode predictions, losing spatial resolution. In the same way, dedicated object detection pipelines, such as YOLOv4 \cite{Bochkovskiy2020}, though they perform well on lower Intersectionover Union (IoU) thresholds, suffer accuracy at higher IoU thresholds. Among those using segmentation on LR images as a means for detection, \cite{Zharkov2019} also tends to not perform well at higher IoU thresholds.&#10;&#10;In this paper, we propose a pipeline for detecting barcodes using deep neural networks, shown in {Fig. \ref{ynet-full-pipeline}}, which consists of two stages trained separately. When compared with classical signal processing methods, neural networks not only provide a faster inference time, but also yield higher accuracy because they learn meaningful filters for optimal feature extraction. As seen in {Fig. \ref{ynet-full-pipeline}}, in the first stage, we expand on the Region Proposal Network (RPN) introduced in Faster R-CNN \cite{Girshick2015} to extract high definition regions of potential locations where barcodes might be. This stage allows us to significantly reduce inference computation time that would have been required otherwise in the second stage. In the second stage, we introduce Y-Net, a semantic segmentation network that detects all instances of barcodes in a given outputted RoI image ($400\times400$). We then apply morphological operations on the predicted masks to separate and extract the corresponding bounding boxes as shown in {Fig. \ref{picture}}. &#10;&#10;One of the limitations of existing work on barcode detection is the insufficient number of training examples. ArTe-Lab 1D Medium Barcode Dataset \cite{Zamberletti2010} and the WWU Muenster Barcode Database \cite{Wachenfeld2010} are two examples of existing available datasets. They contain 365 and 595 images respectively, with ground truth masks at a resolution of $640\times480$. Most of the samples in the ArTe-Lab dataset have only one EAN13 barcode per sample image, and few of them in the Muenster database have more than one barcode instance on a given image. To address this dataset availability problem, we have released 100,000 UHR and 100,000 LR synthetic barcode datasets along with their corresponding bounding boxes ground truths, and their ground truth masks to facilitate further studies. The outline of this paper is as follows: in Section 2, we describe details of our approach; in Section 3, we summarize our experimental results and in Section 4, we conclude and expend on our future work. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Image Processing, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2004.10934" label="2004.10934">
-        <attvalues>
-          <attvalue for="0" value="YOLOv4: Optimal Speed and Accuracy of Object Detection" />
-          <attvalue for="1" value="  There are a huge number of features which are said to improve Convolutional&#10;Neural Network (CNN) accuracy. Practical testing of combinations of such&#10;features on large datasets, and theoretical justification of the result, is&#10;required. Some features operate on certain models exclusively and for certain&#10;problems exclusively, or only for small-scale datasets; while some features,&#10;such as batch-normalization and residual-connections, are applicable to the&#10;majority of models, tasks, and datasets. We assume that such universal features&#10;include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections&#10;(CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT)&#10;and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation,&#10;Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and&#10;combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50)&#10;for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source&#10;code is at https://github.com/AlexeyAB/darknet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.06281" label="1906.06281">
-        <attvalues>
-          <attvalue for="0" value="Universal Barcode Detector via Semantic Segmentation" />
-          <attvalue for="1" value="  Barcodes are used in many commercial applications, thus fast and robust&#10;reading is important. There are many different types of barcodes, some of them&#10;look similar while others are completely different. In this paper we introduce&#10;new fast and robust deep learning detector based on semantic segmentation&#10;approach. It is capable of detecting barcodes of any type simultaneously both&#10;in the document scans and in the wild by means of a single model. The detector&#10;achieves state-of-the-art results on the ArTe-Lab 1D Medium Barcode Dataset&#10;with detection rate 0.995. Moreover, developed detector can deal with more&#10;complicated object shapes like very long but narrow or very small barcodes. The&#10;proposed approach can also identify types of detected barcodes and performs at&#10;real-time speed on CPU environment being much faster than previous&#10;state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00877" label="2104.00877">
-        <attvalues>
-          <attvalue for="0" value="S2R-DepthNet: Learning a Generalizable Depth-specific Structural&#10;  Representation" />
-          <attvalue for="1" value="  Human can infer the 3D geometry of a scene from a sketch instead of a&#10;realistic image, which indicates that the spatial structure plays a fundamental&#10;role in understanding the depth of scenes. We are the first to explore the&#10;learning of a depth-specific structural representation, which captures the&#10;essential feature for depth estimation and ignores irrelevant style&#10;information. Our S2R-DepthNet (Synthetic to Real DepthNet) can be well&#10;generalized to unseen real-world data directly even though it is only trained&#10;on synthetic data. S2R-DepthNet consists of: a) a Structure Extraction (STE)&#10;module which extracts a domaininvariant structural representation from an image&#10;by disentangling the image into domain-invariant structure and domain-specific&#10;style components, b) a Depth-specific Attention (DSA) module, which learns&#10;task-specific knowledge to suppress depth-irrelevant structures for better&#10;depth estimation and generalization, and c) a depth prediction module (DP) to&#10;predict depth from the depth-specific representation. Without access of any&#10;real-world images, our method even outperforms the state-of-the-art&#10;unsupervised domain adaptation methods which use real-world images of the&#10;target domain for training. In addition, when using a small amount of labeled&#10;real-world data, we achieve the state-ofthe-art performance under the&#10;semi-supervised setting. The code and trained models are available at&#10;https://github.com/microsoft/S2R-DepthNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Monocular Depth Estimation.&#10;The monocular depth estimation task aims to estimate depth from a single image. Previous works explore the network structure~\cite{hu2019revisiting, laina2016deeper,fu2018deep,xu2018structured,Chen2019structure-aware,xia2020generating} or are jointly trained with other tasks, \textsl{e.g.}, normal~\cite{qi2018geonet,yin2019enforcing}, optical flow~\cite{yin2018geonet,chen2019self} and segmentation~\cite{guizilini2019semantically,wang2020sdc}. All these works are trained and tested on datasets of a specific domain without considering domain gaps. It is easy for the method to overfit the specific training dataset and lose the capability of generalization. However, obtaining various depth annotations is costly and time-consuming. Some self-supervised methods~\cite{monodepth2,zhou2019moving,watson2019self,yin2018geonet} take video sequences or stereo pairs as training data and use image warping and reconstruction loss to replace explicit depth supervision. However, dynamic objects and low-texture regions are still a challenge for these methods. Our work is the first one to explore another promising direction, which uses only synthetic data for training and tests directly on real data.&#10;Domain Adaptation and Generalization.&#10;The domain adaptation~\cite{ganin2015unsupervised,long2016unsupervised,saito2018maximum} and generalization~\cite{muandet2013domain, shankar2018generalizing} methods deal with the domain gap between the training and testing datasets. Domain adaptation methods transfer the model trained on the source domain by seeing data in the target domain while domain generalization only performs training on the source domain and tests on the unseen domain. Inspired by the domain adaptation works, researchers try to leverage synthetic data for training a depth estimation model and transfer it to real data. These works can be roughly divided into two categories: aligning the feature space of synthetic and real images~\cite{zheng2018t2net,Kundu2018adadepth,abarghouei18monocular} and translating synthetic images to realistic-looking ones~\cite{zheng2018t2net,abarghouei18monocular, zhao2019geometry}. Those methods all need access to real images of the target domain, which means it is still necessary to collect real images for training. Our method gets rid of this limitation since we do not use any real images for training and it holds the potential for wider applications.&#10;Image Translation and Style Transfer.&#10;The image translation task translates images from one domain to another~\cite{huang2018munit,gatys2016image,isola2017image}. &#10;The previous works~\cite{Kazemi2019WACV,huang2017adain, huang2018munit} decompose image representation into a domain-invariant structure component and a domain-specific style component. Specifically, Kazemi \etal\cite{Kazemi2019WACV} present the style and structure disentangled GAN that learns to disentangle style and structure representations for image generation. Huang \etal \cite{huang2018munit} disentangle image representation into structure and style codes, and recombine its structure code with a random style code sampled from the style space of the target domain to translate an image to another domain. Targeting for translating images from one style to another, those works extract a structural representation which is disentangled from domain styles. Our goal is to seek for the essential generalizable structural representation for the depth estimation task by getting rid of the influence of irrelevant factors. &#10;&#10;" />
-          <attvalue for="4" value="Structural Representation, Domain Adaptation, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Depth Estimation, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2104.05044" label="2104.05044">
-        <attvalues>
-          <attvalue for="0" value="USACv20: robust essential, fundamental and homography matrix estimation" />
-          <attvalue for="1" value="  We review the most recent RANSAC-like hypothesize-and-verify robust&#10;estimators. The best performing ones are combined to create a state-of-the-art&#10;version of the Universal Sample Consensus (USAC) algorithm. A recent objective&#10;is to implement a modular and optimized framework, making future RANSAC modules&#10;easy to be included. The proposed method, USACv20, is tested on eight publicly&#10;available real-world datasets, estimating homographies, fundamental and&#10;essential matrices. On average, USACv20 leads to the most geometrically&#10;accurate models and it is the fastest in comparison to the state-of-the-art&#10;robust estimators. All reported properties improved performance of original&#10;USAC algorithm significantly. The pipeline will be made available after&#10;publication.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The RANdom SAmple Consensus (RANSAC) algorithm~\cite{fischler1981random} has been one of the most widely used robust estimators in computer vision. &#10;RANSAC and many of its variants have been successfully applied to a wide range of vision tasks, for instance, short baseline stereo~\cite{torr1993outlier,torr1998robust}, motion segmentation~\cite{torr1993outlier}, detection of geometric primitives~\cite{sminchisescu2005incremental}, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, in structure-from-motion~\cite{agarwal2010bundle,wilson2014robust, schonberger2016structure} (SfM) or simultaneous localization and mapping~\cite{engel2014lsd,mur2015orb} (SLAM) pipelines, image mosaicing~\cite{ghosh2016survey}, and to perform~\cite{zuliani2005multiransac} or initialize multi-model fitting~\cite{isack2012energy,pham2014interacting}.&#10;&#10;In this paper, we review some of the most recent RANSAC modifications, combine them together and propose a state-of-the-art variant of the Universal Sample Consensus~\cite{raguram2013usac} (USAC) algorithm. &#10;Also, an important objective is to make the implemented modular and optimized C++ framework publicly available, therefore, making future RANSAC modules easy to be combined with the proposed USACv20.&#10;&#10;In short, the RANSAC approach repeatedly creates minimal sets of randomly selected points and fits a model to them, e.g., a circle to three 2D points or a homography to four 2D point correspondences.&#10;Next, the quality of the estimated model is measured, for example, by the cardinality of its support, i.e., the number of data points closer than a manually set inlier-outlier threshold. &#10;Finally, the model with the highest score, polished, e.g., by least squares fitting of all inliers, is returned.&#10;&#10;Scoring function. Many modifications have been proposed since the publication of RANSAC, improving the components of the algorithm.&#10;For instance, in MAPSAC~\cite{torr2002bayesian}, the robust estimation is formulated as a process that estimates both the parameters of the data distribution and the quality of the model in terms of maximum a posteriori. &#10;MLESAC~\cite{torr2000mlesac} estimates the model quality by a maximum likelihood process with all its beneficial properties, albeit under certain assumptions about data distributions. &#10;In practice, MLESAC results are often superior to the inlier counting of plain RANSAC, and are less sensitive to the inlier-outlier threshold defined manually.&#10;&#10;Local Optimization. Observing that RANSAC requires in practice more samples than theory predicts, Chum et al.~\cite{chum2003locally,lebeda2012fixing} identified a problem that not all all-inlier samples are ``good'', i.e., lead to a model accurate enough to distinguish all inliers, e.g., due to poor conditioning of the selected random all-inlier sample. They addressed the problem by introducing the locally optimized RANSAC that augments the original approach with a local optimization step applied to the so-far-the-best model. &#10;This approach had been further improved in Graph-Cut RANSAC~\cite{barath2018graph} considering the fact that real-world data often form spatially coherent structures.&#10;Graph-Cut RANSAC exploits the proximity of the points in the local optimization step, leading to results superior to LO-RANSAC in terms of geometric accuracy.&#10;&#10;Sampling Strategies.&#10;Samplers NAPSAC~\cite{nasuto2002napsac} and PROSAC~\cite{chum2005matching} modify the RANSAC sampling strategy to increase the probability of selecting an all-inlier sample early. &#10;PROSAC exploits an a priori predicted inlier probability rank of the points and starts the sampling with the most promising ones.&#10;PROSAC and other RANSAC-like samplers treat models without considering&#10;that inlier points often are in the proximity of each other. &#10;This approach is effective when finding a global model with inliers sparsely distributed in the scene, for instance, the rigid motion induced by changing the viewpoint in two-view matching.&#10;However, as it is often the case in real-world data, if the model is localized with inlier points close to each other, robust estimation can be significantly sped up by exploiting this in the sampling.&#10;NAPSAC assumes that inliers are spatially coherent.&#10;It draws samples from a hyper-sphere centered at the first, randomly selected, point. &#10;If this point is an inlier, the rest of the points sampled in its proximity are more likely to be inliers than the points outside the ball.&#10;Progressive NAPSAC~\cite{barath2019progressive} was proposed to combine NAPSAC-like localized sampling with PROSAC by drawing minimal samples from gradually growing neighborhoods. &#10;&#10;Optimizing Model Verification.&#10;One of the most successful improvement for speeding up the verification is the optimal randomized model verification strategy~\cite{matas2005randomized,chum2008optimal} (WaldSAC) based on Wald’s theory of sequential decision making.&#10;When the level of outlier contamination is known a priori, the WaldSAC strategy is provably optimal. In practice, however, inlier ratios have to be estimated during the evaluation process and WaldSAC adjusted to the current so-far-the-best model.&#10;The performance of the SPRT test is not significantly affected by the imperfect estimation of these parameters. &#10;&#10;Termination criterion.&#10;There were a number of different termination criteria proposed for RANSAC-like hypothesize-and-verify methods. &#10;The original criterion is based on the assumption that the inliers are noise-free. The number of iterations required is calculated from the inlier ratio and the number of points needed for the model estimation. &#10;This criterion was then relaxed by Progressive NAPSAC~\cite{barath2019progressive} by terminating if the probability of finding a model which has significantly more inliers than the previous best falls below a threshold. &#10;In \cite{chum2005matching}, another criterion was proposed. &#10;The PROSAC algorithm terminates if the number of inliers satisfies the following conditions:&#10;(i) non-randomness -- the probability that $i^*$ out of $n$ data points are by chance inliers to an arbitrary incorrect model is smaller than a threshold; (ii) maximality -- the probability that a solution with more than $i^*$ inliers exists and was not found after $k$ samples is smaller than $\mu_0$.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Geometric Model Estimation, Computer Science, Robust Estimation Methods, Computer Vision Algorithms" />
-        </attvalues>
-      </node>
-      <node id="1503.02619" label="1503.02619">
-        <attvalues>
-          <attvalue for="0" value="MODS: Fast and Robust Method for Two-View Matching" />
-          <attvalue for="1" value="  A novel algorithm for wide-baseline matching called MODS - Matching On Demand&#10;with view Synthesis - is presented. The MODS algorithm is experimentally shown&#10;to solve a broader range of wide-baseline problems than the state of the art&#10;while being nearly as fast as standard matchers on simple problems. The&#10;apparent robustness vs. speed trade-off is finessed by the use of progressively&#10;more time-consuming feature detectors and by on-demand generation of&#10;synthesized images that is performed until a reliable estimate of geometry is&#10;obtained.&#10;  We introduce an improved method for tentative correspondence selection,&#10;applicable both with and without view synthesis. A modification of the standard&#10;first to second nearest distance rule increases the number of correct matches&#10;by 5-20% at no additional computational cost.&#10;  Performance of the MODS algorithm is evaluated on several standard publicly&#10;available datasets, and on a new set of geometrically challenging wide baseline&#10;problems that is made public together with the ground truth. Experiments show&#10;that the MODS outperforms the state-of-the-art in robustness and speed.&#10;Moreover, MODS performs well on other classes of difficult two-view problems&#10;like matching of images from different modalities, with wide temporal baseline&#10;or with significant lighting changes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.02295" label="1906.02295">
-        <attvalues>
-          <attvalue for="0" value="Progressive NAPSAC: sampling from gradually growing neighborhoods" />
-          <attvalue for="1" value="  We propose Progressive NAPSAC, P-NAPSAC in short, which merges the advantages&#10;of local and global sampling by drawing samples from gradually growing&#10;neighborhoods. Exploiting the fact that nearby points are more likely to&#10;originate from the same geometric model, P-NAPSAC finds local structures&#10;earlier than global samplers. We show that the progressive spatial sampling in&#10;P-NAPSAC can be integrated with PROSAC sampling, which is applied to the first,&#10;location-defining, point. P-NAPSAC is embedded in USAC, a state-of-the-art&#10;robust estimation pipeline, which we further improve by implementing its local&#10;optimization as in Graph-Cut RANSAC. We call the resulting estimator USAC*. The&#10;method is tested on homography and fundamental matrix fitting on a total of&#10;10,691 models from seven publicly available datasets. USAC* with P-NAPSAC&#10;outperforms reference methods in terms of speed on all problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.04988" label="2002.04988">
-        <attvalues>
-          <attvalue for="0" value="Saliency Driven Perceptual Image Compression" />
-          <attvalue for="1" value="  This paper proposes a new end-to-end trainable model for lossy image&#10;compression, which includes several novel components. The method incorporates&#10;1) an adequate perceptual similarity metric; 2) saliency in the images; 3) a&#10;hierarchical auto-regressive model. This paper demonstrates that the popularly&#10;used evaluations metrics such as MS-SSIM and PSNR are inadequate for judging&#10;the performance of image compression techniques as they do not align with the&#10;human perception of similarity. Alternatively, a new metric is proposed, which&#10;is learned on perceptual similarity data specific to image compression. The&#10;proposed compression model incorporates the salient regions and optimizes on&#10;the proposed perceptual similarity metric. The model not only generates images&#10;which are visually better but also gives superior performance for subsequent&#10;computer vision tasks such as object detection and segmentation when compared&#10;to existing engineered or learned compression techniques.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In the last two decades, the number of images captured and transmitted via the internet has grown exponentially \cite{forbes}. This surge has increased both data storage and transmission requirements. Compression of images can be lossless \cite{boutell1997png,skodras2001jpeg}, that is, the original image can be perfectly reconstructed. A better compression rate can be obtained by using lossy methods such as JPEG \cite{wallace1992jpeg}, JPEG-2000 \cite{skodras2001jpeg} and BPG \cite{bpg}. The objective of these lossy methods is to obtain higher compression by removing the information which is least noticeable to humans. Note that these traditional codecs are hand-crafted and are not learned from the data.&#10;&#10;More recent methods focus on learning to compress images. While learned image compression from data using neural networks is not new \cite{munro1989image,jiang1999image,luttrell1988image}, there has recently been a resurgence of deep learning-based techniques for solving this problem \cite{balle2016end,balle2018variational,mentzer2018conditional,rippel2017real,lee2018context,patel2019deep,theis2017lossy}. These models are trained by jointly minimizing rate (storage or transmission requirement) and distortion (reconstruction quality), leading to a Rate-Distortion trade-off \cite{shannon1948mathematical}.&#10;&#10;The contributions of this paper are three-fold: &#10;\begin{enumerate}&#10; \item A novel hierarchical auto-regressive architecture for image compression is proposed, which is based on an encoder-decoder setup.&#10; \item A learned perceptual similarity metric is proposed. The metric is realized by a fully convolutional network (FCN) which is trained on compression specific artefacts. Note that the metric is differentiable and it is used as a for training the image compression model.&#10; \item The proposed model accounts for salient regions. This is achieved in two ways: (a) more bits are allocated to the salient regions and (b) higher weight is given to their reconstruction.&#10; \end{enumerate}&#10;&#10;The motivations and background for these contributions are subsequently discussed, along with the related work.&#10;&#10;Taxonomy of image compression models. Deep learning models for image compression can be broadly categorized into three generative models: a) Variational Auto-Encoders \cite{kingma2013auto,balle2018variational,balle2016end,lee2018context}; b) Generative Adversarial Networks \cite{rippel2017real,agustsson2018generative} (GAN) \cite{goodfellow2014generative}; c) Auto-Regressive (AR) \cite{van2016conditional,mentzer2018conditional}. Variational auto-encoders (VAEs) and auto-regressive models operate by estimating the probability density explicitly. On the other hand, GANs have an implicit measure of the density \cite{goodfellow2014generative}. GANs are useful in very low bit-rate settings as they can learn to synthesize the images \cite{agustsson2018generative}. However, their superiority over AR and VAE is unclear for higher bit-rates. A key difference between VAE and AR is that the former approximates the density, whereas auto-regressive models such as Pixel-CNNs, Pixel-RNNs \cite{van2016conditional} have an explicit and tractable measure of density either in the pixel space or in the learned quantized space.&#10;&#10;Proposed hierarchical auto-regressive model. A hierarchical auto-regressive model with two-stages is designed, which is realized by two 3D Pixel-CNN \cite{van2016conditional}. The Pixel-CNNs operate on the learned quantized representations. During training, the 3D Pixel-CNNs give an explicit measure of the entropy of the quantized representations, which via information theory directly relates to the bits required to store them after arithmetic coding \cite{marpe2003context}. Minimizing the estimated entropy leads to compression, while the reconstructed image should be as close to the original as possible. Note that auto-regressive models have been used before for compression \cite{mentzer2018conditional}, but, this is the first hierarchical auto-regressive model designed for the task.&#10;&#10;Existing evaluation metrics. Lossy image compression models are compared by plotting the rate-distortion curve, where the rate is in bits-per-pixel vs the value of the distortion function. Common choices of the distortion function for compression models are MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context}. Both of these metrics do not align well with human perception of similarity \cite{patel2019human}. As deep learned models directly optimize on these evaluation metrics, it is natural for them to have high MS-SSIM or PSNR scores when compared to the engineered methods such as JPEG-2000 \cite{skodras2001jpeg}, BPG \cite{bpg}. However, the resulting images often look worse to a human, i.e., the distortion in the images is higher although the metrics report otherwise. Figure \ref{fig:kodak_samples} shows four different techniques ranked in descending order of MS-SSIM values. The $2$nd and $3$rd images have many more artefacts than the last two images which imply that MS-SSIM is arguably not a good evaluation measure for compression. In Figure \ref{fig:kodak_samples} notice that the text is not as clear in the first two approaches.&#10;&#10;The limitations of MS-SSIM and PSNR have been investigated in the past \cite{zhang18}. The super-resolution methods have started using a more sophisticated learned perceptual similarity metric \cite{blau2018perception,blau20182018} for evaluation, exactly for the mentioned reasons. Despite of the short comings, recent image compression literature continues to evaluate models using MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context,patel2019deep,rippel2017real,theis2015generative,theis2017lossy}. Further, it has been observed that techniques trained on PSNR perform well when evaluated on PSNR, but poorly when evaluated using MS-SSIM (and vice-versa). This makes developing practical compression systems difficult. &#10;&#10;Proposed evaluation metric. This paper proposes a learned perceptual similarity metric for evaluating and training image compression models. This paper also studies the compression techniques using human evaluations. The study shows that human evaluation results correlate well with the proposed evaluation metric. Furthermore, the ranking of different compression methods, as judged by humans, correlates with the performance of off-the-shelf object detection and segmentation methods, which are trained on uncompressed images.&#10;&#10;The analysis in this paper starts by evaluating the model proposed by Zhang ~\etal \cite{zhang18}. The model is trained on images with several different artefacts but the only compression artefacts are from JPEG. As Patel ~\etal \cite{patel2019deep} found, using the model \cite{zhang18} has limitations for compression. Therefore, we create a compression specific perceptual similarity dataset. The data include images generated from popular engineered \cite{skodras2001jpeg,wallace1992jpeg,bpg} and learned compression methods \cite{mentzer2018conditional,balle2016end,balle2018variational,patel2019deep}. The data consists of $6$ two alternative forced choices (2AFC) per comparison (see Section \ref{sec:percp_sim_metrices} for details). &#10;&#10;Saliency matters for image compression. While JPEG \cite{wallace1992jpeg} divides an image into uniform $8 \times 8$ blocks, BPG uses a hand-crafted metric to determine homogeneity and divides the more homogeneous regions into larger $64 \times 64$ blocks. In BPG, fewer bits are allocated to homogeneous regions and more bits are allocated to non-homogeneous regions. BPG builds on the hypothesis that humans are more prone to notice artefacts in complex regions of the image. Following this hypothesis of BPG, the proposed method makes use of object saliency in two ways: 1) {rate optimization}: more bits are allocated to the salient regions; 2) {distortion optimization}: artefacts in salient regions are more heavily penalized. To the best of our knowledge, we are the first to incorporate saliency in learned compression.&#10;&#10;The rest of the paper is structured as follows. In Section \ref{sec:percp_sim_metrices}, the compression specific perceptual similarity dataset is presented and various metrics are compared against the human judgements. Section \ref{sec:method} describes the proposed approach which is evaluated in Section \ref{sec:results}. Section \ref{sec:conclusions} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Compression, Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Cognitive Science, Computer Science, Perceptual Similarity Metrics, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1611.01704" label="1611.01704">
-        <attvalues>
-          <attvalue for="0" value="End-to-end Optimized Image Compression" />
-          <attvalue for="1" value="  We describe an image compression method, consisting of a nonlinear analysis&#10;transformation, a uniform quantizer, and a nonlinear synthesis transformation.&#10;The transforms are constructed in three successive stages of convolutional&#10;linear filters and nonlinear activation functions. Unlike most convolutional&#10;neural networks, the joint nonlinearity is chosen to implement a form of local&#10;gain control, inspired by those used to model biological neurons. Using a&#10;variant of stochastic gradient descent, we jointly optimize the entire model&#10;for rate-distortion performance over a database of training images, introducing&#10;a continuous proxy for the discontinuous loss function arising from the&#10;quantizer. Under certain conditions, the relaxed loss function may be&#10;interpreted as the log likelihood of a generative model, as implemented by a&#10;variational autoencoder. Unlike these models, however, the compression model&#10;must operate at any given point along the rate-distortion curve, as specified&#10;by a trade-off parameter. Across an independent set of test images, we find&#10;that the optimized method generally exhibits better rate-distortion performance&#10;than the standard JPEG and JPEG 2000 compression methods. More importantly, we&#10;observe a dramatic improvement in visual quality for all images at all bit&#10;rates, which is supported by objective quality estimates using MS-SSIM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.01436" label="1802.01436">
-        <attvalues>
-          <attvalue for="0" value="Variational image compression with a scale hyperprior" />
-          <attvalue for="1" value="  We describe an end-to-end trainable model for image compression based on&#10;variational autoencoders. The model incorporates a hyperprior to effectively&#10;capture spatial dependencies in the latent representation. This hyperprior&#10;relates to side information, a concept universal to virtually all modern image&#10;codecs, but largely unexplored in image compression using artificial neural&#10;networks (ANNs). Unlike existing autoencoder compression methods, our model&#10;trains a complex prior jointly with the underlying autoencoder. We demonstrate&#10;that this model leads to state-of-the-art image compression when measuring&#10;visual quality using the popular MS-SSIM index, and yields rate-distortion&#10;performance surpassing published ANN-based methods when evaluated using a more&#10;traditional metric based on squared error (PSNR). Furthermore, we provide a&#10;qualitative comparison of models trained for different distortion metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.04260" label="1801.04260">
-        <attvalues>
-          <attvalue for="0" value="Conditional Probability Models for Deep Image Compression" />
-          <attvalue for="1" value="  Deep Neural Networks trained as image auto-encoders have recently emerged as&#10;a promising direction for advancing the state-of-the-art in image compression.&#10;The key challenge in learning such networks is twofold: To deal with&#10;quantization, and to control the trade-off between reconstruction error&#10;(distortion) and entropy (rate) of the latent image representation. In this&#10;paper, we focus on the latter challenge and propose a new technique to navigate&#10;the rate-distortion trade-off for an image compression auto-encoder. The main&#10;idea is to directly model the entropy of the latent representation by using a&#10;context model: A 3D-CNN which learns a conditional probability model of the&#10;latent distribution of the auto-encoder. During training, the auto-encoder&#10;makes use of the context model to estimate the entropy of its representation,&#10;and the context model is concurrently updated to learn the dependencies between&#10;the symbols in the latent representation. Our experiments show that this&#10;approach, when measured in MS-SSIM, yields a state-of-the-art image compression&#10;system based on a simple convolutional auto-encoder.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.05823" label="1705.05823">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Adaptive Image Compression" />
-          <attvalue for="1" value="  We present a machine learning-based approach to lossy image compression which&#10;outperforms all existing codecs, while running in real-time.&#10;  Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG&#10;2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of&#10;generic images across all quality levels. At the same time, our codec is&#10;designed to be lightweight and deployable: for example, it can encode or decode&#10;the Kodak dataset in around 10ms per image on GPU.&#10;  Our architecture is an autoencoder featuring pyramidal analysis, an adaptive&#10;coding module, and regularization of the expected codelength. We also&#10;supplement our approach with adversarial training specialized towards use in a&#10;compression setting: this enables us to produce visually pleasing&#10;reconstructions for very low bitrates.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.10452" label="1809.10452">
-        <attvalues>
-          <attvalue for="0" value="Context-adaptive Entropy Model for End-to-end Optimized Image&#10;  Compression" />
-          <attvalue for="1" value="  We propose a context-adaptive entropy model for use in end-to-end optimized&#10;image compression. Our model exploits two types of contexts, bit-consuming&#10;contexts and bit-free contexts, distinguished based upon whether additional bit&#10;allocation is required. Based on these contexts, we allow the model to more&#10;accurately estimate the distribution of each latent representation with a more&#10;generalized form of the approximation models, which accordingly leads to an&#10;enhanced compression performance. Based on the experimental results, the&#10;proposed method outperforms the traditional image codecs, such as BPG and&#10;JPEG2000, as well as other previous artificial-neural-network (ANN) based&#10;approaches, in terms of the peak signal-to-noise ratio (PSNR) and multi-scale&#10;structural similarity (MS-SSIM) index.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.08310" label="1907.08310">
-        <attvalues>
-          <attvalue for="0" value="Deep Perceptual Compression" />
-          <attvalue for="1" value="  Several deep learned lossy compression techniques have been proposed in the&#10;recent literature. Most of these are optimized by using either MS-SSIM&#10;(multi-scale structural similarity) or MSE (mean squared error) as a loss&#10;function. Unfortunately, neither of these correlate well with human perception&#10;and this is clearly visible from the resulting compressed images. In several&#10;cases, the MS-SSIM for deep learned techniques is higher than say a&#10;conventional, non-deep learned codec such as JPEG-2000 or BPG. However, the&#10;images produced by these deep learned techniques are in many cases clearly&#10;worse to human eyes than those produced by JPEG-2000 or BPG.&#10;  We propose the use of an alternative, deep perceptual metric, which has been&#10;shown to align better with human perceptual similarity. We then propose Deep&#10;Perceptual Compression (DPC) which makes use of an encoder-decoder based image&#10;compression model to jointly optimize on the deep perceptual metric and&#10;MS-SSIM. Via extensive human evaluations, we show that the proposed method&#10;generates visually better results than previous learning based compression&#10;methods and JPEG-2000, and is comparable to BPG. Furthermore, we demonstrate&#10;that for tasks like object-detection, images compressed with DPC give better&#10;accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.00395" label="1703.00395">
-        <attvalues>
-          <attvalue for="0" value="Lossy Image Compression with Compressive Autoencoders" />
-          <attvalue for="1" value="  We propose a new approach to the problem of optimizing autoencoders for lossy&#10;image compression. New media formats, changing hardware technology, as well as&#10;diverse requirements and content types create a need for compression algorithms&#10;which are more flexible than existing codecs. Autoencoders have the potential&#10;to address this need, but are difficult to optimize directly due to the&#10;inherent non-differentiabilty of the compression loss. We here show that&#10;minimal changes to the loss are sufficient to train deep autoencoders&#10;competitive with JPEG 2000 and outperforming recently proposed approaches based&#10;on RNNs. Our network is furthermore computationally efficient thanks to a&#10;sub-pixel architecture, which makes it suitable for high-resolution images.&#10;This is in contrast to previous work on autoencoders for compression using&#10;coarser approximations, shallower architectures, computationally expensive&#10;methods, or focusing on small images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1312.6114" label="1312.6114">
-        <attvalues>
-          <attvalue for="0" value="Auto-Encoding Variational Bayes" />
-          <attvalue for="1" value="  How can we perform efficient inference and learning in directed probabilistic&#10;models, in the presence of continuous latent variables with intractable&#10;posterior distributions, and large datasets? We introduce a stochastic&#10;variational inference and learning algorithm that scales to large datasets and,&#10;under some mild differentiability conditions, even works in the intractable&#10;case. Our contributions are two-fold. First, we show that a reparameterization&#10;of the variational lower bound yields a lower bound estimator that can be&#10;straightforwardly optimized using standard stochastic gradient methods. Second,&#10;we show that for i.i.d. datasets with continuous latent variables per&#10;datapoint, posterior inference can be made especially efficient by fitting an&#10;approximate inference model (also called a recognition model) to the&#10;intractable posterior using the proposed lower bound estimator. Theoretical&#10;advantages are reflected in experimental results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.02958" label="1804.02958">
-        <attvalues>
-          <attvalue for="0" value="Generative Adversarial Networks for Extreme Learned Image Compression" />
-          <attvalue for="1" value="  We present a learned image compression system based on GANs, operating at&#10;extremely low bitrates. Our proposed framework combines an encoder,&#10;decoder/generator and a multi-scale discriminator, which we train jointly for a&#10;generative learned compression objective. The model synthesizes details it&#10;cannot afford to store, obtaining visually pleasing results at bitrates where&#10;previous methods fail and show strong artifacts. Furthermore, if a semantic&#10;label map of the original image is available, our method can fully synthesize&#10;unimportant regions in the decoded image such as streets and trees from the&#10;label map, proportionally reducing the storage cost. A user study confirms that&#10;for low bitrates, our approach is preferred to state-of-the-art methods, even&#10;when they use more than double the bits.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.05328" label="1606.05328">
-        <attvalues>
-          <attvalue for="0" value="Conditional Image Generation with PixelCNN Decoders" />
-          <attvalue for="1" value="  This work explores conditional image generation with a new image density&#10;model based on the PixelCNN architecture. The model can be conditioned on any&#10;vector, including descriptive labels or tags, or latent embeddings created by&#10;other networks. When conditioned on class labels from the ImageNet database,&#10;the model is able to generate diverse, realistic scenes representing distinct&#10;animals, objects, landscapes and structures. When conditioned on an embedding&#10;produced by a convolutional network given a single image of an unseen face, it&#10;generates a variety of new portraits of the same person with different facial&#10;expressions, poses and lighting conditions. We also show that conditional&#10;PixelCNN can serve as a powerful decoder in an image autoencoder. Additionally,&#10;the gated convolutional layers in the proposed model improve the log-likelihood&#10;of PixelCNN to match the state-of-the-art performance of PixelRNN on ImageNet,&#10;with greatly reduced computational cost.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.04187" label="1908.04187">
-        <attvalues>
-          <attvalue for="0" value="Human Perceptual Evaluations for Image Compression" />
-          <attvalue for="1" value="  Recently, there has been much interest in deep learning techniques to do&#10;image compression and there have been claims that several of these produce&#10;better results than engineered compression schemes (such as JPEG, JPEG2000 or&#10;BPG). A standard way of comparing image compression schemes today is to use&#10;perceptual similarity metrics such as PSNR or MS-SSIM (multi-scale structural&#10;similarity). This has led to some deep learning techniques which directly&#10;optimize for MS-SSIM by choosing it as a loss function. While this leads to a&#10;higher MS-SSIM for such techniques, we demonstrate using user studies that the&#10;resulting improvement may be misleading. Deep learning techniques for image&#10;compression with a higher MS-SSIM may actually be perceptually worse than&#10;engineered compression schemes with a lower MS-SSIM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.03924" label="1801.03924">
-        <attvalues>
-          <attvalue for="0" value="The Unreasonable Effectiveness of Deep Features as a Perceptual Metric" />
-          <attvalue for="1" value="  While it is nearly effortless for humans to quickly assess the perceptual&#10;similarity between two images, the underlying processes are thought to be quite&#10;complex. Despite this, the most widely used perceptual metrics today, such as&#10;PSNR and SSIM, are simple, shallow functions, and fail to account for many&#10;nuances of human perception. Recently, the deep learning community has found&#10;that features of the VGG network trained on ImageNet classification has been&#10;remarkably useful as a training loss for image synthesis. But how perceptual&#10;are these so-called &quot;perceptual losses&quot;? What elements are critical for their&#10;success? To answer these questions, we introduce a new dataset of human&#10;perceptual similarity judgments. We systematically evaluate deep features&#10;across different architectures and tasks and compare them with classic metrics.&#10;We find that deep features outperform all previous metrics by large margins on&#10;our dataset. More surprisingly, this result is not restricted to&#10;ImageNet-trained VGG features, but holds across different deep architectures&#10;and levels of supervision (supervised, self-supervised, or even unsupervised).&#10;Our results suggest that perceptual similarity is an emergent property shared&#10;across deep visual representations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.06077" label="1711.06077">
-        <attvalues>
-          <attvalue for="0" value="The Perception-Distortion Tradeoff" />
-          <attvalue for="1" value="  Image restoration algorithms are typically evaluated by some distortion&#10;measure (e.g. PSNR, SSIM, IFC, VIF) or by human opinion scores that quantify&#10;perceived perceptual quality. In this paper, we prove mathematically that&#10;distortion and perceptual quality are at odds with each other. Specifically, we&#10;study the optimal probability for correctly discriminating the outputs of an&#10;image restoration algorithm from real images. We show that as the mean&#10;distortion decreases, this probability must increase (indicating worse&#10;perceptual quality). As opposed to the common belief, this result holds true&#10;for any distortion measure, and is not only a problem of the PSNR or SSIM&#10;criteria. We also show that generative-adversarial-nets (GANs) provide a&#10;principled way to approach the perception-distortion bound. This constitutes&#10;theoretical support to their observed success in low-level vision tasks. Based&#10;on our analysis, we propose a new methodology for evaluating image restoration&#10;methods, and use it to perform an extensive comparison between recent&#10;super-resolution algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.07517" label="1809.07517">
-        <attvalues>
-          <attvalue for="0" value="The 2018 PIRM Challenge on Perceptual Image Super-resolution" />
-          <attvalue for="1" value="  This paper reports on the 2018 PIRM challenge on perceptual super-resolution&#10;(SR), held in conjunction with the Perceptual Image Restoration and&#10;Manipulation (PIRM) workshop at ECCV 2018. In contrast to previous SR&#10;challenges, our evaluation methodology jointly quantifies accuracy and&#10;perceptual quality, therefore enabling perceptual-driven methods to compete&#10;alongside algorithms that target PSNR maximization. Twenty-one participating&#10;teams introduced algorithms which well-improved upon the existing&#10;state-of-the-art methods in perceptual SR, as confirmed by a human opinion&#10;study. We also analyze popular image quality measures and draw conclusions&#10;regarding which of them correlates best with human opinion scores. We conclude&#10;with an analysis of the current trends in perceptual SR, as reflected from the&#10;leading submissions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.03478" label="1506.03478">
-        <attvalues>
-          <attvalue for="0" value="Generative Image Modeling Using Spatial LSTMs" />
-          <attvalue for="1" value="  Modeling the distribution of natural images is challenging, partly because of&#10;strong statistical dependencies which can extend over hundreds of pixels.&#10;Recurrent neural networks have been successful in capturing long-range&#10;dependencies in a number of problems but only recently have found their way&#10;into generative image models. We here introduce a recurrent image model based&#10;on multi-dimensional long short-term memory units which are particularly suited&#10;for image modeling due to their spatial structure. Our model scales to images&#10;of arbitrary size and its likelihood is computationally tractable. We find that&#10;it outperforms the state of the art in quantitative comparisons on several&#10;image datasets and produces promising results when used for texture synthesis&#10;and inpainting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.12535" label="2401.12535">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Vision Transformers Are Efficient Segmentation Learners&#10;  for Imperfect Labels" />
-          <attvalue for="1" value="  This study demonstrates a cost-effective approach to semantic segmentation&#10;using self-supervised vision transformers (SSVT). By freezing the SSVT backbone&#10;and training a lightweight segmentation head, our approach effectively utilizes&#10;imperfect labels, thereby improving robustness to label imperfections.&#10;Empirical experiments show significant performance improvements over existing&#10;methods for various annotation types, including scribble, point-level, and&#10;image-level labels. The research highlights the effectiveness of&#10;self-supervised vision transformers in dealing with imperfect labels, providing&#10;a practical and efficient solution for semantic segmentation while reducing&#10;annotation costs. Through extensive experiments, we confirm that our method&#10;outperforms baseline models for all types of imperfect labels. Especially under&#10;the zero-shot vision-language-model-based label, our model exhibits 11.5\%p&#10;performance gain compared to the baseline.&#10;" />
-          <attvalue for="2" value="&#10;Semantic segmentation is a critical task in computer vision, involving the understanding of an image's semantics and the recognition of objects within it. Unlike image classification, this technique assigns a class to each pixel in an image in order to obtain dense predictions. Semantic segmentation is widely utilized in various fields that demand accurate and detailed predictions, such as autonomous driving and medical imaging~\cite{cordts2016cityscapes,dolz2018ivd}. Despite its importance, semantic segmentation faces challenges due to the requirement for highly precise pixel-level labels, which makes data preparation time-consuming and costly~\cite{cordts2016cityscapes}. This hinders the practical application of semantic segmentation.&#10;&#10;To address the issue of the high annotation cost of semantic segmentation, there has been increasing interest in weakly supervised approaches that utilize annotations that are less expensive than pixel-level labels. These include methods ranging from scribble-level~\cite{pan2021scribble} to point-level~\cite{tel_liang2022tree,bearman2016s}, image-level~\cite{eps_lee2021railroad,SEAM_wang2020self}, and zero-shot approaches based on Vision-Language (VL) models~\cite{maskclip_zhou2022extract} (extracting templates from text). However, these cheaper labels used in weakly supervised approaches are imperfect compared to full supervision~\cite{adele_liu2022adaptive}: (1) they contain significant noise and (2) they provide far fewer labeled pixels. This imperfection can hinder the generalization ability of a model during training, which poses a limitation in real-world applications where low-cost model development is desired. Therefore, our focus is on effectively and efficiently utilizing these imperfect masks.&#10;&#10;We propose a method that utilizes the shape prior of a self-supervised vision transformer (SSVT) to effectively leverage imperfect labels. Recent studies, such as DINO~\cite{dino_caron2021emerging,dinov2_oquab2023dinov2}, have demonstrated that when vision transformers are trained in a self-supervised manner, they develop features suitable for segmentation, including scene layout. To preserve the structural information inherent in SSVT, we propose using the SSVT as a backbone with a frozen state. We only train a lightweight segmentation head to assign classes based on shape-rich features. This approach maintains the robust shape prior of SSVT, resulting in features and segmentation results that are not biased toward imperfect data and exhibit a high level of generalization power. Furthermore, training only the segmentation head significantly reduces the number of parameter updates, thereby reducing the cost of training the entire model.&#10;&#10;Through empirical experiments, we validate that our method outperforms existing techniques across various types of weak annotations. Specifically, we observe performance improvements over the state-of-the-art TEL~\cite{tel_liang2022tree} for scribble and point level labels by 4.1\%p, and over ADELE~\cite{adele_liu2022adaptive} by 1.9\%p, which improves noisy image-level labels. Lastly, when using text-driven labels in a vision-language model like MaskCLIP~\cite{maskclip_zhou2022extract}, our method outperforms existing method~\cite{segformer_xie2021segformer} by 11.5\%p. We also confirm that various types of self-supervised vision transformers~\cite{ibot_zhou2021image,dino_caron2021emerging,dinov2_oquab2023dinov2} are effective in learning from imperfect labels compared to traditional methods for training segmentation networks.&#10;&#10;In summary, our contributions are as follows:&#10;\begin{itemize}&#10; \item We propose a cost effective strategy for training semantic segmentation network under imperfect labels, such as scribble, points, and noisy labels.&#10; \item We introduce a segmentation probe-centric training approach that effectively leverages the shape prior of SSVT models to enhance model generalizability.&#10; \item We validate the superiority of our model through various experiments across different types of imperfect labels. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1811.08305" label="1811.08305">
-        <attvalues>
-          <attvalue for="0" value="IVD-Net: Intervertebral disc localization and segmentation in MRI with a&#10;  multi-modal UNet" />
-          <attvalue for="1" value="  Accurate localization and segmentation of intervertebral disc (IVD) is&#10;crucial for the assessment of spine disease diagnosis. Despite the&#10;technological advances in medical imaging, IVD localization and segmentation&#10;are still manually performed, which is time-consuming and prone to errors. If,&#10;in addition, multi-modal imaging is considered, the burden imposed on disease&#10;assessments increases substantially. In this paper, we propose an architecture&#10;for IVD localization and segmentation in multi-modal MRI, which extends the&#10;well-known UNet. Compared to single images, multi-modal data brings&#10;complementary information, contributing to better data representation and&#10;discriminative power. Our contributions are three-fold. First, how to&#10;effectively integrate and fully leverage multi-modal data remains almost&#10;unexplored. In this work, each MRI modality is processed in a different path to&#10;better exploit their unique information. Second, inspired by HyperDenseNet, the&#10;network is densely-connected both within each path and across different paths,&#10;granting the model the freedom to learn where and how the different modalities&#10;should be processed and combined. Third, we improved standard U-Net modules by&#10;extending inception modules with two dilated convolutions blocks of different&#10;scale, which helps handling multi-scale context. We report experiments over the&#10;data set of the public MICCAI 2018 Challenge on Automatic Intervertebral Disc&#10;Localization and Segmentation, with 13 multi-modal MRI images used for training&#10;and 3 for validation. We trained IVD-Net on an NVidia TITAN XP GPU with 16 GBs&#10;RAM, using ADAM as optimizer and a learning rate of 10e-5 during 200 epochs.&#10;Training took about 5 hours, and segmentation of a whole volume about 2-3&#10;seconds, on average. Several baselines, with different multi-modal fusion&#10;strategies, were used to demonstrate the effectiveness of the proposed&#10;architecture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.09896" label="2102.09896">
-        <attvalues>
-          <attvalue for="0" value="Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on&#10;  Neural Representation and Self-Supervision on Neural Eigenspace" />
-          <attvalue for="1" value="  Scribble-supervised semantic segmentation has gained much attention recently&#10;for its promising performance without high-quality annotations. Due to the lack&#10;of supervision, confident and consistent predictions are usually hard to&#10;obtain. Typically, people handle these problems to either adopt an auxiliary&#10;task with the well-labeled dataset or incorporate the graphical model with&#10;additional requirements on scribble annotations. Instead, this work aims to&#10;achieve semantic segmentation by scribble annotations directly without extra&#10;information and other limitations. Specifically, we propose holistic&#10;operations, including minimizing entropy and a network embedded random walk on&#10;neural representation to reduce uncertainty. Given the probabilistic transition&#10;matrix of a random walk, we further train the network with self-supervision on&#10;its neural eigenspace to impose consistency on predictions between related&#10;images. Comprehensive experiments and ablation studies verify the proposed&#10;approach, which demonstrates superiority over others; it is even comparable to&#10;some full-label supervised ones and works well when scribbles are randomly&#10;shrunk or dropped.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.10739" label="2203.10739">
-        <attvalues>
-          <attvalue for="0" value="Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation" />
-          <attvalue for="1" value="  Sparsely annotated semantic segmentation (SASS) aims to train a segmentation&#10;network with coarse-grained (i.e., point-, scribble-, and block-wise)&#10;supervisions, where only a small proportion of pixels are labeled in each&#10;image. In this paper, we propose a novel tree energy loss for SASS by providing&#10;semantic guidance for unlabeled pixels. The tree energy loss represents images&#10;as minimum spanning trees to model both low-level and high-level pair-wise&#10;affinities. By sequentially applying these affinities to the network&#10;prediction, soft pseudo labels for unlabeled pixels are generated in a&#10;coarse-to-fine manner, achieving dynamic online self-training. The tree energy&#10;loss is effective and easy to be incorporated into existing frameworks by&#10;combining it with a traditional segmentation loss. Compared with previous SASS&#10;methods, our method requires no multistage training strategies, alternating&#10;optimization procedures, additional supervised data, or time-consuming&#10;post-processing while outperforming them in all SASS settings. Code is&#10;available at https://github.com/megvii-research/TreeEnergyLoss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.08965" label="2105.08965">
-        <attvalues>
-          <attvalue for="0" value="Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly&#10;  Supervised Semantic Segmentation" />
-          <attvalue for="1" value="  Existing studies in weakly-supervised semantic segmentation (WSSS) using&#10;image-level weak supervision have several limitations: sparse object coverage,&#10;inaccurate object boundaries, and co-occurring pixels from non-target objects.&#10;To overcome these challenges, we propose a novel framework, namely Explicit&#10;Pseudo-pixel Supervision (EPS), which learns from pixel-level feedback by&#10;combining two weak supervisions; the image-level label provides the object&#10;identity via the localization map and the saliency map from the off-the-shelf&#10;saliency detection model offers rich boundaries. We devise a joint training&#10;strategy to fully utilize the complementary relationship between both&#10;information. Our method can obtain accurate object boundaries and discard&#10;co-occurring pixels, thereby significantly improving the quality of&#10;pseudo-masks. Experimental results show that the proposed method remarkably&#10;outperforms existing methods by resolving key challenges of WSSS and achieves&#10;the new state-of-the-art performance on both PASCAL VOC 2012 and MS COCO 2014&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.04581" label="2004.04581">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Equivariant Attention Mechanism for Weakly Supervised&#10;  Semantic Segmentation" />
-          <attvalue for="1" value="  Image-level weakly supervised semantic segmentation is a challenging problem&#10;that has been deeply studied in recent years. Most of advanced solutions&#10;exploit class activation map (CAM). However, CAMs can hardly serve as the&#10;object mask due to the gap between full and weak supervisions. In this paper,&#10;we propose a self-supervised equivariant attention mechanism (SEAM) to discover&#10;additional supervision and narrow the gap. Our method is based on the&#10;observation that equivariance is an implicit constraint in fully supervised&#10;semantic segmentation, whose pixel-level labels take the same spatial&#10;transformation as the input images during data augmentation. However, this&#10;constraint is lost on the CAMs trained by image-level supervision. Therefore,&#10;we propose consistency regularization on predicted CAMs from various&#10;transformed images to provide self-supervision for network learning. Moreover,&#10;we propose a pixel correlation module (PCM), which exploits context appearance&#10;information and refines the prediction of current pixel by its similar&#10;neighbors, leading to further improvement on CAMs consistency. Extensive&#10;experiments on PASCAL VOC 2012 dataset demonstrate our method outperforms&#10;state-of-the-art methods using the same level of supervision. The code is&#10;released online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.01071" label="2112.01071">
-        <attvalues>
-          <attvalue for="0" value="Extract Free Dense Labels from CLIP" />
-          <attvalue for="1" value="  Contrastive Language-Image Pre-training (CLIP) has made a remarkable&#10;breakthrough in open-vocabulary zero-shot image recognition. Many recent&#10;studies leverage the pre-trained CLIP models for image-level classification and&#10;manipulation. In this paper, we wish examine the intrinsic potential of CLIP&#10;for pixel-level dense prediction, specifically in semantic segmentation. To&#10;this end, with minimal modification, we show that MaskCLIP yields compelling&#10;segmentation results on open concepts across various datasets in the absence of&#10;annotations and fine-tuning. By adding pseudo labeling and self-training,&#10;MaskCLIP+ surpasses SOTA transductive zero-shot semantic segmentation methods&#10;by large margins, e.g., mIoUs of unseen classes on PASCAL VOC/PASCAL&#10;Context/COCO Stuff are improved from 35.6/20.7/30.3 to 86.1/66.7/54.7. We also&#10;test the robustness of MaskCLIP under input corruption and evaluate its&#10;capability in discriminating fine-grained objects and novel concepts. Our&#10;finding suggests that MaskCLIP can serve as a new reliable source of&#10;supervision for dense prediction tasks to achieve annotation-free segmentation.&#10;Source code is available at https://github.com/chongzhou96/MaskCLIP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.03740" label="2110.03740">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Early-Learning Correction for Segmentation from Noisy&#10;  Annotations" />
-          <attvalue for="1" value="  Deep learning in the presence of noisy annotations has been studied&#10;extensively in classification, but much less in segmentation tasks. In this&#10;work, we study the learning dynamics of deep segmentation networks trained on&#10;inaccurately-annotated data. We discover a phenomenon that has been previously&#10;reported in the context of classification: the networks tend to first fit the&#10;clean pixel-level labels during an &quot;early-learning&quot; phase, before eventually&#10;memorizing the false annotations. However, in contrast to classification,&#10;memorization in segmentation does not arise simultaneously for all semantic&#10;categories. Inspired by these findings, we propose a new method for&#10;segmentation from noisy annotations with two key elements. First, we detect the&#10;beginning of the memorization phase separately for each category during&#10;training. This allows us to adaptively correct the noisy annotations in order&#10;to exploit early learning. Second, we incorporate a regularization term that&#10;enforces consistency across scales to boost robustness against annotation&#10;noise. Our method outperforms standard approaches on a medical-imaging&#10;segmentation task where noises are synthesized to mimic human annotation&#10;errors. It also provides robustness to realistic noisy annotations present in&#10;weakly-supervised semantic segmentation, achieving state-of-the-art results on&#10;PASCAL VOC 2012. Code is available at https://github.com/Kangningthu/ADELE&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.15203" label="2105.15203">
-        <attvalues>
-          <attvalue for="0" value="SegFormer: Simple and Efficient Design for Semantic Segmentation with&#10;  Transformers" />
-          <attvalue for="1" value="  We present SegFormer, a simple, efficient yet powerful semantic segmentation&#10;framework which unifies Transformers with lightweight multilayer perception&#10;(MLP) decoders. SegFormer has two appealing features: 1) SegFormer comprises a&#10;novel hierarchically structured Transformer encoder which outputs multiscale&#10;features. It does not need positional encoding, thereby avoiding the&#10;interpolation of positional codes which leads to decreased performance when the&#10;testing resolution differs from training. 2) SegFormer avoids complex decoders.&#10;The proposed MLP decoder aggregates information from different layers, and thus&#10;combining both local attention and global attention to render powerful&#10;representations. We show that this simple and lightweight design is the key to&#10;efficient segmentation on Transformers. We scale our approach up to obtain a&#10;series of models from SegFormer-B0 to SegFormer-B5, reaching significantly&#10;better performance and efficiency than previous counterparts. For example,&#10;SegFormer-B4 achieves 50.3% mIoU on ADE20K with 64M parameters, being 5x&#10;smaller and 2.2% better than the previous best method. Our best model,&#10;SegFormer-B5, achieves 84.0% mIoU on Cityscapes validation set and shows&#10;excellent zero-shot robustness on Cityscapes-C. Code will be released at:&#10;github.com/NVlabs/SegFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.16057" label="2210.16057">
-        <attvalues>
-          <attvalue for="0" value="Semi-UFormer: Semi-supervised Uncertainty-aware Transformer for Image&#10;  Dehazing" />
-          <attvalue for="1" value="  Image dehazing is fundamental yet not well-solved in computer vision. Most&#10;cutting-edge models are trained in synthetic data, leading to the poor&#10;performance on real-world hazy scenarios. Besides, they commonly give&#10;deterministic dehazed images while neglecting to mine their uncertainty. To&#10;bridge the domain gap and enhance the dehazing performance, we propose a novel&#10;semi-supervised uncertainty-aware transformer network, called Semi-UFormer.&#10;Semi-UFormer can well leverage both the real-world hazy images and their&#10;uncertainty guidance information. Specifically, Semi-UFormer builds itself on&#10;the knowledge distillation framework. Such teacher-student networks effectively&#10;absorb real-world haze information for quality dehazing. Furthermore, an&#10;uncertainty estimation block is introduced into the model to estimate the pixel&#10;uncertainty representations, which is then used as a guidance signal to help&#10;the student network produce haze-free images more accurately. Extensive&#10;experiments demonstrate that Semi-UFormer generalizes well from synthetic to&#10;real-world images.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Images captured in hazy weather often suffer from noticeable visibility degradation, color distortions, and contrast reduction, which further drop the performance of downstream vision-based systems such as object detection, autonomous driving, and traffic surveillance \cite{chen2018domain,choi2017sharpness}. Therefore, restoring clean images from their hazy versions is quite important. &#10;&#10;Existing dehazing efforts can be roughly classified into prior-based \cite{he2010single,berman2016non} and learning-based approaches \cite{li2017aod,chen2019gated}. &#10;Traditional prior-based studies often exploit hand-crafted image priors to solve the image dehazing problem based on the atmospheric scattering model \cite{narasimhan2002vision}, such as dark channel prior (DCP) \cite{he2010single}, non-local color prior \cite{berman2016non}, etc. Although these algorithms can improve the overall visibility of the image, they are not always reliable due to over-reliance on assumptions.&#10;&#10;Recent advances in deep learning open up huge opportunities for image dehazing tasks, and a large number of learning-based approaches have sprung up \cite{li2017aod,chen2019gated,qin2020ffa,song2022vision}. While these algorithms are efficient and can produce promising results on various popular benchmarks, most of them are trained on synthetic data, so they cannot generalize well to real-world scenes due to the existence of domain gap. Recently, several semi-supervised \cite{li2019semi,chen2021psd} and unsupervised \cite{li2021you,wang2022cycle} methods have attempted to solve the domain shift issue via training models on real-world images. Although these approaches alleviate the domain shift issue to some extent, their dehazing capacity is usually limited because the ground-truth of real-world hazy images cannot be used as the reconstruction loss to constrain the training of the network. &#10;&#10;In addition, most learning-based dehazing efforts only produce the final dehazed images without discussing the uncertainty of the results, which is important for recovering edge and texture regions in hazy images. Furthermore, current dehazing models typically treat all pixels equally, but pixels in edge and texture regions obviously contain more visual information than pixels in smooth regions \cite{ning2021uncertainty}, and such pixels tend to have a high degree of uncertainty. Therefore, accurate estimation and appropriate use of image uncertainty can guide the network to focus on pixels with large uncertainty, so that pixels in specific regions can be enhanced to improve the quality of the final dehazed images. &#10;&#10;To resolve these issues, we propose a novel semi-supervised uncertainty-aware transformer network (Semi-UFormer) for image dehazing. Semi-UFormer builds itself on the knowledge distillation framework and benefits from uncertainty guidance information, thus producing much clearer images with well-preserved details. In summary, our contributions are three-fold: (1) A novel semi-supervised uncertainty-aware transformer network called Semi-UFormer is proposed for image dehazing, which leverages both real-world data and uncertainty guidance information to boost the model's dehazing ability. (2) An uncertainty estimation block is exploited to predict the epistemic uncertainty of the dehazed images, which is then used to guide the network to better reconstruct the image texture and edge regions. (3) We leverage knowledge distillation technology to align the feature distributions between synthetic and real data, which can help the network generalize well in real-world scenarios. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Uncertainty Estimation, Image Dehazing" />
-        </attvalues>
-      </node>
-      <node id="1803.03243" label="1803.03243">
-        <attvalues>
-          <attvalue for="0" value="Domain Adaptive Faster R-CNN for Object Detection in the Wild" />
-          <attvalue for="1" value="  Object detection typically assumes that training and test data are drawn from&#10;an identical distribution, which, however, does not always hold in practice.&#10;Such a distribution mismatch will lead to a significant performance drop. In&#10;this work, we aim to improve the cross-domain robustness of object detection.&#10;We tackle the domain shift on two levels: 1) the image-level shift, such as&#10;image style, illumination, etc, and 2) the instance-level shift, such as object&#10;appearance, size, etc. We build our approach based on the recent&#10;state-of-the-art Faster R-CNN model, and design two domain adaptation&#10;components, on image level and instance level, to reduce the domain&#10;discrepancy. The two domain adaptation components are based on H-divergence&#10;theory, and are implemented by learning a domain classifier in adversarial&#10;training manner. The domain classifiers on different levels are further&#10;reinforced with a consistency regularization to learn a domain-invariant region&#10;proposal network (RPN) in the Faster R-CNN model. We evaluate our newly&#10;proposed approach using multiple datasets including Cityscapes, KITTI, SIM10K,&#10;etc. The results demonstrate the effectiveness of our proposed approach for&#10;robust object detection in various domain shift scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.08747" label="1811.08747">
-        <attvalues>
-          <attvalue for="0" value="Gated Context Aggregation Network for Image Dehazing and Deraining" />
-          <attvalue for="1" value="  Image dehazing aims to recover the uncorrupted content from a hazy image.&#10;Instead of leveraging traditional low-level or handcrafted image priors as the&#10;restoration constraints, e.g., dark channels and increased contrast, we propose&#10;an end-to-end gated context aggregation network to directly restore the final&#10;haze-free image. In this network, we adopt the latest smoothed dilation&#10;technique to help remove the gridding artifacts caused by the widely-used&#10;dilated convolution with negligible extra parameters, and leverage a gated&#10;sub-network to fuse the features from different levels. Extensive experiments&#10;demonstrate that our method can surpass previous state-of-the-art methods by a&#10;large margin both quantitatively and qualitatively. In addition, to demonstrate&#10;the generality of the proposed method, we further apply it to the image&#10;deraining task, which also achieves the state-of-the-art performance. Code has&#10;been made available at https://github.com/cddlyf/GCANet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.07559" label="1911.07559">
-        <attvalues>
-          <attvalue for="0" value="FFA-Net: Feature Fusion Attention Network for Single Image Dehazing" />
-          <attvalue for="1" value="  In this paper, we propose an end-to-end feature fusion at-tention network&#10;(FFA-Net) to directly restore the haze-free image. The FFA-Net architecture&#10;consists of three key components:&#10;  1) A novel Feature Attention (FA) module combines Channel Attention with&#10;Pixel Attention mechanism, considering that different channel-wise features&#10;contain totally different weighted information and haze distribution is uneven&#10;on the different image pixels. FA treats different features and pixels&#10;unequally, which provides additional flexibility in dealing with different&#10;types of information, expanding the representational ability of CNNs. 2) A&#10;basic block structure consists of Local Residual Learning and Feature&#10;Attention, Local Residual Learning allowing the less important information such&#10;as thin haze region or low-frequency to be bypassed through multiple local&#10;residual connections, let main network architecture focus on more effective&#10;information. 3) An Attention-based different levels Feature Fusion (FFA)&#10;structure, the feature weights are adaptively learned from the Feature&#10;Attention (FA) module, giving more weight to important features. This structure&#10;can also retain the information of shallow layers and pass it into deep layers.&#10;  The experimental results demonstrate that our proposed FFA-Net surpasses&#10;previous state-of-the-art single image dehazing methods by a very large margin&#10;both quantitatively and qualitatively, boosting the best published PSNR metric&#10;from 30.23db to 36.39db on the SOTS indoor test dataset.&#10;  Code has been made available at GitHub.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.03883" label="2204.03883">
-        <attvalues>
-          <attvalue for="0" value="Vision Transformers for Single Image Dehazing" />
-          <attvalue for="1" value="  Image dehazing is a representative low-level vision task that estimates&#10;latent haze-free images from hazy images. In recent years, convolutional neural&#10;network-based methods have dominated image dehazing. However, vision&#10;Transformers, which has recently made a breakthrough in high-level vision&#10;tasks, has not brought new dimensions to image dehazing. We start with the&#10;popular Swin Transformer and find that several of its key designs are&#10;unsuitable for image dehazing. To this end, we propose DehazeFormer, which&#10;consists of various improvements, such as the modified normalization layer,&#10;activation function, and spatial information aggregation scheme. We train&#10;multiple variants of DehazeFormer on various datasets to demonstrate its&#10;effectiveness. Specifically, on the most frequently used SOTS indoor set, our&#10;small model outperforms FFA-Net with only 25% #Param and 5% computational cost.&#10;To the best of our knowledge, our large model is the first method with the PSNR&#10;over 40 dB on the SOTS indoor set, dramatically outperforming the previous&#10;state-of-the-art methods. We also collect a large-scale realistic remote&#10;sensing dehazing dataset for evaluating the method's capability to remove&#10;highly non-homogeneous haze.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.16829" label="2006.16829">
-        <attvalues>
-          <attvalue for="0" value="You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing&#10;  Neural Network" />
-          <attvalue for="1" value="  In this paper, we study two challenging and less-touched problems in single&#10;image dehazing, namely, how to make deep learning achieve image dehazing&#10;without training on the ground-truth clean image (unsupervised) and a image&#10;collection (untrained). An unsupervised neural network will avoid the intensive&#10;labor collection of hazy-clean image pairs, and an untrained model is a&#10;``real'' single image dehazing approach which could remove haze based on only&#10;the observed hazy image itself and no extra images is used. Motivated by the&#10;layer disentanglement idea, we propose a novel method, called you only look&#10;yourself (\textbf{YOLY}) which could be one of the first unsupervised and&#10;untrained neural networks for image dehazing. In brief, YOLY employs three&#10;jointly subnetworks to separate the observed hazy image into several latent&#10;layers, \textit{i.e.}, scene radiance layer, transmission map layer, and&#10;atmospheric light layer. After that, these three layers are further composed to&#10;the hazy image in a self-supervised manner. Thanks to the unsupervised and&#10;untrained characteristics of YOLY, our method bypasses the conventional&#10;training paradigm of deep models on hazy-clean pairs or a large scale dataset,&#10;thus avoids the labor-intensive data collection and the domain shift issue.&#10;Besides, our method also provides an effective learning-based haze transfer&#10;solution thanks to its layer disentanglement mechanism. Extensive experiments&#10;show the promising performance of our method in image dehazing compared with 14&#10;methods on four databases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.09585" label="2305.09585">
-        <attvalues>
-          <attvalue for="0" value="Inductive Graph Neural Networks for Moving Object Segmentation" />
-          <attvalue for="1" value="  Moving Object Segmentation (MOS) is a challenging problem in computer vision,&#10;particularly in scenarios with dynamic backgrounds, abrupt lighting changes,&#10;shadows, camouflage, and moving cameras. While graph-based methods have shown&#10;promising results in MOS, they have mainly relied on transductive learning&#10;which assumes access to the entire training and testing data for evaluation.&#10;However, this assumption is not realistic in real-world applications where the&#10;system needs to handle new data during deployment. In this paper, we propose a&#10;novel Graph Inductive Moving Object Segmentation (GraphIMOS) algorithm based on&#10;a Graph Neural Network (GNN) architecture. Our approach builds a generic model&#10;capable of performing prediction on newly added data frames using the already&#10;trained model. GraphIMOS outperforms previous inductive learning methods and is&#10;more generic than previous transductive techniques. Our proposed algorithm&#10;enables the deployment of graph-based MOS models in real-world applications.&#10;" />
-          <attvalue for="2" value="&#10;&#10;{Moving Object Segmentation (MOS) is an important problem in computer vision, particularly in surveillance system applications \cite{garcia2020background}. The goal of MOS is to identify and separate the pixels or regions in a video that correspond to moving objects from the static background or other static objects. Deep learning models \cite{BOUWMANS20198} have demonstrated strong performance on large-scale datasets. However, as the quality of data improves, these models become increasingly complex and computationally intensive, even with fast algorithms \cite{hou2020super}, few-shot learning methods \cite{9897894}, and specialized architectures \cite{9191151}.}&#10;&#10;{The most common deep models are mostly supervised and can be divided into four groups \cite{hou2023survey}: 2D Convolutional Neural Networks (CNNs) \cite{braham2016deep}, 3D CNNs \cite{akilan20193d}, transformer neural networks \cite{9607687}, and generative adversarial networks \cite{bakkay2018bscgan,SULTANA2022240}. In addition to these, some state-of-the-art (SOTA) techniques have been combined with deep methods to create novel approaches, such as MotionRec \cite{mandal2020motionrec}, RT-SBS \cite{cioppa2020real}, and GraphMOS \cite{giraldo2020graph}. Recent graph-based algorithms like GraphMOS \cite{giraldo2020graph} and GraphMOD-Net \cite{10.1007/978-3-030-81638-4_3} use semi-supervised learning posed as a graph signal reconstruction problem. These methods are inspired by the theory of graph signal processing \cite{ortega2018graph} and have shown promising results.&#10;However, these graph-based methods are transductive in nature, meaning that the model needs to be fully retrained and the graph regenerated whenever a new video is added. To address this issue, we propose inductive techniques for graph-based MOS, where multiple graphs are built instead of a single large graph. This approach reduces the need for rebuilding the whole graph and retraining the model, making it more suitable for real-world deployments, as shown in Fig. \ref{fig:teaser}.}&#10;&#10;In this work, we propose a novel Graph Inductive Moving Object Segmentation (GraphIMOS) algorithm based on Graph Neural Networks (GNNs) \cite{bronstein2021geometric}. We represent each instance of the video as a node in a graph, which is generated using a Mask Region Convolutional Neural Network (Mask R-CNN) with a ResNet-50 and Feature Pyramid Network (FPN) backbone. We use optical flow, intensity, and texture features to represent the nodes. Instead of creating a single large graph, as in \cite{giraldo2020graph}, we generate $\rho + \alpha$ $k$-Nearest Neighbors ($k$-NN) graphs, where each graph can have a different number of nodes. We set $\rho = 3$ in the experiments for the training and validation graphs, which are then fed to the proposed GNN model for training and hyperparameter optimization. To evaluate the performance of our model, we test it on one single graph $\alpha = 1$, built from previously unseen videos. Our approach is evaluated on the challenging Change Detection 2014 (CDNet 2014) dataset \cite{wang2014cdnet}, and we demonstrate competitive results against previous inductive methods.&#10;&#10;This paper makes two main contributions. Firstly, we propose an inductive graph-based framework for Moving Object Segmentation (MOS), which is an important step toward the real-world deployment of graph-based methods in video surveillance applications. Secondly, we introduce a novel GNN architecture specifically designed for MOS, called GraphIMOS. To the best of our knowledge, GraphIMOS is the first graph-based inductive learning approach for MOS. The remainder of this paper is organized as follows. In Section 2, we introduce the preliminary concepts and describe the GraphIMOS algorithm and architecture. Section 3 presents the experimental setup and results. Finally, in Section 4, we present our conclusions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Segmentation, Computer Science, Graph Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="1901.03577" label="1901.03577">
-        <attvalues>
-          <attvalue for="0" value="Background Subtraction in Real Applications: Challenges, Current Models&#10;  and Future Directions" />
-          <attvalue for="1" value="  Computer vision applications based on videos often require the detection of&#10;moving objects in their first step. Background subtraction is then applied in&#10;order to separate the background and the foreground. In literature, background&#10;subtraction is surely among the most investigated field in computer vision&#10;providing a big amount of publications. Most of them concern the application of&#10;mathematical and machine learning models to be more robust to the challenges&#10;met in videos. However, the ultimate goal is that the background subtraction&#10;methods developed in research could be employed in real applications like&#10;traffic surveillance. But looking at the literature, we can remark that there&#10;is often a gap between the current methods used in real applications and the&#10;current methods in fundamental research. In addition, the videos evaluated in&#10;large-scale datasets are not exhaustive in the way that they only covered a&#10;part of the complete spectrum of the challenges met in real applications. In&#10;this context, we attempt to provide the most exhaustive survey as possible on&#10;real applications that used background subtraction in order to identify the&#10;real challenges met in practice, the current used background models and to&#10;provide future directions. Thus, challenges are investigated in terms of&#10;camera, foreground objects and environments. In addition, we identify the&#10;background models that are effectively used in these applications in order to&#10;find potential usable recent background models in terms of robustness, time and&#10;memory requirements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.05255" label="1811.05255">
-        <attvalues>
-          <attvalue for="0" value="Deep Neural Network Concepts for Background Subtraction: A Systematic&#10;  Review and Comparative Evaluation" />
-          <attvalue for="1" value="  Conventional neural networks show a powerful framework for background&#10;subtraction in video acquired by static cameras. Indeed, the well-known SOBS&#10;method and its variants based on neural networks were the leader methods on the&#10;largescale CDnet 2012 dataset during a long time. Recently, convolutional&#10;neural networks which belong to deep learning methods were employed with&#10;success for background initialization, foreground detection and deep learned&#10;features. Currently, the top current background subtraction methods in CDnet&#10;2014 are based on deep neural networks with a large gap of performance in&#10;comparison on the conventional unsupervised approaches based on multi-features&#10;or multi-cues strategies. Furthermore, a huge amount of papers was published&#10;since 2016 when Braham and Van Droogenbroeck published their first work on CNN&#10;applied to background subtraction providing a regular gain of performance. In&#10;this context, we provide the first review of deep neural network concepts in&#10;background subtraction for novices and experts in order to analyze this success&#10;and to provide further directions. For this, we first surveyed the methods used&#10;background initialization, background subtraction and deep learned features.&#10;Then, we discuss the adequacy of deep neural networks for background&#10;subtraction. Finally, experimental results are presented on the CDnet 2014&#10;dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.04993" label="2002.04993">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Semantic Background Subtraction" />
-          <attvalue for="1" value="  Semantic background subtraction SBS has been shown to improve the performance&#10;of most background subtraction algorithms by combining them with semantic&#10;information, derived from a semantic segmentation network. However, SBS&#10;requires high-quality semantic segmentation masks for all frames, which are&#10;slow to compute. In addition, most state-of-the-art background subtraction&#10;algorithms are not real-time, which makes them unsuitable for real-world&#10;applications. In this paper, we present a novel background subtraction&#10;algorithm called Real-Time Semantic Background Subtraction (denoted RT-SBS)&#10;which extends SBS for real-time constrained applications while keeping similar&#10;performances. RT-SBS effectively combines a real-time background subtraction&#10;algorithm with high-quality semantic information which can be provided at a&#10;slower pace, independently for each pixel. We show that RT-SBS coupled with&#10;ViBe sets a new state of the art for real-time background subtraction&#10;algorithms and even competes with the non real-time state-of-the-art ones. Note&#10;that we provide python CPU and GPU implementations of RT-SBS at&#10;https://github.com/cioppaanthony/rt-sbs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.00468" label="1712.00468">
-        <attvalues>
-          <attvalue for="0" value="Graph Signal Processing: Overview, Challenges and Applications" />
-          <attvalue for="1" value="  Research in Graph Signal Processing (GSP) aims to develop tools for&#10;processing data defined on irregular graph domains. In this paper we first&#10;provide an overview of core ideas in GSP and their connection to conventional&#10;digital signal processing. We then summarize recent developments in developing&#10;basic GSP tools, including methods for sampling, filtering or graph learning.&#10;Next, we review progress in several application areas using GSP, including&#10;processing and analysis of sensor network data, biological data, and&#10;applications to image processing and machine learning. We finish by providing a&#10;brief historical perspective to highlight how concepts recently developed in&#10;GSP build on top of prior research in other areas.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.13478" label="2104.13478">
-        <attvalues>
-          <attvalue for="0" value="Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges" />
-          <attvalue for="1" value="  The last decade has witnessed an experimental revolution in data science and&#10;machine learning, epitomised by deep learning methods. Indeed, many&#10;high-dimensional learning tasks previously thought to be beyond reach -- such&#10;as computer vision, playing Go, or protein folding -- are in fact feasible with&#10;appropriate computational scale. Remarkably, the essence of deep learning is&#10;built from two simple algorithmic principles: first, the notion of&#10;representation or feature learning, whereby adapted, often hierarchical,&#10;features capture the appropriate notion of regularity for each task, and&#10;second, learning by local gradient-descent type methods, typically implemented&#10;as backpropagation.&#10;  While learning generic functions in high dimensions is a cursed estimation&#10;problem, most tasks of interest are not generic, and come with essential&#10;pre-defined regularities arising from the underlying low-dimensionality and&#10;structure of the physical world. This text is concerned with exposing these&#10;regularities through unified geometric principles that can be applied&#10;throughout a wide spectrum of applications.&#10;  Such a 'geometric unification' endeavour, in the spirit of Felix Klein's&#10;Erlangen Program, serves a dual purpose: on one hand, it provides a common&#10;mathematical framework to study the most successful neural network&#10;architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand,&#10;it gives a constructive procedure to incorporate prior physical knowledge into&#10;neural architectures and provide principled way to build future architectures&#10;yet to be invented.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.09354" label="2110.09354">
-        <attvalues>
-          <attvalue for="0" value="An Analysis and Implementation of the HDR+ Burst Denoising Method" />
-          <attvalue for="1" value="  HDR+ is an image processing pipeline presented by Google in 2016. At its core&#10;lies a denoising algorithm that uses a burst of raw images to produce a single&#10;higher quality image. Since it is designed as a versatile solution for&#10;smartphone cameras, it does not necessarily aim for the maximization of&#10;standard denoising metrics, but rather for the production of natural, visually&#10;pleasing images. In this article, we specifically discuss and analyze the HDR+&#10;burst denoising algorithm architecture and the impact of its various&#10;parameters. With this publication, we provide an open source Python&#10;implementation of the algorithm, along with an interactive demo.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image noise is a common issue in digital photography, and that issue is even more prominent in smaller sensors used in devices like smartphones or action cameras. Continuous improvement in the processing power of small devices has enabled many advances in computational photography, where algorithms are used to compensate for the limited physical properties of the imaging system. Algorithms that aim at removing digital noise are a very recurrent topic of the computational photography literature. Most of them only use the information stored in the input image (they are usually referred to as ``single-frame'' algorithms), while others suggest the use of information stored in additional images (usually videos or bursts). These multi-frame ``denoisers'' usually make use of similarity measurements in order to combine the information of multiple pixels or patches~\cite{buades2009note}.&#10;&#10;A multi-frame denoising algorithm is at the heart of the HDR+ system. It was first officially introduced as a feature within the Google Camera App in the newly released Nexus 5 and Nexus 6 smartphones in fall 2014. In 2016, a full raw image processing pipeline with the same HDR+ moniker was presented in~\cite{hasinoff2016burst}. This method is the main subject of this article. The pipeline was embedded in smartphones such as the Google Nexus 6, 5X and 6P and first generation Google Pixel. Further algorithm refinements and optimizations were included in the Google Pixel 2 which was released in fall 2017. This device was the first to embed HDR+ in a dedicated Google-designed Image Processing Unit. The same raw burst alignment and merging technique was included in a new ``Night Sight'' mode that was launched alongside the Pixel 3A. The changes and optimizations of this mode were presented in a new publication in 2019~\cite{liba2019handheld}. As such, the complex HDR+ system is not only part of a detailed, frequently compared and quoted scientific publication, but also embedded into multiple recent mass-produced consumer electronics devices (which have received praise for their camera performance), making it a relevant subject of study.&#10;&#10;In this article, we will describe the core part of the system, a raw burst denoising algorithm (although it will not be our main focus, we will also mention the rest of the pipeline). This publication also comes with a publicly available HDR+ Python implementation (with a simplified finishing pipeline), along with an interactive demo on IPOL.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Algorithm Architecture, Mathematics, Image Denoising, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1910.11336" label="1910.11336">
-        <attvalues>
-          <attvalue for="0" value="Handheld Mobile Photography in Very Low Light" />
-          <attvalue for="1" value="  Taking photographs in low light using a mobile phone is challenging and&#10;rarely produces pleasing results. Aside from the physical limits imposed by&#10;read noise and photon shot noise, these cameras are typically handheld, have&#10;small apertures and sensors, use mass-produced analog electronics that cannot&#10;easily be cooled, and are commonly used to photograph subjects that move, like&#10;children and pets. In this paper we describe a system for capturing clean,&#10;sharp, colorful photographs in light as low as 0.3~lux, where human vision&#10;becomes monochromatic and indistinct. To permit handheld photography without&#10;flash illumination, we capture, align, and combine multiple frames. Our system&#10;employs &quot;motion metering&quot;, which uses an estimate of motion magnitudes (whether&#10;due to handshake or moving objects) to identify the number of frames and the&#10;per-frame exposure times that together minimize both noise and motion blur in a&#10;captured burst. We combine these frames using robust alignment and merging&#10;techniques that are specialized for high-noise imagery. To ensure accurate&#10;colors in such low light, we employ a learning-based auto white balancing&#10;algorithm. To prevent the photographs from looking like they were shot in&#10;daylight, we use tone mapping techniques inspired by illusionistic painting:&#10;increasing contrast, crushing shadows to black, and surrounding the scene with&#10;darkness. All of these processes are performed using the limited computational&#10;resources of a mobile device. Our system can be used by novice photographers to&#10;produce shareable pictures in a few seconds based on a single shutter press,&#10;even in environments so dim that humans cannot see clearly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.11802" label="1805.11802">
-        <attvalues>
-          <attvalue for="0" value="CRRN: Multi-Scale Guided Concurrent Reflection Removal Network" />
-          <attvalue for="1" value="  Removing the undesired reflections from images taken through the glass is of&#10;broad application to various computer vision tasks. Non-learning based methods&#10;utilize different handcrafted priors such as the separable sparse gradients&#10;caused by different levels of blurs, which often fail due to their limited&#10;description capability to the properties of real-world reflections. In this&#10;paper, we propose the Concurrent Reflection Removal Network (CRRN) to tackle&#10;this problem in a unified framework. Our proposed network integrates image&#10;appearance information and multi-scale gradient information with human&#10;perception inspired loss function, and is trained on a new dataset with 3250&#10;reflection images taken under diverse real-world scenes. Extensive experiments&#10;on a public benchmark dataset show that the proposed method performs favorably&#10;against state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Image Reflection Removal, Machine Learning, Computer Vision, Mathematics, Physics, Image Processing, Computer Science, Deep Learning Networks" />
-        </attvalues>
-      </node>
-      <node id="1905.12863" label="1905.12863">
-        <attvalues>
-          <attvalue for="0" value="CS-R-FCN: Cross-supervised Learning for Large-Scale Object Detection" />
-          <attvalue for="1" value="  Generic object detection is one of the most fundamental problems in computer&#10;vision, yet it is difficult to provide all the bounding-box-level annotations&#10;aiming at large-scale object detection for thousands of categories. In this&#10;paper, we present a novel cross-supervised learning pipeline for large-scale&#10;object detection, denoted as CS-R-FCN. First, we propose to utilize the data&#10;flow of image-level annotated images in the fully-supervised two-stage object&#10;detection framework, leading to cross-supervised learning combining&#10;bounding-box-level annotated data and image-level annotated data. Second, we&#10;introduce a semantic aggregation strategy utilizing the relationships among the&#10;cross-supervised categories to reduce the unreasonable mutual inhibition&#10;effects during the feature learning. Experimental results show that the&#10;proposed CS-R-FCN improves the mAP by a large margin compared to previous&#10;related works.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object detection is a crucial task in computer vision, aiming to localize and recognize the objects in prescribed images or videos. Benefited from the application of deep convolution networks, object detection has made rapid development in recent years. Large-scale object detection is a challenging project to handle thousands of categories simultaneously in a unified framework, which is still not adequately addressed. The breakthrough of large-scale object detection will be of great significance to the further application of computer vision in reality. &#10;&#10;Object detection in terms of large-scale categories is now facing many difficult problems. First of all, labeling bounding boxes for enormous numbers of categories is so consuming that there are no sufficient samples supporting large-scale fully-supervised object detection training. It is necessary to develop a cross-supervised learning strategy combining the images data with bounding-box-level annotations as well as image-level annotations, as Fig.1 displays. Furthermore, the difficulty of feature learning increases rapidly due to the confusion among categories for cross-supervised large-scale object detection. It is vital to reorganize the supervision information and optimize the learning strategy.&#10;&#10;To deal with the partial lackness of bounding-box-level annotations, LSDA \cite{hoffman2014lsda,tang2018visual} adopts the method of domain adaptation and knowledge transfer, which trains the classifier for all categories and transfers the classifier to object detector. However, related solutions based on transfer learning cannot be trained end-to-end, short of elegance and efficiency. Based on YOLO v2 \cite{redmon2016you, redmon2017yolo9000}, YOLO-9000 proposes an end-to-end one-stage object detection pipeline to jointly train the image-level annotated and bounding-box-level annotated images simultaneously. To further boost the recognition accuracy, YOLO-9000 proposes to train the classifier hierarchically and perform multiple softmax operations over all co-hyponyms. However, limited by the one-stage pipeline, the final performance is still to be improved. Due to the superiority in the accuracy of two-stage object detection, such as Fast R-CNN \cite{girshick2015fast}, Faster R-CNN \cite{ren2015faster}, R-FCN \cite{dai2016r} and R-FCN-3000 \cite{singh2018r}, it is valuable to realize more accurate cross-supervised learning based on the two-stage pipeline. Moreover, the hierarchical classification proposed by YOLO-9000 needs to add several intermediate categories nodes and extra space and time-consuming in the training and inference process, which can be further advanced.&#10;&#10;In this paper, we propose a novel cross-supervised learning pipeline for large-scale object detection, denoted as CS-R-FCN. Our main contributions are as follows. First, we propose to utilize the data flow of image-level annotated images in the fully-supervised two-stage object detector, leading to superior cross-supervised learning. Second, we introduce an efficient semantic aggregation strategy utilizing the relationships among cross-supervised categories to reduce the unreasonable mutual inhibition effects during the feature learning. Experimental results show that the proposed CS-R-FCN can obtain superior performance than previous related frameworks and achieve comparable accuracy with fully-supervised baseline models.&#10;&#10;The remainder part of this paper is organized as follows. Section 2 introduces the proposed method for large-scale cross-supervised object detection. Section 3 states the details of our implementation and analyzes the experimental results and comparison with former works. In Section 4, our work is concluded.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1407.5035" label="1407.5035">
-        <attvalues>
-          <attvalue for="0" value="LSDA: Large Scale Detection Through Adaptation" />
-          <attvalue for="1" value="  A major challenge in scaling object detection is the difficulty of obtaining&#10;labeled images for large numbers of categories. Recently, deep convolutional&#10;neural networks (CNNs) have emerged as clear winners on object classification&#10;benchmarks, in part due to training with 1.2M+ labeled classification images.&#10;Unfortunately, only a small fraction of those labels are available for the&#10;detection task. It is much cheaper and easier to collect large quantities of&#10;image-level labels from search engines than it is to collect detection data and&#10;label it with precise bounding boxes. In this paper, we propose Large Scale&#10;Detection through Adaptation (LSDA), an algorithm which learns the difference&#10;between the two tasks and transfers this knowledge to classifiers for&#10;categories without bounding box annotated data, turning them into detectors.&#10;Our method has the potential to enable detection for the tens of thousands of&#10;categories that lack bounding box annotations, yet have plenty of&#10;classification data. Evaluation on the ImageNet LSVRC-2013 detection challenge&#10;demonstrates the efficacy of our approach. This algorithm enables us to produce&#10;a &gt;7.6K detector by using available classification data from leaf nodes in the&#10;ImageNet tree. We additionally demonstrate how to modify our architecture to&#10;produce a fast detector (running at 2fps for the 7.6K detector). Models and&#10;software are available at&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.03145" label="1801.03145">
-        <attvalues>
-          <attvalue for="0" value="Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised&#10;  Object Detection" />
-          <attvalue for="1" value="  Deep CNN-based object detection systems have achieved remarkable success on&#10;several large-scale object detection benchmarks. However, training such&#10;detectors requires a large number of labeled bounding boxes, which are more&#10;difficult to obtain than image-level annotations. Previous work addresses this&#10;issue by transforming image-level classifiers into object detectors. This is&#10;done by modeling the differences between the two on categories with both&#10;image-level and bounding box annotations, and transferring this information to&#10;convert classifiers to detectors for categories without bounding box&#10;annotations. We improve this previous work by incorporating knowledge about&#10;object similarities from visual and semantic domains during the transfer&#10;process. The intuition behind our proposed method is that visually and&#10;semantically similar categories should exhibit more common transferable&#10;properties than dissimilar categories, e.g. a better detector would result by&#10;transforming the differences between a dog classifier and a dog detector onto&#10;the cat class, than would by transforming from the violin class. Experimental&#10;results on the challenging ILSVRC2013 detection dataset demonstrate that each&#10;of our proposed object similarity based knowledge transfer methods outperforms&#10;the baseline methods. We found strong evidence that visual similarity and&#10;semantic relatedness are complementary for the task, and when combined notably&#10;improve detection, achieving state-of-the-art detection performance in a&#10;semi-supervised setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1504.08083" label="1504.08083">
-        <attvalues>
-          <attvalue for="0" value="Fast R-CNN" />
-          <attvalue for="1" value="  This paper proposes a Fast Region-based Convolutional Network method (Fast&#10;R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently&#10;classify object proposals using deep convolutional networks. Compared to&#10;previous work, Fast R-CNN employs several innovations to improve training and&#10;testing speed while also increasing detection accuracy. Fast R-CNN trains the&#10;very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and&#10;achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains&#10;VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is&#10;implemented in Python and C++ (using Caffe) and is available under the&#10;open-source MIT License at https://github.com/rbgirshick/fast-rcnn.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.06409" label="1605.06409">
-        <attvalues>
-          <attvalue for="0" value="R-FCN: Object Detection via Region-based Fully Convolutional Networks" />
-          <attvalue for="1" value="  We present region-based, fully convolutional networks for accurate and&#10;efficient object detection. In contrast to previous region-based detectors such&#10;as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of&#10;times, our region-based detector is fully convolutional with almost all&#10;computation shared on the entire image. To achieve this goal, we propose&#10;position-sensitive score maps to address a dilemma between&#10;translation-invariance in image classification and translation-variance in&#10;object detection. Our method can thus naturally adopt fully convolutional image&#10;classifier backbones, such as the latest Residual Networks (ResNets), for&#10;object detection. We show competitive results on the PASCAL VOC datasets (e.g.,&#10;83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is&#10;achieved at a test-time speed of 170ms per image, 2.5-20x faster than the&#10;Faster R-CNN counterpart. Code is made publicly available at:&#10;https://github.com/daijifeng001/r-fcn&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.01802" label="1712.01802">
-        <attvalues>
-          <attvalue for="0" value="R-FCN-3000 at 30fps: Decoupling Detection and Classification" />
-          <attvalue for="1" value="  We present R-FCN-3000, a large-scale real-time object detector in which&#10;objectness detection and classification are decoupled. To obtain the detection&#10;score for an RoI, we multiply the objectness score with the fine-grained&#10;classification score. Our approach is a modification of the R-FCN architecture&#10;in which position-sensitive filters are shared across different object classes&#10;for performing localization. For fine-grained classification, these&#10;position-sensitive filters are not needed. R-FCN-3000 obtains an mAP of 34.9%&#10;on the ImageNet detection dataset and outperforms YOLO-9000 by 18% while&#10;processing 30 images per second. We also show that the objectness learned by&#10;R-FCN-3000 generalizes to novel classes and the performance increases with the&#10;number of training object classes - supporting the hypothesis that it is&#10;possible to learn a universal objectness detector. Code will be made available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.06753" label="2306.06753">
-        <attvalues>
-          <attvalue for="0" value="3rd Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation" />
-          <attvalue for="1" value="  In order to deal with the task of video panoptic segmentation in the wild, we&#10;propose a robust integrated video panoptic segmentation solution. In our&#10;solution, we regard the video panoptic segmentation task as a segmentation&#10;target querying task, represent both semantic and instance targets as a set of&#10;queries, and then combine these queries with video features extracted by neural&#10;networks to predict segmentation masks. In order to improve the learning&#10;accuracy and convergence speed of the solution, we add additional tasks of&#10;video semantic segmentation and video instance segmentation for joint training.&#10;In addition, we also add an additional image semantic segmentation model to&#10;further improve the performance of semantic classes. In addition, we also add&#10;some additional operations to improve the robustness of the model. Extensive&#10;experiments on the VIPSeg dataset show that the proposed solution achieves&#10;state-of-the-art performance with 50.04\% VPQ on the VIPSeg test set, which is&#10;3rd place on the video panoptic segmentation track of the PVUW Challenge 2023.&#10;" />
-          <attvalue for="2" value="&#10;Video panoptic segmentation (VPS)~\cite{kim2020video} aims at simultaneously predicting object classes, bounding boxes, masks, instance id associations, and semantic segmentation while assigning unique answers to each pixel in a video.&#10;&#10;In recent years, many VPS datasets have emerged, including Cityscapes-VPS~\cite{kim2020video}, KITTI-STEP~\cite{weber2021step}, VIPSeg~\cite{miao2022large}. For example, Cityscapes-VPS has 400 training videos and 100 validation videos. Each video consists of 30 consecutive frames, with every 5 frames paired with the ground truth annotations. For each video, all 30 frames are predicted, and only the 6 frames with ground truth are evaluated. And KITTI-STEP consists of 21 training sequences and 29 test sequences. In addition, VIPSeg provides 3,536 videos and 84,750 frames with pixel-level panoptic annotations, covering a wide range of real-world scenarios and categories, which is the first attempt to tackle the challenging video panoptic segmentation task in the wild by considering diverse scenarios. In this paper, we focus on the dataset VIPSeg covering a wide range of real-world scenarios and categories. Some visual examples as shown in Fig.~\ref{fig:dataset}.&#10;&#10;However, there still exist several challenges that hinder the development of VPS. First of all, there are many similar objects in the real application scenarios of VPS, where the accurate cross-frame tracking of these objects is very confusing, which leads to the different objects being wrongly matched as the same one. Secondly, stuff classes often occupy a large area, but it is difficult to maintain consistency in a large area, resulting in a lot of noise. In addition, many scenarios are very different, containing different objects and behaviors, which leads to many scenes not being included in the training dataset, thus bringing great challenges to the generalization of the algorithm. Actually, the above points are prominent problems, and there are many other difficulties to be solved in the task of VPS, which together make VPS still a challenging task. &#10;&#10;To deal with the task of VPS, lots of learning-based methods have been proposed in recent years, achieving impressive performance. For example, Video K-Net~\cite{li2022video} is built upon K-Net, a method that unifies image segmentation via a group of learnable kernels. In the method, learnable kernels from K-Net encode object appearances and contexts, and can naturally associate identical instances across video frames. Thus, video K-Net learns to simultaneously segment and track “things” and “stuff” in a video with simple kernel-based appearance modeling and cross-temporal kernel interaction. Tarvis~\cite{athar2023tarvis} proposes a novel, unified network architecture that can be applied to any task that requires segmenting a set of arbitrarily defined `targets' in the video. This approach is flexible with respect to how tasks define these targets, since it models the latter as abstract ‘queries’ which are then used to predict pixel-precise target masks. A single TarViS model can be trained jointly on a collection of datasets spanning different tasks and can hot-swap between tasks during inference without any task-specific retraining. Video-KMax~\cite{shin2023video} propose a unified approach for online and near-online VPS. The meta architecture of the Video-kMaX consists of two components: within-clip segmenter (for clip-level segmentation) and cross-clip associater (for association beyond clips). In addition, clip-kMaX (clip k-means mask transformer) and HiLA-MB (Hierarchical Location-Aware Memory Buffer) are used to instantiate the segmenter and associater, respectively. Tube-link~\cite{li2023tube}, a versatile framework that addresses multiple core tasks of video segmentation with a unified architecture, is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks. To enhance the modeling&#10;of cross-tube relationships, Tube-link introduces a way to perform tube-level linking via attention along the queries, and introduces temporal contrastive learning to instance-wise discriminative features for tube-level association. &#10;&#10;Inspired by these existing methods, we propose a robust integrated video panoptic segmentation solution. For the first challenge of VPS, we first introduce Tarvis to represent both semantic and instance targets as a set of queries, and then combine these queries with video features extracted by neural networks to predict segmentation masks, which ensures that the instance target (thing classes) is tracked accurately. In addition, we add additional tasks of video semantic segmentation and video instance segmentation for joint training to improve the learning accuracy and convergence speed of the solution. For the second challenge, we add an additional image semantic segmentation model ViT-Adapter~\cite{chen2022vision} trained on VSPW~\cite{miao2021vspw} to further improve the performance of semantic classes (stuff classes). For the third challenge, we also add some additional operations to improve the robustness of the model, including exponential moving average (EMA), model ensemble, and so on. Extensive experiments on the VIPSeg dataset show that the proposed solution achieves state-of-the-art performance with 50.04\% VPQ on the VIPSeg test set, which is 3rd place on the video panoptic segmentation track of the PVUW Challenge 2023.&#10;&#10;The main contributions of this paper include: 1) we enhance naive Tarvis with additional tasks of video semantic segmentation and video instance segmentation for joint training on the same data set VIPSeg, to improve the accuracy of thing classes. 2) we introduce ViT-Adapter trained on VSPW to further improve the performance of stuff classes. 3) we add extra operations to improve the robustness on the test set. 4) The proposed solution achieves state-of-the-art performance with 50.04\% VPQ on the VIPSeg test set, which is the 3rd place on the video panoptic segmentation track of the PVUW Challenge 2023.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Neural Network Training, Computer Vision, Mathematics, Artificial Intelligence, Image Processing, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2006.11339" label="2006.11339">
-        <attvalues>
-          <attvalue for="0" value="Video Panoptic Segmentation" />
-          <attvalue for="1" value="  Panoptic segmentation has become a new standard of visual recognition task by&#10;unifying previous semantic segmentation and instance segmentation tasks in&#10;concert. In this paper, we propose and explore a new video extension of this&#10;task, called video panoptic segmentation. The task requires generating&#10;consistent panoptic segmentation as well as an association of instance ids&#10;across video frames. To invigorate research on this new task, we present two&#10;types of video panoptic datasets. The first is a re-organization of the&#10;synthetic VIPER dataset into the video panoptic format to exploit its&#10;large-scale pixel annotations. The second is a temporal extension on the&#10;Cityscapes val. set, by providing new video panoptic annotations&#10;(Cityscapes-VPS). Moreover, we propose a novel video panoptic segmentation&#10;network (VPSNet) which jointly predicts object classes, bounding boxes, masks,&#10;instance id tracking, and semantic segmentation in video frames. To provide&#10;appropriate metrics for this task, we propose a video panoptic quality (VPQ)&#10;metric and evaluate our method and several other baselines. Experimental&#10;results demonstrate the effectiveness of the presented two datasets. We achieve&#10;state-of-the-art results in image PQ on Cityscapes and also in VPQ on&#10;Cityscapes-VPS and VIPER datasets. The datasets and code are made publicly&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.11859" label="2102.11859">
-        <attvalues>
-          <attvalue for="0" value="STEP: Segmenting and Tracking Every Pixel" />
-          <attvalue for="1" value="  The task of assigning semantic classes and track identities to every pixel in&#10;a video is called video panoptic segmentation. Our work is the first that&#10;targets this task in a real-world setting requiring dense interpretation in&#10;both spatial and temporal domains. As the ground-truth for this task is&#10;difficult and expensive to obtain, existing datasets are either constructed&#10;synthetically or only sparsely annotated within short video clips. To overcome&#10;this, we introduce a new benchmark encompassing two datasets, KITTI-STEP, and&#10;MOTChallenge-STEP. The datasets contain long video sequences, providing&#10;challenging examples and a test-bed for studying long-term pixel-precise&#10;segmentation and tracking under real-world conditions. We further propose a&#10;novel evaluation metric Segmentation and Tracking Quality (STQ) that fairly&#10;balances semantic and tracking aspects of this task and is more appropriate for&#10;evaluating sequences of arbitrary length. Finally, we provide several baselines&#10;to evaluate the status of existing methods on this new challenging dataset. We&#10;have made our datasets, metric, benchmark servers, and baselines publicly&#10;available, and hope this will inspire future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\PAR{Panoptic Segmentation.} The task of panoptic segmentation combines semantic segmentation~\cite{He2004CVPR} and instance segmentation~\cite{Hariharan2014ECCV}, and requires assigning a class label and instance ID to all pixels in an image.&#10;It is growing in popularity, thanks to the proposed benchmark~\cite{Kirillov19CVPR}. The corresponding panoptic quality (PQ) metric is the product of recognition quality and segmentation quality.&#10;&#10;\PAR{Video Semantic Segmentation.} Compared to image semantic segmentation~\cite{Everingham10IJCV,Long2015CVPR,Chen2015ICLR}, video semantic segmentation~\cite{Zhu2017CVPR} is less common in the literature, possibly due to the lack of benchmarks. Video semantic segmentation differs from our setting in that it does not require discriminating different instances and hence, also no explicit tracking.&#10;&#10;\PAR{Multi-Object Tracking.} The task of multi-object tracking (MOT) is to accurately track multiple objects by associating bounding boxes in videos~\cite{Breitenstein2009ICCV,Bergmann19ICCV,Peng2020ECCV}. Focusing on tracking, this task does not require any segmentation.&#10;The MOT-Challenge~\cite{dendorfer2020ijcv,Dendorfer2020Arxiv} and KITTI-MOT \cite{Geiger12CVPR} datasets are among the most popular benchmarks, and the tracking performance is measured by the CLEAR MOT metrics~\cite{Bernardin08JIVP} along with a set of track quality measures introduced in~\cite{Wu2006CVPR}.&#10;Recently, \cite{Luiten20IJCV} propose HOTA (Higher Order Tracking Accuracy), which explicitly balances tracking results attributed from accurate detection and association.&#10;&#10;\PAR{Video Instance Segmentation.} Combining instance segmentation and MOT, the goal of video instance segmentation (VIS)~\cite{Yang19ICCV} is to track instance masks across video frames. &#10;This task is also known as multi-object tracking and segmentation (MOTS)~\cite{Voigtlaender19CVPR_mots}.&#10;Yet, the respective work adapt different evaluation metrics, focusing on different perspectives of the task.&#10;The prominent Youtube-VIS dataset~\cite{Yang19ICCV} focuses on short sequences (3-6 seconds, every 5th frame annotated) in which objects are present from start to end. For that, ${AP}^{mask}$~\cite{Hariharan2014ECCV,Lin14ECCV} is adapted to videos by extending the IoU to the temporal domain (3D IoU). The focus of MOTS datasets, KITTI-MOTS, and MOTS-Challenge~\cite{Voigtlaender19CVPR_mots}, is on more general scenarios with appearing and disappearing objects within long videos. Therefore, MOTSA (Multi-Object Tracking and Segmentation Accuracy) is used, the mask-based variants of the CLEAR MOT metrics~\cite{Bernardin08JIVP}.&#10;The two MOTS datasets focus on challenging urban sequences like one encounters in an autonomous driving setting.&#10;In contrast to our task, these benchmarks do not consider non-instance regions and hence, do not require pixel-precise video understanding. &#10;We build on top of these datasets by adding {semantic segmentation} annotations to obtain \kitti and \challenge.&#10;&#10;\PAR{Video Panoptic Segmentation.} Recently, panoptic segmentation has also been extended to the video domain.&#10;Video Panoptic Segmentation (VPS)~\cite{Kim20CVPR} requires generating the instance tracking IDs along with panoptic segmentation results across video frames.&#10;Previous datasets~\cite{Kim20CVPR, Hurtado20CVPRW} and corresponding metrics fail to properly evaluate scenarios that require both short- and long-term segmentation and tracking.&#10;Specifically, due to expensive annotation efforts for VPS datasets, most of the reported experimental results in the literature~\cite{Kim20CVPR,Hurtado20CVPRW} are conducted on synthetic datasets~\cite{Richter2017ICCV,Gaidon2016CVPR}, making it hard to generalize to real-world applications~\cite{Hoffman2018ICML, tsai2018CVPR}.&#10;Exceptionally, \cite{Kim20CVPR} annotates Cityscapes video sequences~\cite{Cordts16CVPR} for 6 frames out of each short 30 frame clip (about 1.8 seconds), hence focusing on short-term tracks.&#10;Both benchmarks~\cite{Kim20CVPR, Hurtado20CVPRW} evaluate with different metrics, VPQ (Video Panoptic Quality) and PTQ (Panoptic Tracking Quality), respectively.&#10;VPQ, tailored for sparse annotations on short clips, was not designed for evaluating long-term tracking, while PTQ penalizes recovery from association errors and may produce hard to interpret negative evaluation results.&#10;Motivated by the issues above, we propose two datasets densely annotated in space and time, \kitti and \challenge requiring {long-term} segmentation and tracking. Additionally, we propose a novel \metric metric that gives equal importance to segmentation and tracking.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2204.04656" label="2204.04656">
-        <attvalues>
-          <attvalue for="0" value="Video K-Net: A Simple, Strong, and Unified Baseline for Video&#10;  Segmentation" />
-          <attvalue for="1" value="  This paper presents Video K-Net, a simple, strong, and unified framework for&#10;fully end-to-end video panoptic segmentation. The method is built upon K-Net, a&#10;method that unifies image segmentation via a group of learnable kernels. We&#10;observe that these learnable kernels from K-Net, which encode object&#10;appearances and contexts, can naturally associate identical instances across&#10;video frames. Motivated by this observation, Video K-Net learns to&#10;simultaneously segment and track &quot;things&quot; and &quot;stuff&quot; in a video with simple&#10;kernel-based appearance modeling and cross-temporal kernel interaction. Despite&#10;the simplicity, it achieves state-of-the-art video panoptic segmentation&#10;results on Citscapes-VPS, KITTI-STEP, and VIPSeg without bells and whistles. In&#10;particular, on KITTI-STEP, the simple method can boost almost 12\% relative&#10;improvements over previous methods. On VIPSeg, Video K-Net boosts almost 15\%&#10;relative improvements and results in 39.8 % VPQ. We also validate its&#10;generalization on video semantic segmentation, where we boost various baselines&#10;by 2\% on the VSPW dataset. Moreover, we extend K-Net into clip-level video&#10;framework for video instance segmentation, where we obtain 40.5% mAP for&#10;ResNet50 backbone and 54.1% mAP for Swin-base on YouTube-2019 validation set.&#10;We hope this simple, yet effective method can serve as a new, flexible baseline&#10;in unified video segmentation design. Both code and models are released at&#10;https://github.com/lxtGH/Video-K-Net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Panoptic Segmentation. &#10;The goal of this task is to unify the semantic segmentation and instance segmentation into one framework with a single metric named Panoptic Quality (PQ)~\cite{kirillov2019panoptic}. Since then, lots of works~\cite{xiong2019upsnet,kirillov2019panopticfpn,li2020panopticFCN,chen2020banet,li2018learning,porzi2019seamless,yang2019sognet,Wu2020AutoPanopticCM,cheng2020panoptic,axialDeeplab} have been proposed to solve this task with various approaches. However, most methods separate thing and stuff segmentation as individual tasks. Recently, starting from DETR~\cite{detr}, query based approaches~\cite{wang2020maxDeeplab,cheng2021maskformer,zhang2021knet,panopticpartformer} unify both things and stuff segmentation as a set prediction problem. In particular, K-Net~\cite{zhang2021knet} unifies segmentation tasks in the view of dot product between kernel and feature maps.&#10;&#10;Video Semantic/Instance Segmentation. Video Semantic Segmentation, a direct extension of semantic segmentation to the video scenario, requires predicting a semantic label to every pixel in each video frame. Several approaches~\cite{shelhamer2016clockwork,DFF,hu2020tdnet,li2021improving} have been proposed in the literature mainly to model the temporal association such as optical flow warping or attention. Recently, the VSPW dataset~\cite{miao2021vspw} is proposed to evaluate large scale video semantic segmentation in the wild. Video Instance Segmentation (VIS)~\cite{vis_dataset} extends instance segmentation into video, and it aims to simultaneously classify, segment and track object instances in a given video sequence. Several methods~\cite{mask_pro_vis,lin2021video,zhou2022transvod} are proposed to link instance-wised feature in video. Recently, several works~\cite{VIS_TR,hwang2021video} extend DETR into VIS. Multi-Object Tracking and Segmentation (MOTS) task~\cite{voigtlaender2019mots} is proposed to evaluate Multi-Object Tracking along with instance segmentation. However, both VIS and MOTS have limited scale distribution and much fewer objects in the scene. &#10;&#10;Video Panoptic Segmentation. Video Panoptic Segmentation (VPS)~\cite{kim2020vps,hurtado2020mopt} requires generating the instance tracking IDs along with panoptic segmentation results across video clips. Kim~\etal~\cite{kim2020vps} use Cityscapes video sequences for 6 frames out of each short 30 frame clip and mainly focus on short-term tracks. They proposed Video Panoptic Quality (VPQ) for evaluation. Several works~\cite{kim2020vps,woo2021learning_associate_vps} are proposed to solve this task respectively. VIP-Deeplab~\cite{ViPDeepLab} extends the Panoptic-Deeplab~\cite{cheng2020panoptic} with next frame center map prediction for DVPS task~\cite{yuan2021polyphonicformer}. Then they use such predicted maps for offline tracking. Since Cityscapes VPS only contains short term clips, to allow long term VPS, STEP dataset~\cite{STEP} is proposed. They propose a new metric named Segmentation and Tracking Quality (STQ) that decouples the segmentation and tracking error. They also provide several baselines for reference. Our Video K-Net is verified to work well on both short- and long-term videos.&#10;&#10;Object Tracking. One of the major tasks in VPS is object tracking. Many studies adopt the tracking-by-detection paradigm~\cite{bewley2016simple,leal2016learning,schulter2017deep,sharma2018beyond,xu2019spatial,zhu2018online,porzi2020learning} and they divide the task into two subtasks where an object detector finds all objects and then a tracking algorithm is employed to associate them. There are also several works~\cite{zhang2018integrated,bergmann2019tracking,peng2020chained,JDE,qdtrack,zhou2020tracking} that detect and track objects at the same time. There are also tracking methods using object queries. For VPS, ViP-DeepLab~\cite{ViPDeepLab} performs object tracking by clustering all instance pixels. By contrast, our method directly extends and links kernels into kernel association embeddings and avoids such complex post-grouping steps. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2301.02657" label="2301.02657">
-        <attvalues>
-          <attvalue for="0" value="TarViS: A Unified Approach for Target-based Video Segmentation" />
-          <attvalue for="1" value="  The general domain of video segmentation is currently fragmented into&#10;different tasks spanning multiple benchmarks. Despite rapid progress in the&#10;state-of-the-art, current methods are overwhelmingly task-specific and cannot&#10;conceptually generalize to other tasks. Inspired by recent approaches with&#10;multi-task capability, we propose TarViS: a novel, unified network architecture&#10;that can be applied to any task that requires segmenting a set of arbitrarily&#10;defined 'targets' in video. Our approach is flexible with respect to how tasks&#10;define these targets, since it models the latter as abstract 'queries' which&#10;are then used to predict pixel-precise target masks. A single TarViS model can&#10;be trained jointly on a collection of datasets spanning different tasks, and&#10;can hot-swap between tasks during inference without any task-specific&#10;retraining. To demonstrate its effectiveness, we apply TarViS to four different&#10;tasks, namely Video Instance Segmentation (VIS), Video Panoptic Segmentation&#10;(VPS), Video Object Segmentation (VOS) and Point Exemplar-guided Tracking&#10;(PET). Our unified, jointly trained model achieves state-of-the-art performance&#10;on 5/7 benchmarks spanning these four tasks, and competitive performance on the&#10;remaining two. Code and model weights are available at:&#10;https://github.com/Ali2500/TarViS&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.04694" label="2304.04694">
-        <attvalues>
-          <attvalue for="0" value="Video-kMaX: A Simple Unified Approach for Online and Near-Online Video&#10;  Panoptic Segmentation" />
-          <attvalue for="1" value="  Video Panoptic Segmentation (VPS) aims to achieve comprehensive pixel-level&#10;scene understanding by segmenting all pixels and associating objects in a&#10;video. Current solutions can be categorized into online and near-online&#10;approaches. Evolving over the time, each category has its own specialized&#10;designs, making it nontrivial to adapt models between different categories. To&#10;alleviate the discrepancy, in this work, we propose a unified approach for&#10;online and near-online VPS. The meta architecture of the proposed Video-kMaX&#10;consists of two components: within clip segmenter (for clip-level segmentation)&#10;and cross-clip associater (for association beyond clips). We propose clip-kMaX&#10;(clip k-means mask transformer) and HiLA-MB (Hierarchical Location-Aware Memory&#10;Buffer) to instantiate the segmenter and associater, respectively. Our general&#10;formulation includes the online scenario as a special case by adopting clip&#10;length of one. Without bells and whistles, Video-kMaX sets a new&#10;state-of-the-art on KITTI-STEP and VIPSeg for video panoptic segmentation, and&#10;VSPW for video semantic segmentation. Code will be made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;{Video Panoptic Segmentation (VPS)}~\quad&#10;Video Panoptic Segmentation~\cite{kim2020video} aims to unify video semantic~\cite{zhu2017deep,gadde2017semantic,jain2019accel, miao2021vspw} and video instance~\cite{voigtlaender2019mots,Yang19ICCV, bertasius2020classifying,hwang2021video} segmentation.&#10;Numerous efforts have been to transform image panoptic segmentation models~\cite{kirillov2018panoptic,kirillov2019panoptic,xiong2019upsnet,yang2019deeperlab,li2018attention,cheng2019panopticworkshop,wang2020axial,wang2021max,li2021fully} to the video domain. &#10;Among them, the online method VPSNet~\cite{kim2020video} adopts task-specific prediction heads from instance segmentation~\cite{he2017mask}, semantic segmentation~\cite{dai2017deformable}, and tracking~\cite{Yang19ICCV}, and jointly trains them to obtain panoptic video results. Similarly, the near-online method ViP-DeepLab~\cite{vip_deeplab} adds a next-frame instance segmentation head on top of Panoptic-DeepLab~\cite{cheng2020panoptic} that provides generic image panoptic segmentation with dual-ASPP~\cite{chen2017deeplab} semantic segmentation module and dual-decoder~\cite{chen2018encoder} based instance segmentation.&#10;More recent works~\cite{kim2022tubeformer, zhou2022slot, li2022video} identifies the limitations of previous methods that require multiple separate networks and complex post-processing (\eg, NMS, fusion for tracking).&#10;To address the issues, they design a transformer architecture~\cite{carion2020end} for end-to-end video panoptic segmentation.&#10;However, all these methods have two fundamental issues.&#10;First, they require specific designs for either online or near-online scenario, \eg, another association module~\cite{kim2020video,li2022video}, temporal consistency loss~\cite{kim2022tubeformer,li2022video}, or clip-segmentation module~\cite{vip_deeplab,kim2022tubeformer}.&#10;Second, the models could only deal with short-term association (\ie, either neighboring frames or a clip).&#10;In this regard, we propose a simple unified online and near-online video panoptic segmentation model for long-term association without adding extra scenario-specific designs. &#10;&#10;{Memory Module for Long-Term Tracking}~\quad&#10;Object queries from the Transformer decoder~\cite{carion2020end} have been used to track objects in multi-object tracking~\cite{sun2020transtrack,meinhardt2022trackformer,cai2022memot,zeng2022motr,zhao2022tracking}, video instance segmentation~\cite{hwang2021video,wu2022efficient,cheng2021mask2former,wu2022seqformer,wu2022defense,huang2022minvis,heo2022vita, heo2022generalized, UNINEXT}, and video panoptic segmentation~\cite{kim2022tubeformer,li2022video,zhou2022slot}.&#10;Some of them exploit queries for the short-term association~\cite{sun2020transtrack,meinhardt2022trackformer}, while the others for the long-term association by additionally exploiting the memory buffer~\cite{zeng2022motr,wu2022defense, heo2022generalized}.&#10;Particularly,&#10;MOTR~\cite{zeng2022motr} proposes a set of track queries to model the tracked objects in the entire video.&#10;MeMOT~\cite{cai2022memot} develops a spatio-temporal memory that stores a long range states of all tracked objects.&#10;MaskTrack R-CNN~\cite{Yang19ICCV} employs a memory module to track detected objects. To make the association robust to challenging scenarios, such as heavy occlusion, IDOL~\cite{wu2022defense} proposes a temporally weighted softmax score for object matching. &#10;Along the same direction, we specialize the memory buffer approach for both online and near-online video panoptic segmentation models, and additionally develop an efficient hierarchical matching scheme.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2205.08534" label="2205.08534">
-        <attvalues>
-          <attvalue for="0" value="Vision Transformer Adapter for Dense Predictions" />
-          <attvalue for="1" value="  This work investigates a simple yet powerful dense prediction task adapter&#10;for Vision Transformer (ViT). Unlike recently advanced variants that&#10;incorporate vision-specific inductive biases into their architectures, the&#10;plain ViT suffers inferior performance on dense predictions due to weak prior&#10;assumptions. To address this issue, we propose the ViT-Adapter, which allows&#10;plain ViT to achieve comparable performance to vision-specific transformers.&#10;Specifically, the backbone in our framework is a plain ViT that can learn&#10;powerful representations from large-scale multi-modal data. When transferring&#10;to downstream tasks, a pre-training-free adapter is used to introduce the&#10;image-related inductive biases into the model, making it suitable for these&#10;tasks. We verify ViT-Adapter on multiple dense prediction tasks, including&#10;object detection, instance segmentation, and semantic segmentation. Notably,&#10;without using extra detection data, our ViT-Adapter-L yields state-of-the-art&#10;60.9 box AP and 53.0 mask AP on COCO test-dev. We hope that the ViT-Adapter&#10;could serve as an alternative for vision-specific transformers and facilitate&#10;future research. The code and models will be released at&#10;https://github.com/czczup/ViT-Adapter.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.06429" label="2002.06429">
-        <attvalues>
-          <attvalue for="0" value="An End-to-End Framework for Unsupervised Pose Estimation of Occluded&#10;  Pedestrians" />
-          <attvalue for="1" value="  Pose estimation in the wild is a challenging problem, particularly in&#10;situations of (i) occlusions of varying degrees and (ii) crowded outdoor&#10;scenes. Most of the existing studies of pose estimation did not report the&#10;performance in similar situations. Moreover, pose annotations for occluded&#10;parts of human figures have not been provided in any of the relevant standard&#10;datasets which in turn creates further difficulties to the required studies for&#10;pose estimation of the entire figure of occluded humans. Well known pedestrian&#10;detection datasets such as CityPersons contains samples of outdoor scenes but&#10;it does not include pose annotations. Here, we propose a novel multi-task&#10;framework for end-to-end training towards the entire pose estimation of&#10;pedestrians including in situations of any kind of occlusion. To tackle this&#10;problem for training the network, we make use of a pose estimation dataset,&#10;MS-COCO, and employ unsupervised adversarial instance-level domain adaptation&#10;for estimating the entire pose of occluded pedestrians. The experimental&#10;studies show that the proposed framework outperforms the SOTA results for pose&#10;estimation, instance segmentation and pedestrian detection in cases of heavy&#10;occlusions (HO) and reasonable + heavy occlusions (R + HO) on the two benchmark&#10;datasets.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pose estimation of pedestrians is an active area of research with several potential applications.&#10;Research studies in this area have recently achieved significant progress due to the development of different sophisticated deep learning architectures \cite{kocabas2018multiposenet, insafutdinov2016deeper, sun2019deep, xiao2018simple, zhang2019semantics} and several large-scale datasets such as \cite{andriluka20142d, lin2014microsoft, li2019crowdpose}.&#10;However, a majority of the existing related studies did not consider estimating the pose of possible occluded portions of the pedestrians.&#10;It is not an uncommon situation when some major portions of a pedestrian figure remain occluded in the captured view and thus their estimated information about the pose may remain ineffective unless the pose of such occluded parts are explored. &#10;In fact, studies of pedestrian detection in occluded scenarios has attracted attentions \cite{Zhou2017, Zhang2018, zhang2018occlusion} during the recent years although the scope of these studies are only limited to the detection of pedestrian figures including their occluded portions while excluding estimation of the corresponding pose.&#10;To the best of our knowledge, pose estimation of the occluded portions of pedestrians was considered only in \cite{kishore2019cluenet}.&#10;On the contrary, Ye et al. \cite{ye2018occlusion} considered hand pose estimation or Reddy et al. \cite{Reddy_2019_CVPR} studied keypoint prediction of 3D shapes of vehicles both for occluded scenarios.&#10;&#10;In this article, we propose a novel end-to-end framework for estimating the entire pose of occluded pedestrians in an unsupervised manner.&#10;This framework tackles the unfavourable situation of the non-availability of two types of training data, such as (i) a pedestrian dataset with pose annotations and (ii) any other dataset with pose annotations for occluded portions of the human figures. &#10;ClueNet \cite{kishore2019cluenet}, a recently introduced two stage framework, attempted to solve this problem but the same fails to produce desirable results in cases of person-to-person occlusions. On the other hand, the proposed network is equally efficient for every type of occlusion. &#10;&#10;To alleviate the problem of non-availability of required pose annotations for pedestrians from training datasets such as CityPersons \cite{zhang2017citypersons}, we make use of relevant samples of MS-COCO dataset \cite{lin2014microsoft} for pose supervision during the training phase and employ instance-level domain adaptation strategy for this purpose.&#10;The human instances from the pose estimation dataset act as the source domain, whereas the ones from the pedestrian detection dataset act as the target domain for domain adaptation.&#10;The proposed framework is divided into two parts, domain specific feature extraction and domain invariant pose estimation, which are described in details in the following sections.&#10;&#10;In summary, contributions of the present study include &#10;\begin{enumerate} [noitemsep,nolistsep, wide]&#10; \item [(1)] A novel end-to-end framework for entire pose estimation of occluded pedestrians in an unsupervised manner. To the best of our knowledge, this is the first such an attempt. &#10; \item [(2)] Existing pedestrian pose estimation study \cite{kishore2019cluenet} did not consider person-to-person occlusion. However, the present solution is capable of handling all types of occlusions.&#10; \item [(3)] The proposed strategy has improved SOTA with respect to pose estimation, instance segmentation and pedestrian detection in cases of heavy occlusions (HO) and reasonable+heavy occlusions (R+HO) on MS-COCO and CityPersons dataset.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Pose Estimation, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1807.04067" label="1807.04067">
-        <attvalues>
-          <attvalue for="0" value="MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual&#10;  Network" />
-          <attvalue for="1" value="  In this paper, we present MultiPoseNet, a novel bottom-up multi-person pose&#10;estimation architecture that combines a multi-task model with a novel&#10;assignment method. MultiPoseNet can jointly handle person detection, keypoint&#10;detection, person segmentation and pose estimation problems. The novel&#10;assignment method is implemented by the Pose Residual Network (PRN) which&#10;receives keypoint and person detections, and produces accurate poses by&#10;assigning keypoints to person instances. On the COCO keypoints dataset, our&#10;pose estimation method outperforms all previous bottom-up methods both in&#10;accuracy (+4-point mAP over previous best result) and speed; it also performs&#10;on par with the best top-down methods while being at least 4x faster. Our&#10;method is the fastest real time system with 23 frames/sec. Source code is&#10;available at: https://github.com/mkocabas/pose-residual-network&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10872" label="1711.10872">
-        <attvalues>
-          <attvalue for="0" value="Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density&#10;  Network" />
-          <attvalue for="1" value="  Learning and predicting the pose parameters of a 3D hand model given an&#10;image, such as locations of hand joints, is challenging due to large viewpoint&#10;changes and articulations, and severe self-occlusions exhibited particularly in&#10;egocentric views. Both feature learning and prediction modeling have been&#10;investigated to tackle the problem. Though effective, most existing&#10;discriminative methods yield a single deterministic estimation of target poses.&#10;Due to their single-value mapping intrinsic, they fail to adequately handle&#10;self-occlusion problems, where occluded joints present multiple modes. In this&#10;paper, we tackle the self-occlusion issue and provide a complete description of&#10;observed poses given an input depth image by a novel method called hierarchical&#10;mixture density networks (HMDN). The proposed method leverages the&#10;state-of-the-art hand pose estimators based on Convolutional Neural Networks to&#10;facilitate feature learning, while it models the multiple modes in a two-level&#10;hierarchy to reconcile single-valued and multi-valued mapping in its output.&#10;The whole framework with a mixture of two differentiable density functions is&#10;naturally end-to-end trainable. In the experiments, HMDN produces interpretable&#10;and diverse candidate samples, and significantly outperforms the&#10;state-of-the-art methods on two benchmarks with occlusions, and performs&#10;comparably on another benchmark free of occlusions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.05693" label="1702.05693">
-        <attvalues>
-          <attvalue for="0" value="CityPersons: A Diverse Dataset for Pedestrian Detection" />
-          <attvalue for="1" value="  Convnets have enabled significant progress in pedestrian detection recently,&#10;but there are still open questions regarding suitable architectures and&#10;training data. We revisit CNN design and point out key adaptations, enabling&#10;plain FasterRCNN to obtain state-of-the-art results on the Caltech dataset.&#10;  To achieve further improvement from more and better data, we introduce&#10;CityPersons, a new set of person annotations on top of the Cityscapes dataset.&#10;The diversity of CityPersons allows us for the first time to train one single&#10;CNN model that generalizes well over multiple benchmarks. Moreover, with&#10;additional training with CityPersons, we obtain top results using FasterRCNN on&#10;Caltech, improving especially for more difficult cases (heavy occlusion and&#10;small scale) and providing higher localization quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.03771" label="2101.03771">
-        <attvalues>
-          <attvalue for="0" value="Investigating the Vision Transformer Model for Image Retrieval Tasks" />
-          <attvalue for="1" value="  This paper introduces a plug-and-play descriptor that can be effectively&#10;adopted for image retrieval tasks without prior initialization or preparation.&#10;The description method utilizes the recently proposed Vision Transformer&#10;network while it does not require any training data to adjust parameters. In&#10;image retrieval tasks, the use of Handcrafted global and local descriptors has&#10;been very successfully replaced, over the last years, by the Convolutional&#10;Neural Networks (CNN)-based methods. However, the experimental evaluation&#10;conducted in this paper on several benchmarking datasets against 36&#10;state-of-the-art descriptors from the literature demonstrates that a neural&#10;network that contains no convolutional layer, such as Vision Transformer, can&#10;shape a global descriptor and achieve competitive results. As fine-tuning is&#10;not required, the presented methodology's low complexity encourages adoption of&#10;the architecture as an image retrieval baseline model, replacing the&#10;traditional and well adopted CNN-based approaches and inaugurating a new era in&#10;image retrieval approaches.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Content-based image retrieval (CBIR) is one of the fundamental scientific challenges that the multimedia, computer vision, and robotics communities have thoroughly researched for decades. Three eras that vary in how researchers export the different features that define an image's visual content characterize content-based image retrieval \cite{zheng2018sift}. The literature strongly focuses on global low-level descriptors during the early years, shaping the first CBIR era. A single vector was used to represent different aspects of an image, such as color, shape, and texture. In the sequel, the community started to concentrate on representations focused on the extraction and use of local features. From 2003 onwards, new approaches adopt local image descriptors to search for salient image patches and points-of-interest, such as edges, corners, and blobs. Since 2015, image retrieval research strongly relies on approaches focused on Deep Learning (DL) and Convolutional Neural Networks (CNN). Although DL-based techniques require a large amount of data for training, the pre-trained networks have been demonstrated in several studies to be particularly useful as feature extractors and achieve high retrieval performance.&#10;&#10;Meanwhile, in natural language processing (NLP), the self-attention-based architecture, particularly Transformers, is now considered as the new standard \cite{vaswani2017attention}. The Transformer is a type of deep-neural network mainly based on self-attention mechanism \cite{wang2018non}. Recently, researchers have expanded transformers for computer vision tasks inspired by the influence of the Transformer in NLP \cite{han2020survey,locatello2020object,hu2018relation}. For example, the authors in \cite{chen2020generative} have trained a sequence transformer to auto-regressively predict pixels and achieve competitive results with CNNs on an image classification task. Recently, the authors in \cite{dosovitskiy2020image} have attempted to apply a typical transformer directly to images, with the fewest adjustments possible shaping an image recognition network. Vision Transformer network (ViT) provides better performance than a traditional convolutional neural network in image recognition tasks. &#10;&#10;This paper builds upon the Vision Transformer architecture to shape a global descriptor for image retrieval tasks. Following the procedure that the vast majority of the deep-learning-based image retrieval models adopt, we discard the fully connected layers of the Transformer's network and use the last layer as a feature to describe the visual content of the image. Best we can tell, this is the first attempt to evaluating the performance of the transformers on image retrieval tasks. &#10;&#10;The remainder of this paper is structured as follows. Section II briefly shows how the vision transformer's architecture operates and presents the process of shaping the image retrieval descriptor. Section III offers a thorough experimental evaluation of the descriptor complemented by a discussion of results. Finally, Section IV concludes the paper, and discusses potential opportunities for future work.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Neural Network Architectures, Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Computer Science, Image Retrieval" />
-        </attvalues>
-      </node>
-      <node id="1608.01807" label="1608.01807">
-        <attvalues>
-          <attvalue for="0" value="SIFT Meets CNN: A Decade Survey of Instance Retrieval" />
-          <attvalue for="1" value="  In the early days, content-based image retrieval (CBIR) was studied with&#10;global features. Since 2003, image retrieval based on local descriptors (de&#10;facto SIFT) has been extensively studied for over a decade due to the advantage&#10;of SIFT in dealing with image transformations. Recently, image representations&#10;based on the convolutional neural network (CNN) have attracted increasing&#10;interest in the community and demonstrated impressive performance. Given this&#10;time of rapid evolution, this article provides a comprehensive survey of&#10;instance retrieval over the last decade. Two broad categories, SIFT-based and&#10;CNN-based methods, are presented. For the former, according to the codebook&#10;size, we organize the literature into using large/medium-sized/small codebooks.&#10;For the latter, we discuss three lines of methods, i.e., using pre-trained or&#10;fine-tuned CNN models, and hybrid methods. The first two perform a single-pass&#10;of an image to the network, while the last category employs a patch-based&#10;feature extraction scheme. This survey presents milestones in modern instance&#10;retrieval, reviews a broad selection of previous works in different categories,&#10;and provides insights on the connection between SIFT and CNN-based methods.&#10;After analyzing and comparing retrieval performance of different categories on&#10;several datasets, we discuss promising directions towards generic and&#10;specialized instance retrieval.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07971" label="1711.07971">
-        <attvalues>
-          <attvalue for="0" value="Non-local Neural Networks" />
-          <attvalue for="1" value="  Both convolutional and recurrent operations are building blocks that process&#10;one local neighborhood at a time. In this paper, we present non-local&#10;operations as a generic family of building blocks for capturing long-range&#10;dependencies. Inspired by the classical non-local means method in computer&#10;vision, our non-local operation computes the response at a position as a&#10;weighted sum of the features at all positions. This building block can be&#10;plugged into many computer vision architectures. On the task of video&#10;classification, even without any bells and whistles, our non-local models can&#10;compete or outperform current competition winners on both Kinetics and Charades&#10;datasets. In static image recognition, our non-local models improve object&#10;detection/segmentation and pose estimation on the COCO suite of tasks. Code is&#10;available at https://github.com/facebookresearch/video-nonlocal-net .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.12556" label="2012.12556">
-        <attvalues>
-          <attvalue for="0" value="A Survey on Visual Transformer" />
-          <attvalue for="1" value="  Transformer, first applied to the field of natural language processing, is a&#10;type of deep neural network mainly based on the self-attention mechanism.&#10;Thanks to its strong representation capabilities, researchers are looking at&#10;ways to apply transformer to computer vision tasks. In a variety of visual&#10;benchmarks, transformer-based models perform similar to or better than other&#10;types of networks such as convolutional and recurrent neural networks. Given&#10;its high performance and less need for vision-specific inductive bias,&#10;transformer is receiving more and more attention from the computer vision&#10;community. In this paper, we review these vision transformer models by&#10;categorizing them in different tasks and analyzing their advantages and&#10;disadvantages. The main categories we explore include the backbone network,&#10;high/mid-level vision, low-level vision, and video processing. We also include&#10;efficient transformer methods for pushing transformer into real device-based&#10;applications. Furthermore, we also take a brief look at the self-attention&#10;mechanism in computer vision, as it is the base component in transformer.&#10;Toward the end of this paper, we discuss the challenges and provide several&#10;further research directions for vision transformers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.15055" label="2006.15055">
-        <attvalues>
-          <attvalue for="0" value="Object-Centric Learning with Slot Attention" />
-          <attvalue for="1" value="  Learning object-centric representations of complex scenes is a promising step&#10;towards enabling efficient abstract reasoning from low-level perceptual&#10;features. Yet, most deep learning approaches learn distributed representations&#10;that do not capture the compositional properties of natural scenes. In this&#10;paper, we present the Slot Attention module, an architectural component that&#10;interfaces with perceptual representations such as the output of a&#10;convolutional neural network and produces a set of task-dependent abstract&#10;representations which we call slots. These slots are exchangeable and can bind&#10;to any object in the input by specializing through a competitive procedure over&#10;multiple rounds of attention. We empirically demonstrate that Slot Attention&#10;can extract object-centric representations that enable generalization to unseen&#10;compositions when trained on unsupervised object discovery and supervised&#10;property prediction tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.11575" label="1711.11575">
-        <attvalues>
-          <attvalue for="0" value="Relation Networks for Object Detection" />
-          <attvalue for="1" value="  Although it is well believed for years that modeling relations between&#10;objects would help object recognition, there has not been evidence that the&#10;idea is working in the deep learning era. All state-of-the-art object detection&#10;systems still rely on recognizing object instances individually, without&#10;exploiting their relations during learning.&#10;  This work proposes an object relation module. It processes a set of objects&#10;simultaneously through interaction between their appearance feature and&#10;geometry, thus allowing modeling of their relations. It is lightweight and&#10;in-place. It does not require additional supervision and is easy to embed in&#10;existing networks. It is shown effective on improving object recognition and&#10;duplicate removal steps in the modern object detection pipeline. It verifies&#10;the efficacy of modeling object relations in CNN based detection. It gives rise&#10;to the first fully end-to-end object detector.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.02603" label="2103.02603">
-        <attvalues>
-          <attvalue for="0" value="Towards Open World Object Detection" />
-          <attvalue for="1" value="  Humans have a natural instinct to identify unknown object instances in their&#10;environments. The intrinsic curiosity about these unknown instances aids in&#10;learning about them, when the corresponding knowledge is eventually available.&#10;This motivates us to propose a novel computer vision problem called: `Open&#10;World Object Detection', where a model is tasked to: 1) identify objects that&#10;have not been introduced to it as `unknown', without explicit supervision to do&#10;so, and 2) incrementally learn these identified unknown categories without&#10;forgetting previously learned classes, when the corresponding labels are&#10;progressively received. We formulate the problem, introduce a strong evaluation&#10;protocol and provide a novel solution, which we call ORE: Open World Object&#10;Detector, based on contrastive clustering and energy based unknown&#10;identification. Our experimental evaluation and ablation studies analyze the&#10;efficacy of ORE in achieving Open World objectives. As an interesting&#10;by-product, we find that identifying and characterizing unknown instances helps&#10;to reduce confusion in an incremental object detection setting, where we&#10;achieve state-of-the-art performance, with no extra methodological effort. We&#10;hope that our work will attract further research into this newly identified,&#10;yet crucial research direction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Open Set Classification: The open set setting considers knowledge acquired through training set to be incomplete, thus new unknown classes can be encountered during testing.&#10;Scheirer \etal \cite{scheirer2013toward} developed open set classifiers in a one-vs-rest setting to balance the performance and the risk of labeling a sample far from the known training examples (termed as open space risk). Follow up works \cite{jain2014multi, scheirer2014probability} extended the open set framework to multi-class classifier setting with probabilistic models to account for the fading away classifier confidences in case of unknown classes.&#10;&#10;Bendale and Boult \cite{bendale2016towards} identified unknowns in the feature space of deep networks and used a Weibull distribution to estimate the set risk (called OpenMax classifier). A generative version of OpenMax was proposed in \cite{ge2017generative} by synthesizing novel class images. Liu \etal \cite{liu2019large} considered a long-tailed recognition setting where majority, minority and unknown classes coexist. They developed a metric learning framework identify unseen classes as unknown. In similar spirit, several dedicated approaches target on detecting the out of distribution samples \cite{liang2018enhancing} or novelties \cite{pidhorskyi2018generative}. Recently, self-supervised learning \cite{Perera_2020_CVPR} and unsupervised learning with reconstruction \cite{Yoshihashi_2019_CVPR} have been explored for open set recognition. However, while these works can recognize unknown instances, they cannot dynamically update themselves in an incremental fashion over multiple training episodes. Further, our energy based unknown detection approach has not been explored before.&#10;&#10;Open World Classification: \cite{bendale2015towards} first proposed the open world setting for image recognition. Instead of a static classifier trained on a fixed set of classes, they proposed a more flexible setting where knowns and unknowns both coexist. The model can recognize both types of objects and adaptively improve itself when new labels for unknown are provided. Their approach extends Nearest Class Mean classifier to operate in an open world setting by re-calibrating the class probabilities to balance open space risk. \cite{pernici2018memory} studies open world face identity learning while \cite{xu2019open} proposed to use an exemplar set of seen classes to match them against a new sample, and rejects it in case of a low match with all previously known classes. &#10;However, they don’t test on image classification benchmarks and study product classification in e-commerce applications.&#10;&#10;Open Set Detection: Dhamija \etal \cite{dhamija2020overlooked} formally studied the impact of open set setting on popular object detectors. They noticed that the state of the art object detectors often classify unknown classes with high confidence to seen classes. This is despite the fact that the detectors are explicitly trained with a background class \cite{ren2016faster, girshick2015fast, liu2016ssd} and/or apply one-vs-rest classifiers to model each class \cite{girshick2014rich, lin2017focal}. A dedicated body of work \cite{miller2018dropout, miller2019evaluating, hall2020probabilistic} focuses on developing measures of (spatial and semantic) uncertainty in object detectors to reject unknown classes. E.g., \cite{miller2018dropout,miller2019evaluating} uses Monte Carlo Dropout \cite{gal2016dropout} sampling in a SSD detector to obtain uncertainty estimates. These methods, however, cannot incrementally adapt their knowledge in a dynamic world.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2405.14017" label="2405.14017">
-        <attvalues>
-          <attvalue for="0" value="MagicPose4D: Crafting Articulated Models with Appearance and Motion&#10;  Control" />
-          <attvalue for="1" value="  With the success of 2D and 3D visual generative models, there is growing&#10;interest in generating 4D content. Existing methods primarily rely on text&#10;prompts to produce 4D content, but they often fall short of accurately defining&#10;complex or rare motions. To address this limitation, we propose MagicPose4D, a&#10;novel framework for refined control over both appearance and motion in 4D&#10;generation. Unlike traditional methods, MagicPose4D accepts monocular videos as&#10;motion prompts, enabling precise and customizable motion generation.&#10;MagicPose4D comprises two key modules:&#10;  i) Dual-Phase 4D Reconstruction Module} which operates in two phases. The&#10;first phase focuses on capturing the model's shape using accurate 2D&#10;supervision and less accurate but geometrically informative 3D&#10;pseudo-supervision without imposing skeleton constraints. The second phase&#10;refines the model using more accurate pseudo-3D supervision, obtained in the&#10;first phase and introduces kinematic chain-based skeleton constraints to ensure&#10;physical plausibility. Additionally, we propose a Global-local Chamfer loss&#10;that aligns the overall distribution of predicted mesh vertices with the&#10;supervision while maintaining part-level alignment without extra annotations.&#10;  ii) Cross-category Motion Transfer Module} leverages the predictions from the&#10;4D reconstruction module and uses a kinematic-chain-based skeleton to achieve&#10;cross-category motion transfer. It ensures smooth transitions between frames&#10;through dynamic rigidity, facilitating robust generalization without additional&#10;training.&#10;  Through extensive experiments, we demonstrate that MagicPose4D significantly&#10;improves the accuracy and consistency of 4D content generation, outperforming&#10;existing methods in various benchmarks.&#10;" />
-          <attvalue for="2" value="&#10;The 4D generation task involves creating a temporal sequence of 3D models of moving objects. Given the difficulty of the general problem, recent approaches have made use of pre-trained models, using prompts to convey the user's intentions. Recently, there has been a significant focus on the use of text/image-prompts to describe appearance and motion~\cite{ling2023align, ren2023dreamgaussian4d, zhao2023animate124}. &#10;The general pipeline of such existing consists of two stpdf: (i) acquiring static geometry through a 3D generation (e.g., a text/image-to-3D) model~\cite{liu2023zero}, which generates 3D representation such as meshes/implicit fields according to text/image prompts, and (ii) obtaining motion information via a video generation model~\cite{blattmann2023stable}, which generates video according to text/image prompts. This approach has achieved impressive 4D content results.&#10;&#10;However, challenges lie in facilitating users to freely and precisely specify the articulated motion of a non-rigid 3D object, and &#10;the generation faithfully reflecting the prompts, in terms of accurately capturing desired object appearance, geometry, and motion.&#10;The main issues with current methods are the following:&#10;(i) Temporal inconsistency in 3D geometry: Most current video generation models fail to ensure temporal consistency of 3D geometry. This involves additional complexity when objects are articulated because the 3D shape and movement must change mutually consistently during motion. For instance, while using the existing methods for 4D animal generation, we have observed unnatural and implausible configurations, such as the number of limbs of the object changing across different frames.&#10;(ii) Limited to common motions: Current video generation models perform well in generating simple, subtle, and common actions, such as walking or small motions like shaking, but they struggle to satisfactorily generate more complex motions, e.g., involving large or uncommon movements such as &quot;King Kong dancing Hip-Hop&quot; and &quot;pig running like a rabbit&quot; as shown in Fig.\ref{exp_poseTransfer}.&#10;(iii) Text is inadequate for accurately describing the details of motion: As shown in Tab.\ref{compare}, most existing methods use text as prompts to describe the desired motion, with some efforts~\cite{bahmani2024tc4d} allowing trajectory (root body transformation) control. However, they are unable to specify detailed motions, e.g., by showing a real-world animal's movement (say, via a monocular short video).&#10;&#10;To address these issues, we propose MagicPose4D, which enables detailed control over both appearance and motion. Unlike existing methods, which rely mostly on text descriptions as motion prompts and for which it is difficult to convey&#10;complex or rare motions, our approach accepts monocular videos or dynamic mesh sequences as motion prompts. This allows for more precise supervision and faithful&#10;4D generation. &#10;MagicPose4D introduces two key modules: Dual-Phase 4D Reconstruction Module and Cross-category Motion Transfer Module. The first module estimates a sequence of 3D models of the object while also simultaneously estimating motion parameters from the motion prompts, and the second module transfers this motion to the target object, which is generated by a 3D generation model controlled by appearance prompts.&#10;&#10;Dual-Phase 4D Reconstruction Module: Given the complexity, especially with articulated objects, we divide this module into two phases. First, we use image descriptors (e.g., segments, flow) as 2D supervision. Additionally, we employ relatively less accurate but geometrically informative pseudo-3D models as geometric priors (using image-to-3D models) to provide 3D supervision for 3D reconstruction.&#10;This phase does not impose constraints emanating from the articulated nature of the object, e.g., captured in the underlying skeletal structure and plausible changes in it during motion, allowing each part to learn arbitrary rotations ($\mathbf{R}$) and translations ($\mathbf{t}$) while focusing on learning the model's shape. In the second phase, we ensure the physical plausibility of the motion by enforcing kinematic chain-based skeletal movement constraints.&#10;Additionally, we propose a Global-local Chamfer loss, which ensures that the overall distribution of predicted mesh vertices aligns with the supervision while maintaining part-level alignment, without requiring additional annotations.&#10;Cross-category Motion Transfer Module: This module achieves cross-species motion transfer by mapping the skeleton of one species to another by establishing joint and limb correspondences through&#10;a kinematic-chain-based representation of the skeleton.&#10;Our motion transfer module is non-training-based. &#10;It helps improve generalization and prevents the poor performance seen in many existing methods when tested on data with significant gaps from the training set. Also, we leverage dynamic rigidity~\cite{zhang2024learning} to guarantee smoothness between frames, unlike the existing approaches~\cite{song20213d, 10076900}, which perform frame-independent pose transfer.&#10;&#10;The following are the main contributions of this paper:&#10;(i) A Novel 4D Generation Framework: Our new framework leverages monocular videos as motion prompts, providing more accurate and more precisely specifiable 4D action generation.&#10;(ii) Skeleton Based 4D Representation: By using skeleton-mediated geometric and 3D prior, we achieve more accurate motion estimation and 3D reconstruction, improving the physical plausibility of the generation.&#10;(iii) Global-local Chamfer Loss: We introduce a novel loss function to better align estimated mesh vertices with the supervisory 3D model overall while maintaining part-level alignment, without additional annotations.&#10;(iv) Cross-category Motion Transfer: Our cross-category mapping in terms of skeleton based dynamic rigidity representation enables smooth transitions between frames and robust generalization without additional training.&#10;(v) Outperforms SOTA: Through extensive experiments, we demonstrate that MagicPose4D provides highly accurate 4D content and significantly outperforms existing methods for 4D reconstruction and pose transfer across all three benchmarks that we experiment with.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Motion Control Systems, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2312.13763" label="2312.13763">
-        <attvalues>
-          <attvalue for="0" value="Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed&#10;  Diffusion Models" />
-          <attvalue for="1" value="  Text-guided diffusion models have revolutionized image and video generation&#10;and have also been successfully used for optimization-based 3D object&#10;synthesis. Here, we instead focus on the underexplored text-to-4D setting and&#10;synthesize dynamic, animated 3D objects using score distillation methods with&#10;an additional temporal dimension. Compared to previous work, we pursue a novel&#10;compositional generation-based approach, and combine text-to-image,&#10;text-to-video, and 3D-aware multiview diffusion models to provide feedback&#10;during 4D object optimization, thereby simultaneously enforcing temporal&#10;consistency, high-quality visual appearance and realistic geometry. Our method,&#10;called Align Your Gaussians (AYG), leverages dynamic 3D Gaussian Splatting with&#10;deformation fields as 4D representation. Crucial to AYG is a novel method to&#10;regularize the distribution of the moving 3D Gaussians and thereby stabilize&#10;the optimization and induce motion. We also propose a motion amplification&#10;mechanism as well as a new autoregressive synthesis scheme to generate and&#10;combine multiple 4D sequences for longer generation. These techniques allow us&#10;to synthesize vivid dynamic scenes, outperform previous work qualitatively and&#10;quantitatively and achieve state-of-the-art text-to-4D performance. Due to the&#10;Gaussian 4D representation, different 4D animations can be seamlessly combined,&#10;as we demonstrate. AYG opens up promising avenues for animation, simulation and&#10;digital content creation as well as synthetic data generation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2311.15127" label="2311.15127">
-        <attvalues>
-          <attvalue for="0" value="Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large&#10;  Datasets" />
-          <attvalue for="1" value="  We present Stable Video Diffusion - a latent video diffusion model for&#10;high-resolution, state-of-the-art text-to-video and image-to-video generation.&#10;Recently, latent diffusion models trained for 2D image synthesis have been&#10;turned into generative video models by inserting temporal layers and finetuning&#10;them on small, high-quality video datasets. However, training methods in the&#10;literature vary widely, and the field has yet to agree on a unified strategy&#10;for curating video data. In this paper, we identify and evaluate three&#10;different stages for successful training of video LDMs: text-to-image&#10;pretraining, video pretraining, and high-quality video finetuning. Furthermore,&#10;we demonstrate the necessity of a well-curated pretraining dataset for&#10;generating high-quality videos and present a systematic curation process to&#10;train a strong base model, including captioning and filtering strategies. We&#10;then explore the impact of finetuning our base model on high-quality data and&#10;train a text-to-video model that is competitive with closed-source video&#10;generation. We also show that our base model provides a powerful motion&#10;representation for downstream tasks such as image-to-video generation and&#10;adaptability to camera motion-specific LoRA modules. Finally, we demonstrate&#10;that our model provides a strong multi-view 3D-prior and can serve as a base to&#10;finetune a multi-view diffusion model that jointly generates multiple views of&#10;objects in a feedforward fashion, outperforming image-based methods at a&#10;fraction of their compute budget. We release code and model weights at&#10;https://github.com/Stability-AI/generative-models .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.17920" label="2403.17920">
-        <attvalues>
-          <attvalue for="0" value="TC4D: Trajectory-Conditioned Text-to-4D Generation" />
-          <attvalue for="1" value="  Recent techniques for text-to-4D generation synthesize dynamic 3D scenes&#10;using supervision from pre-trained text-to-video models. However, existing&#10;representations for motion, such as deformation models or time-dependent neural&#10;representations, are limited in the amount of motion they can generate-they&#10;cannot synthesize motion extending far beyond the bounding box used for volume&#10;rendering. The lack of a more flexible motion model contributes to the gap in&#10;realism between 4D generation methods and recent, near-photorealistic video&#10;generation models. Here, we propose TC4D: trajectory-conditioned text-to-4D&#10;generation, which factors motion into global and local components. We represent&#10;the global motion of a scene's bounding box using rigid transformation along a&#10;trajectory parameterized by a spline. We learn local deformations that conform&#10;to the global trajectory using supervision from a text-to-video model. Our&#10;approach enables the synthesis of scenes animated along arbitrary trajectories,&#10;compositional scene generation, and significant improvements to the realism and&#10;amount of generated motion, which we evaluate qualitatively and through a user&#10;study. Video results can be viewed on our website:&#10;https://sherwinbahmani.github.io/tc4d.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.08809" label="2401.08809">
-        <attvalues>
-          <attvalue for="0" value="Learning Implicit Representation for Reconstructing Articulated Objects" />
-          <attvalue for="1" value="  3D Reconstruction of moving articulated objects without additional&#10;information about object structure is a challenging problem. Current methods&#10;overcome such challenges by employing category-specific skeletal models.&#10;Consequently, they do not generalize well to articulated objects in the wild.&#10;We treat an articulated object as an unknown, semi-rigid skeletal structure&#10;surrounded by nonrigid material (e.g., skin). Our method simultaneously&#10;estimates the visible (explicit) representation (3D shapes, colors, camera&#10;parameters) and the implicit skeletal representation, from motion cues in the&#10;object video without 3D supervision. Our implicit representation consists of&#10;four parts. (1) Skeleton, which specifies how semi-rigid parts are connected.&#10;(2) \textcolor{black}{Skinning Weights}, which associates each surface vertex&#10;with semi-rigid parts with probability. (3) Rigidity Coefficients, specifying&#10;the articulation of the local surface. (4) Time-Varying Transformations, which&#10;specify the skeletal motion and surface deformation parameters. We introduce an&#10;algorithm that uses physical constraints as regularization terms and&#10;iteratively estimates both implicit and explicit representations. Our method is&#10;category-agnostic, thus eliminating the need for category-specific skeletons,&#10;we show that our method outperforms state-of-the-art across standard video&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.15025" label="2109.15025">
-        <attvalues>
-          <attvalue for="0" value="3D Pose Transfer with Correspondence Learning and Mesh Refinement" />
-          <attvalue for="1" value="  3D pose transfer is one of the most challenging 3D generation tasks. It aims&#10;to transfer the pose of a source mesh to a target mesh and keep the identity&#10;(e.g., body shape) of the target mesh. Some previous works require key point&#10;annotations to build reliable correspondence between the source and target&#10;meshes, while other methods do not consider any shape correspondence between&#10;sources and targets, which leads to limited generation quality. In this work,&#10;we propose a correspondence-refinement network to achieve the 3D pose transfer&#10;for both human and animal meshes. The correspondence between source and target&#10;meshes is first established by solving an optimal transport problem. Then, we&#10;warp the source mesh according to the dense correspondence and obtain a coarse&#10;warped mesh. The warped mesh will be better refined with our proposed Elastic&#10;Instance Normalization, which is a conditional normalization layer and can help&#10;to generate high-quality meshes. Extensive experimental results show that the&#10;proposed architecture can effectively transfer the poses from source to target&#10;meshes and produce better results with satisfied visual performance than&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.05877" label="2012.05877">
-        <attvalues>
-          <attvalue for="0" value="INeRF: Inverting Neural Radiance Fields for Pose Estimation" />
-          <attvalue for="1" value="  We present iNeRF, a framework that performs mesh-free pose estimation by&#10;&quot;inverting&quot; a Neural RadianceField (NeRF). NeRFs have been shown to be&#10;remarkably effective for the task of view synthesis - synthesizing&#10;photorealistic novel views of real-world scenes or objects. In this work, we&#10;investigate whether we can apply analysis-by-synthesis via NeRF for mesh-free,&#10;RGB-only 6DoF pose estimation - given an image, find the translation and&#10;rotation of a camera relative to a 3D object or scene. Our method assumes that&#10;no object mesh models are available during either training or test time.&#10;Starting from an initial pose estimate, we use gradient descent to minimize the&#10;residual between pixels rendered from a NeRF and pixels in an observed image.&#10;In our experiments, we first study 1) how to sample rays during pose refinement&#10;for iNeRF to collect informative gradients and 2) how different batch sizes of&#10;rays affect iNeRF on a synthetic dataset. We then show that for complex&#10;real-world scenes from the LLFF dataset, iNeRF can improve NeRF by estimating&#10;the camera poses of novel images and using these images as additional training&#10;data for NeRF. Finally, we show iNeRF can perform category-level object pose&#10;estimation, including object instances not seen during training, with RGB&#10;images by inverting a NeRF model inferred from a single view.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Neural 3D shape representations. Recently, several works have investigated representing 3D shapes implicitly with neural networks. In this formulation, the geometric or appearance properties of a 3D point $\mathbf{x} = (x, y, z)$ is parameterized as the output of a neural network.&#10;The advantage of this approach is that scenes with complex topologies can be represented at high resolution with low memory usage.&#10;When ground truth 3D geometry is available as supervision, neural networks can be optimized to represent the signed distance function~\cite{park2019deepsdf} or occupancy function~\cite{mescheder2019occupancy}.&#10;However, ground truth 3D shapes are hard to obtain in practice. &#10;This motivates subsequent work on relaxing this constraint by formulating differentiable rendering pipelines that allow neural 3D shape representations to be learned using only 2D images as supervision~\cite{kato2020differentiable,lin2020sdfsrn,lin2019photometric}.&#10;Niemeyer~\etal~\cite{niemeyer2020differentiable} represent a surface as a neural 3D occupancy field and texture as a neural 3D texture field. Ray intersectionlocations are first computed with numerical methods using the occupancy field and then provided as inputs to the texture field to output the colors.&#10;Scene Representation Networks~\cite{sitzmann2019scene} learn a neural 3D representation that outputs a feature vector and RGB color at each continuous 3D coordinate and employs a recurrent neural network to perform differentiable ray-marching.&#10;NeRF~\cite{mildenhall2020nerf} shows that by taking view directions as additional inputs, a learned neural network works well in tandem with volume rendering techniques and enables photo-realistic view synthesis.&#10;NeRF in the Wild~\cite{martin2020nerf} extends NeRF to additionally model each image's individual appearance and transient content, thereby allowing high-quality 3D reconstruction of landmarks using unconstrained photo collections.&#10;NSVF~\cite{liu2020neural} improves NeRF by incorporating a sparse voxel octree structure into the scene representation, which accelerates rendering by allowing voxels without scene content to be omitted during rendering.&#10;\yenchen{To generalize across scenes or objects, pixelNeRF~\cite{yu2020pixelnerf} and IBRNet~\cite{wang2021ibrnet} predict NeRF models conditioned on input images.}&#10;Unlike NeRF and its variants, which learn to represent a scene's structure from posed RGB images, we address the inverse problem: how to localize new observations whose camera poses are unknown, using a &#10;NeRF.&#10;&#10;Pose Estimation from RGB Images.&#10;Classical methods for object pose estimation address the task by detecting and matching keypoints with known 3D models~\cite{aubry2014seeing,collet2011moped,ferrari2006simultaneous,rothganger20063d}. Recent approaches based on deep learning have proposed to 1) directly estimate objects pose using CNN-based architectures~\cite{schwarz2015rgb,tulsiani2015viewpoints,xiang2017posecnn} or 2) estimate 2D keypoints~\cite{pavlakos20176,suwajanakorn2018discovery,tekin2018real,tremblay2018deep} and solve for pose using the PnP-RANSAC algorithm. Differentiable mesh renderers~\cite{chen2019learning,palazzi2018end} have also been explored for pose estimation.&#10;Although their results are impressive, all the aforementioned works require access to objects' 3D models during both training and testing, which significantly limits the applicability of these approaches.&#10;\pete{Recently, Chen~\etal~\cite{chen2020category} address category-level object pose estimation~\cite{wang2019normalized}, in particular they impressively estimate object shape and pose across a category from a single image. They use a single-image reconstruction with a 3D voxel-based feature volume and then estimating pose using iterative image alignment.&#10;In contrast, in our work we use continuous implicit 3D representations in the form of NeRF models, which have been empirically shown to produce more photorealistic novel-image rendering \cite{mildenhall2020nerf, martin2020nerf} and scale to large, building-scale volumes \cite{martin2020nerf}, which we hypothesize will enable higher-fidelity pose estimation.&#10;This also presents challenges, however, due to the expensive computational cost of NeRF rendering, for which we introduce a novel importance-sampling approach in Sec.~\ref{subsec:sampling}. Another practical difference in our approach to category-level pose estimation -- while \cite{chen2020category} optimizes for shape with gradient descent, we show we can instead allow pixelNeRF to predict a NeRF model with just a forward pass of a network.&#10;Additionally, since NeRF models scale well to large scenes, we can use the same iNeRF formulation to perform localization, for example in challenging real-world LLFF scenes -- this capability was not demonstrated in \cite{chen2020category}, and may be challenging due to the memory limitations of voxel representations for sufficient fidelity in large scenes.}&#10;While object pose estimation methods are often separate from methods used for visual localization of a camera in a scene as in the SfM literature (i.e. \cite{shotton2013scene,valentin2015exploiting,schmidt2016self}), because NeRF and iNeRF only require posed RGB images as training, iNeRF can be applied to localization as well. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Pose Estimation, Mathematics, Neural Radiance Fields, Physics, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2304.06566" label="2304.06566">
-        <attvalues>
-          <attvalue for="0" value="NeRD: Neural field-based Demosaicking" />
-          <attvalue for="1" value="  We introduce NeRD, a new demosaicking method for generating full-color images&#10;from Bayer patterns. Our approach leverages advancements in neural fields to&#10;perform demosaicking by representing an image as a coordinate-based neural&#10;network with sine activation functions. The inputs to the network are spatial&#10;coordinates and a low-resolution Bayer pattern, while the outputs are the&#10;corresponding RGB values. An encoder network, which is a blend of ResNet and&#10;U-net, enhances the implicit neural representation of the image to improve its&#10;quality and ensure spatial consistency through prior learning. Our experimental&#10;results demonstrate that NeRD outperforms traditional and state-of-the-art&#10;CNN-based methods and significantly closes the gap to transformer-based&#10;methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Raw data acquired by modern digital camera sensors is subject to various types of signal degradation, one of the most severe being the color filter array. To convert the raw data (Fig.~\ref{fig:header}a) into an image suitable for human visual perception (Fig.~\ref{fig:header}b), a demosaicking procedure is necessary \cite{menon2011color}.&#10;&#10;Two main categories of image demosaicking exist: model-based and learning-based methods. Model-based methods, such as bilinear interpolation, Malvar \cite{malvar2004high}, or Menon \cite{menon2006demosaicing}, are still widely used, but they fail to match the performance of recent deep learning-based approaches using deep convolutional networks (CNN) \cite{kokkinos2019iterative, gharbi2016deep, kerepecky2021d3net} or Swin Transformers \cite{xing2022residual}.&#10;&#10;Recently, Transformer networks have seen remarkable success in computer vision tasks and have become a state-of-the-art approach in demosaicking. However, a new paradigm in deep learning, Neural Fields (NF) \cite{xie2022neural}, is gaining attention due to its comparable or superior performance in several computer vision tasks \cite{xie2022neural, mildenhall2021nerf, shangguan2022learning, chen2021nerv, chen2021learning, sitzmann2020implicit, skorokhodov2021adversarial}. The basic idea behind NF is to represent data as the weights of a Multilayer Perceptron (MLP), known as implicit neural representation.&#10;&#10;NF has been applied in various domains and applications including Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf} which achieved state-of-the-art results in representing complex 3D scenes. NeRV \cite{chen2021nerv} encodes entire videos in neural networks. The Local Implicit Image Function (LIIF) \cite{chen2021learning} represents an image as a neural field capable of extrapolating to 30 times higher resolution. SIREN \cite{sitzmann2020implicit} uses a sinusoidal neural representation and demonstrates superiority over classical ReLU MLP in representing complex natural signals such as images.&#10;&#10;Prior information from training data can be encoded into neural representation through conditioning (local or global) using methods such as concatenation, modulation of activation functions \cite{dupont2022data}, or hypernetworks \cite{skorokhodov2021adversarial}. For example, CURE \cite{shangguan2022learning}, a state-of-the-art method for video interpolation based on NF, uses an encoder to impose space-time consistency using local feature codes.&#10;&#10;NF has also been used in image-to-image translation tasks such as superresolution, denoising, inpainting, and generative modeling \cite{xie2022neural}. However, to the best of our knowledge, no NF method has been proposed for demosaicking.&#10;&#10;In this paper, we present NeRD, a novel approach for image demosaicking based on NF. The proposed method employs a joint ResNet and U-Net architecture to extract prior information from high-resolution ground-truth images and their corresponding Bayer patterns. This information is then used to condition the MLP using local feature encodings. The proposed approach offers a unique and innovative solution for image demosaicking.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Neural Network Design, Mathematics, Computer Science, Image Demosaicking, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1807.06403" label="1807.06403">
-        <attvalues>
-          <attvalue for="0" value="Iterative Joint Image Demosaicking and Denoising using a Residual&#10;  Denoising Network" />
-          <attvalue for="1" value="  Modern digital cameras rely on the sequential execution of separate image&#10;processing steps to produce realistic images. The first two steps are usually&#10;related to denoising and demosaicking where the former aims to reduce noise&#10;from the sensor and the latter converts a series of light intensity readings to&#10;color images. Modern approaches try to jointly solve these problems, i.e. joint&#10;denoising-demosaicking which is an inherently ill-posed problem given that&#10;two-thirds of the intensity information is missing and the rest are perturbed&#10;by noise. While there are several machine learning systems that have been&#10;recently introduced to solve this problem, the majority of them relies on&#10;generic network architectures which do not explicitly take into account the&#10;physical image model. In this work we propose a novel algorithm which is&#10;inspired by powerful classical image regularization methods, large-scale&#10;optimization, and deep learning techniques. Consequently, our derived iterative&#10;optimization algorithm, which involves a trainable denoising network, has a&#10;transparent and clear interpretation compared to other black-box data driven&#10;approaches. Our extensive experimentation line demonstrates that our proposed&#10;method outperforms any previous approaches for both noisy and noise-free data&#10;across many different datasets. This improvement in reconstruction quality is&#10;attributed to the rigorous derivation of an iterative solution and the&#10;principled way we design our denoising network architecture, which as a result&#10;requires fewer trainable parameters than the current state-of-the-art solution&#10;and furthermore can be efficiently trained by using a significantly smaller&#10;number of training data than existing deep demosaicking networks. Code and&#10;results can be found at https://github.com/cig-skoltech/deep_demosaick&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.07098" label="2204.07098">
-        <attvalues>
-          <attvalue for="0" value="Residual Swin Transformer Channel Attention Network for Image&#10;  Demosaicing" />
-          <attvalue for="1" value="  Image demosaicing is problem of interpolating full- resolution color images&#10;from raw sensor (color filter array) data. During last decade, deep neural&#10;networks have been widely used in image restoration, and in particular, in&#10;demosaicing, attaining significant performance improvement. In recent years,&#10;vision transformers have been designed and successfully used in various&#10;computer vision applications. One of the recent methods of image restoration&#10;based on a Swin Transformer (ST), SwinIR, demonstrates state-of-the-art&#10;performance with a smaller number of parameters than neural network-based&#10;methods. Inspired by the success of SwinIR, we propose in this paper a novel&#10;Swin Transformer-based network for image demosaicing, called RSTCANet. To&#10;extract image features, RSTCANet stacks several residual Swin Transformer&#10;Channel Attention blocks (RSTCAB), introducing the channel attention for each&#10;two successive ST blocks. Extensive experiments demonstrate that RSTCANet out-&#10;performs state-of-the-art image demosaicing methods, and has a smaller number&#10;of parameters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.13903" label="2110.13903">
-        <attvalues>
-          <attvalue for="0" value="NeRV: Neural Representations for Videos" />
-          <attvalue for="1" value="  We propose a novel neural representation for videos (NeRV) which encodes&#10;videos in neural networks. Unlike conventional representations that treat&#10;videos as frame sequences, we represent videos as neural networks taking frame&#10;index as input. Given a frame index, NeRV outputs the corresponding RGB image.&#10;Video encoding in NeRV is simply fitting a neural network to video frames and&#10;decoding process is a simple feedforward operation. As an image-wise implicit&#10;representation, NeRV output the whole image and shows great efficiency compared&#10;to pixel-wise implicit representation, improving the encoding speed by 25x to&#10;70x, the decoding speed by 38x to 132x, while achieving better video quality.&#10;With such a representation, we can treat videos as neural networks, simplifying&#10;several video-related tasks. For example, conventional video compression&#10;methods are restricted by a long and complex pipeline, specifically designed&#10;for the task. In contrast, with NeRV, we can use any neural network compression&#10;method as a proxy for video compression, and achieve comparable performance to&#10;traditional frame-based video compression approaches (H.264, HEVC \etc).&#10;Besides compression, we demonstrate the generalization of NeRV for video&#10;denoising. The source code and pre-trained model can be found at&#10;https://github.com/haochen-rye/NeRV.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.12026" label="2011.12026">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Generation of Continuous Images" />
-          <attvalue for="1" value="  In most existing learning systems, images are typically viewed as 2D pixel&#10;arrays. However, in another paradigm gaining popularity, a 2D image is&#10;represented as an implicit neural representation (INR) - an MLP that predicts&#10;an RGB pixel value given its (x,y) coordinate. In this paper, we propose two&#10;novel architectural techniques for building INR-based image decoders:&#10;factorized multiplicative modulation and multi-scale INRs, and use them to&#10;build a state-of-the-art continuous image GAN. Previous attempts to adapt INRs&#10;for image generation were limited to MNIST-like datasets and do not scale to&#10;complex real-world data. Our proposed INR-GAN architecture improves the&#10;performance of continuous image generators by several times, greatly reducing&#10;the gap between continuous image GANs and pixel-based ones. Apart from that, we&#10;explore several exciting properties of the INR-based decoders, like&#10;out-of-the-box superresolution, meaningful image-space interpolation,&#10;accelerated inference of low-resolution images, an ability to extrapolate&#10;outside of image boundaries, and strong geometric prior. The project page is&#10;located at https://universome.github.io/inr-gan.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.00137" label="2203.00137">
-        <attvalues>
-          <attvalue for="0" value="Learning Cross-Video Neural Representations for High-Quality Frame&#10;  Interpolation" />
-          <attvalue for="1" value="  This paper considers the problem of temporal video interpolation, where the&#10;goal is to synthesize a new video frame given its two neighbors. We propose&#10;Cross-Video Neural Representation (CURE) as the first video interpolation&#10;method based on neural fields (NF). NF refers to the recent class of methods&#10;for the neural representation of complex 3D scenes that has seen widespread&#10;success and application across computer vision. CURE represents the video as a&#10;continuous function parameterized by a coordinate-based neural network, whose&#10;inputs are the spatiotemporal coordinates and outputs are the corresponding RGB&#10;values. CURE introduces a new architecture that conditions the neural network&#10;on the input frames for imposing space-time consistency in the synthesized&#10;video. This not only improves the final interpolation quality, but also enables&#10;CURE to learn a prior across multiple videos. Experimental evaluations show&#10;that CURE achieves the state-of-the-art performance on video interpolation on&#10;several benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.10008" label="2010.10008">
-        <attvalues>
-          <attvalue for="0" value="Towards Accurate Human Pose Estimation in Videos of Crowded Scenes" />
-          <attvalue for="1" value="  Video-based human pose estimation in crowded scenes is a challenging problem&#10;due to occlusion, motion blur, scale variation and viewpoint change, etc. Prior&#10;approaches always fail to deal with this problem because of (1) lacking of&#10;usage of temporal information; (2) lacking of training data in crowded scenes.&#10;In this paper, we focus on improving human pose estimation in videos of crowded&#10;scenes from the perspectives of exploiting temporal context and collecting new&#10;data. In particular, we first follow the top-down strategy to detect persons&#10;and perform single-person pose estimation for each frame. Then, we refine the&#10;frame-based pose estimation with temporal contexts deriving from the&#10;optical-flow. Specifically, for one frame, we forward the historical poses from&#10;the previous frames and backward the future poses from the subsequent frames to&#10;current frame, leading to stable and accurate human pose estimation in videos.&#10;In addition, we mine new data of similar scenes to HIE dataset from the&#10;Internet for improving the diversity of training set. In this way, our model&#10;achieves best performance on 7 out of 13 videos and 56.33 average w\_AP on test&#10;dataset of HIE challenge.&#10;" />
-          <attvalue for="2" value="&#10;Human pose estimation is important for many computer vision applications, including human action recognition, human-computer interaction, and video surveillance. Due to the viewpoint variance, appearance variance and cluttered background, pose estimation is a very challenging task for large scale image and video datasets. &#10;Recently, significant progress has been made in this area~\cite{sun2019HRNet}. However, the pose estimation in complex events~\cite{lin2020human} is still relatively new and a challenging problem. In this challenge of pose estimation on crowded scenes and complex events, we propose to obtain the pose of single images based pose estimation method, which can be applied to each video frame to get an initial pose estimation, and a further refinement through frames can be applied to make the pose estimation consistent and more accurate.&#10;&#10;The general pipeline for the pose estimation method that we used can be divided into two parts, human detection and pose estimation, respectively. First, we use a human detection method in crowd scenes to detect the bounding box of highly-overlapping human instances in the detection phase. In the second step, we perform pose estimation on every box by two state-of-the-art single-person pose estimation models~\cite{xiao2018SimpleNet, sun2019HRNet}. During the pose estimation phase, we propose a optical flow smoothing algorithm to refine our pose predictions. The framework of our approach is shown in Figure \ref{figure:structure}. &#10;&#10;Since the problem is treated as a two-stage problem to be tackled one by one, each module will be introduced separately. The following of the report is organized as follows: Sec.~\ref{sec:human_det} investigates the common detection methods on HIE2020 challenge and also introduces our detailed method and experiments on human detection; Sec.~\ref{sec:pose_estimation} introduces the pose estimation as well as the final pose generation process. Sec.~\ref{sec:experiments} introduces the experiments and training details of pose estimation. Finally, Sec.~\ref{sec:conclusion} concludes the report.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1902.09212" label="1902.09212">
-        <attvalues>
-          <attvalue for="0" value="Deep High-Resolution Representation Learning for Human Pose Estimation" />
-          <attvalue for="1" value="  This is an official pytorch implementation of Deep High-Resolution&#10;Representation Learning for Human Pose Estimation. In this work, we are&#10;interested in the human pose estimation problem with a focus on learning&#10;reliable high-resolution representations. Most existing methods recover&#10;high-resolution representations from low-resolution representations produced by&#10;a high-to-low resolution network. Instead, our proposed network maintains&#10;high-resolution representations through the whole process. We start from a&#10;high-resolution subnetwork as the first stage, gradually add high-to-low&#10;resolution subnetworks one by one to form more stages, and connect the&#10;mutli-resolution subnetworks in parallel. We conduct repeated multi-scale&#10;fusions such that each of the high-to-low resolution representations receives&#10;information from other parallel representations over and over, leading to rich&#10;high-resolution representations. As a result, the predicted keypoint heatmap is&#10;potentially more accurate and spatially more precise. We empirically&#10;demonstrate the effectiveness of our network through the superior pose&#10;estimation results over two benchmark datasets: the COCO keypoint detection&#10;dataset and the MPII Human Pose dataset. The code and models have been publicly&#10;available at&#10;\url{https://github.com/leoxiaobin/deep-high-resolution-net.pytorch}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.04490" label="2005.04490">
-        <attvalues>
-          <attvalue for="0" value="Human in Events: A Large-Scale Benchmark for Human-centric Video&#10;  Analysis in Complex Events" />
-          <attvalue for="1" value="  Along with the development of modern smart cities, human-centric video&#10;analysis has been encountering the challenge of analyzing diverse and complex&#10;events in real scenes. A complex event relates to dense crowds, anomalous&#10;individuals, or collective behaviors. However, limited by the scale and&#10;coverage of existing video datasets, few human analysis approaches have&#10;reported their performances on such complex events. To this end, we present a&#10;new large-scale dataset with comprehensive annotations, named Human-in-Events&#10;or HiEve (Human-centric video analysis in complex Events), for the&#10;understanding of human motions, poses, and actions in a variety of realistic&#10;events, especially in crowd &amp; complex events. It contains a record number of&#10;poses (&gt;1M), the largest number of action instances (&gt;56k) under complex&#10;events, as well as one of the largest numbers of trajectories lasting for&#10;longer time (with an average trajectory length of &gt;480 frames). Based on its&#10;diverse annotation, we present two simple baselines for action recognition and&#10;pose estimation, respectively. They leverage cross-label information during&#10;training to enhance the feature learning in corresponding visual tasks.&#10;Experiments show that they could boost the performance of existing action&#10;recognition and pose estimation pipelines. More importantly, they prove the&#10;widely ranged annotations in HiEve can improve various video tasks.&#10;Furthermore, we conduct extensive experiments to benchmark recent video&#10;analysis approaches together with our baseline methods, demonstrating HiEve is&#10;a challenging dataset for human-centric video analysis. We expect that the&#10;dataset will advance the development of cutting-edge techniques in&#10;human-centric analysis and the understanding of complex events. The dataset is&#10;available at http://humaninevents.org&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06208" label="1804.06208">
-        <attvalues>
-          <attvalue for="0" value="Simple Baselines for Human Pose Estimation and Tracking" />
-          <attvalue for="1" value="  There has been significant progress on pose estimation and increasing&#10;interests on pose tracking in recent years. At the same time, the overall&#10;algorithm and system complexity increases as well, making the algorithm&#10;analysis and comparison more difficult. This work provides simple and effective&#10;baseline methods. They are helpful for inspiring and evaluating new ideas for&#10;the field. State-of-the-art results are achieved on challenging benchmarks. The&#10;code will be available at https://github.com/leoxiaobin/pose.pytorch.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.06617" label="2409.06617">
-        <attvalues>
-          <attvalue for="0" value="When to Extract ReID Features: A Selective Approach for Improved&#10;  Multiple Object Tracking" />
-          <attvalue for="1" value="  Extracting and matching Re-Identification (ReID) features is used by many&#10;state-of-the-art (SOTA) Multiple Object Tracking (MOT) methods, particularly&#10;effective against frequent and long-term occlusions. While end-to-end object&#10;detection and tracking have been the main focus of recent research, they have&#10;yet to outperform traditional methods in benchmarks like MOT17 and MOT20. Thus,&#10;from an application standpoint, methods with separate detection and embedding&#10;remain the best option for accuracy, modularity, and ease of implementation,&#10;though they are impractical for edge devices due to the overhead involved. In&#10;this paper, we investigate a selective approach to minimize the overhead of&#10;feature extraction while preserving accuracy, modularity, and ease of&#10;implementation. This approach can be integrated into various SOTA methods. We&#10;demonstrate its effectiveness by applying it to StrongSORT and Deep OC-SORT.&#10;Experiments on MOT17, MOT20, and DanceTrack datasets show that our mechanism&#10;retains the advantages of feature extraction during occlusions while&#10;significantly reducing runtime. Additionally, it improves accuracy by&#10;preventing confusion in the feature-matching stage, particularly in cases of&#10;deformation and appearance similarity, which are common in DanceTrack.&#10;https://github.com/emirhanbayar/Fast-StrongSORT,&#10;https://github.com/emirhanbayar/Fast-Deep-OC-SORT&#10;" />
-          <attvalue for="2" value="&#10;&#10;\begin{figure}&#10;&#10;pgfscope{figure}&#10;&#10;Multiple Object Tracking (MOT) is the problem of maintaining the identity of objects of predefined classes across a sequence of frames \cite{strongsort}. Tracking-by-detection (TbD) has long been the dominant paradigm in the field. Methods that follow this paradigm divide tracking into two steps: (1) frame-wise object detection, and (2) data association to link those detections to form trajectories \cite{Seidenschwarz_2023_CVPR}. Real-time TbD methods commonly hold a Kalman Filter \cite{kalman} for each tracked object, match the estimated states of the objects with the detections via the Hungarian algorithm \cite{hungarian} and update the states of the tracklets with matched detection\cite{sort, deepsort, bytetrack, ocsort, deepocsort,strongsort,botsort,boostTrack,pia2,imprasso,lgtrack}.The cost matrix of the matching algorithms can be based on the object's position, motion, appearance, or a combination of multiple factors \cite{deepsort, ocsort, deepocsort}.&#10;&#10;ReID features \cite{reid-bot, fastreid} are widely used in the matching phase of state-of-the-art (SOTA) methods due to their robustness against occlusions and clutter \cite{deepocsort, deepsort, strongsort, botsort, fairmot, rethink}. These methods, which rely on computationally expensive Convolutional Neural Networks (CNNs) for feature extraction, result in extremely low FPS (Frames per Second) when executed on resource-constrained hardware \cite{runtime}. On the other hand, existing MOT methods prioritize achieving high accuracy on prerecorded datasets such as MOT17 \cite{mot16-review}, MOT20 \cite{mot20}, and DanceTrack \cite{dancetrack}. However, in real-time image streaming, frames are captured at wider intervals as FPS decreases, making changes in object positions and movements more significant. Consequently, the error in prediction and matching steps increases \cite{zhou2022apptracker, fpsanalysis}. This renders the mentioned methods impractical for edge artificial intelligence applications.&#10;&#10;Several attempts have been proposed to mitigate the overhead of feature extraction resulting from separate detection and tracking. The idea of joint detection and embedding (JDE) \cite{jde-review} is one such approach, which enables detection and feature extraction in a shared backbone \cite{jde1, jde-review, rethink, retinatrack,fairmot}. Some other approaches replaces data association by regression of previous detections \cite{tbreg1, tbreg2}. With the introduction of DETR \cite{detr}, using transformers, which learn spatiotemporal relation of detections and tracklets, for end-to-end object detection and tracking gained popularity \cite{trans1, trans2, trans3, trans4}. However, they are criticized for lack of modularity and computational demand of training and deployment \cite{Seidenschwarz_2023_CVPR,moyolo}.&#10;&#10;While end-to-end object detection and tracking are trending and intriguing from a research perspective, they have yet to outperform traditional methods in well-known benchmarks like MOT17 and MOT20. Therefore, TbD methods with separate detection and embedding remain the best option from an application point of view due to their accuracy, modularity, and ease of implementation \cite{boxmot, ocmc, conftrack}. Instead of eliminating the feature extraction step, a recent study \cite{ss*} proposed an elegant strategy to minimize the number of detections on which feature extraction is performed. Although this idea is inspirational and outperforms trackers that do not use ReID features, it does reduce the accuracy compared to the original tracker on which the algorithm is applied. In this study, we propose extensions to this algorithm to limit the number of extractions without sacrificing accuracy—in fact, increasing it. The contributions of this work can be summarized as follows:&#10;&#10;\begin{itemize}&#10; \item A selective mechanism to minimize the overhead of feature extractions while preserving accuracy, modularity, and ease of implementation. This method can be applied to any real-time TbD tracker that uses a state estimator (e.g. Kalman Filter) and performs matching based on ReID features.&#10;&#10;\item A feature decay mechanism that increases accuracy by enhancing the temporal locality of matched features.&#10;&#10;\item A showcase of its implementation on StrongSORT and Deep OC-SORT, with detailed analysis on MOT17, MOT20, and DanceTrack datasets and runtime analysis on an edge device.&#10; pgfscope{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Feature Extraction, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2202.13514" label="2202.13514">
-        <attvalues>
-          <attvalue for="0" value="StrongSORT: Make DeepSORT Great Again" />
-          <attvalue for="1" value="  Recently, Multi-Object Tracking (MOT) has attracted rising attention, and&#10;accordingly, remarkable progresses have been achieved. However, the existing&#10;methods tend to use various basic models (e.g, detector and embedding model),&#10;and different training or inference tricks, etc. As a result, the construction&#10;of a good baseline for a fair comparison is essential. In this paper, a classic&#10;tracker, i.e., DeepSORT, is first revisited, and then is significantly improved&#10;from multiple perspectives such as object detection, feature embedding, and&#10;trajectory association. The proposed tracker, named StrongSORT, contributes a&#10;strong and fair baseline for the MOT community. Moreover, two lightweight and&#10;plug-and-play algorithms are proposed to address two inherent &quot;missing&quot;&#10;problems of MOT: missing association and missing detection. Specifically,&#10;unlike most methods, which associate short tracklets into complete trajectories&#10;at high computation complexity, we propose an appearance-free link model&#10;(AFLink) to perform global association without appearance information, and&#10;achieve a good balance between speed and accuracy. Furthermore, we propose a&#10;Gaussian-smoothed interpolation (GSI) based on Gaussian process regression to&#10;relieve the missing detection. AFLink and GSI can be easily plugged into&#10;various trackers with a negligible extra computational cost (1.7 ms and 7.1 ms&#10;per image, respectively, on MOT17). Finally, by fusing StrongSORT with AFLink&#10;and GSI, the final tracker (StrongSORT++) achieves state-of-the-art results on&#10;multiple public benchmarks, i.e., MOT17, MOT20, DanceTrack and KITTI. Codes are&#10;available at https://github.com/dyhBUPT/StrongSORT and&#10;https://github.com/open-mmlab/mmtracking.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.04656" label="2206.04656">
-        <attvalues>
-          <attvalue for="0" value="Simple Cues Lead to a Strong Multi-Object Tracker" />
-          <attvalue for="1" value="  For a long time, the most common paradigm in Multi-Object Tracking was&#10;tracking-by-detection (TbD), where objects are first detected and then&#10;associated over video frames. For association, most models resourced to motion&#10;and appearance cues, e.g., re-identification networks. Recent approaches based&#10;on attention propose to learn the cues in a data-driven manner, showing&#10;impressive results. In this paper, we ask ourselves whether simple good old TbD&#10;methods are also capable of achieving the performance of end-to-end models. To&#10;this end, we propose two key ingredients that allow a standard&#10;re-identification network to excel at appearance-based tracking. We extensively&#10;analyse its failure cases, and show that a combination of our appearance&#10;features with a simple motion model leads to strong tracking results. Our&#10;tracker generalizes to four public datasets, namely MOT17, MOT20, BDD100k, and&#10;DanceTrack, achieving state-of-the-art performance.&#10;https://github.com/dvl-tum/GHOST.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;In the last years, TbD was the most common paradigm used in MOT \cite{DBLP:conf/iccv/BergmannML19,DBLP:conf/cvpr/XuOBHLA20,DBLP:conf/eccv/WangZLLW20,DBLP:journals/ijcv/ZhangWWZL21,DBLP:journals/corr/abs-2103-14258,DBLP:conf/cvpr/PangLZLL20,DBLP:conf/cvpr/PangQLCLDY21,DBLP:conf/cvpr/BrasoL20,DBLP:conf/icml/HornakovaHRS20}. &#10;Pedestrians are first detected using object detectors \cite{DBLP:conf/nips/RenHGS15,DBLP:conf/cvpr/RedmonF17,DBLP:conf/cvpr/YangCL16}. Then, detections are associated across frames to form trajectories corresponding to a certain identity utilizing motion, location, appearance cues, or a combination of them. The association can either be solved frame-by-frame for online applications or offline in a track-wise manner over the sequence.&#10;&#10;Graph-Based Approaches.&#10;One common formalism to perform data association following the TbD paradigm is viewing each detection as a node in a graph with edges linking several nodes over the temporal domain to form trajectories. Determining which nodes are connected can then be solved using maximum flow \cite{DBLP:journals/pami/BerclazFTF11} or minimum cost approaches \cite{DBLP:conf/cvpr/JiangFL07,DBLP:conf/cvpr/PirsiavashRF11,DBLP:conf/cvpr/ZhangLN08} by, e.g, taking motion models into account \cite{DBLP:conf/iccvw/Leal-TaixePR11}. Recent advances combined track-wise graph-based models with neural networks \cite{DBLP:conf/cvpr/BrasoL20}.&#10;We challenge those recent advances by showing that we can obtain strong TbD trackers without using a complex graph model combining our strong but simple cues. &#10;&#10;Motion-Based Association.&#10;Different from the graph-based approaches, many TbD approaches perform frame-by-frame association directly using motion and location cues from detections and existing trajectories \cite{DBLP:conf/icip/BewleyGORU16,DBLP:conf/avss/BochinskiES17,DBLP:conf/eccv/ZhouKK20,DBLP:conf/eccv/PengWWWWTWLHF20,DBLP:conf/cvpr/PangLZLL20}. For short term preservation, those trackers exploit that given two nearby frames object displacements tend to be small. This allows them to utilize spatial proximity for matching by exploiting, \eg, Kalman filters \cite{DBLP:conf/icip/BewleyGORU16}. Taking this idea further, approaches following the tracking-by-regression paradigm utilize object detectors \cite{DBLP:conf/iccv/BergmannML19,DBLP:conf/eccv/ZhouKK20} to regress bounding box positions. Recent advances introduced Transformer-based approaches \cite{DBLP:journals/corr/abs-2101-02702,DBLP:journals/corr/abs-2012-15460,DBLP:journals/corr/abs-2105-03247} that perform tracking following the tracking-by-attention paradigm. &#10;Using sophisticated motion models, those approaches reach outstanding performance on several datasets, especially in short-term associations. However, especially the Transformer-based approaches require sophisticated training strategies. Contrary to all those approaches, we show that a simple linear motion model suffices to model short-term associations in most scenarios. &#10;In scenes with moving cameras or scenarios requiring non-trivial long-term associations, e.g., scenarios with many occlusions, purely motion-based trackers struggle which calls for a combination with appearance-based cues.&#10;&#10;Appearance-Based Association.&#10;To achieve better performance in long-term association scenarios, numerous approaches use additional appearance-based re-identification networks that encode appearance cues to re-identify persons after occlusions \cite{DBLP:conf/eccv/YuLLLSY16,DBLP:conf/iccv/BergmannML19,DBLP:conf/cvpr/BrasoL20,DBLP:conf/cvpr/KimLAR21,DBLP:conf/icip/WojkeBP17,DBLP:conf/cvpr/Leal-TaixeCS16,DBLP:conf/iccv/SadeghianAS17,DBLP:conf/cvpr/SonBCH17}.&#10;Further exploring this direction, a recent work \cite{DBLP:conf/cvpr/PangQLCLDY21} proposed to train a detection network solely utilizing embedding information during the training process. Enhancing MOT towards real-time, several works proposed to jointly compute detections and embeddings in a multi-task setting \cite{DBLP:conf/cvpr/XuOBHLA20,DBLP:conf/cvpr/GuoWWT21,DBLP:conf/eccv/WangZLLW20,DBLP:conf/cvpr/LuRVH20,DBLP:journals/ijcv/ZhangWWZL21}. Some of them introduce more balanced training schemes \cite{DBLP:conf/cvpr/GuoWWT21,DBLP:conf/cvpr/XuOBHLA20} to better leverage the synergies between both cues.&#10;While promising, approaches using appearance additionally to motion cues require rather complex association schemes with several steps \cite{DBLP:journals/ijcv/ZhangWWZL21,DBLP:conf/icip/BewleyGORU16}.&#10;Also, complex and highly differing training schedules or differing inference strategies make it hard to draw conclusions about what really is driving the progress in the field. &#10;In contrast, GHOST does not rely on complex procedures but combines lightweight motion and spiced-up appearance cues in a simple yet strong TbD tracker that only requires little training data. &#10;&#10;Person Re-Identification and Domain Adaptation.&#10;In contrast to the tracking domain, the goal of person reID is to retrieve person bounding boxes from a large gallery set that show the same person as a given query image based on appearance cues. However, state-of-the-art reID models tend to significantly drop in performance when evaluated on out-of-domain samples, i.e., samples coming from other datasets \cite{DBLP:conf/cvpr/ChoiKJPK21,DBLP:conf/cvpr/ZhaoZYLLLS21,DBLP:conf/eccv/ZhuangWXZZWAT20}. As during application, person reID models are applied to different cameras, several approaches on cross-dataset evaluation emerged that transfer the knowledge from a given source, i.e., training to a given target, i.e., test domain utilizing domain adaptation (DA) \cite{DBLP:conf/cvpr/ChoiKJPK21,DBLP:conf/cvpr/ZhaoZYLLLS21,DBLP:conf/eccv/ZhuangWXZZWAT20}. DA often relies on adapting Batch Normalization (BN) statistics to account for distribution shifts between different domains.&#10;The weight matrix and the BN statistics store label and domain-related knowledge, respectively \cite{DBLP:conf/iclr/LiWS0H17}. &#10;To update the latter, the statistics of BN layers can be updated, e.g., by taking the mean and variance of all target domain images \cite{DBLP:conf/iclr/LiWS0H17}, by re-training using pseudo labels \cite{DBLP:conf/cvpr/ChangYSKH19} or combining train and test datatset statistics \cite{DBLP:conf/nips/SchneiderRE0BB20}.&#10;Apart from the statistics, the learned parameters $\beta$ and $\gamma$ can also be updated \cite{DBLP:conf/iclr/WangSLOD21}. &#10;An approach similar to ours but for classification \cite{DBLP:journals/corr/abs-2006-10963} updates BN statistics during test time in a batch-wise manner.&#10;Inspired by those recent advances, we enhance our appearance model to be better suited for MOT using a simple on-the-fly domain adaptation approach. This directly adapts the model's learned training dataset statistics (source) to the sequences (targets).&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1602.00763" label="1602.00763">
-        <attvalues>
-          <attvalue for="0" value="Simple Online and Realtime Tracking" />
-          <attvalue for="1" value="  This paper explores a pragmatic approach to multiple object tracking where&#10;the main focus is to associate objects efficiently for online and realtime&#10;applications. To this end, detection quality is identified as a key factor&#10;influencing tracking performance, where changing the detector can improve&#10;tracking by up to 18.9%. Despite only using a rudimentary combination of&#10;familiar techniques such as the Kalman Filter and Hungarian algorithm for the&#10;tracking components, this approach achieves an accuracy comparable to&#10;state-of-the-art online trackers. Furthermore, due to the simplicity of our&#10;tracking method, the tracker updates at a rate of 260 Hz which is over 20x&#10;faster than other state-of-the-art trackers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.14651" label="2206.14651">
-        <attvalues>
-          <attvalue for="0" value="BoT-SORT: Robust Associations Multi-Pedestrian Tracking" />
-          <attvalue for="1" value="  The goal of multi-object tracking (MOT) is detecting and tracking all the&#10;objects in a scene, while keeping a unique identifier for each object. In this&#10;paper, we present a new robust state-of-the-art tracker, which can combine the&#10;advantages of motion and appearance information, along with camera-motion&#10;compensation, and a more accurate Kalman filter state vector. Our new trackers&#10;BoT-SORT, and BoT-SORT-ReID rank first in the datasets of MOTChallenge [29, 11]&#10;on both MOT17 and MOT20 test sets, in terms of all the main MOT metrics: MOTA,&#10;IDF1, and HOTA. For MOT17: 80.5 MOTA, 80.2 IDF1, and 65.0 HOTA are achieved.&#10;The source code and the pre-trained models are available at&#10;https://github.com/NirAharon/BOT-SORT&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.09765" label="2309.09765">
-        <attvalues>
-          <attvalue for="0" value="Localization-Guided Track: A Deep Association Multi-Object Tracking&#10;  Framework Based on Localization Confidence of Detections" />
-          <attvalue for="1" value="  In currently available literature, no tracking-by-detection (TBD)&#10;paradigm-based tracking method has considered the localization confidence of&#10;detection boxes. In most TBD-based methods, it is considered that objects of&#10;low detection confidence are highly occluded and thus it is a normal practice&#10;to directly disregard such objects or to reduce their priority in matching. In&#10;addition, appearance similarity is not a factor to consider for matching these&#10;objects. However, in terms of the detection confidence fusing classification&#10;and localization, objects of low detection confidence may have inaccurate&#10;localization but clear appearance; similarly, objects of high detection&#10;confidence may have inaccurate localization or unclear appearance; yet these&#10;objects are not further classified. In view of these issues, we propose&#10;Localization-Guided Track (LG-Track). Firstly, localization confidence is&#10;applied in MOT for the first time, with appearance clarity and localization&#10;accuracy of detection boxes taken into account, and an effective deep&#10;association mechanism is designed; secondly, based on the classification&#10;confidence and localization confidence, a more appropriate cost matrix can be&#10;selected and used; finally, extensive experiments have been conducted on MOT17&#10;and MOT20 datasets. The results show that our proposed method outperforms the&#10;compared state-of-art tracking methods. For the benefit of the community, our&#10;code has been made publicly at https://github.com/mengting2023/LG-Track.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.07402" label="1703.07402">
-        <attvalues>
-          <attvalue for="0" value="Simple Online and Realtime Tracking with a Deep Association Metric" />
-          <attvalue for="1" value="  Simple Online and Realtime Tracking (SORT) is a pragmatic approach to&#10;multiple object tracking with a focus on simple, effective algorithms. In this&#10;paper, we integrate appearance information to improve the performance of SORT.&#10;Due to this extension we are able to track objects through longer periods of&#10;occlusions, effectively reducing the number of identity switches. In spirit of&#10;the original framework we place much of the computational complexity into an&#10;offline pre-training stage where we learn a deep association metric on a&#10;large-scale person re-identification dataset. During online application, we&#10;establish measurement-to-track associations using nearest neighbor queries in&#10;visual appearance space. Experimental evaluation shows that our extensions&#10;reduce the number of identity switches by 45%, achieving overall competitive&#10;performance at high frame rates.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.07071" label="1903.07071">
-        <attvalues>
-          <attvalue for="0" value="Bag of Tricks and A Strong Baseline for Deep Person Re-identification" />
-          <attvalue for="1" value="  This paper explores a simple and efficient baseline for person&#10;re-identification (ReID). Person re-identification (ReID) with deep neural&#10;networks has made progress and achieved high performance in recent years.&#10;However, many state-of-the-arts methods design complex network structure and&#10;concatenate multi-branch features. In the literature, some effective training&#10;tricks are briefly appeared in several papers or source codes. This paper will&#10;collect and evaluate these effective training tricks in person ReID. By&#10;combining these tricks together, the model achieves 94.5% rank-1 and 85.9% mAP&#10;on Market1501 with only using global features. Our codes and models are&#10;available at https://github.com/michuanhaohao/reid-strong-baseline.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.11813" label="2302.11813">
-        <attvalues>
-          <attvalue for="0" value="Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification" />
-          <attvalue for="1" value="  Motion-based association for Multi-Object Tracking (MOT) has recently&#10;re-achieved prominence with the rise of powerful object detectors. Despite&#10;this, little work has been done to incorporate appearance cues beyond simple&#10;heuristic models that lack robustness to feature degradation. In this paper, we&#10;propose a novel way to leverage objects' appearances to adaptively integrate&#10;appearance matching into existing high-performance motion-based methods.&#10;Building upon the pure motion-based method OC-SORT, we achieve 1st place on&#10;MOT20 and 2nd place on MOT17 with 63.9 and 64.9 HOTA, respectively. We also&#10;achieve 61.3 HOTA on the challenging DanceTrack benchmark as a new&#10;state-of-the-art even compared to more heavily-designed methods. The code and&#10;models are available at \url{https://github.com/GerardMaggiolino/Deep-OC-SORT}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10; Motion-based Multi-Object Tracking.&#10;Given the rapid improvement of object detectors, many modern end-to-end MOT models still underperform against classic motion model-based tracking algorithms. The Kalman filter~\cite{kalman1960contributions} is the foundation of the most famous line of tracking-by-detection methods. Among this line of work, SORT~\cite{bewley2016simple} uses a linear motion assumption to associate tracks by IoU. ByteTrack~\cite{bytetrack} is recently proposed to fix missing predictions by using low-confidence candidates in association, achieving good performance by balancing the detection quality and tracking confidence. More recently, OC-SORT~\cite{cao2022observation} improves the robustness of tracking in non-linear motion scenarios and relieves the influence from object occlusion or disappearance by more heavily relying on detections directly. &#10;&#10;Appearance-based Multi-Object Tracking.&#10;Visual identification is a straightforward cue to associate targets over time. DeepSORT~\cite{deepsort} is one of the earliest to use deep visual features for object association. Since then more methods ~\cite{zhang2021fairmot,pang2021quasi,cao2022track} have improved upon integrating visual information by training discriminative appearance models in an end-to-end manner. More recently, the rise of transformers~\cite{vaswani2017attention} has started another wave of using appearance for multi-object tracking, where the task of object association is modeled as a query matching problem~\cite{zeng2021motr,sun2020transtrack,meinhardt2021trackformer,cao2022track}. However, appearance-based methods are observed to be less effective when the objects of interest have similar appearance~\cite{sun2021dancetrack} or are occluded~\cite{dendorfer2020mot20,sun2021dancetrack}. Despite having more complicated architectures, these methods fail to outperform simple motion association algorithms that leverage strong detectors. Some recent attempts to add appearance cues~\cite{botsort,strongsort} to motion-based methods use simple moving averages for appearance embedding updates, achieving moderate success. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1603.00831" label="1603.00831">
-        <attvalues>
-          <attvalue for="0" value="MOT16: A Benchmark for Multi-Object Tracking" />
-          <attvalue for="1" value="  Standardized benchmarks are crucial for the majority of computer vision&#10;applications. Although leaderboards and ranking tables should not be&#10;over-claimed, benchmarks often provide the most objective measure of&#10;performance and are therefore important guides for reseach.&#10;  Recently, a new benchmark for Multiple Object Tracking, MOTChallenge, was&#10;launched with the goal of collecting existing and new data and creating a&#10;framework for the standardized evaluation of multiple object tracking methods.&#10;The first release of the benchmark focuses on multiple people tracking, since&#10;pedestrians are by far the most studied object in the tracking community. This&#10;paper accompanies a new release of the MOTChallenge benchmark. Unlike the&#10;initial release, all videos of MOT16 have been carefully annotated following a&#10;consistent protocol. Moreover, it not only offers a significant increase in the&#10;number of labeled boxes, but also provides multiple object classes beside&#10;pedestrians and the level of visibility for every single object of interest.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.09003" label="2003.09003">
-        <attvalues>
-          <attvalue for="0" value="MOT20: A benchmark for multi object tracking in crowded scenes" />
-          <attvalue for="1" value="  Standardized benchmarks are crucial for the majority of computer vision&#10;applications. Although leaderboards and ranking tables should not be&#10;over-claimed, benchmarks often provide the most objective measure of&#10;performance and are therefore important guides for research. The benchmark for&#10;Multiple Object Tracking, MOTChallenge, was launched with the goal to establish&#10;a standardized evaluation of multiple object tracking methods. The challenge&#10;focuses on multiple people tracking, since pedestrians are well studied in the&#10;tracking community, and precise tracking and detection has high practical&#10;relevance. Since the first release, MOT15, MOT16, and MOT17 have tremendously&#10;contributed to the community by introducing a clean dataset and precise&#10;framework to benchmark multi-object trackers. In this paper, we present our&#10;MOT20benchmark, consisting of 8 new sequences depicting very crowded&#10;challenging scenes. The benchmark was presented first at the 4thBMTT MOT&#10;Challenge Workshop at the Computer Vision and Pattern Recognition Conference&#10;(CVPR) 2019, and gives to chance to evaluate state-of-the-art methods for&#10;multiple object tracking when handling extremely crowded scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14690" label="2111.14690">
-        <attvalues>
-          <attvalue for="0" value="DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse&#10;  Motion" />
-          <attvalue for="1" value="  A typical pipeline for multi-object tracking (MOT) is to use a detector for&#10;object localization, and following re-identification (re-ID) for object&#10;association. This pipeline is partially motivated by recent progress in both&#10;object detection and re-ID, and partially motivated by biases in existing&#10;tracking datasets, where most objects tend to have distinguishing appearance&#10;and re-ID models are sufficient for establishing associations. In response to&#10;such bias, we would like to re-emphasize that methods for multi-object tracking&#10;should also work when object appearance is not sufficiently discriminative. To&#10;this end, we propose a large-scale dataset for multi-human tracking, where&#10;humans have similar appearance, diverse motion and extreme articulation. As the&#10;dataset contains mostly group dancing videos, we name it &quot;DanceTrack&quot;. We&#10;expect DanceTrack to provide a better platform to develop more MOT algorithms&#10;that rely less on visual discrimination and depend more on motion analysis. We&#10;benchmark several state-of-the-art trackers on our dataset and observe a&#10;significant performance drop on DanceTrack when compared against existing&#10;benchmarks. The dataset, project code and competition server are released at:&#10;\url{https://github.com/DanceTrack}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;{ Multi-object tracking datasets.}&#10;Many multi-object tracking datasets have been proposed for different scenarios. Similar to our proposed dataset, many existing datasets focus on human tracking. PETS2009~\cite{PETS2009} dataset is one of the earliest in this area. The more recent MOT15~\cite{MOT15}, MOT17~\cite{MOT16} and MOT20~\cite{MOT20} datasets are all popular in this community. These datasets are limited in the aspects of undistinguished appearance and diverse motion. For example, MOT17 contains only a handful of videos and scenarios. Even MOT20 increases the density of objects and emphasizes the occlusion among them, the movements of objects are very regular and they still have distinguishable appearances. Association by pure appearance matching~\cite{quasidense} could easily make success on these datasets and we will show that given the perfect detector, the tracking problem on these datasets can be solved by a very naive association strategy, in Section~\ref{sec:oracle}.&#10;&#10;Besides, many other datasets are proposed for diverse objectives, e.g., WILDTRACK~\cite{wildtrack} for multi-camera tracking, Youtube-VIS~\cite{youtubevos} for video instance segmentation and tracking. With the increasing attraction of autonomous driving, some datasets are specifically built where the objects of interest are vehicles and pedestrians. KITTI~\cite{KITTI} is one of the earliest large-scale multi-object tracking datasets for driving scenarios. More recently, BDD100K~\cite{BDD}, Waymo~\cite{waymo} and KITTI360~\cite{KITTI360} are made available to the public, still focusing on autonomous driving scenarios but providing much larger scale data than KITTI. With the limitation of lanes and traffic rules, the motion patterns of objects in these datasets are even more regular than those focusing on only moving people. There are many datasets focusing on more diverse object categories than persons and vehicles. The ImageNet-Vid~\cite{ImageNet} provides trajectory annotations for 30 object categories in over 1000 videos and TAO~\cite{TAO} annotates even 833 object categories to study object tracking on long-tailed distribution.&#10;&#10;{ Tracking by matching appearance.}&#10;In the recent development of multi-object tracking, appearance similarity serves as the dominant cue in many popular methods. &#10;For example, &#10;JDE~\cite{JDE} and FairMOT~\cite{FairMOT} learn object localization and appearance embedding using a shared backbone for better appearance representation. QDTrack~\cite{quasidense} designs a contrastive training paradigm and dense localization for object detection and uses highly sensitive appearance comparison to match objects across frames. &#10;More recently, with the new focus of applying transformers~\cite{vaswani2017attention} in vision tasks, TransTrack~\cite{Transtrack}, TrackFormer~\cite{Trackformer} and MOTR~\cite{MOTR} make attempts to leverage the attention mechanism in tracking objects in videos. In these works, the features of previous tracklets are passed to the following frames as the query to associate the same objects across frames. The appearance information contained in the query is critical to keep tracklet consistency. &#10;&#10;Although the rise of deep-learning model brings much more powerful visual representations than ever before, we still witness the failure of appearance matching in many real-world situations and expect to improve the tracking performance by taking other cues into account. &#10;&#10;{ Motion analysis in object tracking.} &#10;The displacement of objects-of-interest provides important cues for object tracking. Tracking objects by estimating their motions has inspired a line of researches. These tracking algorithms mainly follow the tracking-by-detection paradigm. &#10;Sequential analysis tools such as Particle filter~\cite{particle1, particle2} and Kalman filter~\cite{kalman1960new} are found efficient in such applications, for example, SORT~\cite{SORT} is developed on the Kalman filter motion model. &#10;Even though motion analysis has been used in many object tracking methods~\cite{JDE,FairMOT,bytetrack}, all these methods can only handle simple linear motion pattern and provide limited help in more complicated situations. Furthermore, as deep networks bring the revolutionary ability to extract high-quality visual features, DeepSORT~\cite{DeepSORT} tries to combine deep visual features and motion models to gain performance gain. Since then, motion-based object tracker has shown weak competitiveness and many focuses are towards appearance cues. &#10;&#10;However, we argue that a more comprehensive and intelligent tracking algorithm should pay more attention to motion analysis since appearance is not always reliable. &#10; " />
-        </attvalues>
-      </node>
-      <node id="1909.12605" label="1909.12605">
-        <attvalues>
-          <attvalue for="0" value="Towards Real-Time Multi-Object Tracking" />
-          <attvalue for="1" value="  Modern multiple object tracking (MOT) systems usually follow the&#10;\emph{tracking-by-detection} paradigm. It has 1) a detection model for target&#10;localization and 2) an appearance embedding model for data association. Having&#10;the two models separately executed might lead to efficiency problems, as the&#10;running time is simply a sum of the two steps without investigating potential&#10;structures that can be shared between them. Existing research efforts on&#10;real-time MOT usually focus on the association step, so they are essentially&#10;real-time association methods but not real-time MOT system. In this paper, we&#10;propose an MOT system that allows target detection and appearance embedding to&#10;be learned in a shared model. Specifically, we incorporate the appearance&#10;embedding model into a single-shot detector, such that the model can&#10;simultaneously output detections and the corresponding embeddings. We further&#10;propose a simple and fast association method that works in conjunction with the&#10;joint model. In both components the computation cost is significantly reduced&#10;compared with former MOT systems, resulting in a neat and fast baseline for&#10;future follow-ups on real-time MOT algorithm design. To our knowledge, this&#10;work reports the first (near) real-time MOT system, with a running speed of 22&#10;to 40 FPS depending on the input resolution. Meanwhile, its tracking accuracy&#10;is comparable to the state-of-the-art trackers embodying separate detection and&#10;embedding (SDE) learning ($64.4\%$ MOTA \vs $66.1\%$ MOTA on MOT-16 challenge).&#10;Code and models are available at&#10;\url{https://github.com/Zhongdao/Towards-Realtime-MOT}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.01850" label="1604.01850">
-        <attvalues>
-          <attvalue for="0" value="Joint Detection and Identification Feature Learning for Person Search" />
-          <attvalue for="1" value="  Existing person re-identification benchmarks and methods mainly focus on&#10;matching cropped pedestrian images between queries and candidates. However, it&#10;is different from real-world scenarios where the annotations of pedestrian&#10;bounding boxes are unavailable and the target person needs to be searched from&#10;a gallery of whole scene images. To close the gap, we propose a new deep&#10;learning framework for person search. Instead of breaking it down into two&#10;separate tasks---pedestrian detection and person re-identification, we jointly&#10;handle both aspects in a single convolutional neural network. An Online&#10;Instance Matching (OIM) loss function is proposed to train the network&#10;effectively, which is scalable to datasets with numerous identities. To&#10;validate our approach, we collect and annotate a large-scale benchmark dataset&#10;for person search. It contains 18,184 images, 8,432 identities, and 96,143&#10;pedestrian bounding boxes. Experiments show that our framework outperforms&#10;other separate approaches, and the proposed OIM loss function converges much&#10;faster and better than the conventional Softmax loss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.01888" label="2004.01888">
-        <attvalues>
-          <attvalue for="0" value="FairMOT: On the Fairness of Detection and Re-Identification in Multiple&#10;  Object Tracking" />
-          <attvalue for="1" value="  Multi-object tracking (MOT) is an important problem in computer vision which&#10;has a wide range of applications. Formulating MOT as multi-task learning of&#10;object detection and re-ID in a single network is appealing since it allows&#10;joint optimization of the two tasks and enjoys high computation efficiency.&#10;However, we find that the two tasks tend to compete with each other which need&#10;to be carefully addressed. In particular, previous works usually treat re-ID as&#10;a secondary task whose accuracy is heavily affected by the primary detection&#10;task. As a result, the network is biased to the primary detection task which is&#10;not fair to the re-ID task. To solve the problem, we present a simple yet&#10;effective approach termed as FairMOT based on the anchor-free object detection&#10;architecture CenterNet. Note that it is not a naive combination of CenterNet&#10;and re-ID. Instead, we present a bunch of detailed designs which are critical&#10;to achieve good tracking results by thorough empirical studies. The resulting&#10;approach achieves high accuracy for both detection and tracking. The approach&#10;outperforms the state-of-the-art methods by a large margin on several public&#10;datasets. The source code and pre-trained models are released at&#10;https://github.com/ifzhang/FairMOT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.05625" label="1903.05625">
-        <attvalues>
-          <attvalue for="0" value="Tracking without bells and whistles" />
-          <attvalue for="1" value="  The problem of tracking multiple objects in a video sequence poses several&#10;challenging tasks. For tracking-by-detection, these include object&#10;re-identification, motion prediction and dealing with occlusions. We present a&#10;tracker (without bells and whistles) that accomplishes tracking without&#10;specifically targeting any of these tasks, in particular, we perform no&#10;training or optimization on tracking data. To this end, we exploit the bounding&#10;box regression of an object detector to predict the position of an object in&#10;the next frame, thereby converting a detector into a Tracktor. We demonstrate&#10;the potential of Tracktor and provide a new state-of-the-art on three&#10;multi-object tracking benchmarks by extending it with a straightforward&#10;re-identification and camera motion compensation. We then perform an analysis&#10;on the performance and failure cases of several state-of-the-art tracking&#10;methods in comparison to our Tracktor. Surprisingly, none of the dedicated&#10;tracking methods are considerably better in dealing with complex tracking&#10;scenarios, namely, small and occluded objects or missing detections. However,&#10;our approach tackles most of the easy tracking scenarios. Therefore, we&#10;motivate our approach as a new tracking paradigm and point out promising future&#10;research directions. Overall, Tracktor yields superior tracking performance&#10;than any current tracking method and our analysis exposes remaining and&#10;unsolved tracking challenges to inspire future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.17170" label="2310.17170">
-        <attvalues>
-          <attvalue for="0" value="DecoderTracker: Decoder-Only Method for Multiple-Object Tracking" />
-          <attvalue for="1" value="  Decoder-only models, such as GPT, have demonstrated superior performance in&#10;many areas compared to traditional encoder-decoder structure transformer&#10;models. Over the years, end-to-end models based on the traditional transformer&#10;structure, like MOTR, have achieved remarkable performance in multi-object&#10;tracking. However, the significant computational resource consumption of these&#10;models leads to less friendly inference speeds and training times. To address&#10;these issues, this paper attempts to construct a lightweight Decoder-only&#10;model: DecoderTracker for end-to-end multi-object tracking. Specifically,&#10;drawing on some real-time detection models, we have developed an image feature&#10;extraction network which can efficiently extract features from images to&#10;replace the encoder structure. In addition to minor innovations in the network,&#10;we analyze the potential reasons for the slow training of MOTR-like models and&#10;propose an effective training strategy to mitigate the issue of prolonged&#10;training times. On the DanceTrack dataset, without any bells and whistles,&#10;DecoderTracker's tracking performance slightly surpasses that of MOTR, with&#10;approximately twice the inference speed. Furthermore, DecoderTracker requires&#10;significantly less training time compared to MOTR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.17296" label="2402.17296">
-        <attvalues>
-          <attvalue for="0" value="Learning Exposure Correction in Dynamic Scenes" />
-          <attvalue for="1" value="  Exposure correction aims to enhance visual data suffering from improper&#10;exposures, which can greatly improve satisfactory visual effects. However,&#10;previous methods mainly focus on the image modality, and the video counterpart&#10;is less explored in the literature. Directly applying prior image-based methods&#10;to videos results in temporal incoherence with low visual quality. Through&#10;thorough investigation, we find that the development of relevant communities is&#10;limited by the absence of a benchmark dataset. Therefore, in this paper, we&#10;construct the first real-world paired video dataset, including both&#10;underexposure and overexposure dynamic scenes. To achieve spatial alignment, we&#10;utilize two DSLR cameras and a beam splitter to simultaneously capture improper&#10;and normal exposure videos. Additionally, we propose an end-to-end video&#10;exposure correction network, in which a dual-stream module is designed to deal&#10;with both underexposure and overexposure factors, enhancing the illumination&#10;based on Retinex theory. The extensive experiments based on various metrics and&#10;user studies demonstrate the significance of our dataset and the effectiveness&#10;of our method. The code and dataset are available at&#10;https://github.com/kravrolens/VECNet.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The images or videos taken across various scenarios may not always be ideal due to the changeable lighting conditions, which face underexposure and overexposure problems and yield unsatisfactory visual effects. How to improve inappropriately exposed visual data is gradually attracting the attention of researchers in the multimedia community.&#10;Due to the diversity of scenes and lighting conditions, the exposure correction process of visual data is particularly complicated, and overexposure and underexposure may exist at the same time (mixed exposure), which further enlarges the discrepancies of operations. Thus, many exposure correction methods have been proposed to handle this challenge problem automatically by utilizing deep learning technologies.&#10;Correcting underexposure and overexposure to normal exposures are much different from each other, which leads to large discrepancies in training an image-based multi-exposure correction network. This challenge is further compounded when dealing with videos captured in dynamic scenes.&#10;&#10;In recent years, fueled by various paired image exposure dataset~\cite{cai2018learning,afifi2021learning}, several methods~\cite{huang2022ecl,huang2022deep,wang2023decoupling,wang2022local,yao2023generalized} that focus on learning image exposure correction have been proposed. However, these methods generally do not handle videos of dynamic scenes, as directly applying them in a frame-by-frame manner often results in inter-frame exposure inconsistency with low visual quality~\cite{lv2018mbllen,li2021low}, as shown in the below sub-figures of Fig.~\ref{fig:intro}. &#10;Some methods~\cite{wei2018deep, wang2021seeing, fu2023dancing} focus on enhancing underexposed videos taken from low-light dynamic scenes. However, these methods can only achieve a single exposure correction. Due to the discrepant representations between underexposure and overexposure, the correction procedures differ greatly from each other, as shown in Fig.~\ref{fig:lux}(a). Training a mixture of multi-exposure data in their models often leads to poor performance across different exposure levels, rendering these methods unsuitable for practical applications.&#10;&#10;To overcome the above challenges, in this work, we focus on the multi-exposure correction presented in the real-world videos for the first time. As of our current understanding, there is a noticeable absence of well-studied works or benchmark datasets in this domain.&#10;On the one hand, we aim to collect a high-quality video dataset with both underexposure and overexposure, and the main difficulties are as follows. &#10;First, collecting paired video datasets requires capturing two videos, one under abnormal exposure conditions and another under normal light of the same dynamic scene and camera motion. &#10;Second, it requires accurate alignment of each pair of corresponding frames in the two videos in both spatial and temporal dimensions.&#10;To address the above issues, we first design a two-camera system with a beam splitter to ensure no parallax between the two cameras with different aperture and ISO parameter settings. Two DSLR cameras simultaneously capture improper and normal exposure videos. Then we perform precise alignment on the captured pairs to make them aligned with each other. Furthermore, the well-exposed videos are manually rendered by professional photographers and then serve as the ground truth of the dataset. &#10;Finally, we construct a new dataset of 119 high-quality videos named DIME, which stands for &quot;Dynamic scenes In Multiple Exposure&quot;. It contains diverse real-world scenes, camera and object motions, and each paired video is accurately aligned along the spatial and temporal dimensions.&#10;&#10;On the other hand, since there are varying degrees of improper exposure in the constructed dataset, existing methods prove inadequate for deployment. Thus, on the other hand, we propose an end-to-end Video Exposure Correction Network (VECNet) to enhance the videos with multiple and improper exposures.&#10;Specifically, to adaptively learn the overexposed and underexposed representations, we formulate a dual-stream strategy to enhance various illumination components based on Retinex theory. For complementary reflectance component learning, we design a novel multi-frame alignment module that aligns neighboring frames into the middle one to effectively obtain comprehensive feature representations.&#10;In brief, our contributions can be summarized as follows:&#10;&#10;(1) We construct the first high-quality paired video exposure correction dataset for dynamic real-world scenes with multiple exposures, camera and object motions, and precise spatial alignment.&#10;&#10;(2) We propose an effective exposure correction network based on Retinex theory to enhance overexposed and underexposed videos.&#10;&#10;(3) We conduct extensive experiments to demonstrate the superiority of our dataset and method.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Engineering, Machine Learning, Physics, Video Exposure Correction, Computer Science, Image Processing Algorithms, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2003.11596" label="2003.11596">
-        <attvalues>
-          <attvalue for="0" value="Learning Multi-Scale Photo Exposure Correction" />
-          <attvalue for="1" value="  Capturing photographs with wrong exposures remains a major source of errors&#10;in camera-based imaging. Exposure problems are categorized as either: (i)&#10;overexposed, where the camera exposure was too long, resulting in bright and&#10;washed-out image regions, or (ii) underexposed, where the exposure was too&#10;short, resulting in dark regions. Both under- and overexposure greatly reduce&#10;the contrast and visual appeal of an image. Prior work mainly focuses on&#10;underexposed images or general image enhancement. In contrast, our proposed&#10;method targets both over- and underexposure errors in photographs. We formulate&#10;the exposure correction problem as two main sub-problems: (i) color enhancement&#10;and (ii) detail enhancement. Accordingly, we propose a coarse-to-fine deep&#10;neural network (DNN) model, trainable in an end-to-end manner, that addresses&#10;each sub-problem separately. A key aspect of our solution is a new dataset of&#10;over 24,000 images exhibiting the broadest range of exposure values to date&#10;with a corresponding properly exposed image. Our method achieves results on par&#10;with existing state-of-the-art methods on underexposed images and yields&#10;significant improvements for images suffering from overexposure errors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.13783" label="2308.13783">
-        <attvalues>
-          <attvalue for="0" value="Generalized Lightness Adaptation with Channel Selective Normalization" />
-          <attvalue for="1" value="  Lightness adaptation is vital to the success of image processing to avoid&#10;unexpected visual deterioration, which covers multiple aspects, e.g., low-light&#10;image enhancement, image retouching, and inverse tone mapping. Existing methods&#10;typically work well on their trained lightness conditions but perform poorly in&#10;unknown ones due to their limited generalization ability. To address this&#10;limitation, we propose a novel generalized lightness adaptation algorithm that&#10;extends conventional normalization techniques through a channel filtering&#10;design, dubbed Channel Selective Normalization (CSNorm). The proposed CSNorm&#10;purposely normalizes the statistics of lightness-relevant channels and keeps&#10;other channels unchanged, so as to improve feature generalization and&#10;discrimination. To optimize CSNorm, we propose an alternating training strategy&#10;that effectively identifies lightness-relevant channels. The model equipped&#10;with our CSNorm only needs to be trained on one lightness condition and can be&#10;well generalized to unknown lightness conditions. Experimental results on&#10;multiple benchmark datasets demonstrate the effectiveness of CSNorm in&#10;enhancing the generalization ability for the existing lightness adaptation&#10;methods. Code is available at https://github.com/mdyao/CSNorm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2104.10729" label="2104.10729">
-        <attvalues>
-          <attvalue for="0" value="Low-Light Image and Video Enhancement Using Deep Learning: A Survey" />
-          <attvalue for="1" value="  Low-light image enhancement (LLIE) aims at improving the perception or&#10;interpretability of an image captured in an environment with poor illumination.&#10;Recent advances in this area are dominated by deep learning-based solutions,&#10;where many learning strategies, network structures, loss functions, training&#10;data, etc. have been employed. In this paper, we provide a comprehensive survey&#10;to cover various aspects ranging from algorithm taxonomy to open issues. To&#10;examine the generalization of existing methods, we propose a low-light image&#10;and video dataset, in which the images and videos are taken by different mobile&#10;phones' cameras under diverse illumination conditions. Besides, for the first&#10;time, we provide a unified online platform that covers many popular LLIE&#10;methods, of which the results can be produced through a user-friendly web&#10;interface. In addition to qualitative and quantitative evaluation of existing&#10;methods on publicly available and our proposed datasets, we also validate their&#10;performance in face detection in the dark.This survey together with the&#10;proposed dataset and online platform could serve as a reference source for&#10;future study and promote the development of this research field. The proposed&#10;platform and dataset as well as the collected methods, datasets, and evaluation&#10;metrics are publicly available and will be regularly updated.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.04560" label="1808.04560">
-        <attvalues>
-          <attvalue for="0" value="Deep Retinex Decomposition for Low-Light Enhancement" />
-          <attvalue for="1" value="  Retinex model is an effective tool for low-light image enhancement. It&#10;assumes that observed images can be decomposed into the reflectance and&#10;illumination. Most existing Retinex-based methods have carefully designed&#10;hand-crafted constraints and parameters for this highly ill-posed&#10;decomposition, which may be limited by model capacity when applied in various&#10;scenes. In this paper, we collect a LOw-Light dataset (LOL) containing&#10;low/normal-light image pairs and propose a deep Retinex-Net learned on this&#10;dataset, including a Decom-Net for decomposition and an Enhance-Net for&#10;illumination adjustment. In the training process for Decom-Net, there is no&#10;ground truth of decomposed reflectance and illumination. The network is learned&#10;with only key constraints including the consistent reflectance shared by paired&#10;low/normal-light images, and the smoothness of illumination. Based on the&#10;decomposition, subsequent lightness enhancement is conducted on illumination by&#10;an enhancement network called Enhance-Net, and for joint denoising there is a&#10;denoising operation on reflectance. The Retinex-Net is end-to-end trainable, so&#10;that the learned decomposition is by nature good for lightness adjustment.&#10;Extensive experiments demonstrate that our method not only achieves visually&#10;pleasing quality for low-light enhancement but also provides a good&#10;representation of image decomposition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.08936" label="2404.08936">
-        <attvalues>
-          <attvalue for="0" value="Shifting Spotlight for Co-supervision: A Simple yet Efficient&#10;  Single-branch Network to See Through Camouflage" />
-          <attvalue for="1" value="  Efficient and accurate camouflaged object detection (COD) poses a challenge&#10;in the field of computer vision. Recent approaches explored the utility of edge&#10;information for network co-supervision, achieving notable advancements.&#10;However, these approaches introduce an extra branch for complex edge&#10;extraction, complicate the model architecture and increases computational&#10;demands. Addressing this issue, our work replicates the effect that animal's&#10;camouflage can be easily revealed under a shifting spotlight, and leverages it&#10;for network co-supervision to form a compact yet efficient single-branch&#10;network, the Co-Supervised Spotlight Shifting Network (CS$^3$Net). The&#10;spotlight shifting strategy allows CS$^3$Net to learn additional prior within a&#10;single-branch framework, obviating the need for resource demanding multi-branch&#10;design. To leverage the prior of spotlight shifting co-supervision, we propose&#10;Shadow Refinement Module (SRM) and Projection Aware Attention (PAA) for feature&#10;refinement and enhancement. To ensure the continuity of multi-scale features&#10;aggregation, we utilize the Extended Neighbor Connection Decoder (ENCD) for&#10;generating the final predictions. Empirical evaluations on public datasets&#10;confirm that our CS$^3$Net offers an optimal balance between efficiency and&#10;performance: it accomplishes a 32.13% reduction in Multiply-Accumulate (MACs)&#10;operations compared to leading efficient COD models, while also delivering&#10;superior performance.&#10;" />
-          <attvalue for="2" value="&#10;Camouflage is an evolutionary adaptation that allows animals to evade predators’ attention by blending into the surroundings. This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}. Camouflaged Object Detection (COD) is particularly difficult because of the visually deceptive nature of camouflage. It often relies on the use of computationally intensive networks for accurate detection. Therefore, developing an efficient and effective COD method is of significant importance. &#10;The major challenge of precisely segmenting camouflaged targets lies in the blurred boundaries between them and the surroundings. Recent deep learning based methods \cite{edge_aware_SCRN,errnet,mutualGL,edge_reconstruion,edge_aware_mirrornet} effectively tackle this challenge by introducing an additional branch to integrate extra prior information (such as edge, depth, etc.). However, the additional branch substantially increases the number of parameters and computational demand, ultimately reducing the overall efficiency.&#10;&#10;To address the inefficiencies caused by multi-branch structures, we propose a spotlight shifting strategy that enables network co-supervision within a single-branch framework. This strategy is inspired by how varying light condition can make the objects' contours more distinct. As presented in Fig.~\ref{fig:teaser_spotlight}, the shifting light source leads to eye-catching changes on shadow projection, enhancing the visibility of the object. Our proposed spotlight shifting strategy mimics this effect by generating shadow maps with different spotlight positions, which are further used for co-supervision. This strategy allows the model to capture informative cues without extra branches.&#10;Our main goal is to develop a COD method that strikes the optimal performance-efficiency trade-off, as evidenced by Fig.~\ref{fig:scatter_chart}. Targeting at which, we present the Co-Supervised Spotlight Shifting Network (CS\(^3\)Net), a single-branch network that leverages spotlight shifting co-supervision for enhanced feature extraction. It integrates the proposed Projection Aware Attention (PAA) and Extended Neighbor Connection Decoder (ENCD) for feature refinement and precise prediction. Our contributions are summarized as follows:&#10;\begin{itemize}&#10; \item We propose spotlight shifting strategy for network co-supervision that enriches the model capability to discern camouflaged objects without introducing extra branches, providing a unique perspective in COD domain.&#10; \item Based on spotlight shifting strategy, a novel single-branch and efficient model, termed as CS\(^3\)Net is crafted for COD task. It integrates the proposed PAA, and ENCD to precisely enhance feature representations.&#10; \item CS\(^3\)Net achieves the optimal balance between model efficiency and performance, reducing MACs by 32.13\% compared to the most advanced efficient COD models, while also delivering superior performance.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Network Architecture, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.11392" label="2006.11392">
-        <attvalues>
-          <attvalue for="0" value="PraNet: Parallel Reverse Attention Network for Polyp Segmentation" />
-          <attvalue for="1" value="  Colonoscopy is an effective technique for detecting colorectal polyps, which&#10;are highly related to colorectal cancer. In clinical practice, segmenting&#10;polyps from colonoscopy images is of great importance since it provides&#10;valuable information for diagnosis and surgery. However, accurate polyp&#10;segmentation is a challenging task, for two major reasons: (i) the same type of&#10;polyps has a diversity of size, color and texture; and (ii) the boundary&#10;between a polyp and its surrounding mucosa is not sharp. To address these&#10;challenges, we propose a parallel reverse attention network (PraNet) for&#10;accurate polyp segmentation in colonoscopy images. Specifically, we first&#10;aggregate the features in high-level layers using a parallel partial decoder&#10;(PPD). Based on the combined feature, we then generate a global map as the&#10;initial guidance area for the following components. In addition, we mine the&#10;boundary cues using a reverse attention (RA) module, which is able to establish&#10;the relationship between areas and boundary cues. Thanks to the recurrent&#10;cooperation mechanism between areas and boundaries, our PraNet is capable of&#10;calibrating any misaligned predictions, improving the segmentation accuracy.&#10;Quantitative and qualitative evaluations on five challenging datasets across&#10;six metrics show that our PraNet improves the segmentation accuracy&#10;significantly, and presents a number of advantages in terms of&#10;generalizability, and real-time segmentation efficiency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.08468" label="2105.08468">
-        <attvalues>
-          <attvalue for="0" value="Progressively Normalized Self-Attention Network for Video Polyp&#10;  Segmentation" />
-          <attvalue for="1" value="  Existing video polyp segmentation (VPS) models typically employ convolutional&#10;neural networks (CNNs) to extract features. However, due to their limited&#10;receptive fields, CNNs can not fully exploit the global temporal and spatial&#10;information in successive video frames, resulting in false-positive&#10;segmentation results. In this paper, we propose the novel PNS-Net&#10;(Progressively Normalized Self-attention Network), which can efficiently learn&#10;representations from polyp videos with real-time speed (~140fps) on a single&#10;RTX 2080 GPU and no post-processing. Our PNS-Net is based solely on a basic&#10;normalized self-attention block, equipping with recurrence and CNNs entirely.&#10;Experiments on challenging VPS datasets demonstrate that the proposed PNS-Net&#10;achieves state-of-the-art performance. We also conduct extensive experiments to&#10;study the effectiveness of the channel split, soft-attention, and progressive&#10;learning strategy. We find that our PNS-Net works well under different&#10;settings, making it a promising solution to the VPS task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.14291" label="2203.14291">
-        <attvalues>
-          <attvalue for="0" value="Video Polyp Segmentation: A Deep Learning Perspective" />
-          <attvalue for="1" value="  We present the first comprehensive video polyp segmentation (VPS) study in&#10;the deep learning era. Over the years, developments in VPS are not moving&#10;forward with ease due to the lack of large-scale fine-grained segmentation&#10;annotations. To address this issue, we first introduce a high-quality&#10;frame-by-frame annotated VPS dataset, named SUN-SEG, which contains 158,690&#10;colonoscopy frames from the well-known SUN-database. We provide additional&#10;annotations with diverse types, i.e., attribute, object mask, boundary,&#10;scribble, and polygon. Second, we design a simple but efficient baseline,&#10;dubbed PNS+, consisting of a global encoder, a local encoder, and normalized&#10;self-attention (NS) blocks. The global and local encoders receive an anchor&#10;frame and multiple successive frames to extract long-term and short-term&#10;spatial-temporal representations, which are then progressively updated by two&#10;NS blocks. Extensive experiments show that PNS+ achieves the best performance&#10;and real-time inference speed (170fps), making it a promising solution for the&#10;VPS task. Third, we extensively evaluate 13 representative polyp/object&#10;segmentation models on our SUN-SEG dataset and provide attribute-based&#10;comparisons. Finally, we discuss several open issues and suggest possible&#10;research directions for the VPS community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.14133" label="2004.14133">
-        <attvalues>
-          <attvalue for="0" value="Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images" />
-          <attvalue for="1" value="  Coronavirus Disease 2019 (COVID-19) spread globally in early 2020, causing&#10;the world to face an existential health crisis. Automated detection of lung&#10;infections from computed tomography (CT) images offers a great potential to&#10;augment the traditional healthcare strategy for tackling COVID-19. However,&#10;segmenting infected regions from CT slices faces several challenges, including&#10;high variation in infection characteristics, and low intensity contrast between&#10;infections and normal tissues. Further, collecting a large amount of data is&#10;impractical within a short time period, inhibiting the training of a deep&#10;model. To address these challenges, a novel COVID-19 Lung Infection&#10;Segmentation Deep Network (Inf-Net) is proposed to automatically identify&#10;infected regions from chest CT slices. In our Inf-Net, a parallel partial&#10;decoder is used to aggregate the high-level features and generate a global map.&#10;Then, the implicit reverse attention and explicit edge-attention are utilized&#10;to model the boundaries and enhance the representations. Moreover, to alleviate&#10;the shortage of labeled data, we present a semi-supervised segmentation&#10;framework based on a randomly selected propagation strategy, which only&#10;requires a few labeled images and leverages primarily unlabeled data. Our&#10;semi-supervised framework can improve the learning ability and achieve a higher&#10;performance. Extensive experiments on our COVID-SemiSeg and real CT volumes&#10;demonstrate that the proposed Inf-Net outperforms most cutting-edge&#10;segmentation models and advances the state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.07054" label="2004.07054">
-        <attvalues>
-          <attvalue for="0" value="JCS: An Explainable COVID-19 Diagnosis System by Joint Classification&#10;  and Segmentation" />
-          <attvalue for="1" value="  Recently, the coronavirus disease 2019 (COVID-19) has caused a pandemic&#10;disease in over 200 countries, influencing billions of humans. To control the&#10;infection, identifying and separating the infected people is the most crucial&#10;step. The main diagnostic tool is the Reverse Transcription Polymerase Chain&#10;Reaction (RT-PCR) test. Still, the sensitivity of the RT-PCR test is not high&#10;enough to effectively prevent the pandemic. The chest CT scan test provides a&#10;valuable complementary tool to the RT-PCR test, and it can identify the&#10;patients in the early-stage with high sensitivity. However, the chest CT scan&#10;test is usually time-consuming, requiring about 21.5 minutes per case. This&#10;paper develops a novel Joint Classification and Segmentation (JCS) system to&#10;perform real-time and explainable COVID-19 chest CT diagnosis. To train our JCS&#10;system, we construct a large scale COVID-19 Classification and Segmentation&#10;(COVID-CS) dataset, with 144,167 chest CT images of 400 COVID-19 patients and&#10;350 uninfected cases. 3,855 chest CT images of 200 patients are annotated with&#10;fine-grained pixel-level labels of opacifications, which are increased&#10;attenuation of the lung parenchyma. We also have annotated lesion counts,&#10;opacification areas, and locations and thus benefit various diagnosis aspects.&#10;Extensive experiments demonstrate that the proposed JCS diagnosis system is&#10;very efficient for COVID-19 classification and segmentation. It obtains an&#10;average sensitivity of 95.0% and a specificity of 93.0% on the classification&#10;test set, and 78.5% Dice score on the segmentation test set of our COVID-CS&#10;dataset. The COVID-CS dataset and code are available at&#10;https://github.com/yuhuan-wu/JCS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.06485" label="2106.06485">
-        <attvalues>
-          <attvalue for="0" value="Pedestrian Attribute Recognition in Video Surveillance Scenarios Based&#10;  on View-attribute Attention Localization" />
-          <attvalue for="1" value="  Pedestrian attribute recognition in surveillance scenarios is still a&#10;challenging task due to the inaccurate localization of specific attributes. In&#10;this paper, we propose a novel view-attribute localization method based on&#10;attention (VALA), which utilizes view information to guide the recognition&#10;process to focus on specific attributes and attention mechanism to localize&#10;specific attribute-corresponding areas. Concretely, view information is&#10;leveraged by the view prediction branch to generate four view weights that&#10;represent the confidences for attributes from different views. View weights are&#10;then delivered back to compose specific view-attributes, which will participate&#10;and supervise deep feature extraction. In order to explore the spatial location&#10;of a view-attribute, regional attention is introduced to aggregate spatial&#10;information and encode inter-channel dependencies of the view feature.&#10;Subsequently, a fine attentive attribute-specific region is localized, and&#10;regional weights for the view-attribute from different spatial locations are&#10;gained by the regional attention. The final view-attribute recognition outcome&#10;is obtained by combining the view weights with the regional weights.&#10;Experiments on three wide datasets (RAP, RAPv2, and PA-100K) demonstrate the&#10;effectiveness of our approach compared with state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.02011" label="1807.02011">
-        <attvalues>
-          <attvalue for="0" value="Improving Unsupervised Defect Segmentation by Applying Structural&#10;  Similarity to Autoencoders" />
-          <attvalue for="1" value="  Convolutional autoencoders have emerged as popular methods for unsupervised&#10;defect segmentation on image data. Most commonly, this task is performed by&#10;thresholding a pixel-wise reconstruction error based on an $\ell^p$ distance.&#10;This procedure, however, leads to large residuals whenever the reconstruction&#10;encompasses slight localization inaccuracies around edges. It also fails to&#10;reveal defective regions that have been visually altered when intensity values&#10;stay roughly consistent. We show that these problems prevent these approaches&#10;from being applied to complex real-world scenarios and that it cannot be easily&#10;avoided by employing more elaborate architectures such as variational or&#10;feature matching autoencoders. We propose to use a perceptual loss function&#10;based on structural similarity which examines inter-dependencies between local&#10;image regions, taking into account luminance, contrast and structural&#10;information, instead of simply comparing single pixel values. It achieves&#10;significant performance gains on a challenging real-world dataset of&#10;nanofibrous materials and a novel dataset of two woven fabrics over the state&#10;of the art approaches for unsupervised defect segmentation that use pixel-wise&#10;reconstruction error metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.08536" label="1903.08536">
-        <attvalues>
-          <attvalue for="0" value="Segmentation-Based Deep-Learning Approach for Surface-Defect Detection" />
-          <attvalue for="1" value="  Automated surface-anomaly detection using machine learning has become an&#10;interesting and promising area of research, with a very high and direct impact&#10;on the application domain of visual inspection. Deep-learning methods have&#10;become the most suitable approaches for this task. They allow the inspection&#10;system to learn to detect the surface anomaly by simply showing it a number of&#10;exemplar images. This paper presents a segmentation-based deep-learning&#10;architecture that is designed for the detection and segmentation of surface&#10;anomalies and is demonstrated on a specific domain of surface-crack detection.&#10;The design of the architecture enables the model to be trained using a small&#10;number of samples, which is an important requirement for practical&#10;applications. The proposed model is compared with the related deep-learning&#10;methods, including the state-of-the-art commercial software, showing that the&#10;proposed approach outperforms the related methods on the specific domain of&#10;surface-crack detection. The large number of experiments also shed light on the&#10;required precision of the annotation, the number of required training samples&#10;and on the required computational cost. Experiments are performed on a newly&#10;created dataset based on a real-world quality control case and demonstrates&#10;that the proposed approach is able to learn on a small number of defected&#10;surfaces, using only approximately 25-30 defective training samples, instead of&#10;hundreds or thousands, which is usually the case in deep-learning applications.&#10;This makes the deep-learning method practical for use in industry where the&#10;number of available defective samples is limited. The dataset is also made&#10;publicly available to encourage the development and evaluation of new methods&#10;for surface-defect detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.03216" label="2111.03216">
-        <attvalues>
-          <attvalue for="0" value="Fast Camouflaged Object Detection via Edge-based Reversible&#10;  Re-calibration Network" />
-          <attvalue for="1" value="  Camouflaged Object Detection (COD) aims to detect objects with similar&#10;patterns (e.g., texture, intensity, colour, etc) to their surroundings, and&#10;recently has attracted growing research interest. As camouflaged objects often&#10;present very ambiguous boundaries, how to determine object locations as well as&#10;their weak boundaries is challenging and also the key to this task. Inspired by&#10;the biological visual perception process when a human observer discovers&#10;camouflaged objects, this paper proposes a novel edge-based reversible&#10;re-calibration network called ERRNet. Our model is characterized by two&#10;innovative designs, namely Selective Edge Aggregation (SEA) and Reversible&#10;Re-calibration Unit (RRU), which aim to model the visual perception behaviour&#10;and achieve effective edge prior and cross-comparison between potential&#10;camouflaged regions and background. More importantly, RRU incorporates diverse&#10;priors with more comprehensive information comparing to existing COD models.&#10;Experimental results show that ERRNet outperforms existing cutting-edge&#10;baselines on three COD datasets and five medical image segmentation datasets.&#10;Especially, compared with the existing top-1 model SINet, ERRNet significantly&#10;improves the performance by $\sim$6% (mean E-measure) with notably high speed&#10;(79.3 FPS), showing that ERRNet could be a general and robust solution for the&#10;COD task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.02613" label="2104.02613">
-        <attvalues>
-          <attvalue for="0" value="Mutual Graph Learning for Camouflaged Object Detection" />
-          <attvalue for="1" value="  Automatically detecting/segmenting object(s) that blend in with their&#10;surroundings is difficult for current models. A major challenge is that the&#10;intrinsic similarities between such foreground objects and background&#10;surroundings make the features extracted by deep model indistinguishable. To&#10;overcome this challenge, an ideal model should be able to seek valuable, extra&#10;clues from the given scene and incorporate them into a joint learning framework&#10;for representation co-enhancement. With this inspiration, we design a novel&#10;Mutual Graph Learning (MGL) model, which generalizes the idea of conventional&#10;mutual learning from regular grids to the graph domain. Specifically, MGL&#10;decouples an image into two task-specific feature maps -- one for roughly&#10;locating the target and the other for accurately capturing its boundary details&#10;-- and fully exploits the mutual benefits by recurrently reasoning their&#10;high-order relations through graphs. Importantly, in contrast to most mutual&#10;learning approaches that use a shared function to model all between-task&#10;interactions, MGL is equipped with typed functions for handling different&#10;complementary relations to maximize information interactions. Experiments on&#10;challenging datasets, including CHAMELEON, CAMO and COD10K, demonstrate the&#10;effectiveness of our MGL with superior performance to existing state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10; {Camouflaged Object Detection.} The camouflaged object detection (COD) task~\cite{mei2021Ming,aixuan_cod_sod21,zhai2021Mutual} has posed new challenges by pushing the boundaries of generic / salient object detection~\cite{redmon2016you,wei2020label,li2016saliency,luo2020webly,he2017mask,li2017multi,lin2017feature,liu2016ssd,lin2017focal,zhuge2021salient,zhao2019egnet,GateNet,Li_2018_ECCV,pang2020multi} to concealed objects blending in with their surroundings. Fan~et~al.~\cite{fan2020camouflaged} present the Search and Identification Net (SINet) to address this challenge by first roughly searching for camouflaged objects and then performing segmentation. Le~et~al.~\cite{ltnghia-CVIU2019} introduce the Anabranch Network (ANet) which incorporates classification information into representation learning. Yan~et~al.~\cite{yan2020} introduce MirrorNet to use both instance segmentation and adversarial attack for COD. The common idea behind these bio-inspired models is that exploring and integrating extra clues into representation learning can greatly outperform the conventional approaches for generic object detection (GOD) and salient object detection (SOD) ~\cite{zhou2020interactive,lin2017feature,he2017mask,zhao2017pyramid,liu2018picanet,huang2019mask,qin2019basnet,qin2021boundary,deng2021re}. Unlike prior works, our novelty is that we use a unified, graph-based model to simultaneously perform camouflaged object detection (COD) and the camouflaged object-aware edge extraction (COEE) by comprehensively reasoning about multi-level relations to boost performance for both tasks.&#10;&#10;{Graph Convolutional Networks.} GCNs are powerful tools for graph data analysis, which have given rise to many applications~\cite{Yang_2020_CVPR,Wei_2020_CVPR,Xie-SAGNN,luo2020hybrid,Mohamed_2020_CVPR,Zhang_2020_CVPR, xie2020region}. In the context of (generic/salient) object detection, GCNs are used to detect or segment 2D/3D objects in images, videos or point clouds~\cite{wang2019dynamic,wang2019graph}. In~\cite{Chen_2019_CVPR,li2020spatial}, the long-range context is modeled by graph convolution for semantic segmentation. Wu~et~al.~\cite{Wu_2020_CVPR} exploit the semantic relations and co-occurrence among objects and background with a bidirectional graph. Luo~et~al.~\cite{Luo2020CascadeGN} introduce a cascade graph model to exploit multi-scale, cross-modality information for salient object detection. In~\cite{Zhang_2020_CVPR}, an adaptive GCN model with attention graph clustering is introduced for co-saliency detection. For camouflaged object detection, we introduce two novel graph-based modules, RIGR and ECGR, to fully reason about complementary information of COD and COEE across different levels, which can better learn representations from image to overcome multiple challenges.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2307.03932" label="2307.03932">
-        <attvalues>
-          <attvalue for="0" value="Edge-Aware Mirror Network for Camouflaged Object Detection" />
-          <attvalue for="1" value="  Existing edge-aware camouflaged object detection (COD) methods normally&#10;output the edge prediction in the early stage. However, edges are important and&#10;fundamental factors in the following segmentation task. Due to the high visual&#10;similarity between camouflaged targets and the surroundings, edge prior&#10;predicted in early stage usually introduces erroneous foreground-background and&#10;contaminates features for segmentation. To tackle this problem, we propose a&#10;novel Edge-aware Mirror Network (EAMNet), which models edge detection and&#10;camouflaged object segmentation as a cross refinement process. More&#10;specifically, EAMNet has a two-branch architecture, where a&#10;segmentation-induced edge aggregation module and an edge-induced integrity&#10;aggregation module are designed to cross-guide the segmentation branch and edge&#10;detection branch. A guided-residual channel attention module which leverages&#10;the residual connection and gated convolution finally better extracts&#10;structural details from low-level features. Quantitative and qualitative&#10;experiment results show that EAMNet outperforms existing cutting-edge baselines&#10;on three widely used COD datasets. Codes are available at&#10;https://github.com/sdy1999/EAMNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.05728" label="2006.05728">
-        <attvalues>
-          <attvalue for="0" value="Diagnosing Rarity in Human-Object Interaction Detection" />
-          <attvalue for="1" value="  Human-object interaction (HOI) detection is a core task in computer vision.&#10;The goal is to localize all human-object pairs and recognize their&#10;interactions. An interaction defined by a &lt;verb, noun&gt; tuple leads to a&#10;long-tailed visual recognition challenge since many combinations are rarely&#10;represented. The performance of the proposed models is limited especially for&#10;the tail categories, but little has been done to understand the reason. To that&#10;end, in this paper, we propose to diagnose rarity in HOI detection. We propose&#10;a three-step strategy, namely Detection, Identification and Recognition where&#10;we carefully analyse the limiting factors by studying state-of-the-art models.&#10;Our findings indicate that detection and identification steps are altered by&#10;the interaction signals like occlusion and relative location, as a result&#10;limiting the recognition accuracy.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The goal of HOI detection is to detect all possible human-object pairs and recognize their interactions from an image. It is a core task in computer vision with many applications in robotics~\cite{katz2008manipulating}. A HOI is defined by a triplet of &lt;human, interaction, object&gt;, where the human and the object is a bounding box and the interaction is a &lt;verb, noun&gt; pair, such as &lt;ride, bicycle&gt;. The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}. The distribution of the training samples for interactions follows a long-tailed distribution where many interactions have few examples, see Figure~\ref{fig:teaser}. Despite the progress in the performance of many-shot interactions, recognizing rare interactions remains a challenge. &#10;&#10;HOI Detection is accomplished in three steps, see Figure~\ref{fig:steps}. 1. Human-object detection: HOI detector initially localizes all possible human-objects from the image. This step is challenged by the fact that interactions transform the human-object appearance, such as occlusions due to grasping, making it hard to localize all human-objects. 2. HOI Identification: Given the exhaustive pairing of all detected humans and objects, the HOI detector needs to identify the real interacting pairs. In the case of Figure~\ref{fig:steps}, only the rider, and the cow are in an interaction. This step is challenging since the cues of an interacting pair is in their subtlety, such as the relative locations of human-objects, human gaze or object parts. 3. HOI Recognition: In this step, HOI detector needs to classify the interaction type of the human-object pair(s), such as &lt;hold, cow&gt; and &lt;sit on, cow&gt; in Figure~\ref{fig:steps}. It is hard to distinguish among many interaction types since only a few examples are available for many interactions. &#10;&#10;Existing models highlight the difficulty in recognizing rare HOIs by reporting the performance on both the rare and nonrare (\ie many shots) splits on the benchmark dataset~\cite{chao2018learning}. However, it is not known what makes rare interactions particularly challenging aside from the low number of examples. Are the human-objects of rare interactions harder to detect? Or rare interacting pairs are harder to identify? The goal of our paper is to answer these questions. Instead of engineering a new model, we try to understand the detectability and identifiability of rare interactions with the help of the state-of-the-art HOI detectors~\cite{gao2018ican,chao2018learning,li2019transferable}. &#10;&#10;Our findings are: (i) Localizing human-objects of rare interactions is not challenging, however, detection is altered by the small and occluded human-objects, (ii) Identifying the rare HOIs is challenging and is altered by the background clutter and human-object distance, and (iii) Recognizing rare interactions is influenced by the detection and identification errors, leaving a big room for improvement. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Artificial Intelligence, Visual Recognition Strategies, Cognitive Science, Human Object Interaction, Computer Science, Computer Vision Challenges, Pattern Recognition" />
-        </attvalues>
-      </node>
-      <node id="1704.07333" label="1704.07333">
-        <attvalues>
-          <attvalue for="0" value="Detecting and Recognizing Human-Object Interactions" />
-          <attvalue for="1" value="  To understand the visual world, a machine must not only recognize individual&#10;object instances but also how they interact. Humans are often at the center of&#10;such interactions and detecting human-object interactions is an important&#10;practical and scientific problem. In this paper, we address the task of&#10;detecting &lt;human, verb, object&gt; triplets in challenging everyday photos. We&#10;propose a novel model that is driven by a human-centric approach. Our&#10;hypothesis is that the appearance of a person -- their pose, clothing, action&#10;-- is a powerful cue for localizing the objects they are interacting with. To&#10;exploit this cue, our model learns to predict an action-specific density over&#10;target object locations based on the appearance of a detected person. Our model&#10;also jointly learns to detect people and objects, and by fusing these&#10;predictions it efficiently infers interaction triplets in a clean, jointly&#10;trained end-to-end system we call InteractNet. We validate our approach on the&#10;recently introduced Verbs in COCO (V-COCO) and HICO-DET datasets, where we show&#10;quantitatively compelling results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.07962" label="1808.07962">
-        <attvalues>
-          <attvalue for="0" value="Learning Human-Object Interactions by Graph Parsing Neural Networks" />
-          <attvalue for="1" value="  This paper addresses the task of detecting and recognizing human-object&#10;interactions (HOI) in images and videos. We introduce the Graph Parsing Neural&#10;Network (GPNN), a framework that incorporates structural knowledge while being&#10;differentiable end-to-end. For a given scene, GPNN infers a parse graph that&#10;includes i) the HOI graph structure represented by an adjacency matrix, and ii)&#10;the node labels. Within a message passing inference framework, GPNN iteratively&#10;computes the adjacency matrices and node labels. We extensively evaluate our&#10;model on three HOI detection benchmarks on images and videos: HICO-DET, V-COCO,&#10;and CAD-120 datasets. Our approach significantly outperforms state-of-art&#10;methods, verifying that GPNN is scalable to large datasets and applies to&#10;spatial-temporal settings. The code is available at&#10;https://github.com/SiyuanQi/gpnn.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.10437" label="1808.10437">
-        <attvalues>
-          <attvalue for="0" value="iCAN: Instance-Centric Attention Network for Human-Object Interaction&#10;  Detection" />
-          <attvalue for="1" value="  Recent years have witnessed rapid progress in detecting and recognizing&#10;individual object instances. To understand the situation in a scene, however,&#10;computers need to recognize how humans interact with surrounding objects. In&#10;this paper, we tackle the challenging task of detecting human-object&#10;interactions (HOI). Our core idea is that the appearance of a person or an&#10;object instance contains informative cues on which relevant parts of an image&#10;to attend to for facilitating interaction prediction. To exploit these cues, we&#10;propose an instance-centric attention module that learns to dynamically&#10;highlight regions in an image conditioned on the appearance of each instance.&#10;Such an attention-based network allows us to selectively aggregate features&#10;relevant for recognizing HOIs. We validate the efficacy of the proposed network&#10;on the Verb in COCO and HICO-DET datasets and show that our approach compares&#10;favorably with the state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.05736" label="1812.05736">
-        <attvalues>
-          <attvalue for="0" value="Detecting unseen visual relations using analogies" />
-          <attvalue for="1" value="  We seek to detect visual relations in images of the form of triplets t =&#10;(subject, predicate, object), such as &quot;person riding dog&quot;, where training&#10;examples of the individual entities are available but their combinations are&#10;unseen at training. This is an important set-up due to the combinatorial nature&#10;of visual relations : collecting sufficient training data for all possible&#10;triplets would be very hard. The contributions of this work are three-fold.&#10;First, we learn a representation of visual relations that combines (i)&#10;individual embeddings for subject, object and predicate together with (ii) a&#10;visual phrase embedding that represents the relation triplet. Second, we learn&#10;how to transfer visual phrase embeddings from existing training triplets to&#10;unseen test triplets using analogies between relations that involve similar&#10;objects. Third, we demonstrate the benefits of our approach on three&#10;challenging datasets : on HICO-DET, our model achieves significant improvement&#10;over a strong baseline for both frequent and unseen triplets, and we observe&#10;similar improvement for the retrieval of unseen triplets with out-of-vocabulary&#10;predicates on the COCO-a dataset as well as the challenging unusual triplets in&#10;the UnRel dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03181" label="1904.03181">
-        <attvalues>
-          <attvalue for="0" value="Detecting Human-Object Interactions via Functional Generalization" />
-          <attvalue for="1" value="  We present an approach for detecting human-object interactions (HOIs) in&#10;images, based on the idea that humans interact with functionally similar&#10;objects in a similar manner. The proposed model is simple and efficiently uses&#10;the data, visual features of the human, relative spatial orientation of the&#10;human and the object, and the knowledge that functionally similar objects take&#10;part in similar interactions with humans. We provide extensive experimental&#10;validation for our approach and demonstrate state-of-the-art results for HOI&#10;detection. On the HICO-Det dataset our method achieves a gain of over 2.5%&#10;absolute points in mean average precision (mAP) over state-of-the-art. We also&#10;show that our approach leads to significant performance gains for zero-shot HOI&#10;detection in the seen object setting. We further demonstrate that using a&#10;generic object detector, our model can generalize to interactions involving&#10;previously unseen objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Human-Object Interaction.&#10;Early methods \cite{yao2010grouplet,yao2011human} relied on structured visual&#10;features which capture contextual relationships between humans and objects. Similarly,&#10;\cite{delaitre2011learning} used structured representations and spatial co-occurrences of body&#10;parts and objects to train models for HOI recognition. Gupta et al.&#10;\cite{gupta2007objects,gupta2009observing} adopted a Bayesian approach that integrated object&#10;classification and localization, action understanding, and perception of object reaction.&#10;\cite{desai2012detecting} constructed a compositional model which combined skeleton models,&#10; poselets, and visual phrases.&#10;&#10;More recently, with the release of large datasets like HICO \cite{chao2015hico}, Visual Genome&#10;\cite{krishna2017visual}, HCVRD \cite{zhuang2017care}, V-COCO \cite{gupta2015visual}, and HICO-Det&#10;\cite{chao2017learning}, the problem of detecting and recognizing HOIs has attracted significant&#10;attention. This has been driven by HICO which is a benchmark dataset for recognizing human-object&#10;interactions. The HICO-Det dataset extended HICO by adding bounding box annotations. V-COCO is a&#10;much smaller dataset containing 26 classes and about 10,000 images. On the other hand, HCVRD and&#10;Visual Genome provide annotations for thousands of relationship categories and hundreds of objects.&#10;However, they suffer from noisy labels. We primarily use the HICO-Det dataset to evaluate our&#10;approach in this paper.&#10;&#10;\cite{gkioxari2017detecting} designed a system which trains object and relationship detectors&#10;simultaneously on the same dataset and classifies a human-object pair into a fixed set of&#10;pre-defined relationship classes. This precludes the method from being useful for detecting novel&#10;relationships. \cite{xu2018interact} used pose and gaze information for HOI detection.&#10;\cite{kolesnikov2018detecting}&#10;introduced the Box Attention module to a standard R-CNN and trained simultaneously for object&#10;detection and relationship triplet prediction. Graph Parsing Neural Networks \cite{qi2018learning}&#10;incorporated structural knowledge and inferred a parse graph in a message passing inference&#10;framework. In contrast, our method does not need iterative processing and requires only a single&#10;pass through a neural network.&#10;&#10;Unlike most prior work, we do not directly classify into a fixed set of&#10;relationship triplets but into predicates. This helps us detect previously unseen interactions. The&#10;method closest in spirit to our approach is \cite{shen2018scaling} which uses a&#10;two branch structure with the first branch responsible for detecting humans and &#10;predicates, and the second for detecting objects. Unlike our proposed approach, their method&#10; solely depends on the appearance of the&#10;human. Also, they do not use any prior information from language. Our model utilizes implicit human&#10;appearance, the object label, human-object geometric relationship, and knowledge about&#10;similarities between objects. Hence, our model achieves much better performance than \cite{shen2018scaling}.&#10;&#10;We also distinguish our work from prior work \cite{kato2018compositional,fang2018pairwise} on HOI&#10;recognition. &#10;We tackle the more difficult problem of detecting HOIs here. &#10;&#10;Zero-shot Learning. Our work also ties well with zero-shot classification&#10;\cite{xian2017zero,kodirov2017semantic} and zero-shot object detection (ZSD)&#10; \cite{bansal2018zero}. \cite{bansal2018zero}&#10; proposed projecting images into the word-vector space to exploit the semantic properties of such&#10; spaces. They also discussed challenges associated with training and evaluating&#10; ZSD. A similar idea was used in \cite{kodirov2017semantic} for zero-shot classification.&#10; \cite{rahman2018zero}, on the other hand, used meta-classes to cluster semantically&#10; similar classes. In this work, we also use word-vectors&#10; as semantic information for our generalization module. This, along with our approach for&#10; generalization during training, helps zero-shot HOI detection.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1910.07721" label="1910.07721">
-        <attvalues>
-          <attvalue for="0" value="Deep Contextual Attention for Human-Object Interaction Detection" />
-          <attvalue for="1" value="  Human-object interaction detection is an important and relatively new class&#10;of visual relationship detection tasks, essential for deeper scene&#10;understanding. Most existing approaches decompose the problem into object&#10;localization and interaction recognition. Despite showing progress, these&#10;approaches only rely on the appearances of humans and objects and overlook the&#10;available context information, crucial for capturing subtle interactions&#10;between them. We propose a contextual attention framework for human-object&#10;interaction detection. Our approach leverages context by learning&#10;contextually-aware appearance features for human and object instances. The&#10;proposed attention module then adaptively selects relevant instance-centric&#10;context information to highlight image regions likely to contain human-object&#10;interactions. Experiments are performed on three benchmarks: V-COCO, HICO-DET&#10;and HCVRD. Our approach outperforms the state-of-the-art on all datasets. On&#10;the V-COCO dataset, our method achieves a relative gain of 4.4% in terms of&#10;role mean average precision ($mAP_{role}$), compared to the existing best&#10;approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.04360" label="2001.04360">
-        <attvalues>
-          <attvalue for="0" value="Classifying All Interacting Pairs in a Single Shot" />
-          <attvalue for="1" value="  In this paper, we introduce a novel human interaction detection approach,&#10;based on CALIPSO (Classifying ALl Interacting Pairs in a Single shOt), a&#10;classifier of human-object interactions. This new single-shot interaction&#10;classifier estimates interactions simultaneously for all human-object pairs,&#10;regardless of their number and class. State-of-the-art approaches adopt a&#10;multi-shot strategy based on a pairwise estimate of interactions for a set of&#10;human-object candidate pairs, which leads to a complexity depending, at least,&#10;on the number of interactions or, at most, on the number of candidate pairs. In&#10;contrast, the proposed method estimates the interactions on the whole image.&#10;Indeed, it simultaneously estimates all interactions between all human subjects&#10;and object targets by performing a single forward pass throughout the image.&#10;Consequently, it leads to a constant complexity and computation time&#10;independent of the number of subjects, objects or interactions in the image. In&#10;detail, interaction classification is achieved on a dense grid of anchors&#10;thanks to a joint multi-task network that learns three complementary tasks&#10;simultaneously: (i) prediction of the types of interaction, (ii) estimation of&#10;the presence of a target and (iii) learning of an embedding which maps&#10;interacting subject and target to a same representation, by using a metric&#10;learning strategy. In addition, we introduce an object-centric passive-voice&#10;verb estimation which significantly improves results. Evaluations on the two&#10;well-known Human-Object Interaction image datasets, V-COCO and HICO-DET,&#10;demonstrate the competitiveness of the proposed method (2nd place) compared to&#10;the state-of-the-art while having constant computation time regardless of the&#10;number of objects and interactions in the image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.05448" label="1702.05448">
-        <attvalues>
-          <attvalue for="0" value="Learning to Detect Human-Object Interactions" />
-          <attvalue for="1" value="  We study the problem of detecting human-object interactions (HOI) in static&#10;images, defined as predicting a human and an object bounding box with an&#10;interaction class label that connects them. HOI detection is a fundamental&#10;problem in computer vision as it provides semantic information about the&#10;interactions among the detected objects. We introduce HICO-DET, a new large&#10;benchmark for HOI detection, by augmenting the current HICO classification&#10;benchmark with instance annotations. To solve the task, we propose Human-Object&#10;Region-based Convolutional Neural Networks (HO-RCNN). At the core of our&#10;HO-RCNN is the Interaction Pattern, a novel DNN input that characterizes the&#10;spatial relations between two bounding boxes. Experiments on HICO-DET&#10;demonstrate that our HO-RCNN, by exploiting human-object spatial relations&#10;through Interaction Patterns, significantly improves the performance of HOI&#10;detection over baseline approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.10292" label="2101.10292">
-        <attvalues>
-          <attvalue for="0" value="Transferable Interactiveness Knowledge for Human-Object Interaction&#10;  Detection" />
-          <attvalue for="1" value="  Human-Object Interaction (HOI) detection is an important problem to&#10;understand how humans interact with objects. In this paper, we explore&#10;interactiveness knowledge which indicates whether a human and an object&#10;interact with each other or not. We found that interactiveness knowledge can be&#10;learned across HOI datasets and bridge the gap between diverse HOI category&#10;settings. Our core idea is to exploit an interactiveness network to learn the&#10;general interactiveness knowledge from multiple HOI datasets and perform&#10;Non-Interaction Suppression (NIS) before HOI classification in inference. On&#10;account of the generalization ability of interactiveness, interactiveness&#10;network is a transferable knowledge learner and can be cooperated with any HOI&#10;detection models to achieve desirable results. We utilize the human instance&#10;and body part features together to learn the interactiveness in hierarchical&#10;paradigm, i.e., instance-level and body part-level interactivenesses.&#10;Thereafter, a consistency task is proposed to guide the learning and extract&#10;deeper interactive visual clues. We extensively evaluate the proposed method on&#10;HICO-DET, V-COCO, and a newly constructed PaStaNet-HOI dataset. With the&#10;learned interactiveness, our method outperforms state-of-the-art HOI detection&#10;methods, verifying its efficacy and flexibility. Code is available at&#10;https://github.com/DirtyHarryLYL/Transferable-Interactiveness-Network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.06105" label="2409.06105">
-        <attvalues>
-          <attvalue for="0" value="SGC-VQGAN: Towards Complex Scene Representation via Semantic Guided&#10;  Clustering Codebook" />
-          <attvalue for="1" value="  Vector quantization (VQ) is a method for deterministically learning features&#10;through discrete codebook representations. Recent works have utilized visual&#10;tokenizers to discretize visual regions for self-supervised representation&#10;learning. However, a notable limitation of these tokenizers is lack of&#10;semantics, as they are derived solely from the pretext task of reconstructing&#10;raw image pixels in an auto-encoder paradigm. Additionally, issues like&#10;imbalanced codebook distribution and codebook collapse can adversely impact&#10;performance due to inefficient codebook utilization. To address these&#10;challenges, We introduce SGC-VQGAN through Semantic Online Clustering method to&#10;enhance token semantics through Consistent Semantic Learning. Utilizing&#10;inference results from segmentation model , our approach constructs a&#10;temporospatially consistent semantic codebook, addressing issues of codebook&#10;collapse and imbalanced token semantics. Our proposed Pyramid Feature Learning&#10;pipeline integrates multi-level features to capture both image details and&#10;semantics simultaneously. As a result, SGC-VQGAN achieves SOTA performance in&#10;both reconstruction quality and various downstream tasks. Its simplicity,&#10;requiring no additional parameter learning, enables its direct application in&#10;downstream tasks, presenting significant potential.&#10;" />
-          <attvalue for="2" value="&#10;Vector quantization (VQ) is a technique used to learn features with discrete codebook representations, often implemented with a variational autoencoding model called VQ-VAE\cite{VQ-VAE}. It is commonly employed alongside deep generative modeling for tasks such as denoising diffusion probabilistic models and autoregressive models\cite{vqgan,gaia1, worlddreamer, VAM}. These methods involve a two-step process: first, learning a compressed discrete representation for each image, and then learning the prior distribution in the discrete latent space. This compression of redundant information greatly aids in training deep generative models on large-scale data. It's important to note that regardless of the performance of the prior model, the compression performance of VQ ultimately limits the overall generation performance.&#10;&#10;Despite the achievements of VQ-VAE \cite{VQ-VAE} and VQGAN \cite{vqgan}, they face limitations when employed for complex scene generation tasks that demand high resolution, emphasize real-world dynamic scenarios, incorporate abundant contextual semantics, and take into account temporal aspects. Firstly, one prominent drawback is their lack of semantic understanding, stemming from their reliance solely on the auto-encoder approach for reconstructing raw image pixels. GAIA-1\cite{gaia1} suggests that enhancing the world model's performance hinges on directing the compression process towards meaningful representations, such as semantics, rather than solely prioritizing high-frequency signals. To address this, they employ the DINOv2\cite{oquab2023dinov2} model to guide compression. However, the semantic features in DINOv2 lack consistency across both time and space, as depicted in pipeline. Consequently, this method fails to consistently improve the semantic aspect of the codebook. Secondly, imbalanced codebook distribution and codebook collaspe can adversely impact performance due to inefficient codebook utilization; Codebook collapse refers to the situation where most tokens become inactive and are not used during the generation stage. \cite{williams2020hierarchical} aims to reset infrequently used or unused entries by leveraging frequently accessed ones, while CVQ-VAE \cite{zheng2023online} proposes an online clustering algorithm that updates codebook entries using associated encoded features during the training stage. However, none of these approaches adequately address the imbalance in codebook utilization concerning semantics, so that the codebook are not fully utilized. As illustrated in Figure~\ref{fig:vis_dist}, we present a visualization of the semantic distribution of the codebook in VQGAN\cite{vqgan} and its improved versions\cite{zheng2023online} on the Nuscenes dataset. VQGAN\cite{vqgan} and CVQ-VAE\cite{zheng2023online} exhibits an alarming $0.7\%$ and $23.5\%$ in terms of semantic uniqueness respectively, indicating a lack of specific semantics for the majority of these tokens. For more details about semantic performance, please refer to Appendix and Table~\ref{tab:results}.&#10;&#10;To address these challenges, we propose the Consistent Semantic Learning approach, which leverages contrastive learning to guide compression while incorporating semantics. By leveraging the inference results from the segmentation model\cite{xie2021segformer}, our approach achieves temporospatially consistent semantic learning, as the semantic IDs remain consistent across both time and space. We formulate the compression task as a Semantic Online Clustering problem. This involves integrating high-level features into the codebook and computing multilevel distances between tokens and encoded features. These distances are then aggregated through a weighted sum to determine the clustering criteria for the codebook and encoded features. As the encoded features within the deep network undergo updates during training rather than remaining fixed, we apply a dynamic updating strategy to update multi-level features in the codebook. Additionally, our Multi-level Feature Learning pipeline combines low-level and high-level features to enhance both reconstruction performance and semantic understanding. Given that the decoder of VQGAN\cite{vqgan} is often not utilized in the second stage in complex scene generation applications\cite{gaia1, worlddreamer}, our pyramid feature learning approach focuses on simplicity without requiring additional parameter learning. This enables direct utilization of our method in downstream tasks, thereby maximizing its potential. The detailed process is demonstrated in Figure~\ref{fig:pipeline}.&#10;&#10;In summary, our main contributions are as follows:&#10;\begin{enumerate}&#10; \item We introduce a Semantic Online Clustering method to enhance token semantics through Consistent Semantic Learning without requiring highly accurate annotations. By utilizing segmentation model inference results, our approach creates a temporospatially consistent semantic codebook, addressing codebook collapse and imbalanced token semantics which improves performance in various tasks leveraging VQ as a prior model.&#10; \item The proposed Pyramid Feature Learning pipeline integrates multi-level features to simultaneously capture image details and semantics. Its simplicity, without requiring additional parameter learning, allows direct application in downstream tasks, offering significant potential.&#10; \item Our method undergoes extensive validation across diverse datasets and tasks to ascertain its performance.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Self-Supervised Image Analysis, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science, Visual Representation Learning, Codebook Optimization Techniques" />
-        </attvalues>
-      </node>
-      <node id="1711.00937" label="1711.00937">
-        <attvalues>
-          <attvalue for="0" value="Neural Discrete Representation Learning" />
-          <attvalue for="1" value="  Learning useful representations without supervision remains a key challenge&#10;in machine learning. In this paper, we propose a simple yet powerful generative&#10;model that learns such discrete representations. Our model, the Vector&#10;Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways:&#10;the encoder network outputs discrete, rather than continuous, codes; and the&#10;prior is learnt rather than static. In order to learn a discrete latent&#10;representation, we incorporate ideas from vector quantisation (VQ). Using the&#10;VQ method allows the model to circumvent issues of &quot;posterior collapse&quot; --&#10;where the latents are ignored when they are paired with a powerful&#10;autoregressive decoder -- typically observed in the VAE framework. Pairing&#10;these representations with an autoregressive prior, the model can generate high&#10;quality images, videos, and speech as well as doing high quality speaker&#10;conversion and unsupervised learning of phonemes, providing further evidence of&#10;the utility of the learnt representations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.17080" label="2309.17080">
-        <attvalues>
-          <attvalue for="0" value="GAIA-1: A Generative World Model for Autonomous Driving" />
-          <attvalue for="1" value="  Autonomous driving promises transformative improvements to transportation,&#10;but building systems capable of safely navigating the unstructured complexity&#10;of real-world scenarios remains challenging. A critical problem lies in&#10;effectively predicting the various potential outcomes that may emerge in&#10;response to the vehicle's actions as the world evolves.&#10;  To address this challenge, we introduce GAIA-1 ('Generative AI for&#10;Autonomy'), a generative world model that leverages video, text, and action&#10;inputs to generate realistic driving scenarios while offering fine-grained&#10;control over ego-vehicle behavior and scene features. Our approach casts world&#10;modeling as an unsupervised sequence modeling problem by mapping the inputs to&#10;discrete tokens, and predicting the next token in the sequence. Emerging&#10;properties from our model include learning high-level structures and scene&#10;dynamics, contextual awareness, generalization, and understanding of geometry.&#10;The power of GAIA-1's learned representation that captures expectations of&#10;future events, combined with its ability to generate realistic samples,&#10;provides new possibilities for innovation in the field of autonomy, enabling&#10;enhanced and accelerated training of autonomous driving technology.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.08111" label="2002.08111">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Quantized Autoencoders" />
-          <attvalue for="1" value="  Despite progress in training neural networks for lossy image compression,&#10;current approaches fail to maintain both perceptual quality and abstract&#10;features at very low bitrates. Encouraged by recent success in learning&#10;discrete representations with Vector Quantized Variational Autoencoders&#10;(VQ-VAEs), we motivate the use of a hierarchy of VQ-VAEs to attain high factors&#10;of compression. We show that the combination of stochastic quantization and&#10;hierarchical latent structure aids likelihood-based image compression. This&#10;leads us to introduce a novel objective for training hierarchical VQ-VAEs. Our&#10;resulting scheme produces a Markovian series of latent variables that&#10;reconstruct images of high-perceptual quality which retain semantically&#10;meaningful features. We provide qualitative and quantitative evaluations on the&#10;CelebA and MNIST datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.15139" label="2307.15139">
-        <attvalues>
-          <attvalue for="0" value="Online Clustered Codebook" />
-          <attvalue for="1" value="  Vector Quantisation (VQ) is experiencing a comeback in machine learning,&#10;where it is increasingly used in representation learning. However, optimizing&#10;the codevectors in existing VQ-VAE is not entirely trivial. A problem is&#10;codebook collapse, where only a small subset of codevectors receive gradients&#10;useful for their optimisation, whereas a majority of them simply ``dies off''&#10;and is never updated or used. This limits the effectiveness of VQ for learning&#10;larger codebooks in complex computer vision tasks that require high-capacity&#10;representations. In this paper, we present a simple alternative method for&#10;online codebook learning, Clustering VQ-VAE (CVQ-VAE). Our approach selects&#10;encoded features as anchors to update the ``dead'' codevectors, while&#10;optimising the codebooks which are alive via the original loss. This strategy&#10;brings unused codevectors closer in distribution to the encoded features,&#10;increasing the likelihood of being chosen and optimized. We extensively&#10;validate the generalization capability of our quantiser on various datasets,&#10;tasks (e.g. reconstruction and generation), and architectures (e.g. VQ-VAE,&#10;VQGAN, LDM). Our CVQ-VAE can be easily integrated into the existing models with&#10;just a few lines of code.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.13974" label="2307.13974">
-        <attvalues>
-          <attvalue for="0" value="Tracking Anything in High Quality" />
-          <attvalue for="1" value="  Visual object tracking is a fundamental video task in computer vision.&#10;Recently, the notably increasing power of perception algorithms allows the&#10;unification of single/multiobject and box/mask-based tracking. Among them, the&#10;Segment Anything Model (SAM) attracts much attention. In this report, we&#10;propose HQTrack, a framework for High Quality Tracking anything in videos.&#10;HQTrack mainly consists of a video multi-object segmenter (VMOS) and a mask&#10;refiner (MR). Given the object to be tracked in the initial frame of a video,&#10;VMOS propagates the object masks to the current frame. The mask results at this&#10;stage are not accurate enough since VMOS is trained on several closeset video&#10;object segmentation (VOS) datasets, which has limited ability to generalize to&#10;complex and corner scenes. To further improve the quality of tracking masks, a&#10;pretrained MR model is employed to refine the tracking results. As a compelling&#10;testament to the effectiveness of our paradigm, without employing any tricks&#10;such as test-time data augmentations and model ensemble, HQTrack ranks the 2nd&#10;place in the Visual Object Tracking and Segmentation (VOTS2023) challenge. Code&#10;and models are available at https://github.com/jiawen-zhu/HQTrack.&#10;" />
-          <attvalue for="2" value="&#10;As a fundamental video task in computer vision, visual object tracking has become the cornerstone of many related areas, such as robot vision and autonomous driving. The task aims at consistently locating the specified object in a video sequence. As one of the most influential challenges in the tracking field, Visual Object Tracking (VOT) challenge \cite{vot21,vot22} attracted a lot of attention, and many SOTA algorithms participate to show their cutting-edge performance. The Visual Object Tracking and Segmentation challenge (VOTS2023) relaxes the constraints enforced in past VOT challenges for considering general object tracking in a broader context. Therefore, VOTS2023 merges short-term and long-term, single-target and multiple-target tracking with segmentation as the only target location specification. This poses more challenges, \eg, inter-object relationship understanding, multi-target trajectory tracking, accurate mask estimation, etc.&#10;&#10;Visual object tracking has made great strides with deep learning techniques\cite{alexnet, resnet, vit}. &#10;Previous methods can be grouped into either online-update trackers\cite{eco, dimp} and Siamese trackers\cite{siamesefc, siamrcnn}. &#10;Recently, Transformer\cite{attention_is_all} sweeps in computer vision, the dominant tracking methods are Transformer-based trackers\cite{transt,stark,mixformer,ostrack}. TransT\cite{transt} proposes transformer-based ECA and CFA modules to replace the long-used correlation calculation.&#10;Benefiting from Transformer's superior long-range modeling capability, TransT outperforms the previous correlation modules which are a capable class of linear modeling. &#10;More recently, some trackers\cite{mixformer, ostrack} introduce pure transformer architecture, and the feature extracting and template-search region interaction is completed in a single backbone, tracking performance is pushed to new records.&#10;These trackers mainly focus on single object tracking and output the bounding box for performance evaluation.&#10;Hence, merely employing SOT trackers is not well-suited to the VOTS2023 challenge.&#10;&#10;Video object segmentation aims to segment out the specific objects of interest in a video sequence. Similar to VOT, semi-supervised video object segmentation also manually provides the annotated in the first frame. The main difference is that the VOS task provides a more fine-grained mask annotation. &#10;Early VOS methods propagate object masks over video frames via motion clues\cite{segflow,tsai2016video} or adopt online learning strategies\cite{caelles2017one,li2018video}.&#10;Recently, Space-Temporal Memory (STM) network~\cite{oh2019video, wang2017learning} extracts the spatio-temporal context from a memory bank to handle the appearance changes and occlusions, offering a promising solution for semi-supervised video object segmentation.&#10;For multi-object segmentation, these methods segment the objects one by one, the final results are merged masks by post ensemble.&#10;AOT\cite{aot} proposes an identification mechanism that can encode, match, and segment multiple objects at the same time. Based on AOT\cite{aot}, DeAOT\cite{deaot} decouples the hierarchical propagation of object-agnostic and object-specific embeddings from previous frames to the current frame, further improving the VOS accuracy.&#10;&#10;Although the above VOS methods can handle tracking task with multi-object and mask output, &#10;challenges in VOTS2023 benchmark remain. &#10;(i) VOTS videos contain a large number of long-term sequences, the longest of which exceeds 10,000 frames, which requires the tracker to be able to discriminate the drastic changes in object appearance and adapt to variations in the environment.&#10;At the same time, long-term video sequences also make some memory-based methods face memory bank space challenges.&#10;(ii) In VOTS videos, targets will leave the field of view and then returns.&#10;Trackers require additional design to accommodate the disappearance and appearance of targets. &#10;(iii) A series of challenges such as fast motion, frequent occlusion, distractors, and tiny objects also make this task more difficult. &#10;&#10;In this work, we propose Tracking Anything in High Quality (termed HQTrack), which mainly consists of a video multi-object segmenter (VMOS) and a mask refiner (MR). VMOS is an improved variant of DeAOT~\cite{deaot}, we cascade a 1/8 scale gated propagation module (GPM) for perceiving small objects in complex scenarios. Besides, Intern-T\cite{internimage} is employed as our feature extractor to enhance object discrimination capabilities.&#10;To save memory usage, a fixed length of long-term memory is used in VMOS, excluding the initial frame, the memory of early frames will be discarded.&#10;On the other hand, it should be beneficial to apply a large segmentation model to refine our tracking masks.&#10;SAM\cite{sam} is prone to failure when predicting objects with complex structures\cite{sam_hq}, and these difficult cases are common in VOTS chanllenge.&#10;To further improve the quality of tracking masks, a pre-trained HQ-SAM\cite{sam_hq} model is employed to refine the tracking masks. &#10;We calculate the outer enclosing boxes of the predicted masks from VMOS as box prompts and feed them into HQ-SAM together with the original images to gain the refined masks, the final tracking results are selected from VMOS and MR.&#10;&#10;Finally, HQTrack obtains an impressive 0.615 quality score on the VOTS2023 test set, achieving runner-up at the VOTS2023 challenge.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1611.09224" label="1611.09224">
-        <attvalues>
-          <attvalue for="0" value="ECO: Efficient Convolution Operators for Tracking" />
-          <attvalue for="1" value="  In recent years, Discriminative Correlation Filter (DCF) based methods have&#10;significantly advanced the state-of-the-art in tracking. However, in the&#10;pursuit of ever increasing tracking performance, their characteristic speed and&#10;real-time capability have gradually faded. Further, the increasingly complex&#10;models, with massive number of trainable parameters, have introduced the risk&#10;of severe over-fitting. In this work, we tackle the key causes behind the&#10;problems of computational complexity and over-fitting, with the aim of&#10;simultaneously improving both speed and performance.&#10;  We revisit the core DCF formulation and introduce: (i) a factorized&#10;convolution operator, which drastically reduces the number of parameters in the&#10;model; (ii) a compact generative model of the training sample distribution,&#10;that significantly reduces memory and time complexity, while providing better&#10;diversity of samples; (iii) a conservative model update strategy with improved&#10;robustness and reduced complexity. We perform comprehensive experiments on four&#10;benchmarks: VOT2016, UAV123, OTB-2015, and TempleColor. When using expensive&#10;deep features, our tracker provides a 20-fold speedup and achieves a 13.0%&#10;relative gain in Expected Average Overlap compared to the top ranked method in&#10;the VOT2016 challenge. Moreover, our fast variant, using hand-crafted features,&#10;operates at 60 Hz on a single CPU, while obtaining 65.0% AUC on OTB-2015.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.09549" label="1606.09549">
-        <attvalues>
-          <attvalue for="0" value="Fully-Convolutional Siamese Networks for Object Tracking" />
-          <attvalue for="1" value="  The problem of arbitrary object tracking has traditionally been tackled by&#10;learning a model of the object's appearance exclusively online, using as sole&#10;training data the video itself. Despite the success of these methods, their&#10;online-only approach inherently limits the richness of the model they can&#10;learn. Recently, several attempts have been made to exploit the expressive&#10;power of deep convolutional networks. However, when the object to track is not&#10;known beforehand, it is necessary to perform Stochastic Gradient Descent online&#10;to adapt the weights of the network, severely compromising the speed of the&#10;system. In this paper we equip a basic tracking algorithm with a novel&#10;fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset&#10;for object detection in video. Our tracker operates at frame-rates beyond&#10;real-time and, despite its extreme simplicity, achieves state-of-the-art&#10;performance in multiple benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.15436" label="2103.15436">
-        <attvalues>
-          <attvalue for="0" value="Transformer Tracking" />
-          <attvalue for="1" value="  Correlation acts as a critical role in the tracking field, especially in&#10;recent popular Siamese-based trackers. The correlation operation is a simple&#10;fusion manner to consider the similarity between the template and the search&#10;region. However, the correlation operation itself is a local linear matching&#10;process, leading to lose semantic information and fall into local optimum&#10;easily, which may be the bottleneck of designing high-accuracy tracking&#10;algorithms. Is there any better feature fusion method than correlation? To&#10;address this issue, inspired by Transformer, this work presents a novel&#10;attention-based feature fusion network, which effectively combines the template&#10;and search region features solely using attention. Specifically, the proposed&#10;method includes an ego-context augment module based on self-attention and a&#10;cross-feature augment module based on cross-attention. Finally, we present a&#10;Transformer tracking (named TransT) method based on the Siamese-like feature&#10;extraction backbone, the designed attention-based fusion mechanism, and the&#10;classification and regression head. Experiments show that our TransT achieves&#10;very promising results on six challenging datasets, especially on large-scale&#10;LaSOT, TrackingNet, and GOT-10k benchmarks. Our tracker runs at approximatively&#10;50 fps on GPU. Code and models are available at&#10;https://github.com/chenxin-dlut/TransT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.17154" label="2103.17154">
-        <attvalues>
-          <attvalue for="0" value="Learning Spatio-Temporal Transformer for Visual Tracking" />
-          <attvalue for="1" value="  In this paper, we present a new tracking architecture with an encoder-decoder&#10;transformer as the key component. The encoder models the global spatio-temporal&#10;feature dependencies between target objects and search regions, while the&#10;decoder learns a query embedding to predict the spatial positions of the target&#10;objects. Our method casts object tracking as a direct bounding box prediction&#10;problem, without using any proposals or predefined anchors. With the&#10;encoder-decoder transformer, the prediction of objects just uses a simple&#10;fully-convolutional network, which estimates the corners of objects directly.&#10;The whole method is end-to-end, does not need any postprocessing steps such as&#10;cosine window and bounding box smoothing, thus largely simplifying existing&#10;tracking pipelines. The proposed tracker achieves state-of-the-art performance&#10;on five challenging short-term and long-term benchmarks, while running at&#10;real-time speed, being 6x faster than Siam R-CNN. Code and models are&#10;open-sourced at https://github.com/researchmm/Stark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.02814" label="2302.02814">
-        <attvalues>
-          <attvalue for="0" value="MixFormer: End-to-End Tracking with Iterative Mixed Attention" />
-          <attvalue for="1" value="  Visual object tracking often employs a multi-stage pipeline of feature&#10;extraction, target information integration, and bounding box estimation. To&#10;simplify this pipeline and unify the process of feature extraction and target&#10;information integration, in this paper, we present a compact tracking&#10;framework, termed as MixFormer, built upon transformers. Our core design is to&#10;utilize the flexibility of attention operations, and propose a Mixed Attention&#10;Module (MAM) for simultaneous feature extraction and target information&#10;integration. This synchronous modeling scheme allows to extract target-specific&#10;discriminative features and perform extensive communication between target and&#10;search area. Based on MAM, we build our MixFormer trackers simply by stacking&#10;multiple MAMs and placing a localization head on top. Specifically, we&#10;instantiate two types of MixFormer trackers, a hierarchical tracker MixCvT, and&#10;a non-hierarchical tracker MixViT. For these two trackers, we investigate a&#10;series of pre-training methods and uncover the different behaviors between&#10;supervised pre-training and self-supervised pre-training in our MixFormer&#10;trackers. We also extend the masked pre-training to our MixFormer trackers and&#10;design the competitive TrackMAE pre-training technique. Finally, to handle&#10;multiple target templates during online tracking, we devise an asymmetric&#10;attention scheme in MAM to reduce computational cost, and propose an effective&#10;score prediction module to select high-quality templates. Our MixFormer&#10;trackers set a new state-of-the-art performance on seven tracking benchmarks,&#10;including LaSOT, TrackingNet, VOT2020, GOT-10k, OTB100 and UAV123. In&#10;particular, our MixViT-L achieves AUC score of 73.3% on LaSOT, 86.1% on&#10;TrackingNet, EAO of 0.584 on VOT2020, and AO of 75.7% on GOT-10k. Code and&#10;trained models are publicly available at https://github.com/MCG-NJU/MixFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.11991" label="2203.11991">
-        <attvalues>
-          <attvalue for="0" value="Joint Feature Learning and Relation Modeling for Tracking: A One-Stream&#10;  Framework" />
-          <attvalue for="1" value="  The current popular two-stream, two-stage tracking framework extracts the&#10;template and the search region features separately and then performs relation&#10;modeling, thus the extracted features lack the awareness of the target and have&#10;limited target-background discriminability. To tackle the above issue, we&#10;propose a novel one-stream tracking (OSTrack) framework that unifies feature&#10;learning and relation modeling by bridging the template-search image pairs with&#10;bidirectional information flows. In this way, discriminative target-oriented&#10;features can be dynamically extracted by mutual guidance. Since no extra heavy&#10;relation modeling module is needed and the implementation is highly&#10;parallelized, the proposed tracker runs at a fast speed. To further improve the&#10;inference efficiency, an in-network candidate early elimination module is&#10;proposed based on the strong similarity prior calculated in the one-stream&#10;framework. As a unified framework, OSTrack achieves state-of-the-art&#10;performance on multiple benchmarks, in particular, it shows impressive results&#10;on the one-shot tracking benchmark GOT-10k, i.e., achieving 73.7% AO, improving&#10;the existing best result (SwinTrack) by 4.3\%. Besides, our method maintains a&#10;good performance-speed trade-off and shows faster convergence. The code and&#10;models are available at https://github.com/botaoye/OSTrack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.06750" label="1709.06750">
-        <attvalues>
-          <attvalue for="0" value="SegFlow: Joint Learning for Video Object Segmentation and Optical Flow" />
-          <attvalue for="1" value="  This paper proposes an end-to-end trainable network, SegFlow, for&#10;simultaneously predicting pixel-wise object segmentation and optical flow in&#10;videos. The proposed SegFlow has two branches where useful information of&#10;object segmentation and optical flow is propagated bidirectionally in a unified&#10;framework. The segmentation branch is based on a fully convolutional network,&#10;which has been proved effective in image segmentation task, and the optical&#10;flow branch takes advantage of the FlowNet model. The unified framework is&#10;trained iteratively offline to learn a generic notion, and fine-tuned online&#10;for specific objects. Extensive experiments on both the video object&#10;segmentation and optical flow datasets demonstrate that introducing optical&#10;flow improves the performance of segmentation and vice versa, against the&#10;state-of-the-art algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.04242" label="1803.04242">
-        <attvalues>
-          <attvalue for="0" value="Video Object Segmentation with Joint Re-identification and&#10;  Attention-Aware Mask Propagation" />
-          <attvalue for="1" value="  The problem of video object segmentation can become extremely challenging&#10;when multiple instances co-exist. While each instance may exhibit large scale&#10;and pose variations, the problem is compounded when instances occlude each&#10;other causing failures in tracking. In this study, we formulate a deep&#10;recurrent network that is capable of segmenting and tracking objects in video&#10;simultaneously by their temporal continuity, yet able to re-identify them when&#10;they re-appear after a prolonged occlusion. We combine both temporal&#10;propagation and re-identification functionalities into a single framework that&#10;can be trained end-to-end. In particular, we present a re-identification module&#10;with template expansion to retrieve missing objects despite their large&#10;appearance changes. In addition, we contribute a new attention-based recurrent&#10;mask propagation approach that is robust to distractors not belonging to the&#10;target segment. Our approach achieves a new state-of-the-art global mean&#10;(Region Jaccard and Boundary F measure) of 68.2 on the challenging DAVIS 2017&#10;benchmark (test-dev set), outperforming the winning solution which achieves a&#10;global mean of 66.1 on the same partition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.00607" label="1904.00607">
-        <attvalues>
-          <attvalue for="0" value="Video Object Segmentation using Space-Time Memory Networks" />
-          <attvalue for="1" value="  We propose a novel solution for semi-supervised video object segmentation. By&#10;the nature of the problem, available cues (e.g. video frame(s) with object&#10;masks) become richer with the intermediate predictions. However, the existing&#10;methods are unable to fully exploit this rich source of information. We resolve&#10;the issue by leveraging memory networks and learn to read relevant information&#10;from all available sources. In our framework, the past frames with object masks&#10;form an external memory, and the current frame as the query is segmented using&#10;the mask information in the memory. Specifically, the query and the memory are&#10;densely matched in the feature space, covering all the space-time pixel&#10;locations in a feed-forward fashion. Contrast to the previous approaches, the&#10;abundant use of the guidance information allows us to better handle the&#10;challenges such as appearance changes and occlussions. We validate our method&#10;on the latest benchmark sets and achieved the state-of-the-art performance&#10;(overall score of 79.4 on Youtube-VOS val set, J of 88.7 and 79.2 on DAVIS&#10;2016/2017 val set respectively) while having a fast runtime (0.16 second/frame&#10;on DAVIS 2016 val set).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02638" label="2106.02638">
-        <attvalues>
-          <attvalue for="0" value="Associating Objects with Transformers for Video Object Segmentation" />
-          <attvalue for="1" value="  This paper investigates how to realize better and more efficient embedding&#10;learning to tackle the semi-supervised video object segmentation under&#10;challenging multi-object scenarios. The state-of-the-art methods learn to&#10;decode features with a single positive object and thus have to match and&#10;segment each target separately under multi-object scenarios, consuming multiple&#10;times computing resources. To solve the problem, we propose an Associating&#10;Objects with Transformers (AOT) approach to match and decode multiple objects&#10;uniformly. In detail, AOT employs an identification mechanism to associate&#10;multiple targets into the same high-dimensional embedding space. Thus, we can&#10;simultaneously process multiple objects' matching and segmentation decoding as&#10;efficiently as processing a single object. For sufficiently modeling&#10;multi-object association, a Long Short-Term Transformer is designed for&#10;constructing hierarchical matching and propagation. We conduct extensive&#10;experiments on both multi-object and single-object benchmarks to examine AOT&#10;variant networks with different complexities. Particularly, our R50-AOT-L&#10;outperforms all the state-of-the-art competitors on three popular benchmarks,&#10;i.e., YouTube-VOS (84.1% J&amp;F), DAVIS 2017 (84.9%), and DAVIS 2016 (91.1%),&#10;while keeping more than $3\times$ faster multi-object run-time. Meanwhile, our&#10;AOT-T can maintain real-time multi-object speed on the above benchmarks. Based&#10;on AOT, we ranked 1st in the 3rd Large-scale VOS Challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Semi-supervised Video Object Segmentation.&#10;Given one or more annotated frames (the first frame in general), semi-supervised VOS methods propagate the manual labeling to the entire video sequence. Traditional methods often solve an optimization problem with an energy defined over a graph structure~\cite{tradition1,tradition3,tradition2}. In recent years, VOS methods have been mainly developed based on deep neural networks (DNN), leading to better results.&#10;&#10;Early DNN methods rely on fine-tuning the networks at test time to make segmentation networks focus on a specific object. Among them, OSVOS~\cite{osvos} and MoNet~\cite{xiao2018monet} fine-tune pre-trained networks on the first-frame ground-truth at test time. OnAVOS~\cite{onavos} extends the first-frame fine-tuning by introducing an online adaptation mechanism. Following these approaches, MaskTrack~\cite{masktrack} and PReM~\cite{premvos} &#10;utilize optical flow to help propagate the segmentation mask from one frame to the next. Despite achieving promising results, the test-time fine-tuning restricts the network efficiency.&#10;&#10;Recent works aim to achieve a better run-time and avoid using online fine-tuning. OSMN~\cite{osmn} employs one convolutional network to extract object embedding and another one to guide segmentation predictions. PML~\cite{pml} learns pixel-wise embedding with a nearest neighbor classifier, and VideoMatch~\cite{videomatch} uses a soft matching layer that maps the pixels of the current frame to the first frame in a learned embedding space. Following PML and VideoMatch, FEELVOS~\cite{feelvos} and CFBI~\cite{cfbi,cfbip} extend the pixel-level matching mechanism by additionally matching between the current frame and the previous frame. RGMP~\cite{rgmp} also gathers guidance information from both the first frame and the previous frame but uses a siamese encoder with two shared streams. STM~\cite{spacetime} and its following works (\eg, EGMN~\cite{EGMN} and KMN~\cite{KMN}) leverage a memory network to embed past-frame predictions into memory and apply a non-local attention mechanism on the memory to decode the segmentation of the current frame. SST~\cite{sstvos} utilizes attention mechanisms in a different way, \ie, transformer blocks~\cite{transformer} are used to extract pixel-level affinity maps and spatial-temporal features. The features are target-agnostic, instead of target-aware like our LSTT, since the mask information in past frames is not propagated and aggregated in the blocks. Instead of using matching mechanisms, LWL~\cite{LWLVOS} proposes to use an online few-shot learner to learn to decode object segmentation.&#10;&#10;The above methods learn to decode features with a single positive object and thus have to match and segment each target separately under multi-object scenarios, consuming multiple times computing resources of single-object cases. The problem restricts the application and development of the VOS with multiple targets. Hence, we propose our AOT to associate and decode multiple targets uniformly and simultaneously, as efficiently as processing a single object.&#10;&#10;Visual Transformers. Transformers~\cite{transformer} was proposed to build hierarchical attention-based networks for machine translation. Similar to Non-local Neural Networks~\cite{nonlocal}, transformer blocks compute correlation with all the input elements and aggregate their information by using attention mechanisms~\cite{att}. Compared to RNNs, transformer networks model global correlation or attention in parallel, leading to better memory efficiency, and thus have been widely used in natural language processing (NLP) tasks~\cite{devlin2018bert,radford2019language,synnaeve2019end}. Recently, transformer blocks were introduced&#10;to many computer vision tasks, such as image classification~\cite{vit,vaswani2021scaling,swin}, object detection~\cite{detr}/segmentation~\cite{vistr}, and image generation~\cite{parmar2018image}, and have shown promising performance compared to CNN-based networks.&#10;&#10;Many VOS methods~\cite{lin2019agss,spacetime,EGMN,KMN} have utilized attention mechanisms to match the object features and propagate the segmentation mask from past frames to the current frames. Nevertheless, these methods consider only one positive target in the attention processes, and how to build hierarchical attention-based propagation has been rarely studied. In this paper, we carefully design a long short-term transformer block, which can effectively construct multi-object matching and propagation within hierarchical structures for VOS.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2210.09782" label="2210.09782">
-        <attvalues>
-          <attvalue for="0" value="Decoupling Features in Hierarchical Propagation for Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  This paper focuses on developing a more effective method of hierarchical&#10;propagation for semi-supervised Video Object Segmentation (VOS). Based on&#10;vision transformers, the recently-developed Associating Objects with&#10;Transformers (AOT) approach introduces hierarchical propagation into VOS and&#10;has shown promising results. The hierarchical propagation can gradually&#10;propagate information from past frames to the current frame and transfer the&#10;current frame feature from object-agnostic to object-specific. However, the&#10;increase of object-specific information will inevitably lead to the loss of&#10;object-agnostic visual information in deep propagation layers. To solve such a&#10;problem and further facilitate the learning of visual embeddings, this paper&#10;proposes a Decoupling Features in Hierarchical Propagation (DeAOT) approach.&#10;Firstly, DeAOT decouples the hierarchical propagation of object-agnostic and&#10;object-specific embeddings by handling them in two independent branches.&#10;Secondly, to compensate for the additional computation from dual-branch&#10;propagation, we propose an efficient module for constructing hierarchical&#10;propagation, i.e., Gated Propagation Module, which is carefully designed with&#10;single-head attention. Extensive experiments show that DeAOT significantly&#10;outperforms AOT in both accuracy and efficiency. On YouTube-VOS, DeAOT can&#10;achieve 86.0% at 22.4fps and 82.0% at 53.4fps. Without test-time augmentations,&#10;we achieve new state-of-the-art performance on four benchmarks, i.e.,&#10;YouTube-VOS (86.2%), DAVIS 2017 (86.2%), DAVIS 2016 (92.9%), and VOT 2020&#10;(0.622). Project page: https://github.com/z-x-yang/AOT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.05778" label="2211.05778">
-        <attvalues>
-          <attvalue for="0" value="InternImage: Exploring Large-Scale Vision Foundation Models with&#10;  Deformable Convolutions" />
-          <attvalue for="1" value="  Compared to the great progress of large-scale vision transformers (ViTs) in&#10;recent years, large-scale models based on convolutional neural networks (CNNs)&#10;are still in an early state. This work presents a new large-scale CNN-based&#10;foundation model, termed InternImage, which can obtain the gain from increasing&#10;parameters and training data like ViTs. Different from the recent CNNs that&#10;focus on large dense kernels, InternImage takes deformable convolution as the&#10;core operator, so that our model not only has the large effective receptive&#10;field required for downstream tasks such as detection and segmentation, but&#10;also has the adaptive spatial aggregation conditioned by input and task&#10;information. As a result, the proposed InternImage reduces the strict inductive&#10;bias of traditional CNNs and makes it possible to learn stronger and more&#10;robust patterns with large-scale parameters from massive data like ViTs. The&#10;effectiveness of our model is proven on challenging benchmarks including&#10;ImageNet, COCO, and ADE20K. It is worth mentioning that InternImage-H achieved&#10;a new record 65.4 mAP on COCO test-dev and 62.9 mIoU on ADE20K, outperforming&#10;current leading CNNs and ViTs. The code will be released at&#10;https://github.com/OpenGVLab/InternImage.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.01567" label="2306.01567">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything in High Quality" />
-          <attvalue for="1" value="  The recent Segment Anything Model (SAM) represents a big leap in scaling up&#10;segmentation models, allowing for powerful zero-shot capabilities and flexible&#10;prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction&#10;quality falls short in many cases, particularly when dealing with objects that&#10;have intricate structures. We propose HQ-SAM, equipping SAM with the ability to&#10;accurately segment any object, while maintaining SAM's original promptable&#10;design, efficiency, and zero-shot generalizability. Our careful design reuses&#10;and preserves the pre-trained model weights of SAM, while only introducing&#10;minimal additional parameters and computation. We design a learnable&#10;High-Quality Output Token, which is injected into SAM's mask decoder and is&#10;responsible for predicting the high-quality mask. Instead of only applying it&#10;on mask-decoder features, we first fuse them with early and final ViT features&#10;for improved mask details. To train our introduced learnable parameters, we&#10;compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is&#10;only trained on the introduced detaset of 44k masks, which takes only 4 hours&#10;on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 10 diverse segmentation&#10;datasets across different downstream tasks, where 8 out of them are evaluated&#10;in a zero-shot transfer protocol. Our code and pretrained models are at&#10;https://github.com/SysCV/SAM-HQ.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\parsection{High-quality Segmentation} Existing works for high-quality segmentation are mostly trained for a specific segmentation task, like image and video instance segmentation~\cite{kirillov2020pointrend,transfiner,vmt,tang2021look,wen2023patchdct}, semantic segmentation~\cite{lin2017refinenet,zhao2018icnet,takikawa2019gated,yuan2020segfix} or panoptic segmentation~\cite{de2023intra}, in a close-world paradigm.&#10;Some of them focus on post-segmentation refinement using with graphical models such as CRF~\cite{krahenbuhl2011efficient} or region growing~\cite{dias2019semantic}.&#10;However, the CRF-based refinement is adhere to low-level color boundaries without fully utilizing high-level semantic context and cannot fix large segmentation errors. &#10;While some refinement-based works adopt separate deep networks for cascade iterative refinement~\cite{cascadepsp,shen2022high}, they are prone to overfitting as shown by our experiment.&#10;Compared to these high-quality segmentation~\cite{transfiner,kirillov2020pointrend,qilu2022finegrained} or segmentation refinement methods, we focus on accurately segmenting diverse objects on new data with flexible prompting, and build a high-quality zero-shot segmentation model that generalizes to various segmentation tasks and domains.&#10;Unlike the post segmentation refinement works~\cite{cascadepsp,shen2022high}, to preserve the zero-shot segmentation capability of SAM,&#10;HQ-SAM predicts the new high-quality mask directly by reusing the image encoder and mask decoder of SAM, instead of taking the coarse mask and images as the input and feeding it into a separate refinement network.&#10;The model architecture of HQ-SAM builds upon SAM with negligible overhead, where we propose efficient token learning for accurate mask predictions. This is completely different from previous high-quality segmentation works, and we show its effectiveness across a wide range of zero-shot experiments.&#10;&#10;\parsection{Fine-tuning and Prompt Tuning for Foundation Models} &#10;Foundation models~\cite{brown2020language,bommasani2021opportunities} first appear in the NLP community, where large language models such as GPT series~\cite{brown2020language} show strong zero-shot generalization to unseen tasks and data.&#10;Then, some prompt-based learning works~\cite{houlsby2019parameter,li2021prefix,hu2021lora} are proposed to help these pre-trained models generalize to the downstream tasks instead of fine-tuning the internal model parameters~\cite{hinterstoisser2018pre} for better transfer learning.&#10;For vision-based foundation models~\cite{sam,seggpt,zou2023segment}, prompt engineering~\cite{zhou2022coop,xing2022class,yao2021cpt,zhou2022zegclip} that freezes the pre-trained model is first explored in vision-language models, such as CLIP~\cite{clip}.&#10;These prompts with learnable parameters are designed to help downstream tasks with better context optimization.&#10;Different from the existing prompt-based or finetuning works, we focus on the minimal adaptation of SAM toward high-quality segmentation. We directly use the proposed HQ-Output Token output for accurate mask prediction, instead of only leveraging some learnable parameters~\cite{zhou2022coop} to help context learning and better generalization. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1910.13136" label="1910.13136">
-        <attvalues>
-          <attvalue for="0" value="An {\alpha}-Matte Boundary Defocus Model Based Cascaded Network for&#10;  Multi-focus Image Fusion" />
-          <attvalue for="1" value="  Capturing an all-in-focus image with a single camera is difficult since the&#10;depth of field of the camera is usually limited. An alternative method to&#10;obtain the all-in-focus image is to fuse several images focusing at different&#10;depths. However, existing multi-focus image fusion methods cannot obtain clear&#10;results for areas near the focused/defocused boundary (FDB). In this paper, a&#10;novel {\alpha}-matte boundary defocus model is proposed to generate realistic&#10;training data with the defocus spread effect precisely modeled, especially for&#10;areas near the FDB. Based on this {\alpha}-matte defocus model and the&#10;generated data, a cascaded boundary aware convolutional network termed MMF-Net&#10;is proposed and trained, aiming to achieve clearer fusion results around the&#10;FDB. More specifically, the MMF-Net consists of two cascaded sub-nets for&#10;initial fusion and boundary fusion, respectively; these two sub-nets are&#10;designed to first obtain a guidance map of FDB and then refine the fusion near&#10;the FDB. Experiments demonstrate that with the help of the new {\alpha}-matte&#10;boundary defocus model, the proposed MMF-Net outperforms the state-of-the-art&#10;methods both qualitatively and quantitatively.&#10;" />
-          <attvalue for="2" value="&#10;&#10;W{hen} photos are taken with cameras, all-in-focus images are often desired as the output, in particular for a large number of computer vision tasks, such as localization, detection and segmentation \cite{hassen2015objective}. However, it is usually hard to obtain an all-in-focus images from a single camera, since the depth of field of the camera is limited \cite{bouzos2019conditional}. Multi-focus image fusion is the approach to generating an all-in-focus image from several images taken on the same scene but focused at different depths, as shown in Fig.~\ref{fig:1} via an example of the fusion image obtained from two source images. &#10;&#10;Existing multi-focus image fusion (MFIF) methods can be broadly categorized into three groups, i.e., transform domain algorithms, spatial domain algorithms, and convolutional neural network (CNN) based algorithms \cite{kaur2015comparative}.&#10;&#10;The transform domain MFIF algorithms first decompose the source images and then fuse the results according to some handcrafted features. Typical transform domain MFIF algorithms include the non-subsampled contourlet transform (NSCT) method \cite{ZHANG20091334}, the sparse representation (SR) method \cite{RN35, zhang2016robust} and the combined NSCT-SR method \cite{LIU2015147}. Due to the imperfection of transformations and handcrafted features, these algorithms often produce non-realistic results, even in the areas far away from the focused/defocused boundary (FDB).&#10;&#10;The spatial domain MFIF algorithms include region based methods and pixel based methods. Region based MFIF algorithms suffer from the blocking effect \cite{RN85}. Pixel based MFIF algorithms first get a 0/1 discrete decision map for fusion \cite{RN43}, and then fuse the source images. The guided filtering (GF) method \cite{RN80} and the dense SIFT (DSIFT) method \cite{RN43} are typical pixel based algorithms. Compared with the transform domain algorithms, the fusion results from the pixel based algorithms are usually better, However, due to the defocus effect, none of the source images are clear in the areas near the FDB, and consequently, the fusion results of these methods are often unclear in these areas.&#10;&#10;The CNN is first explored to extract defocus descriptors in a data-driven way in \cite{wang2010multi}. Existing neural network based MFIF algorithms can be divided into two groups: decision map based algorithms and end-to-end algorithms. Decision map based algorithms \cite{RN91, RN497, 8625482} produce the decision map first as with the pixel based algorithms, which, also similarly to the pixel-based algorithms, lead to the unclear FDB. End-to-end algorithms \cite{ma2019sesf, yan2018unsupervised, zhao2018multi} directly obtain the fusion results, but the results are unfortunately not realistic, as with the transfer domain algorithms. Moreover, since it is hard to get a large number of all-in-focus ground truth images for training, data generation methods need to be adopted in these CNN based algorithms \cite{RN91, RN497, 8625482, ma2019sesf, yan2018unsupervised, zhao2018multi}; however, these methods do not imitate the complex situation of defocus spread near the FDB, and thus some unnatural and unrealistic training data limits the performance of these networks.&#10;&#10;In this paper, to address the issue of unsatisfied fusion results near the FDB, we first present a discussion about the difficulties around the FDB. Then an $\alpha$-matte boundary defocus model is proposed to simulate the defocus spread effect near the FDB. Based on the $\alpha$-matte defocus model and the generated training data, we develop a cascaded network for MFIF, which is called the Matte Model Fusion Net (MMF-Net). The technical underpinnings and contributions of our work are two-fold:&#10;&#10;Firstly, a novel $\alpha$-matte boundary defocus spread model is proposed. Compared with existing defocus models, the proposed $\alpha$-matte model is the first one to specifically model the difference in defocus spread when the defocus happens to the foreground or the background. Therefore, the $\alpha$-matte model can generate simulated defocus images with the valid defocus spread near the FDB, which can be then used as training data to train deep neural networks. &#10;&#10;Secondly, based on the proposed $\alpha$-matte defocus model and the generated training data, a cascaded boundary aware convolutional network termed MMF-Net is designed and trained, to obtain clear fusion results in areas both far away from and near the FDB. Compared with the existing end-to-end CNN algorithms, the proposed MMF-Net generates a guidance map first, acquiring clear and realistic fusion results in areas far away from the FDB. Compared with the decision map based CNN algorithms, the MMF-Net generates the areas near the FDB directly from the source images, like the end to end CNN algorithms, and thus achieve clearer and reasonable fusion results than any source images near the FDB. &#10;&#10;Experiments demonstrate that, on the benchmark dataset, the proposed MMF-Net outperforms the state-of-the-art methods, both qualitatively and quantitatively.&#10;&#10;The rest of this paper is organised as follows. In section~\ref{s:model}, we present the issue to be addressed in this paper, introduce the proposed $\alpha$-matte boundary defocus model, and conduct comparison with the existing defocus models on the simulated scene. In section~\ref{s:matte}, we present the proposed MMF-Net and discuss the corresponding loss functions. Experimental studies including the method comparison are conducted in section~\ref{s:exp}, and conclusions and future work are drawn in section~\ref{s:conclusions}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Optics, Computer Vision Applications, Mathematics, Physics, Computer Science, Image Fusion Techniques, Deep Learning Networks" />
-        </attvalues>
-      </node>
-      <node id="1908.01703" label="1908.01703">
-        <attvalues>
-          <attvalue for="0" value="SESF-Fuse: An Unsupervised Deep Model for Multi-Focus Image Fusion" />
-          <attvalue for="1" value="  In this work, we propose a novel unsupervised deep learning model to address&#10;multi-focus image fusion problem. First, we train an encoder-decoder network in&#10;unsupervised manner to acquire deep feature of input images. And then we&#10;utilize these features and spatial frequency to measure activity level and&#10;decision map. Finally, we apply some consistency verification methods to adjust&#10;the decision map and draw out fused result. The key point behind of proposed&#10;method is that only the objects within the depth-of-field (DOF) have sharp&#10;appearance in the photograph while other objects are likely to be blurred. In&#10;contrast to previous works, our method analyzes sharp appearance in deep&#10;feature instead of original image. Experimental results demonstrate that the&#10;proposed method achieves the state-of-art fusion performance compared to&#10;existing 16 fusion methods in objective and subjective assessment.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.08373" label="1910.08373">
-        <attvalues>
-          <attvalue for="0" value="Deformable Kernel Networks for Joint Image Filtering" />
-          <attvalue for="1" value="  Joint image filters are used to transfer structural details from a guidance&#10;picture used as a prior to a target image, in tasks such as enhancing spatial&#10;resolution and suppressing noise. Previous methods based on convolutional&#10;neural networks (CNNs) combine nonlinear activations of spatially-invariant&#10;kernels to estimate structural details and regress the filtering result. In&#10;this paper, we instead learn explicitly sparse and spatially-variant kernels.&#10;We propose a CNN architecture and its efficient implementation, called the&#10;deformable kernel network (DKN), that outputs sets of neighbors and the&#10;corresponding weights adaptively for each pixel. The filtering result is then&#10;computed as a weighted average. We also propose a fast version of DKN that runs&#10;about seventeen times faster for an image of size 640 x 480. We demonstrate the&#10;effectiveness and flexibility of our models on the tasks of depth map&#10;upsampling, saliency map upsampling, cross-modality image restoration, texture&#10;removal, and semantic segmentation. In particular, we show that the weighted&#10;averaging process with sparsely sampled 3 x 3 kernels outperforms the state of&#10;the art by a significant margin in all cases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Here we briefly describe the context of our approach, and review representative works related to ours. &#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Computer Science, Image Filtering, Neural Network Architecture, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2108.13055" label="2108.13055">
-        <attvalues>
-          <attvalue for="0" value="Efficient Visual Recognition with Deep Neural Networks: A Survey on&#10;  Recent Advances and New Directions" />
-          <attvalue for="1" value="  Visual recognition is currently one of the most important and active research&#10;areas in computer vision, pattern recognition, and even the general field of&#10;artificial intelligence. It has great fundamental importance and strong&#10;industrial needs. Deep neural networks (DNNs) have largely boosted their&#10;performances on many concrete tasks, with the help of large amounts of training&#10;data and new powerful computation resources. Though recognition accuracy is&#10;usually the first concern for new progresses, efficiency is actually rather&#10;important and sometimes critical for both academic research and industrial&#10;applications. Moreover, insightful views on the opportunities and challenges of&#10;efficiency are also highly required for the entire community. While general&#10;surveys on the efficiency issue of DNNs have been done from various&#10;perspectives, as far as we are aware, scarcely any of them focused on visual&#10;recognition systematically, and thus it is unclear which progresses are&#10;applicable to it and what else should be concerned. In this paper, we present&#10;the review of the recent advances with our suggestions on the new possible&#10;directions towards improving the efficiency of DNN-related visual recognition&#10;approaches. We investigate not only from the model but also the data point of&#10;view (which is not the case in existing surveys), and focus on three most&#10;studied data types (images, videos and points). This paper attempts to provide&#10;a systematic summary via a comprehensive survey which can serve as a valuable&#10;reference and inspire both researchers and practitioners who work on visual&#10;recognition problems.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Deep neural networks (DNNs) have achieved great successes on many visual recognition tasks. They have largely improved the performances of long-lasting problems such as handwritten digit recognition \cite{Lecun_1998_MNIST_PIEEE}, face recognition \cite{Hinton_2006_Science}, image categorization \cite{Krizhevsky_2012_AlexNet_NIPS}, etc. They are also enabling exploring new boundaries including the studies on image and video captioning \cite{Lin_2014_COCO_ECCV,Song_2019_MS-RNNs_TNNLS,Gao_2019_HLSTMs_PAMI}, body pose estimation \cite{Wei_2016_CPM_CVPR}, and many others. However, such successes are generally conditioned on huge amounts of high-quality hand labelled training data and the recently greatly advanced computational resources. Obviously, these two conditions are usually too expensive to be satisfied in most applications which are cost-sensitive. Even for the cases that { people} do have enough high-quality training data thanks to the massive efforts of many annotators, it is usually a great challenge to figure out how to train an effective model with limited resources and within acceptable time. Assuming somehow the model can be properly trained (no matter how much efforts it costs), it is still not easy to have the model properly deployed for real applications at end users' side, as the run-time inference has to fit the available or affordable resources, and the running speed has to meet the actual needs which can be real-time or even more than that. Therefore, besides accuracy which is usually the biggest concern in the academia, efficiency is another important issue and in most cases an indispensable demand for real applications.&#10;&#10;Though most of the works on using DNNs for visual recognition tasks are focusing on accuracy, there are still a lot of encouraging progresses on the efficiency side, especially in the recent few years. In the past two years, many survey papers have been published on efficiency issue for DNNs, as detailed in the following section\ref{subsec:related_surveys}. However, none of them pays a major attention to visual recognition tasks, especially missing the coverage of special efforts for efficiently dealing with visual data, which has its own properties. In practice, efficient visual recognition has to be a systematic solution which takes into account of not only compact/compressed networks and hardware acceleration, but also proper handling of visual data, which may be of various types (such as images, videos, and points) with quite different properties. That can be an important reason for the lack of a survey on this topic. Therefore, { as far as we are aware, this paper provides} the first survey on efficient visual recognition with DNNs. It targets at a systematic overview of recent advances and trends from various aspects, based on our expertise and experiences on major types of visual data, their various recognition models, and network compression algorithms. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Deep Neural Networks, Artificial Intelligence, Efficiency Optimization, Computer Science, Pattern Recognition, Visual Recognition" />
-        </attvalues>
-      </node>
-      <node id="1812.11004" label="1812.11004">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical LSTMs with Adaptive Attention for Visual Captioning" />
-          <attvalue for="1" value="  Recent progress has been made in using attention based encoder-decoder&#10;framework for image and video captioning. Most existing decoders apply the&#10;attention mechanism to every generated word including both visual words (e.g.,&#10;&quot;gun&quot; and &quot;shooting&quot;) and non-visual words (e.g. &quot;the&quot;, &quot;a&quot;). However, these&#10;non-visual words can be easily predicted using natural language model without&#10;considering visual signals or attention. Imposing attention mechanism on&#10;non-visual words could mislead and decrease the overall performance of visual&#10;captioning. Furthermore, the hierarchy of LSTMs enables more complex&#10;representation of visual data, capturing information at different scales. To&#10;address these issues, we propose a hierarchical LSTM with adaptive attention&#10;(hLSTMat) approach for image and video captioning. Specifically, the proposed&#10;framework utilizes the spatial or temporal attention for selecting specific&#10;regions or frames to predict the related words, while the adaptive attention is&#10;for deciding whether to depend on the visual information or the language&#10;context information. Also, a hierarchical LSTMs is designed to simultaneously&#10;consider both low-level visual information and high-level language context&#10;information to support the caption generation. We initially design our hLSTMat&#10;for video captioning task. Then, we further refine it and apply it to image&#10;captioning task. To demonstrate the effectiveness of our proposed framework, we&#10;test our method on both video and image captioning tasks. Experimental results&#10;show that our approach achieves the state-of-the-art performance for most of&#10;the evaluation metrics on both tasks. The effect of important components is&#10;also well exploited in the ablation study.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.07142" label="2208.07142">
-        <attvalues>
-          <attvalue for="0" value="Perspective Reconstruction of Human Faces by Joint Mesh and Landmark&#10;  Regression" />
-          <attvalue for="1" value="  Even though 3D face reconstruction has achieved impressive progress, most&#10;orthogonal projection-based face reconstruction methods can not achieve&#10;accurate and consistent reconstruction results when the face is very close to&#10;the camera due to the distortion under the perspective projection. In this&#10;paper, we propose to simultaneously reconstruct 3D face mesh in the world space&#10;and predict 2D face landmarks on the image plane to address the problem of&#10;perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D&#10;landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by&#10;the PnP solver to represent perspective projection. Our approach achieves 1st&#10;place on the leader-board of the ECCV 2022 WCPA challenge and our model is&#10;visually robust under different identities, expressions and poses. The training&#10;code and models are released to facilitate future research.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Monocular 3D face reconstruction has been widely applied in many fields such as VR/AR applications (e.g., movies, sports, games), video editing, and virtual avatars. Reconstructing human faces from monocular RGB data is a well-explored field and most of the approaches can be categorized into optimization-based methods \cite{Blanz1999,booth2018large,ganfit} or regression-based methods \cite{zhu20153ddfa,feng2018joint,deng2020retinaface,deng2019accurate}.&#10;&#10;For the optimization-based methods, a prior of face shape and appearance \cite{Blanz1999,booth2018large} is used. The pioneer work 3D Morphable Model (3DMM) \cite{Blanz1999} represents the face shape and appearance in a PCA-based compact space and the fitting is then based on the principle of analysis-by-synthesis. In \cite{booth2018large}, an in-the-wild texture model is employed to greatly simplify the fitting procedure without optimization on the illumination parameters. To model textures in high fidelity, GANFit \cite{ganfit} harnesses Generative Adversarial Networks (GANs) to train a very powerful generator of facial texture in UV space and constrains the latent parameter by state-of-the-art face recognition model~\cite{deng2019arcface,arcface,deng2021variational,an2022killing,zhu2021webface260m,zhu2022webface260m}. However, the iterative optimization in these methods is not efficient for real-time inference.&#10;&#10;For the regression-based methods, a series of methods are based on synthetic renderings of human faces \cite{genova2018unsupervised,wood20223d} or 3DMM fitted data \cite{zhu20153ddfa} to perform a supervised training of a regressor that predicts the latent representation of a prior face model (e.g., 3DMM~\cite{tran2016regressing}, GCN~\cite{zhou2019dense}, CNN~\cite{tran2018nonlinear}) or 3D vertices in different representation formats~\cite{jackson2017large,gueler2017densereg,feng2018joint,deng2020retinaface}. Genova et al.~\cite{genova2018unsupervised} propose a 3DMM parameter regression technique that is based on synthetic renderings and Tran et al.~\cite{tran2016regressing} directly regress 3DMM parameters using a CNN trained on fitted 3DMM data. Zhu et al.~\cite{zhu20153ddfa} propose 3D Dense Face Alignment (3DDFA) by taking advantage of Projected Normalized Coordinate Code (PNCC). In \cite{zhou2019dense}, joint shape and texture auto-encoder using direct mesh convolutions is proposed based on Graph Convolutional Network (GCN). &#10;In \cite{tran2018nonlinear}, CNN-based shape and texture decoders are trained on unwrapped UV space for non-linear 3D morphable face modelling. Jackson et al.~\cite{jackson2017large} propose a model-free approach that reconstructs a voxel-based representation of the human face. DenseReg~\cite{gueler2017densereg} regresses horizontal and vertical tessellation which is obtained by unwrapping the template shape and transferring it to the image domain.&#10;PRN~\cite{feng2018joint} predicts a position map in the UV space of a template mesh. RetinaFace~\cite{deng2020retinaface} directly predicts projected vertices on the image plane within the face detector. Besides these supervised regression methods, there are also self-supervised regression approaches. Deng et al.~\cite{deng2019accurate} train a 3DMM parameter regressor based on photometric reconstruction loss with skin attention masks, a perception loss based on FaceNet~\cite{facenet}, and multi-image consistency losses. DECA~\cite{deca} robustly produces a UV displacement map from a low-dimensional latent representation.&#10;&#10;Although the above studies have achieved good face reconstruction results, existing methods mainly use orthogonal projection to simplify the projection process of the face. When the face is close to the camera, the distortion caused by perspective projection can not be ignored \cite{kao2022single,zielonka2022towards}. Due to the use of orthogonal projection, existing methods can not well explain the facial distortion caused by perspective projection, leading to poor performance.&#10;&#10;To this end, we propose a joint 3D mesh and 2D landmark regression method in this paper. Monocular 3D face reconstruction usually includes two tasks: 3D face geometric reconstruction and face pose estimation. However, we avoid explicit face pose estimation as in RetinaFace~\cite{deng2020retinaface} and DenseLandmark~\cite{wood20223d}. The insight behind this strategy is that the evaluation metric for face reconstruction is usually in the camera space and the regression error on 6DoF parameters will repeat to every vertex. Even though dense vertex or landmark regression seems redundant, the regression error of each point is individual. Most important, explicit pose parameter regression can result in drift-alignment problem \cite{chaudhuri2019joint}, which will make the 2D visualization of face reconstruction unsatisfying. &#10;By contrast, the 6 Degrees of Freedom (6DoF) face pose can be easily estimated by the PnP solver based on our predicted 3D vertices and 2D landmarks. &#10;&#10;To summarize, the main contributions of this work are:&#10;\begin{itemize}&#10; \item We propose a Joint Mesh and Landmark Regression (JMLR) method to reconstruct 3D face shape under perspective projection, and the 6DoF face pose can be further estimated by PnP.&#10; \item The proposed JMLR achieves first place on the leader-board of the ECCV 2022 WCPA challenge.&#10; \item The visualization results show that the proposed JMLR is robust under different identities, exaggerated expressions and extreme poses.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Face Reconstruction, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1902.05978" label="1902.05978">
-        <attvalues>
-          <attvalue for="0" value="GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face&#10;  Reconstruction" />
-          <attvalue for="1" value="  In the past few years, a lot of work has been done towards reconstructing the&#10;3D facial structure from single images by capitalizing on the power of Deep&#10;Convolutional Neural Networks (DCNNs). In the most recent works, differentiable&#10;renderers were employed in order to learn the relationship between the facial&#10;identity features and the parameters of a 3D morphable model for shape and&#10;texture. The texture features either correspond to components of a linear&#10;texture space or are learned by auto-encoders directly from in-the-wild images.&#10;In all cases, the quality of the facial texture reconstruction of the&#10;state-of-the-art methods is still not capable of modeling textures in high&#10;fidelity. In this paper, we take a radically different approach and harness the&#10;power of Generative Adversarial Networks (GANs) and DCNNs in order to&#10;reconstruct the facial texture and shape from single images. That is, we&#10;utilize GANs to train a very powerful generator of facial texture in UV space.&#10;Then, we revisit the original 3D Morphable Models (3DMMs) fitting approaches&#10;making use of non-linear optimization to find the optimal latent parameters&#10;that best reconstruct the test image but under a new perspective. We optimize&#10;the parameters with the supervision of pretrained deep identity features&#10;through our end-to-end differentiable framework. We demonstrate excellent&#10;results in photorealistic and identity preserving 3D face reconstructions and&#10;achieve for the first time, to the best of our knowledge, facial texture&#10;reconstruction with high-frequency details.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.07212" label="1511.07212">
-        <attvalues>
-          <attvalue for="0" value="Face Alignment Across Large Poses: A 3D Solution" />
-          <attvalue for="1" value="  Face alignment, which fits a face model to an image and extracts the semantic&#10;meanings of facial pixels, has been an important topic in CV community.&#10;However, most algorithms are designed for faces in small to medium poses (below&#10;45 degree), lacking the ability to align faces in large poses up to 90 degree.&#10;The challenges are three-fold: Firstly, the commonly used landmark-based face&#10;model assumes that all the landmarks are visible and is therefore not suitable&#10;for profile views. Secondly, the face appearance varies more dramatically&#10;across large poses, ranging from frontal view to profile view. Thirdly,&#10;labelling landmarks in large poses is extremely challenging since the invisible&#10;landmarks have to be guessed. In this paper, we propose a solution to the three&#10;problems in an new alignment framework, called 3D Dense Face Alignment (3DDFA),&#10;in which a dense 3D face model is fitted to the image via convolutional neutral&#10;network (CNN). We also propose a method to synthesize large-scale training&#10;samples in profile views to solve the third problem of data labelling.&#10;Experiments on the challenging AFLW database show that our approach achieves&#10;significant improvements over state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07835" label="1803.07835">
-        <attvalues>
-          <attvalue for="0" value="Joint 3D Face Reconstruction and Dense Alignment with Position Map&#10;  Regression Network" />
-          <attvalue for="1" value="  We propose a straightforward method that simultaneously reconstructs the 3D&#10;facial structure and provides dense alignment. To achieve this, we design a 2D&#10;representation called UV position map which records the 3D shape of a complete&#10;face in UV space, then train a simple Convolutional Neural Network to regress&#10;it from a single 2D image. We also integrate a weight mask into the loss&#10;function during training to improve the performance of the network. Our method&#10;does not rely on any prior face model, and can reconstruct full facial geometry&#10;along with semantic meaning. Meanwhile, our network is very light-weighted and&#10;spends only 9.8ms to process an image, which is extremely faster than previous&#10;works. Experiments on multiple challenging datasets show that our method&#10;surpasses other state-of-the-art methods on both reconstruction and alignment&#10;tasks by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.08527" label="1903.08527">
-        <attvalues>
-          <attvalue for="0" value="Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From&#10;  Single Image to Image Set" />
-          <attvalue for="1" value="  Recently, deep learning based 3D face reconstruction methods have shown&#10;promising results in both quality and efficiency.However, training deep neural&#10;networks typically requires a large volume of data, whereas face images with&#10;ground-truth 3D face shapes are scarce. In this paper, we propose a novel deep&#10;3D face reconstruction approach that 1) leverages a robust, hybrid loss&#10;function for weakly-supervised learning which takes into account both low-level&#10;and perception-level information for supervision, and 2) performs multi-image&#10;face reconstruction by exploiting complementary information from different&#10;images for shape aggregation. Our method is fast, accurate, and robust to&#10;occlusion and large pose. We provide comprehensive experiments on three&#10;datasets, systematically comparing our method with fifteen recent methods and&#10;demonstrating its state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.07698" label="1801.07698">
-        <attvalues>
-          <attvalue for="0" value="ArcFace: Additive Angular Margin Loss for Deep Face Recognition" />
-          <attvalue for="1" value="  Recently, a popular line of research in face recognition is adopting margins&#10;in the well-established softmax loss function to maximize class separability.&#10;In this paper, we first introduce an Additive Angular Margin Loss (ArcFace),&#10;which not only has a clear geometric interpretation but also significantly&#10;enhances the discriminative power. Since ArcFace is susceptible to the massive&#10;label noise, we further propose sub-center ArcFace, in which each class&#10;contains $K$ sub-centers and training samples only need to be close to any of&#10;the $K$ positive sub-centers. Sub-center ArcFace encourages one dominant&#10;sub-class that contains the majority of clean faces and non-dominant&#10;sub-classes that include hard or noisy faces. Based on this self-propelled&#10;isolation, we boost the performance through automatically purifying raw web&#10;faces under massive real-world noise. Besides discriminative feature embedding,&#10;we also explore the inverse problem, mapping feature vectors to face images.&#10;Without training any additional generator or discriminator, the pre-trained&#10;ArcFace model can generate identity-preserved face images for both subjects&#10;inside and outside the training data only by using the network gradient and&#10;Batch Normalization (BN) priors. Extensive experiments demonstrate that ArcFace&#10;can enhance the discriminative feature embedding as well as strengthen the&#10;generative face synthesis.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;{Face Recognition with Margin Penalty.} {As shown in Figure \ref{fig:comparisons},} the pioneering work \cite{schroff2015facenet} uses the Triplet loss to exploit triplet data such that faces from the same class are closer than faces from different classes by a clear Euclidean distance margin. Even though the Triplet loss makes perfect sense for face recognition, the sample-to-sample comparisons are local within mini-batch and the training procedure for the Triplet loss is very challenging as there is a combinatorial explosion in the number of triplets especially for large-scale datasets, requiring effective sampling strategies to select informative mini-batch~\cite{rippel2015metric,schroff2015facenet} and choose representative triplets within the mini-batch~\cite{oh2016deep,sohn2016improved}.&#10;{&#10;As the Triplet loss trained with semi-hard negative mining converges slower due to the ignorance of too many examples, a double-margin contrastive loss is proposed in \cite{wu2017sampling} to explore more informative and stable examples by distance weighted sampling, thus it converges faster and more accurately.}&#10;Some other works tried to reduce the total number of triplets with proxies~\cite{movshovitz2017no,qian2018large}, \ie, sample-to-sample comparison is changed into sample-to-proxy comparison. However, sampling and proxy methods only optimize the embedding of partial classes instead of all classes in one iteration step.&#10;&#10;Margin-based softmax methods~\cite{liu2017sphereface,deng2017marginal,tencent2017CosineFace,wang2018additive} focused on incorporating margin penalty into a more feasible framework, softmax loss, which has extensive sample-to-class comparisons. Compared to deep metric learning methods (\eg, Triplet~\cite{schroff2015facenet}, Tuplet~\cite{oh2016deep,sohn2016improved}), margin-based softmax methods conduct global comparisons at the cost of memory consumption on holding the center of each class {as illustrated in Figure \ref{fig:comparisons}.} Sample-to-class comparison is more efficient and stable than sample-to-sample comparison as (1) the class number is much smaller than sample number, and (2) each class can be represented by a smoothed center vector which can be updated online during training. To further improve the margin-based softmax loss, recent works focus on the exploration of adaptive parameters \cite{zhang2019p2sgrad,zhang2019adacos,liu2019adaptiveface}, inter-class regularization \cite{zhao2019regularface,duan2019uniformface}, mining \cite{wang2019mis,huang2020curricularface}, grouping \cite{kim2020groupface}, etc.&#10;&#10;{Face Recognition under Noise.} Most of the face recognition datasets~\cite{yi2014learning,guo2016ms,cao2017vggface2,cao2018celeb} are downloaded from the Internet by searching a pre-defined celebrity list, and the original labels are likely to be ambiguous and inaccurate~\cite{wang2018devil}.&#10;Learning with massive noisy data has recently drawn much attention in face recognition~\cite{wu2018light,hu2019noise,Zhong2019Unequal,wang2019co} as accurate manual annotations can be expensive~\cite{wang2018devil} or even unavailable.&#10;&#10;Wu~\etal~\cite{wu2018light} proposed a semantic bootstrap strategy, which re-labels the noisy samples according to the probabilities of the softmax function. However, automatic cleaning by the bootstrapping rule requires time-consuming iterations (\eg twice refinement steps are used in~\cite{wu2018light}) and the labelling quality is affected by the capacity of the original model. Hu~\etal~\cite{hu2019noise} found that the cleanness possibility of a sample can be dynamically reflected by its position in the target logit distribution and presented a noise-tolerant end-to-end paradigm by employing the idea of weighting training samples. Zhong~\etal~\cite{Zhong2019Unequal} devised a noise-resistant loss by introducing a hypothetical training label, which is a convex combination of the original label with probability $\rho$ and the predicted label by the current model with probability $1-\rho$.&#10;However, computing time-varying fusion weight~\cite{hu2019noise} and designing piece-wise loss~\cite{Zhong2019Unequal} contain many hand-designed hyper-parameters. Besides, re-weighting methods are susceptible to the performance of the initial model. Wang~\etal~\cite{wang2019co} proposed a co-mining strategy which uses the loss values as the cue to simultaneously detect noisy labels, exchange the high-confidence clean faces to alleviate the error accumulation caused by the sampling bias, and re-weight the predicted clean faces to make them dominate the discriminative model training. However, the co-mining method requires training twin networks simultaneously and it is challenging to train large networks (\eg ResNet100~\cite{he2016deep}) on a large-scale dataset (\eg MS1MV0~\cite{guo2016ms} and Celeb500K~\cite{cao2018celeb}). &#10;&#10;{Face Recognition with Sub-classes.} Practices and theories that lead to ``sub-class'' have been studied for a long&#10;time~\cite{zhu2004optimal,zhu2006subclass}. The concept of ``sub-class'' applied in face recognition was first introduced in~\cite{zhu2004optimal,zhu2006subclass}, &#10;where a mixture of Gaussians was used to approximate the underlying distribution of each class. For instance, a person’s face images may be frontal view or side view, resulting in different modalities when all images are represented in the same data space. In~\cite{zhu2004optimal,zhu2006subclass}, experimental results showed that subclass divisions can be used to effectively adapt to different face modalities thus improve the performance of face recognition.&#10;Wan~\etal~\cite{wan2017separability} further proposed a separability criterion to divide every class into sub-classes, which have much less overlaps. The new within-class scatter can represent multi-modality information, therefore optimizing this within-class scatter will separate different modalities more clearly and further increase the accuracy of face recognition. However, these work~\cite{zhu2004optimal,zhu2006subclass,wan2017separability} only employed hand-designed feature descriptor on tiny under-controlled datasets. &#10;&#10;Concurrent with our work, Softtriple~\cite{qian2019softtriple} presents a multi-center softmax loss with class-wise regularizer.&#10;These multi-centers can depict the hidden distribution of the data~\cite{muller2020subclass} due to the fact that they can capture the complex geometry of the original data and help reduce the intra-class variance. On the fine-grained visual retrieval problem, the Softtriple~\cite{qian2019softtriple} loss achieves better performance than the softmax loss as capturing local clusters is essential for this task.&#10;Even though the concept of ``sub-class'' has been employed in face recognition~\cite{zhu2004optimal,zhu2006subclass,wan2017separability} and fine-grained visual retrieval~\cite{qian2019softtriple}, none of these work has considered the large-scale (\eg 0.5 million classes) face recognition problem under massive noise (\eg around $50\%$ noisy samples within the training data). &#10;&#10;{Face Synthesis by Model Inversion.} {Identity-preserving face generation \cite{shen2018faceid,bao2018towards,li2020advancing,duong2020vec2face} has been extensively explored under the framework of GAN \cite{goodfellow2014generative}. Even though GAN models can yield high-fidelity images \cite{brock2018large,karras2019style}, training a GAN's generator requires access to the original data. Due to the emerging concern of data privacy, an alternative line of work in security focuses on model inversion, that is, image synthesis from a single CNN. &#10;Model inversion can not only help researchers to visualize neural networks to understand their properties \cite{samek2019explainable} but also can be used for data-free distillation, quantization and pruning \cite{yin2020dreaming,haroush2020knowledge,cai2020zeroq}.}&#10;Fredrikson \etal \cite{fredrikson2015model} propose the model inversion attack to obtain class images from a network through a gradient descent on the input. As the pixel space is so large compared to the feature space, optimizing the image pixels by gradient descent \cite{mahendran2015understanding} requires heavy regularization terms, such as total variation \cite{mahendran2015understanding} or Gaussian blur \cite{yosinski2015understanding}. Even though previous model inversion methods \cite{fredrikson2015model,mordvintsev2015inceptionism} can transform an input image (random noise or a natural image) to yield a high output activation for a chosen class, it leaves intermediate representations constraint-free. Therefore, the resulting images are not realistic, lacking natural image statistics.&#10;&#10;The pioneer generative face recognition model is Eigenface~\cite{turk1991face}, which can project a training face image or a new face image (mean-subtracted) on the eigenfaces and thereby record how that face differs from the mean face. The eigenvalue associated with each eigenface represents how much the image vary from the mean image in that direction. The recognition process with the eigenface method is to project query images into the face-space spanned by eigenfaces calculated, and to find the closest match to a face class in that face-space. Even though raw pixel features used in Eigenface are substituted by the deep convolutional features, the procedure of employing the statistic prior (\eg mean and variance) to reconstruct face images can be an inspiration. {Recently, \cite{yin2020dreaming,haroush2020knowledge,cai2020zeroq} have proposed a data-free method employing the statistics (\eg mean and variance) stored in the BN layers to restore ImageNet images.} &#10;Inspired by these works, we synthesize face images by inverting the pre-trained ArcFace model and considering the face prior (\eg mean and variance) stored in the BN layers. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2103.04098" label="2103.04098">
-        <attvalues>
-          <attvalue for="0" value="WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face&#10;  Recognition" />
-          <attvalue for="1" value="  In this paper, we contribute a new million-scale face benchmark containing&#10;noisy 4M identities/260M faces (WebFace260M) and cleaned 2M identities/42M&#10;faces (WebFace42M) training data, as well as an elaborately designed&#10;time-constrained evaluation protocol. Firstly, we collect 4M name list and&#10;download 260M faces from the Internet. Then, a Cleaning Automatically utilizing&#10;Self-Training (CAST) pipeline is devised to purify the tremendous WebFace260M,&#10;which is efficient and scalable. To the best of our knowledge, the cleaned&#10;WebFace42M is the largest public face recognition training set and we expect to&#10;close the data gap between academia and industry. Referring to practical&#10;scenarios, Face Recognition Under Inference Time conStraint (FRUITS) protocol&#10;and a test set are constructed to comprehensively evaluate face matchers.&#10;  Equipped with this benchmark, we delve into million-scale face recognition&#10;problems. A distributed framework is developed to train face recognition models&#10;efficiently without tampering with the performance. Empowered by WebFace42M, we&#10;reduce relative 40% failure rate on the challenging IJB-C set, and ranks the&#10;3rd among 430 entries on NIST-FRVT. Even 10% data (WebFace4M) shows superior&#10;performance compared with public training set. Furthermore, comprehensive&#10;baselines are established on our rich-attribute test set under&#10;FRUITS-100ms/500ms/1000ms protocol, including MobileNet, EfficientNet,&#10;AttentionNet, ResNet, SENet, ResNeXt and RegNet families. Benchmark website is&#10;https://www.face-benchmark.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.10149" label="2204.10149">
-        <attvalues>
-          <attvalue for="0" value="WebFace260M: A Benchmark for Million-Scale Deep Face Recognition" />
-          <attvalue for="1" value="  Face benchmarks empower the research community to train and evaluate&#10;high-performance face recognition systems. In this paper, we contribute a new&#10;million-scale recognition benchmark, containing uncurated 4M identities/260M&#10;faces (WebFace260M) and cleaned 2M identities/42M faces (WebFace42M) training&#10;data, as well as an elaborately designed time-constrained evaluation protocol.&#10;Firstly, we collect 4M name lists and download 260M faces from the Internet.&#10;Then, a Cleaning Automatically utilizing Self-Training (CAST) pipeline is&#10;devised to purify the tremendous WebFace260M, which is efficient and scalable.&#10;To the best of our knowledge, the cleaned WebFace42M is the largest public face&#10;recognition training set and we expect to close the data gap between academia&#10;and industry. Referring to practical deployments, Face Recognition Under&#10;Inference Time conStraint (FRUITS) protocol and a new test set with rich&#10;attributes are constructed. Besides, we gather a large-scale masked face&#10;sub-set for biometrics assessment under COVID-19. For a comprehensive&#10;evaluation of face matchers, three recognition tasks are performed under&#10;standard, masked and unbiased settings, respectively. Equipped with this&#10;benchmark, we delve into million-scale face recognition problems. A distributed&#10;framework is developed to train face recognition models efficiently without&#10;tampering with the performance. Enabled by WebFace42M, we reduce 40% failure&#10;rate on the challenging IJB-C set and rank 3rd among 430 entries on NIST-FRVT.&#10;Even 10% data (WebFace4M) shows superior performance compared with the public&#10;training sets. Furthermore, comprehensive baselines are established under the&#10;FRUITS-100/500/1000 milliseconds protocols. The proposed benchmark shows&#10;enormous potential on standard, masked and unbiased face recognition scenarios.&#10;Our WebFace260M website is https://www.face-benchmark.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.06098" label="1806.06098">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Training for 3D Morphable Model Regression" />
-          <attvalue for="1" value="  We present a method for training a regression network from image pixels to 3D&#10;morphable model coordinates using only unlabeled photographs. The training loss&#10;is based on features from a facial recognition network, computed on-the-fly by&#10;rendering the predicted faces with a differentiable renderer. To make training&#10;from features feasible and avoid network fooling effects, we introduce three&#10;objectives: a batch distribution loss that encourages the output distribution&#10;to match the distribution of the morphable model, a loopback loss that ensures&#10;the network can correctly reinterpret its own output, and a multi-view identity&#10;loss that compares the features of the predicted 3D face and the input&#10;photograph from multiple viewing angles. We train a regression network using&#10;these objectives, a set of unlabeled photographs, and the morphable model&#10;itself, and demonstrate state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.04904" label="1612.04904">
-        <attvalues>
-          <attvalue for="0" value="Regressing Robust and Discriminative 3D Morphable Models with a very&#10;  Deep Neural Network" />
-          <attvalue for="1" value="  The 3D shapes of faces are well known to be discriminative. Yet despite this,&#10;they are rarely used for face recognition and always under controlled viewing&#10;conditions. We claim that this is a symptom of a serious but often overlooked&#10;problem with existing methods for single view 3D face reconstruction: when&#10;applied &quot;in the wild&quot;, their 3D estimates are either unstable and change for&#10;different photos of the same subject or they are over-regularized and generic.&#10;In response, we describe a robust method for regressing discriminative 3D&#10;morphable face models (3DMM). We use a convolutional neural network (CNN) to&#10;regress 3DMM shape and texture parameters directly from an input photo. We&#10;overcome the shortage of training data required for this purpose by offering a&#10;method for generating huge numbers of labeled examples. The 3D estimates&#10;produced by our CNN surpass state of the art accuracy on the MICC data set.&#10;Coupled with a 3D-3D face matching pipeline, we show the first competitive face&#10;recognition results on the LFW, YTF and IJB-A benchmarks using 3D face shapes&#10;as representations, rather than the opaque deep feature vectors used by other&#10;modern systems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.03786" label="1804.03786">
-        <attvalues>
-          <attvalue for="0" value="Nonlinear 3D Face Morphable Model" />
-          <attvalue for="1" value="  As a classic statistical model of 3D facial shape and texture, 3D Morphable&#10;Model (3DMM) is widely used in facial analysis, e.g., model fitting, image&#10;synthesis. Conventional 3DMM is learned from a set of well-controlled 2D face&#10;images with associated 3D face scans, and represented by two sets of PCA basis&#10;functions. Due to the type and amount of training data, as well as the linear&#10;bases, the representation power of 3DMM can be limited. To address these&#10;problems, this paper proposes an innovative framework to learn a nonlinear 3DMM&#10;model from a large set of unconstrained face images, without collecting 3D face&#10;scans. Specifically, given a face image as input, a network encoder estimates&#10;the projection, shape and texture parameters. Two decoders serve as the&#10;nonlinear 3DMM to map from the shape and texture parameters to the 3D shape and&#10;texture, respectively. With the projection parameter, 3D shape, and texture, a&#10;novel analytically-differentiable rendering layer is designed to reconstruct&#10;the original input face. The entire network is end-to-end trainable with only&#10;weak supervision. We demonstrate the superior representation power of our&#10;nonlinear 3DMM over its linear counterpart, and its contribution to face&#10;alignment and 3D reconstruction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.07834" label="1703.07834">
-        <attvalues>
-          <attvalue for="0" value="Large Pose 3D Face Reconstruction from a Single Image via Direct&#10;  Volumetric CNN Regression" />
-          <attvalue for="1" value="  3D face reconstruction is a fundamental Computer Vision problem of&#10;extraordinary difficulty. Current systems often assume the availability of&#10;multiple facial images (sometimes from the same subject) as input, and must&#10;address a number of methodological challenges such as establishing dense&#10;correspondences across large facial poses, expressions, and non-uniform&#10;illumination. In general these methods require complex and inefficient&#10;pipelines for model building and fitting. In this work, we propose to address&#10;many of these limitations by training a Convolutional Neural Network (CNN) on&#10;an appropriate dataset consisting of 2D images and 3D facial models or scans.&#10;Our CNN works with just a single 2D facial image, does not require accurate&#10;alignment nor establishes dense correspondence between images, works for&#10;arbitrary facial poses and expressions, and can be used to reconstruct the&#10;whole 3D facial geometry (including the non-visible parts of the face)&#10;bypassing the construction (during training) and fitting (during testing) of a&#10;3D Morphable Model. We achieve this via a simple CNN architecture that performs&#10;direct regression of a volumetric representation of the 3D facial geometry from&#10;a single 2D image. We also demonstrate how the related task of facial landmark&#10;localization can be incorporated into the proposed framework and help improve&#10;reconstruction quality, especially for the cases of large poses and facial&#10;expressions. Testing code will be made available online, along with pre-trained&#10;models http://aaronsplace.co.uk/papers/jackson2017recon&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.02188" label="1803.02188">
-        <attvalues>
-          <attvalue for="0" value="DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild" />
-          <attvalue for="1" value="  In this work we use deep learning to establish dense correspondences between&#10;a 3D object model and an image &quot;in the wild&quot;. We introduce &quot;DenseReg&quot;, a&#10;fully-convolutional neural network (F-CNN) that densely regresses at every&#10;foreground pixel a pair of U-V template coordinates in a single feedforward&#10;pass. To train DenseReg we construct a supervision signal by combining 3D&#10;deformable model fitting and 2D landmark annotations. We define the regression&#10;task in terms of the intrinsic, U-V coordinates of a 3D deformable model that&#10;is brought into correspondence with image instances at training time. A host of&#10;other object-related tasks (e.g. part segmentation, landmark localization) are&#10;shown to be by-products of this task, and to largely improve thanks to its&#10;introduction. We obtain highly-accurate regression results by combining ideas&#10;from semantic segmentation with regression networks, yielding a 'quantized&#10;regression' architecture that first obtains a quantized estimate of position&#10;through classification, and refines it through regression of the residual. We&#10;show that such networks can boost the performance of existing state-of-the-art&#10;systems for pose estimation. Firstly, we show that our system can serve as an&#10;initialization for Statistical Deformable Models, as well as an element of&#10;cascaded architectures that jointly localize landmarks and estimate dense&#10;correspondences. We also show that the obtained dense correspondence can act as&#10;a source of 'privileged information' that complements and extends the pure&#10;landmark-level annotations, accelerating and improving the training of pose&#10;estimation networks. We report state-of-the-art performance on the challenging&#10;300W benchmark for facial landmark localization and on the MPII and LSP&#10;datasets for human pose estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.03832" label="1503.03832">
-        <attvalues>
-          <attvalue for="0" value="FaceNet: A Unified Embedding for Face Recognition and Clustering" />
-          <attvalue for="1" value="  Despite significant recent advances in the field of face recognition,&#10;implementing face verification and recognition efficiently at scale presents&#10;serious challenges to current approaches. In this paper we present a system,&#10;called FaceNet, that directly learns a mapping from face images to a compact&#10;Euclidean space where distances directly correspond to a measure of face&#10;similarity. Once this space has been produced, tasks such as face recognition,&#10;verification and clustering can be easily implemented using standard techniques&#10;with FaceNet embeddings as feature vectors.&#10;  Our method uses a deep convolutional network trained to directly optimize the&#10;embedding itself, rather than an intermediate bottleneck layer as in previous&#10;deep learning approaches. To train, we use triplets of roughly aligned matching&#10;/ non-matching face patches generated using a novel online triplet mining&#10;method. The benefit of our approach is much greater representational&#10;efficiency: we achieve state-of-the-art face recognition performance using only&#10;128-bytes per face.&#10;  On the widely used Labeled Faces in the Wild (LFW) dataset, our system&#10;achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves&#10;95.12%. Our system cuts the error rate in comparison to the best published&#10;result by 30% on both datasets.&#10;  We also introduce the concept of harmonic embeddings, and a harmonic triplet&#10;loss, which describe different versions of face embeddings (produced by&#10;different networks) that are compatible to each other and allow for direct&#10;comparison between each other.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.04012" label="2012.04012">
-        <attvalues>
-          <attvalue for="0" value="Learning an Animatable Detailed 3D Face Model from In-The-Wild Images" />
-          <attvalue for="1" value="  While current monocular 3D face reconstruction methods can recover fine&#10;geometric details, they suffer several limitations. Some methods produce faces&#10;that cannot be realistically animated because they do not model how wrinkles&#10;vary with expression. Other methods are trained on high-quality face scans and&#10;do not generalize well to in-the-wild images. We present the first approach&#10;that regresses 3D face shape and animatable details that are specific to an&#10;individual but change with expression. Our model, DECA (Detailed Expression&#10;Capture and Animation), is trained to robustly produce a UV displacement map&#10;from a low-dimensional latent representation that consists of person-specific&#10;detail parameters and generic expression parameters, while a regressor is&#10;trained to predict detail, shape, albedo, expression, pose and illumination&#10;parameters from a single image. To enable this, we introduce a novel&#10;detail-consistency loss that disentangles person-specific details from&#10;expression-dependent wrinkles. This disentanglement allows us to synthesize&#10;realistic person-specific wrinkles by controlling expression parameters while&#10;keeping person-specific details unchanged. DECA is learned from in-the-wild&#10;images with no paired 3D supervision and achieves state-of-the-art shape&#10;reconstruction accuracy on two benchmarks. Qualitative results on in-the-wild&#10;data demonstrate DECA's robustness and its ability to disentangle identity- and&#10;expression-dependent details enabling animation of reconstructed faces. The&#10;model and code are publicly available at https://deca.is.tue.mpg.de.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.06607" label="2204.06607">
-        <attvalues>
-          <attvalue for="0" value="Towards Metrical Reconstruction of Human Faces" />
-          <attvalue for="1" value="  Face reconstruction and tracking is a building block of numerous applications&#10;in AR/VR, human-machine interaction, as well as medical applications. Most of&#10;these applications rely on a metrically correct prediction of the shape,&#10;especially, when the reconstructed subject is put into a metrical context&#10;(i.e., when there is a reference object of known size). A metrical&#10;reconstruction is also needed for any application that measures distances and&#10;dimensions of the subject (e.g., to virtually fit a glasses frame).&#10;State-of-the-art methods for face reconstruction from a single image are&#10;trained on large 2D image datasets in a self-supervised fashion. However, due&#10;to the nature of a perspective projection they are not able to reconstruct the&#10;actual face dimensions, and even predicting the average human face outperforms&#10;some of these methods in a metrical sense. To learn the actual shape of a face,&#10;we argue for a supervised training scheme. Since there exists no large-scale 3D&#10;dataset for this task, we annotated and unified small- and medium-scale&#10;databases. The resulting unified dataset is still a medium-scale dataset with&#10;more than 2k identities and training purely on it would lead to overfitting. To&#10;this end, we take advantage of a face recognition network pretrained on a&#10;large-scale 2D image dataset, which provides distinct features for different&#10;faces and is robust to expression, illumination, and camera changes. Using&#10;these features, we train our face shape estimator in a supervised fashion,&#10;inheriting the robustness and generalization of the face recognition network.&#10;Our method, which we call MICA (MetrIC fAce), outperforms the state-of-the-art&#10;reconstruction methods by a large margin, both on current non-metric benchmarks&#10;as well as on our metric benchmarks (15% and 24% lower average error on NoW,&#10;respectively).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Reconstructing human faces and heads from monocular RGB, RGB-D, or multiview data is a well-explored field at the intersectionof computer vision and computer graphics.&#10;Zollhöfer et al.~\cite{Zollhoefer2018FaceSTAR} provide an extensive review of reconstruction methods, focusing on optimization-based techniques that follow the principle of analysis-by-synthesis.&#10;Primarily, the approaches that are based on monocular inputs are based on a prior of face shape and appearance \cite{Garrido2014,Garrido2015,Weise2009,Weise2011,face2face,Thies15,Blanz2003,Blanz2004,deepvideoportraits,deferredneuralrendering,HeadOn,thies2018facevr,thies2020nvp}.&#10;The seminal work of Blanz et al.~\cite{Blanz1999} introduced such a 3D morphable model (3DMM), which represents the shape and appearance of a human in a compressed, low-dimensional, PCA-based space (which can be interpreted as a decoder with a single linear layer).&#10;There is a large corpus of different morphable models~\cite{3DMM_survey}, but the majority of reconstruction methods use either the Basel Face Model~\cite{Blanz1999,bfm09} or the Flame head model~\cite{flame}.&#10;Besides using these models for an analysis-by-synthesis approach, there is a series of learned regression-based methods.&#10;An overview of these methods is given by Morales et al.~\cite{morales2021survey}.&#10;In the following, we will discuss the most relevant related work for monocular RGB-based reconstruction methods.&#10;Optimization-based Reconstruction of Human Faces.&#10;Along with the introduction of a 3D morphable model for faces, Blanz et al.~\cite{Blanz1999} proposed an optimization-based reconstruction method that is based on the principle of analysis-by-synthesis.&#10;While they used a sparse sampling scheme to optimize the color reproduction, Thies et al.~\cite{Thies15,face2face} introduced a dense color term considering the entire face region that is represented by a morphable model using differentiable rendering.&#10;This method has been adapted for avatar digitization from a single image~\cite{liwen2017} including hair, is used to reconstruct high-fidelity facial reflectance and geometry from a single images~\cite{Yamaguchi2018}, for reconstruction and animation of entire upper bodies~\cite{HeadOn}, or avatars with dynamic textures~\cite{nagano2018}.&#10;Recently, these optimization-based methods are combined with learnable components such as surface offsets or view-dependent surface radiance fields~\cite{grassal2021neural}.&#10;In addition to a photometric reconstruction objective, additional terms based on dense correspondence~\cite{gueler2017densereg} or normal~\cite{Abrevaya_2020_CVPR,grassal2021neural} estimations of neural network can be employed.&#10;Optimization-based methods are also used as a building block for neural rendering methods such as deep video portraits~\cite{deepvideoportraits}, deferred neural rendering~\cite{deferredneuralrendering}, or neural voice puppetry~\cite{thies2020nvp}.&#10;Note that differentiable rendering is not only used in neural rendering frameworks but is also a key component for self-supervised learning of regression-based reconstruction methods covered in the following.&#10;Regression-based Reconstruction of Human Faces.&#10;Learning-based face reconstruction methods can be categorized into supervised and self-supervised approaches.&#10;A series of methods are based on synthetic renderings of human faces to perform a supervised training of a regressor that predicts the parameters of a 3D morphable model~\cite{InverseFaceNet,richardson20163d,dou2017endtoend,richardson2017learning}.&#10;Genova et al.~\cite{genova2018unsupervised} propose a 3DMM parameter regression technique that is based on synthetic renderings (where ground truth parameters are available) and real images (where multi-view identity losses are applied). It uses FaceNet~\cite{facenet} to extract features for the 3DMM regression task.&#10;Tran et al.~\cite{tran2016regressing} and Chang et al.~\cite{chang2018expnet} (ExpNet) directly regress 3DMM parameters using a CNN trained on fitted 3DMM data.&#10;Tu et al.~\cite{Tu2019} propose a dual training pass for images with and without 3DMM fittings.&#10;Jackson et al.~\cite{jackson2017large} propose a model-free approach that reconstructs a voxel-based representation of the human face and is trained on paired 2D image and 3D scan data.&#10;PRN~\cite{feng2018joint} is trained on 'in-the-wild' images with fitted 3DMM reconstructions~\cite{zhu20153ddfa}. It is not restricted to a 3DMM model space and predicts a position map in the UV-space of a template mesh. Instead of working in UV-space, Wei et al.~\cite{wei20193d} propose to use graph convolutions to regress the coordinates of the vertices.&#10;MoFA~\cite{tewari17MoFA} is a network trained to regress the 3DMM parameters in a self-supervised fashion. As a supervision signal, it uses the dense photometric losses of Face2Face~\cite{face2face}.&#10;Within this framework, Tewari et al. proposed to refine the identity shape and appearance~\cite{tewari2017self} as well as the expression basis~\cite{tewari2019fml} of a linear 3DMM.&#10;In a similar setup, one can also train a non-linear 3DMM~\cite{tran2019towards} or personalized models~\cite{chaudhuri2020personalized}.&#10;RingNet~\cite{RingNet:CVPR:2019} regresses 3DMM parameters and is trained on 2D images using losses on the reproduction of 2D landmarks and shape consistency (different images of the same subject) and shape inconsistency (images of different subjects) losses.&#10;DECA~\cite{deca} extends RingNet with expression dependent offset predictions in UV space. It uses dense photometric losses to train the 3DMM parameter regression and the offset prediction network.&#10;This separation of a coarse 3DMM model and a detailed bump map has been introduced by Tran et al.~\cite{tran2017extreme}.&#10;Chen et al.~\cite{chen2019photo} use a hybrid training composed of self-supervised and supervised training based on renderings to predict texture and displacement maps.&#10;Deng et al.~\cite{deng2019accurate} train a 3DMM parameter regressor based on multi-image consistency losses and 'hybrid-level' losses (photometric reconstruction loss with skin attention masks, and a perception-level loss based on FaceNet~\cite{facenet}).&#10;On the NoW challenge~\cite{RingNet:CVPR:2019}, DECA~\cite{deca} and the method of Deng et al.~\cite{deng2019accurate} show on-par state-of-the-art results.&#10;Similar to DECA's offset prediction, there are GAN-based methods that predict detailed color maps~\cite{Gecer_2019_CVPR,gecer2021fast} or skin properties~\cite{saito2016photorealistic,Yamaguchi2018,lattas2020avatarme,lattas2021avatarme++} (e.g., albedo, reflectance, normals) in UV-space of a 3DMM-based face reconstruction.&#10;In contrast to these methods, we are interested in reconstructing a metrical 3D representation of a human face and not fine-scale details.&#10;Self-supervised methods suffer from the depth-scale ambiguity (the face scale, translation away from the camera, and the perspective projection are ambiguous) and, thus, predict a wrongly scaled face, even though 3DMM models are by construction in a metrical space.&#10;We rely on a strong supervision signal to learn the metrical reconstruction of a face using high-quality 3D scan datasets which we unified.&#10;In combination with an identity encoder~\cite{arcface} trained on in-the-wild 2D data, including occlusions, different illumination, poses, and expressions, we achieve robust geometry estimations that significantly outperform state-of-the-art methods.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1902.10744" label="1902.10744">
-        <attvalues>
-          <attvalue for="0" value="Joint Face Detection and Facial Motion Retargeting for Multiple Faces" />
-          <attvalue for="1" value="  Facial motion retargeting is an important problem in both computer graphics&#10;and vision, which involves capturing the performance of a human face and&#10;transferring it to another 3D character. Learning 3D morphable model (3DMM)&#10;parameters from 2D face images using convolutional neural networks is common in&#10;2D face alignment, 3D face reconstruction etc. However, existing methods either&#10;require an additional face detection step before retargeting or use a cascade&#10;of separate networks to perform detection followed by retargeting in a&#10;sequence. In this paper, we present a single end-to-end network to jointly&#10;predict the bounding box locations and 3DMM parameters for multiple faces.&#10;First, we design a novel multitask learning framework that learns a&#10;disentangled representation of 3DMM parameters for a single face. Then, we&#10;leverage the trained single face model to generate ground truth 3DMM parameters&#10;for multiple faces to train another network that performs joint face detection&#10;and motion retargeting for images with multiple faces. Experimental results&#10;show that our joint detection and retargeting network has high face detection&#10;accuracy and is robust to extreme expressions and poses while being faster than&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.00524" label="2109.00524">
-        <attvalues>
-          <attvalue for="0" value="On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation" />
-          <attvalue for="1" value="  Benchmark datasets that measure camera pose accuracy have driven progress in&#10;visual re-localisation research. To obtain poses for thousands of images, it is&#10;common to use a reference algorithm to generate pseudo ground truth. Popular&#10;choices include Structure-from-Motion (SfM) and&#10;Simultaneous-Localisation-and-Mapping (SLAM) using additional sensors like&#10;depth cameras if available. Re-localisation benchmarks thus measure how well&#10;each method replicates the results of the reference algorithm. This begs the&#10;question whether the choice of the reference algorithm favours a certain family&#10;of re-localisation methods. This paper analyzes two widely used re-localisation&#10;datasets and shows that evaluation outcomes indeed vary with the choice of the&#10;reference algorithm. We thus question common beliefs in the re-localisation&#10;literature, namely that learning-based scene coordinate regression outperforms&#10;classical feature-based methods, and that RGB-D-based methods outperform&#10;RGB-based methods. We argue that any claims on ranking re-localisation methods&#10;should take the type of the reference algorithm, and the similarity of the&#10;methods to the reference algorithm, into account.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The difficulty of obtaining ground truth varies for different tasks in computer visions. &#10;For tasks with a low dimensional structure, \eg image classification and object detection, human annotations are effective &#10;\cite{pascal-voc-2012} &#10;and can be scaled via crowd sourcing \cite{Deng2009CVPR}.&#10;For tasks with a more complex output, \eg image segmentation or optical flow, annotation time quickly rises &#10;to a level which severely affects the scale and cost of associated datasets \cite{Donath2013ICVS, Cordts2016CVPR,geiger2013vision,Menze2015CVPR}.&#10;&#10;The 6DoF camera pose estimation task comes with the added difficulty that humans are not skilled at directly annotating the&#10;camera poses. &#10;Instead they annotate image correspondences as input to an optimization problem to recover the pose indirectly \cite{Sattler2018CVPR,Taira2018CVPR, Torii2019TPAMI}.&#10;Since many correspondences are required for a stable pose estimate, such an annotation approach does not scale beyond a few hundred images.&#10;Further, &#10;the annotations are usually only precise up to a few pixels, which, depending on the distance to the scene, can result in significant pose uncertainty \cite{Sattler2018CVPR,Zhang2020IJCV,Torii2019TPAMI}.&#10;&#10;As an alternative, &#10;the recording camera can be &#10;tracked by an external tracking system \cite{sturm12iros, Burri2016IJRR, Schoeps2019CVPR}.&#10;While providing high precision poses, capturing a diverse set of scenes is challenging due to the complicated setup, \ie, &#10;installation and calibration, &#10;ensuring good visibility of the sensors, \etc &#10;Similarly, industrial level LiDAR scanners have been used to produce high quality scans of landmarks, but the corresponding datasets provide only few scenes with limited spatial extent \cite{Strecha2008CVPR, Knapitsch2017TOG, Schoeps2017CVPR}.&#10;GPS-INS systems that combine GPS with inertial navigation systems (INS) have also been used to track camera poses on a large scale \cite{geiger2013vision, Maddern2017IJRR}. &#10;Yet, post-processing is still required to obtain a higher accuracy~\cite{Sattler2018CVPR}. &#10;&#10;Synthetic datasets come with true ground truth, but most current datasets, \eg, Habitat\cite{habitat19iccv}, are limited in diversity of low-level noise, illumination conditions or specular reflections.&#10;Therefore, data association, which is at the core of re-localisation, can become too easy.&#10;An example are the very low errors reported in~\cite{Sattler2019CVPR} for Active Search~\cite{Sattler2012ECCV, Sattler2017PAMI} on a synthetic version of the Cambridge Landmarks dataset. &#10;&#10;The vast majority of re-localisation benchmarks follow an automatic approach to ground truth recovery using a reference algorithm \cite{Li2010ECCV, Chen2011CVPR,Li2012ECCV, Shotton2013CVPR,Kendall2015ICCV,Valentin20163DV,Sattler2018CVPR, Taira2018CVPR, Torii2019TPAMI,Zhang2020IJCV, Wald2020ECCV, Jin2020}.&#10;Popular choices are &#10;SfM~\cite{Schoenberger2016CVPR,Wu133DV}, often for large-scale outdoor environments \cite{Li2010ECCV,Chen2011CVPR, Li2012ECCV, Torii2019TPAMI, Kendall2015ICCV, Sattler2018CVPR}, and depth-based &#10;SLAM~\cite{Newcombe2011ISMAR,Izadi2011UIST,Schoeps2019CVPR,Dai2017TOG}, often for small-scale indoor environments \cite{Shotton2013CVPR,Valentin20163DV, Wald2020ECCV}. &#10;Hybrid solutions also exist, such as ICP-based registration of LiDAR scans followed by an SfM-based registration of RGB images \cite{Sattler2018CVPR, Taira2018CVPR}.&#10;Some benchmarks use human visual inspection &#10;as a final quality control and verification stage \cite{Sattler2018CVPR, Taira2018CVPR, Jin2020, Zhang2020IJCV}, and state-of-the-art reference algorithms are found to provide high-quality reconstructions and pose tracks.&#10;However, as&#10;shown in Fig.~\ref{fig:teaser}, subtle differences in the output of reference algorithms, unlikely to be recognized by visual checks, can have significant influence on the evaluation outcome of a benchmark.&#10;&#10;Such evaluation artifacts have the potential to challenge some conclusions that have previously been drawn in the literature.&#10;Fig.~\ref{fig:relwork} shows published results in re-localisation research on the popular indoor datasets 7Scenes \cite{Shotton2013CVPR} and 12Scenes \cite{Valentin20163DV}, and the popular outdoor datasets Cambridge Landmarks \cite{Kendall2015ICCV} and Aachen Day \cite{Sattler2012BMVC, Sattler2018CVPR}.&#10;We compare the dominant families of re-localisation methods, scene coordinate regression and sparse feature-based matching. &#10;Scene coordinate regression methods use a learned model, a neural network or a random forest, to predict dense image-to-scene correspondences \cite{Shotton2013CVPR, Valentin2015CVPR, Brachmann2016CVPR, Brachmann2017CVPR, Brachmann2018CVPR, Brachmann2019ICCVa, Brachmann2019ICCVb, brachmann2020ARXIV, Cavallari2017CVPR, Cavallari2019TPAMI, Yang2019ICCV, li2020hierarchical}.&#10;RGB-D variants of scene coordinate regression methods dominate rankings for indoor re-localisation, which has been attributed to the inherent difficulty of the indoor scenario regarding texture-less surfaces and ambiguous structures that make it difficult to find and match sparse features \cite{Shotton2013CVPR, Kendall2015ICCV, Walch2017ICCV, Brachmann2017CVPR}. &#10;For outdoor re-localisation, classical approaches, which match hand-crafted \cite{Sattler2017PAMI, Shotton2013CVPR, Valentin20163DV} or learned descriptors \cite{Sarlin2019CVPR, DeTone2018CVPRWorkshops, Sarlin2020CVPR, HumenbergerX20Kapture} at sparse feature locations to a 3D SfM reconstruction, achieve vastly superior results compared to scene coordinate regression.&#10;This has been attributed to an inability of scene coordinate regression to scale to spatially large scenes \cite{Sattler2018CVPR,Taira2019TPAMI}. &#10;We offer a different explanation regarding the performance of re-localisers in different settings by taking the reference algorithms into account that were used to create the associated benchmarks.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Reference Algorithm Bias, Computer Vision Evaluation, Mathematics, Computer Science, Visual Re-localisation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2101.10801" label="2101.10801">
-        <attvalues>
-          <attvalue for="0" value="Global-Local Propagation Network for RGB-D Semantic Segmentation" />
-          <attvalue for="1" value="  Depth information matters in RGB-D semantic segmentation task for providing&#10;additional geometric information to color images. Most existing methods exploit&#10;a multi-stage fusion strategy to propagate depth feature to the RGB branch.&#10;However, at the very deep stage, the propagation in a simple element-wise&#10;addition manner can not fully utilize the depth information. We propose&#10;Global-Local propagation network (GLPNet) to solve this problem. Specifically,&#10;a local context fusion module(L-CFM) is introduced to dynamically align both&#10;modalities before element-wise fusion, and a global context fusion&#10;module(G-CFM) is introduced to propagate the depth information to the RGB&#10;branch by jointly modeling the multi-modal global context features. Extensive&#10;experiments demonstrate the effectiveness and complementarity of the proposed&#10;fusion modules. Embedding two fusion modules into a two-stream encoder-decoder&#10;structure, our GLPNet achieves new state-of-the-art performance on two&#10;challenging indoor scene segmentation datasets, i.e., NYU-Depth v2 and SUN-RGBD&#10;dataset.&#10;" />
-          <attvalue for="2" value="&#10;&#10;With the rapid development of RGB-D sensors like Microsoft Kinect, people get access to depth data more easily. Depth data can naturally describe 3D geometric information and reflect the structure of objects in the scene, thus can serve as a complementary modality for RGB data which captures rich color and texture information to improve semantic segmentation results. However, how to fully utilize depth information and effectively fuse the two complementary modalities still remains an open problem.&#10;&#10;Early approaches \cite{long2015fully} attempt to use a two-stream network to extract features from RGB and depth modality respectively and fuse them at the last layer to predict the final segmentation results. This kind of ``late fusion&quot; strategy fuses two modalities too late, resulting in that the RGB branch can not get needed geometric information guidance at the early stages. Later, researchers tend to propagate features of the depth branch to the RGB branch in a multi-stage manner, i.e., add depth feature to the RGB branch at the end of every stage of the encoder network \cite{hazirbas2016fusenet,jiang2018rednet}. This strategy leverages geometric cues more early and sufficiently, which is proved to be effective. &#10;&#10;However, we rethink that the feature propagation way in conventional methods can not fully utilize the depth information due to the following two reasons. Firstly, the consecutive use of convolution and down-sampling operations in the two-stream encoder makes the two modality features are not aligned each other as well as the early layers, so the geometric information provided by depth feature at the deep stage is not precise enough to assist RGB feature. Secondly, compared with the depth information, the RGB information reflects more semantic information, and the fact is more obvious at the higher level of the network. Thus, the element-wise addition is not an appropriate solution, and the RGB should receive more focus for the semantic prediction.&#10;&#10;To address the above problems, we propose Global-Local Propagation Network (GLPNet) to jointly utilize the complementary modalities of the depth and RGB features, in which a local context fusion module (L-CFM) and a global context fusion module (G-CFM) are designed to solve the problems of spatial disalignment and semantic propagation in the feature fusion, respectively. Instead of directly adding depth feature to the RGB branch, L-CFM dynamically aligns the features of both modalities first before modality fusion. Specifically, the alignment process is to simultaneously warp the feature maps of both modalities according to the offsets predicted through a convolution layer which is inspired by the optical flow in video processing field and Semantic Flow \cite{li2020semantic}. Besides, the G-CFM is proposed to propagate the depth feature to the RGB branch through the joint multi-modal context modeling. Concretely, we extract global context features from both modalities and aggregate them to every RGB pixel using attention mechanism. Compared to L-CFM which precisely aligns the local features of two modalities, G-CFM aims at utilizing depth information from the view of global context. Given that the proposed two fusion modules help depth feature propagation from orthogonal perspectives (i.e., globally and locally),&#10;combining them in a parallel way further improves the propagation effectiveness at deep stage. &#10;&#10;The proposed GLPNet achieves new state-of-the-art performance on two challenging RGB-D semantic segmentation datasets, i.e., NYU-Depth v2 and SUN-RGBD dataset. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Depth Information Fusion" />
-        </attvalues>
-      </node>
-      <node id="1806.01054" label="1806.01054">
-        <attvalues>
-          <attvalue for="0" value="RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Indoor semantic segmentation has always been a difficult task in computer&#10;vision. In this paper, we propose an RGB-D residual encoder-decoder&#10;architecture, named RedNet, for indoor RGB-D semantic segmentation. In RedNet,&#10;the residual module is applied to both the encoder and decoder as the basic&#10;building block, and the skip-connection is used to bypass the spatial feature&#10;between the encoder and decoder. In order to incorporate the depth information&#10;of the scene, a fusion structure is constructed, which makes inference on RGB&#10;image and depth image separately, and fuses their features over several layers.&#10;In order to efficiently optimize the network's parameters, we propose a&#10;`pyramid supervision' training scheme, which applies supervised learning over&#10;different layers in the decoder, to cope with the problem of gradients&#10;vanishing. Experiment results show that the proposed RedNet(ResNet-50) achieves&#10;a state-of-the-art mIoU accuracy of 47.8% on the SUN RGB-D benchmark dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.10120" label="2002.10120">
-        <attvalues>
-          <attvalue for="0" value="Semantic Flow for Fast and Accurate Scene Parsing" />
-          <attvalue for="1" value="  In this paper, we focus on designing effective method for fast and accurate&#10;scene parsing. A common practice to improve the performance is to attain high&#10;resolution feature maps with strong semantic representation. Two strategies are&#10;widely used -- atrous convolutions and feature pyramid fusion, are either&#10;computation intensive or ineffective. Inspired by the Optical Flow for motion&#10;alignment between adjacent video frames, we propose a Flow Alignment Module&#10;(FAM) to learn Semantic Flow between feature maps of adjacent levels, and&#10;broadcast high-level features to high resolution features effectively and&#10;efficiently. Furthermore, integrating our module to a common feature pyramid&#10;structure exhibits superior performance over other real-time methods even on&#10;light-weight backbone networks, such as ResNet-18. Extensive experiments are&#10;conducted on several challenging datasets, including Cityscapes, PASCAL&#10;Context, ADE20K and CamVid. Especially, our network is the first to achieve&#10;80.4\% mIoU on Cityscapes with a frame rate of 26 FPS. The code is available at&#10;\url{https://github.com/lxtGH/SFSegNets}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;For scene parsing, there are mainly two paradigms for high-resolution semantic map prediction. One paradigm tries to keep both spatial and semantic information along the main network pathway, while the other paradigm distributes spatial and semantic information to different parts in a network, then merges them back via different strategies.&#10;&#10;The first paradigm mostly relies on some network operations to retain high-resolution feature maps in the latter network stages. Many state-of-the-art accurate methods~\cite{pspnet,DAnet,nvidia_seg_video} follow this paradigm to design sophisticated head networks to capture contextual information. PSPNet~\cite{pspnet} proposes to leverage pyramid pooling module (PPM) to model multi-scale contexts, whilst DeepLab series~\cite{deeplabv2,deeplabv3,deeplabv3p,denseaspp} uses astrous spatial pyramid pooling (ASPP). In~\cite{DAnet,AdaptivePyramid_seg,ocnet,emanet,annet,ccnet,DMF_seg}, non-local operator~\cite{non_local} and self-attention mechanism~\cite{transformer} are adopted to harvest pixel-wise context from the whole image. Meanwhile, several works~\cite{graph_cnn,beyond_grids,spg_net,DGM_net,zhangli_dgcn} use graph convolutional neural networks to propagate information over the image by projecting features into an interaction space. &#10;&#10;The second paradigm contains several state-of-the-art fast methods, where high-level semantics are represented by low-resolution feature maps. A common strategy is to fuse multi-level feature maps for high-resolution spatiality and strong semantics~\cite{fcn,segnet,upernet,unet,xiangtl_gff}. ICNet~\cite{ICnet} uses multi-scale images as input and a cascade network to be more efficient. DFANet~\cite{dfanet} utilizes a light-weight backbone to speed up its network and proposes a cross-level feature aggregation to boost accuracy, while SwiftNet~\cite{swiftnet} uses lateral connections as the cost-effective solution to restore the prediction resolution while maintaining the speed. To further speed up, low-resolution images are used as input for high-level semantics~\cite{ICnet,gum_bmvc} which reduce features into low resolution and then upsample them back by a large factor. The direct consequence of using a large upsample factor is performance degradation, especially for small objects and object boundaries. Guided upsampling~\cite{gum_bmvc} is related to our method, where the semantic map is upsampled back to the input image size guided by the feature map from an early layer. However, this guidance is still insufficient for some cases due to the information gap between the semantics and resolution. In contrast, our method aligns feature maps from adjacent levels and further enhances the feature maps using a feature pyramid framework towards both high resolution and strong semantics, consequently resulting in the state-of-the-art performance considering the trade-off between high accuracy and fast speed. &#10;&#10;There is another set of works focusing on designing light-weight backbone networks to achieve real-time performances. ESPNets~\cite{ESPNet,ESPNetv2} save computation by decomposing standard convolution into point-wise convolution and spatial pyramid of dilated convolutions. BiSeNet~\cite{bisenet} introduces spatial path and semantic path to reduce computation. Recently, several methods~\cite{fast_cell_search_seg,custom_search_seg,DF-seg-net} use AutoML techniques to search efficient architectures for scene parsing. Our method is complementary to some of these works, which further boosts their accuracy. &#10;Since our proposed semantic flow is inspired by optical flow~\cite{FlowNet}, which is used in video semantic segmentation, we also discuss several works in video semantic segmentation. &#10;For accurate results, temporal information is exceedingly exploited by using optical flow. &#10;Gadde~et. al.~\cite{Netwarp} warps internal feature maps and Nilsson~et. al.~\cite{GRFP_video} warps final semantic maps from nearby frame predictions to the current map. To pursue faster speed, optical flow is used to bypass the low-level feature computation of some frames by warping features from their preceding frames~\cite{DFF,Lowlatency_net}. Our work is different from theirs by propagating information hierarchically in another dimension, which is orthogonal to the temporal propagation for videos. &#10;" />
-        </attvalues>
-      </node>
-      <node id="2004.04400" label="2004.04400">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image&#10;  Synthesis" />
-          <attvalue for="1" value="  Camera captured human pose is an outcome of several sources of variation.&#10;Performance of supervised 3D pose estimation approaches comes at the cost of&#10;dispensing with variations, such as shape and appearance, that may be useful&#10;for solving other related tasks. As a result, the learned model not only&#10;inculcates task-bias but also dataset-bias because of its strong reliance on&#10;the annotated samples, which also holds true for weakly-supervised models.&#10;Acknowledging this, we propose a self-supervised learning framework to&#10;disentangle such variations from unlabeled video frames. We leverage the prior&#10;knowledge on human skeleton and poses in the form of a single part-based 2D&#10;puppet model, human pose articulation constraints, and a set of unpaired 3D&#10;poses. Our differentiable formalization, bridging the representation gap&#10;between the 3D pose and spatial part maps, not only facilitates discovery of&#10;interpretable pose disentanglement but also allows us to operate on videos with&#10;diverse camera movements. Qualitative results on unseen in-the-wild datasets&#10;establish our superior generalization across multiple tasks beyond the primary&#10;tasks of 3D pose estimation and part segmentation. Furthermore, we demonstrate&#10;state-of-the-art weakly-supervised 3D pose estimation performance on both&#10;Human3.6M and MPI-INF-3DHP datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Human 3D pose estimation is a well established problem in computer vision, specifically in fully supervised paradigm. Earlier approaches~\cite{ramakrishna2014pose,yang2011articulated,tompson2014joint,chen2014articulated} proposed to infer the underlying graphical model for articulated pose estimation. However, the recent CNN based approaches~\cite{bulat2016human,newell2016stacked,VNect_SIGGRAPH2017} focus on regressing spatial keypoint heat-maps, without explicitly accounting for &#10;the underlying limb connectivity information. However, the performance of such models heavily relies on a large set of paired 2D or 3D pose annotations. As a different approach, \cite{katircioglu2018learning} proposed to regress latent representation of a trained 3D pose autoencoder to indirectly endorse a plausibility bound on the output predictions. Recently, several weakly supervised approaches utilize varied set of auxiliary supervision other than the direct 3D pose supervision (see Table~\ref{tab:char}).&#10;In this paper, we address a more challenging scenario where we consider access to only a set of unaligned 2D pose data to facilitate the learning of a plausible 2D pose prior (see Table~\ref{tab:char}).&#10;&#10;In literature, while &#10;several supervised shape and appearance disentangling techniques~\cite{balakrishnan2018synthesizing,ma2018disentangled,ma2017pose,esser2018variational,siarohin2018deformable} exist, the available unsupervised pose estimation works (\ie in the absence of multi-view or camera extrinsic supervision), are mostly limited to 2D landmark estimation~\cite{denton2017unsupervised,jakab2018unsupervised} for rigid or mildly deformable structures, such as facial landmark detection, constrained torso pose recovery etc. The general idea~\cite{kanazawa2016warpnet,rocco2017convolutional,thewlis2017unsupervised,thewlis2017nips} is to utilize the relative transformation between a pair of images depicting a consistent appearance with varied pose. Such image pairs are usually sampled from a video satisfying appearance invariance~\cite{jakab2018unsupervised} or synthetically generated deformations~\cite{rocco2017convolutional}. &#10;&#10;Beyond landmarks, object parts~\cite{lorenz2019unsupervised} can infer shape alongside the pose. Part representations are best suited for 3D articulated objects as a result of its occlusion-aware property as opposed to simple landmarks. In general, the available unsupervised part learning techniques~\cite{singh2012unsupervised,hung2019scops} are mostly limited to segmentation based discriminative tasks. On the other hand, \cite{Yang2016end,novotny2017anchornet} explicitly leverage the consistency between geometry and the semantic part segments. However, the kinematic articulation constraints are well defined in 3D rather than in 2D~\cite{akhter2015pose}. Motivated by this, we aim to leverage the advantages of both non-spatial 3D pose~\cite{katircioglu2018learning} and spatial part-based representation~\cite{lorenz2019unsupervised} by proposing a novel 2D pose-anchored part deformation model. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Computer Science, Human Pose Estimation, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2105.01859" label="2105.01859">
-        <attvalues>
-          <attvalue for="0" value="Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer&#10;  RGBD Sensors" />
-          <attvalue for="1" value="  Human volumetric capture is a long-standing topic in computer vision and&#10;computer graphics. Although high-quality results can be achieved using&#10;sophisticated off-line systems, real-time human volumetric capture of complex&#10;scenarios, especially using light-weight setups, remains challenging. In this&#10;paper, we propose a human volumetric capture method that combines temporal&#10;volumetric fusion and deep implicit functions. To achieve high-quality and&#10;temporal-continuous reconstruction, we propose dynamic sliding fusion to fuse&#10;neighboring depth observations together with topology consistency. Moreover,&#10;for detailed and complete surface generation, we propose detail-preserving deep&#10;implicit functions for RGBD input which can not only preserve the geometric&#10;details on the depth inputs but also generate more plausible texturing results.&#10;Results and experiments show that our method outperforms existing methods in&#10;terms of view sparsity, generalization capacity, reconstruction quality, and&#10;run-time efficiency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In the following, we focus on 3D human volumetric/performance capture and classify existing methods into $4$ categories according to their underlying techniques. &#10;&#10;Volumetric capture from multi-view stereo. &#10;Multi-view volumetric capture is an active research area in the computer vision and graphics community. Previous works use multi-view images for human model reconstruction~\cite{Kanade97,StarckCGA07,liu2009point}. Shape cues like silhouette, stereo, shading, and cloth priors have been integrated to improve reconstruction/rendering performance ~\cite{StarckCGA07,liu2009point,WuShadingHuman,WaschbuschWCSG05,VlasicPBDPRM09,bradley2008markerless,Mustafa16,pons2017clothcap,Wu_2020_CVPR}. &#10;State-of-the-art methods build extremely sophisticated systems where up to 100 cameras~\cite{collet2015high} and even custom-designed gradient lighting~\cite{guo2019relightables} for high-quality volumetric capture. In particular, the methods by~\cite{collet2015high} and~\cite{guo2019relightables} first perform multi-view stereo for the point cloud generation, followed by mesh construction, simplification, tracking, and post-processing steps such as UV mapping. Although the results are compelling, the reliance on well-controlled multi-camera studios and a huge amount of computational resources prohibit them from being used in living spaces. &#10;&#10;Template-based Performance Capture. &#10;For performance capture, some of the previous works leverage pre-scanned templates and exploit multi-view geometry to track the motion of the templates. For instance, the methods in~\cite{Vlasic08,gall2009motion,brox2009combined} adopted a template with an embedded skeleton driven by multi-view silhouettes and temporal feature constraints. Their methods are then extended to handle multiple interacting characters in~\cite{liu2011markerless,LiuPAMI13}. Besides templates with an embedded skeleton, some works adopted non-rigid deformation for template motion tracking. Li et,al~\cite{li2009robust} utilized embedded deformation graph in~\cite{Sumner2007embedded} to parameterize the non-rigid deformations of the pre-scanned template. Guo et,al.~\cite{guo2015robust} adopted an $\ell_0$ norm constraint to generate articulated motions for bodies and faces without explicitly constructing a skeleton. Zollh{\&quot;o}fer et, al.~\cite{zollhofer2014real} took advantage of GPU to parallelize the non-rigid registration algorithm and achieved real-time performance of general non-rigid tracking. Recently, capturing 3D dense human body deformation with coarse-to-fine registration from a single RGB camera has been enabled~\cite{xu2018monoperfcap} and improved for real-time performance~\cite{habermann2019livecap}. DeepCap~\cite{habermann2020deepcap} introduced a deep learning method that jointly infers the articulated and non-rigid 3D deformation parameters in a single feed-forward pass. Although template-based approaches require less input than multi-view stereo methods, they are incapable of handling topological changes due to the prerequisite of a template with fixed topology. &#10;&#10;Volumetric Fusion for Dynamic 3D Reconstruction. &#10;To get rid of template priors and realize convenient deployment, researchers turn to use one or sparse depth sensors for 3D reconstruction. In 2015, DynamicFusion~\cite{newcombe2015dynamicfusion} proposed the first template-free, single-view, real-time dynamic 3D reconstruction system, which integrates multiple frames into a canonical model to reconstruct a complete surface model. However, it only handles controlled and relatively slow motions due to the challenges of real-time non-rigid tracking. In order to improve the robustness of DynamicFusion, the following works incorporated killing/sobolev constraints~\cite{slavcheva2017killingfusion, Slavcheva_2018_CVPR}, articulated priors~\cite{chao2018ArticulatedFusion}, skeleton tracking~\cite{yu2017BodyFusion}, parametric body models~\cite{yu2018doublefusion}, sparse inertial measurements~\cite{Zheng2018HybridFusion}, data-driven prior~\cite{su2020robustfusion} or learned correspondences~\cite{bozic2020deepdeform} into the non-rigid fusion pipeline. However, all of these methods are prone to tracking failure in invisible areas, which is an inherent drawback of single-view systems. To overcome this limitation, Fusion4D~\cite{dou2016fusion4d} and Motion2fusion~\cite{Motion2Fusion} focused on real-time multi-view setups using high-end custom-design sensors, with the notion of key volume updating and learning-based surface matching. Even though the pipelines were carefully designed in~\cite{dou2016fusion4d} and~\cite{Motion2Fusion}, they still suffer from incomplete and noisy reconstructions when severe topological changes occur especially under very sparse system setups. &#10;&#10;Learning-based 3D Human Reconstruction. &#10;Fueled by the rapid developments in neural 3D representations(e.g., ~\cite{park2019deepsdf,Peng2020ECCV,DeepLocalShapes,Occupancy_Networks} etc.), a lot of data-driven methods for 3D human reconstruction have been proposed in recent years. &#10;Methods in~\cite{zhu2019hmd,alldieck2019tex2shape} proposed to deform a parametric body model to fit the image observations including keypoints, silhouettes, and shading. DeepHuman~\cite{zheng2019deephuman} combined the parametric body model with a coarse-scale volumetric reconstruction network to reconstruct 3D human models from a single RGB image. Some methods infer human shapes on 2D image domains using multi-view silhouettes~\cite{natsume2019siclope} or front-back depth pairs~\cite{gabeur2019moulding,wang2020normalgan}. PIFu~\cite{saito2019pifu} proposed to regress an implicit function using pixel-aligned image features. Unlike voxel-based methods, PIFu is able to reconstruct high-resolution results thanks to the compactness of the implicit representations. PIFuHD~\cite{saito2020pifuhd} extended PIFu to capture more local details. However, both PIFu~\cite{saito2019pifu} and PIFuHD~\cite{saito2020pifuhd} fail to reconstruct plausible models in cases of challenging poses and self-occlusions. PaMIR~\cite{zheng2020pamir} resolved this challenge by using the SMPL model as a prior but suffers from run-time inefficiency since it requires a post-processing optimization step. &#10;IFNet~\cite{chibane20ifnet} and IPNet~\cite{bhatnagar2020ipnet} can recover impressive 3D humans from partial point clouds, but the dependency on multi-scale 3D convolutions and parametric body models block the realization of real-time reconstruction performance. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Engineering, Machine Learning, Mathematics, Computer Science, Human Volumetric Capture, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2402.15321" label="2402.15321">
-        <attvalues>
-          <attvalue for="0" value="OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene&#10;  Understanding" />
-          <attvalue for="1" value="  This report provides an overview of the challenge hosted at the OpenSUN3D&#10;Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with&#10;ICCV 2023. The goal of this workshop series is to provide a platform for&#10;exploration and discussion of open-vocabulary 3D scene understanding tasks,&#10;including but not limited to segmentation, detection and mapping. We provide an&#10;overview of the challenge hosted at the workshop, present the challenge&#10;dataset, the evaluation methodology, and brief descriptions of the winning&#10;methods. For additional details, please see&#10;https://opensun3d.github.io/index_iccv23.html.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The ability to perceive, understand and interact with arbitrary 3D environments is a long-standing research goal with applications in AR/VR, robotics, health and industry.&#10;Current 3D scene understanding methods are largely limited to recognizing a closed-set of pre-defined object classes.&#10;Recently, large visual-language models (VLMs), such as CLIP~\cite{clip}, have demonstrated impressive generalization capabilities trained on internet-scale image-language pairs.&#10;Initial works have shown that VLMs can enable 3D scene understanding towards open-vocabulary recognition,&#10;and offer capabilities to understand affordances, materials, activities, and properties of unseen environments without the need to train on costly 3D labeled data.&#10;The development of such frameworks enables intelligent agents to perform arbitrary and complex tasks in novel environments.&#10;&#10;Driven by these motivations, the number of open-vocabulary 3D scene understanding approaches has been growing rapidly.&#10;On one hand, there are explicit point-cloud based approaches such as OpenScene~\cite{Peng2022OpenScene}, PLA~\cite{Ding2022PLALO}, CLIP-FO3D~\cite{Zhang2023CLIPFO3DLF} and ConceptFusion~\cite{conceptfusion} which aim to create scene representations that allow querying the scene based on arbitrary open-vocabulary descriptions.&#10;On the other hand, there are works such as LERF~\cite{Kerr2023LERFLE}, OpenNeRF~\cite{opennerf} and DFF~\cite{kobayashi2022distilledfeaturefields} that embed open-vocabulary features on an implicit scene representation.&#10;Methods focusing on 3D instance segmentation include OpenMask3D~\cite{takmaz2023openmask3d},&#10;OpenIns3D~\cite{Huang2023OpenIns3DSA} and Open3DIS~\cite{Nguyen2023Open3DISO3}.&#10;&#10;There has also been a growing interest in creating a 3D scene graph representation \cite{sarkar2023sgaligner,Armeni_2019_ICCV} that allows for open-vocabulary querying such as ConceptGraphs~\cite{Gu2023ConceptGraphsO3}.&#10;These open-vocabulary approaches go beyond the pre-defined capabilities of traditional 3D scene understanding methods operating in a closed-vocabulary setting. &#10;&#10;The goal of the workshop is to bundle these efforts towards 3D open-vocabulary scene understanding, and to discuss and establish clear task definitions, evaluation metrics, and benchmark datasets.&#10;&#10;One of the important goals of this challenge is to help enable the quantitative comparison of open-vocabulary 3D scene segmentation methods.&#10;In fact, due to the lack of benchmarks addressing various open-vocabulary concepts,&#10;existing 3D open-vocabulary approaches mainly evaluate their capabilities qualitatively on individually selected queries and scenarios,&#10;which differ from work to work.&#10;The key goal of the workshop is to drive the research community inspire the research community to reason about the usability and generalization of their approaches in real-world scenarios.&#10;&#10;In this iteration of the workshop, the challenge focuses on the open-vocabulary setting with arbitrary object descriptions,&#10;which describe long-tail objects as well as object properties such as semantics, materials, affordances, and situational context.&#10;In summary, the challenge is:&#10;&#10;\begin{enumerate}&#10; \item Task: Given an open-vocabulary, text-based query, the aim is to localize and segment the object instances that fit best with the given prompt, which can describe object properties such as semantics, material type, affordances and situational context. &#10; \item Input: An RGB-D image sequence and the corresponding 3D reconstructed geometry of a given scene, camera parameters (intrinsics, extrinsics), and an input query.&#10; \item Output: Instance segmentation masks in the point cloud that corresponds to the vertices of the provided 3D mesh reconstruction, segmenting the objects that fit best with the provided text prompt.&#10;\end{enumerate}&#10;&#10;In the following sections, we first introduce the challenge task, dataset and evaluation methodology (Sec.~\ref{sec:challenge}). Then, we share results from the top three winning teams, as well as the respective methods proposed by each team (Sec.~\ref{sec:methods}). &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Open-Vocabulary Tasks, Mathematics, Artificial Intelligence, Computer Science, Computer Vision Challenges, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2211.15654" label="2211.15654">
-        <attvalues>
-          <attvalue for="0" value="OpenScene: 3D Scene Understanding with Open Vocabularies" />
-          <attvalue for="1" value="  Traditional 3D scene understanding approaches rely on labeled 3D datasets to&#10;train a model for a single task with supervision. We propose OpenScene, an&#10;alternative approach where a model predicts dense features for 3D scene points&#10;that are co-embedded with text and image pixels in CLIP feature space. This&#10;zero-shot approach enables task-agnostic training and open-vocabulary queries.&#10;For example, to perform SOTA zero-shot 3D semantic segmentation it first infers&#10;CLIP features for every 3D point and later classifies them based on&#10;similarities to embeddings of arbitrary class labels. More interestingly, it&#10;enables a suite of open-vocabulary scene understanding applications that have&#10;never been done before. For example, it allows a user to enter an arbitrary&#10;text query and then see a heat map indicating which parts of a scene match. Our&#10;approach is effective at identifying objects, materials, affordances,&#10;activities, and room types in complex 3D scenes, all using a single model&#10;trained without any labeled 3D data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.16312" label="2211.16312">
-        <attvalues>
-          <attvalue for="0" value="PLA: Language-Driven Open-Vocabulary 3D Scene Understanding" />
-          <attvalue for="1" value="  Open-vocabulary scene understanding aims to localize and recognize unseen&#10;categories beyond the annotated label space. The recent breakthrough of 2D&#10;open-vocabulary perception is largely driven by Internet-scale paired&#10;image-text data with rich vocabulary concepts. However, this success cannot be&#10;directly transferred to 3D scenarios due to the inaccessibility of large-scale&#10;3D-text pairs. To this end, we propose to distill knowledge encoded in&#10;pre-trained vision-language (VL) foundation models through captioning&#10;multi-view images from 3D, which allows explicitly associating 3D and&#10;semantic-rich captions. Further, to foster coarse-to-fine visual-semantic&#10;representation learning from captions, we design hierarchical 3D-caption pairs,&#10;leveraging geometric constraints between 3D scenes and multi-view images.&#10;Finally, by employing contrastive learning, the model learns language-aware&#10;embeddings that connect 3D and text for open-vocabulary tasks. Our method not&#10;only remarkably outperforms baseline methods by 25.8% $\sim$ 44.7% hIoU and&#10;14.5% $\sim$ 50.4% hAP$_{50}$ in open-vocabulary semantic and instance&#10;segmentation, but also shows robust transferability on challenging zero-shot&#10;domain transfer tasks. See the project website at&#10;https://dingry.github.io/projects/PLA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.04748" label="2303.04748">
-        <attvalues>
-          <attvalue for="0" value="CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D&#10;  Dense CLIP" />
-          <attvalue for="1" value="  Training a 3D scene understanding model requires complicated human&#10;annotations, which are laborious to collect and result in a model only encoding&#10;close-set object semantics. In contrast, vision-language pre-training models&#10;(e.g., CLIP) have shown remarkable open-world reasoning properties. To this&#10;end, we propose directly transferring CLIP's feature space to 3D scene&#10;understanding model without any form of supervision. We first modify CLIP's&#10;input and forwarding process so that it can be adapted to extract dense pixel&#10;features for 3D scene contents. We then project multi-view image features to&#10;the point cloud and train a 3D scene understanding model with feature&#10;distillation. Without any annotations or additional training, our model&#10;achieves promising annotation-free semantic segmentation results on&#10;open-vocabulary semantics and long-tailed concepts. Besides, serving as a&#10;cross-modal pre-training framework, our method can be used to improve data&#10;efficiency during fine-tuning. Our model outperforms previous SOTA methods in&#10;various zero-shot and data-efficient learning benchmarks. Most importantly, our&#10;model successfully inherits CLIP's rich-structured knowledge, allowing 3D scene&#10;understanding models to recognize not only object concepts but also open-world&#10;semantics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.07241" label="2302.07241">
-        <attvalues>
-          <attvalue for="0" value="ConceptFusion: Open-set Multimodal 3D Mapping" />
-          <attvalue for="1" value="  Building 3D maps of the environment is central to robot navigation, planning,&#10;and interaction with objects in a scene. Most existing approaches that&#10;integrate semantic concepts with 3D maps largely remain confined to the&#10;closed-set setting: they can only reason about a finite set of concepts,&#10;pre-defined at training time. Further, these maps can only be queried using&#10;class labels, or in recent work, using text prompts.&#10;  We address both these issues with ConceptFusion, a scene representation that&#10;is (1) fundamentally open-set, enabling reasoning beyond a closed set of&#10;concepts and (ii) inherently multimodal, enabling a diverse range of possible&#10;queries to the 3D map, from language, to images, to audio, to 3D geometry, all&#10;working in concert. ConceptFusion leverages the open-set capabilities of&#10;today's foundation models pre-trained on internet-scale data to reason about&#10;concepts across modalities such as natural language, images, and audio. We&#10;demonstrate that pixel-aligned open-set features can be fused into 3D maps via&#10;traditional SLAM and multi-view fusion approaches. This enables effective&#10;zero-shot spatial reasoning, not needing any additional training or finetuning,&#10;and retains long-tailed concepts better than supervised approaches,&#10;outperforming them by more than 40% margin on 3D IoU. We extensively evaluate&#10;ConceptFusion on a number of real-world datasets, simulated home environments,&#10;a real-world tabletop manipulation task, and an autonomous driving platform. We&#10;showcase new avenues for blending foundation models with 3D open-set multimodal&#10;mapping.&#10;  For more information, visit our project page https://concept-fusion.github.io&#10;or watch our 5-minute explainer video&#10;https://www.youtube.com/watch?v=rkXgws8fiDs&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.09553" label="2303.09553">
-        <attvalues>
-          <attvalue for="0" value="LERF: Language Embedded Radiance Fields" />
-          <attvalue for="1" value="  Humans describe the physical world using natural language to refer to&#10;specific 3D locations based on a vast range of properties: visual appearance,&#10;semantics, abstract associations, or actionable affordances. In this work we&#10;propose Language Embedded Radiance Fields (LERFs), a method for grounding&#10;language embeddings from off-the-shelf models like CLIP into NeRF, which enable&#10;these types of open-ended language queries in 3D. LERF learns a dense,&#10;multi-scale language field inside NeRF by volume rendering CLIP embeddings&#10;along training rays, supervising these embeddings across training views to&#10;provide multi-view consistency and smooth the underlying language field. After&#10;optimization, LERF can extract 3D relevancy maps for a broad range of language&#10;prompts interactively in real-time, which has potential use cases in robotics,&#10;understanding vision-language models, and interacting with 3D scenes. LERF&#10;enables pixel-aligned, zero-shot queries on the distilled 3D CLIP embeddings&#10;without relying on region proposals or masks, supporting long-tail&#10;open-vocabulary queries hierarchically across the volume. The project website&#10;can be found at https://lerf.io .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.15585" label="2205.15585">
-        <attvalues>
-          <attvalue for="0" value="Decomposing NeRF for Editing via Feature Field Distillation" />
-          <attvalue for="1" value="  Emerging neural radiance fields (NeRF) are a promising scene representation&#10;for computer graphics, enabling high-quality 3D reconstruction and novel view&#10;synthesis from image observations. However, editing a scene represented by a&#10;NeRF is challenging, as the underlying connectionist representations such as&#10;MLPs or voxel grids are not object-centric or compositional. In particular, it&#10;has been difficult to selectively edit specific regions or objects. In this&#10;work, we tackle the problem of semantic scene decomposition of NeRFs to enable&#10;query-based local editing of the represented 3D scenes. We propose to distill&#10;the knowledge of off-the-shelf, self-supervised 2D image feature extractors&#10;such as CLIP-LSeg or DINO into a 3D feature field optimized in parallel to the&#10;radiance field. Given a user-specified query of various modalities such as&#10;text, an image patch, or a point-and-click selection, 3D feature fields&#10;semantically decompose 3D space without the need for re-training and enable us&#10;to semantically select and edit regions in the radiance field. Our experiments&#10;validate that the distilled feature fields (DFFs) can transfer recent progress&#10;in 2D vision and language foundation models to 3D scene representations,&#10;enabling convincing 3D segmentation and selective editing of emerging neural&#10;graphics representations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.13631" label="2306.13631">
-        <attvalues>
-          <attvalue for="0" value="OpenMask3D: Open-Vocabulary 3D Instance Segmentation" />
-          <attvalue for="1" value="  We introduce the task of open-vocabulary 3D instance segmentation. Current&#10;approaches for 3D instance segmentation can typically only recognize object&#10;categories from a pre-defined closed set of classes that are annotated in the&#10;training datasets. This results in important limitations for real-world&#10;applications where one might need to perform tasks guided by novel,&#10;open-vocabulary queries related to a wide variety of objects. Recently,&#10;open-vocabulary 3D scene understanding methods have emerged to address this&#10;problem by learning queryable features for each point in the scene. While such&#10;a representation can be directly employed to perform semantic segmentation,&#10;existing methods cannot separate multiple object instances. In this work, we&#10;address this limitation, and propose OpenMask3D, which is a zero-shot approach&#10;for open-vocabulary 3D instance segmentation. Guided by predicted&#10;class-agnostic 3D instance masks, our model aggregates per-mask features via&#10;multi-view fusion of CLIP-based image embeddings. Experiments and ablation&#10;studies on ScanNet200 and Replica show that OpenMask3D outperforms other&#10;open-vocabulary methods, especially on the long-tail distribution. Qualitative&#10;experiments further showcase OpenMask3D's ability to segment object properties&#10;based on free-form queries describing geometry, affordances, and materials.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.00616" label="2309.00616">
-        <attvalues>
-          <attvalue for="0" value="OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation" />
-          <attvalue for="1" value="  In this work, we introduce OpenIns3D, a new 3D-input-only framework for 3D&#10;open-vocabulary scene understanding. The OpenIns3D framework employs a&#10;&quot;Mask-Snap-Lookup&quot; scheme. The &quot;Mask&quot; module learns class-agnostic mask&#10;proposals in 3D point clouds, the &quot;Snap&quot; module generates synthetic scene-level&#10;images at multiple scales and leverages 2D vision-language models to extract&#10;interesting objects, and the &quot;Lookup&quot; module searches through the outcomes of&#10;&quot;Snap&quot; to assign category names to the proposed masks. This approach, yet&#10;simple, achieves state-of-the-art performance across a wide range of 3D&#10;open-vocabulary tasks, including recognition, object detection, and instance&#10;segmentation, on both indoor and outdoor datasets. Moreover, OpenIns3D&#10;facilitates effortless switching between different 2D detectors without&#10;requiring retraining. When integrated with powerful 2D open-world models, it&#10;achieves excellent results in scene understanding tasks. Furthermore, when&#10;combined with LLM-powered 2D models, OpenIns3D exhibits an impressive&#10;capability to comprehend and process highly complex text queries that demand&#10;intricate reasoning and real-world knowledge. Project page:&#10;https://zheninghuang.github.io/OpenIns3D/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.10671" label="2312.10671">
-        <attvalues>
-          <attvalue for="0" value="Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance" />
-          <attvalue for="1" value="  We introduce Open3DIS, a novel solution designed to tackle the problem of&#10;Open-Vocabulary Instance Segmentation within 3D scenes. Objects within 3D&#10;environments exhibit diverse shapes, scales, and colors, making precise&#10;instance-level identification a challenging task. Recent advancements in&#10;Open-Vocabulary scene understanding have made significant strides in this area&#10;by employing class-agnostic 3D instance proposal networks for object&#10;localization and learning queryable features for each 3D mask. While these&#10;methods produce high-quality instance proposals, they struggle with identifying&#10;small-scale and geometrically ambiguous objects. The key idea of our method is&#10;a new module that aggregates 2D instance masks across frames and maps them to&#10;geometrically coherent point cloud regions as high-quality object proposals&#10;addressing the above limitations. These are then combined with 3D&#10;class-agnostic instance proposals to include a wide range of objects in the&#10;real world. To validate our approach, we conducted experiments on three&#10;prominent datasets, including ScanNet200, S3DIS, and Replica, demonstrating&#10;significant performance gains in segmenting objects with diverse categories&#10;over the state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.14880" label="2304.14880">
-        <attvalues>
-          <attvalue for="0" value="SGAligner : 3D Scene Alignment with Scene Graphs" />
-          <attvalue for="1" value="  Building 3D scene graphs has recently emerged as a topic in scene&#10;representation for several embodied AI applications to represent the world in a&#10;structured and rich manner. With their increased use in solving downstream&#10;tasks (eg, navigation and room rearrangement), can we leverage and recycle them&#10;for creating 3D maps of environments, a pivotal step in agent operation? We&#10;focus on the fundamental problem of aligning pairs of 3D scene graphs whose&#10;overlap can range from zero to partial and can contain arbitrary changes. We&#10;propose SGAligner, the first method for aligning pairs of 3D scene graphs that&#10;is robust to in-the-wild scenarios (ie, unknown overlap -- if any -- and&#10;changes in the environment). We get inspired by multi-modality knowledge graphs&#10;and use contrastive learning to learn a joint, multi-modal embedding space. We&#10;evaluate on the 3RScan dataset and further showcase that our method can be used&#10;for estimating the transformation between pairs of 3D scenes. Since benchmarks&#10;for these tasks are missing, we create them on this dataset. The code,&#10;benchmark, and trained models are available on the project website.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.02527" label="1910.02527">
-        <attvalues>
-          <attvalue for="0" value="3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera" />
-          <attvalue for="1" value="  A comprehensive semantic understanding of a scene is important for many&#10;applications - but in what space should diverse semantic information (e.g.,&#10;objects, scene categories, material types, texture, etc.) be grounded and what&#10;should be its structure? Aspiring to have one unified structure that hosts&#10;diverse types of semantics, we follow the Scene Graph paradigm in 3D,&#10;generating a 3D Scene Graph. Given a 3D mesh and registered panoramic images,&#10;we construct a graph that spans the entire building and includes semantics on&#10;objects (e.g., class, material, and other attributes), rooms (e.g., scene&#10;category, volume, etc.) and cameras (e.g., location, etc.), as well as the&#10;relationships among these entities.&#10;  However, this process is prohibitively labor heavy if done manually. To&#10;alleviate this we devise a semi-automatic framework that employs existing&#10;detection methods and enhances them using two main constraints: I. framing of&#10;query images sampled on panoramas to maximize the performance of 2D detectors,&#10;and II. multi-view consistency enforcement across 2D detections that originate&#10;in different camera locations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.16650" label="2309.16650">
-        <attvalues>
-          <attvalue for="0" value="ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and&#10;  Planning" />
-          <attvalue for="1" value="  For robots to perform a wide variety of tasks, they require a 3D&#10;representation of the world that is semantically rich, yet compact and&#10;efficient for task-driven perception and planning. Recent approaches have&#10;attempted to leverage features from large vision-language models to encode&#10;semantics in 3D representations. However, these approaches tend to produce maps&#10;with per-point feature vectors, which do not scale well in larger environments,&#10;nor do they contain semantic spatial relationships between entities in the&#10;environment, which are useful for downstream planning. In this work, we propose&#10;ConceptGraphs, an open-vocabulary graph-structured representation for 3D&#10;scenes. ConceptGraphs is built by leveraging 2D foundation models and fusing&#10;their output to 3D by multi-view association. The resulting representations&#10;generalize to novel semantic classes, without the need to collect large 3D&#10;datasets or finetune models. We demonstrate the utility of this representation&#10;through a number of downstream planning tasks that are specified through&#10;abstract (language) prompts and require complex reasoning over spatial and&#10;semantic concepts. (Project page: https://concept-graphs.github.io/ Explainer&#10;video: https://youtu.be/mRhNkQwRYnc )&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.08126" label="2206.08126">
-        <attvalues>
-          <attvalue for="0" value="Channel Importance Matters in Few-Shot Image Classification" />
-          <attvalue for="1" value="  Few-Shot Learning (FSL) requires vision models to quickly adapt to brand-new&#10;classification tasks with a shift in task distribution. Understanding the&#10;difficulties posed by this task distribution shift is central to FSL. In this&#10;paper, we show that a simple channel-wise feature transformation may be the key&#10;to unraveling this secret from a channel perspective. When facing novel&#10;few-shot tasks in the test-time datasets, this transformation can greatly&#10;improve the generalization ability of learned image representations, while&#10;being agnostic to the choice of training algorithms and datasets. Through an&#10;in-depth analysis of this transformation, we find that the difficulty of&#10;representation transfer in FSL stems from the severe channel bias problem of&#10;image representations: channels may have different importance in different&#10;tasks, while convolutional neural networks are likely to be insensitive, or&#10;respond incorrectly to such a shift. This points out a core problem of the&#10;generalization ability of modern vision systems and needs further attention in&#10;the future. Our code is available at&#10;https://github.com/Frankluox/Channel_Importance_FSL.&#10;" />
-          <attvalue for="2" value="&#10;Deep convolutional neural networks~\cite{Alexnet,Resnet} have revolutionized computer vision in the last decade, making it possible to automatically learn representations from a large number of images. The learned representations can generalize well to brand-new images. As a result, image classification performance is close to humans on most benchmarks. However, in addition to recognizing previously-seen categories, humans can quickly change their focus of image patterns in changing environments and recognize new categories given only a few observations. This fast learning capability, known as Few-Shot Learning (FSL), challenges current vision models on the ability to quickly adapt to novel classification tasks that are different from those in training. This task distribution shift means that categories, domains of images or granularity of categories in new tasks deviate from those in the training tasks.&#10;&#10;Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}. Thus it is crucial to understand how task distribution shift affects the generalization ability of image representations in few-shot learning.&#10;&#10;As shown in Figure \ref{example_channelchange}, task distribution shift may lead to changes in discriminative image features that are critical to the classification task at hand. For example, in the task of recognizing animals, a convolutional neural network trained on miniImageNet can successfully identify the discriminative information related to animals. Although the representations learned by the network may encode some plant information (from image background), plants do not appear as a main category in miniImageNet and it may be insufficient for the network to distinguish various plants in a novel few-shot task sampled from the iNaturalist dataset. Even when the network is well trained to recognize plants on iNaturalist, it is difficult to be adapted to the novel task of identifying plant diseases due to the granularity shift, since the discriminative information now becomes the more fine-grained lesion part of leaves.&#10;&#10;In this paper, we show that this difficulty encountered in few-shot learning leads to a channel bias problem in learned image representations (i.e., features). Specifically, in the layer after global pooling, different channels in the learned feature seek for different patterns~(as verified in~\cite{emergingobject,dissection}) during training, and the channels are weighted (in a biased way) based on their importance to the training task. However, when applied to novel few-shot classification tasks, the learned image features usually do not change much or have inappropriately changed without adapting to categories in novel tasks. This bias towards training tasks may result in imprecise attention to image features in novel tasks.&#10;&#10;What leads to our discovery of the channel bias problem is a simple transformation function that we found in a mathematical analysis textbook. Applied to top of image representations channel-wisely only at test time on the fly, this transformation function can consistently and largely improve predictions for out-of-distribution few-shot classification tasks, being agnostic to the choice of datasets and training algorithms~(e.g., 0.5-7.5\% average improvement over 5-way 5-shot tasks on 19 different test-time datasets, as shown in Table~\ref{performance}). Through analysis, we reveal the existence of channel bias problem, and show that this transformation rectifies channel emphasis by adjusting the Mean Magnitude of Channels (MMC) of image representations over the target &#10;task. Concretely, it serves as a smoothing function that suppresses channels of large MMC and largely amplifies channels of small MMC. &#10;&#10;To further understand the channel bias problem, we derive an oracle adjustment on the MMC of image representations in binary classification tasks. Such studies demonstrate that the channel bias problem exists in many different target tasks with various types of task distributions shift, and it becomes severe with the distribution shift expanding (as shown in Figure~\ref{visiualize_optimal}). In addition, through test-time shot analysis, we verify that the channel bias problem requires more attention in few-shot setting, while simple fine-tuning can help address this problem in many-shot setting.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Few-Shot Learning, Mathematics, Artificial Intelligence, Vision Systems, Cognitive Science, Computer Science, Image Representations" />
-        </attvalues>
-      </node>
-      <node id="1909.09157" label="1909.09157">
-        <attvalues>
-          <attvalue for="0" value="Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness&#10;  of MAML" />
-          <attvalue for="1" value="  An important research direction in machine learning has centered around&#10;developing meta-learning algorithms to tackle few-shot learning. An especially&#10;successful algorithm has been Model Agnostic Meta-Learning (MAML), a method&#10;that consists of two optimization loops, with the outer loop finding a&#10;meta-initialization, from which the inner loop can efficiently learn new tasks.&#10;Despite MAML's popularity, a fundamental open question remains -- is the&#10;effectiveness of MAML due to the meta-initialization being primed for rapid&#10;learning (large, efficient changes in the representations) or due to feature&#10;reuse, with the meta initialization already containing high quality features?&#10;We investigate this question, via ablation studies and analysis of the latent&#10;representations, finding that feature reuse is the dominant factor. This leads&#10;to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we&#10;remove the inner loop for all but the (task-specific) head of a MAML-trained&#10;network. ANIL matches MAML's performance on benchmark few-shot image&#10;classification and RL and offers computational improvements over MAML. We&#10;further study the precise contributions of the head and body of the network,&#10;showing that performance on the test tasks is entirely determined by the&#10;quality of the learned features, and we can remove even the head of the network&#10;(the NIL algorithm). We conclude with a discussion of the rapid learning vs&#10;feature reuse question for meta-learning algorithms more broadly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11498" label="2007.11498">
-        <attvalues>
-          <attvalue for="0" value="CrossTransformers: spatially-aware few-shot transfer" />
-          <attvalue for="1" value="  Given new tasks with very little data$-$such as new classes in a&#10;classification problem or a domain shift in the input$-$performance of modern&#10;vision systems degrades remarkably quickly. In this work, we illustrate how the&#10;neural network representations which underpin modern vision systems are subject&#10;to supervision collapse, whereby they lose any information that is not&#10;necessary for performing the training task, including information that may be&#10;necessary for transfer to new tasks or domains. We then propose two methods to&#10;mitigate this problem. First, we employ self-supervised learning to encourage&#10;general-purpose features that transfer better. Second, we propose a novel&#10;Transformer based neural network architecture called CrossTransformers, which&#10;can take a small number of labeled images and an unlabeled query, find coarse&#10;spatial correspondence between the query and the labeled images, and then infer&#10;class membership by computing distances between spatially-corresponding&#10;features. The result is a classifier that is more robust to task and domain&#10;shift, which we demonstrate via state-of-the-art performance on Meta-Dataset, a&#10;recent dataset for evaluating transfer from ImageNet to many other vision&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.02729" label="1909.02729">
-        <attvalues>
-          <attvalue for="0" value="A Baseline for Few-Shot Image Classification" />
-          <attvalue for="1" value="  Fine-tuning a deep network trained with the standard cross-entropy loss is a&#10;strong baseline for few-shot learning. When fine-tuned transductively, this&#10;outperforms the current state-of-the-art on standard datasets such as&#10;Mini-ImageNet, Tiered-ImageNet, CIFAR-FS and FC-100 with the same&#10;hyper-parameters. The simplicity of this approach enables us to demonstrate the&#10;first few-shot learning results on the ImageNet-21k dataset. We find that using&#10;a large number of meta-training classes results in high few-shot accuracies&#10;even for a large number of few-shot classes. We do not advocate our approach as&#10;the solution for few-shot learning, but simply use the results to highlight&#10;limitations of current benchmarks and few-shot protocols. We perform extensive&#10;studies on benchmark datasets to propose a metric that quantifies the&#10;&quot;hardness&quot; of a few-shot episode. This metric can be used to report the&#10;performance of few-shot algorithms in a more systematic way.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.11539" label="2003.11539">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Few-Shot Image Classification: a Good Embedding Is All You&#10;  Need?" />
-          <attvalue for="1" value="  The focus of recent meta-learning research has been on the development of&#10;learning algorithms that can quickly adapt to test time tasks with limited data&#10;and low computational cost. Few-shot learning is widely used as one of the&#10;standard benchmarks in meta-learning. In this work, we show that a simple&#10;baseline: learning a supervised or self-supervised representation on the&#10;meta-training set, followed by training a linear classifier on top of this&#10;representation, outperforms state-of-the-art few-shot learning methods. An&#10;additional boost can be achieved through the use of self-distillation. This&#10;demonstrates that using a good learned embedding model can be more effective&#10;than sophisticated meta-learning algorithms. We believe that our findings&#10;motivate a rethinking of few-shot image classification benchmarks and the&#10;associated role of meta-learning algorithms. Code is available at:&#10;http://github.com/WangYueFt/rfs/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.01315" label="2103.01315">
-        <attvalues>
-          <attvalue for="0" value="Exploring Complementary Strengths of Invariant and Equivariant&#10;  Representations for Few-Shot Learning" />
-          <attvalue for="1" value="  In many real-world problems, collecting a large number of labeled samples is&#10;infeasible. Few-shot learning (FSL) is the dominant approach to address this&#10;issue, where the objective is to quickly adapt to novel categories in presence&#10;of a limited number of samples. FSL tasks have been predominantly solved by&#10;leveraging the ideas from gradient-based meta-learning and metric learning&#10;approaches. However, recent works have demonstrated the significance of&#10;powerful feature representations with a simple embedding network that can&#10;outperform existing sophisticated FSL algorithms. In this work, we build on&#10;this insight and propose a novel training mechanism that simultaneously&#10;enforces equivariance and invariance to a general set of geometric&#10;transformations. Equivariance or invariance has been employed standalone in the&#10;previous works; however, to the best of our knowledge, they have not been used&#10;jointly. Simultaneous optimization for both of these contrasting objectives&#10;allows the model to jointly learn features that are not only independent of the&#10;input transformation but also the features that encode the structure of&#10;geometric transformations. These complementary sets of features help generalize&#10;well to novel classes with only a few data samples. We achieve additional&#10;improvements by incorporating a novel self-supervised distillation objective.&#10;Our extensive experimentation shows that even without knowledge distillation&#10;our proposed method can outperform current state-of-the-art FSL methods on five&#10;popular benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04390" label="2003.04390">
-        <attvalues>
-          <attvalue for="0" value="Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning" />
-          <attvalue for="1" value="  Meta-learning has been the most common framework for few-shot learning in&#10;recent years. It learns the model from collections of few-shot classification&#10;tasks, which is believed to have a key advantage of making the training&#10;objective consistent with the testing objective. However, some recent works&#10;report that by training for whole-classification, i.e. classification on the&#10;whole label-set, it can get comparable or even better embedding than many&#10;meta-learning algorithms. The edge between these two lines of works has yet&#10;been underexplored, and the effectiveness of meta-learning in few-shot learning&#10;remains unclear. In this paper, we explore a simple process: meta-learning over&#10;a whole-classification pre-trained model on its evaluation metric. We observe&#10;this simple method achieves competitive performance to state-of-the-art methods&#10;on standard benchmarks. Our further analysis shed some light on understanding&#10;the trade-offs between the meta-learning objective and the whole-classification&#10;objective in few-shot learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.13953" label="2110.13953">
-        <attvalues>
-          <attvalue for="0" value="On sensitivity of meta-learning to support data" />
-          <attvalue for="1" value="  Meta-learning algorithms are widely used for few-shot learning. For example,&#10;image recognition systems that readily adapt to unseen classes after seeing&#10;only a few labeled examples. Despite their success, we show that modern&#10;meta-learning algorithms are extremely sensitive to the data used for&#10;adaptation, i.e. support data. In particular, we demonstrate the existence of&#10;(unaltered, in-distribution, natural) images that, when used for adaptation,&#10;yield accuracy as low as 4\% or as high as 95\% on standard few-shot image&#10;classification benchmarks. We explain our empirical findings in terms of class&#10;margins, which in turn suggests that robust and safe meta-learning requires&#10;larger margins than supervised learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1412.6856" label="1412.6856">
-        <attvalues>
-          <attvalue for="0" value="Object Detectors Emerge in Deep Scene CNNs" />
-          <attvalue for="1" value="  With the success of new computational architectures for visual processing,&#10;such as convolutional neural networks (CNN) and access to image databases with&#10;millions of labeled examples (e.g., ImageNet, Places), the state of the art in&#10;computer vision is advancing rapidly. One important factor for continued&#10;progress is to understand the representations that are learned by the inner&#10;layers of these deep architectures. Here we show that object detectors emerge&#10;from training CNNs to perform scene classification. As scenes are composed of&#10;objects, the CNN for scene classification automatically discovers meaningful&#10;objects detectors, representative of the learned scene categories. With object&#10;detectors emerging as a result of learning to recognize scenes, our work&#10;demonstrates that the same network can perform both scene recognition and&#10;object localization in a single forward-pass, without ever having been&#10;explicitly taught the notion of objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.05796" label="1704.05796">
-        <attvalues>
-          <attvalue for="0" value="Network Dissection: Quantifying Interpretability of Deep Visual&#10;  Representations" />
-          <attvalue for="1" value="  We propose a general framework called Network Dissection for quantifying the&#10;interpretability of latent representations of CNNs by evaluating the alignment&#10;between individual hidden units and a set of semantic concepts. Given any CNN&#10;model, the proposed method draws on a broad data set of visual concepts to&#10;score the semantics of hidden units at each intermediate convolutional layer.&#10;The units with semantics are given labels across a range of objects, parts,&#10;scenes, textures, materials, and colors. We use the proposed method to test the&#10;hypothesis that interpretability of units is equivalent to random linear&#10;combinations of units, then we apply our method to compare the latent&#10;representations of various networks when trained to solve different supervised&#10;and self-supervised training tasks. We further analyze the effect of training&#10;iterations, compare networks trained with different initializations, examine&#10;the impact of network depth and width, and measure the effect of dropout and&#10;batch normalization on the interpretability of deep visual representations. We&#10;demonstrate that the proposed method can shed light on characteristics of CNN&#10;models and training methods that go beyond measurements of their discriminative&#10;power.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2311.16589" label="2311.16589">
-        <attvalues>
-          <attvalue for="0" value="HD Maps are Lane Detection Generalizers: A Novel Generative Framework&#10;  for Single-Source Domain Generalization" />
-          <attvalue for="1" value="  Lane detection is a vital task for vehicles to navigate and localize their&#10;position on the road. To ensure reliable driving, lane detection models must&#10;have robust generalization performance in various road environments. However,&#10;despite the advanced performance in the trained domain, their generalization&#10;performance still falls short of expectations due to the domain discrepancy. To&#10;bridge this gap, we propose a novel generative framework using HD Maps for&#10;Single-Source Domain Generalization (SSDG) in lane detection. We first generate&#10;numerous front-view images from lane markings of HD Maps. Next, we&#10;strategically select a core subset among the generated images using (i) lane&#10;structure and (ii) road surrounding criteria to maximize their diversity. In&#10;the end, utilizing this core set, we train lane detection models to boost their&#10;generalization performance. We validate that our generative framework from HD&#10;Maps outperforms the Domain Adaptation model MLDA with +3.01%p accuracy&#10;improvement, even though we do not access the target domain images.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Lane detection is a fundamental task in autonomous driving. &#10;By detecting lane markings on the road, vehicles can localize their position and identify drivable spaces. &#10;However, current data-driven lane detection models still suffer from performance degradation when tested on unseen domains.&#10;Such discrepancies between the source and target datasets result in a severe performance decrease. &#10;&#10;Recent studies~\cite{dg1,dg2,dg3,dg4,dg5} have attempted to address this performance degradation.&#10;These approaches include Domain Adaptation (DA), which involves making use of a limited number of data samples from the target domain, and Domain Generalization (DG), which leverages data from multiple source domains.&#10;However, in realistic driving scenarios, it is technically impossible to obtain even a small number of data samples from the target domain since they cannot be predetermined. &#10;Collecting and annotating data manually from numerous regions also have practical limitations.&#10;Therefore, it is more realistic to achieve generalization solely relying on a single-source domain without any prior information about the target domain.&#10;&#10;From the Single-Source Domain Generalization (SSDG) perspective, it is imperative to obtain domain-invariant features, which necessitates a sufficiently diverse set of source domain data. &#10;To achieve this diversity, previous works~\cite{hendrycks2019augmix, volpi2018generalizing, qiao2020learning, ssdg1, ssdg2, ssdg3} actively adopt augmentation techniques such as Augmix~\cite{hendrycks2019augmix} or adversarial methods~\cite{volpi2018generalizing, qiao2020learning}. &#10;However, SSDG for lane detection has been unexplored, even though the significance for the real-world application. &#10;&#10;Therefore, we propose a model-agnostic generative framework to generalize lane detection models adopting HD Maps as a single-source domain.&#10;To the best of our knowledge, this is the first paper addressing the domain shifts in lane detection with SSDG setup. &#10;Notably, we notify that HD Maps are effective lane-detection generalizers thanks to their flexibility in collecting numerous datasets depending on the camera parameters. &#10;Utilizing this advantage, we first get the 2D lane masks from pre-built HD Maps and generate lane-conditioned images with various surroundings. &#10;Next, we strategically select a core set among the generated images to maximize diversity by using (i) lane structure and (ii) road surrounding criteria. &#10;This can help to boost the robustness of our single-source domain from HD Maps generation. &#10;In the end, we train lane detection models regarding this core set as a single-source domain. &#10;&#10;The main contributions of the paper are summarized as follows:&#10;&#10;\begin{itemize}&#10;&#10;\item We propose a model-agnostic generative framework to generalize a lane detection model utilizing a single-source domain from HD Maps, addressing the SSDG problem in lane detection for the first time.&#10;&#10;\item After decomposing the factor of domain shifts with (i) lane structure and (ii) road surrounding, we propose the core set selection criteria to maximize the diversity of single-source domain. &#10;&#10;\item Diverse experimental validations have demonstrated that the proposed training framework outperforms previous DA methods without any target domain information.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Lane Detection, Mathematics, Autonomous Systems, Computer Science, Domain Generalization" />
-        </attvalues>
-      </node>
-      <node id="1710.03077" label="1710.03077">
-        <attvalues>
-          <attvalue for="0" value="Deeper, Broader and Artier Domain Generalization" />
-          <attvalue for="1" value="  The problem of domain generalization is to learn from multiple training&#10;domains, and extract a domain-agnostic model that can then be applied to an&#10;unseen domain. Domain generalization (DG) has a clear motivation in contexts&#10;where there are target domains with distinct characteristics, yet sparse data&#10;for training. For example recognition in sketch images, which are distinctly&#10;more abstract and rarer than photos. Nevertheless, DG methods have primarily&#10;been evaluated on photo-only benchmarks focusing on alleviating the dataset&#10;bias where both problems of domain distinctiveness and data sparsity can be&#10;minimal. We argue that these benchmarks are overly straightforward, and show&#10;that simple deep learning baselines perform surprisingly well on them. In this&#10;paper, we make two main contributions: Firstly, we build upon the favorable&#10;domain shift-robust properties of deep learning methods, and develop a low-rank&#10;parameterized CNN model for end-to-end DG learning. Secondly, we develop a DG&#10;benchmark dataset covering photo, sketch, cartoon and painting domains. This is&#10;both more practically relevant, and harder (bigger domain shift) than existing&#10;benchmarks. The results show that our method outperforms existing DG&#10;alternatives, and our dataset provides a more significant DG challenge to drive&#10;future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1301.2115" label="1301.2115">
-        <attvalues>
-          <attvalue for="0" value="Domain Generalization via Invariant Feature Representation" />
-          <attvalue for="1" value="  This paper investigates domain generalization: How to take knowledge acquired&#10;from an arbitrary number of related domains and apply it to previously unseen&#10;domains? We propose Domain-Invariant Component Analysis (DICA), a kernel-based&#10;optimization algorithm that learns an invariant transformation by minimizing&#10;the dissimilarity across domains, whilst preserving the functional relationship&#10;between input and output variables. A learning-theoretic analysis shows that&#10;reducing dissimilarity improves the expected generalization ability of&#10;classifiers on new domains, motivating the proposed algorithm. Experimental&#10;results on synthetic and real-world datasets demonstrate that DICA successfully&#10;learns invariant features and improves classifier performance in practice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.13580" label="1910.13580">
-        <attvalues>
-          <attvalue for="0" value="Domain Generalization via Model-Agnostic Learning of Semantic Features" />
-          <attvalue for="1" value="  Generalization capability to unseen domains is crucial for machine learning&#10;models when deploying to real-world conditions. We investigate the challenging&#10;problem of domain generalization, i.e., training a model on multi-domain source&#10;data such that it can directly generalize to target domains with unknown&#10;statistics. We adopt a model-agnostic learning paradigm with gradient-based&#10;meta-train and meta-test procedures to expose the optimization to domain shift.&#10;Further, we introduce two complementary losses which explicitly regularize the&#10;semantic structure of the feature space. Globally, we align a derived soft&#10;confusion matrix to preserve general knowledge about inter-class relationships.&#10;Locally, we promote domain-independent class-specific cohesion and separation&#10;of sample features with a metric-learning component. The effectiveness of our&#10;method is demonstrated with new state-of-the-art results on two common object&#10;recognition benchmarks. Our method also shows consistent improvement on a&#10;medical image segmentation task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.12018" label="1805.12018">
-        <attvalues>
-          <attvalue for="0" value="Generalizing to Unseen Domains via Adversarial Data Augmentation" />
-          <attvalue for="1" value="  We are concerned with learning models that generalize well to different&#10;\emph{unseen} domains. We consider a worst-case formulation over data&#10;distributions that are near the source domain in the feature space. Only using&#10;training data from a single source distribution, we propose an iterative&#10;procedure that augments the dataset with examples from a fictitious target&#10;domain that is &quot;hard&quot; under the current model. We show that our iterative&#10;scheme is an adaptive data augmentation method where we append adversarial&#10;examples at each iteration. For softmax losses, we show that our method is a&#10;data-dependent regularization scheme that behaves differently from classical&#10;regularizers that regularize towards zero (e.g., ridge or lasso). On digit&#10;recognition and semantic segmentation tasks, our method learns models improve&#10;performance across a range of a priori unknown target domains.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.02052" label="2201.02052">
-        <attvalues>
-          <attvalue for="0" value="A Unified Framework for Attention-Based Few-Shot Object Detection" />
-          <attvalue for="1" value="  Few-Shot Object Detection (FSOD) is a rapidly growing field in computer&#10;vision. It consists in finding all occurrences of a given set of classes with&#10;only a few annotated examples for each class. Numerous methods have been&#10;proposed to address this challenge and most of them are based on attention&#10;mechanisms. However, the great variety of classic object detection frameworks&#10;and training strategies makes performance comparison between methods difficult.&#10;In particular, for attention-based FSOD methods, it is laborious to compare the&#10;impact of the different attention mechanisms on performance. This paper aims at&#10;filling this shortcoming. To do so, a flexible framework is proposed to allow&#10;the implementation of most of the attention techniques available in the&#10;literature. To properly introduce such a framework, a detailed review of the&#10;existing FSOD methods is firstly provided. Some different attention mechanisms&#10;are then reimplemented within the framework and compared with all other&#10;parameters fixed.&#10;" />
-          <attvalue for="2" value="&#10;Few-Shot Object Detection (FSOD) is a challenging problem that aims to find all&#10;occurrences of a class in an image given only a few examples. Impressive&#10;progress has been made in object detection in the past decade, mostly because of&#10;deep convolutional networks (see e.g. \cite{ren2015faster,redmon2016you}).&#10;Current state-of-the-art performs high-quality detection, but it requires large&#10;annotated datasets and days of training to achieve that quality. Often, such&#10;requirements could not be met, and it is quite hard to achieve good performance&#10;for a specific task. The few-shot learning trend focuses specifically on this&#10;kind of use cases where data is scarce. Most few-shot approaches rely on a&#10;two-steps training strategy. First, the model is trained on a large dataset,&#10;slightly different from the desired task, to acquire general knowledge. Second,&#10;a fine-tuning step refines the model to learn the actual task. Of course, plenty&#10;of techniques have been introduced so that the fine-tuning does not disrupt base&#10;learning (see e.g. \cite{kirkpatrick2017overcoming}). This has been extensively&#10;studied for classification in the past years. However, detection is a more&#10;challenging task than classification and has been only tackled recently from a&#10;few-shot perspective.&#10;&#10;Current FSOD state-of-the-art is mainly based on attention mechanisms, which aim&#10;at extracting information about the task (i.e. semantic information about the&#10;classes that should be detected) from support examples. This information allows&#10;the network to condition the detection on the examples. It allows the network to&#10;adapt on the fly, provided with a few annotated images and a short fine-tuning&#10;step. A seminal work in this direction is presented in reference&#10;\cite{kang2019few} which reweights features from the query image (i.e. the&#10;images in which the model performs the detection) with the features extracted&#10;from the support images. Plenty of methods based on the same idea have been&#10;introduced (see e.g. \cite{deng2020few,fan2020few,wallach2019one,li2020one}).&#10;Although the attention mechanisms, deployed in most FSOD methods, might differ&#10;from the original one proposed in \cite{kang2019few}, the main principle remains&#10;the same. Indeed, it combines information from the query image and the support&#10;set to detect only the objects annotated in the support.&#10;&#10;The FSOD field is rapidly growing, and most new papers propose a novel attention&#10;technique. However, there are a lot of design choices that can be considered to&#10;address the FSOD problem. First, the detection framework (e.g. Faster R-CNN&#10;\cite{ren2015faster} or YOLO \cite{redmon2016you}) and its backbone (e.g.&#10;ResNet-50 or 101), then the different loss functions (e.g. L1, IoU, Focal Loss)&#10;to train each part of the network and finally all the hyperparameters that are&#10;tied to these methods (i.e. from the learning rate to the class splits for&#10;evaluation). All this makes the comparison between FSOD methods difficult. In&#10;this paper, the attention mechanism is considered as the heart of FSOD because&#10;it combines general features extracted from the input image and the conditioning&#10;features extracted from the support examples. Of course, it should be noted that&#10;not all works on FSOD are attention-based. Indeed, there exist some papers that&#10;use metric learning to solve FSOD (see e.g.&#10;\cite{karlinsky2019repmet,jeune2021experience,sun2021fsce}). Instead of&#10;combining query and support features, a generic embedding function is learned.&#10;The detection is then obtained from the comparison of the query and support&#10;embeddings. There are also approaches based only on fine-tuning. Although quite&#10;straightforward these methods generally do not perform as well as&#10;attention-based ones.&#10;&#10;The main goal of this paper is to review a part of the attention-based FSOD&#10;methods and compare them fairly. To do so, a modular attention framework is&#10;proposed. It allows regrouping most attention-based methods under the same&#10;notations. Specifically, this framework is divided into three parts: spatial&#10;alignment, global attention, and fusion layer. This separation helps to easily&#10;implement the different mechanisms and facilitates the comparison. Most&#10;importantly, this makes it possible to completely fix the other parameters and&#10;design choices without reimplementing each method. That way, a fair comparison&#10;of the performance of the different mechanisms can be obtained. The modular&#10;splitting also allows a deep understanding of what component of the attention is&#10;key for FSOD and gives insights to build better approaches. Moreover, to help&#10;the development of new techniques for FSOD and future comparisons, the code of&#10;the proposed framework will be made&#10;available. To our&#10;knowledge, there exist two papers that review FSOD methods&#10;\cite{huang2021survey, jiaxu2021comparative}. However, they focus on general&#10;FSOD while this work targets attention-based FSOD.&#10;&#10;The rest of this paper is organized as follows. First, a review of existing&#10;methods for FSOD is conducted in Section \ref{sec:related}. A formal description&#10;of the attention framework is then proposed in Section \ref{sec:framework}.&#10;Section \ref{sec:comparison} confirms the flexibility of the framework with a&#10;comparative analysis of some existing methods for FSOD. All these methods have&#10;been reimplemented through the proposed framework, achieving similar results to&#10;those reported in the original papers. Some insights about the key elements of&#10;the attention mechanisms are then provided. Section \ref{sec:conclusion}&#10;concludes this paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Attention Mechanisms, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1612.00796" label="1612.00796">
-        <attvalues>
-          <attvalue for="0" value="Overcoming catastrophic forgetting in neural networks" />
-          <attvalue for="1" value="  The ability to learn tasks in a sequential fashion is crucial to the&#10;development of artificial intelligence. Neural networks are not, in general,&#10;capable of this and it has been widely thought that catastrophic forgetting is&#10;an inevitable feature of connectionist models. We show that it is possible to&#10;overcome this limitation and train networks that can maintain expertise on&#10;tasks which they have not experienced for a long time. Our approach remembers&#10;old tasks by selectively slowing down learning on the weights important for&#10;those tasks. We demonstrate our approach is scalable and effective by solving a&#10;set of classification tasks based on the MNIST hand written digit dataset and&#10;by learning several Atari 2600 games sequentially.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.01866" label="1812.01866">
-        <attvalues>
-          <attvalue for="0" value="Few-shot Object Detection via Feature Reweighting" />
-          <attvalue for="1" value="  Conventional training of a deep CNN based object detector demands a large&#10;number of bounding box annotations, which may be unavailable for rare&#10;categories. In this work we develop a few-shot object detector that can learn&#10;to detect novel objects from only a few annotated examples. Our proposed model&#10;leverages fully labeled base classes and quickly adapts to novel classes, using&#10;a meta feature learner and a reweighting module within a one-stage detection&#10;architecture. The feature learner extracts meta features that are generalizable&#10;to detect novel object classes, using training data from base classes with&#10;sufficient samples. The reweighting module transforms a few support examples&#10;from the novel classes to a global vector that indicates the importance or&#10;relevance of meta features for detecting the corresponding objects. These two&#10;modules, together with a detection prediction module, are trained end-to-end&#10;based on an episodic few-shot learning scheme and a carefully designed loss&#10;function. Through extensive experiments we demonstrate that our model&#10;outperforms well-established baselines by a large margin for few-shot object&#10;detection, on multiple datasets and settings. We also present analysis on&#10;various aspects of our proposed model, aiming to provide some inspiration for&#10;future few-shot detection works.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.07826" label="2006.07826">
-        <attvalues>
-          <attvalue for="0" value="Few-shot Object Detection on Remote Sensing Images" />
-          <attvalue for="1" value="  In this paper, we deal with the problem of object detection on remote sensing&#10;images. Previous methods have developed numerous deep CNN-based methods for&#10;object detection on remote sensing images and the report remarkable&#10;achievements in detection performance and efficiency. However, current&#10;CNN-based methods mostly require a large number of annotated samples to train&#10;deep neural networks and tend to have limited generalization abilities for&#10;unseen object categories. In this paper, we introduce a few-shot learning-based&#10;method for object detection on remote sensing images where only a few annotated&#10;samples are provided for the unseen object categories. More specifically, our&#10;model contains three main components: a meta feature extractor that learns to&#10;extract feature representations from input images, a reweighting module that&#10;learn to adaptively assign different weights for each feature representation&#10;from the support images, and a bounding box prediction module that carries out&#10;object detection on the reweighted feature maps. We build our few-shot object&#10;detection model upon YOLOv3 architecture and develop a multi-scale object&#10;detection framework. Experiments on two benchmark datasets demonstrate that&#10;with only a few annotated samples our model can still achieve a satisfying&#10;detection performance on remote sensing images and the performance of our model&#10;is significantly better than the well-established baseline models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.01998" label="1908.01998">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector" />
-          <attvalue for="1" value="  Conventional methods for object detection typically require a substantial&#10;amount of training data and preparing such high-quality training data is very&#10;labor-intensive. In this paper, we propose a novel few-shot object detection&#10;network that aims at detecting objects of unseen categories with only a few&#10;annotated examples. Central to our method are our Attention-RPN, Multi-Relation&#10;Detector and Contrastive Training strategy, which exploit the similarity&#10;between the few shot support set and query set to detect novel objects while&#10;suppressing false detection in the background. To train our network, we&#10;contribute a new dataset that contains 1000 categories of various objects with&#10;high-quality annotations. To the best of our knowledge, this is one of the&#10;first datasets specifically designed for few-shot object detection. Once our&#10;few-shot network is trained, it can detect objects of unseen categories without&#10;further training or fine-tuning. Our method is general and has a wide range of&#10;potential applications. We produce a new state-of-the-art performance on&#10;different datasets in the few-shot setting. The dataset link is&#10;https://github.com/fanq15/Few-Shot-Object-Detection-Dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.12529" label="1911.12529">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Object Detection with Co-Attention and Co-Excitation" />
-          <attvalue for="1" value="  This paper aims to tackle the challenging problem of one-shot object&#10;detection. Given a query image patch whose class label is not included in the&#10;training data, the goal of the task is to detect all instances of the same&#10;class in a target image. To this end, we develop a novel {\em co-attention and&#10;co-excitation} (CoAE) framework that makes contributions in three key technical&#10;aspects. First, we propose to use the non-local operation to explore the&#10;co-attention embodied in each query-target pair and yield region proposals&#10;accounting for the one-shot situation. Second, we formulate a&#10;squeeze-and-co-excitation scheme that can adaptively emphasize correlated&#10;feature channels to help uncover relevant proposals and eventually the target&#10;objects. Third, we design a margin-based ranking loss for implicitly learning a&#10;metric to predict the similarity of a region proposal to the underlying query,&#10;no matter its class label is seen or unseen in training. The resulting model is&#10;therefore a two-stage detector that yields a strong baseline on both VOC and&#10;MS-COCO under one-shot setting of detecting objects from both seen and&#10;never-seen classes. Codes are available at&#10;https://github.com/timy90022/One-Shot-Object-Detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Object detection State-of-the-art object detectors unanimously adopt variants of deep convolutional neural networks as their backbones and have been improving the performance on large-scale benchmarks. Two types of pipeline designs are often taken into consideration by recent object detectors: one-stage (proposal-free) \cite{Kong19FoveaBox,Law18CornerNet,Lin17RetinaNet,Liu16SSD,Redmon16YOLO,Sermanet13OverFeat} and two-stage (proposal-based) \cite{Cai18CascadeRCNN,He15SPPNet,He17MaskRCNN,Girshick14RCNN,Girshick15FastRCNN,Ren15FasterRCNN}. Two-stage detectors generate a set of region proposals at the first stage, and then classify the proposals as well as refine their locations at the second stage. The two-stage pipeline is first demonstrated by R-CNN \cite{Girshick14RCNN} and further improved by Faster R-CNN \cite{Ren15FasterRCNN}, which replaces the grouping-based proposal method with a region proposal network (RPN), making the whole pipeline end-to-end trainable. Subsequently, state-of-the-art two-stage object detectors \cite{Cai18CascadeRCNN,He17MaskRCNN} mainly follow Faster R-CNN in the design of architecture. In contrast, one-stage detectors like \cite{Kong19FoveaBox,Law18CornerNet,Lin17RetinaNet,Liu16SSD,Redmon16YOLO,Sermanet13OverFeat} trade localization performance for fast inference speed by skipping the region-proposal step and directly predicting the bounding boxes and the corresponding class labels with respect to a fixed set of anchors.&#10;&#10;Few-shot classification via metric learning&#10;The aim of metric-learning based few-shot classification is to derive a similarity metric that can be directly applied to the inference of unseen classes supported by a set of labeled examples (\ie, {support set}). The setting of {$N$-way $K$-shot} classification is considered to have a support set containing $K$ labeled examples for each of $N$ classes, where $K = 0$, $K = 1$, and $K &gt; 1$ mean zero-shot, one-shot, and few-shot, respectively.&#10;\cite{Koch15SiameseNN} presents the first principled approach that employs Siamese networks for one-shot image classification. The Siamese networks learn a general similarity metric from pairs of input images to decide whether the two images belong to the same class. Then, during inference, the Siamese networks can be used to match unlabeled images of either seen or unseen classes with the one-shot support set. The prediction is done by assigning the test image with the class label of the most similar example in the support set.&#10;\cite{Vinyals16MatchingNN} propose the matching networks and an episodic training strategy tailored to the few-shot criterion. The matching networks learn a more powerful similarity metric using an attention mechanism \cite{Bahdanau14NMT,Vinyals15PointerNN} over the examples in the support set.&#10;Instead of associating unlabeled samples with their nearest support examples,&#10;the prototypical networks proposed by \cite{Snell17PrototypicalNN} map the unlabeled samples to the nearest `class prototype' of the support set.&#10;\cite{Snell17PrototypicalNN} also show that the prototypical networks can be applied to zero-shot setting where the class prototype becomes the semantic vector of the class.&#10;\cite{Sung18RelationNN} present the relation network, which is similar to \cite{Koch15SiameseNN,Vinyals16MatchingNN,Snell17PrototypicalNN} but learns the similarity metric fully based on a relational convolutional block instead of the Euclidean or cosine distance.&#10;&#10;Few-shot object detection Similar to few-shot classification, the problem of object detection can also be addressed under a few-shot setting.&#10;This problem is relatively new and less explored, and only a few preliminary results are reported from the perspectives of transfer learning \cite{Chen18LSTD}, meta learning \cite{Kang18FeatureReweighting}, or metric learning \cite{Michaelis18SiameseMask,Schwartz18RepMet,Zhang19ComparisonNN}.&#10;For transfer learning, \cite{Chen18LSTD} present the regularization techniques to address overfitting when directly training on a handful of labeled images of unseen classes.&#10;For meta learning, \cite{Kang18FeatureReweighting} propose a meta-model that is trained to reweight the features of an input image extracted from a base detection model. The reweighted features can be adapted to the detection of novel objects from a few examples.&#10;For metric learning, \cite{Michaelis18SiameseMask,Schwartz18RepMet,Zhang19ComparisonNN} share a similar framework that replaces the conventional classifier in the object detector with a metric-based classifier akin to \cite{Koch15SiameseNN,Vinyals16MatchingNN,Snell17PrototypicalNN,Sung18RelationNN}.&#10;&#10;The problem formulation of our work is more closely related to \cite{Koch15SiameseNN,Vinyals16MatchingNN,Snell17PrototypicalNN,Sung18RelationNN,Michaelis18SiameseMask,Zhang19ComparisonNN} than \cite{Chen18LSTD,Kang18FeatureReweighting,Schwartz18RepMet}. Our formulation is class-agnostic and training-free for unseen novel classes. Once the training process is done, our model can be used to detect objects of unseen classes without either knowing the classes beforehand or the need of fine-tuning.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2005.03819" label="2005.03819">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Object Detection without Fine-Tuning" />
-          <attvalue for="1" value="  Deep learning has revolutionized object detection thanks to large-scale&#10;datasets, but their object categories are still arguably very limited. In this&#10;paper, we attempt to enrich such categories by addressing the one-shot object&#10;detection problem, where the number of annotated training examples for learning&#10;an unseen class is limited to one. We introduce a two-stage model consisting of&#10;a first stage Matching-FCOS network and a second stage Structure-Aware Relation&#10;Module, the combination of which integrates metric learning with an anchor-free&#10;Faster R-CNN-style detection pipeline, eventually eliminating the need to&#10;fine-tune on the support images. We also propose novel training strategies that&#10;effectively improve detection performance. Extensive quantitative and&#10;qualitative evaluations were performed and our method exceeds the&#10;state-of-the-art one-shot performance consistently on multiple datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.13027" label="2109.13027">
-        <attvalues>
-          <attvalue for="0" value="Experience feedback using Representation Learning for Few-Shot Object&#10;  Detection on Aerial Images" />
-          <attvalue for="1" value="  This paper proposes a few-shot method based on Faster R-CNN and&#10;representation learning for object detection in aerial images. The two&#10;classification branches of Faster R-CNN are replaced by prototypical networks&#10;for online adaptation to new classes. These networks produce embeddings vectors&#10;for each generated box, which are then compared with class prototypes. The&#10;distance between an embedding and a prototype determines the corresponding&#10;classification score. The resulting networks are trained in an episodic manner.&#10;A new detection task is randomly sampled at each epoch, consisting in detecting&#10;only a subset of the classes annotated in the dataset. This training strategy&#10;encourages the network to adapt to new classes as it would at test time. In&#10;addition, several ideas are explored to improve the proposed method such as a&#10;hard negative examples mining strategy and self-supervised clustering for&#10;background objects. The performance of our method is assessed on DOTA, a&#10;large-scale remote sensing images dataset. The experiments conducted provide a&#10;broader understanding of the capabilities of representation learning. It&#10;highlights in particular some intrinsic weaknesses for the few-shot object&#10;detection task. Finally, some suggestions and perspectives are formulated&#10;according to these insights.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.05950" label="2103.05950">
-        <attvalues>
-          <attvalue for="0" value="FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding" />
-          <attvalue for="1" value="  Emerging interests have been brought to recognize previously unseen objects&#10;given very few training examples, known as few-shot object detection (FSOD).&#10;Recent researches demonstrate that good feature embedding is the key to reach&#10;favorable few-shot learning performance. We observe object proposals with&#10;different Intersection-of-Union (IoU) scores are analogous to the intra-image&#10;augmentation used in contrastive approaches. And we exploit this analogy and&#10;incorporate supervised contrastive learning to achieve more robust objects&#10;representations in FSOD. We present Few-Shot object detection via Contrastive&#10;proposals Encoding (FSCE), a simple yet effective approach to learning&#10;contrastive-aware object proposal encodings that facilitate the classification&#10;of detected objects. We notice the degradation of average precision (AP) for&#10;rare objects mainly comes from misclassifying novel instances as confusable&#10;classes. And we ease the misclassification issues by promoting instance level&#10;intra-class compactness and inter-class variance via our contrastive proposal&#10;encoding loss (CPE loss). Our design outperforms current state-of-the-art works&#10;in any shot and all data splits, with up to +8.8% on standard benchmark PASCAL&#10;VOC and +2.7% on challenging COCO benchmark. Code is available at: https:&#10;//github.com/MegviiDetection/FSCE&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Few-shot learning. Few-shot learning aims to recognize new concepts given limited labeled examples. Meta-learning approaches aim at training a meta-model on episodes of individual tasks such that it can adapt to new tasks with few samples \cite{finn_model-agnostic_2017,ren_meta-learning_2018,sun_meta-transfer_nodate,khodadadeh_unsupervised_2019,ravi_optimization_2017,nichol_reptile_nodate,rusu_meta-learning_2018}, known as ``learning-to-learn''. Deep metric-learning based approaches emphasize learning good feature representation embeddings that facilitate downstream tasks. The most intuitive metrics including cosine similarity~\cite{cos_face,cos_softmax,chen_closer_2020,deng_arcface_2019}, euclidean distance to class center~\cite{snell_prototypical_2017}, and graph distances~\cite{garcia_few-shot_2018}. Interestingly, hallucinator-based methods solve the data deficiency via learning to generate fake-data~\cite{wang_low-shot_2018}. Existing few-shot learners are mostly developed in the context of classification. In comparison, few-shot detection is more challenging as it involves both classification and localization, yet under-researched.&#10;&#10;Few-shot object detection. There are two lines of work addressing the challenging few-shot object detection (FSOD) problem. First, meta-learning based approaches devise a stage-wise and periodic meta-training paradigm to train a meta-learner to help knowledge transfer from base classes. Meta R-CNN \cite{yan_meta_2020} meta-learns channel-wise attention layer for remodeling the RoI head. MetaDet \cite{wang_meta-learning_2019} applies a weight prediction meta-model to dynamically transfer category-specific parameters from the base detector. FSIW \cite{xiao_few_shot_2020} improves upon Meta R-CNN and FSRW \cite{kang_few-shot_2019} by more complex feature aggregation and meta-training on a balanced dataset. With the balanced dataset introduced in TFA \cite{wang_frustratingly_2020}, fine-tune based detectors are rowing over meta-learning based methods in performance, MPSR~\cite{wu_multi-scale_2020} sets the current state-of-the-art by mitigating the scale scarcity in few-shot datasets, but its generalizability is limited because the positive refinement branch contains manual decisions. RepMet~\cite{karlinsky_repmet_2018} attaches an embedding sub-net in RoI head to model a posterior class distribution. It utilizes advanced tricks including OHEM~\cite{shrivastava_training_2016} and SoftNMS~\cite{bodla_soft-nms_2017} but fails to catch up with current SOTA. We criticize complex algorithms as they can easily overfit and exhibit poor test results in FSOD. Instead, our insight here is that the degeneration of average precision (AP) for novel categories mainly comes from misclassifying novel instances as confusable categories, and we resort to contrastive learning to learn discriminative object proposal representations without complexing the model.&#10;&#10;Contrastive learning&#10;The recent success of self-supervised models can be attributed to the renewed interest in exploring contrastive learning. \cite{hjelm_learning_2018,wu_unsupervised_2018,oord_representation_2019,chuang_debiased_2020,he_momentum_2020,chen_improved_2020,chen_simple_2020,chen_big_2020}. Optimizing the contrastive objectives \cite{oord_representation_2019,cos_face,deng_arcface_2019,supervised_contrastive_learning} simultaneously maximize the agreement between similar instances defined as positive pairs and encourage the difference among dissimilar instances or negative pairs. With contrastive learning, the algorithm learns to build representations that do not concentrate on pixel-level details, but encoding high-level features effective enough to distinguish different images \cite{chen_simple_2020,he_momentum_2020,chen_improved_2020,chen_big_2020}. Supervised contrastive learning~\cite{supervised_contrastive_learning} extends the batch contrastive approach to supervised setting, but for image classification. &#10;&#10;To our best knowledge, this work is the first to integrate supervised contrastive learning~\cite{sun2014deep,supervised_contrastive_learning} into few-shot object detection. The state-of-the-art few-shot detection performance in any shot and all benchmarks demonstrate the effectiveness of our proposed method.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2110.14711" label="2110.14711">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Self-Supervised and Few-Shot Object Detection" />
-          <attvalue for="1" value="  Labeling data is often expensive and time-consuming, especially for tasks&#10;such as object detection and instance segmentation, which require dense&#10;labeling of the image. While few-shot object detection is about training a&#10;model on novel (unseen) object classes with little data, it still requires&#10;prior training on many labeled examples of base (seen) classes. On the other&#10;hand, self-supervised methods aim at learning representations from unlabeled&#10;data which transfer well to downstream tasks such as object detection.&#10;Combining few-shot and self-supervised object detection is a promising research&#10;direction. In this survey, we review and characterize the most recent&#10;approaches on few-shot and self-supervised object detection. Then, we give our&#10;main takeaways and discuss future research directions. Project page at&#10;https://gabrielhuang.github.io/fsod-survey/&#10;" />
-          <attvalue for="2" value="&#10;&#10;T{raditional} object detectors rely on large supervised object detection datasets such as PASCAL VOC~\cite{pascal} and MS COCO~\cite{coco}, which have over hundreds and thousands of annotated examples per object category. &#10;However, labeling data is often expensive and time-consuming. This is especially true in the case of object detection and instance segmentation, which require dense labeling of bounding boxes/masks for each object, a process that is slower and requires more annotator training than for object classification. &#10;Moreover, for fine-grained object detection applications such as plant or animal species recognition, pre-labeled datasets may not exist, and labels may have to be collected on the spot by expert annotators.&#10;&#10;To try to solve these problems, few-shot object detection (FSOD) methods attempt to recognize novel (unseen) object classes based only on a few examples, after training on many labeled examples of base (seen) classes.&#10;Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr}&#10;However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}.&#10;&#10;The problem with typical self-supervised pretrained methods such as SimCLR~\cite{simclr} or MoCo~\cite{moco} is that they are geared towards classification, and often engineered to maximize Top-1 performance on ImageNet~\cite{densecl}. However, &#10;some of the learned invariances in classification (e.g. to translation) might not be desirable in localization tasks, and thus the representation might discard critical information for object detection. Moreover, it has been shown that higher ImageNet Top-1 accuracy does not necessarily guarantee higher object detection performance~\cite{densecl}.&#10;&#10;In response to such shortcomings, there has been a growing number of methods for self-supervised object detection. These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods.&#10;In particular, the current state-of-the-art for FSOD on MS COCO is a method which does self-supervised pretraining of both the backbone and the object detector~\cite{detreg}.&#10;&#10;Thus, this motivates a survey combining the most recent approaches on few-shot and self-supervised object detection, both of which having not been surveyed before (see Section~\ref{sec:related-surveys}). In the following sections, we briefly summarize key object detection concepts (Section~\ref{sec:background}). Then we review the few-shot object detection task and benchmarks (Section~\ref{sec:fsod}) and we discuss the most recent developments in few-shot object detection (Section~\ref{sec:fsod}) and self-supervised object detection pretraining (Section~\ref{sec:ss}). We conclude this survey by summing up the main takeaways, future trends, and related tasks (Sections~\ref{sec:takeaways}~and~\ref{sec:related-approaches}). We provide a taxonomy of popular few-shot and self-supervised object detection methods in Figure~\ref{fig:taxonomy}, on the base of which this survey is structured.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.00343" label="2401.00343">
-        <attvalues>
-          <attvalue for="0" value="SHARE: Single-view Human Adversarial REconstruction" />
-          <attvalue for="1" value="  The accuracy of 3D Human Pose and Shape reconstruction (HPS) from an image is&#10;progressively improving. Yet, no known method is robust across all image&#10;distortion. To address issues due to variations of camera poses, we introduce&#10;SHARE, a novel fine-tuning method that utilizes adversarial data augmentation&#10;to enhance the robustness of existing HPS techniques. We perform a&#10;comprehensive analysis on the impact of camera poses on HPS reconstruction&#10;outcomes. We first generated large-scale image datasets captured systematically&#10;from diverse camera perspectives. We then established a mapping between camera&#10;poses and reconstruction errors as a continuous function that characterizes the&#10;relationship between camera poses and HPS quality. Leveraging this&#10;representation, we introduce RoME (Regions of Maximal Error), a novel sampling&#10;technique for our adversarial fine-tuning method.&#10;  The SHARE framework is generalizable across various single-view HPS methods&#10;and we demonstrate its performance on HMR, SPIN, PARE, CLIFF and ExPose. Our&#10;results illustrate a reduction in mean joint errors across single-view HPS&#10;techniques, for images captured from multiple camera positions without&#10;compromising their baseline performance. In many challenging cases, our method&#10;surpasses the performance of existing models, highlighting its practical&#10;significance for diverse real-world applications.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The reconstruction of human body pose and shape (HPS) has gained attention from industries like fashion, healthcare, special effects, surveillance, computer animation, and virtual and augmented reality~\cite{Liang01,hu2018personalized,ghadi2021syntactic}. Single-view 3D human pose and shape recovery is of particular interest due to its simplicity and practicality, sparking renewed research efforts to enhance its accuracy and robustness.&#10;&#10;Common issues affecting HPS reconstruction results, such as self-occlusion, low-contrast lighting or poor depth perception are often due to suboptimal camera poses~\cite{Puscas01,kocabas2021spec,pang2022benchmarking}.&#10;Therefore, it is vital to understand how camera poses impact reconstruction quality. Furthermore, in large-scale consumer applications of HPS, such as virtual try-on or healthcare, these effects can significantly influence the user experience.&#10;&#10;In this paper, we study the influence of various camera poses on HPS using images. We propose novel methods to minimize disparities due to such camera pose variations. &#10;Our adversarial fine-tuning method complements numerous pre-trained HPS models and improves their robustness against diverse camera poses not commonly found in their training data.&#10;&#10;The key contributions of this work include:&#10;\begin{enumerate}&#10;\item A framework for the automatic creation of large-scale image datasets for given bodies, camera poses, and scene settings (Sec.~\ref{section:GenSyn}) to be publicly released;&#10;\item A systematic study and analysis on the {impact of camera poses} on the quality of human pose and shape reconstruction (Sec.~\ref{section:AnalysisOfPose});&#10;\item An {adversarial data augmentation} technique for fine-tuning pre-trained HPS models against image variation due to camera poses using {differentiable sampling techniques} (Sec.~\ref{section:AdvDataAug});&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Human Pose Estimation, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1908.05794" label="1908.05794">
-        <attvalues>
-          <attvalue for="0" value="Structured Coupled Generative Adversarial Networks for Unsupervised&#10;  Monocular Depth Estimation" />
-          <attvalue for="1" value="  Inspired by the success of adversarial learning, we propose a new end-to-end&#10;unsupervised deep learning framework for monocular depth estimation consisting&#10;of two Generative Adversarial Networks (GAN), deeply coupled with a structured&#10;Conditional Random Field (CRF) model. The two GANs aim at generating distinct&#10;and complementary disparity maps and at improving the generation quality via&#10;exploiting the adversarial learning strategy. The deep CRF coupling model is&#10;proposed to fuse the generative and discriminative outputs from the dual GAN&#10;nets. As such, the model implicitly constructs mutual constraints on the two&#10;network branches and between the generator and discriminator. This facilitates&#10;the optimization of the whole network for better disparity generation.&#10;Extensive experiments on the KITTI, Cityscapes, and Make3D datasets clearly&#10;demonstrate the effectiveness of the proposed approach and show superior&#10;performance compared to state of the art methods. The code and models are&#10;available at https://github.com/mihaipuscas/ 3dv---coupled-crf-disparity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.00620" label="2110.00620">
-        <attvalues>
-          <attvalue for="0" value="SPEC: Seeing People in the Wild with an Estimated Camera" />
-          <attvalue for="1" value="  Due to the lack of camera parameter information for in-the-wild images,&#10;existing 3D human pose and shape (HPS) estimation methods make several&#10;simplifying assumptions: weak-perspective projection, large constant focal&#10;length, and zero camera rotation. These assumptions often do not hold and we&#10;show, quantitatively and qualitatively, that they cause errors in the&#10;reconstructed 3D shape and pose. To address this, we introduce SPEC, the first&#10;in-the-wild 3D HPS method that estimates the perspective camera from a single&#10;image and employs this to reconstruct 3D human bodies more accurately. First,&#10;we train a neural network to estimate the field of view, camera pitch, and roll&#10;given an input image. We employ novel losses that improve the calibration&#10;accuracy over previous work. We then train a novel network that concatenates&#10;the camera calibration to the image features and uses these together to regress&#10;3D body shape and pose. SPEC is more accurate than the prior art on the&#10;standard benchmark (3DPW) as well as two new datasets with more challenging&#10;camera views and varying focal lengths. Specifically, we create a new&#10;photorealistic synthetic dataset (SPEC-SYN) with ground truth 3D bodies and a&#10;novel in-the-wild dataset (SPEC-MTP) with calibration and high-quality&#10;reference bodies. Both qualitative and quantitative analysis confirm that&#10;knowing camera parameters during inference regresses better human bodies. Code&#10;and datasets are available for research purposes at https://spec.is.tue.mpg.de.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.10529" label="2209.10529">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond&#10;  Algorithms" />
-          <attvalue for="1" value="  3D human pose and shape estimation (a.k.a. &quot;human mesh recovery&quot;) has&#10;achieved substantial progress. Researchers mainly focus on the development of&#10;novel algorithms, while less attention has been paid to other critical factors&#10;involved. This could lead to less optimal baselines, hindering the fair and&#10;faithful evaluations of newly designed methodologies. To address this problem,&#10;this work presents the first comprehensive benchmarking study from three&#10;under-explored perspectives beyond algorithms. 1) Datasets. An analysis on 31&#10;datasets reveals the distinct impacts of data samples: datasets featuring&#10;critical attributes (i.e. diverse poses, shapes, camera characteristics,&#10;backbone features) are more effective. Strategical selection and combination of&#10;high-quality datasets can yield a significant boost to the model performance.&#10;2) Backbones. Experiments with 10 backbones, ranging from CNNs to transformers,&#10;show the knowledge learnt from a proximity task is readily transferable to&#10;human mesh recovery. 3) Training strategies. Proper augmentation techniques and&#10;loss designs are crucial. With the above findings, we achieve a PA-MPJPE of&#10;47.3 mm on the 3DPW test set with a relatively simple model. More importantly,&#10;we provide strong baselines for fair comparisons of algorithms, and&#10;recommendations for building effective training configurations in the future.&#10;Codebase is available at http://github.com/smplbody/hmr-benchmarks&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.01649" label="2205.01649">
-        <attvalues>
-          <attvalue for="0" value="Learning Enriched Features for Fast Image Restoration and Enhancement" />
-          <attvalue for="1" value="  Given a degraded input image, image restoration aims to recover the missing&#10;high-quality image content. Numerous applications demand effective image&#10;restoration, e.g., computational photography, surveillance, autonomous&#10;vehicles, and remote sensing. Significant advances in image restoration have&#10;been made in recent years, dominated by convolutional neural networks (CNNs).&#10;The widely-used CNN-based methods typically operate either on full-resolution&#10;or on progressively low-resolution representations. In the former case, spatial&#10;details are preserved but the contextual information cannot be precisely&#10;encoded. In the latter case, generated outputs are semantically reliable but&#10;spatially less accurate. This paper presents a new architecture with a holistic&#10;goal of maintaining spatially-precise high-resolution representations through&#10;the entire network, and receiving complementary contextual information from the&#10;low-resolution representations. The core of our approach is a multi-scale&#10;residual block containing the following key elements: (a) parallel&#10;multi-resolution convolution streams for extracting multi-scale features, (b)&#10;information exchange across the multi-resolution streams, (c) non-local&#10;attention mechanism for capturing contextual information, and (d) attention&#10;based multi-scale feature aggregation. Our approach learns an enriched set of&#10;features that combines contextual information from multiple scales, while&#10;simultaneously preserving the high-resolution spatial details. Extensive&#10;experiments on six real image benchmark datasets demonstrate that our method,&#10;named as MIRNet-v2 , achieves state-of-the-art results for a variety of image&#10;processing tasks, including defocus deblurring, image denoising,&#10;super-resolution, and image enhancement. The source code and pre-trained models&#10;are available at https://github.com/swz30/MIRNetv2&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Rapidly growing image content necessitates the need to develop effective image restoration and enhancement algorithms. &#10;In this paper, we propose a new method capable of performing dual-pixel defocus deblurring, image denoising, super-resolution, and image enhancement.&#10;{Unlike existing works for these problems, our approach processes features at the original resolution in order to preserve spatial details, while effectively fuses contextual information from multiple parallel branches.} &#10;Next, we briefly describe the representative methods for each of the studied problems. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Convolutional Neural Networks, Mathematics, Computer Science, Image Restoration, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2109.07036" label="2109.07036">
-        <attvalues>
-          <attvalue for="0" value="PnP-DETR: Towards Efficient Visual Analysis with Transformers" />
-          <attvalue for="1" value="  Recently, DETR pioneered the solution of vision tasks with transformers, it&#10;directly translates the image feature map into the object detection result.&#10;Though effective, translating the full feature map can be costly due to&#10;redundant computation on some area like the background. In this work, we&#10;encapsulate the idea of reducing spatial redundancy into a novel poll and pool&#10;(PnP) sampling module, with which we build an end-to-end PnP-DETR architecture&#10;that adaptively allocates its computation spatially to be more efficient.&#10;Concretely, the PnP module abstracts the image feature map into fine foreground&#10;object feature vectors and a small number of coarse background contextual&#10;feature vectors. The transformer models information interaction within the&#10;fine-coarse feature space and translates the features into the detection&#10;result. Moreover, the PnP-augmented model can instantly achieve various desired&#10;trade-offs between performance and computation with a single model by varying&#10;the sampled feature length, without requiring to train multiple models as&#10;existing methods. Thus it offers greater flexibility for deployment in diverse&#10;scenarios with varying computation constraint. We further validate the&#10;generalizability of the PnP module on panoptic segmentation and the recent&#10;transformer-based image recognition model ViT and show consistent efficiency&#10;gain. We believe our method makes a step for efficient visual analysis with&#10;transformers, wherein spatial redundancy is commonly observed. Code will be&#10;available at \url{https://github.com/twangnh/pnp-detr}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Object Detection&#10;&#10;In recent years performance of object detection has been substantially improved~\cite{girshick2014rich,girshick2015fast,ren2015faster,liu2016ssd,lin2017focal,tian2019fcos,wang2019distilling,wang2019few} over traditional approaches~\cite{sung1998example,felzenszwalb2009object}.&#10;Those modern methods mainly address the task with a relaxed learning objective, i.e., learning on a set of matched positive anchor box samples and predicting with post-processing (NMS) to suppress duplicates. The handcraft designs &#10;Recently, \cite{carion2020end} proposed an end-to-end DETR framework that learns an explicit set based objective with transformers~\cite{vaswani2017attention}, showing decent performance compared to previous two-stage methods~\cite{ren2015faster}. &#10;Our work aims at improving efficiency of end-to-end objectors by reducing spatial redundancy. Compared to most recent deformable DETR~\cite{zhu2020deformable} that improves the attention efficiency, we aim to directly compress the feature map, which is from different perspective and could be potentially combined together. For example, by implementing bilinear interpolation kernel in the irregular sampled space~\cite{skorokhodov2020interpolating_git,skorokhodov2020interpolating} to enable the learning of deformable offset prediction.&#10;&#10;Sparse Execution and Sampling&#10;Lots of works explored sparse execution in convolution layers~\cite{figurnov2016perforatedcnns,ren2018sbnet,cao2019seernet,xie2020spatially,figurnov2017spatially,dong2017more,figurnov2017spatially}, saving computation by avoiding convolution operations on some less informative spatial locations. In this work, we are partially inspired by the sparse convolution and explore sparse execution of transformers~\cite{vaswani2017attention} by developing a dynamic image feature sampling method for efficient subsequent processing.&#10;Our work is also related to literature on learning a sampling policy for point cloud understanding tasks~\cite{dovrat2019learning,lang2020samplenet,nezhadarya2020adaptive}.&#10;Different from these works where sampling is achieved by new data point generation, we directly address discrete sampling by using a novel sampling as ranking strategy. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Transformer Models, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science, Efficient Computing" />
-        </attvalues>
-      </node>
-      <node id="2408.05953" label="2408.05953">
-        <attvalues>
-          <attvalue for="0" value="A Simple Task-aware Contrastive Local Descriptor Selection Strategy for&#10;  Few-shot Learning between inter class and intra class" />
-          <attvalue for="1" value="  Few-shot image classification aims to classify novel classes with few labeled&#10;samples. Recent research indicates that deep local descriptors have better&#10;representational capabilities. These studies recognize the impact of background&#10;noise on classification performance. They typically filter query descriptors&#10;using all local descriptors in the support classes or engage in bidirectional&#10;selection between local descriptors in support and query sets. However, they&#10;ignore the fact that background features may be useful for the classification&#10;performance of specific tasks. This paper proposes a novel task-aware&#10;contrastive local descriptor selection network (TCDSNet). First, we calculate&#10;the contrastive discriminative score for each local descriptor in the support&#10;class, and select discriminative local descriptors to form a support descriptor&#10;subset. Finally, we leverage support descriptor subsets to adaptively select&#10;discriminative query descriptors for specific tasks. Extensive experiments&#10;demonstrate that our method outperforms state-of-the-art methods on both&#10;general and fine-grained datasets.&#10;" />
-          <attvalue for="2" value="&#10;The purpose of few-shot learning is to enable models to adapt quickly to new tasks with only a small number of training samples in scenarios where data is scarce. Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}.&#10;&#10;This work is based on a few-shot learning method using local descriptors, falling within the realm of metric learning. Features based on local descriptors exhibit superior representational capabilities compared to image-level features. In previous works, \cite{li2019revisiting} proposed DN4, which directly utilizes all query descriptors. It selects k support descriptors directly for each query local descriptor through k-nearest neighbors (k-NN) and approximates the relationship between query samples and support classes using cosine similarity distances. Based on DN4, \cite{liu2022dmn4} introduced DMN4, which believes that not all query descriptors are task-relevant and contain significant background noise. DMN4 establishes mutual nearest neighbor (MNN) relationships, explicitly selecting query descriptors most relevant to each task, thereby avoiding the impact of background noise on classification performance. Similarly, based on DN4, \cite{dong2021learning} and \cite{yan2023few} proposed ATL-Net and TADNet, respectively. Both methods measure the relationships between each query local descriptor and all support classes, adaptively selecting discriminative query descriptors for classification. \cite{qiao2024talds} introduced TALDS-Net, recognizing background noise in query descriptors. It first adaptively selects optimal descriptor subsets composed of support class local descriptors and then adaptively chooses query descriptors for classification from the optimal descriptor subset. However, all these methods aim to eliminate background noise to prevent its influence on the feature representation of local descriptors, either by filtering query descriptors through all support class local descriptors or by bidirectionally selecting between support class local descriptors and query descriptors. Their goal is to remove background noise.&#10;We observe that from a human cognitive perspective, for example, considering an image of a dog and an image of a dolphin, not only do the target features differ significantly, but the background features also exhibit substantial distinctions (e.g., dolphin backgrounds are unlikely to be grassy, whereas dog backgrounds might include grass). In such cases, background features can contribute to classification. Conversely, for two images both belonging to the dolphin category, the differences in background features might not be as pronounced, and in this scenario, background features can be considered as noise. For instance, when dealing with unfamiliar images, if the background is an ocean, it can help narrow down the classification to objects commonly found only in the ocean. This aids in identifying the target category among familiar images. Thus, background features within the same category might positively impact classification performance. Furthermore, background features between different categories might also contribute to enhancing classification performance. Determining discriminative local descriptors for methods based on local descriptors is a challenging task. Moreover, it is essential to judiciously retain or discard background noise in the process.&#10;&#10;In response to this challenge, a straightforward solution is to select local descriptors in the support class to form a support descriptor subset, and then use the support descriptor subset to select query descriptors. Experimental results have also demonstrated the effectiveness of this simple method.&#10;&#10;The above-described method is our proposed Task-Aware Contrastive Discriminative Local Descriptor Selection Network (TCDSNet). Specifically, we first select local descriptors from the support class. For each support descriptor, we calculate the sum of its similarities with the remaining support descriptors in the same category as the intra-class similarity score. Next, we compute the sum of its similarities with support descriptors from other categories as the inter-class similarity score. A high intra-class similarity score indicates that the support descriptor has strong representational capabilities for that class, while a low inter-class similarity score suggests that the support descriptor has high discriminative power across other classes. We calculate the discriminative score by dividing the intra-class similarity score by the inter-class similarity score, which we term the contrastive discriminative score. Then we select $K$ support descriptors in descending order of their discriminative scores. Finally, we utilize the selected support descriptors to choose query descriptors. For selecting discriminative query descriptors, we employ a simple learnable module to adaptively predict a threshold. Using the learned threshold and a score map, we select the most discriminative descriptors for final classification. This approach enhances the model's classification and generalization capabilities.&#10;&#10;In summary, our main contributions are three folds:&#10;\begin{itemize}&#10;\item We propose a novel method that calculates discriminative scores ($\mathcal{CDS}$) for local descriptors in the support class. This enhances the model's adaptability to different tasks and strengthens the performance of local descriptors in few-shot learning tasks.&#10;&#10;\item We propose a novel Task-Aware Contrastive Discriminative Local Descriptor Selection Network (TCDSNet) that not only selects a subset of support descriptors based on discriminative scores but also incorporates a learnable module for adaptively choosing the discriminative query descriptors. &#10;&#10;\item Extensive experimental results demonstrate that TCDSNet outperforms state-of-the-art methods on multiple general and fine-grained datasets.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Descriptor Selection, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1703.03400" label="1703.03400">
-        <attvalues>
-          <attvalue for="0" value="Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks" />
-          <attvalue for="1" value="  We propose an algorithm for meta-learning that is model-agnostic, in the&#10;sense that it is compatible with any model trained with gradient descent and&#10;applicable to a variety of different learning problems, including&#10;classification, regression, and reinforcement learning. The goal of&#10;meta-learning is to train a model on a variety of learning tasks, such that it&#10;can solve new learning tasks using only a small number of training samples. In&#10;our approach, the parameters of the model are explicitly trained such that a&#10;small number of gradient steps with a small amount of training data from a new&#10;task will produce good generalization performance on that task. In effect, our&#10;method trains the model to be easy to fine-tune. We demonstrate that this&#10;approach leads to state-of-the-art performance on two few-shot image&#10;classification benchmarks, produces good results on few-shot regression, and&#10;accelerates fine-tuning for policy gradient reinforcement learning with neural&#10;network policies.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.09502" label="1810.09502">
-        <attvalues>
-          <attvalue for="0" value="How to train your MAML" />
-          <attvalue for="1" value="  The field of few-shot learning has recently seen substantial advancements.&#10;Most of these advancements came from casting few-shot learning as a&#10;meta-learning problem. Model Agnostic Meta Learning or MAML is currently one of&#10;the best approaches for few-shot learning via meta-learning. MAML is simple,&#10;elegant and very powerful, however, it has a variety of issues, such as being&#10;very sensitive to neural network architectures, often leading to instability&#10;during training, requiring arduous hyperparameter searches to stabilize&#10;training and achieve high generalization and being very computationally&#10;expensive at both training and inference times. In this paper, we propose&#10;various modifications to MAML that not only stabilize the system, but also&#10;substantially improve the generalization performance, convergence speed and&#10;computational overhead of MAML, which we call MAML++.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.05175" label="1703.05175">
-        <attvalues>
-          <attvalue for="0" value="Prototypical Networks for Few-shot Learning" />
-          <attvalue for="1" value="  We propose prototypical networks for the problem of few-shot classification,&#10;where a classifier must generalize to new classes not seen in the training set,&#10;given only a small number of examples of each new class. Prototypical networks&#10;learn a metric space in which classification can be performed by computing&#10;distances to prototype representations of each class. Compared to recent&#10;approaches for few-shot learning, they reflect a simpler inductive bias that is&#10;beneficial in this limited-data regime, and achieve excellent results. We&#10;provide an analysis showing that some simple design decisions can yield&#10;substantial improvements over recent approaches involving complicated&#10;architectural choices and meta-learning. We further extend prototypical&#10;networks to zero-shot learning and achieve state-of-the-art results on the&#10;CU-Birds dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.06025" label="1711.06025">
-        <attvalues>
-          <attvalue for="0" value="Learning to Compare: Relation Network for Few-Shot Learning" />
-          <attvalue for="1" value="  We present a conceptually simple, flexible, and general framework for&#10;few-shot learning, where a classifier must learn to recognise new classes given&#10;only few examples from each. Our method, called the Relation Network (RN), is&#10;trained end-to-end from scratch. During meta-learning, it learns to learn a&#10;deep distance metric to compare a small number of images within episodes, each&#10;of which is designed to simulate the few-shot setting. Once trained, a RN is&#10;able to classify images of new classes by computing relation scores between&#10;query images and the few examples of each new class without further updating&#10;the network. Besides providing improved performance on few-shot learning, our&#10;framework is easily extended to zero-shot learning. Extensive experiments on&#10;five benchmarks demonstrate that our simple approach provides a unified and&#10;effective approach for both of these two tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.04340" label="1711.04340">
-        <attvalues>
-          <attvalue for="0" value="Data Augmentation Generative Adversarial Networks" />
-          <attvalue for="1" value="  Effective training of neural networks requires much data. In the low-data&#10;regime, parameters are underdetermined, and learnt networks generalise poorly.&#10;Data Augmentation alleviates this by using existing data more effectively.&#10;However standard data augmentation produces only limited plausible alternative&#10;data. Given there is potential to generate a much broader set of augmentations,&#10;we design and train a generative model to do data augmentation. The model,&#10;based on image conditional Generative Adversarial Networks, takes data from a&#10;source domain and learns to take any data item and generalise it to generate&#10;other within-class data items. As this generative process does not depend on&#10;the classes themselves, it can be applied to novel unseen classes of data. We&#10;show that a Data Augmentation Generative Adversarial Network (DAGAN) augments&#10;standard vanilla classifiers well. We also show a DAGAN can enhance few-shot&#10;learning systems such as Matching Networks. We demonstrate these approaches on&#10;Omniglot, on EMNIST having learnt the DAGAN on Omniglot, and VGG-Face data. In&#10;our experiments we can see over 13% increase in accuracy in the low-data regime&#10;experiments in Omniglot (from 69% to 82%), EMNIST (73.9% to 76%) and VGG-Face&#10;(4.5% to 12%); in Matching Networks for Omniglot we observe an increase of 0.5%&#10;(from 96.9% to 97.4%) and an increase of 1.8% in EMNIST (from 59.5% to 61.3%).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.04734" label="1806.04734">
-        <attvalues>
-          <attvalue for="0" value="Delta-encoder: an effective sample synthesis method for few-shot object&#10;  recognition" />
-          <attvalue for="1" value="  Learning to classify new categories based on just one or a few examples is a&#10;long-standing challenge in modern computer vision. In this work, we proposes a&#10;simple yet effective method for few-shot (and one-shot) object recognition. Our&#10;approach is based on a modified auto-encoder, denoted Delta-encoder, that&#10;learns to synthesize new samples for an unseen category just by seeing few&#10;examples from it. The synthesized samples are then used to train a classifier.&#10;The proposed approach learns to both extract transferable intra-class&#10;deformations, or &quot;deltas&quot;, between same-class pairs of training examples, and&#10;to apply those deltas to the few provided examples of a novel class (unseen&#10;during training) in order to efficiently synthesize samples from that new&#10;class. The proposed method improves over the state-of-the-art in one-shot&#10;object-recognition and compares favorably in the few-shot case. Upon acceptance&#10;code will be made available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.10132" label="1903.10132">
-        <attvalues>
-          <attvalue for="0" value="f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning" />
-          <attvalue for="1" value="  When labeled training data is scarce, a promising data augmentation approach&#10;is to generate visual features of unknown classes using their attributes. To&#10;learn the class conditional distribution of CNN features, these models rely on&#10;pairs of image features and class attributes. Hence, they can not make use of&#10;the abundance of unlabeled data samples. In this paper, we tackle any-shot&#10;learning problems i.e. zero-shot and few-shot, in a unified feature generating&#10;framework that operates in both inductive and transductive learning settings.&#10;We develop a conditional generative model that combines the strength of VAE and&#10;GANs and in addition, via an unconditional discriminator, learns the marginal&#10;feature distribution of unlabeled images. We empirically show that our model&#10;learns highly discriminative CNN features for five datasets, i.e. CUB, SUN, AWA&#10;and ImageNet, and establish a new state-of-the-art in any-shot learning, i.e.&#10;inductive and transductive (generalized) zero- and few-shot learning settings.&#10;We also demonstrate that our learned features are interpretable: we visualize&#10;them by inverting them back to the pixel space and we explain them by&#10;generating textual arguments of why they are associated with a certain label.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.15636" label="2401.15636">
-        <attvalues>
-          <attvalue for="0" value="FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion&#10;  Models" />
-          <attvalue for="1" value="  The rapid development of generative diffusion models has significantly&#10;advanced the field of style transfer. However, most current style transfer&#10;methods based on diffusion models typically involve a slow iterative&#10;optimization process, e.g., model fine-tuning and textual inversion of style&#10;concept. In this paper, we introduce FreeStyle, an innovative style transfer&#10;method built upon a pre-trained large diffusion model, requiring no further&#10;optimization. Besides, our method enables style transfer only through a text&#10;description of the desired style, eliminating the necessity of style images.&#10;Specifically, we propose a dual-stream encoder and single-stream decoder&#10;architecture, replacing the conventional U-Net in diffusion models. In the&#10;dual-stream encoder, two distinct branches take the content image and style&#10;text prompt as inputs, achieving content and style decoupling. In the decoder,&#10;we further modulate features from the dual streams based on a given content&#10;image and the corresponding style text prompt for precise style transfer. Our&#10;experimental results demonstrate high-quality synthesis and fidelity of our&#10;method across various content images and style text prompts. Compared with&#10;state-of-the-art methods that require training, our FreeStyle approach notably&#10;reduces the computational burden by thousands of iterations, while achieving&#10;comparable or superior performance across multiple evaluation metrics including&#10;CLIP Aesthetic Score, CLIP Score, and Preference. We have released the code&#10;anonymously at:&#10;\href{https://anonymous.4open.science/r/FreeStyleAnonymous-0F9B}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.12290" label="1903.12290">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Local Descriptor based Image-to-Class Measure for Few-shot&#10;  Learning" />
-          <attvalue for="1" value="  Few-shot learning in image classification aims to learn a classifier to&#10;classify images when only few training examples are available for each class.&#10;Recent work has achieved promising classification performance, where an&#10;image-level feature based measure is usually used. In this paper, we argue that&#10;a measure at such a level may not be effective enough in light of the scarcity&#10;of examples in few-shot learning. Instead, we think a local descriptor based&#10;image-to-class measure should be taken, inspired by its surprising success in&#10;the heydays of local invariant features. Specifically, building upon the recent&#10;episodic training mechanism, we propose a Deep Nearest Neighbor Neural Network&#10;(DN4 in short) and train it in an end-to-end manner. Its key difference from&#10;the literature is the replacement of the image-level feature based measure in&#10;the final layer by a local descriptor based image-to-class measure. This&#10;measure is conducted online via a $k$-nearest neighbor search over the deep&#10;local descriptors of convolutional feature maps. The proposed DN4 not only&#10;learns the optimal deep local descriptors for the image-to-class measure, but&#10;also utilizes the higher efficiency of such a measure in the case of example&#10;scarcity, thanks to the exchangeability of visual patterns across the images in&#10;the same class. Our work leads to a simple, effective, and computationally&#10;efficient framework for few-shot learning. Experimental study on benchmark&#10;datasets consistently shows its superiority over the related state-of-the-art,&#10;with the largest absolute improvement of $17\%$ over the next best. The source&#10;code can be available from \UrlFont{https://github.com/WenbinLee/DN4.git}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.08160" label="2103.08160">
-        <attvalues>
-          <attvalue for="0" value="DMN4: Few-shot Learning via Discriminative Mutual Nearest Neighbor&#10;  Neural Network" />
-          <attvalue for="1" value="  Few-shot learning (FSL) aims to classify images under low-data regimes, where&#10;the conventional pooled global feature is likely to lose useful local&#10;characteristics. Recent work has achieved promising performances by using deep&#10;descriptors. They generally take all deep descriptors from neural networks into&#10;consideration while ignoring that some of them are useless in classification&#10;due to their limited receptive field, e.g., task-irrelevant descriptors could&#10;be misleading and multiple aggregative descriptors from background clutter&#10;could even overwhelm the object's presence. In this paper, we argue that a&#10;Mutual Nearest Neighbor (MNN) relation should be established to explicitly&#10;select the query descriptors that are most relevant to each task and discard&#10;less relevant ones from aggregative clutters in FSL. Specifically, we propose&#10;Discriminative Mutual Nearest Neighbor Neural Network (DMN4) for FSL. Extensive&#10;experiments demonstrate that our method outperforms the existing&#10;state-of-the-arts on both fine-grained and generalized datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.05449" label="2312.05449">
-        <attvalues>
-          <attvalue for="0" value="TALDS-Net: Task-Aware Adaptive Local Descriptors Selection for Few-shot&#10;  Image Classification" />
-          <attvalue for="1" value="  Few-shot image classification aims to classify images from unseen novel&#10;classes with few samples. Recent works demonstrate that deep local descriptors&#10;exhibit enhanced representational capabilities compared to image-level&#10;features. However, most existing methods solely rely on either employing all&#10;local descriptors or directly utilizing partial descriptors, potentially&#10;resulting in the loss of crucial information. Moreover, these methods primarily&#10;emphasize the selection of query descriptors while overlooking support&#10;descriptors. In this paper, we propose a novel Task-Aware Adaptive Local&#10;Descriptors Selection Network (TALDS-Net), which exhibits the capacity for&#10;adaptive selection of task-aware support descriptors and query descriptors.&#10;Specifically, we compare the similarity of each local support descriptor with&#10;other local support descriptors to obtain the optimal support descriptor subset&#10;and then compare the query descriptors with the optimal support subset to&#10;obtain discriminative query descriptors. Extensive experiments demonstrate that&#10;our TALDS-Net outperforms state-of-the-art methods on both general and&#10;fine-grained datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.09388" label="2308.09388">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Models for Image Restoration and Enhancement -- A&#10;  Comprehensive Survey" />
-          <attvalue for="1" value="  Image restoration (IR) has been an indispensable and challenging task in the&#10;low-level vision field, which strives to improve the subjective quality of&#10;images distorted by various forms of degradation. Recently, the diffusion model&#10;has achieved significant advancements in the visual generation of AIGC, thereby&#10;raising an intuitive question, &quot;whether diffusion model can boost image&#10;restoration&quot;. To answer this, some pioneering studies attempt to integrate&#10;diffusion models into the image restoration task, resulting in superior&#10;performances than previous GAN-based methods. Despite that, a comprehensive and&#10;enlightening survey on diffusion model-based image restoration remains scarce.&#10;In this paper, we are the first to present a comprehensive review of recent&#10;diffusion model-based methods on image restoration, encompassing the learning&#10;paradigm, conditional strategy, framework design, modeling strategy, and&#10;evaluation. Concretely, we first introduce the background of the diffusion&#10;model briefly and then present two prevalent workflows that exploit diffusion&#10;models in image restoration. Subsequently, we classify and emphasize the&#10;innovative designs using diffusion models for both IR and blind/real-world IR,&#10;intending to inspire future development. To evaluate existing methods&#10;thoroughly, we summarize the commonly-used dataset, implementation details, and&#10;evaluation metrics. Additionally, we present the objective comparison for&#10;open-sourced methods across three tasks, including image super-resolution,&#10;deblurring, and inpainting. Ultimately, informed by the limitations in existing&#10;works, we propose five potential and challenging directions for the future&#10;research of diffusion model-based IR, including sampling efficiency, model&#10;compression, distortion simulation and estimation, distortion invariant&#10;learning, and framework design.&#10;" />
-          <attvalue for="2" value="&#10;&#10;I{mage} Restoration (IR) has been a long-term research topic in low-level vision tasks, which plays an irreplaceable role in improving the subjective quality of images. The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc. The visual illustration of some IR tasks are shown in Fig.~\ref{fig:restoration_tasks}. To restore the distorted images, traditional IR methods treated the restoration as the signal processing and reduce the artifacts with hand-crafted algorithms from the spatial or frequency perspective~\cite{dabov2007image-traditional1,traditional3,mairal2008learning-traditional-4,xu2010image-traditional-5,kim2010single-traditional-6,dong2012nonlocally-traditional-7,schmidt2014shrinkage-traditional-8}. With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc. Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}. Although these works achieve superior progress in the objective quality (\egno, PSNR, and SSIM), the restored images still suffer from unsatisfied texture generation, hindering the application of IR methods in real-world scenarios. &#10;&#10;Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability. For instance, SRGAN~\cite{SRGAN} and DeblurGAN~\cite{kupyn2018deblurgan-old2} utilize a combination of pixel-wise loss and adversarial loss to achieve perception-oriented SR network and deblurring network, respectively. Following them, two primary directions to improve GAN-based IR arise from enhancing the generator (\ieno, the restoration network)~\cite{SRGAN,zhang2019ranksrgan-gene1,wang2018esrgan,wang2021realesrgan,zhang2021designing-gene2} and discriminator~\cite{schonfeld2020u-pixel-discri,isola2017image-patch-wise-discri,simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2}. In particular, ESRGAN~\cite{wang2018esrgan} introduces the powerful RRDB~\cite{wang2018esrgan} as a generator for GAN-based SR tasks. Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global). Although the above progresses, most researches on GAN-based IR still meet two inevitable but crucial problems: 1) The training of GAN-based IR is susceptible to mode corruption and unstable optimization and 2) The textures of most generated images seem to be fake and counterfactual.&#10;&#10;Recently, diffusion models have emerged as a new branch of generative models, leading to a series of breakthroughs in visual generation tasks. The prototype of the diffusion model can be traced back to the work~\cite{sohl2015deep-dpm}, and has been developed by DDPM~\cite{DDPM}, NCSN~\cite{NCSN} and SDE~\cite{SDE}. In general, the diffusion model is composed of the forward/diffusion process and reverse process, where the forward process progressively increases the pixel-wise noise to the image until it satisfies the Gaussian noise, and the reverse process aims to reconstruct the image by denoising with score estimating~\cite{NCSN} or noise prediction~\cite{DDPM}. Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}. With the advancement of the vision-language model, diffusion model has been extended to the cross-modality generation, such as StableDiffusion~\cite{LDM}, and DALLE-2~\cite{ramesh2022hierarchical-dalle2}. This greatly promotes the development of artificial intelligence-generated content (AIGC). We have listed the representative works on diffusion models based on the timeline in Fig.~\ref{fig:timeline}. &#10;&#10;Inspired by the superior generative capability of diffusion models, numerous studies have investigated their application in image restoration tasks, targeting to facilitate the texture recovery. According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR. Typically, supervised learning-based methods necessitate the collection of large-scale distorted/clean image pairs, while zero-shot-based methods predominantly rely on known degradation modes. These limitations impede the application of these diffusion model-based methods in real-world scenarios, where the distortions are typically diverse and unknown. To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning. &#10;&#10;Although the diffusion models have shown significant efficacy in image restoration, the associated techniques and benchmarks exhibit considerable diversity and complexity, making them hard to be followed and improved. Moreover, the absence of a comprehensive review for diffusion model-based IR further limits its development. In this paper, we are the first to review and summarize the works on diffusion model-based image restoration methods, aiming to provide a well-structured and in-depth knowledge base and facilitate its evolution within the image restoration community.&#10;&#10;In this survey, we start by presenting the background of diffusion models in Sec.~\ref{sec:background}, highlighting three foundational modeling methods, \ieno, NCSN~\cite{NCSN}, DDPM~\cite{DDPM}, and SDE~\cite{SDE}, along with the further improvements on diffusion models from the perspectives of optimization strategy, sampling efficiency, model architecture, and conditional strategies. Based on these preliminaries, we shed light on the advances of diffusion models in image restoration from two distinct directions in Sec.~\ref{sec:Diffuion-based IR}: 1) supervised diffusion model-based IR, and 2) zero-shot diffusion model-based IR. In Sec.~\ref{sec:blindir}, we summarize the diffusion model-based IR under more practical and challenging scenarios, \ieno, blind/real-world degradation. This intends to further enhance the capability of the diffusion model-based IR methods in fulfilling the demands of practical application. To facilitate a reasonable and exhaustive comparison, in Sec.~\ref{sec:experiments}, we clarify the commonly-used datasets, and experimental settings in different diffusion model-based IR tasks. Furthermore, the comprehensive comparisons between the benchmarks across different tasks are provided. In Sec.~\ref{sec:future}, we delve into the insightful analysis of the primary challenges and potential directions in diffusion model-based IR. The final conclusion for this review is summarized in Sec.~\ref{sec:conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Diffusion Models, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Image Restoration, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1511.04587" label="1511.04587">
-        <attvalues>
-          <attvalue for="0" value="Accurate Image Super-Resolution Using Very Deep Convolutional Networks" />
-          <attvalue for="1" value="  We present a highly accurate single-image super-resolution (SR) method. Our&#10;method uses a very deep convolutional network inspired by VGG-net used for&#10;ImageNet classification \cite{simonyan2015very}. We find increasing our network&#10;depth shows a significant improvement in accuracy. Our final model uses 20&#10;weight layers. By cascading small filters many times in a deep network&#10;structure, contextual information over large image regions is exploited in an&#10;efficient way. With very deep networks, however, convergence speed becomes a&#10;critical issue during training. We propose a simple yet effective training&#10;procedure. We learn residuals only and use extremely high learning rates&#10;($10^4$ times higher than SRCNN \cite{dong2015image}) enabled by adjustable&#10;gradient clipping. Our proposed method performs better than existing methods in&#10;accuracy and visual improvements in our results are easily noticeable.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.02921" label="1707.02921">
-        <attvalues>
-          <attvalue for="0" value="Enhanced Deep Residual Networks for Single Image Super-Resolution" />
-          <attvalue for="1" value="  Recent research on super-resolution has progressed with the development of&#10;deep convolutional neural networks (DCNN). In particular, residual learning&#10;techniques exhibit improved performance. In this paper, we develop an enhanced&#10;deep super-resolution network (EDSR) with performance exceeding those of&#10;current state-of-the-art SR methods. The significant performance improvement of&#10;our model is due to optimization by removing unnecessary modules in&#10;conventional residual networks. The performance is further improved by&#10;expanding the model size while we stabilize the training procedure. We also&#10;propose a new multi-scale deep super-resolution system (MDSR) and training&#10;method, which can reconstruct high-resolution images of different upscaling&#10;factors in a single model. The proposed methods show superior performance over&#10;the state-of-the-art methods on benchmark datasets and prove its excellence by&#10;winning the NTIRE2017 Super-Resolution Challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.04802" label="1609.04802">
-        <attvalues>
-          <attvalue for="0" value="Photo-Realistic Single Image Super-Resolution Using a Generative&#10;  Adversarial Network" />
-          <attvalue for="1" value="  Despite the breakthroughs in accuracy and speed of single image&#10;super-resolution using faster and deeper convolutional neural networks, one&#10;central problem remains largely unsolved: how do we recover the finer texture&#10;details when we super-resolve at large upscaling factors? The behavior of&#10;optimization-based super-resolution methods is principally driven by the choice&#10;of the objective function. Recent work has largely focused on minimizing the&#10;mean squared reconstruction error. The resulting estimates have high peak&#10;signal-to-noise ratios, but they are often lacking high-frequency details and&#10;are perceptually unsatisfying in the sense that they fail to match the fidelity&#10;expected at the higher resolution. In this paper, we present SRGAN, a&#10;generative adversarial network (GAN) for image super-resolution (SR). To our&#10;knowledge, it is the first framework capable of inferring photo-realistic&#10;natural images for 4x upscaling factors. To achieve this, we propose a&#10;perceptual loss function which consists of an adversarial loss and a content&#10;loss. The adversarial loss pushes our solution to the natural image manifold&#10;using a discriminator network that is trained to differentiate between the&#10;super-resolved images and original photo-realistic images. In addition, we use&#10;a content loss motivated by perceptual similarity instead of similarity in&#10;pixel space. Our deep residual network is able to recover photo-realistic&#10;textures from heavily downsampled images on public benchmarks. An extensive&#10;mean-opinion-score (MOS) test shows hugely significant gains in perceptual&#10;quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of&#10;the original high-resolution images than to those obtained with any&#10;state-of-the-art method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.00219" label="1809.00219">
-        <attvalues>
-          <attvalue for="0" value="ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks" />
-          <attvalue for="1" value="  The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work&#10;that is capable of generating realistic textures during single image&#10;super-resolution. However, the hallucinated details are often accompanied with&#10;unpleasant artifacts. To further enhance the visual quality, we thoroughly&#10;study three key components of SRGAN - network architecture, adversarial loss&#10;and perceptual loss, and improve each of them to derive an Enhanced SRGAN&#10;(ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block&#10;(RRDB) without batch normalization as the basic network building unit.&#10;Moreover, we borrow the idea from relativistic GAN to let the discriminator&#10;predict relative realness instead of the absolute value. Finally, we improve&#10;the perceptual loss by using the features before activation, which could&#10;provide stronger supervision for brightness consistency and texture recovery.&#10;Benefiting from these improvements, the proposed ESRGAN achieves consistently&#10;better visual quality with more realistic and natural textures than SRGAN and&#10;won the first place in the PIRM2018-SR Challenge. The code is available at&#10;https://github.com/xinntao/ESRGAN .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.11084" label="2108.11084">
-        <attvalues>
-          <attvalue for="0" value="Transformer for Single Image Super-Resolution" />
-          <attvalue for="1" value="  Single image super-resolution (SISR) has witnessed great strides with the&#10;development of deep learning. However, most existing studies focus on building&#10;more complex networks with a massive number of layers. Recently, more and more&#10;researchers start to explore the application of Transformer in computer vision&#10;tasks. However, the heavy computational cost and high GPU memory occupation of&#10;the vision Transformer cannot be ignored. In this paper, we propose a novel&#10;Efficient Super-Resolution Transformer (ESRT) for SISR. ESRT is a hybrid model,&#10;which consists of a Lightweight CNN Backbone (LCB) and a Lightweight&#10;Transformer Backbone (LTB). Among them, LCB can dynamically adjust the size of&#10;the feature map to extract deep features with a low computational cost. LTB is&#10;composed of a series of Efficient Transformers (ET), which occupies a small GPU&#10;memory occupation, thanks to the specially designed Efficient Multi-Head&#10;Attention (EMHA). Extensive experiments show that ESRT achieves competitive&#10;results with low computational costs. Compared with the original Transformer&#10;which occupies 16,057M GPU memory, ESRT only occupies 4,191M GPU memory. All&#10;codes are available at https://github.com/luissen/ESRT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.11247" label="2208.11247">
-        <attvalues>
-          <attvalue for="0" value="SwinFIR: Revisiting the SwinIR with Fast Fourier Convolution and&#10;  Improved Training for Image Super-Resolution" />
-          <attvalue for="1" value="  Transformer-based methods have achieved impressive image restoration&#10;performance due to their capacities to model long-range dependency compared to&#10;CNN-based methods. However, advances like SwinIR adopts the window-based and&#10;local attention strategy to balance the performance and computational overhead,&#10;which restricts employing large receptive fields to capture global information&#10;and establish long dependencies in the early layers. To further improve the&#10;efficiency of capturing global information, in this work, we propose SwinFIR to&#10;extend SwinIR by replacing Fast Fourier Convolution (FFC) components, which&#10;have the image-wide receptive field. We also revisit other advanced techniques,&#10;i.e, data augmentation, pre-training, and feature ensemble to improve the&#10;effect of image reconstruction. And our feature ensemble method enables the&#10;performance of the model to be considerably enhanced without increasing the&#10;training and testing time. We applied our algorithm on multiple popular&#10;large-scale benchmarks and achieved state-of-the-art performance comparing to&#10;the existing methods. For example, our SwinFIR achieves the PSNR of 32.83 dB on&#10;Manga109 dataset, which is 0.8 dB higher than the state-of-the-art SwinIR&#10;method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.02177" label="1612.02177">
-        <attvalues>
-          <attvalue for="0" value="Deep Multi-scale Convolutional Neural Network for Dynamic Scene&#10;  Deblurring" />
-          <attvalue for="1" value="  Non-uniform blind deblurring for general dynamic scenes is a challenging&#10;computer vision problem as blurs arise not only from multiple object motions&#10;but also from camera shake, scene depth variation. To remove these complicated&#10;motion blurs, conventional energy optimization based methods rely on simple&#10;assumptions such that blur kernel is partially uniform or locally linear.&#10;Moreover, recent machine learning based methods also depend on synthetic blur&#10;datasets generated under these assumptions. This makes conventional deblurring&#10;methods fail to remove blurs where blur kernel is difficult to approximate or&#10;parameterize (e.g. object motion boundaries). In this work, we propose a&#10;multi-scale convolutional neural network that restores sharp images in an&#10;end-to-end manner where blur is caused by various sources. Together, we present&#10;multi-scale loss function that mimics conventional coarse-to-fine approaches.&#10;Furthermore, we propose a new large-scale dataset that provides pairs of&#10;realistic blurry image and the corresponding ground truth sharp image that are&#10;obtained by a high-speed camera. With the proposed model trained on this&#10;dataset, we demonstrate empirically that our method achieves the&#10;state-of-the-art performance in dynamic scene deblurring not only&#10;qualitatively, but also quantitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07064" label="1711.07064">
-        <attvalues>
-          <attvalue for="0" value="DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial&#10;  Networks" />
-          <attvalue for="1" value="  We present DeblurGAN, an end-to-end learned method for motion deblurring. The&#10;learning is based on a conditional GAN and the content loss . DeblurGAN&#10;achieves state-of-the art performance both in the structural similarity measure&#10;and visual appearance. The quality of the deblurring model is also evaluated in&#10;a novel way on a real-world problem -- object detection on (de-)blurred images.&#10;The method is 5 times faster than the closest competitor -- DeepDeblur. We also&#10;introduce a novel method for generating synthetic motion blurred images from&#10;sharp ones, allowing realistic dataset augmentation.&#10;  The model, code and the dataset are available at&#10;https://github.com/KupynOrest/DeblurGAN&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.07820" label="2204.07820">
-        <attvalues>
-          <attvalue for="0" value="FCL-GAN: A Lightweight and Real-Time Baseline for Unsupervised Blind&#10;  Image Deblurring" />
-          <attvalue for="1" value="  Blind image deblurring (BID) remains a challenging and significant task.&#10;Benefiting from the strong fitting ability of deep learning, paired data-driven&#10;supervised BID method has obtained great progress. However, paired data are&#10;usually synthesized by hand, and the realistic blurs are more complex than&#10;synthetic ones, which makes the supervised methods inept at modeling realistic&#10;blurs and hinders their real-world applications. As such, unsupervised deep BID&#10;method without paired data offers certain advantages, but current methods still&#10;suffer from some drawbacks, e.g., bulky model size, long inference time, and&#10;strict image resolution and domain requirements. In this paper, we propose a&#10;lightweight and real-time unsupervised BID baseline, termed Frequency-domain&#10;Contrastive Loss Constrained Lightweight CycleGAN (shortly, FCL-GAN), with&#10;attractive properties, i.e., no image domain limitation, no image resolution&#10;limitation, 25x lighter than SOTA, and 5x faster than SOTA. To guarantee the&#10;lightweight property and performance superiority, two new collaboration units&#10;called lightweight domain conversion unit(LDCU) and parameter-free&#10;frequency-domain contrastive unit(PFCU) are designed. LDCU mainly implements&#10;inter-domain conversion in lightweight manner. PFCU further explores the&#10;similarity measure, external difference and internal connection between the&#10;blurred domain and sharp domain images in frequency domain, without involving&#10;extra parameters. Extensive experiments on several image datasets demonstrate&#10;the effectiveness of our FCL-GAN in terms of performance, model size and&#10;reference time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.09652" label="2202.09652">
-        <attvalues>
-          <attvalue for="0" value="MSSNet: Multi-Scale-Stage Network for Single Image Deblurring" />
-          <attvalue for="1" value="  Most of traditional single image deblurring methods before deep learning&#10;adopt a coarse-to-fine scheme that estimates a sharp image at a coarse scale&#10;and progressively refines it at finer scales. While this scheme has also been&#10;adopted to several deep learning-based approaches, recently a number of&#10;single-scale approaches have been introduced showing superior performance to&#10;previous coarse-to-fine approaches both in quality and computation time. In&#10;this paper, we revisit the coarse-to-fine scheme, and analyze defects of&#10;previous coarse-to-fine approaches that degrade their performance. Based on the&#10;analysis, we propose Multi-Scale-Stage Network (MSSNet), a novel deep&#10;learning-based approach to single image deblurring that adopts our remedies to&#10;the defects. Specifically, MSSNet adopts three novel technical components:&#10;stage configuration reflecting blur scales, an inter-scale information&#10;propagation scheme, and a pixel-shuffle-based multi-scale scheme. Our&#10;experiments show that MSSNet achieves the state-of-the-art performance in terms&#10;of quality, network size, and computation time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09881" label="2111.09881">
-        <attvalues>
-          <attvalue for="0" value="Restormer: Efficient Transformer for High-Resolution Image Restoration" />
-          <attvalue for="1" value="  Since convolutional neural networks (CNNs) perform well at learning&#10;generalizable image priors from large-scale data, these models have been&#10;extensively applied to image restoration and related tasks. Recently, another&#10;class of neural architectures, Transformers, have shown significant performance&#10;gains on natural language and high-level vision tasks. While the Transformer&#10;model mitigates the shortcomings of CNNs (i.e., limited receptive field and&#10;inadaptability to input content), its computational complexity grows&#10;quadratically with the spatial resolution, therefore making it infeasible to&#10;apply to most image restoration tasks involving high-resolution images. In this&#10;work, we propose an efficient Transformer model by making several key designs&#10;in the building blocks (multi-head attention and feed-forward network) such&#10;that it can capture long-range pixel interactions, while still remaining&#10;applicable to large images. Our model, named Restoration Transformer&#10;(Restormer), achieves state-of-the-art results on several image restoration&#10;tasks, including image deraining, single-image motion deblurring, defocus&#10;deblurring (single-image and dual-pixel data), and image denoising (Gaussian&#10;grayscale/color denoising, and real image denoising). The source code and&#10;pre-trained models are available at https://github.com/swz30/Restormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.04627" label="2204.04627">
-        <attvalues>
-          <attvalue for="0" value="Stripformer: Strip Transformer for Fast Image Deblurring" />
-          <attvalue for="1" value="  Images taken in dynamic scenes may contain unwanted motion blur, which&#10;significantly degrades visual quality. Such blur causes short- and long-range&#10;region-specific smoothing artifacts that are often directional and non-uniform,&#10;which is difficult to be removed. Inspired by the current success of&#10;transformers on computer vision and image processing tasks, we develop,&#10;Stripformer, a transformer-based architecture that constructs intra- and&#10;inter-strip tokens to reweight image features in the horizontal and vertical&#10;directions to catch blurred patterns with different orientations. It stacks&#10;interlaced intra-strip and inter-strip attention layers to reveal blur&#10;magnitudes. In addition to detecting region-specific blurred patterns of&#10;various orientations and magnitudes, Stripformer is also a token-efficient and&#10;parameter-efficient transformer model, demanding much less memory usage and&#10;computation cost than the vanilla transformer but works better without relying&#10;on tremendous training data. Experimental results show that Stripformer&#10;performs favorably against state-of-the-art models in dynamic scene deblurring.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.03981" label="1608.03981">
-        <attvalues>
-          <attvalue for="0" value="Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image&#10;  Denoising" />
-          <attvalue for="1" value="  Discriminative model learning for image denoising has been recently&#10;attracting considerable attentions due to its favorable denoising performance.&#10;In this paper, we take one step forward by investigating the construction of&#10;feed-forward denoising convolutional neural networks (DnCNNs) to embrace the&#10;progress in very deep architecture, learning algorithm, and regularization&#10;method into image denoising. Specifically, residual learning and batch&#10;normalization are utilized to speed up the training process as well as boost&#10;the denoising performance. Different from the existing discriminative denoising&#10;models which usually train a specific model for additive white Gaussian noise&#10;(AWGN) at a certain noise level, our DnCNN model is able to handle Gaussian&#10;denoising with unknown noise level (i.e., blind Gaussian denoising). With the&#10;residual learning strategy, DnCNN implicitly removes the latent clean image in&#10;the hidden layers. This property motivates us to train a single DnCNN model to&#10;tackle with several general image denoising tasks such as Gaussian denoising,&#10;single image super-resolution and JPEG image deblocking. Our extensive&#10;experiments demonstrate that our DnCNN model can not only exhibit high&#10;effectiveness in several general image denoising tasks, but also be efficiently&#10;implemented by benefiting from GPU computing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1508.02848" label="1508.02848">
-        <attvalues>
-          <attvalue for="0" value="Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast&#10;  and Effective Image Restoration" />
-          <attvalue for="1" value="  Image restoration is a long-standing problem in low-level computer vision&#10;with many interesting applications. We describe a flexible learning framework&#10;based on the concept of nonlinear reaction diffusion models for various image&#10;restoration problems. By embodying recent improvements in nonlinear diffusion&#10;models, we propose a dynamic nonlinear reaction diffusion model with&#10;time-dependent parameters (\ie, linear filters and influence functions). In&#10;contrast to previous nonlinear diffusion models, all the parameters, including&#10;the filters and the influence functions, are simultaneously learned from&#10;training data through a loss based approach. We call this approach TNRD --&#10;\textit{Trainable Nonlinear Reaction Diffusion}. The TNRD approach is&#10;applicable for a variety of image restoration tasks by incorporating&#10;appropriate reaction force. We demonstrate its capabilities with three&#10;representative applications, Gaussian image denoising, single image super&#10;resolution and JPEG deblocking. Experiments show that our trained nonlinear&#10;diffusion models largely benefit from the training of the parameters and&#10;finally lead to the best reported performance on common test datasets for the&#10;tested applications. Our trained models preserve the structural simplicity of&#10;diffusion models and take only a small number of diffusion steps, thus are&#10;highly efficient. Moreover, they are also well-suited for parallel computation&#10;on GPUs, which makes the inference procedure extremely fast.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.08448" label="1907.08448">
-        <attvalues>
-          <attvalue for="0" value="Deep Graph-Convolutional Image Denoising" />
-          <attvalue for="1" value="  Non-local self-similarity is well-known to be an effective prior for the&#10;image denoising problem. However, little work has been done to incorporate it&#10;in convolutional neural networks, which surpass non-local model-based methods&#10;despite only exploiting local information. In this paper, we propose a novel&#10;end-to-end trainable neural network architecture employing layers based on&#10;graph convolution operations, thereby creating neurons with non-local receptive&#10;fields. The graph convolution operation generalizes the classic convolution to&#10;arbitrary graphs. In this work, the graph is dynamically computed from&#10;similarities among the hidden features of the network, so that the powerful&#10;representation learning capabilities of the network are exploited to uncover&#10;self-similar patterns. We introduce a lightweight Edge-Conditioned Convolution&#10;which addresses vanishing gradient and over-parameterization issues of this&#10;particular graph convolution. Extensive experiments show state-of-the-art&#10;performance with improved qualitative and quantitative results on both&#10;synthetic Gaussian noise and real noise.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.14009" label="2202.14009">
-        <attvalues>
-          <attvalue for="0" value="SUNet: Swin Transformer UNet for Image Denoising" />
-          <attvalue for="1" value="  Image restoration is a challenging ill-posed problem which also has been a&#10;long-standing issue. In the past few years, the convolution neural networks&#10;(CNNs) almost dominated the computer vision and had achieved considerable&#10;success in different levels of vision tasks including image restoration.&#10;However, recently the Swin Transformer-based model also shows impressive&#10;performance, even surpasses the CNN-based methods to become the&#10;state-of-the-art on high-level vision tasks. In this paper, we proposed a&#10;restoration model called SUNet which uses the Swin Transformer layer as our&#10;basic block and then is applied to UNet architecture for image denoising. The&#10;source code and pre-trained models are available at&#10;https://github.com/FanChiMao/SUNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.12394" label="2209.12394">
-        <attvalues>
-          <attvalue for="0" value="Multi-stage image denoising with the wavelet transform" />
-          <attvalue for="1" value="  Deep convolutional neural networks (CNNs) are used for image denoising via&#10;automatically mining accurate structure information. However, most of existing&#10;CNNs depend on enlarging depth of designed networks to obtain better denoising&#10;performance, which may cause training difficulty. In this paper, we propose a&#10;multi-stage image denoising CNN with the wavelet transform (MWDCNN) via three&#10;stages, i.e., a dynamic convolutional block (DCB), two cascaded wavelet&#10;transform and enhancement blocks (WEBs) and a residual block (RB). DCB uses a&#10;dynamic convolution to dynamically adjust parameters of several convolutions&#10;for making a tradeoff between denoising performance and computational costs.&#10;WEB uses a combination of signal processing technique (i.e., wavelet&#10;transformation) and discriminative learning to suppress noise for recovering&#10;more detailed information in image denoising. To further remove redundant&#10;features, RB is used to refine obtained features for improving denoising&#10;effects and reconstruct clean images via improved residual dense architectures.&#10;Experimental results show that the proposed MWDCNN outperforms some popular&#10;denoising methods in terms of quantitative and qualitative analysis. Codes are&#10;available at https://github.com/hellloxiaotian/MWDCNN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.01598" label="2304.01598">
-        <attvalues>
-          <attvalue for="0" value="MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask&#10;  based on Blind-Spot Network" />
-          <attvalue for="1" value="  Recent advances in deep learning have been pushing image denoising techniques&#10;to a new level. In self-supervised image denoising, blind-spot network (BSN) is&#10;one of the most common methods. However, most of the existing BSN algorithms&#10;use a dot-based central mask, which is recognized as inefficient for images&#10;with large-scale spatially correlated noise. In this paper, we give the&#10;definition of large-noise and propose a multi-mask strategy using multiple&#10;convolutional kernels masked in different shapes to further break the noise&#10;spatial correlation. Furthermore, we propose a novel self-supervised image&#10;denoising method that combines the multi-mask strategy with BSN (MM-BSN). We&#10;show that different masks can cause significant performance differences, and&#10;the proposed MM-BSN can efficiently fuse the features extracted by multi-masked&#10;layers, while recovering the texture structures destroyed by multi-masking and&#10;information transmission. Our MM-BSN can be used to address the problem of&#10;large-noise denoising, which cannot be efficiently handled by other BSN&#10;methods. Extensive experiments on public real-world datasets demonstrate that&#10;the proposed MM-BSN achieves state-of-the-art performance among self-supervised&#10;and even unpaired image denoising methods for sRGB images denoising, without&#10;any labelling effort or prior knowledge. Code can be found in&#10;https://github.com/dannie125/MM-BSN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.07379" label="1604.07379">
-        <attvalues>
-          <attvalue for="0" value="Context Encoders: Feature Learning by Inpainting" />
-          <attvalue for="1" value="  We present an unsupervised visual feature learning algorithm driven by&#10;context-based pixel prediction. By analogy with auto-encoders, we propose&#10;Context Encoders -- a convolutional neural network trained to generate the&#10;contents of an arbitrary image region conditioned on its surroundings. In order&#10;to succeed at this task, context encoders need to both understand the content&#10;of the entire image, as well as produce a plausible hypothesis for the missing&#10;part(s). When training context encoders, we have experimented with both a&#10;standard pixel-wise reconstruction loss, as well as a reconstruction plus an&#10;adversarial loss. The latter produces much sharper results because it can&#10;better handle multiple modes in the output. We found that a context encoder&#10;learns a representation that captures not just appearance but also the&#10;semantics of visual structures. We quantitatively demonstrate the effectiveness&#10;of our learned features for CNN pre-training on classification, detection, and&#10;segmentation tasks. Furthermore, context encoders can be used for semantic&#10;inpainting tasks, either stand-alone or as initialization for non-parametric&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.07892" label="1801.07892">
-        <attvalues>
-          <attvalue for="0" value="Generative Image Inpainting with Contextual Attention" />
-          <attvalue for="1" value="  Recent deep learning based approaches have shown promising results for the&#10;challenging task of inpainting large missing regions in an image. These methods&#10;can generate visually plausible image structures and textures, but often create&#10;distorted structures or blurry textures inconsistent with surrounding areas.&#10;This is mainly due to ineffectiveness of convolutional neural networks in&#10;explicitly borrowing or copying information from distant spatial locations. On&#10;the other hand, traditional texture and patch synthesis approaches are&#10;particularly suitable when it needs to borrow textures from the surrounding&#10;regions. Motivated by these observations, we propose a new deep generative&#10;model-based approach which can not only synthesize novel image structures but&#10;also explicitly utilize surrounding image features as references during network&#10;training to make better predictions. The model is a feed-forward, fully&#10;convolutional neural network which can process images with multiple holes at&#10;arbitrary locations and with variable sizes during the test time. Experiments&#10;on multiple datasets including faces (CelebA, CelebA-HQ), textures (DTD) and&#10;natural images (ImageNet, Places2) demonstrate that our proposed approach&#10;generates higher-quality inpainting results than existing ones. Code, demo and&#10;models are available at: https://github.com/JiahuiYu/generative_inpainting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.06790" label="2304.06790">
-        <attvalues>
-          <attvalue for="0" value="Inpaint Anything: Segment Anything Meets Image Inpainting" />
-          <attvalue for="1" value="  Modern image inpainting systems, despite the significant progress, often&#10;struggle with mask selection and holes filling. Based on Segment-Anything Model&#10;(SAM), we make the first attempt to the mask-free image inpainting and propose&#10;a new paradigm of ``clicking and filling'', which is named as Inpaint Anything&#10;(IA). The core idea behind IA is to combine the strengths of different models&#10;in order to build a very powerful and user-friendly pipeline for solving&#10;inpainting-related problems. IA supports three main features: (i) Remove&#10;Anything: users could click on an object and IA will remove it and smooth the&#10;``hole'' with the context; (ii) Fill Anything: after certain objects removal,&#10;users could provide text-based prompts to IA, and then it will fill the hole&#10;with the corresponding generative content via driving AIGC models like Stable&#10;Diffusion; (iii) Replace Anything: with IA, users have another option to retain&#10;the click-selected object and replace the remaining background with the newly&#10;generated scenes. We are also very willing to help everyone share and promote&#10;new projects based on our Inpaint Anything (IA). Our codes are available at&#10;https://github.com/geekyutao/Inpaint-Anything.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.15270" label="2203.15270">
-        <attvalues>
-          <attvalue for="0" value="MAT: Mask-Aware Transformer for Large Hole Image Inpainting" />
-          <attvalue for="1" value="  Recent studies have shown the importance of modeling long-range interactions&#10;in the inpainting problem. To achieve this goal, existing approaches exploit&#10;either standalone attention techniques or transformers, but usually under a low&#10;resolution in consideration of computational cost. In this paper, we present a&#10;novel transformer-based model for large hole inpainting, which unifies the&#10;merits of transformers and convolutions to efficiently process high-resolution&#10;images. We carefully design each component of our framework to guarantee the&#10;high fidelity and diversity of recovered images. Specifically, we customize an&#10;inpainting-oriented transformer block, where the attention module aggregates&#10;non-local information only from partial valid tokens, indicated by a dynamic&#10;mask. Extensive experiments demonstrate the state-of-the-art performance of the&#10;new model on multiple benchmark datasets. Code is released at&#10;https://github.com/fenglinglwb/MAT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1504.06993" label="1504.06993">
-        <attvalues>
-          <attvalue for="0" value="Compression Artifacts Reduction by a Deep Convolutional Network" />
-          <attvalue for="1" value="  Lossy compression introduces complex compression artifacts, particularly the&#10;blocking artifacts, ringing effects and blurring. Existing algorithms either&#10;focus on removing blocking artifacts and produce blurred output, or restores&#10;sharpened images that are accompanied with ringing effects. Inspired by the&#10;deep convolutional networks (DCN) on super-resolution, we formulate a compact&#10;and efficient network for seamless attenuation of different compression&#10;artifacts. We also demonstrate that a deeper model can be effectively trained&#10;with the features learned in a shallow network. Following a similar &quot;easy to&#10;hard&quot; idea, we systematically investigate several practical transfer settings&#10;and show the effectiveness of transfer learning in low-level vision problems.&#10;Our method shows superior performance than the state-of-the-arts both on the&#10;benchmark datasets and the real-world use case (i.e. Twitter). In addition, we&#10;show that our method can be applied as pre-processing to facilitate other&#10;low-level vision routines when they take compressed images as input.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.00366" label="1605.00366">
-        <attvalues>
-          <attvalue for="0" value="Compression Artifacts Removal Using Convolutional Neural Networks" />
-          <attvalue for="1" value="  This paper shows that it is possible to train large and deep convolutional&#10;neural networks (CNN) for JPEG compression artifacts reduction, and that such&#10;networks can provide significantly better reconstruction quality compared to&#10;previously used smaller networks as well as to any other state-of-the-art&#10;methods. We were able to train networks with 8 layers in a single step and in&#10;relatively short time by combining residual learning, skip architecture, and&#10;symmetric weight initialization. We provide further insights into convolution&#10;networks for JPEG artifact reduction by evaluating three different objectives,&#10;generalization with respect to training dataset size, and generalization with&#10;respect to JPEG quality level.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.09320" label="2004.09320">
-        <attvalues>
-          <attvalue for="0" value="Quantization Guided JPEG Artifact Correction" />
-          <attvalue for="1" value="  The JPEG image compression algorithm is the most popular method of image&#10;compression because of its ability for large compression ratios. However, to&#10;achieve such high compression, information is lost. For aggressive quantization&#10;settings, this leads to a noticeable reduction in image quality. Artifact&#10;correction has been studied in the context of deep neural networks for some&#10;time, but the current state-of-the-art methods require a different model to be&#10;trained for each quality setting, greatly limiting their practical application.&#10;We solve this problem by creating a novel architecture which is parameterized&#10;by the JPEG files quantization matrix. This allows our single model to achieve&#10;state-of-the-art performance over models trained for specific quality settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.14573" label="2109.14573">
-        <attvalues>
-          <attvalue for="0" value="Towards Flexible Blind JPEG Artifacts Removal" />
-          <attvalue for="1" value="  Training a single deep blind model to handle different quality factors for&#10;JPEG image artifacts removal has been attracting considerable attention due to&#10;its convenience for practical usage. However, existing deep blind methods&#10;usually directly reconstruct the image without predicting the quality factor,&#10;thus lacking the flexibility to control the output as the non-blind methods. To&#10;remedy this problem, in this paper, we propose a flexible blind convolutional&#10;neural network, namely FBCNN, that can predict the adjustable quality factor to&#10;control the trade-off between artifacts removal and details preservation.&#10;Specifically, FBCNN decouples the quality factor from the JPEG image via a&#10;decoupler module and then embeds the predicted quality factor into the&#10;subsequent reconstructor module through a quality factor attention block for&#10;flexible control. Besides, we find existing methods are prone to fail on&#10;non-aligned double JPEG images even with only a one-pixel shift, and we thus&#10;propose a double JPEG degradation model to augment the training data. Extensive&#10;experiments on single JPEG images, more general double JPEG images, and&#10;real-world JPEG images demonstrate that our proposed FBCNN achieves favorable&#10;performance against state-of-the-art methods in terms of both quantitative&#10;metrics and visual quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.07335" label="2207.07335">
-        <attvalues>
-          <attvalue for="0" value="Learning Parallax Transformer Network for Stereo Image JPEG Artifacts&#10;  Removal" />
-          <attvalue for="1" value="  Under stereo settings, the performance of image JPEG artifacts removal can be&#10;further improved by exploiting the additional information provided by a second&#10;view. However, incorporating this information for stereo image JPEG artifacts&#10;removal is a huge challenge, since the existing compression artifacts make&#10;pixel-level view alignment difficult. In this paper, we propose a novel&#10;parallax transformer network (PTNet) to integrate the information from stereo&#10;image pairs for stereo image JPEG artifacts removal. Specifically, a&#10;well-designed symmetric bi-directional parallax transformer module is proposed&#10;to match features with similar textures between different views instead of&#10;pixel-level view alignment. Due to the issues of occlusions and boundaries, a&#10;confidence-based cross-view fusion module is proposed to achieve better feature&#10;fusion for both views, where the cross-view features are weighted with&#10;confidence maps. Especially, we adopt a coarse-to-fine design for the&#10;cross-view interaction, leading to better performance. Comprehensive&#10;experimental results demonstrate that our PTNet can effectively remove&#10;compression artifacts and achieves superior performance than other testing&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.02760" label="2305.02760">
-        <attvalues>
-          <attvalue for="0" value="Multi-Modality Deep Network for JPEG Artifacts Reduction" />
-          <attvalue for="1" value="  In recent years, many convolutional neural network-based models are designed&#10;for JPEG artifacts reduction, and have achieved notable progress. However, few&#10;methods are suitable for extreme low-bitrate image compression artifacts&#10;reduction. The main challenge is that the highly compressed image loses too&#10;much information, resulting in reconstructing high-quality image difficultly.&#10;To address this issue, we propose a multimodal fusion learning method for&#10;text-guided JPEG artifacts reduction, in which the corresponding text&#10;description not only provides the potential prior information of the highly&#10;compressed image, but also serves as supplementary information to assist in&#10;image deblocking. We fuse image features and text semantic features from the&#10;global and local perspectives respectively, and design a contrastive loss built&#10;upon contrastive learning to produce visually pleasing results. Extensive&#10;experiments, including a user study, prove that our method can obtain better&#10;deblocking results compared to the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.05957" label="1701.05957">
-        <attvalues>
-          <attvalue for="0" value="Image De-raining Using a Conditional Generative Adversarial Network" />
-          <attvalue for="1" value="  Severe weather conditions such as rain and snow adversely affect the visual&#10;quality of images captured under such conditions thus rendering them useless&#10;for further usage and sharing. In addition, such degraded images drastically&#10;affect performance of vision systems. Hence, it is important to solve the&#10;problem of single image de-raining/de-snowing. However, this is a difficult&#10;problem to solve due to its inherent ill-posed nature. Existing approaches&#10;attempt to introduce prior information to convert it into a well-posed problem.&#10;In this paper, we investigate a new point of view in addressing the single&#10;image de-raining problem. Instead of focusing only on deciding what is a good&#10;prior or a good framework to achieve good quantitative and qualitative&#10;performance, we also ensure that the de-rained image itself does not degrade&#10;the performance of a given computer vision algorithm such as detection and&#10;classification. In other words, the de-rained result should be&#10;indistinguishable from its corresponding clear image to a given discriminator.&#10;This criterion can be directly incorporated into the optimization framework by&#10;using the recently introduced conditional generative adversarial networks&#10;(GANs). To minimize artifacts introduced by GANs and ensure better visual&#10;quality, a new refined loss function is introduced. Based on this, we propose a&#10;novel single image de-raining method called Image De-raining Conditional&#10;General Adversarial Network (ID-CGAN), which considers quantitative, visual and&#10;also discriminative performance into the objective function. Experiments&#10;evaluated on synthetic images and real images show that the proposed method&#10;outperforms many recent state-of-the-art single image de-raining methods in&#10;terms of quantitative and visual performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10098" label="1711.10098">
-        <attvalues>
-          <attvalue for="0" value="Attentive Generative Adversarial Network for Raindrop Removal from a&#10;  Single Image" />
-          <attvalue for="1" value="  Raindrops adhered to a glass window or camera lens can severely hamper the&#10;visibility of a background scene and degrade an image considerably. In this&#10;paper, we address the problem by visually removing raindrops, and thus&#10;transforming a raindrop degraded image into a clean one. The problem is&#10;intractable, since first the regions occluded by raindrops are not given.&#10;Second, the information about the background scene of the occluded regions is&#10;completely lost for most part. To resolve the problem, we apply an attentive&#10;generative network using adversarial training. Our main idea is to inject&#10;visual attention into both the generative and discriminative networks. During&#10;the training, our visual attention learns about raindrop regions and their&#10;surroundings. Hence, by injecting this information, the generative network will&#10;pay more attention to the raindrop regions and the surrounding structures, and&#10;the discriminative network will be able to assess the local consistency of the&#10;restored regions. This injection of visual attention to both generative and&#10;discriminative networks is the main contribution of this paper. Our experiments&#10;show the effectiveness of our approach, which outperforms the state of the art&#10;methods quantitatively and qualitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.07412" label="1802.07412">
-        <attvalues>
-          <attvalue for="0" value="Density-aware Single Image De-raining using a Multi-stream Dense Network" />
-          <attvalue for="1" value="  Single image rain streak removal is an extremely challenging problem due to&#10;the presence of non-uniform rain densities in images. We present a novel&#10;density-aware multi-stream densely connected convolutional neural network-based&#10;algorithm, called DID-MDN, for joint rain density estimation and de-raining.&#10;The proposed method enables the network itself to automatically determine the&#10;rain-density information and then efficiently remove the corresponding&#10;rain-streaks guided by the estimated rain-density label. To better characterize&#10;rain-streaks with different scales and shapes, a multi-stream densely connected&#10;de-raining network is proposed which efficiently leverages features from&#10;different scales. Furthermore, a new dataset containing images with&#10;rain-density labels is created and used to train the proposed density-aware&#10;network. Extensive experiments on synthetic and real datasets demonstrate that&#10;the proposed method achieves significant improvements over the recent&#10;state-of-the-art methods. In addition, an ablation study is performed to&#10;demonstrate the improvements obtained by different modules in the proposed&#10;method. Code can be found at: https://github.com/hezhangsprinter&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.03106" label="2106.03106">
-        <attvalues>
-          <attvalue for="0" value="Uformer: A General U-Shaped Transformer for Image Restoration" />
-          <attvalue for="1" value="  In this paper, we present Uformer, an effective and efficient&#10;Transformer-based architecture for image restoration, in which we build a&#10;hierarchical encoder-decoder network using the Transformer block. In Uformer,&#10;there are two core designs. First, we introduce a novel locally-enhanced window&#10;(LeWin) Transformer block, which performs nonoverlapping window-based&#10;self-attention instead of global self-attention. It significantly reduces the&#10;computational complexity on high resolution feature map while capturing local&#10;context. Second, we propose a learnable multi-scale restoration modulator in&#10;the form of a multi-scale spatial bias to adjust features in multiple layers of&#10;the Uformer decoder. Our modulator demonstrates superior capability for&#10;restoring details for various image restoration tasks while introducing&#10;marginal extra parameters and computational cost. Powered by these two designs,&#10;Uformer enjoys a high capability for capturing both local and global&#10;dependencies for image restoration. To evaluate our approach, extensive&#10;experiments are conducted on several image restoration tasks, including image&#10;denoising, motion deblurring, defocus deblurring and deraining. Without bells&#10;and whistles, our Uformer achieves superior or comparable performance compared&#10;with the state-of-the-art algorithms. The code and models are available at&#10;https://github.com/ZhendongWang6/Uformer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1601.06759" label="1601.06759">
-        <attvalues>
-          <attvalue for="0" value="Pixel Recurrent Neural Networks" />
-          <attvalue for="1" value="  Modeling the distribution of natural images is a landmark problem in&#10;unsupervised learning. This task requires an image model that is at once&#10;expressive, tractable and scalable. We present a deep neural network that&#10;sequentially predicts the pixels in an image along the two spatial dimensions.&#10;Our method models the discrete probability of the raw pixel values and encodes&#10;the complete set of dependencies in the image. Architectural novelties include&#10;fast two-dimensional recurrent layers and an effective use of residual&#10;connections in deep recurrent networks. We achieve log-likelihood scores on&#10;natural images that are considerably better than the previous state of the art.&#10;Our main results also provide benchmarks on the diverse ImageNet dataset.&#10;Samples generated from the model appear crisp, varied and globally coherent.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.05517" label="1701.05517">
-        <attvalues>
-          <attvalue for="0" value="PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture&#10;  Likelihood and Other Modifications" />
-          <attvalue for="1" value="  PixelCNNs are a recently proposed class of powerful generative models with&#10;tractable likelihood. Here we discuss our implementation of PixelCNNs which we&#10;make available at https://github.com/openai/pixel-cnn. Our implementation&#10;contains a number of modifications to the original model that both simplify its&#10;structure and improve its performance. 1) We use a discretized logistic mixture&#10;likelihood on the pixels, rather than a 256-way softmax, which we find to speed&#10;up training. 2) We condition on whole pixels, rather than R/G/B sub-pixels,&#10;simplifying the model structure. 3) We use downsampling to efficiently capture&#10;structure at multiple resolutions. 4) We introduce additional short-cut&#10;connections to further speed up optimization. 5) We regularize the model using&#10;dropout. Finally, we present state-of-the-art log likelihood results on&#10;CIFAR-10 to demonstrate the usefulness of these modifications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.05770" label="1505.05770">
-        <attvalues>
-          <attvalue for="0" value="Variational Inference with Normalizing Flows" />
-          <attvalue for="1" value="  The choice of approximate posterior distribution is one of the core problems&#10;in variational inference. Most applications of variational inference employ&#10;simple families of posterior approximations in order to allow for efficient&#10;inference, focusing on mean-field or other simple structured approximations.&#10;This restriction has a significant impact on the quality of inferences made&#10;using variational methods. We introduce a new approach for specifying flexible,&#10;arbitrarily complex and scalable approximate posterior distributions. Our&#10;approximations are distributions constructed through a normalizing flow,&#10;whereby a simple initial density is transformed into a more complex one by&#10;applying a sequence of invertible transformations until a desired level of&#10;complexity is attained. We use this view of normalizing flows to develop&#10;categories of finite and infinitesimal flows and provide a unified view of&#10;approaches for constructing rich posterior approximations. We demonstrate that&#10;the theoretical advantages of having posteriors that better match the true&#10;posterior, combined with the scalability of amortized variational approaches,&#10;provides a clear improvement in performance and applicability of variational&#10;inference.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.07057" label="1705.07057">
-        <attvalues>
-          <attvalue for="0" value="Masked Autoregressive Flow for Density Estimation" />
-          <attvalue for="1" value="  Autoregressive models are among the best performing neural density&#10;estimators. We describe an approach for increasing the flexibility of an&#10;autoregressive model, based on modelling the random numbers that the model uses&#10;internally when generating data. By constructing a stack of autoregressive&#10;models, each modelling the random numbers of the next model in the stack, we&#10;obtain a type of normalizing flow suitable for density estimation, which we&#10;call Masked Autoregressive Flow. This type of flow is closely related to&#10;Inverse Autoregressive Flow and is a generalization of Real NVP. Masked&#10;Autoregressive Flow achieves state-of-the-art performance in a range of&#10;general-purpose density estimation tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.07035" label="1710.07035">
-        <attvalues>
-          <attvalue for="0" value="Generative Adversarial Networks: An Overview" />
-          <attvalue for="1" value="  Generative adversarial networks (GANs) provide a way to learn deep&#10;representations without extensively annotated training data. They achieve this&#10;through deriving backpropagation signals through a competitive process&#10;involving a pair of networks. The representations that can be learned by GANs&#10;may be used in a variety of applications, including image synthesis, semantic&#10;image editing, style transfer, image super-resolution and classification. The&#10;aim of this review paper is to provide an overview of GANs for the signal&#10;processing community, drawing on familiar analogies and concepts where&#10;possible. In addition to identifying different methods for training and&#10;constructing GANs, we also point to remaining challenges in their theory and&#10;application.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.10196" label="1710.10196">
-        <attvalues>
-          <attvalue for="0" value="Progressive Growing of GANs for Improved Quality, Stability, and&#10;  Variation" />
-          <attvalue for="1" value="  We describe a new training methodology for generative adversarial networks.&#10;The key idea is to grow both the generator and discriminator progressively:&#10;starting from a low resolution, we add new layers that model increasingly fine&#10;details as training progresses. This both speeds the training up and greatly&#10;stabilizes it, allowing us to produce images of unprecedented quality, e.g.,&#10;CelebA images at 1024^2. We also propose a simple way to increase the variation&#10;in generated images, and achieve a record inception score of 8.80 in&#10;unsupervised CIFAR10. Additionally, we describe several implementation details&#10;that are important for discouraging unhealthy competition between the generator&#10;and discriminator. Finally, we suggest a new metric for evaluating GAN results,&#10;both in terms of image quality and variation. As an additional contribution, we&#10;construct a higher-quality version of the CelebA dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.03126" label="1609.03126">
-        <attvalues>
-          <attvalue for="0" value="Energy-based Generative Adversarial Network" />
-          <attvalue for="1" value="  We introduce the &quot;Energy-based Generative Adversarial Network&quot; model (EBGAN)&#10;which views the discriminator as an energy function that attributes low&#10;energies to the regions near the data manifold and higher energies to other&#10;regions. Similar to the probabilistic GANs, a generator is seen as being&#10;trained to produce contrastive samples with minimal energies, while the&#10;discriminator is trained to assign high energies to these generated samples.&#10;Viewing the discriminator as an energy function allows to use a wide variety of&#10;architectures and loss functionals in addition to the usual binary classifier&#10;with logistic output. Among them, we show one instantiation of EBGAN framework&#10;as using an auto-encoder architecture, with the energy being the reconstruction&#10;error, in place of the discriminator. We show that this form of EBGAN exhibits&#10;more stable behavior than regular GANs during training. We also show that a&#10;single-scale architecture can be trained to generate high-resolution images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1602.02644" label="1602.02644">
-        <attvalues>
-          <attvalue for="0" value="Generating Images with Perceptual Similarity Metrics based on Deep&#10;  Networks" />
-          <attvalue for="1" value="  Image-generating machine learning models are typically trained with loss&#10;functions based on distance in the image space. This often leads to&#10;over-smoothed results. We propose a class of loss functions, which we call deep&#10;perceptual similarity metrics (DeePSiM), that mitigate this problem. Instead of&#10;computing distances in the image space, we compute distances between image&#10;features extracted by deep neural networks. This metric better reflects&#10;perceptually similarity of images and thus leads to better results. We show&#10;three applications: autoencoder training, a modification of a variational&#10;autoencoder, and inversion of deep convolutional networks. In all cases, the&#10;generated images look sharp and resemble natural images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.08155" label="1603.08155">
-        <attvalues>
-          <attvalue for="0" value="Perceptual Losses for Real-Time Style Transfer and Super-Resolution" />
-          <attvalue for="1" value="  We consider image transformation problems, where an input image is&#10;transformed into an output image. Recent methods for such problems typically&#10;train feed-forward convolutional neural networks using a \emph{per-pixel} loss&#10;between the output and ground-truth images. Parallel work has shown that&#10;high-quality images can be generated by defining and optimizing&#10;\emph{perceptual} loss functions based on high-level features extracted from&#10;pretrained networks. We combine the benefits of both approaches, and propose&#10;the use of perceptual loss functions for training feed-forward networks for&#10;image transformation tasks. We show results on image style transfer, where a&#10;feed-forward network is trained to solve the optimization problem proposed by&#10;Gatys et al in real-time. Compared to the optimization-based method, our&#10;network gives similar qualitative results but is three orders of magnitude&#10;faster. We also experiment with single-image super-resolution, where replacing&#10;a per-pixel loss with a perceptual loss gives visually pleasing results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.05666" label="1511.05666">
-        <attvalues>
-          <attvalue for="0" value="Super-Resolution with Deep Convolutional Sufficient Statistics" />
-          <attvalue for="1" value="  Inverse problems in image and audio, and super-resolution in particular, can&#10;be seen as high-dimensional structured prediction problems, where the goal is&#10;to characterize the conditional distribution of a high-resolution output given&#10;its low-resolution corrupted observation. When the scaling ratio is small,&#10;point estimates achieve impressive performance, but soon they suffer from the&#10;regression-to-the-mean problem, result of their inability to capture the&#10;multi-modality of this conditional distribution. Modeling high-dimensional&#10;image and audio distributions is a hard task, requiring both the ability to&#10;model complex geometrical structures and textured regions. In this paper, we&#10;propose to use as conditional model a Gibbs distribution, where its sufficient&#10;statistics are given by deep convolutional neural networks. The features&#10;computed by the network are stable to local deformation, and have reduced&#10;variance when the input is a stationary texture. These properties imply that&#10;the resulting sufficient statistics minimize the uncertainty of the target&#10;signals given the degraded observations, while being highly informative. The&#10;filters of the CNN are initialized by multiscale complex wavelets, and then we&#10;propose an algorithm to fine-tune them by estimating the gradient of the&#10;conditional log-likelihood, which bears some similarities with Generative&#10;Adversarial Networks. We evaluate experimentally the proposed approach in the&#10;image super-resolution task, but the approach is general and could be used in&#10;other challenging ill-posed problems such as audio bandwidth extension.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.06382" label="1908.06382">
-        <attvalues>
-          <attvalue for="0" value="RankSRGAN: Generative Adversarial Networks with Ranker for Image&#10;  Super-Resolution" />
-          <attvalue for="1" value="  Generative Adversarial Networks (GAN) have demonstrated the potential to&#10;recover realistic details for single image super-resolution (SISR). To further&#10;improve the visual quality of super-resolved results, PIRM2018-SR Challenge&#10;employed perceptual metrics to assess the perceptual quality, such as PI, NIQE,&#10;and Ma. However, existing methods cannot directly optimize these&#10;indifferentiable perceptual metrics, which are shown to be highly correlated&#10;with human ratings. To address the problem, we propose Super-Resolution&#10;Generative Adversarial Networks with Ranker (RankSRGAN) to optimize generator&#10;in the direction of perceptual metrics. Specifically, we first train a Ranker&#10;which can learn the behavior of perceptual metrics and then introduce a novel&#10;rank-content loss to optimize the perceptual quality. The most appealing part&#10;is that the proposed method can combine the strengths of different SR methods&#10;to generate better results. Extensive experiments show that RankSRGAN achieves&#10;visually pleasing results and reaches state-of-the-art performance in&#10;perceptual metrics. Project page:&#10;https://wenlongzhang0724.github.io/Projects/RankSRGAN&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.10833" label="2107.10833">
-        <attvalues>
-          <attvalue for="0" value="Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure&#10;  Synthetic Data" />
-          <attvalue for="1" value="  Though many attempts have been made in blind super-resolution to restore&#10;low-resolution images with unknown and complex degradations, they are still far&#10;from addressing general real-world degraded images. In this work, we extend the&#10;powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN),&#10;which is trained with pure synthetic data. Specifically, a high-order&#10;degradation modeling process is introduced to better simulate complex&#10;real-world degradations. We also consider the common ringing and overshoot&#10;artifacts in the synthesis process. In addition, we employ a U-Net&#10;discriminator with spectral normalization to increase discriminator capability&#10;and stabilize the training dynamics. Extensive comparisons have shown its&#10;superior visual performance than prior works on various real datasets. We also&#10;provide efficient implementations to synthesize training pairs on the fly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.14006" label="2103.14006">
-        <attvalues>
-          <attvalue for="0" value="Designing a Practical Degradation Model for Deep Blind Image&#10;  Super-Resolution" />
-          <attvalue for="1" value="  It is widely acknowledged that single image super-resolution (SISR) methods&#10;would not perform well if the assumed degradation model deviates from those in&#10;real images. Although several degradation models take additional factors into&#10;consideration, such as blur, they are still not effective enough to cover the&#10;diverse degradations of real images. To address this issue, this paper proposes&#10;to design a more complex but practical degradation model that consists of&#10;randomly shuffled blur, downsampling and noise degradations. Specifically, the&#10;blur is approximated by two convolutions with isotropic and anisotropic&#10;Gaussian kernels; the downsampling is randomly chosen from nearest, bilinear&#10;and bicubic interpolations; the noise is synthesized by adding Gaussian noise&#10;with different noise levels, adopting JPEG compression with different quality&#10;factors, and generating processed camera sensor noise via reverse-forward&#10;camera image signal processing (ISP) pipeline model and RAW image noise model.&#10;To verify the effectiveness of the new degradation model, we have trained a&#10;deep blind ESRGAN super-resolver and then applied it to super-resolve both&#10;synthetic and real images with diverse degradations. The experimental results&#10;demonstrate that the new degradation model can help to significantly improve&#10;the practicability of deep super-resolvers, thus providing a powerful&#10;alternative solution for real SISR applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.12655" label="2002.12655">
-        <attvalues>
-          <attvalue for="0" value="A U-Net Based Discriminator for Generative Adversarial Networks" />
-          <attvalue for="1" value="  Among the major remaining challenges for generative adversarial networks&#10;(GANs) is the capacity to synthesize globally and locally coherent images with&#10;object shapes and textures indistinguishable from real images. To target this&#10;issue we propose an alternative U-Net based discriminator architecture,&#10;borrowing the insights from the segmentation literature. The proposed U-Net&#10;based architecture allows to provide detailed per-pixel feedback to the&#10;generator while maintaining the global coherence of synthesized images, by&#10;providing the global image feedback as well. Empowered by the per-pixel&#10;response of the discriminator, we further propose a per-pixel consistency&#10;regularization technique based on the CutMix data augmentation, encouraging the&#10;U-Net discriminator to focus more on semantic and structural changes between&#10;real and fake images. This improves the U-Net discriminator training, further&#10;enhancing the quality of generated samples. The novel discriminator improves&#10;over the state of the art in terms of the standard distribution and image&#10;quality metrics, enabling the generator to synthesize images with varying&#10;structure, appearance and levels of detail, maintaining global and local&#10;realism. Compared to the BigGAN baseline, we achieve an average improvement of&#10;2.7 FID points across FFHQ, CelebA, and the newly introduced COCO-Animals&#10;dataset. The code is available at https://github.com/boschresearch/unetgan.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.04958" label="1912.04958">
-        <attvalues>
-          <attvalue for="0" value="Analyzing and Improving the Image Quality of StyleGAN" />
-          <attvalue for="1" value="  The style-based GAN architecture (StyleGAN) yields state-of-the-art results&#10;in data-driven unconditional generative image modeling. We expose and analyze&#10;several of its characteristic artifacts, and propose changes in both model&#10;architecture and training methods to address them. In particular, we redesign&#10;the generator normalization, revisit progressive growing, and regularize the&#10;generator to encourage good conditioning in the mapping from latent codes to&#10;images. In addition to improving image quality, this path length regularizer&#10;yields the additional benefit that the generator becomes significantly easier&#10;to invert. This makes it possible to reliably attribute a generated image to a&#10;particular network. We furthermore visualize how well the generator utilizes&#10;its output resolution, and identify a capacity problem, motivating us to train&#10;larger models for additional quality improvements. Overall, our improved model&#10;redefines the state of the art in unconditional image modeling, both in terms&#10;of existing distribution quality metrics as well as perceived image quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.04962" label="2203.04962">
-        <attvalues>
-          <attvalue for="0" value="Learning the Degradation Distribution for Blind Image Super-Resolution" />
-          <attvalue for="1" value="  Synthetic high-resolution (HR) \&amp; low-resolution (LR) pairs are widely used&#10;in existing super-resolution (SR) methods. To avoid the domain gap between&#10;synthetic and test images, most previous methods try to adaptively learn the&#10;synthesizing (degrading) process via a deterministic model. However, some&#10;degradations in real scenarios are stochastic and cannot be determined by the&#10;content of the image. These deterministic models may fail to model the random&#10;factors and content-independent parts of degradations, which will limit the&#10;performance of the following SR models. In this paper, we propose a&#10;probabilistic degradation model (PDM), which studies the degradation&#10;$\mathbf{D}$ as a random variable, and learns its distribution by modeling the&#10;mapping from a priori random variable $\mathbf{z}$ to $\mathbf{D}$. Compared&#10;with previous deterministic degradation models, PDM could model more diverse&#10;degradations and generate HR-LR pairs that may better cover the various&#10;degradations of test images, and thus prevent the SR model from over-fitting to&#10;specific ones. Extensive experiments have demonstrated that our degradation&#10;model can help the SR model achieve better performance on different datasets.&#10;The source codes are released at \url{git@github.com:greatlog/UnpairedSR.git}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.03524" label="2205.03524">
-        <attvalues>
-          <attvalue for="0" value="Dual Adversarial Adaptation for Cross-Device Real-World Image&#10;  Super-Resolution" />
-          <attvalue for="1" value="  Due to the sophisticated imaging process, an identical scene captured by&#10;different cameras could exhibit distinct imaging patterns, introducing distinct&#10;proficiency among the super-resolution (SR) models trained on images from&#10;different devices. In this paper, we investigate a novel and practical task&#10;coded cross-device SR, which strives to adapt a real-world SR model trained on&#10;the paired images captured by one camera to low-resolution (LR) images captured&#10;by arbitrary target devices. The proposed task is highly challenging due to the&#10;absence of paired data from various imaging devices. To address this issue, we&#10;propose an unsupervised domain adaptation mechanism for real-world SR, named&#10;Dual ADversarial Adaptation (DADA), which only requires LR images in the target&#10;domain with available real paired data from a source camera. DADA employs the&#10;Domain-Invariant Attention (DIA) module to establish the basis of target model&#10;training even without HR supervision. Furthermore, the dual framework of DADA&#10;facilitates an Inter-domain Adversarial Adaptation (InterAA) in one branch for&#10;two LR input images from two domains, and an Intra-domain Adversarial&#10;Adaptation (IntraAA) in two branches for an LR input image. InterAA and IntraAA&#10;together improve the model transferability from the source domain to the&#10;target. We empirically conduct experiments under six Real to Real adaptation&#10;settings among three different cameras, and achieve superior performance&#10;compared with existing state-of-the-art approaches. We also evaluate the&#10;proposed DADA to address the adaptation to the video camera, which presents a&#10;promising research topic to promote the wide applications of real-world&#10;super-resolution. Our source code is publicly available at&#10;https://github.com/lonelyhope/DADA.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.02900" label="1804.02900">
-        <attvalues>
-          <attvalue for="0" value="A Fully Progressive Approach to Single-Image Super-Resolution" />
-          <attvalue for="1" value="  Recent deep learning approaches to single image super-resolution have&#10;achieved impressive results in terms of traditional error measures and&#10;perceptual quality. However, in each case it remains challenging to achieve&#10;high quality results for large upsampling factors. To this end, we propose a&#10;method (ProSR) that is progressive both in architecture and training: the&#10;network upsamples an image in intermediate steps, while the learning process is&#10;organized from easy to hard, as is done in curriculum learning. To obtain more&#10;photorealistic results, we design a generative adversarial network (GAN), named&#10;ProGanSR, that follows the same progressive multi-scale design principle. This&#10;not only allows to scale well to high upsampling factors (e.g., 8x) but&#10;constitutes a principled multi-scale approach that increases the reconstruction&#10;quality for all upsampling factors simultaneously. In particular ProSR ranks&#10;2nd in terms of SSIM and 4th in terms of PSNR in the NTIRE2018 SISR challenge&#10;[34]. Compared to the top-ranking team, our model is marginally lower, but runs&#10;5 times faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.05600" label="1907.05600">
-        <attvalues>
-          <attvalue for="0" value="Generative Modeling by Estimating Gradients of the Data Distribution" />
-          <attvalue for="1" value="  We introduce a new generative model where samples are produced via Langevin&#10;dynamics using gradients of the data distribution estimated with score&#10;matching. Because gradients can be ill-defined and hard to estimate when the&#10;data resides on low-dimensional manifolds, we perturb the data with different&#10;levels of Gaussian noise, and jointly estimate the corresponding scores, i.e.,&#10;the vector fields of gradients of the perturbed data distribution for all noise&#10;levels. For sampling, we propose an annealed Langevin dynamics where we use&#10;gradients corresponding to gradually decreasing noise levels as the sampling&#10;process gets closer to the data manifold. Our framework allows flexible model&#10;architectures, requires no sampling during training or the use of adversarial&#10;methods, and provides a learning objective that can be used for principled&#10;model comparisons. Our models produce samples comparable to GANs on MNIST,&#10;CelebA and CIFAR-10 datasets, achieving a new state-of-the-art inception score&#10;of 8.87 on CIFAR-10. Additionally, we demonstrate that our models learn&#10;effective representations via image inpainting experiments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.17076" label="2303.17076">
-        <attvalues>
-          <attvalue for="0" value="DiffCollage: Parallel Generation of Large Content with Diffusion Models" />
-          <attvalue for="1" value="  We present DiffCollage, a compositional diffusion model that can generate&#10;large content by leveraging diffusion models trained on generating pieces of&#10;the large content. Our approach is based on a factor graph representation where&#10;each factor node represents a portion of the content and a variable node&#10;represents their overlap. This representation allows us to aggregate&#10;intermediate outputs from diffusion models defined on individual nodes to&#10;generate content of arbitrary size and shape in parallel without resorting to&#10;an autoregressive generation procedure. We apply DiffCollage to various tasks,&#10;including infinite image generation, panorama image generation, and&#10;long-duration text-guided motion generation. Extensive experimental results&#10;with a comparison to strong autoregressive baselines verify the effectiveness&#10;of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.03859" label="2205.03859">
-        <attvalues>
-          <attvalue for="0" value="On Conditioning the Input Noise for Controlled Image Generation with&#10;  Diffusion Models" />
-          <attvalue for="1" value="  Conditional image generation has paved the way for several breakthroughs in&#10;image editing, generating stock photos and 3-D object generation. This&#10;continues to be a significant area of interest with the rise of new&#10;state-of-the-art methods that are based on diffusion models. However, diffusion&#10;models provide very little control over the generated image, which led to&#10;subsequent works exploring techniques like classifier guidance, that provides a&#10;way to trade off diversity with fidelity. In this work, we explore techniques&#10;to condition diffusion models with carefully crafted input noise artifacts.&#10;This allows generation of images conditioned on semantic attributes. This is&#10;different from existing approaches that input Gaussian noise and further&#10;introduce conditioning at the diffusion model's inference step. Our experiments&#10;over several examples and conditional settings show the potential of our&#10;approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.06337" label="2105.06337">
-        <attvalues>
-          <attvalue for="0" value="Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech" />
-          <attvalue for="1" value="  Recently, denoising diffusion probabilistic models and generative score&#10;matching have shown high potential in modelling complex data distributions&#10;while stochastic calculus has provided a unified point of view on these&#10;techniques allowing for flexible inference schemes. In this paper we introduce&#10;Grad-TTS, a novel text-to-speech model with score-based decoder producing&#10;mel-spectrograms by gradually transforming noise predicted by encoder and&#10;aligned with text input by means of Monotonic Alignment Search. The framework&#10;of stochastic differential equations helps us to generalize conventional&#10;diffusion probabilistic models to the case of reconstructing data from noise&#10;with different parameters and allows to make this reconstruction flexible by&#10;explicitly controlling trade-off between sound quality and inference speed.&#10;Subjective human evaluation shows that Grad-TTS is competitive with&#10;state-of-the-art text-to-speech approaches in terms of Mean Opinion Score. We&#10;will make the code publicly available shortly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.02246" label="2206.02246">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models" />
-          <attvalue for="1" value="  We present a novel way of conditioning a pretrained denoising diffusion&#10;speech model to produce speech in the voice of a novel person unseen during&#10;training. The method requires a short (~3 seconds) sample from the target&#10;person, and generation is steered at inference time, without any training&#10;steps. At the heart of the method lies a sampling process that combines the&#10;estimation of the denoising model with a low-pass version of the new speaker's&#10;sample. The objective and subjective evaluations show that our sampling method&#10;can generate a voice similar to that of the target speaker in terms of&#10;frequency, with an accuracy comparable to state-of-the-art methods, and without&#10;training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.14792" label="2209.14792">
-        <attvalues>
-          <attvalue for="0" value="Make-A-Video: Text-to-Video Generation without Text-Video Data" />
-          <attvalue for="1" value="  We propose Make-A-Video -- an approach for directly translating the&#10;tremendous recent progress in Text-to-Image (T2I) generation to Text-to-Video&#10;(T2V). Our intuition is simple: learn what the world looks like and how it is&#10;described from paired text-image data, and learn how the world moves from&#10;unsupervised video footage. Make-A-Video has three advantages: (1) it&#10;accelerates training of the T2V model (it does not need to learn visual and&#10;multimodal representations from scratch), (2) it does not require paired&#10;text-video data, and (3) the generated videos inherit the vastness (diversity&#10;in aesthetic, fantastical depictions, etc.) of today's image generation models.&#10;We design a simple yet effective way to build on T2I models with novel and&#10;effective spatial-temporal modules. First, we decompose the full temporal U-Net&#10;and attention tensors and approximate them in space and time. Second, we design&#10;a spatial temporal pipeline to generate high resolution and frame rate videos&#10;with a video decoder, interpolation model and two super resolution models that&#10;can enable various applications besides T2V. In all aspects, spatial and&#10;temporal resolution, faithfulness to text, and quality, Make-A-Video sets the&#10;new state-of-the-art in text-to-video generation, as determined by both&#10;qualitative and quantitative measures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.10440" label="2211.10440">
-        <attvalues>
-          <attvalue for="0" value="Magic3D: High-Resolution Text-to-3D Content Creation" />
-          <attvalue for="1" value="  DreamFusion has recently demonstrated the utility of a pre-trained&#10;text-to-image diffusion model to optimize Neural Radiance Fields (NeRF),&#10;achieving remarkable text-to-3D synthesis results. However, the method has two&#10;inherent limitations: (a) extremely slow optimization of NeRF and (b)&#10;low-resolution image space supervision on NeRF, leading to low-quality 3D&#10;models with a long processing time. In this paper, we address these limitations&#10;by utilizing a two-stage optimization framework. First, we obtain a coarse&#10;model using a low-resolution diffusion prior and accelerate with a sparse 3D&#10;hash grid structure. Using the coarse representation as the initialization, we&#10;further optimize a textured 3D mesh model with an efficient differentiable&#10;renderer interacting with a high-resolution latent diffusion model. Our method,&#10;dubbed Magic3D, can create high quality 3D mesh models in 40 minutes, which is&#10;2x faster than DreamFusion (reportedly taking 1.5 hours on average), while also&#10;achieving higher resolution. User studies show 61.7% raters to prefer our&#10;approach over DreamFusion. Together with the image-conditioned generation&#10;capabilities, we provide users with new ways to control 3D synthesis, opening&#10;up new avenues to various creative applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.05543" label="2302.05543">
-        <attvalues>
-          <attvalue for="0" value="Adding Conditional Control to Text-to-Image Diffusion Models" />
-          <attvalue for="1" value="  We present ControlNet, a neural network architecture to add spatial&#10;conditioning controls to large, pretrained text-to-image diffusion models.&#10;ControlNet locks the production-ready large diffusion models, and reuses their&#10;deep and robust encoding layers pretrained with billions of images as a strong&#10;backbone to learn a diverse set of conditional controls. The neural&#10;architecture is connected with &quot;zero convolutions&quot; (zero-initialized&#10;convolution layers) that progressively grow the parameters from zero and ensure&#10;that no harmful noise could affect the finetuning. We test various conditioning&#10;controls, eg, edges, depth, segmentation, human pose, etc, with Stable&#10;Diffusion, using single or multiple conditions, with or without prompts. We&#10;show that the training of ControlNets is robust with small (&lt;50k) and large&#10;(&gt;1m) datasets. Extensive results show that ControlNet may facilitate wider&#10;applications to control image diffusion models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.08332" label="2211.08332">
-        <attvalues>
-          <attvalue for="0" value="Versatile Diffusion: Text, Images and Variations All in One Diffusion&#10;  Model" />
-          <attvalue for="1" value="  Recent advances in diffusion models have set an impressive milestone in many&#10;generation tasks, and trending works such as DALL-E2, Imagen, and Stable&#10;Diffusion have attracted great interest. Despite the rapid landscape changes,&#10;recent new approaches focus on extensions and performance rather than capacity,&#10;thus requiring separate models for separate tasks. In this work, we expand the&#10;existing single-flow diffusion pipeline into a multi-task multimodal network,&#10;dubbed Versatile Diffusion (VD), that handles multiple flows of text-to-image,&#10;image-to-text, and variations in one unified model. The pipeline design of VD&#10;instantiates a unified multi-flow diffusion framework, consisting of sharable&#10;and swappable layer modules that enable the crossmodal generality beyond images&#10;and text. Through extensive experiments, we demonstrate that VD successfully&#10;achieves the following: a) VD outperforms the baseline approaches and handles&#10;all its base tasks with competitive quality; b) VD enables novel extensions&#10;such as disentanglement of style and semantics, dual- and multi-context&#10;blending, etc.; c) The success of our multi-flow multimodal framework over&#10;images and text may inspire further diffusion-based universal AI research. Our&#10;code and models are open-sourced at&#10;https://github.com/SHI-Labs/Versatile-Diffusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14822" label="2111.14822">
-        <attvalues>
-          <attvalue for="0" value="Vector Quantized Diffusion Model for Text-to-Image Synthesis" />
-          <attvalue for="1" value="  We present the vector quantized diffusion (VQ-Diffusion) model for&#10;text-to-image generation. This method is based on a vector quantized&#10;variational autoencoder (VQ-VAE) whose latent space is modeled by a conditional&#10;variant of the recently developed Denoising Diffusion Probabilistic Model&#10;(DDPM). We find that this latent-space method is well-suited for text-to-image&#10;generation tasks because it not only eliminates the unidirectional bias with&#10;existing methods but also allows us to incorporate a mask-and-replace diffusion&#10;strategy to avoid the accumulation of errors, which is a serious problem with&#10;existing methods. Our experiments show that the VQ-Diffusion produces&#10;significantly better text-to-image generation results when compared with&#10;conventional autoregressive (AR) models with similar numbers of parameters.&#10;Compared with previous GAN-based text-to-image methods, our VQ-Diffusion can&#10;handle more complex scenes and improve the synthesized image quality by a large&#10;margin. Finally, we show that the image generation computation in our method&#10;can be made highly efficient by reparameterization. With traditional AR&#10;methods, the text-to-image generation time increases linearly with the output&#10;image resolution and hence is quite time consuming even for normal size images.&#10;The VQ-Diffusion allows us to achieve a better trade-off between quality and&#10;speed. Our experiments indicate that the VQ-Diffusion model with the&#10;reparameterization is fifteen times faster than traditional AR methods while&#10;achieving a better image quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.08453" label="2302.08453">
-        <attvalues>
-          <attvalue for="0" value="T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for&#10;  Text-to-Image Diffusion Models" />
-          <attvalue for="1" value="  The incredible generative ability of large-scale text-to-image (T2I) models&#10;has demonstrated strong power of learning complex structures and meaningful&#10;semantics. However, relying solely on text prompts cannot fully take advantage&#10;of the knowledge learned by the model, especially when flexible and accurate&#10;controlling (e.g., color and structure) is needed. In this paper, we aim to&#10;``dig out&quot; the capabilities that T2I models have implicitly learned, and then&#10;explicitly use them to control the generation more granularly. Specifically, we&#10;propose to learn simple and lightweight T2I-Adapters to align internal&#10;knowledge in T2I models with external control signals, while freezing the&#10;original large T2I models. In this way, we can train various adapters according&#10;to different conditions, achieving rich control and editing effects in the&#10;color and structure of the generation results. Further, the proposed&#10;T2I-Adapters have attractive properties of practical value, such as&#10;composability and generalization ability. Extensive experiments demonstrate&#10;that our T2I-Adapter has promising generation quality and a wide range of&#10;applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.05768" label="2305.05768">
-        <attvalues>
-          <attvalue for="0" value="DifFIQA: Face Image Quality Assessment Using Denoising Diffusion&#10;  Probabilistic Models" />
-          <attvalue for="1" value="  Modern face recognition (FR) models excel in constrained scenarios, but often&#10;suffer from decreased performance when deployed in unconstrained (real-world)&#10;environments due to uncertainties surrounding the quality of the captured&#10;facial data. Face image quality assessment (FIQA) techniques aim to mitigate&#10;these performance degradations by providing FR models with sample-quality&#10;predictions that can be used to reject low-quality samples and reduce false&#10;match errors. However, despite steady improvements, ensuring reliable quality&#10;estimates across facial images with diverse characteristics remains&#10;challenging. In this paper, we present a powerful new FIQA approach, named&#10;DifFIQA, which relies on denoising diffusion probabilistic models (DDPM) and&#10;ensures highly competitive results. The main idea behind the approach is to&#10;utilize the forward and backward processes of DDPMs to perturb facial images&#10;and quantify the impact of these perturbations on the corresponding image&#10;embeddings for quality prediction. Because the diffusion-based perturbations&#10;are computationally expensive, we also distill the knowledge encoded in DifFIQA&#10;into a regression-based quality predictor, called DifFIQA(R), that balances&#10;performance and execution time. We evaluate both models in comprehensive&#10;experiments on 7 datasets, with 4 target FR models and against 10&#10;state-of-the-art FIQA techniques with highly encouraging results. The source&#10;code will be made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.08818" label="2304.08818">
-        <attvalues>
-          <attvalue for="0" value="Align your Latents: High-Resolution Video Synthesis with Latent&#10;  Diffusion Models" />
-          <attvalue for="1" value="  Latent Diffusion Models (LDMs) enable high-quality image synthesis while&#10;avoiding excessive compute demands by training a diffusion model in a&#10;compressed lower-dimensional latent space. Here, we apply the LDM paradigm to&#10;high-resolution video generation, a particularly resource-intensive task. We&#10;first pre-train an LDM on images only; then, we turn the image generator into a&#10;video generator by introducing a temporal dimension to the latent space&#10;diffusion model and fine-tuning on encoded image sequences, i.e., videos.&#10;Similarly, we temporally align diffusion model upsamplers, turning them into&#10;temporally consistent video super resolution models. We focus on two relevant&#10;real-world applications: Simulation of in-the-wild driving data and creative&#10;content creation with text-to-video modeling. In particular, we validate our&#10;Video LDM on real driving videos of resolution 512 x 1024, achieving&#10;state-of-the-art performance. Furthermore, our approach can easily leverage&#10;off-the-shelf pre-trained image LDMs, as we only need to train a temporal&#10;alignment model in that case. Doing so, we turn the publicly available,&#10;state-of-the-art text-to-image LDM Stable Diffusion into an efficient and&#10;expressive text-to-video model with resolution up to 1280 x 2048. We show that&#10;the temporal layers trained in this way generalize to different fine-tuned&#10;text-to-image LDMs. Utilizing this property, we show the first results for&#10;personalized text-to-video generation, opening exciting directions for future&#10;content creation. Project page:&#10;https://research.nvidia.com/labs/toronto-ai/VideoLDM/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14951" label="2104.14951">
-        <attvalues>
-          <attvalue for="0" value="SRDiff: Single Image Super-Resolution with Diffusion Probabilistic&#10;  Models" />
-          <attvalue for="1" value="  Single image super-resolution (SISR) aims to reconstruct high-resolution (HR)&#10;images from the given low-resolution (LR) ones, which is an ill-posed problem&#10;because one LR image corresponds to multiple HR images. Recently,&#10;learning-based SISR methods have greatly outperformed traditional ones, while&#10;suffering from over-smoothing, mode collapse or large model footprint issues&#10;for PSNR-oriented, GAN-driven and flow-based methods respectively. To solve&#10;these problems, we propose a novel single image super-resolution diffusion&#10;probabilistic model (SRDiff), which is the first diffusion-based model for&#10;SISR. SRDiff is optimized with a variant of the variational bound on the data&#10;likelihood and can provide diverse and realistic SR predictions by gradually&#10;transforming the Gaussian noise into a super-resolution (SR) image conditioned&#10;on an LR input through a Markov chain. In addition, we introduce residual&#10;prediction to the whole framework to speed up convergence. Our extensive&#10;experiments on facial and general benchmarks (CelebA and DIV2K datasets) show&#10;that 1) SRDiff can generate diverse SR results in rich details with&#10;state-of-the-art performance, given only one LR input; 2) SRDiff is easy to&#10;train with a small footprint; and 3) SRDiff can perform flexible image&#10;manipulation including latent space interpolation and content fusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.12831" label="2302.12831">
-        <attvalues>
-          <attvalue for="0" value="CDPMSR: Conditional Diffusion Probabilistic Models for Single Image&#10;  Super-Resolution" />
-          <attvalue for="1" value="  Diffusion probabilistic models (DPM) have been widely adopted in&#10;image-to-image translation to generate high-quality images. Prior attempts at&#10;applying the DPM to image super-resolution (SR) have shown that iteratively&#10;refining a pure Gaussian noise with a conditional image using a U-Net trained&#10;on denoising at various-level noises can help obtain a satisfied&#10;high-resolution image for the low-resolution one. To further improve the&#10;performance and simplify current DPM-based super-resolution methods, we propose&#10;a simple but non-trivial DPM-based super-resolution post-process&#10;framework,i.e., cDPMSR. After applying a pre-trained SR model on the to-be-test&#10;LR image to provide the conditional input, we adapt the standard DPM to conduct&#10;conditional image generation and perform super-resolution through a&#10;deterministic iterative denoising process. Our method surpasses prior attempts&#10;on both qualitative and quantitative results and can generate more&#10;photo-realistic counterparts for the low-resolution images with various&#10;benchmark datasets including Set5, Set14, Urban100, BSD100, and Manga109. Code&#10;will be published after accepted.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.12064" label="2209.12064">
-        <attvalues>
-          <attvalue for="0" value="Face Super-Resolution Using Stochastic Differential Equations" />
-          <attvalue for="1" value="  Diffusion models have proven effective for various applications such as&#10;images, audio and graph generation. Other important applications are image&#10;super-resolution and the solution of inverse problems. More recently, some&#10;works have used stochastic differential equations (SDEs) to generalize&#10;diffusion models to continuous time. In this work, we introduce SDEs to&#10;generate super-resolution face images. To the best of our knowledge, this is&#10;the first time SDEs have been used for such an application. The proposed method&#10;provides an improved peak signal-to-noise ratio (PSNR), structural similarity&#10;index measure (SSIM), and consistency than the existing super-resolution&#10;methods based on diffusion models. In particular, we also assess the potential&#10;application of this method for the face recognition task. A generic facial&#10;feature extractor is used to compare the super-resolution images with the&#10;ground truth and superior results were obtained compared with other methods.&#10;Our code is publicly available at https://github.com/marcelowds/sr-sde&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.14626" label="2207.14626">
-        <attvalues>
-          <attvalue for="0" value="Restoring Vision in Adverse Weather Conditions with Patch-Based&#10;  Denoising Diffusion Models" />
-          <attvalue for="1" value="  Image restoration under adverse weather conditions has been of significant&#10;interest for various computer vision applications. Recent successful methods&#10;rely on the current progress in deep neural network architectural designs&#10;(e.g., with vision transformers). Motivated by the recent progress achieved&#10;with state-of-the-art conditional generative models, we present a novel&#10;patch-based image restoration algorithm based on denoising diffusion&#10;probabilistic models. Our patch-based diffusion modeling approach enables&#10;size-agnostic image restoration by using a guided denoising process with&#10;smoothed noise estimates across overlapping patches during inference. We&#10;empirically evaluate our model on benchmark datasets for image desnowing,&#10;combined deraining and dehazing, and raindrop removal. We demonstrate our&#10;approach to achieve state-of-the-art performances on both weather-specific and&#10;multi-weather image restoration, and experimentally show strong generalization&#10;to real-world test images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.02475" label="2112.02475">
-        <attvalues>
-          <attvalue for="0" value="Deblurring via Stochastic Refinement" />
-          <attvalue for="1" value="  Image deblurring is an ill-posed problem with multiple plausible solutions&#10;for a given input image. However, most existing methods produce a deterministic&#10;estimate of the clean image and are trained to minimize pixel-level distortion.&#10;These metrics are known to be poorly correlated with human perception, and&#10;often lead to unrealistic reconstructions. We present an alternative framework&#10;for blind deblurring based on conditional diffusion models. Unlike existing&#10;techniques, we train a stochastic sampler that refines the output of a&#10;deterministic predictor and is capable of producing a diverse set of plausible&#10;reconstructions for a given input. This leads to a significant improvement in&#10;perceptual quality over existing state-of-the-art methods across multiple&#10;standard benchmarks. Our predict-and-refine approach also enables much more&#10;efficient sampling compared to typical diffusion models. Combined with a&#10;carefully tuned network architecture and inference procedure, our method is&#10;competitive in terms of distortion metrics such as PSNR. These results show&#10;clear benefits of our diffusion-based method for deblurring and challenge the&#10;widely used strategy of producing a single, deterministic reconstruction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;The goal of image deblurring is to generate a plausible reconstruction of the unobserved sharp, clean image $\bx$ from a blurry input $\by$.&#10;Deblurring techniques differ in what they aim to obtain. For example, one could try to directly sample from the posterior $p(\bx \given \by)$. Another viable option is to compute a point-estimate such as the conditional mean $\E \brac{\bx \given \by}$ or the maximum a posteriori estimate $\argmax_{\bx} p(\bx\given\by)$.&#10;&#10;Deblurring through point estimates.&#10;Traditional deblurring methods formulate the problem as one of blind deconvolution~\cite{chan1998total,fergus2006removing,shan2008high,levin2009understanding,zhu2012deconvolving,xu2013unnatural,lai2016comparative,jin2018normalized,delbracio2021polyblur,chen2019blind}. In this setup, the blur is generally modeled as a noisy linear operator acting on the clean image. While the exact values of the blur operator are not assumed to be known, one can enforce some prior distribution on the blur and the sharp image and try to find the most likely solution.&#10;&#10;Alternatively, many recent methods adopt an end-to-end approach where a deep neural network is trained to directly produce a point estimate~\cite{chakrabarti2016neural,gao2019dynamic,nah2017deep,ramakrishnan2017deep, kupyn2018deblurgan,kupyn2019deblurgan,su2017deep,sun2015learning,tao2018scale,wieschollek2017learning, ren2021deblurring,chen2021hinet,cho2021rethinking}. These methods generally rely on pairs of blurry-sharp images as training data and cast the deblurring problem as a supervised regression task. Much of the efforts have gone into developing specialized network architectures and loss functions to achieve better pixel-level reconstruction metrics such as PSNR or SSIM~\cite{wang2004image}.&#10;For example, MIMO-UNet \cite{cho2021rethinking} proposed an architecture that facilitates information flow across different image resolutions in a multi-scale U-Net \cite{ronneberger2015u}. Another work HINet \cite{chen2021hinet} introduced Half Instance Normalization \cite{ulyanov2016instance}, which can be used as a building block for image restoration networks. MPRNet \cite{zamir2021multi} presented an improved multi-stage architecture designed to incorporate both high-level global features as well as local details.&#10;&#10;Issue of regression to the mean.&#10;While the aforementioned approaches lead to state-of-the-art PSNR, they share the limitation that they can only produce a deterministic output. This is at odds with the nature of blind image deblurring, which is an inherently ill-posed inverse problem with multiple valid solutions for a single input.&#10;In fact, the current trend of developing point-estimators that directly minimize a distortion loss suffers from the problem of ``regression to the mean''. If there are multiple possible clean images that correspond to the blurry input, the optimal reconstruction according to the given loss function will be an average of them.&#10;Consequently, the resultant deterministic reconstruction often lacks details as it learns to produce the average of all possible solutions at best.&#10;&#10;Diverse image restoration. One way to circumvent the regression to the mean phenomenon is to avoid point estimations and directly learn to generate samples from the posterior distribution~\cite{kawar2021stochastic,kawar2021snips,ohayon2021high,kadkhodaie2021stochastic}.&#10;While techniques based on adversarial training have been explored for blind deblurring~\cite{kupyn2018deblurgan,kupyn2019deblurgan}, in general they are not trained to produce multiple samples.&#10;Additionally, non-reference based adversarial losses can introduce significant hallucinations and distortions~\cite{cohen2018distribution}.&#10;&#10;Likelihood-based deep generative models such as&#10;Variational Autoencoders~\cite{prakash2020fully}, Normalizing Flows~\cite{lugmayr2020srflow,lugmayr2021ntire}, and Diffusion Probabilistic Models (DPMs)~\cite{saharia2021image,li2021srdiff} have also been successfully applied to other image enhancement tasks such as super-resolution, where a diverse set of candidates can be generated from the learned posterior~\cite{prakash2020fully}.&#10;Compared to point estimates, solving imaging inverse problems by sampling from the posterior has additional benefits such as uncertainty quantification~\cite{kawar2021stochastic,whang2021composing,kawar2021snips}, near-optimal sample complexity~\cite{jalal2020robust} and better fairness guarantees~\cite{jalal2021fairness}.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2304.08291" label="2304.08291">
-        <attvalues>
-          <attvalue for="0" value="Refusion: Enabling Large-Size Realistic Image Restoration with&#10;  Latent-Space Diffusion Models" />
-          <attvalue for="1" value="  This work aims to improve the applicability of diffusion models in realistic&#10;image restoration. Specifically, we enhance the diffusion model in several&#10;aspects such as network architecture, noise level, denoising steps, training&#10;image size, and optimizer/scheduler. We show that tuning these hyperparameters&#10;allows us to achieve better performance on both distortion and perceptual&#10;scores. We also propose a U-Net based latent diffusion model which performs&#10;diffusion in a low-resolution latent space while preserving high-resolution&#10;information from the original input for the decoding process. Compared to the&#10;previous latent-diffusion model which trains a VAE-GAN to compress the image,&#10;our proposed U-Net compression strategy is significantly more stable and can&#10;recover highly accurate images without relying on adversarial optimization.&#10;Importantly, these modifications allow us to apply diffusion models to various&#10;image restoration tasks, including real-world shadow removal, HR&#10;non-homogeneous dehazing, stereo super-resolution, and bokeh effect&#10;transformation. By simply replacing the datasets and slightly changing the&#10;noise network, our model, named Refusion, is able to deal with large-size&#10;images (e.g., 6000 x 4000 x 3 in HR dehazing) and produces good results on all&#10;the above restoration problems. Our Refusion achieves the best perceptual&#10;performance in the NTIRE 2023 Image Shadow Removal Challenge and wins 2nd place&#10;overall.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.10028" label="2305.10028">
-        <attvalues>
-          <attvalue for="0" value="Pyramid Diffusion Models For Low-light Image Enhancement" />
-          <attvalue for="1" value="  Recovering noise-covered details from low-light images is challenging, and&#10;the results given by previous methods leave room for improvement. Recent&#10;diffusion models show realistic and detailed image generation through a&#10;sequence of denoising refinements and motivate us to introduce them to&#10;low-light image enhancement for recovering realistic details. However, we found&#10;two problems when doing this, i.e., 1) diffusion models keep constant&#10;resolution in one reverse process, which limits the speed; 2) diffusion models&#10;sometimes result in global degradation (e.g., RGB shift). To address the above&#10;problems, this paper proposes a Pyramid Diffusion model (PyDiff) for low-light&#10;image enhancement. PyDiff uses a novel pyramid diffusion method to perform&#10;sampling in a pyramid resolution style (i.e., progressively increasing&#10;resolution in one reverse process). Pyramid diffusion makes PyDiff much faster&#10;than vanilla diffusion models and introduces no performance degradation.&#10;Furthermore, PyDiff uses a global corrector to alleviate the global degradation&#10;that may occur in the reverse process, significantly improving the performance&#10;and making the training of diffusion models easier with little additional&#10;computational consumption. Extensive experiments on popular benchmarks show&#10;that PyDiff achieves superior performance and efficiency. Moreover, PyDiff can&#10;generalize well to unseen noise and illumination distributions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.06682" label="2303.06682">
-        <attvalues>
-          <attvalue for="0" value="DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for&#10;  Hyperspectral Image Restoration" />
-          <attvalue for="1" value="  Diffusion models have recently received a surge of interest due to their&#10;impressive performance for image restoration, especially in terms of noise&#10;robustness. However, existing diffusion-based methods are trained on a large&#10;amount of training data and perform very well in-distribution, but can be quite&#10;susceptible to distribution shift. This is especially inappropriate for&#10;data-starved hyperspectral image (HSI) restoration. To tackle this problem,&#10;this work puts forth a self-supervised diffusion model for HSI restoration,&#10;namely Denoising Diffusion Spatio-Spectral Model (\texttt{DDS2M}), which works&#10;by inferring the parameters of the proposed Variational Spatio-Spectral Module&#10;(VS2M) during the reverse diffusion process, solely using the degraded HSI&#10;without any extra training data. In VS2M, a variational inference-based loss&#10;function is customized to enable the untrained spatial and spectral networks to&#10;learn the posterior distribution, which serves as the transitions of the&#10;sampling chain to help reverse the diffusion process. Benefiting from its&#10;self-supervised nature and the diffusion process, \texttt{DDS2M} enjoys&#10;stronger generalization ability to various HSIs compared to existing&#10;diffusion-based methods and superior robustness to noise compared to existing&#10;HSI restoration methods. Extensive experiments on HSI denoising, noisy HSI&#10;completion and super-resolution on a variety of HSIs demonstrate&#10;\texttt{DDS2M}'s superiority over the existing task-specific state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.14951" label="2105.14951">
-        <attvalues>
-          <attvalue for="0" value="SNIPS: Solving Noisy Inverse Problems Stochastically" />
-          <attvalue for="1" value="  In this work we introduce a novel stochastic algorithm dubbed SNIPS, which&#10;draws samples from the posterior distribution of any linear inverse problem,&#10;where the observation is assumed to be contaminated by additive white Gaussian&#10;noise. Our solution incorporates ideas from Langevin dynamics and Newton's&#10;method, and exploits a pre-trained minimum mean squared error (MMSE) Gaussian&#10;denoiser. The proposed approach relies on an intricate derivation of the&#10;posterior score function that includes a singular value decomposition (SVD) of&#10;the degradation operator, in order to obtain a tractable iterative algorithm&#10;for the desired sampling. Due to its stochasticity, the algorithm can produce&#10;multiple high perceptual quality samples for the same noisy observation. We&#10;demonstrate the abilities of the proposed paradigm for image deblurring,&#10;super-resolution, and compressive sensing. We show that the samples produced&#10;are sharp, detailed and consistent with the given measurements, and their&#10;diversity exposes the inherent uncertainty in the inverse problem being solved.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.11888" label="2209.11888">
-        <attvalues>
-          <attvalue for="0" value="JPEG Artifact Correction using Denoising Diffusion Restoration Models" />
-          <attvalue for="1" value="  Diffusion models can be used as learned priors for solving various inverse&#10;problems. However, most existing approaches are restricted to linear inverse&#10;problems, limiting their applicability to more general cases. In this paper, we&#10;build upon Denoising Diffusion Restoration Models (DDRM) and propose a method&#10;for solving some non-linear inverse problems. We leverage the pseudo-inverse&#10;operator used in DDRM and generalize this concept for other measurement&#10;operators, which allows us to use pre-trained unconditional diffusion models&#10;for applications such as JPEG artifact correction. We empirically demonstrate&#10;the effectiveness of our approach across various quality factors, attaining&#10;performance levels that are on par with state-of-the-art methods trained&#10;specifically for the JPEG restoration task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.11793" label="2201.11793">
-        <attvalues>
-          <attvalue for="0" value="Denoising Diffusion Restoration Models" />
-          <attvalue for="1" value="  Many interesting tasks in image restoration can be cast as linear inverse&#10;problems. A recent family of approaches for solving these problems uses&#10;stochastic algorithms that sample from the posterior distribution of natural&#10;images given the measurements. However, efficient solutions often require&#10;problem-specific supervised training to model the posterior, whereas&#10;unsupervised methods that are not problem-specific typically rely on&#10;inefficient iterative methods. This work addresses these issues by introducing&#10;Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised&#10;posterior sampling method. Motivated by variational inference, DDRM takes&#10;advantage of a pre-trained denoising diffusion generative model for solving any&#10;linear inverse problem. We demonstrate DDRM's versatility on several image&#10;datasets for super-resolution, deblurring, inpainting, and colorization under&#10;various amounts of measurement noise. DDRM outperforms the current leading&#10;unsupervised methods on the diverse ImageNet dataset in reconstruction quality,&#10;perceptual quality, and runtime, being 5x faster than the nearest competitor.&#10;DDRM also generalizes well for natural images out of the distribution of the&#10;observed ImageNet training set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.00490" label="2212.00490">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model" />
-          <attvalue for="1" value="  Most existing Image Restoration (IR) models are task-specific, which can not&#10;be generalized to different degradation operators. In this work, we propose the&#10;Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for&#10;arbitrary linear IR problems, including but not limited to image&#10;super-resolution, colorization, inpainting, compressed sensing, and deblurring.&#10;DDNM only needs a pre-trained off-the-shelf diffusion model as the generative&#10;prior, without any extra training or network modifications. By refining only&#10;the null-space contents during the reverse diffusion process, we can yield&#10;diverse results satisfying both data consistency and realness. We further&#10;propose an enhanced and robust version, dubbed DDNM+, to support noisy&#10;restoration and improve restoration quality for hard tasks. Our experiments on&#10;several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot&#10;IR methods. We also demonstrate that DDNM+ can solve complex real-world&#10;applications, e.g., old photo restoration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2209.14687" label="2209.14687">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Posterior Sampling for General Noisy Inverse Problems" />
-          <attvalue for="1" value="  Diffusion models have been recently studied as powerful generative inverse&#10;problem solvers, owing to their high quality reconstructions and the ease of&#10;combining existing iterative solvers. However, most works focus on solving&#10;simple linear inverse problems in noiseless settings, which significantly&#10;under-represents the complexity of real-world problems. In this work, we extend&#10;diffusion solvers to efficiently handle general noisy (non)linear inverse&#10;problems via approximation of the posterior sampling. Interestingly, the&#10;resulting posterior sampling scheme is a blended version of diffusion sampling&#10;with the manifold constrained gradient without a strict measurement consistency&#10;projection step, yielding a more desirable generative path in noisy settings&#10;compared to the previous studies. Our method demonstrates that diffusion models&#10;can incorporate various measurement noise statistics such as Gaussian and&#10;Poisson, and also efficiently handle noisy nonlinear inverse problems such as&#10;Fourier phase retrieval and non-uniform deblurring. Code available at&#10;https://github.com/DPS2022/diffusion-posterior-sampling&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.08005" label="2111.08005">
-        <attvalues>
-          <attvalue for="0" value="Solving Inverse Problems in Medical Imaging with Score-Based Generative&#10;  Models" />
-          <attvalue for="1" value="  Reconstructing medical images from partial measurements is an important&#10;inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging&#10;(MRI). Existing solutions based on machine learning typically train a model to&#10;directly map measurements to medical images, leveraging a training dataset of&#10;paired images and measurements. These measurements are typically synthesized&#10;from images using a fixed physical model of the measurement process, which&#10;hinders the generalization capability of models to unknown measurement&#10;processes. To address this issue, we propose a fully unsupervised technique for&#10;inverse problem solving, leveraging the recently introduced score-based&#10;generative models. Specifically, we first train a score-based generative model&#10;on medical images to capture their prior distribution. Given measurements and a&#10;physical model of the measurement process at test time, we introduce a sampling&#10;method to reconstruct an image consistent with both the prior and the observed&#10;measurements. Our method does not assume a fixed measurement process during&#10;training, and can thus be flexibly adapted to different measurement processes&#10;at test time. Empirically, we observe comparable or better performance to&#10;supervised learning techniques in several medical imaging tasks in CT and MRI,&#10;while demonstrating significantly better generalization to unknown measurement&#10;processes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.11105" label="2304.11105">
-        <attvalues>
-          <attvalue for="0" value="Improved Diffusion-based Image Colorization via Piggybacked Models" />
-          <attvalue for="1" value="  Image colorization has been attracting the research interests of the&#10;community for decades. However, existing methods still struggle to provide&#10;satisfactory colorized results given grayscale images due to a lack of&#10;human-like global understanding of colors. Recently, large-scale Text-to-Image&#10;(T2I) models have been exploited to transfer the semantic information from the&#10;text prompts to the image domain, where text provides a global control for&#10;semantic objects in the image. In this work, we introduce a colorization model&#10;piggybacking on the existing powerful T2I diffusion model. Our key idea is to&#10;exploit the color prior knowledge in the pre-trained T2I diffusion model for&#10;realistic and diverse colorization. A diffusion guider is designed to&#10;incorporate the pre-trained weights of the latent diffusion model to output a&#10;latent color prior that conforms to the visual semantics of the grayscale&#10;input. A lightness-aware VQVAE will then generate the colorized result with&#10;pixel-perfect alignment to the given grayscale image. Our model can also&#10;achieve conditional colorization with additional inputs (e.g. user hints and&#10;texts). Extensive experiments show that our method achieves state-of-the-art&#10;performance in terms of perceptual quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.03221" label="2212.03221">
-        <attvalues>
-          <attvalue for="0" value="ADIR: Adaptive Diffusion for Image Reconstruction" />
-          <attvalue for="1" value="  In recent years, denoising diffusion models have demonstrated outstanding&#10;image generation performance. The information on natural images captured by&#10;these models is useful for many image reconstruction applications, where the&#10;task is to restore a clean image from its degraded observations. In this work,&#10;we propose a conditional sampling scheme that exploits the prior learned by&#10;diffusion models while retaining agreement with the observations. We then&#10;combine it with a novel approach for adapting pretrained diffusion denoising&#10;networks to their input. We examine two adaption strategies: the first uses&#10;only the degraded image, while the second, which we advocate, is performed&#10;using images that are ``nearest neighbors'' of the degraded image, retrieved&#10;from a diverse dataset using an off-the-shelf visual-language model. To&#10;evaluate our method, we test it on two state-of-the-art publicly available&#10;diffusion models, Stable Diffusion and Guided Diffusion. We show that our&#10;proposed `adaptive diffusion for image reconstruction' (ADIR) approach achieves&#10;a significant improvement in the super-resolution, deblurring, and text-based&#10;editing tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.07864" label="2302.07864">
-        <attvalues>
-          <attvalue for="0" value="Denoising Diffusion Probabilistic Models for Robust Image&#10;  Super-Resolution in the Wild" />
-          <attvalue for="1" value="  Diffusion models have shown promising results on single-image&#10;super-resolution and other image- to-image translation tasks. Despite this&#10;success, they have not outperformed state-of-the-art GAN models on the more&#10;challenging blind super-resolution task, where the input images are out of&#10;distribution, with unknown degradations. This paper introduces SR3+, a&#10;diffusion-based model for blind super-resolution, establishing a new&#10;state-of-the-art. To this end, we advocate self-supervised training with a&#10;combination of composite, parameterized degradations for self-supervised&#10;training, and noise-conditioing augmentation during training and testing. With&#10;these innovations, a large-scale convolutional architecture, and large-scale&#10;datasets, SR3+ greatly outperforms SR3. It outperforms Real-ESRGAN when trained&#10;on the same data, with a DRealSR FID score of 36.82 vs. 37.22, which further&#10;improves to FID of 32.37 with larger models, and further still with larger&#10;training sets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.12686" label="2301.12686">
-        <attvalues>
-          <attvalue for="0" value="GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse&#10;  Problems with Denoising Diffusion Restoration" />
-          <attvalue for="1" value="  Pre-trained diffusion models have been successfully used as priors in a&#10;variety of linear inverse problems, where the goal is to reconstruct a signal&#10;from noisy linear measurements. However, existing approaches require knowledge&#10;of the linear operator. In this paper, we propose GibbsDDRM, an extension of&#10;Denoising Diffusion Restoration Models (DDRM) to a blind setting in which the&#10;linear measurement operator is unknown. GibbsDDRM constructs a joint&#10;distribution of the data, measurements, and linear operator by using a&#10;pre-trained diffusion model for the data prior, and it solves the problem by&#10;posterior sampling with an efficient variant of a Gibbs sampler. The proposed&#10;method is problem-agnostic, meaning that a pre-trained diffusion model can be&#10;applied to various inverse problems without fine-tuning. In experiments, it&#10;achieved high performance on both blind image deblurring and vocal&#10;dereverberation tasks, despite the use of simple generic priors for the&#10;underlying linear operators.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.10656" label="2211.10656">
-        <attvalues>
-          <attvalue for="0" value="Parallel Diffusion Models of Operator and Image for Blind Inverse&#10;  Problems" />
-          <attvalue for="1" value="  Diffusion model-based inverse problem solvers have demonstrated&#10;state-of-the-art performance in cases where the forward operator is known (i.e.&#10;non-blind). However, the applicability of the method to blind inverse problems&#10;has yet to be explored. In this work, we show that we can indeed solve a family&#10;of blind inverse problems by constructing another diffusion prior for the&#10;forward operator. Specifically, parallel reverse diffusion guided by gradients&#10;from the intermediate stages enables joint optimization of both the forward&#10;operator parameters as well as the image, such that both are jointly estimated&#10;at the end of the parallel reverse diffusion procedure. We show the efficacy of&#10;our method on two representative tasks -- blind deblurring, and imaging through&#10;turbulence -- and show that our method yields state-of-the-art performance,&#10;while also being flexible to be applicable to general blind inverse problems&#10;when we know the functional forms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.07015" label="2305.07015">
-        <attvalues>
-          <attvalue for="0" value="Exploiting Diffusion Prior for Real-World Image Super-Resolution" />
-          <attvalue for="1" value="  We present a novel approach to leverage prior knowledge encapsulated in&#10;pre-trained text-to-image diffusion models for blind super-resolution (SR).&#10;Specifically, by employing our time-aware encoder, we can achieve promising&#10;restoration results without altering the pre-trained synthesis model, thereby&#10;preserving the generative prior and minimizing training cost. To remedy the&#10;loss of fidelity caused by the inherent stochasticity of diffusion models, we&#10;employ a controllable feature wrapping module that allows users to balance&#10;quality and fidelity by simply adjusting a scalar value during the inference&#10;process. Moreover, we develop a progressive aggregation sampling strategy to&#10;overcome the fixed-size constraints of pre-trained diffusion models, enabling&#10;adaptation to resolutions of any size. A comprehensive evaluation of our method&#10;using both synthetic and real-world benchmarks demonstrates its superiority&#10;over current state-of-the-art approaches. Code and models are available at&#10;https://github.com/IceClear/StableSR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.06885" label="2303.06885">
-        <attvalues>
-          <attvalue for="0" value="DR2: Diffusion-based Robust Degradation Remover for Blind Face&#10;  Restoration" />
-          <attvalue for="1" value="  Blind face restoration usually synthesizes degraded low-quality data with a&#10;pre-defined degradation model for training, while more complex cases could&#10;happen in the real world. This gap between the assumed and actual degradation&#10;hurts the restoration performance where artifacts are often observed in the&#10;output. However, it is expensive and infeasible to include every type of&#10;degradation to cover real-world cases in the training data. To tackle this&#10;robustness issue, we propose Diffusion-based Robust Degradation Remover (DR2)&#10;to first transform the degraded image to a coarse but degradation-invariant&#10;prediction, then employ an enhancement module to restore the coarse prediction&#10;to a high-quality image. By leveraging a well-performing denoising diffusion&#10;probabilistic model, our DR2 diffuses input images to a noisy status where&#10;various types of degradation give way to Gaussian noise, and then captures&#10;semantic information through iterative denoising steps. As a result, DR2 is&#10;robust against common degradation (e.g. blur, resize, noise and compression)&#10;and compatible with different designs of enhancement modules. Experiments in&#10;various settings show that our framework outperforms state-of-the-art methods&#10;on heavily degraded synthetic and real-world datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.03400" label="2012.03400">
-        <attvalues>
-          <attvalue for="0" value="CompFeat: Comprehensive Feature Aggregation for Video Instance&#10;  Segmentation" />
-          <attvalue for="1" value="  Video instance segmentation is a complex task in which we need to detect,&#10;segment, and track each object for any given video. Previous approaches only&#10;utilize single-frame features for the detection, segmentation, and tracking of&#10;objects and they suffer in the video scenario due to several distinct&#10;challenges such as motion blur and drastic appearance change. To eliminate&#10;ambiguities introduced by only using single-frame features, we propose a novel&#10;comprehensive feature aggregation approach (CompFeat) to refine features at&#10;both frame-level and object-level with temporal and spatial context&#10;information. The aggregation process is carefully designed with a new attention&#10;mechanism which significantly increases the discriminative power of the learned&#10;features. We further improve the tracking capability of our model through a&#10;siamese design by incorporating both feature similarities and spatial&#10;similarities. Experiments conducted on the YouTube-VIS dataset validate the&#10;effectiveness of proposed CompFeat. Our code will be available at&#10;https://github.com/SHI-Labs/CompFeat-for-Video-Instance-Segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="~&#10;In this sectionwe review video instance segmentation and several closely-related tasks such as video object detection and video object tracking.&#10;&#10;{Video Object Detection.} Video object detection aims to detect all objects in videos such as shown in the ImageNet VID challenge~\cite{russakovsky2015imagenet,han2016seq}. Feature aggregation is widely used in video detection~\cite{zhu2017flow,feichtenhofer2017detect,chen2018optimizing,liu2019looking}. For instance, Zhu~et al. proposed to aggregate features from nearby frames to enhance the feature quality of an input frame. However, its speed is pretty slow due to the dense detection and optical flow estimation. In~\cite{chen2018optimizing}, Chen~et al. proposed to use a scale-time lattice to generate detection on sparse key frames and designed a temporal propagation approach for detection in an effective way. Inspired by these work, we propose to improve the feature quality for video instance segmentation via feature aggregation using attention mechanism.&#10;&#10;{Video Object Tracking}. Video object tracking can be viewed as a sub-task of video instance segmentation, which has two scenarios: detection-based tracking and detection-free tracking. In detection-free tracking, given the ground truth location of the target object in the first frame, algorithm is required to track the target object through the whole video. Recently, the Siamese network based trackers have received significant attentions due to their well-balanced accuracy and efficiency~\cite{li2019siamrpn++, zhang2019deeper, wang2018learning, valmadre2017end}. In particular, these trackers attempt to produce a similarity map from cross-correlation of the two feature branches, one for the target object and the other for the search region, where the similarity map embeds more semantic meanings. On the other hand, detection-based tracking~\cite{sadeghian2017tracking, son2017multi,shi2018geometry} simultaneously detect and track multiple video objects, which is more similar to the setting of video instance segmentation. In our proposed CompFeat, we borrow ideas from both detection-based tracking and detection-free tracking.&#10;&#10;{Video Instance Segmentation}. MaskTrack-RCNN~\cite{yang2019video} is the first attempt to address the video instance segmentation problem. It proposes a large-scale video dateset named YouTube-VIS for benchmarking video instance segmentation algorithms. Several methods in the Large-Scale Video Object Segmentation Challenge achieve impressive results but they utilize large quantity of external data and complex algorithm pipelines~\cite{wang2019empirical, dong2019temporal, luiten2019video}. A closely related work is multi-object tracking and segmentation~\cite{voigtlaender2019mots} which is proposed to evaluate multi-object tracking along with instance segmentation. However, because of its limited data scale and few object categories, we do not compare with it in this paper. MaskTrack-RCNN only uses image features but not temporal information of video sequences. We extend this work with a more sophisticated comprehensive feature aggregation approach which greatly boosts the performance on video instance segmentation.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Feature Aggregation Techniques, Object Tracking Methods, Video Object Detection" />
-        </attvalues>
-      </node>
-      <node id="2108.11021" label="2108.11021">
-        <attvalues>
-          <attvalue for="0" value="Layer-wise Customized Weak Segmentation Block and AIoU Loss for Accurate&#10;  Object Detection" />
-          <attvalue for="1" value="  The anchor-based detectors handle the problem of scale variation by building&#10;the feature pyramid and directly setting different scales of anchors on each&#10;cell in different layers. However, it is difficult for box-wise anchors to&#10;guide the adaptive learning of scale-specific features in each layer because&#10;there is no one-to-one correspondence between box-wise anchors and pixel-level&#10;features. In order to alleviate the problem, in this paper, we propose a&#10;scale-customized weak segmentation (SCWS) block at the pixel level for scale&#10;customized object feature learning in each layer. By integrating the SCWS&#10;blocks into the single-shot detector, a scale-aware object detector (SCOD) is&#10;constructed to detect objects of different sizes naturally and accurately.&#10;Furthermore, the standard location loss neglects the fact that the hard and&#10;easy samples may be seriously imbalanced. A forthcoming problem is that it is&#10;unable to get more accurate bounding boxes due to the imbalance. To address&#10;this problem, an adaptive IoU (AIoU) loss via a simple yet effective squeeze&#10;operation is specified in our SCOD. Extensive experiments on PASCAL VOC and MS&#10;COCO demonstrate the superiority of our SCOD.&#10;" />
-          <attvalue for="2" value="&#10;Recently, object detection, as one of the core tasks in computer vision, has achieved significant advances in recent years. On the whole, all the detectors that use deep CNN can be divided into two categories: two-stage approaches \cite{girshick2015fast, ren2015faster, lin2017feature,dai2016r, he2017mask, cai2018cascade, Cao_2020_CVPR, pang2019libra,li2019scale} and one-stage approaches \cite{liu2016ssd, fu2017dssd, redmon2016you, lin2017focal, zhang2018single, zhao2019m2det, redmon2016you,wang2020single, wu2019iou, Zhu_2019_CVPR}. The two-stage detectors have achieved the top performance on benchmark datasets, while the one-stage approaches achieve time efficiency.&#10;&#10;It is well known that the problem of scale variation across object instances has a great influence on the object detection task. Most of the anchor based detectors, such as SSD \cite{liu2016ssd}, FPN \cite{lin2017feature}, handle this problem by building the feature pyramid and directly setting different scales of anchors on each feature map cell in multiple feature layers, which makes each level feature map perform its duty. The box-wise anchors are used to guide the learning of scale-specific features at each layer. However, as the feature of each cell is not equal to the box-wise anchor, the scale-specific features of each layer can not always be obtained under the guidance of box-wise anchors due to the lack of adaptive guidance. That is, the anchor based detectors lack of scale-awareness, which is harmful for accurately detecting objects of different scales.&#10;&#10;Since the scale-specific features can not be obtained under the guidance of box-wise anchors, we propose an idea to construct an adaptive layer-wise guidance module at the pixel level instead of box level to materialize the scale of each layered features. Motivated by this, in this paper, we introduce a scale-customized weak segmentation (SCWS) block, which is supervised by weakly segmented ground-truth (coarse annotation). The SCWS blocks specify the role of each layer at pixel level according to scale and assist each layer in learning scale-specific features. With the SCWS blocks, the detector will naturally realize the scale-awareness. As shown in Fig.~\ref{example}, we present a picture with six object instances of four different scales. Take the smallest scale (i.e., the little girl) as an example, this bounding box of the little girl is assigned to the lowest layer (layer 1). All pixels falling into the bounding box of the little girl will be uniquely classified as ``person'' category in layer 1, while in other layers, the pixels in the location of the girl will be classified as ``background''. Similarly, each object instance can be assigned to a certain layer at pixel level.&#10;&#10;Furthermore, the matching strategy based on IoU is used for preparing the samples in detection tasks. However, this strategy neglects a fact that the IoU distribution of all the samples is imbalanced. Generally, the number of samples at low IoU levels (i.e. hard samples) is significantly larger than that at high IoU levels (i.e. easy samples). Needless to say, the hard samples dominate the gradients of the location loss during training, which enables the model to be biased toward the hard samples.&#10;In order to alleviate the model bias, we propose an adaptive IoU (AIoU) via a simple yet effective squeeze operation, as shown in Fig.~\ref{pious}. And based on the AIoU, we propose a novel AIoU loss, which can essentially improve the hard samples dominated imbalance problem. There are two advantages of our AIoU loss over the standard IoU loss. First, the $\mathcal{L}_{AIoU} \ge \mathcal{L}_{IoU}$, which means our AIoU loss is stricter than standard IoU loss for localization. Second, the relative change rate of AIoU loss to the standard IoU loss, i.e., $\Delta^{'}\mathcal{L}_{AIoU}$, increases with the increase of IoU value. This means that our $\mathcal{L}_{AIoU}$ pays more attention to the samples at high IoU levels, which can prevent the network from being biased toward the hard samples.&#10;&#10;With the SCWS blocks for layer-wise scale-specific feature learning and the AIoU loss for training, a scale-aware object detector (SCOD) is finally proposed in this paper. In summary, this paper makes three contributions:&#10;(1) We propose a pixel-level scale-customized weak segmentation (SCWS) idea to learn scale-specific feature in each layer, which can make the detector attentively realize scale-awareness.&#10;(2) We introduce an adaptive IoU (AIoU) loss to prevent the gradients of localization loss from being dominated by outliers and ensure accurate bounding box regression of the detector.&#10;(3) With multiple SCWS blocks and the AIoU loss, a scale-aware object detector (SCOD) is materialized in this paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Scale Variation Handling, Mathematics, Loss Function Optimization, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1901.01892" label="1901.01892">
-        <attvalues>
-          <attvalue for="0" value="Scale-Aware Trident Networks for Object Detection" />
-          <attvalue for="1" value="  Scale variation is one of the key challenges in object detection. In this&#10;work, we first present a controlled experiment to investigate the effect of&#10;receptive fields for scale variation in object detection. Based on the findings&#10;from the exploration experiments, we propose a novel Trident Network&#10;(TridentNet) aiming to generate scale-specific feature maps with a uniform&#10;representational power. We construct a parallel multi-branch architecture in&#10;which each branch shares the same transformation parameters but with different&#10;receptive fields. Then, we adopt a scale-aware training scheme to specialize&#10;each branch by sampling object instances of proper scales for training. As a&#10;bonus, a fast approximation version of TridentNet could achieve significant&#10;improvements without any additional parameters and computational cost compared&#10;with the vanilla detector. On the COCO dataset, our TridentNet with ResNet-101&#10;backbone achieves state-of-the-art single-model results of 48.4 mAP. Codes are&#10;available at https://git.io/fj5vR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.03511" label="2008.03511">
-        <attvalues>
-          <attvalue for="0" value="Single-Shot Two-Pronged Detector with Rectified IoU Loss" />
-          <attvalue for="1" value="  In the CNN based object detectors, feature pyramids are widely exploited to&#10;alleviate the problem of scale variation across object instances. These object&#10;detectors, which strengthen features via a top-down pathway and lateral&#10;connections, are mainly to enrich the semantic information of low-level&#10;features, but ignore the enhancement of high-level features. This can lead to&#10;an imbalance between different levels of features, in particular a serious lack&#10;of detailed information in the high-level features, which makes it difficult to&#10;get accurate bounding boxes. In this paper, we introduce a novel two-pronged&#10;transductive idea to explore the relationship among different layers in both&#10;backward and forward directions, which can enrich the semantic information of&#10;low-level features and detailed information of high-level features at the same&#10;time. Under the guidance of the two-pronged idea, we propose a Two-Pronged&#10;Network (TPNet) to achieve bidirectional transfer between high-level features&#10;and low-level features, which is useful for accurately detecting object at&#10;different scales. Furthermore, due to the distribution imbalance between the&#10;hard and easy samples in single-stage detectors, the gradient of localization&#10;loss is always dominated by the hard examples that have poor localization&#10;accuracy. This will enable the model to be biased toward the hard samples. So&#10;in our TPNet, an adaptive IoU based localization loss, named Rectified IoU&#10;(RIoU) loss, is proposed to rectify the gradients of each kind of samples. The&#10;Rectified IoU loss increases the gradients of examples with high IoU while&#10;suppressing the gradients of examples with low IoU, which can improve the&#10;overall localization accuracy of model. Extensive experiments demonstrate the&#10;superiority of our TPNet and RIoU loss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.11638" label="2303.11638">
-        <attvalues>
-          <attvalue for="0" value="Human Pose as Compositional Tokens" />
-          <attvalue for="1" value="  Human pose is typically represented by a coordinate vector of body joints or&#10;their heatmap embeddings. While easy for data processing, unrealistic pose&#10;estimates are admitted due to the lack of dependency modeling between the body&#10;joints. In this paper, we present a structured representation, named Pose as&#10;Compositional Tokens (PCT), to explore the joint dependency. It represents a&#10;pose by M discrete tokens with each characterizing a sub-structure with several&#10;interdependent joints. The compositional design enables it to achieve a small&#10;reconstruction error at a low cost. Then we cast pose estimation as a&#10;classification task. In particular, we learn a classifier to predict the&#10;categories of the M tokens from an image. A pre-learned decoder network is used&#10;to recover the pose from the tokens without further post-processing. We show&#10;that it achieves better or comparable pose estimation results as the existing&#10;methods in general scenarios, yet continues to work well when occlusion occurs,&#10;which is ubiquitous in practice. The code and models are publicly available at&#10;https://github.com/Gengzigang/PCT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In this section, we first briefly discuss the widely used pose representations. Then we discuss the methods that explore joint dependencies.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Image Classification, Human Pose Estimation, Compositional Representation" />
-        </attvalues>
-      </node>
-      <node id="2102.10202" label="2102.10202">
-        <attvalues>
-          <attvalue for="0" value="Camera Calibration with Pose Guidance" />
-          <attvalue for="1" value="  Camera calibration plays a critical role in various computer vision tasks&#10;such as autonomous driving or augmented reality. Widely used camera calibration&#10;tools utilize plane pattern based methodology, such as using a chessboard or&#10;AprilTag board, user's calibration expertise level significantly affects&#10;calibration accuracy and consistency when without clear instruction.&#10;Furthermore, calibration is a recurring task that has to be performed each time&#10;the camera is changed or moved. It's also a great burden to calibrate huge&#10;amounts of cameras such as Driver Monitoring System (DMS) cameras in a&#10;production line with millions of vehicles. To resolve above issues, we propose&#10;a calibration system called Calibration with Pose Guidance to improve&#10;calibration accuracy, reduce calibration variance among different users or&#10;different trials of the same person. Experiment result shows that our proposed&#10;method achieves more accurate and consistent calibration than traditional&#10;calibration tools.&#10;" />
-          <attvalue for="2" value="&#10;Camera calibration models and estimates a camera's intrinsic and extrinsic parameters, and is an essential first step for many robotic and computer vision applications~\cite{liu2017calibration,sochor2017traffic,wang2017calibration, ren2013techniques}. Intrinsic parameters deal with the camera's internal characteristics, such as, its focal length, principle point, skew, and lens distortion~\cite{zhang2000flexible}. Extrinsic parameters describe camera's position and orientation~\cite{huang2019research,ramirez2020improve}. Knowing a camera's calibration parameters allows us to remove its lens distortion, which is necessary in many applications that demands accuracy such as vehicle or pedestrian detection in wide Field of View autonomous vehicle cameras. However, reliable and accurate camera calibration usually requires an expert intuition to reliably constrain all of the parameters in the camera model. Existing calibration toolboxes~\cite{bradski2000opencv, bouguet2004camera} ask users to capture images from a posed calibration pattern board (chessboard~\cite{zhang2000flexible, chen2019plane, li2011geometric}, circle grid pattern~\cite{ha2017deltille}, AprilTag~\cite{xie2018infrastructure, sagitov2017artag}, etc.) in positions of their choosing, after which the maximum-likelihood calibrations parameters are computed using all images in a batch optimization. Tan et al.~\cite{tan2017automatic} proposed to use monitor to display poses, however how to choose optimal poses to display on monitor screen is not considered. Richardson et al.~\cite{richardson2013aprilcal} and Rojtberg et al.~\cite{rojtberg2018efficient} proposed pose selection for interactive calibration which depends on a good pose initialization. The existing calibrators have common issues: 1) Calibration result consistency is not guaranteed if a tool is ran by users with different level of expertise. Even for the same user, different runs of the same tool may result in significant difference. 2) The widely used re-projection error alone is not sufficient to control estimated parameters' error. 3) User has to guess the chessboard pose and whether the pose number and variation can lead to a successful calibration, this is challenging especially for the novice without domain expertise. It can cause frustrated user experience and also make quality control hard~\cite{sturm1999plane}. &#10;&#10;In this work, we try to close these gaps by proposing the Calibration with Pose Guidance system, the diagram of which is shown in Fig.~\ref{flow}. In the first step, a set of optimal 3D virtual poses are selected using a novel score function which narrows down solution search space and avoid degenerated poses. In the second step, expected virtual poses are displayed on top of video streaming to guide the users to move calibration pattern respectively, with adjustment instructions shown in red arrows.&#10;&#10;We summarize our major contributions as follows: 1) A method to automatically generate an optimal set of poses for calibrating a camera is proposed. The pose set automatically avoids degenerated cases, such as feeding the images captured at the same place many times into the tool and helps narrow down solution search space for calibration optimization. 2) A novel score function to evaluate pose sets to find the optimized set for a specific application scenario. 3) A gamified Human Computer Interface (HCI) that is simple and straightforward to guide any user, no matter of its expertise, to capture sufficient and desired pre-defined poses accurately and consistently, with visual hints for adjustment of the pattern board for each pose. Our method saves a lot of training time for novices to conduct calibration. The optimal pose set achieves higher accuracy and consistency than human involved calibration method especially for the novices.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Camera Calibration, Autonomous Systems, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1702.06451" label="1702.06451">
-        <attvalues>
-          <attvalue for="0" value="Traffic Surveillance Camera Calibration by 3D Model Bounding Box&#10;  Alignment for Accurate Vehicle Speed Measurement" />
-          <attvalue for="1" value="  In this paper, we focus on fully automatic traffic surveillance camera&#10;calibration, which we use for speed measurement of passing vehicles. We improve&#10;over a recent state-of-the-art camera calibration method for traffic&#10;surveillance based on two detected vanishing points. More importantly, we&#10;propose a novel automatic scene scale inference method. The method is based on&#10;matching bounding boxes of rendered 3D models of vehicles with detected&#10;bounding boxes in the image. The proposed method can be used from arbitrary&#10;viewpoints, since it has no constraints on camera placement. We evaluate our&#10;method on the recent comprehensive dataset for speed measurement BrnoCompSpeed.&#10;Experiments show that our automatic camera calibration method by detection of&#10;two vanishing points reduces error by 50% (mean distance ratio error reduced&#10;from 0.18 to 0.09) compared to the previous state-of-the-art method. We also&#10;show that our scene scale inference method is more precise, outperforming both&#10;state-of-the-art automatic calibration method for speed measurement (error&#10;reduction by 86% -- 7.98km/h to 1.10km/h) and manual calibration (error&#10;reduction by 19% -- 1.35km/h to 1.10km/h). We also present qualitative results&#10;of the proposed automatic camera calibration method on video sequences obtained&#10;from real surveillance cameras in various places, and under different lighting&#10;conditions (night, dawn, day).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.04096" label="1907.04096">
-        <attvalues>
-          <attvalue for="0" value="Efficient Pose Selection for Interactive Camera Calibration" />
-          <attvalue for="1" value="  The choice of poses for camera calibration with planar patterns is only&#10;rarely considered - yet the calibration precision heavily depends on it. This&#10;work presents a pose selection method that finds a compact and robust set of&#10;calibration poses and is suitable for interactive calibration. Consequently,&#10;singular poses that would lead to an unreliable solution are avoided&#10;explicitly, while poses reducing the uncertainty of the calibration are&#10;favoured. For this, we use uncertainty propagation.&#10;  Our method takes advantage of a self-identifying calibration pattern to track&#10;the camera pose in real-time. This allows to iteratively guide the user to the&#10;target poses, until the desired quality level is reached. Therefore, only a&#10;sparse set of key-frames is needed for calibration.&#10;  The method is evaluated on separate training and testing sets, as well as on&#10;synthetic data. Our approach performs better than comparable solutions while&#10;requiring 30% less calibration frames.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.07937" label="2406.07937">
-        <attvalues>
-          <attvalue for="0" value="IFTD: Image Feature Triangle Descriptor for Loop Detection in Driving&#10;  Scenes" />
-          <attvalue for="1" value="  In this work, we propose a fast and robust Image Feature Triangle Descriptor&#10;(IFTD) based on the STD method, aimed at improving the efficiency and accuracy&#10;of place recognition in driving scenarios. We extract keypoints from BEV&#10;projection image of point cloud and construct these keypoints into triangle&#10;descriptors. By matching these feature triangles, we achieved precise place&#10;recognition and calculated the 4-DOF pose estimation between two keyframes.&#10;Furthermore, we employ image similarity inspection to perform the final place&#10;recognition. Experimental results on three public datasets demonstrate that our&#10;IFTD can achieve greater robustness and accuracy than state-of-the-art methods&#10;with low computational overhead.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Place recognition, also known as loop detection, is the process of identifying and matching the current position of a robot with previously visited locations in a known environment. Loop detection is important for correcting odometry drift and ensuring global consistency. Due to the widespread use of visual sensors, many vision-based loop detection methods, such as DBoW \cite{galvez2012DBoW2}, have become popular. However, visual sensors are susceptible to challenges such as lighting conditions, viewpoint changes, and occlusions, making loop detection difficult. Compared to visual sensors, LiDAR is invariant to lighting and appearance changes and can directly capture the structural information of the surrounding environment, thus enabling more accurate loop detection.&#10;LiDAR point based loop detection methods should possess the following three fundamental characteristics: (1) the descriptor for place recognition should possess rotational and translational invariance regardless of viewpoint changes. (2) Considering the complexity of 3D point clouds, the construction and retrieval of descriptors must be efficient to swiftly detect loops. (3) The similarity constraint should be sufficiently reliable to solve the relative pose between the current frame and the loop frame. Guided by the above fundamental requirements, Yuan et al. proposed the STD \cite{yuan2023std} algorithm, which directly extracts planar boundary feature points from 3D point clouds to construct feature triangles.The descriptors generated by this method possess invariance to rotation and translation, but the computational cost of extracting feature points and constructing feature triangles is relatively high. Furthermore, in complex environments such as forests, the method is difficult to identify stable planar edge feature points, limiting its applicability in such challenging scenarios.&#10;To address this issue, we propose IFTD, an image feature triangle descriptor based on BEV images, suitable for driving scenarios. First, we perform height-based layering of the 3D point cloud, followed by BEV projection based on the layered heights. Next, we extract Shi-Tomasi feature points \cite{shi1994good} from the BEV images, which are invariant to rotation and translation, and construct feature triangles from these feature points. Subsequently, we determine whether the loop closure has occurred based on the similarity of these feature triangles and BEV images..Experimental results on the NCLT \cite{carlevaris2016NCLT}, KITTI \cite{geiger2012KITTI}, and Mulran \cite{kim2020mulran} datasets demonstrate that IFTD shows excellent accuracy and robustness on most sequences compared to state-of-the-art methods.In addition, it is significantly better than STD in terms of speed and robustness.&#10;Specifically, our contributions are as follows:&#10;\begin{itemize}&#10;&#09;\item We designed a triangle descriptor based on BEV image features, which is more robust in complex environments and achieves faster computational efficiency.&#10;&#09;\item We proposed a rapid geometric verification method based on BEV projection images. which can quickly identify and match scene similarities and provide precise 4-DoF (x, y, z, and yaw) pose estimation.&#10;&#09;\item We have released the source code of our system to benefit the development of the community.&#10;\end{itemize}&#10;The rest of this paper is organized as follows: In Sec.\ref{Related Work}, we provide a brief overview of relevant literature. Sec.\ref{METHODOLOGY} elaborates on the proposed Image Feature Triangle Descriptor. Sec.\ref{Experiments} presents the experimental evaluation. Finally, we conclude the paper in Sec.\Ref{Conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Pose Estimation, Mathematics, Computer Science, Place Recognition, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2209.12435" label="2209.12435">
-        <attvalues>
-          <attvalue for="0" value="STD: Stable Triangle Descriptor for 3D place recognition" />
-          <attvalue for="1" value="  In this work, we present a novel global descriptor termed stable triangle&#10;descriptor (STD) for 3D place recognition. For a triangle, its shape is&#10;uniquely determined by the length of the sides or included angles. Moreover,&#10;the shape of triangles is completely invariant to rigid transformations. Based&#10;on this property, we first design an algorithm to efficiently extract local key&#10;points from the 3D point cloud and encode these key points into triangular&#10;descriptors. Then, place recognition is achieved by matching the side lengths&#10;(and some other information) of the descriptors between point clouds. The point&#10;correspondence obtained from the descriptor matching pair can be further used&#10;in geometric verification, which greatly improves the accuracy of place&#10;recognition. In our experiments, we extensively compare our proposed system&#10;against other state-of-the-art systems (i.e., M2DP, Scan Context) on public&#10;datasets (i.e., KITTI, NCLT, and Complex-Urban) and our self-collected dataset&#10;(with a non-repetitive scanning solid-state LiDAR). All the quantitative&#10;results show that STD has stronger adaptability and a great improvement in&#10;precision over its counterparts. To share our findings and make contributions&#10;to the community, we open source our code on our GitHub:&#10;https://github.com/hku-mars/STD.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.13115" label="2202.13115">
-        <attvalues>
-          <attvalue for="0" value="Analysis of Visual Reasoning on One-Stage Object Detection" />
-          <attvalue for="1" value="  Current state-of-the-art one-stage object detectors are limited by treating&#10;each image region separately without considering possible relations of the&#10;objects. This causes dependency solely on high-quality convolutional feature&#10;representations for detecting objects successfully. However, this may not be&#10;possible sometimes due to some challenging conditions. In this paper, the usage&#10;of reasoning features on one-stage object detection is analyzed. We attempted&#10;different architectures that reason the relations of the image regions by using&#10;self-attention. YOLOv3-Reasoner2 model spatially and semantically enhances&#10;features in the reasoning layer and fuses them with the original convolutional&#10;features to improve performance. The YOLOv3-Reasoner2 model achieves around&#10;2.5% absolute improvement with respect to baseline YOLOv3 on COCO in terms of&#10;mAP while still running in real-time.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object detection aims to classify and localize objects of interest in a given image. It has attracted great attention of the community because of its close ties with other computer vision applications. Many traditional methods have been proposed to solve object detection problem before the major breakthrough in deep learning area. These methods \cite{viola1, viola2, hog, dpm1, dpm2} were built on handcrafted feature representations. Inevitable dependency on handcrafted features limited the performance of traditional approaches. The great impact of the AlexNet \cite{alexnet} has put a new complexion on the object detection approaches and then deep learning based methods have completely dominated literature. Deep learning based detectors can be divided into two categories: two-stage object detectors and one-stage object detectors. Two-stage detectors have low inference speeds due to the intermediate layer used to propose possible object regions. Region proposal layer extracts regions of objects in the first stage. In the second stage, these proposed regions are used for classification and bounding box regression. On the other hand, one-stage detectors could predict all the bounding boxes and class probabilities in a single pass with high inference speeds. This makes one-stage detectors more suitable for real-time applications.&#10;&#10;Recent one-stage object detectors \cite{yolov1, ssd, yolov3, efficientdet, yolov4} achieve good performance on datasets such as MS COCO \cite{coco} and PASCAL VOC \cite{pascalvoc}. However, they lack of ability to consider possible relations between image regions. The current one-stage detectors treat each image region separately. They are unaware of distinct image regions due to small receptive fields when image size is considered. They depend solely on high-quality local convolutional features to detect objects successfully. However, this is not the way how human visual system works. Humans have an ability of reasoning to carry out visual tasks with the help of acquired knowledge. Many methods \cite{smn, relationnetwork, iterativevr, detr, vityolo} have been proposed to mimic human reasoning ability in object detection. On the other hand, these methods are mostly complicated and uses two-stage detection architectures. Thus, they are not applicable for real-time applications.&#10;&#10;In this paper, we propose a new approach to incorporate visual reasoning into one-stage object detection. Different from ViT-YOLO \cite{vityolo}, we integrate multi-head attention based reasoning layer on top of the neck instead of the backbone. By this way, reasoning information about the relationships between different image regions can be extracted by using more meaningful, fine-grained and enhanced feature maps. &#10;&#10;This paper's contributions can be summarized as follows:&#10;&#10;\begin{itemize}&#10;\item We present that one-stage object detection can be improved by visual reasoning. A new architecture is proposed that can extract semantic relations between image regions to predict bounding boxes and class probabilities. &#10;\item We analyze the effect of using only reasoning features on object detection performance. We demonstrate that fusing backbone output only-convolutional and reasoning features achieves the best performance improvement over the baseline model while still running in real-time. &#10;\item We analyze the effect of utilizing reasoning on average precision improvement for each object category.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1704.04224" label="1704.04224">
-        <attvalues>
-          <attvalue for="0" value="Spatial Memory for Context Reasoning in Object Detection" />
-          <attvalue for="1" value="  Modeling instance-level context and object-object relationships is extremely&#10;challenging. It requires reasoning about bounding boxes of different classes,&#10;locations \etc. Above all, instance-level spatial reasoning inherently requires&#10;modeling conditional distributions on previous detections. Unfortunately, our&#10;current object detection systems do not have any {\bf memory} to remember what&#10;to condition on! The state-of-the-art object detectors still detect all object&#10;in parallel followed by non-maximal suppression (NMS). While memory has been&#10;used for tasks such as captioning, they mostly use image-level memory cells&#10;without capturing the spatial layout. On the other hand, modeling object-object&#10;relationships requires {\bf spatial} reasoning -- not only do we need a memory&#10;to store the spatial layout, but also a effective reasoning module to extract&#10;spatial patterns. This paper presents a conceptually simple yet powerful&#10;solution -- Spatial Memory Network (SMN), to model the instance-level context&#10;efficiently and effectively. Our spatial memory essentially assembles object&#10;instances back into a pseudo &quot;image&quot; representation that is easy to be fed into&#10;another ConvNet for object-object context reasoning. This leads to a new&#10;sequential reasoning architecture where image and memory are processed in&#10;parallel to obtain detections which update the memory again. We show our SMN&#10;direction is promising as it provides 2.2\% improvement over baseline Faster&#10;RCNN on the COCO dataset so far.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.13004" label="2308.13004">
-        <attvalues>
-          <attvalue for="0" value="Spherical Vision Transformer for 360-degree Video Saliency Prediction" />
-          <attvalue for="1" value="  The growing interest in omnidirectional videos (ODVs) that capture the full&#10;field-of-view (FOV) has gained 360-degree saliency prediction importance in&#10;computer vision. However, predicting where humans look in 360-degree scenes&#10;presents unique challenges, including spherical distortion, high resolution,&#10;and limited labelled data. We propose a novel vision-transformer-based model&#10;for omnidirectional videos named SalViT360 that leverages tangent image&#10;representations. We introduce a spherical geometry-aware spatiotemporal&#10;self-attention mechanism that is capable of effective omnidirectional video&#10;understanding. Furthermore, we present a consistency-based unsupervised&#10;regularization term for projection-based 360-degree dense-prediction models to&#10;reduce artefacts in the predictions that occur after inverse projection. Our&#10;approach is the first to employ tangent images for omnidirectional saliency&#10;prediction, and our experimental results on three ODV saliency datasets&#10;demonstrate its effectiveness compared to the state-of-the-art.&#10;" />
-          <attvalue for="2" value="&#10;As an important computer vision task, visual saliency prediction aims at predicting where people look in a scene. It is widely used in various areas, such as saliency-guided image and video compression~\cite{7962230, zhu2018spatiotemporal, mishra2021multi, 8692638, patney2016towards}, super-resolution~\cite{wang2020visual, hou2022multi}, and quality assessment~\cite{yang2019sgdnet, zhu2021saliency, guan2017visual, 8572733, qiu2021blind} to exploit human perceptual features for enhancement. With the growing popularity of virtual reality (VR) applications and multimedia streaming, predicting saliency in $360^\circ$ videos has received more attention recently.&#10;One primary challenge in processing $360^\circ$ scenes is effectively representing omnidirectional data. Equirectangular Projection (ERP), where the full-FOV scene is projected on a 2D plane, is a common representation due to its computational simplicity. However, ERP suffers from spherical distortion, particularly towards the poles, which can significantly affect the geometric structure of the scene and degrade model performance. While previous works have proposed kernel transformations~\cite{su2017learning} and spherical convolutions~\cite{coors2018spherenet, esteves2018learning} to minimize this distortion on ERP, these methods come at the cost of computational complexity and the loss of global context in $360^\circ$ scenes. Cubemap projection~\cite{4056759} is another common approach that addresses the distortion problem by approximately expressing the spherical scene with six faces of a cube. Although this approach eliminates the distortion to an extent, it breaks the continuity of neighbouring faces and introduces discrepancies in the predictions around the edges.&#10;&#10;Previous works in $360^\circ$ saliency prediction primarily focused on addressing this representation problem, with each method trying to balance representative power and computational complexity. Chao~et~al.~\cite{chao2018salgan360} employed cubemap projection and fine-tuned the 2D image saliency model SalGAN~\cite{Pan_2017_SalGAN} on each cube face. Cheng~et~al.~\cite{cheng2018cube} extended cubemap projection with cube-padding to address the discontinuities on the face boundaries. Chao~et~al.~\cite{9122430} extended SalGAN360 with multi-view fusion. Dahou~et~al.~\cite{dahou2021atsal} proposed a two-stream architecture to compute global and local saliency in omnidirectional videos. Their approach uses global prediction as a rough attention estimate, and the local stream on cube faces predicts local saliency. Zhang~et~al.~\cite{Zhang_2018_ECCV} proposed spherical convolutions for saliency prediction. Qiao~et~al.~\cite{qiao2020viewport} showed that the eye fixation distribution bias depends on the viewport locations, which motivated us to introduce {spherical position information} into our model. Yun~et~al~\cite{yun2022panoramic} use local undistorted patches with deformable CNNs and use a ViT variant for self-attention across space and time. &#10;Djilali~et~al.~\cite{Djilali_2021_ICCV} used a self-supervised pre-training based on learning the association between several different views of the same scene and trained a supervised decoder for $360^\circ$ saliency prediction as a downstream task. Although their approach considers the global relationship between viewports, it ignores the temporal dimension that is crucial for video understanding.&#10;&#10;The methods mentioned above share a common limitation in processing $360^\circ$ data through projections or modified kernels and ignore the full field-of-view, which is critical for global scene understanding and saliency prediction. Thus, there is a need for an effective omnidirectional data processing method that minimizes spherical distortion while preserving the global context and avoiding computational overhead and artefacts introduced by the previous methods. Recently, Eder~et~al.~\cite{eder2020tangent} proposed tangent image representations, which use gnomonic projection to map a spherical image into multiple overlapping patches, where each patch is tangent to the faces of an icosahedron. This method tackles the problem of spherical distortion on the scene. However, to our interest, the dense-prediction models particularly suffer from discrepancies and artefacts on overlapping regions of tangent image patches after inverse projection to ERP. In this work, we propose using tangent images to process undistorted local viewports and develop a transformer-based model to learn their global association for saliency prediction in 360$^\circ$ videos. This is the first work that employs tangent images for omnidirectional saliency prediction, which is also capable of modelling the temporal dimension, motivated by the recent video transformer architectures, e.g., ~\cite{gberta_2021_ICML}.&#10;&#10;Proposed method and contributions: We use gnomonic projection to obtain multiple undistorted tangent images, which enables us to extract rich local spatial features using any pre-trained and fixed 2D backbone. Our extensive experiments&#10;demonstrate the effectiveness of our proposed SalViT360~model against the state-of-the-art on VR-EyeTracking~\cite{xu2018gaze}, PVS-HMEM~\cite{xu2018predicting}, and 360-AV-HM~\cite{9301766} datasets. Our contributions are three-fold:&#10;\begin{itemize}&#10; \item {Spherical geometry-aware spatial attention.} We aggregate global information on the sphere by computing self-attention among tangent viewports. We use tangent features as viewport tokens to address the quadratic complexity associated with spatial self-attention. We introduce $360^\circ$ geometry awareness to the transformer by using learnable spherical position embeddings guided by per-pixel angular coordinates $(\phi, \theta)$, denoting latitude and longitude angles, respectively. The proposed position embedding method outperforms standard 1D embeddings, and it can easily be integrated into any transformer architecture designed for $360^\circ$ images. &#10;&#10;\item {Viewport Spatio-Temporal Attention (VSTA).} The complexity of joint spatio-temporal self-attention increases quadratically with respect to the number of frames. With VSTA, we optimize this joint computation on tangent viewports from consecutive frames, where the spatio-temporal self-attention is performed in two stages: (1) Viewport Spatial Attention (VSA) and (2) Viewport Temporal Attention (VTA). In VSA, spherical geometry-aware self-attention is computed intra-frame level. The temporal information in the videos is encoded by a VTA, among tangent planes that point to the same direction in the inter-frame level.&#10;&#10;\item {Viewport Augmentation Consistency (VAC).} We propose an unsupervised, consistency-based loss for omnidirectional images, minimizing the discrepancies in overlapping regions of tangent predictions. The loss is computed between the weight-sharing saliency predictions of two tangent image sets which are generated with different configurations. This regularization method is suitable for any projection-based dense-prediction model. Importantly, VAC does not introduce any memory or time overhead during test time, as only one set of predictions is sufficient for inference.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Saliency Prediction, Human-Computer Interaction, Cognitive Science, Computer Science, Omnidirectional Videos" />
-        </attvalues>
-      </node>
-      <node id="1701.01081" label="1701.01081">
-        <attvalues>
-          <attvalue for="0" value="SalGAN: Visual Saliency Prediction with Generative Adversarial Networks" />
-          <attvalue for="1" value="  We introduce SalGAN, a deep convolutional neural network for visual saliency&#10;prediction trained with adversarial examples. The first stage of the network&#10;consists of a generator model whose weights are learned by back-propagation&#10;computed from a binary cross entropy (BCE) loss over downsampled versions of&#10;the saliency maps. The resulting prediction is processed by a discriminator&#10;network trained to solve a binary classification task between the saliency maps&#10;generated by the generative stage and the ground truth ones. Our experiments&#10;show how adversarial training allows reaching state-of-the-art performance&#10;across different metrics when combined with a widely-used loss function like&#10;BCE. Our results can be reproduced with the source code and trained models&#10;available at https://imatge-upc.github.io/saliency-salgan-2017/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.10600" label="2011.10600">
-        <attvalues>
-          <attvalue for="0" value="ATSal: An Attention Based Architecture for Saliency Prediction in 360&#10;  Videos" />
-          <attvalue for="1" value="  The spherical domain representation of 360 video/image presents many&#10;challenges related to the storage, processing, transmission and rendering of&#10;omnidirectional videos (ODV). Models of human visual attention can be used so&#10;that only a single viewport is rendered at a time, which is important when&#10;developing systems that allow users to explore ODV with head mounted displays&#10;(HMD). Accordingly, researchers have proposed various saliency models for 360&#10;video/images. This paper proposes ATSal, a novel attention based (head-eye)&#10;saliency model for 360\degree videos. The attention mechanism explicitly&#10;encodes global static visual attention allowing expert models to focus on&#10;learning the saliency on local patches throughout consecutive frames. We&#10;compare the proposed approach to other state-of-the-art saliency models on two&#10;datasets: Salient360! and VR-EyeTracking. Experimental results on over 80 ODV&#10;videos (75K+ frames) show that the proposed method outperforms the existing&#10;state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.09390" label="1912.09390">
-        <attvalues>
-          <attvalue for="0" value="Tangent Images for Mitigating Spherical Distortion" />
-          <attvalue for="1" value="  In this work, we propose &quot;tangent images,&quot; a spherical image representation&#10;that facilitates transferable and scalable $360^\circ$ computer vision.&#10;Inspired by techniques in cartography and computer graphics, we render a&#10;spherical image to a set of distortion-mitigated, locally-planar image grids&#10;tangent to a subdivided icosahedron. By varying the resolution of these grids&#10;independently of the subdivision level, we can effectively represent high&#10;resolution spherical images while still benefiting from the low-distortion&#10;icosahedral spherical approximation. We show that training standard&#10;convolutional neural networks on tangent images compares favorably to the many&#10;specialized spherical convolutional kernels that have been developed, while&#10;also scaling efficiently to handle significantly higher spherical resolutions.&#10;Furthermore, because our approach does not require specialized kernels, we show&#10;that we can transfer networks trained on perspective images to spherical data&#10;without fine-tuning and with limited performance drop-off. Finally, we&#10;demonstrate that tangent images can be used to improve the quality of sparse&#10;feature detection on spherical images, illustrating its usefulness for&#10;traditional computer vision tasks like structure-from-motion and SLAM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.05095" label="2102.05095">
-        <attvalues>
-          <attvalue for="0" value="Is Space-Time Attention All You Need for Video Understanding?" />
-          <attvalue for="1" value="  We present a convolution-free approach to video classification built&#10;exclusively on self-attention over space and time. Our method, named&#10;&quot;TimeSformer,&quot; adapts the standard Transformer architecture to video by&#10;enabling spatiotemporal feature learning directly from a sequence of&#10;frame-level patches. Our experimental study compares different self-attention&#10;schemes and suggests that &quot;divided attention,&quot; where temporal attention and&#10;spatial attention are separately applied within each block, leads to the best&#10;video classification accuracy among the design choices considered. Despite the&#10;radically new design, TimeSformer achieves state-of-the-art results on several&#10;action recognition benchmarks, including the best reported accuracy on&#10;Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks,&#10;our model is faster to train, it can achieve dramatically higher test&#10;efficiency (at a small drop in accuracy), and it can also be applied to much&#10;longer video clips (over one minute long). Code and models are available at:&#10;https://github.com/facebookresearch/TimeSformer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.10755" label="1710.10755">
-        <attvalues>
-          <attvalue for="0" value="Predicting Head Movement in Panoramic Video: A Deep Reinforcement&#10;  Learning Approach" />
-          <attvalue for="1" value="  Panoramic video provides immersive and interactive experience by enabling&#10;humans to control the field of view (FoV) through head movement (HM). Thus, HM&#10;plays a key role in modeling human attention on panoramic video. This paper&#10;establishes a database collecting subjects' HM in panoramic video sequences.&#10;From this database, we find that the HM data are highly consistent across&#10;subjects. Furthermore, we find that deep reinforcement learning (DRL) can be&#10;applied to predict HM positions, via maximizing the reward of imitating human&#10;HM scanpaths through the agent's actions. Based on our findings, we propose a&#10;DRL-based HM prediction (DHP) approach with offline and online versions, called&#10;offline-DHP and online-DHP. In offline-DHP, multiple DRL workflows are run to&#10;determine potential HM positions at each panoramic frame. Then, a heat map of&#10;the potential HM positions, named the HM map, is generated as the output of&#10;offline-DHP. In online-DHP, the next HM position of one subject is estimated&#10;given the currently observed HM position, which is achieved by developing a DRL&#10;algorithm upon the learned offline-DHP model. Finally, the experiments validate&#10;that our approach is effective in both offline and online prediction of HM&#10;positions for panoramic video, and that the learned offline-DHP model can&#10;improve the performance of online-DHP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.01530" label="2104.01530">
-        <attvalues>
-          <attvalue for="0" value="High-resolution Depth Maps Imaging via Attention-based Hierarchical&#10;  Multi-modal Fusion" />
-          <attvalue for="1" value="  Depth map records distance between the viewpoint and objects in the scene,&#10;which plays a critical role in many real-world applications. However, depth map&#10;captured by consumer-grade RGB-D cameras suffers from low spatial resolution.&#10;Guided depth map super-resolution (DSR) is a popular approach to address this&#10;problem, which attempts to restore a high-resolution (HR) depth map from the&#10;input low-resolution (LR) depth and its coupled HR RGB image that serves as the&#10;guidance.&#10;  The most challenging problems for guided DSR are how to correctly select&#10;consistent structures and propagate them, and properly handle inconsistent&#10;ones. In this paper, we propose a novel attention-based hierarchical&#10;multi-modal fusion (AHMF) network for guided DSR. Specifically, to effectively&#10;extract and combine relevant information from LR depth and HR guidance, we&#10;propose a multi-modal attention based fusion (MMAF) strategy for hierarchical&#10;convolutional layers, including a feature enhance block to select valuable&#10;features and a feature recalibration block to unify the similarity metrics of&#10;modalities with different appearance characteristics. Furthermore, we propose a&#10;bi-directional hierarchical feature collaboration (BHFC) module to fully&#10;leverage low-level spatial information and high-level structure information&#10;among multi-scale features. Experimental results show that our approach&#10;outperforms state-of-the-art methods in terms of reconstruction accuracy,&#10;running speed and memory efficiency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Image Super Resolution, Computer Science, Depth Map Enhancement, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2309.14745" label="2309.14745">
-        <attvalues>
-          <attvalue for="0" value="SSPFusion: A Semantic Structure-Preserving Approach for Infrared and&#10;  Visible Image Fusion" />
-          <attvalue for="1" value="  Most existing learning-based infrared and visible image fusion (IVIF) methods&#10;exhibit massive redundant information in the fusion images, i.e., yielding&#10;edge-blurring effect or unrecognizable for object detectors. To alleviate these&#10;issues, we propose a semantic structure-preserving approach for IVIF, namely&#10;SSPFusion. At first, we design a Structural Feature Extractor (SFE) to extract&#10;the structural features of infrared and visible images. Then, we introduce a&#10;multi-scale Structure-Preserving Fusion (SPF) module to fuse the structural&#10;features of infrared and visible images, while maintaining the consistency of&#10;semantic structures between the fusion and source images. Owing to these two&#10;effective modules, our method is able to generate high-quality fusion images&#10;from pairs of infrared and visible images, which can boost the performance of&#10;downstream computer-vision tasks. Experimental results on three benchmarks&#10;demonstrate that our method outperforms eight state-of-the-art image fusion&#10;methods in terms of both qualitative and quantitative evaluations. The code for&#10;our method, along with additional comparison results, will be made available&#10;at: https://github.com/QiaoYang-CV/SSPFUSION.&#10;" />
-          <attvalue for="2" value="&#10;&#10;To improve the ability of both human and computer vision systems to perceive their surrounding environment, multiple modalities of images, e.g., visible and infrared images, are frequently captured simultaneously to provide comprehensive information about the monitored scene~\cite{ZHANG2017227}.&#10;Researchers have developed various methods to fuse the pairs of infrared and visible images as high-quality images, which can significantly enrich the image details and support the downstream computer-vision tasks (e.g., object tracking~\cite{fusiontracking}, multi-target detection~\cite{TarDAL}, real-time semantic segmentation~\cite{SeAFusion}). Intuitively, images fused from different modalities of images can comprehensively reflect conditions of the monitored scene. &#10;&#10;Recently, in order to produce high-quality fused images, many learning-based IVIF methods have been developed. &#10;These methods~\cite{GTF, DIDFuse, IFCNN, RFN-Nest, icassppaper2, Diffusion} implicitly utilize various loss functions to maximize the structural similarity between the fused and original images. &#10;However, these loss functions are primarily employed to manipulate gradient or color consistency rather than directly acting on the underlying structural consistency, which leads to redundant information in the fused image and low structural similarity between the fused and original images. Although some works ~\cite{TarDAL, UMF-CMGR} pointed out that exploiting structural information could help improve the performance of IVIF methods, &#10;these methods encounter a challenge regarding structure inconsistency between the fused and source images.&#10;As shown in the red boxed Fig.~\ref{figure22222}, the tree structure is clearly showed in the visible image, but absent in the fused image of UMF~\cite{UMF-CMGR}. &#10;&#10;To address the above issues, we propose a semantic structure-preserving approach for IVIF, namely SSPFusion. &#10;First, we design a Structural Feature Extractor (SFE) to extract structural features for the infrared and visible images and provide supervision signals for fused structures throughout the fusion process. &#10;Second, we develop a novel Structure-Preserving Fusion (SPF) module. With the supervision signals provided by SFE, SPF integrates the structural features of the source images, which are further reconstructed as a fusion image with well-preserved strutures by the Decoder. Owing to the proposed SFE and SPF modules, our method can ultimately preserve the structural features from source images to the fusion image and generate high-quality fusion image, as shown in Fig.~\ref{figure22222} (f).&#10;Finally, extensive experiments verify that our method outperforms eight state-of-the-art image fusion methods on three public datasets, and can significantly boost the performance of semantic segmentation tasks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Science, Machine Learning, Computer Vision, Image Processing, Electrical Engineering, Deep Learning Methods, Computer Science, Image Fusion Techniques, Computer Vision Systems" />
-        </attvalues>
-      </node>
-      <node id="2203.16220" label="2203.16220">
-        <attvalues>
-          <attvalue for="0" value="Target-aware Dual Adversarial Learning and a Multi-scenario&#10;  Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection" />
-          <attvalue for="1" value="  This study addresses the issue of fusing infrared and visible images that&#10;appear differently for object detection. Aiming at generating an image of high&#10;visual quality, previous approaches discover commons underlying the two&#10;modalities and fuse upon the common space either by iterative optimization or&#10;deep networks. These approaches neglect that modality differences implying the&#10;complementary information are extremely important for both fusion and&#10;subsequent detection task. This paper proposes a bilevel optimization&#10;formulation for the joint problem of fusion and detection, and then unrolls to&#10;a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a&#10;commonly used detection network. The fusion network with one generator and dual&#10;discriminators seeks commons while learning from differences, which preserves&#10;structural information of targets from the infrared and textural details from&#10;the visible. Furthermore, we build a synchronized imaging system with&#10;calibrated infrared and optical sensors, and collect currently the most&#10;comprehensive benchmark covering a wide range of scenarios. Extensive&#10;experiments on several public datasets and our benchmark demonstrate that our&#10;method outputs not only visually appealing fusion but also higher detection mAP&#10;than the state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.11876" label="2205.11876">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Misaligned Infrared and Visible Image Fusion via&#10;  Cross-Modality Image Generation and Registration" />
-          <attvalue for="1" value="  Recent learning-based image fusion methods have marked numerous progress in&#10;pre-registered multi-modality data, but suffered serious ghosts dealing with&#10;misaligned multi-modality data, due to the spatial deformation and the&#10;difficulty narrowing cross-modality discrepancy. To overcome the obstacles, in&#10;this paper, we present a robust cross-modality generation-registration paradigm&#10;for unsupervised misaligned infrared and visible image fusion (IVIF).&#10;Specifically, we propose a Cross-modality Perceptual Style Transfer Network&#10;(CPSTN) to generate a pseudo infrared image taking a visible image as input.&#10;Benefiting from the favorable geometry preservation ability of the CPSTN, the&#10;generated pseudo infrared image embraces a sharp structure, which is more&#10;conducive to transforming cross-modality image alignment into mono-modality&#10;registration coupled with the structure-sensitive of the infrared image. In&#10;this case, we introduce a Multi-level Refinement Registration Network (MRRN) to&#10;predict the displacement vector field between distorted and pseudo infrared&#10;images and reconstruct registered infrared image under the mono-modality&#10;setting. Moreover, to better fuse the registered infrared images and visible&#10;images, we present a feature Interaction Fusion Module (IFM) to adaptively&#10;select more meaningful features for fusion in the Dual-path Interaction Fusion&#10;Network (DIFN). Extensive experimental results suggest that the proposed method&#10;performs superior capability on misaligned cross-modality image fusion.&#10;" />
-          <attvalue for="2" value="&#10;Image fusion is to extract the complementary information from different modality images and aggregate them to generate richer and more meaningful feature representations. Typically, Infrared and Visible Image Fusion (IVIF) enjoys the merit and presents conduciveness to practical applications such as autonomous driving and video surveillance.&#10;&#10;Most existing IVIF methods~\cite{liu2020bilevel,liu2021searching,liu2022attention} are designed for the hand-crafted pre-registered images, which are very labor-intensive and time-sensitive, despite numerous progress.&#10;And, they are sensitive to the discrepancy of intensity of the misaligned infrared and visible images, resulting in serious ghosting artifacts on fused images, once slight offset and deformation exist.&#10;The intrinsic reason is that the large cross-modality variation between infrared and visible images makes it impractical to straightly bridge the domain gap between them in a shared feature space. Coupled with the absence of cross-modality similarity constraints, few attempts have ever been made to fuse the misaligned infrared and visible images. The primary obstacle is cross-modality image alignment.&#10;&#10;More often than not, existing widely-used image alignment methods~\cite{flownet,voxelmorph} perform pixel-wise and feature-level alignment by explicitly estimating the deformation field between the distorted image and its reference. However, they are worked under only a single modality setting~\cite{BlockMix_20} due to their high dependence on the similarity of distribution and appearance on either synthetic or real data with neighborhood reference.&#10;Moreover, the designs of the similarity measures~\cite{TANG2022108792} to optimize the cross-modality alignment process have been proven to be quite challenging.&#10;These obstacles gave rise to the development of cross-modality medical image translation~\cite{medireg,LiuLZFL20,Liu_pami_21} and cross-modality Re-ID tasks~\cite{AlignGAN,D2RL}. The former leverages the cGAN~\cite{pix2pix} to perform NeuroImage-to-NeuroImage translation, while the latter learns the cross-modality match in pedestrian images of a person across disjoint camera views by an RGB-to-IR (infrared) translation. A recent study~\cite{NeMAR} presents a multi-modality image registration method by training an image-to-image translation network on the two input modalities.&#10;The basic idea of the above-mentioned approaches is using the image-to-image translation to implement cross-modality transformation to narrow the large variation between different modalities. &#10;&#10;Motivated by this idea, we propose a specialized cross-modality generation-registration paradigm (CGRP) for unsupervised misaligned infrared and visible image fusion.&#10;An inherent characteristic of infrared images is the emphasis on sharp geometry structures over texture details.&#10;Generating a pseudo infrared image preserving sharp structure information is more conducive to implementing mono-modality registration between it with the corresponding distorted infrared image.&#10;To better preserve geometry structure during generating pseudo infrared images from visible images, we proposed a Cross-modality Perceptual Style Transfer Network (CPSTN), which inherits the basic cycle consistent learning manner of the CycleGAN~\cite{cyclegan}, while developing a perceptual style transfer constraint and cross regularizations across two-cycle learning path to further guide the CPSTN to generate sharper structures. Without them, the generation capability of our CPSTN will degrade to that in~\cite{NeMAR}. &#10;Doing this lays the foundation for the mono-modality registration of infrared images. We utilize a Multi-level Refinement Registration Network (MRRN) to predict the deformation field from coarse to fine between distorted and pseudo infrared images and reconstruct the registered infrared image. &#10;In this case, we further perform the fusion of the registered infrared image and visible image.&#10;To enable the fusion network to focus more on faithful texture details, while avoiding feature smoothing caused by some unsophisticated fusion rules (e.g., concatenation, weighted average), we develop an Interaction Fusion Module (IFM) to adaptively select meaningful features from infrared and visible images to achieve fused results with sharp textures.&#10;We evaluate the proposed method on manually processed unaligned datasets and demonstrate its strengths through comprehensive analysis. &#10;The main contributions are summarized as follows:&#10;\begin{itemize}&#10;&#09;\item We propose a highly-robust unsupervised infrared and visible image fusion framework, which is more focused on mitigating ghosting artifacts caused by the fusion of misaligned image pairs compared with learning-based fusion methods specialized for pre-registered images.&#10;&#09;\item Considering the difficulty of the cross-modality image alignment, we exploit a specialized cross-modality generation-registration paradigm to bridge the large discrepancy between modalities, thus achieving effective infrared and visible image alignment. &#09;&#10;&#09;\item An interaction fusion module is developed to adaptively fuse multi-modality features, which avoids feature smoothing caused by unsophisticated fusion rules and emphasizes faithful texture details.&#10;\end{itemize}&#10;The extensive experimental results demonstrate that the proposed method performs superior capability on misaligned cross-modality image fusion.&#10;The code will be available at.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.10734" label="2110.10734">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervision and Spatial-Sequential Attention Based Loss for&#10;  Multi-Person Pose Estimation" />
-          <attvalue for="1" value="  Bottom-up based multi-person pose estimation approaches use heatmaps with&#10;auxiliary predictions to estimate joint positions and belonging at one time.&#10;Recently, various combinations between auxiliary predictions and heatmaps have&#10;been proposed for higher performance, these predictions are supervised by the&#10;corresponding L2 loss function directly. However, the lack of more explicit&#10;supervision results in low features utilization and contradictions between&#10;predictions in one model. To solve these problems, this paper proposes (i) a&#10;new loss organization method which uses self-supervised heatmaps to reduce&#10;prediction contradictions and spatial-sequential attention to enhance networks'&#10;features extraction; (ii) a new combination of predictions composed by&#10;heatmaps, Part Affinity Fields (PAFs) and our block-inside offsets to fix&#10;pixel-level joints positions and further demonstrates the effectiveness of&#10;proposed loss function. Experiments are conducted on the MS COCO keypoint&#10;dataset and adopting OpenPose as the baseline model. Our method outperforms the&#10;baseline overall. On the COCO verification dataset, the mAP of OpenPose trained&#10;with our proposals outperforms the OpenPose baseline by over 5.5%.&#10;" />
-          <attvalue for="2" value="&#10;M{ulti-Person} pose estimation refers to estimating pixel-level joints positions for each person in images. Specifically, the number of people is larger than one in each image, which makes judging joints belonging and locating joints positions to become two necessary tasks. &#10;Multi-Person estimation has received widespread attention in plenty of applications, such as sports analysis\cite{zhu2007human}, human-computer interaction\cite{cai2015effective}, and video surveillance\cite{gualdi2008video}. &#10;With the development of Convolutional Neural Networks (CNNs), most of recent pose estimation frameworks are based on CNNs\cite{huang2017coarse,carreira2016human,chu2017multi,lifshitz2016human,belagiannis2017recurrent,ramakrishna2014pose} and achieve superior performance. These methods to generate multi-person pose could be divided into two types: Top-Down methods\cite{newell2016stacked,he2017mask,fang2017rmpe,chen2018cascaded,papandreou2017towards,xiao2018simple,li2019crowdpose,li2019rethinking,zhang2019human,chou2018self} and Bottom-Up methods\cite{cao2017realtime,cao2018openpose,newell2017associative,kocabas2018multiposenet,papandreou2018personlab,nie2019single,kreiss2019pifpaf,varadarajan2018greedy,gkioxari2013articulated, liu2020resolution}. The difference between these two kinds of methods is whether the time consumption increases linearly with the number of people in the image. Top-Down methods will generate human boundary boxes\cite{fang2017rmpe} or proposals\cite{he2017mask} in the first step and then estimate joints positions for each proposal, the processing speed is related to the number of boundary boxes or proposals in one image. For the Bottom-Up methods, except estimating joints positions predictions, the model will also generate one or more auxiliary predictions for assembling joints, these procedures will be finished by only look at the image once. In this paper, we focus on the Bottom-Up methods, which are commonly used in real-time applications\cite{angelini20192d} because of its speed advantages. &#10;&#10;However, the Bottom-Up methods are difficult to realize as high accuracy as Top-Down methods for the following reasons. Firstly, the addition of auxiliary predictions leads the indeterminate influence to the final accuracy. The Associative Embedding\cite{newell2017associative} has joints jitter error even use groundtruth joints positions as inputs, which means the auxiliary predictions they designed leading to negative influences to joints positions estimation. However, the Part Affinity fields (PAFs), have been proofed that it has a positive influence to locate joints in the OpenPose model\cite{cao2017realtime}. Secondly, different person size varies in one image dramatically. For example, the smallest person only occupies about 1/100 area of one image but the largest person could occupy the image with the upper limb. Lastly, the single person joints refinement is easier to implement compares with the multi-person case. For example, the Online Hard Example Mining (OHEM)\cite{chen2018cascaded} and Graph Neural Networks (GNNs)\cite{zhang2019human} have been applied for single person pose refinement, however, these methods are hard to implement in the multi-person case directly because of the uncertainty of joints numbers. &#10;&#10;Much research has been investigated on the Bottom-Up methods because of speed stability in real-world applications. The tradition method of Bottom-Up based pose estimation is part-based\cite{hua2005learning,felzenszwalb2005pictorial,ramanan2005strike,andriluka2010monocular,andriluka2009pictorial,pishchulin2013poselet,yang2012articulated,johnson2010clustered}. They accumulate prior knowledge of 2D body shapes for their data-driven Monte Carlo algorithm. Since Deep Neural Networks (DNNs)\cite{hinton2012improving} have achieved superior performance in prior knowledge learning, it is expanded into various fields. Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously. At the same time, the MicroSort COCO keypoints detection dataset\cite{ruggero2017benchmarking} and the benchmark has been published, The OpenPose\cite{cao2017realtime}, which combined part-based methods and CNNs, archived first place in the COCO keypoints task in 2016. The network will generate Part Affinity Fields (PAFs) as auxiliary predictions for assembling joints. &#10;After 2017, the research of Bottom-Up based pose estimation is main focus on proposing more efficient auxiliary predictions\cite{newell2017associative,papandreou2018personlab,nie2019single,kreiss2019pifpaf}. Associative Embedding\cite{newell2017associative} uses several equal resolution images as auxiliary predictions, the difference between pixels value in the joints region stands for the different person belonging. Recently, the single-stage multi-person pose machines\cite{nie2019single} proposes a new auxiliary prediction by embedding the limb length information into previous part-based predictions. The PifPaf\cite{kreiss2019pifpaf} proposes a combination of auxiliary predictions by combining G-RMI\cite{papandreou2017towards} proposed offsets and their modified middle-level offsets\cite{papandreou2018personlab}. &#10;&#10;So far, lots of auxiliary predictions combinations' effectivity has been proofed. The simplest loss function, e.g., L2 loss or L1 loss, is commonly used in these works. These simple loss function selections are usually for controlling experimental variables but result in several problems.&#10;&#10;Contradiction between predictions. Firstly, the relationship between auxiliary predictions and heatmaps (joints location predictions) has not been supervised to learn from the loss function level. The results of OpenPose demonstrate that there have contradictions between PAFmaps and heatmaps results, i.e., The heatmaps and PAFmaps results stand for 'person-exist' regions for one image. In some cases, the heatmaps results are correct while the PAFmaps results are incorrect. &#10;&#10;Inefficient features utilization. The post-processing algorithm, which uses the auxiliary predictions and heatmaps to generate the final pose result, is necessary for Bottom-Up based pose estimation methods. But not all information (pixels value) of auxiliary predictions has been used in the post-processing algorithm\cite{cao2017realtime,cao2018openpose,wang2008multiple,newell2017associative,nie2019single,kreiss2019pifpaf}, which means different regions of auxiliary predictions have different importance but be treated the same way by simple loss functions. &#10;&#10;Demerits for multi-stage loss. Excepting generate several auxiliary predictions, most State-Of-The-Art (SOTA) pose networks use multi-stage structures\cite{cao2017realtime,newell2017associative,nie2019single} to refine predictions. The different stages are usually supervised by the same loss function, e.g., Mean Square Error (MSE) loss. The performance of OpenPose\cite{cao2017realtime} and Hourglass\cite{newell2016stacked} demonstrate that there is only about 0.6\% mAP improvement for the later network stages. i.e., stage 4 to 6 in the OpenPose model and stage 5 to 8 in the Hourglass model. Adopting the same loss function in a multi-stage network leads to the unbalanced learning difficulty for different stages, there are few new features for the final several stages to learn for refining the performance\cite{li2019rethinking}.&#10;&#10;Demerits for higher-level supervision. In our opinion, both auxiliary predictions and loss functions belong to supervision strategies. The auxiliary predictions are high-level supervision that defines the network outputs' types; the loss functions are lower-level supervision that defines how does the network generate expectant outputs\cite{lin2017focal}. For the high-level supervision, the resolution of inputs and outputs predictions in the previous works\cite{cao2017realtime,kreiss2019pifpaf} is different because of the down-sample layers in the network. There has an inevitable up-sample error for recovering the predictions to the resolution of inputs, which is a negative impact on the model performance. &#10;&#10;In summary, to solve the aforementioned challenges for Bottom-up based pose estimation methods, this paper proposes the self-supervision and spatial-sequential attention based loss function for the three problems in lower-level supervision. Besides, we make a new combination of predictions composed of heatmaps, Part Affinity Fields (PAFs), and our proposed block-inside offsets, which is called resolution irrelevant encoding, to fix pixel-level joints positions. The new combination is also for further demonstrating the effectiveness of our proposed loss functions. These proposals are network-agnostic and could be implemented for common Bottom-Up\cite{cao2017realtime,papandreou2018personlab,kocabas2018multiposenet,kreiss2019pifpaf,wang2018magnify} frameworks. &#10;&#10;Overall, as shown in Fig. \ref{fig:concept}, this paper's contributions are two folds:&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Pose Estimation, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1507.06550" label="1507.06550">
-        <attvalues>
-          <attvalue for="0" value="Human Pose Estimation with Iterative Error Feedback" />
-          <attvalue for="1" value="  Hierarchical feature extractors such as Convolutional Networks (ConvNets)&#10;have achieved impressive performance on a variety of classification tasks using&#10;purely feedforward processing. Feedforward architectures can learn rich&#10;representations of the input space but do not explicitly model dependencies in&#10;the output spaces, that are quite structured for tasks such as articulated&#10;human pose estimation or object segmentation. Here we propose a framework that&#10;expands the expressive power of hierarchical feature extractors to encompass&#10;both input and output spaces, by introducing top-down feedback. Instead of&#10;directly predicting the outputs in one go, we use a self-correcting model that&#10;progressively changes an initial solution by feeding back error predictions, in&#10;a process we call Iterative Error Feedback (IEF). IEF shows excellent&#10;performance on the task of articulated pose estimation in the challenging MPII&#10;and LSP benchmarks, matching the state-of-the-art without requiring ground&#10;truth scale annotation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.07432" label="1702.07432">
-        <attvalues>
-          <attvalue for="0" value="Multi-Context Attention for Human Pose Estimation" />
-          <attvalue for="1" value="  In this paper, we propose to incorporate convolutional neural networks with a&#10;multi-context attention mechanism into an end-to-end framework for human pose&#10;estimation. We adopt stacked hourglass networks to generate attention maps from&#10;features at multiple resolutions with various semantics. The Conditional Random&#10;Field (CRF) is utilized to model the correlations among neighboring regions in&#10;the attention map. We further combine the holistic attention model, which&#10;focuses on the global consistency of the full human body, and the body part&#10;attention model, which focuses on the detailed description for different body&#10;parts. Hence our model has the ability to focus on different granularity from&#10;local salient regions to global semantic-consistent spaces. Additionally, we&#10;design novel Hourglass Residual Units (HRUs) to increase the receptive field of&#10;the network. These units are extensions of residual units with a side branch&#10;incorporating filters with larger receptive fields, hence features with various&#10;scales are learned and combined within the HRUs. The effectiveness of the&#10;proposed multi-context attention mechanism and the hourglass residual units is&#10;evaluated on two widely used human pose estimation benchmarks. Our approach&#10;outperforms all existing methods on both benchmarks over all the body parts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.08212" label="1603.08212">
-        <attvalues>
-          <attvalue for="0" value="Human Pose Estimation using Deep Consensus Voting" />
-          <attvalue for="1" value="  In this paper we consider the problem of human pose estimation from a single&#10;still image. We propose a novel approach where each location in the image votes&#10;for the position of each keypoint using a convolutional neural net. The voting&#10;scheme allows us to utilize information from the whole image, rather than rely&#10;on a sparse set of keypoint locations. Using dense, multi-target votes, not&#10;only produces good keypoint predictions, but also enables us to compute&#10;image-dependent joint keypoint probabilities by looking at consensus voting.&#10;This differs from most previous methods where joint probabilities are learned&#10;from relative keypoint locations and are independent of the image. We finally&#10;combine the keypoints votes and joint probabilities in order to identify the&#10;optimal pose configuration. We show our competitive performance on the MPII&#10;Human Pose and Leeds Sports Pose datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.02914" label="1605.02914">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Human Pose Estimation" />
-          <attvalue for="1" value="  We propose a novel ConvNet model for predicting 2D human body poses in an&#10;image. The model regresses a heatmap representation for each body keypoint, and&#10;is able to learn and represent both the part appearances and the context of the&#10;part configuration. We make the following three contributions: (i) an&#10;architecture combining a feed forward module with a recurrent module, where the&#10;recurrent module can be run iteratively to improve the performance, (ii) the&#10;model can be trained end-to-end and from scratch, with auxiliary losses&#10;incorporated to improve performance, (iii) we investigate whether keypoint&#10;visibility can also be predicted. The model is evaluated on two benchmark&#10;datasets. The result is a simple architecture that achieves performance on par&#10;with the state of the art, but without the complexity of a graphical model&#10;stage (or layers).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07319" label="1711.07319">
-        <attvalues>
-          <attvalue for="0" value="Cascaded Pyramid Network for Multi-Person Pose Estimation" />
-          <attvalue for="1" value="  The topic of multi-person pose estimation has been largely improved recently,&#10;especially with the development of convolutional neural network. However, there&#10;still exist a lot of challenging cases, such as occluded keypoints, invisible&#10;keypoints and complex background, which cannot be well addressed. In this&#10;paper, we present a novel network structure called Cascaded Pyramid Network&#10;(CPN) which targets to relieve the problem from these &quot;hard&quot; keypoints. More&#10;specifically, our algorithm includes two stages: GlobalNet and RefineNet.&#10;GlobalNet is a feature pyramid network which can successfully localize the&#10;&quot;simple&quot; keypoints like eyes and hands but may fail to precisely recognize the&#10;occluded or invisible keypoints. Our RefineNet tries explicitly handling the&#10;&quot;hard&quot; keypoints by integrating all levels of feature representations from the&#10;GlobalNet together with an online hard keypoint mining loss. In general, to&#10;address the multi-person pose estimation problem, a top-down pipeline is&#10;adopted to first generate a set of human bounding boxes based on a detector,&#10;followed by our CPN for keypoint localization in each human bounding box. Based&#10;on the proposed algorithm, we achieve state-of-art results on the COCO keypoint&#10;benchmark, with average precision at 73.0 on the COCO test-dev dataset and 72.1&#10;on the COCO test-challenge dataset, which is a 19% relative improvement&#10;compared with 60.5 from the COCO 2016 keypoint challenge.Code&#10;(https://github.com/chenyilun95/tf-cpn.git) and the detection results are&#10;publicly available for further research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.00324" label="1812.00324">
-        <attvalues>
-          <attvalue for="0" value="CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark" />
-          <attvalue for="1" value="  Multi-person pose estimation is fundamental to many computer vision tasks and&#10;has made significant progress in recent years. However, few previous methods&#10;explored the problem of pose estimation in crowded scenes while it remains&#10;challenging and inevitable in many scenarios. Moreover, current benchmarks&#10;cannot provide an appropriate evaluation for such cases. In this paper, we&#10;propose a novel and efficient method to tackle the problem of pose estimation&#10;in the crowd and a new dataset to better evaluate algorithms. Our model&#10;consists of two key components: joint-candidate single person pose estimation&#10;(SPPE) and global maximum joints association. With multi-peak prediction for&#10;each joint and global association using graph model, our method is robust to&#10;inevitable interference in crowded scenes and very efficient in inference. The&#10;proposed method surpasses the state-of-the-art methods on CrowdPose dataset by&#10;5.2 mAP and results on MSCOCO dataset demonstrate the generalization ability of&#10;our method. Source code and dataset will be made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.00148" label="1901.00148">
-        <attvalues>
-          <attvalue for="0" value="Rethinking on Multi-Stage Networks for Human Pose Estimation" />
-          <attvalue for="1" value="  Existing pose estimation approaches fall into two categories: single-stage&#10;and multi-stage methods. While multi-stage methods are seemingly more suited&#10;for the task, their performance in current practice is not as good as&#10;single-stage methods. This work studies this issue. We argue that the current&#10;multi-stage methods' unsatisfactory performance comes from the insufficiency in&#10;various design choices. We propose several improvements, including the&#10;single-stage module design, cross stage feature aggregation, and coarse-to-fine&#10;supervision. The resulting method establishes the new state-of-the-art on both&#10;MS COCO and MPII Human Pose dataset, justifying the effectiveness of a&#10;multi-stage architecture. The source code is publicly available for further&#10;research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.01760" label="1901.01760">
-        <attvalues>
-          <attvalue for="0" value="Human Pose Estimation with Spatial Contextual Information" />
-          <attvalue for="1" value="  We explore the importance of spatial contextual information in human pose&#10;estimation. Most state-of-the-art pose networks are trained in a multi-stage&#10;manner and produce several auxiliary predictions for deep supervision. With&#10;this principle, we present two conceptually simple and yet computational&#10;efficient modules, namely Cascade Prediction Fusion (CPF) and Pose Graph Neural&#10;Network (PGNN), to exploit underlying contextual information. Cascade&#10;prediction fusion accumulates prediction maps from previous stages to extract&#10;informative signals. The resulting maps also function as a prior to guide&#10;prediction at following stages. To promote spatial correlation among joints,&#10;our PGNN learns a structured representation of human pose as a graph. Direct&#10;message passing between different joints is enabled and spatial relation is&#10;captured. These two modules require very limited computational complexity.&#10;Experimental results demonstrate that our method consistently outperforms&#10;previous methods on MPII and LSP benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.02439" label="1707.02439">
-        <attvalues>
-          <attvalue for="0" value="Self Adversarial Training for Human Pose Estimation" />
-          <attvalue for="1" value="  This paper presents a deep learning based approach to the problem of human&#10;pose estimation. We employ generative adversarial networks as our learning&#10;paradigm in which we set up two stacked hourglass networks with the same&#10;architecture, one as the generator and the other as the discriminator. The&#10;generator is used as a human pose estimator after the training is done. The&#10;discriminator distinguishes ground-truth heatmaps from generated ones, and&#10;back-propagates the adversarial loss to the generator. This process enables the&#10;generator to learn plausible human body configurations and is shown to be&#10;useful for improving the prediction accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08225" label="1803.08225">
-        <attvalues>
-          <attvalue for="0" value="PersonLab: Person Pose Estimation and Instance Segmentation with a&#10;  Bottom-Up, Part-Based, Geometric Embedding Model" />
-          <attvalue for="1" value="  We present a box-free bottom-up approach for the tasks of pose estimation and&#10;instance segmentation of people in multi-person images using an efficient&#10;single-shot model. The proposed PersonLab model tackles both semantic-level&#10;reasoning and object-part associations using part-based modeling. Our model&#10;employs a convolutional network which learns to detect individual keypoints and&#10;predict their relative displacements, allowing us to group keypoints into&#10;person pose instances. Further, we propose a part-induced geometric embedding&#10;descriptor which allows us to associate semantic person pixels with their&#10;corresponding person instance, delivering instance-level person segmentations.&#10;Our system is based on a fully-convolutional architecture and allows for&#10;efficient inference, with runtime essentially independent of the number of&#10;people present in the scene. Trained on COCO data alone, our system achieves&#10;COCO test-dev keypoint average precision of 0.665 using single-scale inference&#10;and 0.687 using multi-scale inference, significantly outperforming all previous&#10;bottom-up pose estimation systems. We are also the first bottom-up method to&#10;report competitive results for the person class in the COCO instance&#10;segmentation task, achieving a person category average precision of 0.417.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09220" label="1908.09220">
-        <attvalues>
-          <attvalue for="0" value="Single-Stage Multi-Person Pose Machines" />
-          <attvalue for="1" value="  Multi-person pose estimation is a challenging problem. Existing methods are&#10;mostly two-stage based--one stage for proposal generation and the other for&#10;allocating poses to corresponding persons. However, such two-stage methods&#10;generally suffer low efficiency. In this work, we present the first&#10;single-stage model, Single-stage multi-person Pose Machine (SPM), to simplify&#10;the pipeline and lift the efficiency for multi-person pose estimation. To&#10;achieve this, we propose a novel Structured Pose Representation (SPR) that&#10;unifies person instance and body joint position representations. Based on SPR,&#10;we develop the SPM model that can directly predict structured poses for&#10;multiple persons in a single stage, and thus offer a more compact pipeline and&#10;attractive efficiency advantage over two-stage methods. In particular, SPR&#10;introduces the root joints to indicate different person instances and human&#10;body joint positions are encoded into their displacements w.r.t. the roots. To&#10;better predict long-range displacements for some joints, SPR is further&#10;extended to hierarchical representations. Based on SPR, SPM can efficiently&#10;perform multi-person poses estimation by simultaneously predicting root joints&#10;(location of instances) and body joint displacements via CNNs. Moreover, to&#10;demonstrate the generality of SPM, we also apply it to multi-person 3D pose&#10;estimation. Comprehensive experiments on benchmarks MPII, extended&#10;PASCAL-Person-Part, MSCOCO and CMU Panoptic clearly demonstrate the&#10;state-of-the-art efficiency of SPM for multi-person 2D/3D pose estimation,&#10;together with outstanding accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06593" label="1903.06593">
-        <attvalues>
-          <attvalue for="0" value="PifPaf: Composite Fields for Human Pose Estimation" />
-          <attvalue for="1" value="  We propose a new bottom-up method for multi-person 2D human pose estimation&#10;that is particularly well suited for urban mobility such as self-driving cars&#10;and delivery robots. The new method, PifPaf, uses a Part Intensity Field (PIF)&#10;to localize body parts and a Part Association Field (PAF) to associate body&#10;parts with each other to form full human poses. Our method outperforms previous&#10;methods at low resolution and in crowded, cluttered and occluded scenes thanks&#10;to (i) our new composite field PAF encoding fine-grained information and (ii)&#10;the choice of Laplace loss for regressions which incorporates a notion of&#10;uncertainty. Our architecture is based on a fully convolutional, single-shot,&#10;box-free design. We perform on par with the existing state-of-the-art bottom-up&#10;method on the standard COCO keypoint task and produce state-of-the-art results&#10;on a modified COCO keypoint task for the transportation domain.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.09182" label="1708.09182">
-        <attvalues>
-          <attvalue for="0" value="A Greedy Part Assignment Algorithm for Real-time Multi-person 2D Pose&#10;  Estimation" />
-          <attvalue for="1" value="  Human pose-estimation in a multi-person image involves detection of various&#10;body parts and grouping them into individual person clusters. While the former&#10;task is challenging due to mutual occlusions, the combinatorial complexity of&#10;the latter task is very high. We propose a greedy part assignment algorithm&#10;that exploits the inherent structure of the human body to achieve a lower&#10;complexity, compared to any of the prior published works. This is accomplished&#10;by (i) reducing the number of part-candidates using the estimated number of&#10;people in the image, (ii) doing a greedy sequential assignment of part-classes,&#10;following the kinematic chain from head to ankle (iii) doing a greedy&#10;assignment of parts in each part-class set, to person-clusters (iv) limiting&#10;the candidate person clusters to the most proximal clusters using human&#10;anthropometric data and (v) using only a specific subset of pre-assigned parts&#10;for establishing pairwise structural constraints. We show that, these steps&#10;result in a sparse body parts relationship graph and reduces the complexity. We&#10;also propose methods for improving the accuracy of pose-estimation by (i)&#10;spawning person-clusters from any unassigned significant body part and (ii)&#10;suppressing hallucinated parts. On the MPII multi-person pose database,&#10;pose-estimation using the proposed method takes only 0.14 seconds per image. We&#10;show that, our proposed algorithm, by using a large spatial and structural&#10;context, achieves the state-of-the-art accuracy on both MPII and WAF&#10;multi-person pose datasets, demonstrating the robustness of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1207.0580" label="1207.0580">
-        <attvalues>
-          <attvalue for="0" value="Improving neural networks by preventing co-adaptation of feature&#10;  detectors" />
-          <attvalue for="1" value="  When a large feedforward neural network is trained on a small training set,&#10;it typically performs poorly on held-out test data. This &quot;overfitting&quot; is&#10;greatly reduced by randomly omitting half of the feature detectors on each&#10;training case. This prevents complex co-adaptations in which a feature detector&#10;is only helpful in the context of several other specific feature detectors.&#10;Instead, each neuron learns to detect a feature that is generally helpful for&#10;producing the correct answer given the combinatorially large variety of&#10;internal contexts in which it must operate. Random &quot;dropout&quot; gives big&#10;improvements on many benchmark tasks and sets new records for speech and object&#10;recognition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.4280" label="1411.4280">
-        <attvalues>
-          <attvalue for="0" value="Efficient Object Localization Using Convolutional Networks" />
-          <attvalue for="1" value="  Recent state-of-the-art performance on human-body pose estimation has been&#10;achieved with Deep Convolutional Networks (ConvNets). Traditional ConvNet&#10;architectures include pooling and sub-sampling layers which reduce&#10;computational requirements, introduce invariance and prevent over-training.&#10;These benefits of pooling come at the cost of reduced localization accuracy. We&#10;introduce a novel architecture which includes an efficient `position&#10;refinement' model that is trained to estimate the joint offset location within&#10;a small region of the image. This refinement model is jointly trained in&#10;cascade with a state-of-the-art ConvNet model to achieve improved accuracy in&#10;human joint location estimation. We show that the variance of our detector&#10;approaches the variance of human annotations on the FLIC dataset and&#10;outperforms all existing approaches on the MPII-human-pose dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.2984" label="1406.2984">
-        <attvalues>
-          <attvalue for="0" value="Joint Training of a Convolutional Network and a Graphical Model for&#10;  Human Pose Estimation" />
-          <attvalue for="1" value="  This paper proposes a new hybrid architecture that consists of a deep&#10;Convolutional Network and a Markov Random Field. We show how this architecture&#10;is successfully applied to the challenging problem of articulated human pose&#10;estimation in monocular images. The architecture can exploit structural domain&#10;constraints such as geometric relationships between body joint locations. We&#10;show that joint training of these two model paradigms improves performance and&#10;allows us to significantly outperform existing state-of-the-art techniques.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1312.4659" label="1312.4659">
-        <attvalues>
-          <attvalue for="0" value="DeepPose: Human Pose Estimation via Deep Neural Networks" />
-          <attvalue for="1" value="  We propose a method for human pose estimation based on Deep Neural Networks&#10;(DNNs). The pose estimation is formulated as a DNN-based regression problem&#10;towards body joints. We present a cascade of such DNN regressors which results&#10;in high precision pose estimates. The approach has the advantage of reasoning&#10;about pose in a holistic fashion and has a simple but yet powerful formulation&#10;which capitalizes on recent advances in Deep Learning. We present a detailed&#10;empirical analysis with state-of-art or better performance on four academic&#10;benchmarks of diverse real-world images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1508.00271" label="1508.00271">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Network Models for Human Dynamics" />
-          <attvalue for="1" value="  We propose the Encoder-Recurrent-Decoder (ERD) model for recognition and&#10;prediction of human body pose in videos and motion capture. The ERD model is a&#10;recurrent neural network that incorporates nonlinear encoder and decoder&#10;networks before and after recurrent layers. We test instantiations of ERD&#10;architectures in the tasks of motion capture (mocap) generation, body pose&#10;labeling and body pose forecasting in videos. Our model handles mocap training&#10;data across multiple subjects and activity domains, and synthesizes novel&#10;motions while avoid drifting for long periods of time. For human pose labeling,&#10;ERD outperforms a per frame body part detector by resolving left-right body&#10;part confusions. For video pose forecasting, ERD predicts body joint&#10;displacements across a temporal horizon of 400ms and outperforms a first order&#10;motion model based on optical flow. ERDs extend previous Long Short Term Memory&#10;(LSTM) models in the literature to jointly learn representations and their&#10;dynamics. Our experiments show such representation learning is crucial for both&#10;labeling and prediction in space-time. We find this is a distinguishing feature&#10;between the spatio-temporal visual domain in comparison to 1D text, speech or&#10;handwriting, where straightforward hard coded representations have shown&#10;excellent results when directly combined with recurrent units.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.01743" label="1609.01743">
-        <attvalues>
-          <attvalue for="0" value="Human pose estimation via Convolutional Part Heatmap Regression" />
-          <attvalue for="1" value="  This paper is on human pose estimation using Convolutional Neural Networks.&#10;Our main contribution is a CNN cascaded architecture specifically designed for&#10;learning part relationships and spatial context, and robustly inferring pose&#10;even for the case of severe part occlusions. To this end, we propose a&#10;detection-followed-by-regression CNN cascade. The first part of our cascade&#10;outputs part detection heatmaps and the second part performs regression on&#10;these heatmaps. The benefits of the proposed architecture are multi-fold: It&#10;guides the network where to focus in the image and effectively encodes part&#10;constraints and context. More importantly, it can effectively cope with&#10;occlusions because part detection heatmaps for occluded parts provide low&#10;confidence scores which subsequently guide the regression part of our network&#10;to rely on contextual information in order to predict the location of these&#10;parts. Additionally, we show that the proposed cascade is flexible enough to&#10;readily allow the integration of various CNN architectures for both detection&#10;and regression, including recent ones based on residual learning. Finally, we&#10;illustrate that our cascade achieves top performance on the MPII and LSP data&#10;sets. Code can be downloaded from http://www.cs.nott.ac.uk/~psxab5/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.01101" label="1708.01101">
-        <attvalues>
-          <attvalue for="0" value="Learning Feature Pyramids for Human Pose Estimation" />
-          <attvalue for="1" value="  Articulated human pose estimation is a fundamental yet challenging task in&#10;computer vision. The difficulty is particularly pronounced in scale variations&#10;of human body parts when camera view changes or severe foreshortening happens.&#10;Although pyramid methods are widely used to handle scale changes at inference&#10;time, learning feature pyramids in deep convolutional neural networks (DCNNs)&#10;is still not well explored. In this work, we design a Pyramid Residual Module&#10;(PRMs) to enhance the invariance in scales of DCNNs. Given input features, the&#10;PRMs learn convolutional filters on various scales of input features, which are&#10;obtained with different subsampling ratios in a multi-branch network. Moreover,&#10;we observe that it is inappropriate to adopt existing methods to initialize the&#10;weights of multi-branch networks, which achieve superior performance than plain&#10;networks in many tasks recently. Therefore, we provide theoretic derivation to&#10;extend the current weight initialization scheme to multi-branch network&#10;structures. We investigate our method on two standard benchmarks for human pose&#10;estimation. Our approach obtains state-of-the-art results on both benchmarks.&#10;Code is available at https://github.com/bearpaw/PyraNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.00389" label="1705.00389">
-        <attvalues>
-          <attvalue for="0" value="Adversarial PoseNet: A Structure-aware Convolutional Network for Human&#10;  Pose Estimation" />
-          <attvalue for="1" value="  For human pose estimation in monocular images, joint occlusions and&#10;overlapping upon human bodies often result in deviated pose predictions. Under&#10;these circumstances, biologically implausible pose predictions may be produced.&#10;In contrast, human vision is able to predict poses by exploiting geometric&#10;constraints of joint inter-connectivity. To address the problem by&#10;incorporating priors about the structure of human bodies, we propose a novel&#10;structure-aware convolutional network to implicitly take such priors into&#10;account during training of the deep network. Explicit learning of such&#10;constraints is typically challenging. Instead, we design discriminators to&#10;distinguish the real poses from the fake ones (such as biologically implausible&#10;ones). If the pose generator (G) generates results that the discriminator fails&#10;to distinguish from real ones, the network successfully learns the priors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.09894" label="1803.09894">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale Structure-Aware Network for Human Pose Estimation" />
-          <attvalue for="1" value="  We develop a robust multi-scale structure-aware neural network for human pose&#10;estimation. This method improves the recent deep conv-deconv hourglass models&#10;with four key improvements: (1) multi-scale supervision to strengthen&#10;contextual feature learning in matching body keypoints by combining feature&#10;heatmaps across scales, (2) multi-scale regression network at the end to&#10;globally optimize the structural matching of the multi-scale features, (3)&#10;structure-aware loss used in the intermediate supervision and at the regression&#10;to improve the matching of keypoints and respective neighbors to infer a&#10;higher-order matching configurations, and (4) a keypoint masking training&#10;scheme that can effectively fine-tune our network to robustly localize occluded&#10;keypoints via adjacent matches. Our method can effectively improve&#10;state-of-the-art pose estimation methods that suffer from difficulties in scale&#10;varieties, occlusions, and complex multi-person scenarios. This multi-scale&#10;supervision tightly integrates with the regression network to effectively (i)&#10;localize keypoints using the ensemble of multi-scale features, and (ii) infer&#10;global pose configuration by maximizing structural consistencies across&#10;multiple keypoints and scales. The keypoint masking training enhances these&#10;advantages to focus learning on hard occlusion samples. Our method achieves the&#10;leading position in the MPII challenge leaderboard among the state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.05388" label="1707.05388">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation" />
-          <attvalue for="1" value="  We propose a new method to analyze the impact of errors in algorithms for&#10;multi-instance pose estimation and a principled benchmark that can be used to&#10;compare them. We define and characterize three classes of errors -&#10;localization, scoring, and background - study how they are influenced by&#10;instance attributes and their impact on an algorithm's performance. Our&#10;technique is applied to compare the two leading methods for human pose&#10;estimation on the COCO Dataset, measure the sensitivity of pose estimation with&#10;respect to instance size, type and number of visible keypoints, clutter due to&#10;multiple instances, and the relative score of instances. The performance of&#10;algorithms, and the types of error they make, are highly dependent on all these&#10;variables, but mostly on the number of keypoints and the clutter. The analysis&#10;and software tools we propose offer a novel and insightful approach for&#10;understanding the behavior of pose estimation algorithms and an effective&#10;method for measuring their strengths and weaknesses.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.02002" label="1708.02002">
-        <attvalues>
-          <attvalue for="0" value="Focal Loss for Dense Object Detection" />
-          <attvalue for="1" value="  The highest accuracy object detectors to date are based on a two-stage&#10;approach popularized by R-CNN, where a classifier is applied to a sparse set of&#10;candidate object locations. In contrast, one-stage detectors that are applied&#10;over a regular, dense sampling of possible object locations have the potential&#10;to be faster and simpler, but have trailed the accuracy of two-stage detectors&#10;thus far. In this paper, we investigate why this is the case. We discover that&#10;the extreme foreground-background class imbalance encountered during training&#10;of dense detectors is the central cause. We propose to address this class&#10;imbalance by reshaping the standard cross entropy loss such that it&#10;down-weights the loss assigned to well-classified examples. Our novel Focal&#10;Loss focuses training on a sparse set of hard examples and prevents the vast&#10;number of easy negatives from overwhelming the detector during training. To&#10;evaluate the effectiveness of our loss, we design and train a simple dense&#10;detector we call RetinaNet. Our results show that when trained with the focal&#10;loss, RetinaNet is able to match the speed of previous one-stage detectors&#10;while surpassing the accuracy of all existing state-of-the-art two-stage&#10;detectors. Code is at: https://github.com/facebookresearch/Detectron.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.11078" label="2011.11078">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Differentiable 6DoF Object Pose Estimation with Local and&#10;  Global Constraints" />
-          <attvalue for="1" value="  Inferring the 6DoF pose of an object from a single RGB image is an important&#10;but challenging task, especially under heavy occlusion. While recent approaches&#10;improve upon the two stage approaches by training an end-to-end pipeline, they&#10;do not leverage local and global constraints. In this paper, we propose&#10;pairwise feature extraction to integrate local constraints, and triplet&#10;regularization to integrate global constraints for improved 6DoF object pose&#10;estimation. Coupled with better augmentation, our approach achieves state of&#10;the art results on the challenging Occlusion Linemod dataset, with a 9%&#10;improvement over the previous state of the art, and achieves competitive&#10;results on the Linemod dataset.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Estimating the 6DoF pose of an object is an important problem with applications in various domains like robotics~\cite{correll2016analysis}, augmented reality~\cite{arpose} and autonomous driving~\cite{song2019apollocar3d}. With the pervasion of inexpensive RGB sensors, it is cost effective and highly beneficial to perform 6DoF pose estimation from a single RGB image without using additional depth sensors.&#10;&#10;Some studies~\cite{kehl2017ssd}\cite{xiang2018posecnn} attempted to regress the 6DoF pose directly from the image, however, these were not as competitive as recent two stage approaches. In the first stage of two stage approaches, a correspondence estimator detects the object and estimates the 2D image projections of the 3D object points (referred to as 2D keypoints). This establishes correspondences between the 2D and 3D points. \cite{rad2017bb8}\cite{tekin2018real}\cite{hu2019segmentation} used a CNN based architecture to segment out regions containing the object and regress the 2D keypoints from those regions. A recent study regressed direction vectors to the 2D keypoints from the segmented regions of the object~\cite{peng2019pvnet}. The 2D keypoints were then estimated from intersections of pairs of direction vectors. This approach was found to be more robust to occlusions of the object. &#10;&#10;In the second stage, a RANSAC based Perspective-n-Point(PnP) algorithm serves as a pose estimator to predict the 6DoF object pose using the established 2D-3D correspondences. However, Hu et al.~\cite{hu2020single} showed that RANSAC is sensitive to the ordering of the 2D-3D correspondences and computationally costly when there are many of them. Further, the non-differentiable nature of the RANSAC based pose estimator &#10;does not allow for end-to-end training of the two stage approaches with respect to the final objective, namely the object pose. Hence, Hu et al.~\cite{hu2020single} proposed to replace the non-differentiable RANSAC based pose estimator with a trainable neural network to estimate the 6DoF object pose. Their end-to-end trainable model showed improved results compared to the two stage approach as validated with two state of the art correspondence estimators~\cite{hu2019segmentation}~\cite{peng2019pvnet}. We follow up on their model with~\cite{peng2019pvnet} as the correspondence estimator as it shows superior performance and refer to it as SSPE.&#10;&#10;While SSPE shows improved performance using end-to-end training, it does not utilize local and global geometric constraints. In this work, we propose pairwise features to utilize local information between direction vectors associated with the same 3D point, and triplet regularization to account for the global geometry between pairwise features associated with different 3D points.&#10; Coupled with increased masking augmentation, our model achieves state of the art results on the Occlusion Linemod~\cite{occlusionlinemod} dataset and competitive results on the Linemod~\cite{linemod} dataset. In summary, our main contributions are:&#10;\begin{itemize}&#10;&#10;\item Pairwise feature extraction from direction vectors to better utilize local information&#10; \item Triplet regularization to account for the global geometry of the pairwise features&#10; \item State of the art results on Occlusion Linemod and competitive results on Linemod &#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1601.05484" label="1601.05484">
-        <attvalues>
-          <attvalue for="0" value="Analysis and Observations from the First Amazon Picking Challenge" />
-          <attvalue for="1" value="  This paper presents a overview of the inaugural Amazon Picking Challenge&#10;along with a summary of a survey conducted among the 26 participating teams.&#10;The challenge goal was to design an autonomous robot to pick items from a&#10;warehouse shelf. This task is currently performed by human workers, and there&#10;is hope that robots can someday help increase efficiency and throughput while&#10;lowering cost. We report on a 28-question survey posed to the teams to learn&#10;about each team's background, mechanism design, perception apparatus, planning&#10;and control approach. We identify trends in this data, correlate it with each&#10;team's success in the competition, and discuss observations and lessons learned&#10;based on survey results and the authors' personal experiences during the&#10;challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.12222" label="1811.12222">
-        <attvalues>
-          <attvalue for="0" value="ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for&#10;  Autonomous Driving" />
-          <attvalue for="1" value="  Autonomous driving has attracted remarkable attention from both industry and&#10;academia. An important task is to estimate 3D properties(e.g.translation,&#10;rotation and shape) of a moving or parked vehicle on the road. This task, while&#10;critical, is still under-researched in the computer vision community -&#10;partially owing to the lack of large scale and fully-annotated 3D car database&#10;suitable for autonomous driving research. In this paper, we contribute the&#10;first large-scale database suitable for 3D car instance understanding -&#10;ApolloCar3D. The dataset contains 5,277 driving images and over 60K car&#10;instances, where each car is fitted with an industry-grade 3D CAD model with&#10;absolute model size and semantically labelled keypoints. This dataset is above&#10;20 times larger than PASCAL3D+ and KITTI, the current state-of-the-art. To&#10;enable efficient labelling in 3D, we build a pipeline by considering 2D-3D&#10;keypoint correspondences for a single instance and 3D relationship among&#10;multiple instances. Equipped with such dataset, we build various baseline&#10;algorithms with the state-of-the-art deep convolutional neural networks.&#10;Specifically, we first segment each car with a pre-trained Mask R-CNN, and then&#10;regress towards its 3D pose and shape based on a deformable 3D car model with&#10;or without using semantic keypoints. We show that using keypoints significantly&#10;improves fitting performance. Finally, we develop a new 3D metric jointly&#10;considering 3D pose and 3D shape, allowing for comprehensive evaluation and&#10;ablation study. By comparing with human performance we suggest several future&#10;directions for further improvements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10006" label="1711.10006">
-        <attvalues>
-          <attvalue for="0" value="SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again" />
-          <attvalue for="1" value="  We present a novel method for detecting 3D model instances and estimating&#10;their 6D poses from RGB data in a single shot. To this end, we extend the&#10;popular SSD paradigm to cover the full 6D pose space and train on synthetic&#10;model data only. Our approach competes or surpasses current state-of-the-art&#10;methods that leverage RGB-D data on multiple challenging datasets. Furthermore,&#10;our method produces these results at around 10Hz, which is many times faster&#10;than the related methods. For the sake of reproducibility, we make our trained&#10;networks and detection code publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.10896" label="1703.10896">
-        <attvalues>
-          <attvalue for="0" value="BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for&#10;  Predicting the 3D Poses of Challenging Objects without Using Depth" />
-          <attvalue for="1" value="  We introduce a novel method for 3D object detection and pose estimation from&#10;color images only. We first use segmentation to detect the objects of interest&#10;in 2D even in presence of partial occlusions and cluttered background. By&#10;contrast with recent patch-based methods, we rely on a &quot;holistic&quot; approach: We&#10;apply to the detected objects a Convolutional Neural Network (CNN) trained to&#10;predict their 3D poses in the form of 2D projections of the corners of their 3D&#10;bounding boxes. This, however, is not sufficient for handling objects from the&#10;recent T-LESS dataset: These objects exhibit an axis of rotational symmetry,&#10;and the similarity of two images of such an object under two different poses&#10;makes training the CNN challenging. We solve this problem by restricting the&#10;range of poses used for training, and by introducing a classifier to identify&#10;the range of a pose at run-time before estimating it. We also use an optional&#10;additional step that refines the predicted poses. We improve the&#10;state-of-the-art on the LINEMOD dataset from 73.7% to 89.3% of correctly&#10;registered RGB frames. We are also the first to report results on the Occlusion&#10;dataset using color images only. We obtain 54% of frames passing the Pose 6D&#10;criterion on average on several sequences of the T-LESS dataset, compared to&#10;the 67% of the state-of-the-art on the same sequences which uses both color and&#10;depth. The full approach is also scalable, as a single network can be trained&#10;for multiple objects simultaneously.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08848" label="1711.08848">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Seamless Single Shot 6D Object Pose Prediction" />
-          <attvalue for="1" value="  We propose a single-shot approach for simultaneously detecting an object in&#10;an RGB image and predicting its 6D pose without requiring multiple stages or&#10;having to examine multiple hypotheses. Unlike a recently proposed single-shot&#10;technique for this task (Kehl et al., ICCV'17) that only predicts an&#10;approximate 6D pose that must then be refined, ours is accurate enough not to&#10;require additional post-processing. As a result, it is much faster - 50 fps on&#10;a Titan X (Pascal) GPU - and more suitable for real-time processing. The key&#10;component of our method is a new CNN architecture inspired by the YOLO network&#10;design that directly predicts the 2D image locations of the projected vertices&#10;of the object's 3D bounding box. The object's 6D pose is then estimated using a&#10;PnP algorithm.&#10;  For single object and multiple object pose estimation on the LINEMOD and&#10;OCCLUSION datasets, our approach substantially outperforms other recent&#10;CNN-based approaches when they are all used without post-processing. During&#10;post-processing, a pose refinement step can be used to boost the accuracy of&#10;the existing methods, but at 10 fps or less, they are much slower than our&#10;method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.02541" label="1812.02541">
-        <attvalues>
-          <attvalue for="0" value="Segmentation-driven 6D Object Pose Estimation" />
-          <attvalue for="1" value="  The most recent trend in estimating the 6D pose of rigid objects has been to&#10;train deep networks to either directly regress the pose from the image or to&#10;predict the 2D locations of 3D keypoints, from which the pose can be obtained&#10;using a PnP algorithm. In both cases, the object is treated as a global entity,&#10;and a single pose estimate is computed. As a consequence, the resulting&#10;techniques can be vulnerable to large occlusions.&#10;  In this paper, we introduce a segmentation-driven 6D pose estimation&#10;framework where each visible part of the objects contributes a local pose&#10;prediction in the form of 2D keypoint locations. We then use a predicted&#10;measure of confidence to combine these pose candidates into a robust set of&#10;3D-to-2D correspondences, from which a reliable pose estimate can be obtained.&#10;We outperform the state-of-the-art on the challenging Occluded-LINEMOD and&#10;YCB-Video datasets, which is evidence that our approach deals well with&#10;multiple poorly-textured objects occluding each other. Furthermore, it relies&#10;on a simple enough architecture to achieve real-time performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.11788" label="1812.11788">
-        <attvalues>
-          <attvalue for="0" value="PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation" />
-          <attvalue for="1" value="  This paper addresses the challenge of 6DoF pose estimation from a single RGB&#10;image under severe occlusion or truncation. Many recent works have shown that a&#10;two-stage approach, which first detects keypoints and then solves a&#10;Perspective-n-Point (PnP) problem for pose estimation, achieves remarkable&#10;performance. However, most of these methods only localize a set of sparse&#10;keypoints by regressing their image coordinates or heatmaps, which are&#10;sensitive to occlusion and truncation. Instead, we introduce a Pixel-wise&#10;Voting Network (PVNet) to regress pixel-wise unit vectors pointing to the&#10;keypoints and use these vectors to vote for keypoint locations using RANSAC.&#10;This creates a flexible representation for localizing occluded or truncated&#10;keypoints. Another important feature of this representation is that it provides&#10;uncertainties of keypoint locations that can be further leveraged by the PnP&#10;solver. Experiments show that the proposed approach outperforms the state of&#10;the art on the LINEMOD, Occlusion LINEMOD and YCB-Video datasets by a large&#10;margin, while being efficient for real-time pose estimation. We further create&#10;a Truncation LINEMOD dataset to validate the robustness of our approach against&#10;truncation. The code will be avaliable at https://zju-3dv.github.io/pvnet/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.08324" label="1911.08324">
-        <attvalues>
-          <attvalue for="0" value="Single-Stage 6D Object Pose Estimation" />
-          <attvalue for="1" value="  Most recent 6D pose estimation frameworks first rely on a deep network to&#10;establish correspondences between 3D object keypoints and 2D image locations&#10;and then use a variant of a RANSAC-based Perspective-n-Point (PnP) algorithm.&#10;This two-stage process, however, is suboptimal: First, it is not end-to-end&#10;trainable. Second, training the deep network relies on a surrogate loss that&#10;does not directly reflect the final 6D pose estimation task.&#10;  In this work, we introduce a deep architecture that directly regresses 6D&#10;poses from correspondences. It takes as input a group of candidate&#10;correspondences for each 3D keypoint and accounts for the fact that the order&#10;of the correspondences within each group is irrelevant, while the order of the&#10;groups, that is, of the 3D keypoints, is fixed. Our architecture is generic and&#10;can thus be exploited in conjunction with existing correspondence-extraction&#10;networks so as to yield single-stage 6D pose estimation frameworks. Our&#10;experiments demonstrate that these single-stage frameworks consistently&#10;outperform their two-stage counterparts in terms of both accuracy and speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.10776" label="2204.10776">
-        <attvalues>
-          <attvalue for="0" value="Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB&#10;  Images" />
-          <attvalue for="1" value="  In this paper, we present a generalizable model-free 6-DoF object pose&#10;estimator called Gen6D. Existing generalizable pose estimators either need&#10;high-quality object models or require additional depth maps or object masks in&#10;test time, which significantly limits their application scope. In contrast, our&#10;pose estimator only requires some posed images of the unseen object and is able&#10;to accurately predict the poses of the object in arbitrary environments. Gen6D&#10;consists of an object detector, a viewpoint selector and a pose refiner, all of&#10;which do not require the 3D object model and can generalize to unseen objects.&#10;Experiments show that Gen6D achieves state-of-the-art results on two model-free&#10;datasets: the MOPED dataset and a new GenMOP dataset collected by us. In&#10;addition, on the LINEMOD dataset, Gen6D achieves competitive results compared&#10;with instance-specific pose estimators. Project page:&#10;https://liuyuan-pal.github.io/Gen6D/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2110.03479" label="2110.03479">
-        <attvalues>
-          <attvalue for="0" value="Camera Calibration through Camera Projection Loss" />
-          <attvalue for="1" value="  Camera calibration is a necessity in various tasks including 3D&#10;reconstruction, hand-eye coordination for a robotic interaction, autonomous&#10;driving, etc. In this work we propose a novel method to predict extrinsic&#10;(baseline, pitch, and translation), intrinsic (focal length and principal point&#10;offset) parameters using an image pair. Unlike existing methods, instead of&#10;designing an end-to-end solution, we proposed a new representation that&#10;incorporates camera model equations as a neural network in multi-task learning&#10;framework. We estimate the desired parameters via novel camera projection loss&#10;(CPL) that uses the camera model neural network to reconstruct the 3D points&#10;and uses the reconstruction loss to estimate the camera parameters. To the best&#10;of our knowledge, ours is the first method to jointly estimate both the&#10;intrinsic and extrinsic parameters via a multi-task learning methodology that&#10;combines analytical equations in learning framework for the estimation of&#10;camera parameters. We also proposed a novel dataset using CARLA Simulator.&#10;Empirically, we demonstrate that our proposed approach achieves better&#10;performance with respect to both deep learning-based and traditional methods on&#10;8 out of 10 parameters evaluated using both synthetic and real data. Our code&#10;and generated dataset are available at&#10;https://github.com/thanif/Camera-Calibration-through-Camera-Projection-Loss.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Camera calibration deals with finding the five intrinsic ( focal length, image sensor format, and principal point) and six extrinsic (rotation, translation) parameters of the specific camera. &#10;&#10;Most of the existing methods usually ignore the underlying mathematical formulation of the camera model and instead propose an end-to-end framework to directly estimate the desired parameters~\cite{workman2015deepfocal,rong2016radial,hold2018perceptual, lopez2019deep, zhai2016detecting, detone2016deep,bogdan2018deepcalib,zhang2020deepptz,workman2016horizon,barreto2006unifying}. Thus they are difficult to interpret for real-world applications and have so far been able to mainly estimate the focal length of the camera via single image only~\cite{detone2016deep,bogdan2018deepcalib,zhang2020deepptz}. The major contributions of our work are as follows:&#10;&#10;\begin{itemize}&#10; \item To the best of our knowledge, this work is the first learning-based method to jointly estimate both intrinsic and extrinsic camera parameters including camera baseline, disparity, pitch, translation, focal length and principal point offset.&#10; \item The existing learning based approaches~\cite{detone2016deep, workman2015deepfocal, zhang2020deepptz} have not been applied to the estimation of all $10$ camera parameters due to lack of any dataset. We addressed this limitation by generating a synthetic dataset from two towns in CARLA~\cite{Dosovitskiy17} simulation consisting of $49$ different camera settings.&#10; \item Unlike existing methods, instead of designing an end-to-end solution to directly estimate the desired parameters~\cite{detone2016deep}, we proposed a new representation that represents camera model equations as a neural network in a multi-task learning (MTL) framework.&#10; \item We proposed a novel camera projection loss (CPL) that combines analytical equations in learning framework. \end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Camera Calibration, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1712.01259" label="1712.01259">
-        <attvalues>
-          <attvalue for="0" value="A Perceptual Measure for Deep Single Image Camera Calibration" />
-          <attvalue for="1" value="  Most current single image camera calibration methods rely on specific image&#10;features or user input, and cannot be applied to natural images captured in&#10;uncontrolled settings. We propose directly inferring camera calibration&#10;parameters from a single image using a deep convolutional neural network. This&#10;network is trained using automatically generated samples from a large-scale&#10;panorama dataset, and considerably outperforms other methods, including recent&#10;deep learning-based approaches, in terms of standard L2 error. However, we&#10;argue that in many cases it is more important to consider how humans perceive&#10;errors in camera estimation. To this end, we conduct a large-scale human&#10;perception study where we ask users to judge the realism of 3D objects&#10;composited with and without ground truth camera calibration. Based on this&#10;study, we develop a new perceptual measure for camera calibration, and&#10;demonstrate that our deep calibration network outperforms other methods on this&#10;measure. Finally, we demonstrate the use of our calibration network for a&#10;number of applications including virtual object insertion, image retrieval and&#10;compositing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.02129" label="1604.02129">
-        <attvalues>
-          <attvalue for="0" value="Horizon Lines in the Wild" />
-          <attvalue for="1" value="  The horizon line is an important contextual attribute for a wide variety of&#10;image understanding tasks. As such, many methods have been proposed to estimate&#10;its location from a single image. These methods typically require the image to&#10;contain specific cues, such as vanishing points, coplanar circles, and regular&#10;textures, thus limiting their real-world applicability. We introduce a large,&#10;realistic evaluation dataset, Horizon Lines in the Wild (HLW), containing&#10;natural images with labeled horizon lines. Using this dataset, we investigate&#10;the application of convolutional neural networks for directly estimating the&#10;horizon line, without requiring any explicit geometric constraints or other&#10;special cues. An extensive evaluation shows that using our CNNs, either in&#10;isolation or in conjunction with a previous geometric approach, we achieve&#10;state-of-the-art results on the challenging HLW dataset and two existing&#10;benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.03798" label="1606.03798">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Homography Estimation" />
-          <attvalue for="1" value="  We present a deep convolutional neural network for estimating the relative&#10;homography between a pair of images. Our feed-forward network has 10 layers,&#10;takes two stacked grayscale images as input, and produces an 8 degree of&#10;freedom homography which can be used to map the pixels from the first image to&#10;the second. We present two convolutional neural network architectures for&#10;HomographyNet: a regression network which directly estimates the real-valued&#10;homography parameters, and a classification network which produces a&#10;distribution over quantized homographies. We use a 4-point homography&#10;parameterization which maps the four corners from one image into the second&#10;image. Our networks are trained in an end-to-end fashion using warped MS-COCO&#10;images. Our approach works without the need for separate local feature&#10;detection and transformation estimation stages. Our deep models are compared to&#10;a traditional homography estimator based on ORB features and we highlight the&#10;scenarios where HomographyNet outperforms the traditional technique. We also&#10;describe a variety of applications powered by deep homography estimation, thus&#10;showcasing the flexibility of a deep learning approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.13006" label="2202.13006">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Instance Segmentation using Motion Information via&#10;  Optical Flow" />
-          <attvalue for="1" value="  Weakly supervised instance segmentation has gained popularity because it&#10;reduces high annotation cost of pixel-level masks required for model training.&#10;Recent approaches for weakly supervised instance segmentation detect and&#10;segment objects using appearance information obtained from a static image.&#10;However, it poses the challenge of identifying objects with a&#10;non-discriminatory appearance. In this study, we address this problem by using&#10;motion information from image sequences. We propose a two-stream encoder that&#10;leverages appearance and motion features extracted from images and optical&#10;flows. Additionally, we propose a novel pairwise loss that considers both&#10;appearance and motion information to supervise segmentation. We conducted&#10;extensive evaluations on the YouTube-VIS 2019 benchmark dataset. Our results&#10;demonstrate that the proposed method improves the Average Precision of the&#10;state-of-the-art method by 3.1.&#10;" />
-          <attvalue for="2" value="&#10;Instance segmentation, which is a task of detecting objects in an image and segmenting the region of the objects at the pixel level, is one of the key technologies behind recently advanced machine vision applications.&#10;It has been actively studied because of its wide range of applications, such as autonomous driving.&#10;However, the annotation cost of pixel-level masks required to train models has been one of the major challenges.&#10;Therefore, weakly supervised instance segmentation, which can train models without relying on mask annotations, has recently gained popularity.&#10;&#10;Box-supervised instance segmentation is a common weakly supervised approach in which a model is trained using information from bounding boxes as pseudo-labels for the segmentation mask.&#10;The main challenge in weakly supervised mask generation with box annotations is the separation of foreground as an object and its background regions in the box. If the entire box region is labeled as foreground, many false positive labels adversely affect the model training for mask generation.&#10;In previous works~\cite{Khoreva_2017_CVPR_simple_does_it, Hsu_2019_NIPS_BBTP, Lee_2021_CVPR_BBAM, Tian_2021_CVPR_boxinst}, foreground is separated based on image appearance information.&#10;For example, BoxInst~\cite{Tian_2021_CVPR_boxinst}, which is the state-of-the-art approach, has addressed the separation problem by segmenting regions based on the color similarity of local pixel pairs.&#10;However, it suffers from segmentation errors, which caused by false connections of regions with similar colors in the foreground and background (Figure~\ref{fig:results} upper left).&#10;Additionally, it often fails to detect objects with non-discriminative appearances (Figure~\ref{fig:results} lower left).&#10;&#10;While previous studies have solely relied on information from a static image, they ignored information from image sequences. In fact, most segmentation technology applications, such as autonomous driving and video editing, use image sequences as system inputs.&#10;&#10;In this study, we address the above-mentioned problems by introducing temporal information from sequential images as complement signals for weakly supervised instance segmentation.&#10;In particular, we propose a method for weakly supervised instance segmentation using motion information.&#10;The basic idea is to use motion information in a form of optical flow~\cite{Baker_2007_ICCV_optical_flow}, which could support object detection and segmentation when dealing with a foreground object moving differently than background, as shown in Figure~\ref{fig:results}.&#10;&#10;It has been demonstrated that motion information is effective in achieving the task of so-called motion segmentation~\cite{Jain_2017_CVPR_FusionSeg, Siam_2018_ITSC_MODNet, Rashed_2019_ICCV_FuseMODNet, Mohamed_2021_IV_instance_motion_segmentation}, which attempts to segment the region of moving objects in a class-agnostic manner.&#10;In the motion segmentation, motion feature is extracted from optical flow using a pre-trained object detection model~\cite{Jain_2017_CVPR_FusionSeg} as its backbone. &#10;Previous studies showed that integrating motion feature maps into appearance feature is effective~\cite{Siam_2018_ITSC_MODNet}.&#10;Moreover, further fine-grained architectures, such as multi-level feature maps~\cite{Rashed_2019_ICCV_FuseMODNet} and feature pyramid networks~\cite{Mohamed_2021_IV_instance_motion_segmentation}, have recently been introduced. &#10;&#10;Several recent approaches for the instance segmentation have also explored the use of temporal information by aggregating features across frames~\cite{Fu_2021_AAAI_CompFeat, Bertasius_2020_CVPR_MaskProp} or using 3D convolutions~\cite{Voigtlaender_2019_CVPR_mots, Athar_2020_ECCV_StemSeg}. &#10;Previous work~\cite{Liu_2021_CVPR_temporal_mask_consistency} attempted to use motion information for instance segmentation in a weakly supervised manner. However, it simply amplifies the foreground scores of the regions with large motion. &#10;&#10;We propose a two-stream encoder that uses motion features extracted from optical flow for object detection and segmentation, with the goal of weakly supervised instance segmentation using motion information.&#10;Additionally, we propose a novel pairwise loss that uses optical flow to complement the segmentation of foreground and background regions with similar colors.&#10;The contributions of this study are summarized as follows:(1) &#10;We explore the use of motion information in box-supervised instance segmentation. (2) We propose a two-stream encoder to provide both appearance and motion features for the detection and mask heads. Additionally, we propose a novel pairwise loss that uses both appearance and motion information to improve segmentation accuracy. (3) Extensive evaluation on the YouTube-VIS 2019~\cite{yang_ICCV_2019_VIS} benchmark dataset demonstrates that the proposed method outperforms the state-of-the-art approach even with less complex pre-trained backbones.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Weak Supervision, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Instance Segmentation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1603.07485" label="1603.07485">
-        <attvalues>
-          <attvalue for="0" value="Simple Does It: Weakly Supervised Instance and Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic labelling and instance segmentation are two tasks that require&#10;particularly costly annotations. Starting from weak supervision in the form of&#10;bounding box detection annotations, we propose a new approach that does not&#10;require modification of the segmentation training procedure. We show that when&#10;carefully designing the input labels from given bounding boxes, even a single&#10;round of training is enough to improve over previously reported weakly&#10;supervised results. Overall, our weak supervision approach reaches ~95% of the&#10;quality of the fully supervised model, both for semantic labelling and instance&#10;segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.02310" label="2012.02310">
-        <attvalues>
-          <attvalue for="0" value="BoxInst: High-Performance Instance Segmentation with Box Annotations" />
-          <attvalue for="1" value="  We present a high-performance method that can achieve mask-level instance&#10;segmentation with only bounding-box annotations for training. While this&#10;setting has been studied in the literature, here we show significantly stronger&#10;performance with a simple design (e.g., dramatically improving previous best&#10;reported mask AP of 21.1% in Hsu et al. (2019) to 31.6% on the COCO dataset).&#10;Our core idea is to redesign the loss of learning masks in instance&#10;segmentation, with no modification to the segmentation network itself. The new&#10;loss functions can supervise the mask training without relying on mask&#10;annotations. This is made possible with two loss terms, namely, 1) a surrogate&#10;term that minimizes the discrepancy between the projections of the ground-truth&#10;box and the predicted mask; 2) a pairwise loss that can exploit the prior that&#10;proximal pixels with similar colors are very likely to have the same category&#10;label. Experiments demonstrate that the redesigned mask loss can yield&#10;surprisingly high-quality instance masks with only box annotations. For&#10;example, without using any mask annotations, with a ResNet-101 backbone and 3x&#10;training schedule, we achieve 33.2% mask AP on COCO test-dev split (vs. 39.1%&#10;of the fully supervised counterpart). Our excellent experiment results on COCO&#10;and Pascal VOC indicate that our method dramatically narrows the performance&#10;gap between weakly and fully supervised instance segmentation.&#10;  Code is available at: https://git.io/AdelaiDet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.05395" label="1910.05395">
-        <attvalues>
-          <attvalue for="0" value="FuseMODNet: Real-Time Camera and LiDAR based Moving Object Detection for&#10;  robust low-light Autonomous Driving" />
-          <attvalue for="1" value="  Moving object detection is a critical task for autonomous vehicles. As&#10;dynamic objects represent higher collision risk than static ones, our own&#10;ego-trajectories have to be planned attending to the future states of the&#10;moving elements of the scene. Motion can be perceived using temporal&#10;information such as optical flow. Conventional optical flow computation is&#10;based on camera sensors only, which makes it prone to failure in conditions&#10;with low illumination. On the other hand, LiDAR sensors are independent of&#10;illumination, as they measure the time-of-flight of their own emitted lasers.&#10;In this work, we propose a robust and real-time CNN architecture for Moving&#10;Object Detection (MOD) under low-light conditions by capturing motion&#10;information from both camera and LiDAR sensors. We demonstrate the impact of&#10;our algorithm on KITTI dataset where we simulate a low-light environment&#10;creating a novel dataset &quot;Dark KITTI&quot;. We obtain a 10.1% relative improvement&#10;on Dark-KITTI, and a 4.25% improvement on standard KITTI relative to our&#10;baselines. The proposed algorithm runs at 18 fps on a standard desktop GPU&#10;using $256\times1224$ resolution images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.07008" label="2008.07008">
-        <attvalues>
-          <attvalue for="0" value="Monocular Instance Motion Segmentation for Autonomous Driving: KITTI&#10;  InstanceMotSeg Dataset and Multi-task Baseline" />
-          <attvalue for="1" value="  Moving object segmentation is a crucial task for autonomous vehicles as it&#10;can be used to segment objects in a class agnostic manner based on their motion&#10;cues. It enables the detection of unseen objects during training (e.g., moose&#10;or a construction truck) based on their motion and independent of their&#10;appearance. Although pixel-wise motion segmentation has been studied in&#10;autonomous driving literature, it has been rarely addressed at the instance&#10;level, which would help separate connected segments of moving objects leading&#10;to better trajectory planning. As the main issue is the lack of large public&#10;datasets, we create a new InstanceMotSeg dataset comprising of 12.9K samples&#10;improving upon our KITTIMoSeg dataset. In addition to providing instance level&#10;annotations, we have added 4 additional classes which is crucial for studying&#10;class agnostic motion segmentation. We adapt YOLACT and implement a&#10;motion-based class agnostic instance segmentation model which would act as a&#10;baseline for the dataset. We also extend it to an efficient multi-task model&#10;which additionally provides semantic instance segmentation sharing the encoder.&#10;The model then learns separate prototype coefficients within the class agnostic&#10;and semantic heads providing two independent paths of object detection for&#10;redundant safety. To obtain real-time performance, we study different efficient&#10;encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an&#10;improvement of 10% mAP relative to the baseline. Our model improves the&#10;previous state of the art motion segmentation method by 3.3%. The dataset and&#10;qualitative results video are shared in our website at&#10;https://sites.google.com/view/instancemotseg/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.12886" label="2103.12886">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Instance Segmentation for Videos with Temporal Mask&#10;  Consistency" />
-          <attvalue for="1" value="  Weakly supervised instance segmentation reduces the cost of annotations&#10;required to train models. However, existing approaches which rely only on&#10;image-level class labels predominantly suffer from errors due to (a) partial&#10;segmentation of objects and (b) missing object predictions. We show that these&#10;issues can be better addressed by training with weakly labeled videos instead&#10;of images. In videos, motion and temporal consistency of predictions across&#10;frames provide complementary signals which can help segmentation. We are the&#10;first to explore the use of these video signals to tackle weakly supervised&#10;instance segmentation. We propose two ways to leverage this information in our&#10;model. First, we adapt inter-pixel relation network (IRN) to effectively&#10;incorporate motion information during training. Second, we introduce a new&#10;MaskConsist module, which addresses the problem of missing object instances by&#10;transferring stable predictions between neighboring frames during training. We&#10;demonstrate that both approaches together improve the instance segmentation&#10;metric $AP_{50}$ on video frames of two datasets: Youtube-VIS and Cityscapes by&#10;$5\%$ and $3\%$ respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Different types of weak supervision have been used in the past for semantic segmentation: bounding boxes~\cite{dai2015boxsup,papandreou2015weakly,khoreva2017simple,song2019box}, scribbles~\cite{lin2016scribblesup,vernaza2017learning,tang2018normalized}, and image-level class labels~\cite{kolesnikov2016seed,jin2017webly,hou2018self,wei2018revisiting,huang2018weakly,lee2019ficklenet,shimoda2019self,sun2020mining}. Similarly, for instance segmentation, image-level~\cite{zhou2018weakly,cholakkal2019object,zhu2019learning,ahn2019weakly,laradji2019masks,ge2019label,shen2019cyclic,arun2020weakly} and bounding box supervision~\cite{khoreva2017simple,hsu2019bbtp} have been explored. In this work, we focus on only using class labels for weakly supervised instance segmentation. &#10;&#10;Weakly supervised semantic segmentation:&#10;Most weakly supervised semantic segmentation approaches rely on class attention maps (CAMs)~\cite{zhou2016learning} to provide noisy pseudo-labels as supervision~\cite{kolesnikov2016seed,huang2018weakly,shimoda2019self,sun2020mining}. Sun et al.~\cite{sun2020mining} used co-attention maps generated from image pairs to train the semantic segmentation network. Another line of work leverages motion and temporal consistency in videos~\cite{tsai2016semantic,tokmakov2016learning,saleh2017bringing,hong2017weakly,lee2019frame,wang2020deep} to learn more robust representation. For instance, frame-to-frame (F2F)~\cite{lee2019frame} used optical flow to warp CAMs from neighboring frames and aggregated the warped CAMs to obtain more robust pseudo-labels.&#10;&#10;Weakly supervised instance segmentation:&#10;For training instance segmentation models with bounding box supervision, Hsu et al.~\cite{hsu2019bbtp} proposed a bounding box tightness&#10;constraint and multiple instance learning (MIL) based objective. Another line of work that only uses class labels extracts semantic responses from CAMs or other attention maps and then combines them with object proposals~\cite{rother2004grabcut,pont2016multiscale} to generate instance segmentation masks ~\cite{zhou2018weakly,cholakkal2019object,zhu2019learning,laradji2019masks,shen2019cyclic}. However, these methods' performance heavily depends on the quality of proposals used, which are mostly pre-trained on other datasets.&#10;Shen et al.~\cite{shen2019cyclic} extracted attention maps from a detection network and then jointly learn the detection and segmentation networks in a cyclic manner. Arun et al.~\cite{arun2020weakly} proposed a conditional network to model the noise in weak supervision and combined it with object proposals to generate instance masks.&#10;The first end-to-end network (IRN)~\cite{ahn2019weakly} was proposed by Ahn et al. to directly predict instance offset and semantic boundary which were combined with CAMs to generate instance mask predictions. Our method adapts \cite{ahn2019weakly} for the first step of training and combines it with a novel MaskConsist module. However, other weakly supervised can also be integrated into our framework if code is available. &#10;&#10;Segmentation in videos: A series of approaches have emerged for segmentation in videos~\cite{Bertasius_2020_CVPR,hu2019learning,athar2020stem,mohamed2020instancemotseg,liang2020polytransform}. Some works proposed to leverage the video consistency~\cite{vondrick2018tracking,wang2019learning,lu2020learning,luiten2020unovost}. Recently, Yang et al.~\cite{yang2019video} extended the traditional instance segmentation task from images to videos and proposed Video Instance Segmentation task (VIS). VIS aims to simultaneously segment and track all object instances in the video. Every pixel is labeled with a class label and an instance track-ID. MaskTrack~\cite{yang2019video} added a tracking-head to Mask R-CNN~\cite{he2017mask} to build a new model for this task. Bertasius et al.~\cite{Bertasius_2020_CVPR} improved MaskTrack by proposing a mask propagation head. This head propagated instance features across frames in a clip to get more stable predictions. To the best of our knowledge, there has been no work that has explored weakly supervised learning for the video instance segmentation task. We evaluate our method on this task by combining it with a simple tracking approach. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1905.04804" label="1905.04804">
-        <attvalues>
-          <attvalue for="0" value="Video Instance Segmentation" />
-          <attvalue for="1" value="  In this paper we present a new computer vision task, named video instance&#10;segmentation. The goal of this new task is simultaneous detection, segmentation&#10;and tracking of instances in videos. In words, it is the first time that the&#10;image instance segmentation problem is extended to the video domain. To&#10;facilitate research on this new task, we propose a large-scale benchmark called&#10;YouTube-VIS, which consists of 2883 high-resolution YouTube videos, a&#10;40-category label set and 131k high-quality instance masks. In addition, we&#10;propose a novel algorithm called MaskTrack R-CNN for this task. Our new method&#10;introduces a new tracking branch to Mask R-CNN to jointly perform the&#10;detection, segmentation and tracking tasks simultaneously. Finally, we evaluate&#10;the proposed method and several strong baselines on our new dataset.&#10;Experimental results clearly demonstrate the advantages of the proposed&#10;algorithm and reveal insight for future improvement. We believe the video&#10;instance segmentation task will motivate the community along the line of&#10;research for video understanding.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.15903" label="2103.15903">
-        <attvalues>
-          <attvalue for="0" value="Iterative Gradient Encoding Network with Feature Co-Occurrence Loss for&#10;  Single Image Reflection Removal" />
-          <attvalue for="1" value="  Removing undesired reflections from a photo taken in front of glass is of&#10;great importance for enhancing visual computing systems' efficiency. Previous&#10;learning-based approaches have produced visually plausible results for some&#10;reflections type, however, failed to generalize against other reflection types.&#10;There is a dearth of literature for efficient methods concerning single image&#10;reflection removal, which can generalize well in large-scale reflection types.&#10;In this study, we proposed an iterative gradient encoding network for single&#10;image reflection removal. Next, to further supervise the network in learning&#10;the correlation between the transmission layer features, we proposed a feature&#10;co-occurrence loss. Extensive experiments on the public benchmark dataset of&#10;SIR$^2$ demonstrated that our method can remove reflection favorably against&#10;the existing state-of-the-art method on all imaging settings, including diverse&#10;backgrounds. Moreover, as the reflection strength increases, our method can&#10;still remove reflection even where other state of the art methods failed.&#10;" />
-          <attvalue for="2" value="&#10;How to obtain a reflection-free image taken through the glass is of great interest to computer vision researchers. Removing the undesired reflection enhances the target object's visibility and benefits various computer vision tasks, such as image classification, segmentation, and object detection. The initial efforts of reflection removal employed multiple images to disentangle the reflection from the transmission \cite{guo2014robust}, \cite{sarel2004separating}, \cite{szeliski2000layer}. More recently, the endeavor to solve the more common and practically significant scenario of a single input image has received a lot of appreciation \cite{li2014single}, \cite{shih2015reflection}, \cite{li2020single}. However, single-image reflection removal is a challenging process because of the ill-posed nature of the problem \cite{levin2007user}. To deal with the problem's ill-posedness, the recent learning-based method has utilized different auxiliary information as prior and constraint \cite{fan2017generic}, \cite{wan2018crrn}, \cite{lyu2019reflection}, \cite{sun2019multi}. Among them, one school of researchers had tried to exploit auxiliary information embedded in the image's gradient \cite{abiko2019single}, \cite{wan2019corrn}. For example, in \cite{wan2018crrn} Wan et al. proposed to use an auxiliary network to restore the gradient of the corrupted image and fused the features of the gradient network with the image restoration network. In a different work, M Ikehara et al.\cite{abiko2019single} proposed a feature space gradient constraint loss. In \cite{wan2019corrn} Ket et al. proposed an MMD loss based on the higher-order statics of image gradient. Together with other learning-based methods, these methods perform adequately when the effect of reflection is low; however, they fail miserably if the intensity of reflection increases. Moreover, due to different complexities such as insufficient training data, varying imaging conditions, diverging scene content, and little physical understanding of this problem, these methods often fail to remove some reflection but perform sufficiently for another reflection type.&#10;In this study, we proposed a novel iterative gradient encoding network for single image reflection removal. Our proposed iterative method first computes the gradient of the estimated transmission layer of the current iteration and utilizes it to estimate the transmission layer in the next iteration. We have employed a multi-scale feature fusion scheme to use the feature of gradient image. Note, our proposed method is different from other iterative methods, as we have not used the estimated transmission as an input to the network in the next iteration; only a subpart of the network is iterated, holding the feature extracted from the mixture image fixed.&#10;\\Next, we reintroduced the well-known style loss \cite{gatys2015neural} as feature co-occurrence loss for reflection removal. The correlation between the features of the transmission layer and the reflection layer provides useful information for separating these layers. Style loss is well known for capturing correlation between different features, yet this consequential but straightforward loss is unused in reflection removal. To the best of our knowledge, this is the first study to adopt style loss as feature co-occurrence loss for reflection removal. &#10;\\We evaluated our proposed method in the recent benchmark data set of SIR$^2$ \cite{wan2017benchmarking}. This data set contains real-world reflection images with diverse imaging settings and backgrounds. Extensive experiment on this data set has shown our method can remove reflection more efficiently than the current state of the art in all imaging conditions. Moreover, our iterative methods have helped to remove strong reflections from the background where most of the state of the art method failed.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Reflection Removal, Machine Learning, Computer Vision, Mathematics, Physics, Deep Learning Methods, Computer Science, Computer Vision Systems, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1911.06634" label="1911.06634">
-        <attvalues>
-          <attvalue for="0" value="Single Image Reflection Removal through Cascaded Refinement" />
-          <attvalue for="1" value="  We address the problem of removing undesirable reflections from a single&#10;image captured through a glass surface, which is an ill-posed, challenging but&#10;practically important problem for photo enhancement. Inspired by iterative&#10;structure reduction for hidden community detection in social networks, we&#10;propose an Iterative Boost Convolutional LSTM Network (IBCLN) that enables&#10;cascaded prediction for reflection removal. IBCLN is a cascaded network that&#10;iteratively refines the estimates of transmission and reflection layers in a&#10;manner that they can boost the prediction quality to each other, and&#10;information across steps of the cascade is transferred using an LSTM. The&#10;intuition is that the transmission is the strong, dominant structure while the&#10;reflection is the weak, hidden structure. They are complementary to each other&#10;in a single image and thus a better estimate and reduction on one side from the&#10;original image leads to a more accurate estimate on the other side. To&#10;facilitate training over multiple cascade steps, we employ LSTM to address the&#10;vanishing gradient problem, and propose residual reconstruction loss as further&#10;training guidance. Besides, we create a dataset of real-world images with&#10;reflection and ground-truth transmission layers to mitigate the problem of&#10;insufficient data. Comprehensive experiments demonstrate that the proposed&#10;method can effectively remove reflections in real and synthetic images compared&#10;with state-of-the-art reflection removal methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Mathematically speaking, SIRR operates on a captured image $\mathbf{I}$, which is generally assumed to be a linear combination of a transmission layer $\mathbf{T}$ and a reflection layer $\mathbf{R}$. The goal is to infer a transmission layer $\mathbf{T}$ that is free of reflections. &#10;In this work, we focus on deep learning-based SIRR, which has produced state-of-the-art results. Previous multiple-image methods~\cite{xue2015computational, guo2014robust, li2013exploiting, sun2016automatic, sarel2004separating, gai2011blind, szeliski2000layer, han2017reflection} and single-image-priors based methods~\cite{levin2004separating, li2014single, levin2003learning, springer2017reflection, arvanitopoulos2017single, wan2016depth, li2013exploiting, tuncer2011ground} are not considered here.&#10;&#10;Due to the advantages in robustness and performance, there is an emerging interest in applying neural networks to SIRR. Fan \etal~\cite{fan2017generic} provide the first neural network model to solve this ill-posed problem. They propose a linear method for synthesizing images with reflection for training, and use an edge map as auxiliary information to guide the reflection removal. Wan \etal~\cite{wan2018crrn} develop two cooperative sub-networks, which predict the transmission layer intensity and gradients concurrently. Both of these works~\cite{fan2017generic, wan2018crrn} utilize edge or gradient information of the captured layer $\mathbf{I}$, motivated by the idea that the reflection layers are usually not in focus and thus blurry as compared to the transmission layers. &#10;From the edge information of the captured image $\mathbf{I}$, the edge map of the transmission image $\mathbf{T}$ is predicted and used in estimating the transmission result.&#10;Instead, BDN~\cite{yang2018seeing} predicts reflection layers which are then used as auxiliary information in a subsequent network to estimate the transmission.&#10;&#10;In several recent methods, improved formulations of the objective function are presented. These include the adoption of perceptual losses~\cite{johnson2016perceptual} to account for both low-level and high-level image information~\cite{chi2018single, jin2018learning, zhang2018single}. In these works, images are fed to a deep network pre-trained on ImageNet, and comparisons are made based on extracted multi-stage features. Adversarial losses have also been applied, specifically to improve the realism of predicted transmission layers~\cite{zhang2018single, lee2018generative, wen2019single, wei2019single}.&#10;&#10;Another direction of study focuses on datasets for training. Moving beyond improvements for the linear synthesis method in~\cite{fan2017generic} and~\cite{zhang2018single}, Wen \etal~\cite{wen2019single} synthesize training data with learned non-linear alpha blending masks that better model the real-world imaging conditions. These masks are also used in forming a reconstruction loss that guides the prediction of transmission layers. To deal with the insufficiency of densely-labeled training data, Wei \etal~\cite{wei2019single} present a technique for utilizing misaligned real-world images as the training data, as they are less burdensome to acquire than aligned images and are more realistic than synthetic images.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1708.03474" label="1708.03474">
-        <attvalues>
-          <attvalue for="0" value="A Generic Deep Architecture for Single Image Reflection Removal and&#10;  Image Smoothing" />
-          <attvalue for="1" value="  This paper proposes a deep neural network structure that exploits edge&#10;information in addressing representative low-level vision tasks such as layer&#10;separation and image filtering. Unlike most other deep learning strategies&#10;applied in this context, our approach tackles these challenging problems by&#10;estimating edges and reconstructing images using only cascaded convolutional&#10;layers arranged such that no handcrafted or application-specific&#10;image-processing components are required. We apply the resulting transferrable&#10;pipeline to two different problem domains that are both sensitive to edges,&#10;namely, single image reflection removal and image smoothing. For the former,&#10;using a mild reflection smoothness assumption and a novel synthetic data&#10;generation method that acts as a type of weak supervision, our network is able&#10;to solve much more difficult reflection cases that cannot be handled by&#10;previous methods. For the latter, we also exceed the state-of-the-art&#10;quantitative and qualitative results by wide margins. In all cases, the&#10;proposed framework is simple, fast, and easy to transfer across disparate&#10;domains.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1508.06576" label="1508.06576">
-        <attvalues>
-          <attvalue for="0" value="A Neural Algorithm of Artistic Style" />
-          <attvalue for="1" value="  In fine art, especially painting, humans have mastered the skill to create&#10;unique visual experiences through composing a complex interplay between the&#10;content and style of an image. Thus far the algorithmic basis of this process&#10;is unknown and there exists no artificial system with similar capabilities.&#10;However, in other key areas of visual perception such as object and face&#10;recognition near-human performance was recently demonstrated by a class of&#10;biologically inspired vision models called Deep Neural Networks. Here we&#10;introduce an artificial system based on a Deep Neural Network that creates&#10;artistic images of high perceptual quality. The system uses neural&#10;representations to separate and recombine content and style of arbitrary&#10;images, providing a neural algorithm for the creation of artistic images.&#10;Moreover, in light of the striking similarities between performance-optimised&#10;artificial neural networks and biological vision, our work offers a path&#10;forward to an algorithmic understanding of how humans create and perceive&#10;artistic imagery.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.12785" label="2210.12785">
-        <attvalues>
-          <attvalue for="0" value="An Improved RaftStereo Trained with A Mixed Dataset for the Robust&#10;  Vision Challenge 2022" />
-          <attvalue for="1" value="  Stereo-matching is a fundamental problem in computer vision. Despite recent&#10;progress by deep learning, improving the robustness is ineluctable when&#10;deploying stereo-matching models to real-world applications. Different from the&#10;common practices, i.e., developing an elaborate model to achieve robustness, we&#10;argue that collecting multiple available datasets for training is a cheaper way&#10;to increase generalization ability. Specifically, this report presents an&#10;improved RaftStereo trained with a mixed dataset of seven public datasets for&#10;the robust vision challenge (denoted as iRaftStereo_RVC). When evaluated on the&#10;training sets of Middlebury, KITTI-2015, and ETH3D, the model outperforms its&#10;counterparts trained with only one dataset, such as the popular Sceneflow.&#10;After fine-tuning the pre-trained model on the three datasets of the challenge,&#10;it ranks at 2nd place on the stereo leaderboard, demonstrating the benefits of&#10;mixed dataset pre-training.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Stereo matching is an everlasting research topic in computer vision, as it is a feasible way to recover the depth of environments and facilitates many downstream tasks, such as 3D reconstruction and robot navigation.&#10;In recent years, research has shifted from traditional methods, such as SGM~\cite{hirschmuller2007stereo} and ELAS~\cite{geiger2010efficient} to deep learning based ones~\cite{sceneflow,iccv19_autodispnet,cheng2020hierarchical,xu2022attention,li2022practical,cheng2019learning,ganet,wang2021pvstereo,du2019amnet,zhang2020adaptive,lipson2021raft}. &#10;These deep-learning methods have considerably improved stereo matching. Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet}). &#10;&#10;More recently, methods based on the strategy of excellent work on optical flow estimation, RAFT~\cite{teed2020raft}, iteratively update the disparity field using recurrent modules with the features sampled from the correlation volume, i.e., RaftStereo~\cite{lipson2021raft} and CREStereo~\cite{li2022practical}, establishing the state of the arts. &#10;However, although significant progress has been made on the model design, there is still a problem when deploying these models to diverse scenarios. This motivates the holding of the Robust Vision Challenge, which encourages people to develop robust vision systems that work well in various scenes. &#10;&#10;To take part in the challenge, we first build the technical scheme upon RaftStereo~\cite{lipson2021raft}. &#10;In terms of robustness, Ranftl~et al.~\cite{ranftl2020towards} revealed that training with mixed datasets is powerful even for zero-shot cross-dataset generalization. &#10;We believe that it is also true for stereo matching. Therefore, we collect seven public available datasets, i.e., Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}. Experiments show that pre-training on the mixed dataset is beneficial compared with training on a single large dataset. After fine-tuning the pre-trained model on the three training sets, it ranks at the 2nd place on the stereo track of the challenge. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Stereo Matching, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1905.07443" label="1905.07443">
-        <attvalues>
-          <attvalue for="0" value="AutoDispNet: Improving Disparity Estimation With AutoML" />
-          <attvalue for="1" value="  Much research work in computer vision is being spent on optimizing existing&#10;network architectures to obtain a few more percentage points on benchmarks.&#10;Recent AutoML approaches promise to relieve us from this effort. However, they&#10;are mainly designed for comparatively small-scale classification tasks. In this&#10;work, we show how to use and extend existing AutoML techniques to efficiently&#10;optimize large-scale U-Net-like encoder-decoder architectures. In particular,&#10;we leverage gradient-based neural architecture search and Bayesian optimization&#10;for hyperparameter search. The resulting optimization does not require a&#10;large-scale compute cluster. We show results on disparity estimation that&#10;clearly outperform the manually optimized baseline and reach state-of-the-art&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.13501" label="2010.13501">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Neural Architecture Search for Deep Stereo Matching" />
-          <attvalue for="1" value="  To reduce the human efforts in neural network design, Neural Architecture&#10;Search (NAS) has been applied with remarkable success to various high-level&#10;vision tasks such as classification and semantic segmentation. The underlying&#10;idea for the NAS algorithm is straightforward, namely, to enable the network&#10;the ability to choose among a set of operations (e.g., convolution with&#10;different filter sizes), one is able to find an optimal architecture that is&#10;better adapted to the problem at hand. However, so far the success of NAS has&#10;not been enjoyed by low-level geometric vision tasks such as stereo matching.&#10;This is partly due to the fact that state-of-the-art deep stereo matching&#10;networks, designed by humans, are already sheer in size. Directly applying the&#10;NAS to such massive structures is computationally prohibitive based on the&#10;currently available mainstream computing resources. In this paper, we propose&#10;the first end-to-end hierarchical NAS framework for deep stereo matching by&#10;incorporating task-specific human knowledge into the neural architecture search&#10;framework. Specifically, following the gold standard pipeline for deep stereo&#10;matching (i.e., feature extraction -- feature volume construction and dense&#10;matching), we optimize the architectures of the entire pipeline jointly.&#10;Extensive experiments show that our searched network outperforms all&#10;state-of-the-art deep stereo matching architectures and is ranked at the top 1&#10;accuracy on KITTI stereo 2012, 2015 and Middlebury benchmarks, as well as the&#10;top 1 on SceneFlow dataset with a substantial improvement on the size of the&#10;network and the speed of inference. The code is available at&#10;https://github.com/XuelianCheng/LEAStereo.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.02146" label="2203.02146">
-        <attvalues>
-          <attvalue for="0" value="Attention Concatenation Volume for Accurate and Efficient Stereo&#10;  Matching" />
-          <attvalue for="1" value="  Stereo matching is a fundamental building block for many vision and robotics&#10;applications. An informative and concise cost volume representation is vital&#10;for stereo matching of high accuracy and efficiency. In this paper, we present&#10;a novel cost volume construction method which generates attention weights from&#10;correlation clues to suppress redundant information and enhance&#10;matching-related information in the concatenation volume. To generate reliable&#10;attention weights, we propose multi-level adaptive patch matching to improve&#10;the distinctiveness of the matching cost at different disparities even for&#10;textureless regions. The proposed cost volume is named attention concatenation&#10;volume (ACV) which can be seamlessly embedded into most stereo matching&#10;networks, the resulting networks can use a more lightweight aggregation network&#10;and meanwhile achieve higher accuracy, e.g. using only 1/25 parameters of the&#10;aggregation network can achieve higher accuracy for GwcNet. Furthermore, we&#10;design a highly accurate network (ACVNet) based on our ACV, which achieves&#10;state-of-the-art performance on several benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Recently, CNN-based stereo models~\cite{zbontar2015computing, dispNetC2016large,kendall2017end,guo2019group,nie2019multi, yang2019hierarchical} have achieved impressive performance on most of the standard benchmarks. &#10;Most of them devote to improving the accuracy and efficiency of cost volume construction and cost aggregation, which are the two key steps of stereo matching. &#10;&#10;Cost volume construction. Existing cost volume representation can be roughly categorized into three types: the correlation volume, the concatenation volume and a combined volume by concatenating the two volumes. &#10;DispNetC~\cite{dispNetC2016large} utilizes a correlation layer to directly measures the similarities of left and right image features to form a single-channel cost volume for each disparity level. Then, 2D convolution is applied to aggregate contextual information. Such full correlation volume demands low memory and computational complexity, yet the encoded information is too limited (i.e. large content information loss in the channel dimension) to achieve a satisfactory accuracy. GC-Net~\cite{kendall2017end} uses the concatenation volume, which concatenate the left and right CNN features to form a 4D cost volume for all disparities. Such 4D concatenation volume preserves abundant content information from all feature channels and thus outperforms the correlation volume in terms of better accuracy. However, as the concatenation volume does not explicitly encodes similarity measures, it requires a deep stack of 3D convolutions to aggregate costs of all disparities from scratch. &#10;To overcome the above drawbacks, GwcNet~\cite{guo2019group} proposes the group-wise correlation volume and concatenates it with a compact concatenation volume to form a combined volume, which aims to combine the advantages of two volumes.&#10;However, directly concatenating two types of volumes without considering their respective characteristics yields an inefficient use of the complementary strengths in the two volumes. As a result, deep stacking 3D convolutions in the hourglass architecture are still demanded for cost aggregation in GwcNet~\cite{guo2019group}. &#10;&#10;Following the 4D combined cost volume,&#10;cascade cost volumes~\cite{shen2021cfnet,gu2020cascade,wang2021patchmatchnet} further reduce the memory and computational complexity of cost volume construction by building a cost volume pyramid in a coarse-to-fine manner to progressively narrow down the target disparity range and refine the depth map. However, such coarse-to-fine strategy inevitably involves accumulated errors, i.e. errors in a previous stage can hardly be compensated in the latter stages and in turn yield large errors for some cases. While our ACV only adjusts the weights of different disparities. Thus, although the attention weights are imperfect, the concatenation volume which contains rich context, can help amend errors via the subsequent aggregation network.&#10;&#10;Cost aggregation. The goal of this step is to aggregate contextual information in the initial cost volume to derive accurate similarity measures. Many existing methods ~\cite{chang2018pyramid, nie2019multi,chabra2019stereodrnet} exploit a deep 3D CNN to learn an effective similarity function from the cost volume. However, the computational and memory consumption is too high for time-constrained applications. To reduce the complexity, AANet~\cite{xu2020aanet} proposes an intra-scale and cross-scale cost aggregation algorithm to replace the conventional 3D convolutions which can achieve very fast inference speed with a sacrifice of nontrivial accuracy degradation. GANet~\cite{zhang2019ga} also tries to replace 3D convolutions with two guided aggregation layers, which achieves a higher accuracy using spatially dependent 3D aggregation, but at the cost of a higher aggregation time due to the two guided aggregation layers. Even, their final model still uses fifteen 3D convolutions.&#10;&#10;Cost volume construction and aggregation are two tightly-coupled modules which jointly determine the accuracy and efficiency of a stereo matching network. In this work, we propose a highly efficient yet informative cost volume representation, named attention concatenation volume, by using the similarity information encoded in the correlation volume to regularize the concatenation volume so that only a lightweight aggregation network is demanded to achieve an overall high efficiency and accuracy. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2203.11483" label="2203.11483">
-        <attvalues>
-          <attvalue for="0" value="Practical Stereo Matching via Cascaded Recurrent Network with Adaptive&#10;  Correlation" />
-          <attvalue for="1" value="  With the advent of convolutional neural networks, stereo matching algorithms&#10;have recently gained tremendous progress. However, it remains a great challenge&#10;to accurately extract disparities from real-world image pairs taken by&#10;consumer-level devices like smartphones, due to practical complicating factors&#10;such as thin structures, non-ideal rectification, camera module inconsistencies&#10;and various hard-case scenes. In this paper, we propose a set of innovative&#10;designs to tackle the problem of practical stereo matching: 1) to better&#10;recover fine depth details, we design a hierarchical network with recurrent&#10;refinement to update disparities in a coarse-to-fine manner, as well as a&#10;stacked cascaded architecture for inference; 2) we propose an adaptive group&#10;correlation layer to mitigate the impact of erroneous rectification; 3) we&#10;introduce a new synthetic dataset with special attention to difficult cases for&#10;better generalizing to real-world scenes. Our results not only rank 1st on both&#10;Middlebury and ETH3D benchmarks, outperforming existing state-of-the-art&#10;methods by a notable margin, but also exhibit high-quality details for&#10;real-life photos, which clearly demonstrates the efficacy of our contributions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06587" label="1904.06587">
-        <attvalues>
-          <attvalue for="0" value="GA-Net: Guided Aggregation Net for End-to-end Stereo Matching" />
-          <attvalue for="1" value="  In the stereo matching task, matching cost aggregation is crucial in both&#10;traditional methods and deep neural network models in order to accurately&#10;estimate disparities. We propose two novel neural net layers, aimed at&#10;capturing local and the whole-image cost dependencies respectively. The first&#10;is a semi-global aggregation layer which is a differentiable approximation of&#10;the semi-global matching, the second is the local guided aggregation layer&#10;which follows a traditional cost filtering strategy to refine thin structures.&#10;These two layers can be used to replace the widely used 3D convolutional layer&#10;which is computationally costly and memory-consuming as it has cubic&#10;computational/memory complexity. In the experiments, we show that nets with a&#10;two-layer guided aggregation block easily outperform the state-of-the-art&#10;GC-Net which has nineteen 3D convolutional layers. We also train a deep guided&#10;aggregation network (GA-Net) which gets better accuracies than state-of-the-art&#10;methods on both Scene Flow dataset and KITTI benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.07094" label="2103.07094">
-        <attvalues>
-          <attvalue for="0" value="PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo&#10;  Matching" />
-          <attvalue for="1" value="  Supervised learning with deep convolutional neural networks (DCNNs) has seen&#10;huge adoption in stereo matching. However, the acquisition of large-scale&#10;datasets with well-labeled ground truth is cumbersome and labor-intensive,&#10;making supervised learning-based approaches often hard to implement in&#10;practice. To overcome this drawback, we propose a robust and effective&#10;self-supervised stereo matching approach, consisting of a pyramid voting module&#10;(PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically,&#10;our OptStereo first builds multi-scale cost volumes, and then adopts a&#10;recurrent unit to iteratively update disparity estimations at high resolution;&#10;while our PVM can generate reliable semi-dense disparity images, which can be&#10;employed to supervise OptStereo training. Furthermore, we publish the&#10;HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under&#10;different illumination and weather conditions for research purposes. Extensive&#10;experimental results demonstrate the effectiveness and efficiency of our&#10;self-supervised stereo matching approach on the KITTI Stereo benchmarks and our&#10;HKUST-Drive dataset. PVStereo, our best-performing implementation, greatly&#10;outperforms all other state-of-the-art self-supervised stereo matching&#10;approaches. Our project page is available at sites.google.com/view/pvstereo.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.09099" label="1904.09099">
-        <attvalues>
-          <attvalue for="0" value="AMNet: Deep Atrous Multiscale Stereo Disparity Estimation Networks" />
-          <attvalue for="1" value="  In this paper, a new deep learning architecture for stereo disparity&#10;estimation is proposed. The proposed atrous multiscale network (AMNet) adopts&#10;an efficient feature extractor with depthwise-separable convolutions and an&#10;extended cost volume that deploys novel stereo matching costs on the deep&#10;features. A stacked atrous multiscale network is proposed to aggregate rich&#10;multiscale contextual information from the cost volume which allows for&#10;estimating the disparity with high accuracy at multiple scales. AMNet can be&#10;further modified to be a foreground-background aware network, FBA-AMNet, which&#10;is capable of discriminating between the foreground and the background objects&#10;in the scene at multiple scales. An iterative multitask learning method is&#10;proposed to train FBA-AMNet end-to-end. The proposed disparity estimation&#10;networks, AMNet and FBA-AMNet, show accurate disparity estimates and advance&#10;the state of the art on the challenging Middlebury, KITTI 2012, KITTI 2015, and&#10;Sceneflow stereo disparity estimation benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.03751" label="1909.03751">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching" />
-          <attvalue for="1" value="  State-of-the-art deep learning based stereo matching approaches treat&#10;disparity estimation as a regression problem, where loss function is directly&#10;defined on true disparities and their estimated ones. However, disparity is&#10;just a byproduct of a matching process modeled by cost volume, while indirectly&#10;learning cost volume driven by disparity regression is prone to overfitting&#10;since the cost volume is under constrained. In this paper, we propose to&#10;directly add constraints to the cost volume by filtering cost volume with&#10;unimodal distribution peaked at true disparities. In addition, variances of the&#10;unimodal distributions for each pixel are estimated to explicitly model&#10;matching uncertainty under different contexts. The proposed architecture&#10;achieves state-of-the-art performance on Scene Flow and two KITTI stereo&#10;benchmarks. In particular, our method ranked the $1^{st}$ place of KITTI 2012&#10;evaluation and the $4^{th}$ place of KITTI 2015 evaluation (recorded on&#10;2019.8.20). The codes of AcfNet are available at:&#10;https://github.com/DeepMotionAIResearch/DenseMatchingBenchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.07547" label="2109.07547">
-        <attvalues>
-          <attvalue for="0" value="RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching" />
-          <attvalue for="1" value="  We introduce RAFT-Stereo, a new deep architecture for rectified stereo based&#10;on the optical flow network RAFT. We introduce multi-level convolutional GRUs,&#10;which more efficiently propagate information across the image. A modified&#10;version of RAFT-Stereo can perform accurate real-time inference. RAFT-stereo&#10;ranks first on the Middlebury leaderboard, outperforming the next best method&#10;on 1px error by 29% and outperforms all published work on the ETH3D two-view&#10;stereo benchmark. Code is available at&#10;https://github.com/princeton-vl/RAFT-Stereo.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.01341" label="1907.01341">
-        <attvalues>
-          <attvalue for="0" value="Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot&#10;  Cross-dataset Transfer" />
-          <attvalue for="1" value="  The success of monocular depth estimation relies on large and diverse&#10;training sets. Due to the challenges associated with acquiring dense&#10;ground-truth depth across different environments at scale, a number of datasets&#10;with distinct characteristics and biases have emerged. We develop tools that&#10;enable mixing multiple datasets during training, even if their annotations are&#10;incompatible. In particular, we propose a robust training objective that is&#10;invariant to changes in depth range and scale, advocate the use of principled&#10;multi-objective learning to combine data from different sources, and highlight&#10;the importance of pretraining encoders on auxiliary tasks. Armed with these&#10;tools, we experiment with five diverse training datasets, including a new,&#10;massive data source: 3D films. To demonstrate the generalization power of our&#10;approach we use zero-shot cross-dataset transfer}, i.e. we evaluate on datasets&#10;that were not seen during training. The experiments confirm that mixing data&#10;from complementary sources greatly improves monocular depth estimation. Our&#10;approach clearly outperforms competing methods across diverse datasets, setting&#10;a new state of the art for monocular depth estimation. Some results are shown&#10;in the supplementary video at https://youtu.be/D46FzVyL9I8&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.14338" label="2003.14338">
-        <attvalues>
-          <attvalue for="0" value="TartanAir: A Dataset to Push the Limits of Visual SLAM" />
-          <attvalue for="1" value="  We present a challenging dataset, the TartanAir, for robot navigation tasks&#10;and more. The data is collected in photo-realistic simulation environments with&#10;the presence of moving objects, changing light and various weather conditions.&#10;By collecting data in simulations, we are able to obtain multi-modal sensor&#10;data and precise ground truth labels such as the stereo RGB image, depth image,&#10;segmentation, optical flow, camera poses, and LiDAR point cloud. We set up&#10;large numbers of environments with various styles and scenes, covering&#10;challenging viewpoints and diverse motion patterns that are difficult to&#10;achieve by using physical data collection platforms. In order to enable data&#10;collection at such a large scale, we develop an automatic pipeline, including&#10;mapping, trajectory sampling, data processing, and data verification. We&#10;evaluate the impact of various factors on visual SLAM algorithms using our&#10;data. The results of state-of-the-art algorithms reveal that the visual SLAM&#10;problem is far from solved. Methods that show good performance on established&#10;datasets such as KITTI do not perform well in more difficult scenarios.&#10;Although we use the simulation, our goal is to push the limits of Visual SLAM&#10;algorithms in the real world by providing a challenging benchmark for testing&#10;new methods, while also using a large diverse training data for learning-based&#10;methods. Our dataset is available at&#10;\url{http://theairlab.org/tartanair-dataset}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06534" label="1804.06534">
-        <attvalues>
-          <attvalue for="0" value="Falling Things: A Synthetic Dataset for 3D Object Detection and Pose&#10;  Estimation" />
-          <attvalue for="1" value="  We present a new dataset, called Falling Things (FAT), for advancing the&#10;state-of-the-art in object detection and 3D pose estimation in the context of&#10;robotics. By synthetically combining object models and backgrounds of complex&#10;composition and high graphical quality, we are able to generate photorealistic&#10;images with accurate 3D pose annotations for all objects in all images. Our&#10;dataset contains 60k annotated photos of 21 household objects taken from the&#10;YCB dataset. For each image, we provide the 3D poses, per-pixel class&#10;segmentation, and 2D/3D bounding box coordinates for all objects. To facilitate&#10;testing different input modalities, we provide mono and stereo RGB images,&#10;along with registered dense depth images. We describe in detail the generation&#10;process and statistical analysis of the data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.10716" label="2409.10716">
-        <attvalues>
-          <attvalue for="0" value="Online Learning via Memory: Retrieval-Augmented Detector Adaptation" />
-          <attvalue for="1" value="  This paper presents a novel way of online adapting any off-the-shelf object&#10;detection model to a novel domain without retraining the detector model.&#10;Inspired by how humans quickly learn knowledge of a new subject (e.g.,&#10;memorization), we allow the detector to look up similar object concepts from&#10;memory during test time. This is achieved through a retrieval augmented&#10;classification (RAC) module together with a memory bank that can be flexibly&#10;updated with new domain knowledge. We experimented with various off-the-shelf&#10;open-set detector and close-set detectors. With only a tiny memory bank (e.g.,&#10;10 images per category) and being training-free, our online learning method&#10;could significantly outperform baselines in adapting a detector to novel&#10;domains.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object detection has advanced significantly over the years. Traditionally, effective object detectors were developed by collecting a large amount of annotated training data and training the model offline, a process that could take days or weeks and result in models tightly coupled to specific datasets or domains, such as Faster-RCNN~\cite{fasterrcnn}, YOLO~\cite{yolo} and etc. on COCO dataset~\cite{cocodataset}. &#10;Recently, with advancements in foundational model-accelerated pseudo labeling pipelines and more comprehensive training datasets, object detectors have evolved from closed-set to open-world models. These new models, like GLIP~\cite{glip} and Grounding-DINO~\cite{g-dino}, are capable of detecting wide range of objects.&#10;&#10;However, applying these models to novel domains often results in poor detection performance. For example, the state-of-the-art open-set model Grounding-DINO\cite{g-dino}, primarily trained on large-scale natural scene datasets, achieves less than 1\% mAP on the aerial dataset DOTA~\cite{dota}.&#10;To adapt to new visual domains and class vocabularies, these models still require offline retraining or fine-tuning. Traditional offline learning paradigms face two major bottlenecks: (1) the need for extensive labeling of target-domain data to enable stable training, and (2) a time-consuming retraining/fine-tuning process. These overheads hinder the efficient adaptation of detector models in real-world scenarios.&#10;&#10;To address these challenges, we propose an innovative online learning framework that adapts any detector to new domains with minimal overhead.&#10;The overview of our framework is shown in Fig.~\ref{fig:offline_online_comparison} (b). Comparing to traditional offline learning in Fig.~\ref{fig:offline_online_comparison} (a) that requires detector re-training/finetuning, our method could freeze the off-the-shelf detector. Instead, we design two modules: (1) a updatable memory bank with a small set of label images in target domain; and (2) a retrieval-augmented module (RAC) to help classify new concepts by retrieving similar-apperance images and objects from the memory bank. &#10;&#10;Such an online learning paradigm provides following benefits: (1) it enables rapid any-domain adaptation as soon as minimal amounts of target-domain data (e.g., as less as 10 images per category) are labeled in the memory bank; (2) it allows the detector to conduct online continual learning for continuous improvement when more and more data are inferenced and/or labeled and added to the updatable memory bank; (3) all such capabilities comes without any exra computing cost in re-training/finetuning, which greatly facilites the real-world application deployment. &#10;&#10;The retrieval-augmented paradigm is relatively underexplored in online visual domain adaptation, but many of prior works has highlightened its potential in addressing visual online learning challenges. Notably, the retrieval-augmented paradigm has proven effective for large language models (LLMs) in addressing forgetting and hallucination in novel text domains (e.g., retrieval-augmented generation (RAG~\cite{rag}) and KNN-LM~\cite{knn-lm}). There are also existing works on retrieval-augmented visual models with three main categories: (1) Generalizing vision tasks by retrieving task-related images and tuning an additional task-specific encoder (e.g., REACT~\cite{learning-customized-visual-models-with-retrieval-augmented-knowledge}). (2) Solving long-tail classification-only problem by a huge augmented knowledge base~\cite{rac}. (3) Open-vocabulary object detection, which requires a complex multi-step training process and LLMs to function~\cite{retrieval-augmented-open-vocabulary-object-detection}. &#10;&#10;We construct an aerial domain adaptation benchmark with DOTA~\cite{dota} dataset. Experiments demonstrate the bottlenecks of pretrained close-set and open-set models (e.g., faster-rcnn, and grounding dino) in such novel domains. Without bells and whistles, our online learning method enables them to improve performance by a large margin and with minimal labeling efforts. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Cognitive Science, Online Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2303.05499" label="2303.05499">
-        <attvalues>
-          <attvalue for="0" value="Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set&#10;  Object Detection" />
-          <attvalue for="1" value="  In this paper, we present an open-set object detector, called Grounding DINO,&#10;by marrying Transformer-based detector DINO with grounded pre-training, which&#10;can detect arbitrary objects with human inputs such as category names or&#10;referring expressions. The key solution of open-set object detection is&#10;introducing language to a closed-set detector for open-set concept&#10;generalization. To effectively fuse language and vision modalities, we&#10;conceptually divide a closed-set detector into three phases and propose a tight&#10;fusion solution, which includes a feature enhancer, a language-guided query&#10;selection, and a cross-modality decoder for cross-modality fusion. While&#10;previous works mainly evaluate open-set object detection on novel categories,&#10;we propose to also perform evaluations on referring expression comprehension&#10;for objects specified with attributes. Grounding DINO performs remarkably well&#10;on all three settings, including benchmarks on COCO, LVIS, ODinW, and&#10;RefCOCO/+/g. Grounding DINO achieves a $52.5$ AP on the COCO detection&#10;zero-shot transfer benchmark, i.e., without any training data from COCO. It&#10;sets a new record on the ODinW zero-shot benchmark with a mean $26.1$ AP. Code&#10;will be available at \url{https://github.com/IDEA-Research/GroundingDINO}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10398" label="1711.10398">
-        <attvalues>
-          <attvalue for="0" value="DOTA: A Large-scale Dataset for Object Detection in Aerial Images" />
-          <attvalue for="1" value="  Object detection is an important and challenging problem in computer vision.&#10;Although the past decade has witnessed major advances in object detection in&#10;natural scenes, such successes have been slow to aerial imagery, not only&#10;because of the huge variation in the scale, orientation and shape of the object&#10;instances on the earth's surface, but also due to the scarcity of&#10;well-annotated datasets of objects in aerial scenes. To advance object&#10;detection research in Earth Vision, also known as Earth Observation and Remote&#10;Sensing, we introduce a large-scale Dataset for Object deTection in Aerial&#10;images (DOTA). To this end, we collect $2806$ aerial images from different&#10;sensors and platforms. Each image is of the size about 4000-by-4000 pixels and&#10;contains objects exhibiting a wide variety of scales, orientations, and shapes.&#10;These DOTA images are then annotated by experts in aerial image interpretation&#10;using $15$ common object categories. The fully annotated DOTA images contains&#10;$188,282$ instances, each of which is labeled by an arbitrary (8 d.o.f.)&#10;quadrilateral To build a baseline for object detection in Earth Vision, we&#10;evaluate state-of-the-art object detection algorithms on DOTA. Experiments&#10;demonstrate that DOTA well represents real Earth Vision applications and are&#10;quite challenging.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.11401" label="2005.11401">
-        <attvalues>
-          <attvalue for="0" value="Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" />
-          <attvalue for="1" value="  Large pre-trained language models have been shown to store factual knowledge&#10;in their parameters, and achieve state-of-the-art results when fine-tuned on&#10;downstream NLP tasks. However, their ability to access and precisely manipulate&#10;knowledge is still limited, and hence on knowledge-intensive tasks, their&#10;performance lags behind task-specific architectures. Additionally, providing&#10;provenance for their decisions and updating their world knowledge remain open&#10;research problems. Pre-trained models with a differentiable access mechanism to&#10;explicit non-parametric memory can overcome this issue, but have so far been&#10;only investigated for extractive downstream tasks. We explore a general-purpose&#10;fine-tuning recipe for retrieval-augmented generation (RAG) -- models which&#10;combine pre-trained parametric and non-parametric memory for language&#10;generation. We introduce RAG models where the parametric memory is a&#10;pre-trained seq2seq model and the non-parametric memory is a dense vector index&#10;of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG&#10;formulations, one which conditions on the same retrieved passages across the&#10;whole generated sequence, the other can use different passages per token. We&#10;fine-tune and evaluate our models on a wide range of knowledge-intensive NLP&#10;tasks and set the state-of-the-art on three open domain QA tasks, outperforming&#10;parametric seq2seq models and task-specific retrieve-and-extract architectures.&#10;For language generation tasks, we find that RAG models generate more specific,&#10;diverse and factual language than a state-of-the-art parametric-only seq2seq&#10;baseline.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.00172" label="1911.00172">
-        <attvalues>
-          <attvalue for="0" value="Generalization through Memorization: Nearest Neighbor Language Models" />
-          <attvalue for="1" value="  We introduce $k$NN-LMs, which extend a pre-trained neural language model (LM)&#10;by linearly interpolating it with a $k$-nearest neighbors ($k$NN) model. The&#10;nearest neighbors are computed according to distance in the pre-trained LM&#10;embedding space, and can be drawn from any text collection, including the&#10;original LM training data. Applying this augmentation to a strong Wikitext-103&#10;LM, with neighbors drawn from the original training set, our $k$NN-LM achieves&#10;a new state-of-the-art perplexity of 15.79 - a 2.9 point improvement with no&#10;additional training. We also show that this approach has implications for&#10;efficiently scaling up to larger training sets and allows for effective domain&#10;adaptation, by simply varying the nearest neighbor datastore, again without&#10;further training. Qualitatively, the model is particularly helpful in&#10;predicting rare patterns, such as factual knowledge. Together, these results&#10;strongly suggest that learning similarity between sequences of text is easier&#10;than predicting the next word, and that nearest neighbor search is an effective&#10;approach for language modeling in the long tail.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.07094" label="2301.07094">
-        <attvalues>
-          <attvalue for="0" value="Learning Customized Visual Models with Retrieval-Augmented Knowledge" />
-          <attvalue for="1" value="  Image-text contrastive learning models such as CLIP have demonstrated strong&#10;task transfer ability. The high generality and usability of these visual models&#10;is achieved via a web-scale data collection process to ensure broad concept&#10;coverage, followed by expensive pre-training to feed all the knowledge into&#10;model weights. Alternatively, we propose REACT, REtrieval-Augmented&#10;CusTomization, a framework to acquire the relevant web knowledge to build&#10;customized visual models for target domains. We retrieve the most relevant&#10;image-text pairs (~3% of CLIP pre-training data) from the web-scale database as&#10;external knowledge, and propose to customize the model by only training new&#10;modualized blocks while freezing all the original weights. The effectiveness of&#10;REACT is demonstrated via extensive experiments on classification, retrieval,&#10;detection and segmentation tasks, including zero, few, and full-shot settings.&#10;Particularly, on the zero-shot classification task, compared with CLIP, it&#10;achieves up to 5.4% improvement on ImageNet and 3.7% on the ELEVATER benchmark&#10;(20 datasets).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.11233" label="2202.11233">
-        <attvalues>
-          <attvalue for="0" value="Retrieval Augmented Classification for Long-Tail Visual Recognition" />
-          <attvalue for="1" value="  We introduce Retrieval Augmented Classification (RAC), a generic approach to&#10;augmenting standard image classification pipelines with an explicit retrieval&#10;module. RAC consists of a standard base image encoder fused with a parallel&#10;retrieval branch that queries a non-parametric external memory of pre-encoded&#10;images and associated text snippets. We apply RAC to the problem of long-tail&#10;classification and demonstrate a significant improvement over previous&#10;state-of-the-art on Places365-LT and iNaturalist-2018 (14.5% and 6.7%&#10;respectively), despite using only the training datasets themselves as the&#10;external information source. We demonstrate that RAC's retrieval module,&#10;without prompting, learns a high level of accuracy on tail classes. This, in&#10;turn, frees the base encoder to focus on common classes, and improve its&#10;performance thereon. RAC represents an alternative approach to utilizing large,&#10;pretrained models without requiring fine-tuning, as well as a first step&#10;towards more effectively making use of external memory within common computer&#10;vision architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.05687" label="2404.05687">
-        <attvalues>
-          <attvalue for="0" value="Retrieval-Augmented Open-Vocabulary Object Detection" />
-          <attvalue for="1" value="  Open-vocabulary object detection (OVD) has been studied with Vision-Language&#10;Models (VLMs) to detect novel objects beyond the pre-trained categories.&#10;Previous approaches improve the generalization ability to expand the knowledge&#10;of the detector, using 'positive' pseudo-labels with additional 'class' names,&#10;e.g., sock, iPod, and alligator. To extend the previous methods in two aspects,&#10;we propose Retrieval-Augmented Losses and visual Features (RALF). Our method&#10;retrieves related 'negative' classes and augments loss functions. Also, visual&#10;features are augmented with 'verbalized concepts' of classes, e.g., worn on the&#10;feet, handheld music player, and sharp teeth. Specifically, RALF consists of&#10;two modules: Retrieval Augmented Losses (RAL) and Retrieval-Augmented visual&#10;Features (RAF). RAL constitutes two losses reflecting the semantic similarity&#10;with negative vocabularies. In addition, RAF augments visual features with the&#10;verbalized concepts from a large language model (LLM). Our experiments&#10;demonstrate the effectiveness of RALF on COCO and LVIS benchmark datasets. We&#10;achieve improvement up to 3.4 box AP$_{50}^{\text{N}}$ on novel categories of&#10;the COCO dataset and 3.6 mask AP$_{\text{r}}$ gains on the LVIS dataset. Code&#10;is available at https://github.com/mlvlab/RALF .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.05913" label="2407.05913">
-        <attvalues>
-          <attvalue for="0" value="Submodular video object proposal selection for semantic object&#10;  segmentation" />
-          <attvalue for="1" value="  Learning a data-driven spatio-temporal semantic representation of the objects&#10;is the key to coherent and consistent labelling in video. This paper proposes&#10;to achieve semantic video object segmentation by learning a data-driven&#10;representation which captures the synergy of multiple instances from continuous&#10;frames. To prune the noisy detections, we exploit the rich information among&#10;multiple instances and select the discriminative and representative subset.&#10;This selection process is formulated as a facility location problem solved by&#10;maximising a submodular function. Our method retrieves the longer term&#10;contextual dependencies which underpins a robust semantic video object&#10;segmentation algorithm. We present extensive experiments on a challenging&#10;dataset that demonstrate the superior performance of our approach compared with&#10;the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10; The proliferation of user-uploaded videos which are frequently associated with semantic tags provides a vast resource for computer vision research. These semantic tags, albeit not spatially or temporally located in the video, suggest visual concepts appearing in the video. This social trend has led to an increasing interest in exploring the idea of segmenting video objects with weak supervision or labels. &#10;&#10;Hartmann {et al.} \cite{HartmannGHTKMVERS12} firstly formulated the problem as learning weakly supervised classifiers for a set of independent spatio-temporal segments.&#10;Tang {et al.} \cite{TangSY013} learned discrimative model by leveraging labelled positive videos and a large collection of negative examples based on distance matrix. Liu {et al.} \cite{LiuTSRCB14} extended the traditional binary classifition problem to multi-class and proposed nearest neighbor-based label transfer algorithm which encourages smoothness between regions that are spatio-temporally adjacent and similar in appearance. Zhang {et al.} \cite{ZhangCLWX15} utilized pre-trained object detector to generate a set of detections and then pruned noisy detections and regions by preserving spatio-temporal constraints. &#10;&#10;Graphical models have emerged as powerful tools in computer vision&#10;\cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, offering a versatile framework for representing and analyzing complex visual data.&#10;These approaches leverage the inherent structure and relationships within images&#10;\cite{wang2015robust,tinghuai2016method,xing2021learning} and videos&#10;\cite{wang2010video,wang2014wide,chen2020fine}, enabling more sophisticated and context-aware analysis. By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation&#10;\cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}. &#10;&#10;In contrast to previous works, we propose to learn a class-specific representation which captures the synergy of multiple instances from continuous frames. To prune the noisy detections, we exploit the rich information among multiple instances and select the discriminative and representative &#10;subset. In this framework, our algorithm is able to bridge the gap between image classification and video object segmentation, leveraging the ample pre-trained image recognition models rather than strongly-trained object detectors. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Machine Learning Algorithms, Mathematics, Artificial Intelligence, Computer Science, Video Object Segmentation, Spatio-Temporal Representation" />
-        </attvalues>
-      </node>
-      <node id="2001.06807" label="2001.06807">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks" />
-          <attvalue for="1" value="  This work proposes a novel attentive graph neural network (AGNN) for&#10;zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this&#10;task as a process of iterative information fusion over video graphs.&#10;Specifically, AGNN builds a fully connected graph to efficiently represent&#10;frames as nodes, and relations between arbitrary frame pairs as edges. The&#10;underlying pair-wise relations are described by a differentiable attention&#10;mechanism. Through parametric message passing, AGNN is able to efficiently&#10;capture and mine much richer and higher-order relations between video frames,&#10;thus enabling a more complete understanding of video content and more accurate&#10;foreground estimation. Experimental results on three video segmentation&#10;datasets show that AGNN sets a new state-of-the-art in each case. To further&#10;demonstrate the generalizability of our framework, we extend AGNN to an&#10;additional task: image object co-segmentation (IOCS). We perform experiments on&#10;two famous IOCS datasets and observe again the superiority of our AGNN model.&#10;The extensive experiments verify that AGNN is able to learn the underlying&#10;semantic/appearance relationships among video frames or related images, and&#10;discover the common objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.05924" label="2407.05924">
-        <attvalues>
-          <attvalue for="0" value="Graph-Boosted Attentive Network for Semantic Body Parsing" />
-          <attvalue for="1" value="  Human body parsing remains a challenging problem in natural scenes due to&#10;multi-instance and inter-part semantic confusions as well as occlusions. This&#10;paper proposes a novel approach to decomposing multiple human bodies into&#10;semantic part regions in unconstrained environments. Specifically we propose a&#10;convolutional neural network (CNN) architecture which comprises of novel&#10;semantic and contour attention mechanisms across feature hierarchy to resolve&#10;the semantic ambiguities and boundary localization issues related to semantic&#10;body parsing. We further propose to encode estimated pose as higher-level&#10;contextual information which is combined with local semantic cues in a novel&#10;graphical model in a principled manner. In this proposed model, the lower-level&#10;semantic cues can be recursively updated by propagating higher-level contextual&#10;information from estimated pose and vice versa across the graph, so as to&#10;alleviate erroneous pose information and pixel level predictions. We further&#10;propose an optimization technique to efficiently derive the solutions. Our&#10;proposed method achieves the state-of-art results on the challenging Pascal&#10;Person-Part dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.03772" label="2106.03772">
-        <attvalues>
-          <attvalue for="0" value="Learning Dynamics via Graph Neural Networks for Human Pose Estimation&#10;  and Tracking" />
-          <attvalue for="1" value="  Multi-person pose estimation and tracking serve as crucial steps for video&#10;understanding. Most state-of-the-art approaches rely on first estimating poses&#10;in each frame and only then implementing data association and refinement.&#10;Despite the promising results achieved, such a strategy is inevitably prone to&#10;missed detections especially in heavily-cluttered scenes, since this&#10;tracking-by-detection paradigm is, by nature, largely dependent on visual&#10;evidences that are absent in the case of occlusion. In this paper, we propose a&#10;novel online approach to learning the pose dynamics, which are independent of&#10;pose detections in current fame, and hence may serve as a robust estimation&#10;even in challenging scenarios including occlusion. Specifically, we derive this&#10;prediction of dynamics through a graph neural network~(GNN) that explicitly&#10;accounts for both spatial-temporal and visual information. It takes as input&#10;the historical pose tracklets and directly predicts the corresponding poses in&#10;the following frame for each tracklet. The predicted poses will then be&#10;aggregated with the detected poses, if any, at the same frame so as to produce&#10;the final pose, potentially recovering the occluded joints missed by the&#10;estimator. Experiments on PoseTrack 2017 and PoseTrack 2018 datasets&#10;demonstrate that the proposed method achieves results superior to the state of&#10;the art on both human pose estimation and tracking tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.01319" label="2107.01319">
-        <attvalues>
-          <attvalue for="0" value="Learning Hierarchical Graph Neural Networks for Image Clustering" />
-          <attvalue for="1" value="  We propose a hierarchical graph neural network (GNN) model that learns how to&#10;cluster a set of images into an unknown number of identities using a training&#10;set of images annotated with labels belonging to a disjoint set of identities.&#10;Our hierarchical GNN uses a novel approach to merge connected components&#10;predicted at each level of the hierarchy to form a new graph at the next level.&#10;Unlike fully unsupervised hierarchical clustering, the choice of grouping and&#10;complexity criteria stems naturally from supervision in the training set. The&#10;resulting method, Hi-LANDER, achieves an average of 54% improvement in F-score&#10;and 8% increase in Normalized Mutual Information (NMI) relative to current&#10;GNN-based clustering algorithms. Additionally, state-of-the-art GNN-based&#10;methods rely on separate models to predict linkage probabilities and node&#10;densities as intermediate steps of the clustering process. In contrast, our&#10;unified framework achieves a seven-fold decrease in computational cost. We&#10;release our training and inference code at&#10;https://github.com/dmlc/dgl/tree/master/examples/pytorch/hilander.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.00392" label="2003.00392">
-        <attvalues>
-          <attvalue for="0" value="Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning" />
-          <attvalue for="1" value="  Cross-modal retrieval between videos and texts has attracted growing&#10;attentions due to the rapid emergence of videos on the web. The current&#10;dominant approach for this problem is to learn a joint embedding space to&#10;measure cross-modal similarities. However, simple joint embeddings are&#10;insufficient to represent complicated visual and textual details, such as&#10;scenes, objects, actions and their compositions. To improve fine-grained&#10;video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model,&#10;which decomposes video-text matching into global-to-local levels. To be&#10;specific, the model disentangles texts into hierarchical semantic graph&#10;including three levels of events, actions, entities and relationships across&#10;levels. Attention-based graph reasoning is utilized to generate hierarchical&#10;textual embeddings, which can guide the learning of diverse and hierarchical&#10;video representations. The HGR model aggregates matchings from different&#10;video-text levels to capture both global and local details. Experimental&#10;results on three video-text datasets demonstrate the advantages of our model.&#10;Such hierarchical decomposition also enables better generalization across&#10;datasets and improves the ability to distinguish fine-grained semantic&#10;differences.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.00580" label="2108.00580">
-        <attvalues>
-          <attvalue for="0" value="GraphFPN: Graph Feature Pyramid Network for Object Detection" />
-          <attvalue for="1" value="  Feature pyramids have been proven powerful in image understanding tasks that&#10;require multi-scale features. State-of-the-art methods for multi-scale feature&#10;learning focus on performing feature interactions across space and scales using&#10;neural networks with a fixed topology. In this paper, we propose graph feature&#10;pyramid networks that are capable of adapting their topological structures to&#10;varying intrinsic image structures and supporting simultaneous feature&#10;interactions across all scales. We first define an image-specific superpixel&#10;hierarchy for each input image to represent its intrinsic image structures. The&#10;graph feature pyramid network inherits its structure from this superpixel&#10;hierarchy. Contextual and hierarchical layers are designed to achieve feature&#10;interactions within the same scale and across different scales. To make these&#10;layers more powerful, we introduce two types of local channel attention for&#10;graph neural networks by generalizing global channel attention for&#10;convolutional neural networks. The proposed graph feature pyramid network can&#10;enhance the multiscale features from a convolutional feature pyramid network.&#10;We evaluate our graph feature pyramid network in the object detection task by&#10;integrating it into the Faster R-CNN algorithm. The modified algorithm&#10;outperforms not only previous state-of-the-art feature pyramid-based methods&#10;with a clear margin but also other popular detection methods on both MS-COCO&#10;2017 validation and test datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.07108" label="2001.07108">
-        <attvalues>
-          <attvalue for="0" value="Spectral Pyramid Graph Attention Network for Hyperspectral Image&#10;  Classification" />
-          <attvalue for="1" value="  Convolutional neural networks (CNN) have made significant advances in&#10;hyperspectral image (HSI) classification. However, standard convolutional&#10;kernel neglects the intrinsic connections between data points, resulting in&#10;poor region delineation and small spurious predictions. Furthermore, HSIs have&#10;a unique continuous data distribution along the high dimensional spectrum&#10;domain - much remains to be addressed in characterizing the spectral contexts&#10;considering the prohibitively high dimensionality and improving reasoning&#10;capability in light of the limited amount of labelled data. This paper presents&#10;a novel architecture which explicitly addresses these two issues. Specifically,&#10;we design an architecture to encode the multiple spectral contextual&#10;information in the form of spectral pyramid of multiple embedding spaces. In&#10;each spectral embedding space, we propose graph attention mechanism to&#10;explicitly perform interpretable reasoning in the spatial domain based on the&#10;connection in spectral feature space. Experiments on three HSI datasets&#10;demonstrate that the proposed architecture can significantly improve the&#10;classification accuracy compared with the existing methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.02280" label="1606.02280">
-        <attvalues>
-          <attvalue for="0" value="Semi-Supervised Domain Adaptation for Weakly Labeled Semantic Video&#10;  Object Segmentation" />
-          <attvalue for="1" value="  Deep convolutional neural networks (CNNs) have been immensely successful in&#10;many high-level computer vision tasks given large labeled datasets. However,&#10;for video semantic object segmentation, a domain where labels are scarce,&#10;effectively exploiting the representation power of CNN with limited training&#10;data remains a challenge. Simply borrowing the existing pretrained CNN image&#10;recognition model for video segmentation task can severely hurt performance. We&#10;propose a semi-supervised approach to adapting CNN image recognition model&#10;trained from labeled image data to the target domain exploiting both semantic&#10;evidence learned from CNN, and the intrinsic structures of video data. By&#10;explicitly modeling and compensating for the domain shift from the source&#10;domain to the target domain, this proposed approach underpins a robust semantic&#10;object segmentation method against the changes in appearance, shape and&#10;occlusion in natural videos. We present extensive experiments on challenging&#10;datasets that demonstrate the superior performance of our approach compared&#10;with the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.07020" label="2007.07020">
-        <attvalues>
-          <attvalue for="0" value="Video Object Segmentation with Episodic Graph Memory Networks" />
-          <attvalue for="1" value="  How to make a segmentation model efficiently adapt to a specific video and to&#10;online target appearance variations are fundamentally crucial issues in the&#10;field of video object segmentation. In this work, a graph memory network is&#10;developed to address the novel idea of &quot;learning to update the segmentation&#10;model&quot;. Specifically, we exploit an episodic memory network, organized as a&#10;fully connected graph, to store frames as nodes and capture cross-frame&#10;correlations by edges. Further, learnable controllers are embedded to ease&#10;memory reading and writing, as well as maintain a fixed memory scale. The&#10;structured, external memory design enables our model to comprehensively mine&#10;and quickly store new knowledge, even with limited visual information, and the&#10;differentiable memory controllers slowly learn an abstract method for storing&#10;useful representations in the memory and how to later use these representations&#10;for prediction, via gradient descent. In addition, the proposed graph memory&#10;network yields a neat yet principled framework, which can generalize well both&#10;one-shot and zero-shot video object segmentation tasks. Extensive experiments&#10;on four challenging benchmark datasets verify that our graph memory network is&#10;able to facilitate the adaptation of the segmentation network for case-by-case&#10;video object segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.03145" label="2203.03145">
-        <attvalues>
-          <attvalue for="0" value="End-to-end video instance segmentation via spatial-temporal graph neural&#10;  networks" />
-          <attvalue for="1" value="  Video instance segmentation is a challenging task that extends image instance&#10;segmentation to the video domain. Existing methods either rely only on&#10;single-frame information for the detection and segmentation subproblems or&#10;handle tracking as a separate post-processing step, which limit their&#10;capability to fully leverage and share useful spatial-temporal information for&#10;all the subproblems. In this paper, we propose a novel graph-neural-network&#10;(GNN) based method to handle the aforementioned limitation. Specifically, graph&#10;nodes representing instance features are used for detection and segmentation&#10;while graph edges representing instance relations are used for tracking. Both&#10;inter and intra-frame information is effectively propagated and shared via&#10;graph updates and all the subproblems (i.e. detection, segmentation and&#10;tracking) are jointly optimized in an unified framework. The performance of our&#10;method shows great improvement on the YoutubeVIS validation dataset compared to&#10;existing methods and achieves 35.2% AP with a ResNet-50 backbone, operating at&#10;22 FPS. Code is available at http://github.com/lucaswithai/visgraph.git .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Image Instance Segmentation: The aim of instance segmentation is to group pixels into different object instances to get classes, segmentation masks and unique identities of objects\cite{hariharan2014simultaneous}. Most of the instance segmentation can be divided into two-stage methods and one-stage methods. Two-stage methods always generate detection bounding boxes first and predict masks on these boxes. Mask R-CNN \cite{he2017mask} is still the mostly widely used two-stage framework in image instance segmentation and video instance segmentation, which generate object proposal by region proposal network and use two seperate branches to predict bounding boxes and segmentation masks on ROI features obtained from proposal areas on the feature map. Mask Scoring R-CNN \cite{huang2019mask} designs a MaskIOU branch to evaluate the accuracy of mask, rather than directly using the classification score in Mask R-CNN. PANet \cite{wang2019panet} use bottom-up path augmentation, adaptive feature pooling, and fully-connected fusion to improve the accuracy of segmentation.&#10;In one-stage instance segmentation methods, masks are directly predicted from images without generating a large amount of proposals first. PolarMask \cite{xie2020polarmask} uses 16 rays to represent a mask and add these representation on FCOS \cite{tian2019fcos} framework to achieve instance segmentation. CondInst \cite{tian2020conditional} introduces conditional convolution into FCOS and uses dynamic instance-sensitive convolution filters to represent the masks. YOLACT \cite{bolya2019yolact} uses a group of templates and coefficients to represent masks and designs a fully convolutional network to predict them. &#10;&#10;Video instance segmentation: The video instance segmentation task is more challenging than video object segmentation\cite{li2020delving}, which requires classifying, segmenting instances in each frame and linking&#10;the same instance across frames \cite{yang2019video}. It can be applied in many other computer vision fields, including human behavior analysis\cite{lin2020human} and object position estimation\cite{zhang2021regional}. The baseline method MaskTrack R-CNN \cite{yang2019video} adds a embedding branch on Mask R-CNN \cite{he2017mask} and uses a historical instance feature queue to associate instance masks across frame. Maskprop \cite{bertasius2020classifying} propagates masks in previous frames to predict their positions and shapes in the later frames. VisTR \cite{wang2020end} introduces transformers and uses queries on encoding features to obtain instance sequence from video clips. SipMask \cite{cao2020sipmask} designs a novel image instance segmentation method and uses the same association strategy of MaskTrack R-CNN \cite{yang2019video} to link instance masks across frames. STEm-Seg \cite{athar2020stem} models video clips as 3D space-time volume and generate instance sequences by clustering pixels based on space-time embedding. {We found that most of these methods tracks and segments each instance separately, without extracting and utilizing relation information between instances, while our method use instance-level relations extracted by graph neural network to guide the feature fusion and learn joint detection, segmentation as well as tracking in one framework, which is not only more efficient but also can improve the accuracy of all VIS subproblems.}&#10;&#10;Graph Neural Networks for Relation and Tracking: Graph neural network was first proposed to process data with a graph&#10;structure using neural networks \cite{gori2005new}. In GNN, &#10;a graph with nodes and edges relating each other is constructed and features on nodes and edges will be updated based on relations. GNN has already been used in object detection and multi object tracking to improve the accuracy. In object detection, GNN is often used to fuse the features of related objects or proposals to increase the detection accuracy \cite{wang2020joint,chen2020relation, shi2020point, xu2020cross}. As for&#10;MOT, most of the methods formulates data association as an edge classification problem using GNNs \cite{braso2020learning, jiang2019graph, li2020graph, weng2020gnn3dmot}, where&#10;each node represents a detection or a sequence of an object and each edge denotes the similarity between detection and sequences.&#10;After aggregating node features and edge features based on object relations, the final association results can be obtained by classifying the edges. However, in MOT, tracking is usually a separate step from other tasks such as detection. While in our method graph is leveraged to fuse spatial-temporal features which are useful for not only tracking but also other tasks such as detection and segmentation.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2106.07409" label="2106.07409">
-        <attvalues>
-          <attvalue for="0" value="3rd Place Solution for Short-video Face Parsing Challenge" />
-          <attvalue for="1" value="  This is a short technical report introducing the solution of Team Rat for&#10;Short-video Parsing Face Parsing Track of The 3rd Person in Context (PIC)&#10;Workshop and Challenge at CVPR 2021.&#10;  In this report, we propose an Edge-Aware Network (EANet) that uses edge&#10;information to refine the segmentation edge. To further obtain the finer edge&#10;results, we introduce edge attention loss that only compute cross entropy on&#10;the edges, it can effectively reduce the classification error around edge and&#10;get more smooth boundary. Benefiting from the edge information and edge&#10;attention loss, the proposed EANet achieves 86.16\% accuracy in the Short-video&#10;Face Parsing track of the 3rd Person in Context (PIC) Workshop and Challenge,&#10;ranked the third place.&#10;" />
-          <attvalue for="2" value="&#10;Face Parsing is a particular task in semantic segmentation, it needs to predict the semantic category of each pixel on the human face, such as hair, eye, mouth, nose, etc. Compared with face detection and facial keypoint detection, the face parsing can provide more attribute information for further application. &#10;&#10;There are a lot of works~\cite{2020Object, li2020improving, chen2018encoderdecoder, liu2021swin, wu2020ainnoseg, long2015fully,wang2020solov2, he2018mask, tao2020hierarchical, lin2019face,ruan2018devil} studied face parsing and scene parsing. But the some of them ignore the edge information, mean while, in this challenge, there are accuracy requirements for edge segmentation. Motivated by this, we propose Edge-Aware Network (EANet) mining the edge information to improve segmentation results. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Edge Detection" />
-        </attvalues>
-      </node>
-      <node id="1809.05996" label="1809.05996">
-        <attvalues>
-          <attvalue for="0" value="Devil in the Details: Towards Accurate Single and Multiple Human Parsing" />
-          <attvalue for="1" value="  Human parsing has received considerable interest due to its wide application&#10;potentials. Nevertheless, it is still unclear how to develop an accurate human&#10;parsing system in an efficient and elegant way. In this paper, we identify&#10;several useful properties, including feature resolution, global context&#10;information and edge details, and perform rigorous analyses to reveal how to&#10;leverage them to benefit the human parsing task. The advantages of these useful&#10;properties finally result in a simple yet effective Context Embedding with Edge&#10;Perceiving (CE2P) framework for single human parsing. Our CE2P is end-to-end&#10;trainable and can be easily adopted for conducting multiple human parsing.&#10;Benefiting the superiority of CE2P, we achieved the 1st places on all three&#10;human parsing benchmarks. Without any bells and whistles, we achieved 56.50\%&#10;(mIoU), 45.31\% (mean $AP^r$) and 33.34\% ($AP^p_{0.5}$) in LIP, CIHP and MHP&#10;v2.0, which outperform the state-of-the-arts more than 2.06\%, 3.81\% and&#10;1.87\%, respectively. We hope our CE2P will serve as a solid baseline and help&#10;ease future research in single/multiple human parsing. Code has been made&#10;available at \url{https://github.com/liutinglt/CE2P}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.04887" label="2011.04887">
-        <attvalues>
-          <attvalue for="0" value="CoADNet: Collaborative Aggregation-and-Distribution Networks for&#10;  Co-Salient Object Detection" />
-          <attvalue for="1" value="  Co-Salient Object Detection (CoSOD) aims at discovering salient objects that&#10;repeatedly appear in a given query group containing two or more relevant&#10;images. One challenging issue is how to effectively capture co-saliency cues by&#10;modeling and exploiting inter-image relationships. In this paper, we present an&#10;end-to-end collaborative aggregation-and-distribution network (CoADNet) to&#10;capture both salient and repetitive visual patterns from multiple images.&#10;First, we integrate saliency priors into the backbone features to suppress the&#10;redundant background information through an online intra-saliency guidance&#10;structure. After that, we design a two-stage aggregate-and-distribute&#10;architecture to explore group-wise semantic interactions and produce the&#10;co-saliency features. In the first stage, we propose a group-attentional&#10;semantic aggregation module that models inter-image relationships to generate&#10;the group-wise semantic representations. In the second stage, we propose a&#10;gated group distribution module that adaptively distributes the learned group&#10;semantics to different individuals in a dynamic gating mechanism. Finally, we&#10;develop a group consistency preserving decoder tailored for the CoSOD task,&#10;which maintains group constraints during feature decoding to predict more&#10;consistent full-resolution co-saliency maps. The proposed CoADNet is evaluated&#10;on four prevailing CoSOD benchmark datasets, which demonstrates the remarkable&#10;performance improvement over ten state-of-the-art competitors.&#10;" />
-          <attvalue for="2" value="&#10;Different from salient object detection (SOD) that extracts the most attractive regions/objects from a single image, co-salient object detection (CoSOD) aims to discover the salient and repetitive objects from an image group containing two or more relevant images, which is consistent with the collaborative processing mechanism of human visual systems \cite{crm_review, deng2020re, zdw_review}. Mining co-salient patterns across image groups has been widely applied in various down-streaming computer vision tasks, such as image co-segmentation \cite{cosal_app_1, cosal_app_6, DeepCO3}, object detection \cite{cosal_app_7, cosal_app_8}, and image retrieval \cite{cosal_app_9}.&#10;Conceptually, the co-salient objects belong to the same semantic category, but their category attributes are unknown under the context of CoSOD, which is different from SOD for single image or video sequences \cite{crm2020tc, crm2019tip, crm2016spl, crm2019tgrs, lcy2020tc, eccv20}. Therefore, co-saliency models are supposed to capture discriminative inter-image semantic relationships without the supervision of specific category labels, which makes CoSOD a challenging problem especially when facing co-salient objects with distinct appearance variations. Starting from the pair-wise CoSOD model \cite{CSpair}, researchers have designed different unsupervised approaches for inter-image relationship extraction, such as clustering \cite{CCS}, rank constraint \cite{SWCS}, similarity matching \cite{ECS,icme18}, and depth cue \cite{MFLP,crm2019tmm,crm2019tc}. Due to the limited discrimination of hand-crafted descriptors, these methods cannot well extract the high-level object semantics, leading to unsatisfactory performance in complex scenes. In recent years, learning based methods have achieved more competitive performance by generating more robust and representative co-saliency features, including un-/semi- supervised learning methods \cite{DeepCO3, GoNet, HC, SAFS, CODW, FASS}, fully supervised methods based on Convolutional Neural Networks (CNNs) \cite{TIP18, RCNet, MCFIP, RCGS, GCS, CSMG}, and Graph Convolutional Networks (GCNs) \cite{MGCNet,MGL,GCAGC}. However, based on our observations, there are still three aspects of critical issues that need to be further considered and addressed in terms of modeling and exploiting inter-image relationships for co-saliency detection:&#10;&#10;(1) Insufficient group-wise relationship modeling. Previous studies adopted the recurrent learning \cite{RCNet} and concatenation-convolution \cite{GCS} strategies to model the inter-image relationships. However, the learned group representations vary with different order of the input group images, leading to unstable training and vulnerable inference. Thus, we propose a group-attentional semantic aggregation module to generate order-insensitive group semantics. Besides, in order to tackle the spatial variation of co-saliency patterns, we further consider long-range semantic dependencies across group images through a global attention mechanism when exploring the inter-image correspondences.&#10;&#10;(2) Competition between intra-image saliency and inter-image correspondence. Despite the great efforts to boost the learning of group-wise representations, little attention is paid to the effective and reasonable group-individual combination. In existing works \cite{RCNet, RCGS}, the learned group semantics were directly duplicated and concatenated with individual features. In fact, this operation implies that different individuals receive identical group semantics, which may propagate redundant and distracting information from the interactions among other images. To solve this issue, we design a gated group distribution module to learn dynamic combination weights between individual features and group-wise representations, thereby allowing different individuals to adaptively select useful group-wise information that is conducive to their own co-saliency prediction.&#10;&#10;(3) Weakened group consistency during feature decoding.&#10;In the feature decoding of the CoSOD task, existing up-sampling or deconvolution based methods \cite{CSMG,GCAGC} ignore the maintenance of inter-image consistency, which may lead to the inconsistency of co-salient objects among different images and introduce additional artifacts. Motivated by this problem, we develop a group consistency preserving decoder that further maintains inter-image relationships during the feature decoding process, predicting more consistent full-resolution co-saliency maps.&#10;&#10;In this paper, we present an end-to-end collaborative aggregation-and-distribution network (CoADNet) for the CoSOD task, and the main contributions can be summarized as follows.&#10;&#10;(1) The proposed CoADNet provides some insights and improvements in terms of modeling and exploiting inter-image relationships in the CoSOD workflow, and produces more accurate and consistent co-saliency results on four prevailing co-saliency benchmark datasets.&#10;&#10;(2) We design an online intra-saliency guidance (OIaSG) module for supplying saliency prior knowledge, which is jointly optimized to generate trainable saliency guidance information. In this way, the network dynamically learns how to combine the saliency cues with deep individual features, which has better flexibility and expansibility in providing reliable saliency priors.&#10;&#10;(3) We propose a two-stage aggregate-and-distribute architecture to learn group-wise correspondences and co-saliency features. In the first stage, a group-attentional semantic aggregation (GASA) module is proposed to model inter-image relationships with long-range semantic dependencies. In the second stage, we propose a gated group distribution (GGD) module to distribute the learned group semantics to different individuals in a dynamic and unique way.&#10;&#10;(4) A group consistency preserving decoder (GCPD) is designed to replace conventional up-sampling or deconvolution driven feature decoding structures, which exploits more sufficient inter-image constraints to generate full-resolution co-saliency maps while maintaining group-wise consistency.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Cognitive Science, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1710.05172" label="1710.05172">
-        <attvalues>
-          <attvalue for="0" value="Co-saliency Detection for RGBD Images Based on Multi-constraint Feature&#10;  Matching and Cross Label Propagation" />
-          <attvalue for="1" value="  Co-saliency detection aims at extracting the common salient regions from an&#10;image group containing two or more relevant images. It is a newly emerging&#10;topic in computer vision community. Different from the most existing&#10;co-saliency methods focusing on RGB images, this paper proposes a novel&#10;co-saliency detection model for RGBD images, which utilizes the depth&#10;information to enhance identification of co-saliency. First, the intra saliency&#10;map for each image is generated by the single image saliency model, while the&#10;inter saliency map is calculated based on the multi-constraint feature&#10;matching, which represents the constraint relationship among multiple images.&#10;Then, the optimization scheme, namely Cross Label Propagation (CLP), is used to&#10;refine the intra and inter saliency maps in a cross way. Finally, all the&#10;original and optimized saliency maps are integrated to generate the final&#10;co-saliency result. The proposed method introduces the depth information and&#10;multi-constraint feature matching to improve the performance of co-saliency&#10;detection. Moreover, the proposed method can effectively exploit any existing&#10;single image saliency model to work well in co-saliency scenarios. Experiments&#10;on two RGBD co-saliency datasets demonstrate the effectiveness of our proposed&#10;model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.06679" label="1811.06679">
-        <attvalues>
-          <attvalue for="0" value="HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images" />
-          <attvalue for="1" value="  Co-saliency detection aims to discover common and salient objects in an image&#10;group containing more than two relevant images. Moreover, depth information has&#10;been demonstrated to be effective for many computer vision tasks. In this&#10;paper, we propose a novel co-saliency detection method for RGBD images based on&#10;hierarchical sparsity reconstruction and energy function refinement. With the&#10;assistance of the intra saliency map, the inter-image correspondence is&#10;formulated as a hierarchical sparsity reconstruction framework. The global&#10;sparsity reconstruction model with a ranking scheme focuses on capturing the&#10;global characteristics among the whole image group through a common foreground&#10;dictionary. The pairwise sparsity reconstruction model aims to explore the&#10;corresponding relationship between pairwise images through a set of pairwise&#10;dictionaries. In order to improve the intra-image smoothness and inter-image&#10;consistency, an energy function refinement model is proposed, which includes&#10;the unary data term, spatial smooth term, and holistic consistency term.&#10;Experiments on two RGBD co-saliency detection benchmarks demonstrate that the&#10;proposed method outperforms the state-of-the-art algorithms both qualitatively&#10;and quantitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.01371" label="1711.01371">
-        <attvalues>
-          <attvalue for="0" value="An Iterative Co-Saliency Framework for RGBD Images" />
-          <attvalue for="1" value="  As a newly emerging and significant topic in computer vision community,&#10;co-saliency detection aims at discovering the common salient objects in&#10;multiple related images. The existing methods often generate the co-saliency&#10;map through a direct forward pipeline which is based on the designed cues or&#10;initialization, but lack the refinement-cycle scheme. Moreover, they mainly&#10;focus on RGB image and ignore the depth information for RGBD images. In this&#10;paper, we propose an iterative RGBD co-saliency framework, which utilizes the&#10;existing single saliency maps as the initialization, and generates the final&#10;RGBD cosaliency map by using a refinement-cycle model. Three schemes are&#10;employed in the proposed RGBD co-saliency framework, which include the addition&#10;scheme, deletion scheme, and iteration scheme. The addition scheme is used to&#10;highlight the salient regions based on intra-image depth propagation and&#10;saliency propagation, while the deletion scheme filters the saliency regions&#10;and removes the non-common salient regions based on interimage constraint. The&#10;iteration scheme is proposed to obtain more homogeneous and consistent&#10;co-saliency map. Furthermore, a novel descriptor, named depth shape prior, is&#10;proposed in the addition scheme to introduce the depth information to enhance&#10;identification of co-salient objects. The proposed method can effectively&#10;exploit any existing 2D saliency model to work well in RGBD co-saliency&#10;scenarios. The experiments on two RGBD cosaliency datasets demonstrate the&#10;effectiveness of our proposed framework.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.06167" label="2003.06167">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Graph Convolutional Network with Attention Graph Clustering for&#10;  Co-saliency Detection" />
-          <attvalue for="1" value="  Co-saliency detection aims to discover the common and salient foregrounds&#10;from a group of relevant images. For this task, we present a novel adaptive&#10;graph convolutional network with attention graph clustering (GCAGC). Three&#10;major contributions have been made, and are experimentally shown to have&#10;substantial practical merits. First, we propose a graph convolutional network&#10;design to extract information cues to characterize the intra- and interimage&#10;correspondence. Second, we develop an attention graph clustering algorithm to&#10;discriminate the common objects from all the salient foreground objects in an&#10;unsupervised fashion. Third, we present a unified framework with&#10;encoder-decoder structure to jointly train and optimize the graph convolutional&#10;network, attention graph cluster, and co-saliency detection decoder in an&#10;end-to-end manner. We evaluate our proposed GCAGC method on three cosaliency&#10;detection benchmark datasets (iCoseg, Cosal2015 and COCO-SEG). Our GCAGC method&#10;obtains significant improvements over the state-of-the-arts on most of them.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.07381" label="1707.07381">
-        <attvalues>
-          <attvalue for="0" value="Group-wise Deep Co-saliency Detection" />
-          <attvalue for="1" value="  In this paper, we propose an end-to-end group-wise deep co-saliency detection&#10;approach to address the co-salient object discovery problem based on the fully&#10;convolutional network (FCN) with group input and group output. The proposed&#10;approach captures the group-wise interaction information for group images by&#10;learning a semantics-aware image representation based on a convolutional neural&#10;network, which adaptively learns the group-wise features for co-saliency&#10;detection. Furthermore, the proposed approach discovers the collaborative and&#10;interactive relationships between group-wise feature representation and&#10;single-image individual feature representation, and model this in a&#10;collaborative learning framework. Finally, we set up a unified end-to-end deep&#10;learning scheme to jointly optimize the process of group-wise feature&#10;representation learning and the collaborative learning, leading to more&#10;reliable and robust co-saliency detection results. Experimental results&#10;demonstrate the effectiveness of our approach in comparison with the&#10;state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.13393" label="2007.13393">
-        <attvalues>
-          <attvalue for="0" value="Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D&#10;  Reconstruction with Symmetry" />
-          <attvalue for="1" value="  Deep implicit field regression methods are effective for 3D reconstruction&#10;from single-view images. However, the impact of different sampling patterns on&#10;the reconstruction quality is not well-understood. In this work, we first study&#10;the effect of point set discrepancy on the network training. Based on Farthest&#10;Point Sampling algorithm, we propose a sampling scheme that theoretically&#10;encourages better generalization performance, and results in fast convergence&#10;for SGD-based optimization algorithms. Secondly, based on the reflective&#10;symmetry of an object, we propose a feature fusion method that alleviates&#10;issues due to self-occlusions which makes it difficult to utilize local image&#10;features. Our proposed system Ladybird is able to create high quality 3D object&#10;reconstructions from a single input image. We evaluate Ladybird on a large&#10;scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms&#10;of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).&#10;" />
-          <attvalue for="2" value="&#10;&#10;\blfootnote{$*$These two authors contribute equally. &#10;~~~~~~~~~~~$\dagger$Corresponding author.&#10;\\ ORCIDs: Yi Yuan\orcidID{0000-0003-2507-8181} Gurprit Singh\orcidID{0000-0003-0970-5835}~}&#10;&#10;Due to the under-constrained nature of the problem, 3D object reconstruction from a single-view image has been a challenging task. Large shape and structure variations among objects make it difficult to define one dedicated parameterized model. Methods based on template deformation are often restricted by the initial topology of the template, and are not able to recover holes for instance.&#10;Recently, deep learning based implicit fields regression methods have shown great potential in monocular 3D reconstruction. &#10;Mescheder et al.~\cite{mescheder2019occupancy} and DISN~\cite{xu2019disn} create visually pleasing smooth shape reconstruction, with consistent normal and complex topology using implicit fields.&#10;&#10;An implicit field is a real-valued function defined on $\mathbb{R}^3$ whose iso-surface recovers the mesh of interest. Common choices of implicit field are signed distance field, truncated signed distance field, or occupancy probability field. A network $g_w(I, p)$ is trained to predict the implicit field of point $p \in \mathbb{R}^3$, based on the input image $I$, where $w$ are the parameters which are optimized with stochastic gradient descent (SGD) type algorithms. This is followed by post-processing methods like marching cube and sphere tracing to reconstruct the mesh.&#10;&#10;The loss function for the implicit field regression problem is the $L_2$ distance between the ground truth implicit field and the network $g_w$ predicted output. During training, a sparse set of 3D points need to be sampled in a compact region containing the mesh to approximate the optimization objective. &#10;We formulate this empirical loss as a Monte Carlo estimator.&#10;&#10;While most prior discussion on sampling~\cite{mescheder2019occupancy} focuses on designing a probability measure for the integral that puts different weights for regions of different distance to the mesh surface, &#10;we look at the problem from a point view of discrepancy of the sample sets. &#10;When approximating an integral, different samplers have different error convergence rates with respect to the sample size~\cite{pilleboue2015variance} \cite{niederreiter1988low}.&#10;Low discrepancy sequences/points or blue noise (in 2D) samples give better estimation, for instance, compared to random samples (white noise). &#10;&#10;Given a set of locally uniform samples whose distance to the target mesh is bounded by a threshold, we show that farthest point sampling algorithm (FPS) can be used to select a sparse subset with low discrepancy for training $g_w$. An overview of our method is shown in Figure~\ref{fig:first}. Our proposed sampling scheme results in better generalization performance as it provides better approximation to the expected loss, thanks to the Koksma-Hlawka inequality \cite{kuipers2012uniform}. Empirically our sampling scheme also results in faster convergence for SGD-based optimization algorithms, which speeds up the training process significantly as shown in Figure~\ref{fig:first}(e,f).&#10;&#10;Many deep 3D implicit field reconstruction works~\cite{mescheder2019occupancy} \cite{chen2019learning} explore the use of global shape encoding. While being good at capturing the general shape and obtaining interesting interpolation in the latent space, sometimes it is difficult to recover fine geometric details with only global features. Local features found via aligning image to mesh by modeling the camera are used to address the issue. However, for occluded points, it is ambiguous what local features should be used. Usually all the sampled points are projected to the images \cite{xu2019disn}, and hence points in the back use features of the points that occlude them.&#10;&#10;As most man-made objects are symmetric about a plane, we observe that this problem can be alleviated via the consideration of reflective symmetry. For a symmetric pair of points $p$ and $q$, the implicit fields at $p$ and $q$ are the same, and often at least one of them is visible in the image. Hence we can use the local features of $q$ to improve the implicit field predication of $p$, which can also be understood as utilizing two-view information. Our feature fusion method imposes a symmetry prior on the network $g_w$, which gives significant improvement of the reconstruction quality as shown in Figure~\ref{fig:big}. Unlike previous works \cite{wang20193dn} \cite{yao2019front2back} that focus on the design of loss function, detection or encoding of symmetry, our method naturally integrates into the pixel-to-mesh alignment framework.&#10;&#10;The advantage of spatially aligning the image to mesh and utilizing the corresponding local features is that the fine shape details and textures can be better recovered. However, when $p$ is occluded, the feature obtained by such alignment no longer has an intuitive meaning. Recently Front2Back \cite{yao2019front2back} addresses such issues by detecting reflective symmetries from the data and synthesizing the opposite orthographic view. Our approach is simpler and does not depend on symmetry detection.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1812.03828" label="1812.03828">
-        <attvalues>
-          <attvalue for="0" value="Occupancy Networks: Learning 3D Reconstruction in Function Space" />
-          <attvalue for="1" value="  With the advent of deep neural networks, learning-based approaches for 3D&#10;reconstruction have gained popularity. However, unlike for images, in 3D there&#10;is no canonical representation which is both computationally and memory&#10;efficient yet allows for representing high-resolution geometry of arbitrary&#10;topology. Many of the state-of-the-art learning-based 3D reconstruction&#10;approaches can hence only represent very coarse 3D geometry or are limited to a&#10;restricted domain. In this paper, we propose Occupancy Networks, a new&#10;representation for learning-based 3D reconstruction methods. Occupancy networks&#10;implicitly represent the 3D surface as the continuous decision boundary of a&#10;deep neural network classifier. In contrast to existing approaches, our&#10;representation encodes a description of the 3D output at infinite resolution&#10;without excessive memory footprint. We validate that our representation can&#10;efficiently encode 3D structure and can be inferred from various kinds of&#10;input. Our experiments demonstrate competitive results, both qualitatively and&#10;quantitatively, for the challenging tasks of 3D reconstruction from single&#10;images, noisy point clouds and coarse discrete voxel grids. We believe that&#10;occupancy networks will become a useful tool in a wide variety of&#10;learning-based 3D tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.02822" label="1812.02822">
-        <attvalues>
-          <attvalue for="0" value="Learning Implicit Fields for Generative Shape Modeling" />
-          <attvalue for="1" value="  We advocate the use of implicit fields for learning generative models of&#10;shapes and introduce an implicit field decoder, called IM-NET, for shape&#10;generation, aimed at improving the visual quality of the generated shapes. An&#10;implicit field assigns a value to each point in 3D space, so that a shape can&#10;be extracted as an iso-surface. IM-NET is trained to perform this assignment by&#10;means of a binary classifier. Specifically, it takes a point coordinate, along&#10;with a feature vector encoding a shape, and outputs a value which indicates&#10;whether the point is outside the shape or not. By replacing conventional&#10;decoders by our implicit decoder for representation learning (via IM-AE) and&#10;shape generation (via IM-GAN), we demonstrate superior results for tasks such&#10;as generative shape modeling, interpolation, and single-view 3D reconstruction,&#10;particularly in terms of visual quality. Code and supplementary material are&#10;available at https://github.com/czq142857/implicit-decoder.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.03322" label="1903.03322">
-        <attvalues>
-          <attvalue for="0" value="3DN: 3D Deformation Network" />
-          <attvalue for="1" value="  Applications in virtual and augmented reality create a demand for rapid&#10;creation and easy access to large sets of 3D models. An effective way to&#10;address this demand is to edit or deform existing 3D models based on a&#10;reference, e.g., a 2D image which is very easy to acquire. Given such a source&#10;3D model and a target which can be a 2D image, 3D model, or a point cloud&#10;acquired as a depth scan, we introduce 3DN, an end-to-end network that deforms&#10;the source model to resemble the target. Our method infers per-vertex offset&#10;displacements while keeping the mesh connectivity of the source model fixed. We&#10;present a training strategy which uses a novel differentiable operation, mesh&#10;sampling operator, to generalize our method across source and target models&#10;with varying mesh densities. Mesh sampling operator can be seamlessly&#10;integrated into the network to handle meshes with different topologies.&#10;Qualitative and quantitative results show that our method generates higher&#10;quality results compared to the state-of-the art learning-based methods for 3D&#10;shape generation. Code is available at github.com/laughtervv/3DN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.10589" label="1912.10589">
-        <attvalues>
-          <attvalue for="0" value="Front2Back: Single View 3D Shape Reconstruction via Front to Back&#10;  Prediction" />
-          <attvalue for="1" value="  Reconstruction of a 3D shape from a single 2D image is a classical computer&#10;vision problem, whose difficulty stems from the inherent ambiguity of&#10;recovering occluded or only partially observed surfaces. Recent methods address&#10;this challenge through the use of largely unstructured neural networks that&#10;effectively distill conditional mapping and priors over 3D shape. In this work,&#10;we induce structure and geometric constraints by leveraging three core&#10;observations: (1) the surface of most everyday objects is often almost entirely&#10;exposed from pairs of typical opposite views; (2) everyday objects often&#10;exhibit global reflective symmetries which can be accurately predicted from&#10;single views; (3) opposite orthographic views of a 3D shape share consistent&#10;silhouettes. Following these observations, we first predict orthographic 2.5D&#10;visible surface maps (depth, normal and silhouette) from perspective 2D images,&#10;and detect global reflective symmetries in this data; second, we predict the&#10;back facing depth and normal maps using as input the front maps and, when&#10;available, the symmetric reflections of these maps; and finally, we reconstruct&#10;a 3D mesh from the union of these maps using a surface reconstruction method&#10;best suited for this data. Our experiments demonstrate that our framework&#10;outperforms state-of-the art approaches for 3D shape reconstructions from 2D&#10;and 2.5D data in terms of input fidelity and details preservation.&#10;Specifically, we achieve 12% better performance on average in ShapeNet&#10;benchmark dataset, and up to 19% for certain classes of objects (e.g., chairs&#10;and vessels).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11876" label="2007.11876">
-        <attvalues>
-          <attvalue for="0" value="Real-time CNN-based Segmentation Architecture for Ball Detection in a&#10;  Single View Setup" />
-          <attvalue for="1" value="  This paper considers the task of detecting the ball from a single viewpoint&#10;in the challenging but common case where the ball interacts frequently with&#10;players while being poorly contrasted with respect to the background. We&#10;propose a novel approach by formulating the problem as a segmentation task&#10;solved by an efficient CNN architecture. To take advantage of the ball&#10;dynamics, the network is fed with a pair of consecutive images. Our inference&#10;model can run in real time without the delay induced by a temporal analysis. We&#10;also show that test-time data augmentation allows for a significant increase&#10;the detection accuracy. As an additional contribution, we publicly release the&#10;dataset on which this work is based.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Locating the ball in team sports is an important complement to player detection~\cite{Parisot2017} and tracking~\cite{Kumar2013a}, both to feed sport analytics~\cite{Thomas2017a} and to enrich broadcasted content~\cite{Fernandez2010}.&#10;In the context of real-time automated production of team sports events~\cite{Chen2011,Chen2010d,Chen2010c,Chen2016a,Keemotion}, knowing the ball position with accuracy and without delay is even more critical.&#10;&#10;In the past ten years, the task of detecting the ball has been largely investigated, leading to industrial products like the automated line calling system in tennis or the goal-line technology in soccer~\cite{HawkEye}. However, the ball detection problem remains unsolved for cases of important practical interest because of two main issues.&#10;&#10;First, the task becomes quite challenging when the ball is partially occluded due to frequent interactions with players, as often encountered in team sports. Earlier works have generally addressed this problem by considering a calibrated multi-view acquisition setup to increase the candidates detection reliability by checking their consistency across the views~\cite{Parisot2011, Ren2008, intelTrueView, HawkEye, Lampert2012}. To better deal with instants at which the ball is held by players, some works have even proposed to enrich multiview ball detection with cues derived from players tracking~\cite{Wang2014a, Wang2014, Maksai2016}. However, those solutions require the installation of multiple cameras around the field, which is significantly more expensive than single viewpoint acquisition systems~\cite{intelTrueView} and appears difficult to deploy in many venues.&#10;&#10;Second, the weakly contrasted appearance of the ball, as generally encountered in indoor environments, leads to many false detections. This problem is often handled by exploiting a ballistic trajectory prior in order to discriminate between true and false detected candidates~\cite{Chen2008, Chen2012, Chakraborty2013, Kumar2011, Zhou2013, Yan2006, Parisot2011}.&#10;In addition to this prior, some works also use the cues derived from the players tracking~\cite{Zhang2008, Maksai2016}, or even use the identity and team affiliation of every player in every frame~\cite{Wei2016}.&#10;However, this prior induces a significant delay, making those solutions inappropriate for real-time applications. Furthermore, they require detecting the ball at high frame rates, inducing hardware and computational constraints and making them unusable for single instants applications.&#10;&#10;Overall, for scenarios considering instant images in indoor scenes captured from a single view point, the ball detection problem remains largely unsolved. This is especially true for basketball (involving fast dynamics and many players interactions) where state-of-the-art solutions saturates around $40\%$ detection rate, restricted in the basket area~\cite{Parisot2019b}.&#10;Those limitations force current real-time game analysis solutions to rely on a connected ball~\cite{youtube}, which requires to insert a transmitter within the ball ; or use multiple cameras and multiple servers to process the different data streams~\cite{HawkEye, Pingali2000, intelTrueView}.&#10;&#10;In this paper, we propose a learning-based method reaching close to $70\%$ ball detection rate on unseen venues, with very few false positives, demonstrated on basketball images. In order to train and validate our method, we created a new dataset of single view basketball scenes featuring many interactions between the ball and players, and low-contrasted backgrounds.&#10;&#10;Similar to most modern image analysis solutions, our method builds on a Convolutional Neural Network (CNN)~\cite{Lecun2015a}.&#10;In earlier works, several CNNs have been designed to address the object detection problem, and among the most accurate, Mask~R-CNN~\cite{He2017} has been trained to detect a large variety of objects, including balls.&#10;The experiments presented in Section~\ref{subsec:compare} reveal that applying the universal Mask~R-CNN detector to our problem largely fails. However, fine tuning the pre-trained weights to deal with our dataset significantly improves the detection performance (despite staying $\sim 5\%$ below our method). However, Mask~R-CNN is too complex to run in real-time on an affordable architecture.&#10;&#10;To provide a computationally simple alternative to Mask~R-CNN, a few works have designed a CNN model to specifically detect the ball in a sports context.&#10;\cite{Reno2018a} adopts a classification strategy by splitting the image in a grid of overlapping patches, each of which is fed to a CNN that assesses whether or not it contains a ball. However, the model is far from real-time, and only two different games are considered to train and validate the method.&#10;In the Robocup Soccer context, \cite{Speck2017a} formulates the detection problem as a regression task aiming at predicting the coordinates of the ball in the image. The performance remains relatively poor despite the reasonable contrast&#10;between the white ball and the green field. Furthermore, regression of object coordinates is known to be poorly addressed by CNNs~\cite{Liu2018}. Hence, this strategy is expected to poorly generalize to real team sport scenes.&#10;&#10;In comparison to those initial attempts to exploit CNNs to detect the ball, the contributions of our work are multiple and multifaceted.&#10;&#10;Primarily, we propose to formulate the ball detection problem as a segmentation problem, for which CNNs are known to be quite effective~\cite{Chen2017a}. Such formulation is especially relevant in our team sport ball detection context since there is only one object-of-interest that we aim to detect. Hence, the CNN does not need to handle the object instantiation problem.&#10;In addition, we use a pair of consecutive images coming from a fixed (or motion compensated) viewpoint to take advantage of the ball dynamics without the delay caused by a temporal regularization, allowing low-latency applications.&#10;We show that this approach allows fast and reliable ball detection in weakly contrasted or cluttered scenes.&#10;&#10;Furthermore, we show that the use of test-time data augmentation permits a significant increase in the detection accuracy at small false positive rates. This is in the continuation of recent works showing that test-time data augmentation can be used to estimate the prediction uncertainty of a CNN model~\cite{Ayhan2018, Wang2019}.&#10;&#10;Finally, we make the royalty-free part of our dataset publicly available.&#10;Beyond providing pairs of consecutive images, it offers a representative sample of professional basketball images gathered on multiple different arenas. It features a large variety of game actions and lighting conditions, cluttered scenes and complex backgrounds. This solves a weakness of several previous works suffering from a limited set of validation data, having very few different games considered~\cite{Parisot2019b, Reno2018a, Speck2017a}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1807.07987" label="1807.07987">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning" />
-          <attvalue for="1" value="  Deep learning (DL) is a high dimensional data reduction technique for&#10;constructing high-dimensional predictors in input-output models. DL is a form&#10;of machine learning that uses hierarchical layers of latent features. In this&#10;article, we review the state-of-the-art of deep learning from a modeling and&#10;algorithmic perspective. We provide a list of successful areas of applications&#10;in Artificial Intelligence (AI), Image Processing, Robotics and Automation.&#10;Deep learning is predictive in its nature rather then inferential and can be&#10;viewed as a black-box methodology for high-dimensional function estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.03247" label="1807.03247">
-        <attvalues>
-          <attvalue for="0" value="An Intriguing Failing of Convolutional Neural Networks and the CoordConv&#10;  Solution" />
-          <attvalue for="1" value="  Few ideas have enjoyed as large an impact on deep learning as convolution.&#10;For any problem involving pixels or spatial representations, common intuition&#10;holds that convolutional neural networks may be appropriate. In this paper we&#10;show a striking counterexample to this intuition via the seemingly trivial&#10;coordinate transform problem, which simply requires learning a mapping between&#10;coordinates in (x,y) Cartesian space and one-hot pixel space. Although&#10;convolutional networks would seem appropriate for this task, we show that they&#10;fail spectacularly. We demonstrate and carefully analyze the failure first on a&#10;toy problem, at which point a simple fix becomes obvious. We call this solution&#10;CoordConv, which works by giving convolution access to its own input&#10;coordinates through the use of extra coordinate channels. Without sacrificing&#10;the computational and parametric efficiency of ordinary convolution, CoordConv&#10;allows networks to learn either complete translation invariance or varying&#10;degrees of translation dependence, as required by the end task. CoordConv&#10;solves the coordinate transform problem with perfect generalization and 150&#10;times faster with 10--100 times fewer parameters than convolution. This stark&#10;contrast raises the question: to what extent has this inability of convolution&#10;persisted insidiously inside other tasks, subtly hampering performance from&#10;within? A complete answer to this question will require further investigation,&#10;but we show preliminary evidence that swapping convolution for CoordConv can&#10;improve models on a diverse set of tasks. Using CoordConv in a GAN produced&#10;less mode collapse as the transform between high-level spatial latents and&#10;pixels becomes easier to learn. A Faster R-CNN detection model trained on MNIST&#10;showed 24% better IOU when using CoordConv, and in the RL domain agents playing&#10;Atari games benefit significantly from the use of CoordConv layers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.05587" label="1706.05587">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Atrous Convolution for Semantic Image Segmentation" />
-          <attvalue for="1" value="  In this work, we revisit atrous convolution, a powerful tool to explicitly&#10;adjust filter's field-of-view as well as control the resolution of feature&#10;responses computed by Deep Convolutional Neural Networks, in the application of&#10;semantic image segmentation. To handle the problem of segmenting objects at&#10;multiple scales, we design modules which employ atrous convolution in cascade&#10;or in parallel to capture multi-scale context by adopting multiple atrous&#10;rates. Furthermore, we propose to augment our previously proposed Atrous&#10;Spatial Pyramid Pooling module, which probes convolutional features at multiple&#10;scales, with image-level features encoding global context and further boost&#10;performance. We also elaborate on implementation details and share our&#10;experience on training our system. The proposed `DeepLabv3' system&#10;significantly improves over our previous DeepLab versions without DenseCRF&#10;post-processing and attains comparable performance with other state-of-art&#10;models on the PASCAL VOC 2012 semantic image segmentation benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.07825" label="2006.07825">
-        <attvalues>
-          <attvalue for="0" value="Working with scale: 2nd place solution to Product Detection in Densely&#10;  Packed Scenes [Technical Report]" />
-          <attvalue for="1" value="  This report describes a 2nd place solution of the detection challenge which&#10;is held within CVPR 2020 Retail-Vision workshop. Instead of going further&#10;considering previous results this work mainly aims to verify previously&#10;observed takeaways by re-experimenting. The reliability and reproducibility of&#10;the results are reached by incorporating a popular object detection toolbox -&#10;MMDetection. In this report, I firstly represent the results received for&#10;Faster-RCNN and RetinaNet models, which were taken for comparison in the&#10;original work. Then I describe the experiment results with more advanced&#10;models. The final section reviews two simple tricks for Faster-RCNN model that&#10;were used for my final submission: changing default anchor scale parameter and&#10;train-time image tiling. The source code is available at&#10;https://github.com/tyomj/product_detection.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The majority of works in object detection mainly describe results on two datasets: Pascal VOC \cite{everingham2011pascal} and MS-COCO \cite{lin2014microsoft}. While this information might be helpful in forming the relative intuition of a model behavior for common scenes, it is still unobvious when it comes to a dataset whose distribution of objects deviates. Goldman et al. \cite{goldman2019precise} proposed a dataset with the following key features: 1) a high number of objects on an image, 2) objects are densely packed.&#10;&#10;Although in the comparison table the authors declared the dataset to have 110,712 classes, it is important to note that in this context they used the term class to describe the appearance variations among the objects. De facto the dataset represents a single-class object detection task. Considering this information, we might reference an example of a similar task from a different domain \cite{shao2018crowdhuman} where both RetinaNet \cite{lin2017focal} and Faster-RCNN \cite{ren2015faster} were taking as baseline solutions and showed good results. Hence, the first step of my plan was to revisit some of the baseline architectures.&#10;&#10;A rapidly growing number of papers with different new methods in object detection makes it difficult to maintain the reliable and up-to-date benchmarking table especially when a custom implementation of the new method also comes with different parts of the training pipeline such as data pre-processing or results post-processing. In order to address this issue, many attempts have been made for the last several years to create a highly reusable codebase that covers most of the popular models in the field. For now there are plenty of such toolboxes based on different deep learning frameworks: Tensorflow Object Detection API \cite{huang2016speedaccuracy}, Detectron \cite{Detectron2018} and Detectron2 \cite{wu2019detectron2}, MMDetection \cite{chen2019mmdetection}, and so on. Each of these toolboxes has reusable parts of code, which makes it easier to compare the performance of models in vitro. For my experiments I chose MMDetection v1.&#10;&#10;The rest of the report is structured as follows: in Section \ref{sec:dataset} I briefly review the parameters of the dataset and additionally highlight the features which I found important. Section \ref{sec:exps} &#10;contains all the experiment results where I consequentially inspect the results starting from baseline models \ref{sec:basleine} then testing a few approaches that are different from widely-used anchoring scheme \ref{sec:adv}. In the final section\ref{sec:scale} I describe two simple tricks that helped to achieve considerably good performance for Faster-RCNN, testing some commonly-used tricks along the way.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1904.00853" label="1904.00853">
-        <attvalues>
-          <attvalue for="0" value="Precise Detection in Densely Packed Scenes" />
-          <attvalue for="1" value="  Man-made scenes can be densely packed, containing numerous objects, often&#10;identical, positioned in close proximity. We show that precise object detection&#10;in such scenes remains a challenging frontier even for state-of-the-art object&#10;detectors. We propose a novel, deep-learning based method for precise object&#10;detection, designed for such challenging settings. Our contributions include:&#10;(1) A layer for estimating the Jaccard index as a detection quality score; (2)&#10;a novel EM merging unit, which uses our quality scores to resolve detection&#10;overlap ambiguities; finally, (3) an extensive, annotated data set, SKU-110K,&#10;representing packed retail environments, released for training and testing&#10;under such extreme settings. Detection tests on SKU-110K and counting tests on&#10;the CARPK and PUCPR+ show our method to outperform existing state-of-the-art&#10;with substantial margins. The code and data will be made available on&#10;\url{www.github.com/eg4000/SKU110K_CVPR19}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.00123" label="1805.00123">
-        <attvalues>
-          <attvalue for="0" value="CrowdHuman: A Benchmark for Detecting Human in a Crowd" />
-          <attvalue for="1" value="  Human detection has witnessed impressive progress in recent years. However,&#10;the occlusion issue of detecting human in highly crowded environments is far&#10;from solved. To make matters worse, crowd scenarios are still under-represented&#10;in current human detection benchmarks. In this paper, we introduce a new&#10;dataset, called CrowdHuman, to better evaluate detectors in crowd scenarios.&#10;The CrowdHuman dataset is large, rich-annotated and contains high diversity.&#10;There are a total of $470K$ human instances from the train and validation&#10;subsets, and $~22.6$ persons per image, with various kinds of occlusions in the&#10;dataset. Each human instance is annotated with a head bounding-box, human&#10;visible-region bounding-box and human full-body bounding-box. Baseline&#10;performance of state-of-the-art detection frameworks on CrowdHuman is&#10;presented. The cross-dataset generalization results of CrowdHuman dataset&#10;demonstrate state-of-the-art performance on previous dataset including&#10;Caltech-USA, CityPersons, and Brainwash without bells and whistles. We hope our&#10;dataset will serve as a solid baseline and help promote future research in&#10;human detection tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.10012" label="1611.10012">
-        <attvalues>
-          <attvalue for="0" value="Speed/accuracy trade-offs for modern convolutional object detectors" />
-          <attvalue for="1" value="  The goal of this paper is to serve as a guide for selecting a detection&#10;architecture that achieves the right speed/memory/accuracy balance for a given&#10;application and platform. To this end, we investigate various ways to trade&#10;accuracy for speed and memory usage in modern convolutional object detection&#10;systems. A number of successful systems have been proposed in recent years, but&#10;apples-to-apples comparisons are difficult due to different base feature&#10;extractors (e.g., VGG, Residual Networks), different default image resolutions,&#10;as well as different hardware and software platforms. We present a unified&#10;implementation of the Faster R-CNN [Ren et al., 2015], R-FCN [Dai et al., 2016]&#10;and SSD [Liu et al., 2015] systems, which we view as &quot;meta-architectures&quot; and&#10;trace out the speed/accuracy trade-off curve created by using alternative&#10;feature extractors and varying other critical parameters such as image size&#10;within each of these meta-architectures. On one extreme end of this spectrum&#10;where speed and memory are critical, we present a detector that achieves real&#10;time speeds and can be deployed on a mobile device. On the opposite end in&#10;which accuracy is critical, we present a detector that achieves&#10;state-of-the-art performance measured on the COCO detection task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.07155" label="1906.07155">
-        <attvalues>
-          <attvalue for="0" value="MMDetection: Open MMLab Detection Toolbox and Benchmark" />
-          <attvalue for="1" value="  We present MMDetection, an object detection toolbox that contains a rich set&#10;of object detection and instance segmentation methods as well as related&#10;components and modules. The toolbox started from a codebase of MMDet team who&#10;won the detection track of COCO Challenge 2018. It gradually evolves into a&#10;unified platform that covers many popular detection methods and contemporary&#10;modules. It not only includes training and inference codes, but also provides&#10;weights for more than 200 network models. We believe this toolbox is by far the&#10;most complete detection toolbox. In this paper, we introduce the various&#10;features of this toolbox. In addition, we also conduct a benchmarking study on&#10;different methods, components, and their hyper-parameters. We wish that the&#10;toolbox and benchmark could serve the growing research community by providing a&#10;flexible toolkit to reimplement existing methods and develop their own new&#10;detectors. Code and models are available at&#10;https://github.com/open-mmlab/mmdetection. The project is under active&#10;development and we will keep this document updated.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.09598" label="2302.09598">
-        <attvalues>
-          <attvalue for="0" value="Guided Depth Map Super-resolution: A Survey" />
-          <attvalue for="1" value="  Guided depth map super-resolution (GDSR), which aims to reconstruct a&#10;high-resolution (HR) depth map from a low-resolution (LR) observation with the&#10;help of a paired HR color image, is a longstanding and fundamental problem, it&#10;has attracted considerable attention from computer vision and image processing&#10;communities. A myriad of novel and effective approaches have been proposed&#10;recently, especially with powerful deep learning techniques. This survey is an&#10;effort to present a comprehensive survey of recent progress in GDSR. We start&#10;by summarizing the problem of GDSR and explaining why it is challenging. Next,&#10;we introduce some commonly used datasets and image quality assessment methods.&#10;In addition, we roughly classify existing GDSR methods into three categories,&#10;i.e., filtering-based methods, prior-based methods, and learning-based methods.&#10;In each category, we introduce the general description of the published&#10;algorithms and design principles, summarize the representative methods, and&#10;discuss their highlights and limitations. Moreover, the depth related&#10;applications are introduced. Furthermore, we conduct experiments to evaluate&#10;the performance of some representative methods based on unified experimental&#10;configurations, so as to offer a systematic and fair performance evaluation to&#10;readers. Finally, we conclude this survey with possible directions and open&#10;problems for further research. All the related materials can be found at&#10;\url{https://github.com/zhwzhong/Guided-Depth-Map-Super-resolution-A-Survey}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Depth information plays a fundamental role in a wide range of computer vision applications, such as 3D reconstruction~\cite{han2019image, peng2021shape}, semantic scene understanding~\cite{wang2021salient, xu2022weakly, wang2022learning} and autonomous driving~\cite{du2021ago, wang2021progressive}. Currently, there are two main range sensing technologies to capture depth information, either passive or active. For passive sensing technologies, the most famous and commonly used approach is stereo reconstruction, which first obtains two images of the scene meanwhile through two conventional monochrome or color cameras separated by a certain distance and then estimates the scene depth by using stereo matching algorithms. Over the past decades, the performance of stereo matching algorithms has improved markedly~\cite{wang2021pvstereo, shankar2022learned}. However, as these methods are based on finding corresponds between the two views, the effect of depth missing and occlusion will occur for texture-less regions. In contrast to passive ones, active methods can capture depth maps more efficiently and robustly, especially for texture-less regions. Time-of-flight (TOF) camera~\cite{lange2001solid} and structure light scanner~\cite{herrera2012joint} are two representative depth sensors for real-time active depth sensing. Despite the rapid development of active depth sensing, there is still a large gap between depth camera and color camera, especially in terms of spatial resolution. For instance, the spatial resolution of consumer-level ToF cameras is only about 200 kilo pixels, which is much lower than off-the-shelf conventional color cameras (e.g., 40 Megapixels for Huawei Mate 30). &#10;&#10;The low-resolution depth maps acquired by these depth sensors would greatly affect the performance of algorithms in downstream applications. Therefore, it is highly desirable to develop effective and efficient depth super-resolution algorithms to facilitate the usage of depth information. In general, depth cameras are equipped with an additional RGB sensor which can capture color images with a resolution higher than that of depth maps. It is natural to improve the quality of the depth map by employing the high-resolution color counterpart as guidance, which is known as guided depth map super-resolution (GDSR). Compared to the general image super-resolution, the guided depth image super-resolution has two unique characteristics : 1) the depth map contains smooth areas separated by sharp boundaries, which is known as piece-wise smooth, 2) an additional high-resolution color image can be used to guide the depth map reconstruction process. Generally, the depth map and its corresponding color image are geometric and photometric descriptions of the same scene, and they have strong structural similarity. Thus, most existing GDSR algorithms are elaborately designed to exploit this piece-wise smoothness and take full advantage of the guidance image. Recently, GDSR attracts more and more attention in both academic and industrial circles, which encourages us to offer this comprehensive survey of recent advances.&#10;&#10;Fundamentally, GDSR is an ill-posed problem since there can be various HR depth maps with a slight difference in camera angle, illumination, material properties, and other variables for a certain LR depth map. During the past decades, a wide variety of methods have been developed to solve this inverse problem. To clearly describe the progress of the GDSR methods, we provide a brief chronology in Fig.~\ref{fig:fig2}.&#10;Here, we classify these methods into the following three categories: filtering-based methods, optimization-based methods, and learning-based methods. &#10;&#10;Filtering-based methods estimate the depth of pixels by performing a weighted average of local pixels, and the weights are obtained by the affinity calculated from RGB-D image pairs. The bilateral filters~\cite{BF, yang2007spatial, jbu, qiao2021fast}, non-local mean filters~\cite{huhle2010fusion, buades2005non}, and guided filters~\cite{GF, DGF, WGFS, WGF} are representative methods. Filtering-based methods enjoy simplicity in design and low computational complexity, but are prone to generate annoying artifacts when the depth discontinuities are not consistent with those in their corresponding color image. Moreover, the filter kernel is typically designed for a specific task, which lacks flexibility. Contrary to local filtering-based methods, optimization-based methods formulate the problem of GDSR as a global optimization framework. The objective function usually contains two terms: data fidelity term and regularization term. The data fidelity term is used to preserve depth consistency between the super-resolved result and the input depth map. The regularization term encourages the edges of the super-resolved result to consist with the guidance image as much as possible. &#10;The shortcomings of local filtering-based methods can be resolved to some extent by optimization-based methods. Nevertheless, since the optimization-based methods depend on hand-crafted object functions, they may not reflect the real complex image priors. Furthermore, the iterative optimization process involved in these algorithms is usually time-consuming, which impedes their applications in real-world scenarios. To obtain a better and more efficient GDSR model, learning-based methods are proposed and applied to large-scale datasets to solve the depth map super-resolution task. The learning-based methods can be further divided into two categories: sparse dictionary learning methods and deep learning methods. Dictionary learning, also known as sparse coding, aims at deriving a set of dictionary atoms in which only a small number of atoms can be linearly combined to approximate a given image. Due to its simplicity and flexibility for data representation, numerous methods have been introduced in the past decade and achieved impressive results~\cite{9681224, deng2019deep, CUNet}. Recent advances of deep learning algorithms have revolutionized the area of computer vision; tremendous progress has been achieved in a variety of domains, such as image classification~\cite{he2016deep, dosovitskiy2021an}, object localization~\cite{zhang2021weakly, wu2022edn} and semantic segmentation~\cite{strudel2021segmenter, zhao2018psanet}. GDSR is no exception: a myriad of neural networks have been developed and advanced the state of the art~\cite{AHMF, zhao2021discrete, GraphSR}. The learning-based method is suitable for a scene where there is a large amount of training data. When training data are insufficient, the traditional optimization-based or filter-based method may be a better choice.&#10;&#10;Although significant progress has been made, there is no unified framework to understand and classify existing methods, in particular the recently popular deep learning based approaches. To the best of our knowledge, there are only two surveys~\cite{eichhardt2017image, yang2019depth} available in the literature for the area of GDSR. The paper of \cite{eichhardt2017image} is published in 2017 and its main purpose is to summarize the methods that are tailored for ToF depth maps. The paper of~\cite{yang2019depth} is published in 2019, which only introduces some representative methods that are published before 2019. Many advanced methods that achieve state-of-the-art performance have been proposed since then. Furthermore, neither of these two surveys introduces the benchmark datasets, evaluation metrics, or experimental comparisons of different methods. A comprehensive survey is desired that summarizes and compares these approaches with a unified perspective. In addition, inconsistent experimental protocols are utilized in different approaches. It lacks a benchmark for fairly evaluating state-of-the-art approaches. &#10;In view of these issues, we provide a comprehensive review of existing GDSR methods. &#10;&#10;We summarize the key contributions of this survey as follows:&#10;\begin{itemize}&#10; \item We present a systematic investigation of recent GDSR techniques, including problem formulation, benchmark dataset, widely used evaluation metrics, and representative methods. To introduce these methods clearly, we classify them into three categories: filtering-based, optimization-based and learning-based. For each category, we review representative methods and discuss their contributions, benefits, and weaknesses.&#10; \item We provide comprehensive and fair experimental comparison to evaluate state-of-the-art methods quantitatively and qualitatively on benchmark datasets based on the same experimental configurations. Moreover, we present some tricks that can be used to further improve the performance of depth map reconstruction.&#10; \item We discuss the open problems, challenges, and envision prospects for future research.&#10;\end{itemize}&#10;&#10;The remainder of this paper is organized as follows. In Sect.~\ref{fund}, we introduce some background concepts and basic knowledge, including the formulation of the GDSR problem, widely used RGB-D datasets, and evaluation metrics. Then detailed reviews of filtering-based (Sect.~\ref{filter}), optimization-based (Sect.~\ref{prior}) and learning-based (Sect.~\ref{learn}) methods are given. In Sect.~\ref{application}, we introduce the depth related applications. In Sect.~\ref{comparison}, we systematically compare several representative methods using publicly available datasets with unified experimental configurations. Finally, we conclude the paper and present some challenging issues as well as promising future research directions in Sect.~\ref{conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Computer Vision Applications, Mathematics, Image Processing, Depth Map Super-resolution, Computer Science, Image Processing Techniques" />
-        </attvalues>
-      </node>
-      <node id="1906.06543" label="1906.06543">
-        <attvalues>
-          <attvalue for="0" value="Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the&#10;  Deep Learning Era" />
-          <attvalue for="1" value="  3D reconstruction is a longstanding ill-posed problem, which has been&#10;explored for decades by the computer vision, computer graphics, and machine&#10;learning communities. Since 2015, image-based 3D reconstruction using&#10;convolutional neural networks (CNN) has attracted increasing interest and&#10;demonstrated an impressive performance. Given this new era of rapid evolution,&#10;this article provides a comprehensive survey of the recent developments in this&#10;field. We focus on the works which use deep learning techniques to estimate the&#10;3D shape of generic objects either from a single or multiple RGB images. We&#10;organize the literature based on the shape representations, the network&#10;architectures, and the training mechanisms they use. While this survey is&#10;intended for methods which reconstruct generic objects, we also review some of&#10;the recent works which focus on specific object classes such as human body&#10;shapes and faces. We provide an analysis and comparison of the performance of&#10;some key papers, summarize some of the open problems in this field, and discuss&#10;promising directions for future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.09146" label="1904.09146">
-        <attvalues>
-          <attvalue for="0" value="Salient Object Detection in the Deep Learning Era: An In-Depth Survey" />
-          <attvalue for="1" value="  As an essential problem in computer vision, salient object detection (SOD)&#10;has attracted an increasing amount of research attention over the years. Recent&#10;advances in SOD are predominantly led by deep learning-based solutions (named&#10;deep SOD). To enable in-depth understanding of deep SOD, in this paper, we&#10;provide a comprehensive survey covering various aspects, ranging from algorithm&#10;taxonomy to unsolved issues. In particular, we first review deep SOD algorithms&#10;from different perspectives, including network architecture, level of&#10;supervision, learning paradigm, and object-/instance-level detection. Following&#10;that, we summarize and analyze existing SOD datasets and evaluation metrics.&#10;Then, we benchmark a large group of representative SOD models, and provide&#10;detailed analyses of the comparison results. Moreover, we study the performance&#10;of SOD algorithms under different attribute settings, which has not been&#10;thoroughly explored previously, by constructing a novel SOD dataset with rich&#10;attribute annotations covering various salient object types, challenging&#10;factors, and scene categories. We further analyze, for the first time in the&#10;field, the robustness of SOD models to random input perturbations and&#10;adversarial attacks. We also look into the generalization and difficulty of&#10;existing SOD datasets. Finally, we discuss several open issues of SOD and&#10;outline future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.11658" label="2208.11658">
-        <attvalues>
-          <attvalue for="0" value="AGO-Net: Association-Guided 3D Point Cloud Object Detection Network" />
-          <attvalue for="1" value="  The human brain can effortlessly recognize and localize objects, whereas&#10;current 3D object detection methods based on LiDAR point clouds still report&#10;inferior performance for detecting occluded and distant objects: the point&#10;cloud appearance varies greatly due to occlusion, and has inherent variance in&#10;point densities along the distance to sensors. Therefore, designing feature&#10;representations robust to such point clouds is critical. Inspired by human&#10;associative recognition, we propose a novel 3D detection framework that&#10;associates intact features for objects via domain adaptation. We bridge the gap&#10;between the perceptual domain, where features are derived from real scenes with&#10;sub-optimal representations, and the conceptual domain, where features are&#10;extracted from augmented scenes that consist of non-occlusion objects with rich&#10;detailed information. A feasible method is investigated to construct conceptual&#10;scenes without external datasets. We further introduce an attention-based&#10;re-weighting module that adaptively strengthens the feature adaptation of more&#10;informative regions. The network's feature enhancement ability is exploited&#10;without introducing extra cost during inference, which is plug-and-play in&#10;various 3D detection frameworks. We achieve new state-of-the-art performance on&#10;the KITTI 3D detection benchmark in both accuracy and speed. Experiments on&#10;nuScenes and Waymo datasets also validate the versatility of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.05633" label="2105.05633">
-        <attvalues>
-          <attvalue for="0" value="Segmenter: Transformer for Semantic Segmentation" />
-          <attvalue for="1" value="  Image segmentation is often ambiguous at the level of individual image&#10;patches and requires contextual information to reach label consensus. In this&#10;paper we introduce Segmenter, a transformer model for semantic segmentation. In&#10;contrast to convolution-based methods, our approach allows to model global&#10;context already at the first layer and throughout the network. We build on the&#10;recent Vision Transformer (ViT) and extend it to semantic segmentation. To do&#10;so, we rely on the output embeddings corresponding to image patches and obtain&#10;class labels from these embeddings with a point-wise linear decoder or a mask&#10;transformer decoder. We leverage models pre-trained for image classification&#10;and show that we can fine-tune them on moderate sized datasets available for&#10;semantic segmentation. The linear decoder allows to obtain excellent results&#10;already, but the performance can be further improved by a mask transformer&#10;generating class masks. We conduct an extensive ablation study to show the&#10;impact of the different parameters, in particular the performance is better for&#10;large models and small patch sizes. Segmenter attains excellent results for&#10;semantic segmentation. It outperforms the state of the art on both ADE20K and&#10;Pascal Context datasets and is competitive on Cityscapes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.08639" label="2211.08639">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Dynamic Image Harmonization" />
-          <attvalue for="1" value="  Image harmonization is a critical task in computer vision, which aims to&#10;adjust the foreground to make it compatible with the background. Recent works&#10;mainly focus on using global transformations (i.e., normalization and color&#10;curve rendering) to achieve visual consistency. However, these models ignore&#10;local visual consistency and their huge model sizes limit their harmonization&#10;ability on edge devices. In this paper, we propose a hierarchical dynamic&#10;network (HDNet) to adapt features from local to global view for better feature&#10;transformation in efficient image harmonization. Inspired by the success of&#10;various dynamic models, local dynamic (LD) module and mask-aware global dynamic&#10;(MGD) module are proposed in this paper. Specifically, LD matches local&#10;representations between the foreground and background regions based on semantic&#10;similarities, then adaptively adjust every foreground local representation&#10;according to the appearance of its $K$-nearest neighbor background regions. In&#10;this way, LD can produce more realistic images at a more fine-grained level,&#10;and simultaneously enjoy the characteristic of semantic alignment. The MGD&#10;effectively applies distinct convolution to the foreground and background,&#10;learning the representations of foreground and background regions as well as&#10;their correlations to the global harmonization, facilitating local visual&#10;consistency for the images much more efficiently. Experimental results&#10;demonstrate that the proposed HDNet significantly reduces the total model&#10;parameters by more than 80\% compared to previous methods, while still&#10;attaining state-of-the-art performance on the popular iHarmony4 dataset.&#10;Notably, the HDNet achieves a 4\% improvement in PSNR and a 19\% reduction in&#10;MSE compared to the prior state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Harmonization" />
-        </attvalues>
-      </node>
-      <node id="2006.07807" label="2006.07807">
-        <attvalues>
-          <attvalue for="0" value="Relative Pose Estimation for Stereo Rolling Shutter Cameras" />
-          <attvalue for="1" value="  In this paper, we present a novel linear algorithm to estimate the 6 DoF&#10;relative pose from consecutive frames of stereo rolling shutter (RS) cameras.&#10;Our method is derived based on the assumption that stereo cameras undergo&#10;motion with constant velocity around the center of the baseline, which needs 9&#10;pairs of correspondences on both left and right consecutive frames. The stereo&#10;RS images enable the recovery of depth maps from the semi-global matching (SGM)&#10;algorithm. With the estimated camera motion and depth map, we can correct the&#10;RS images to get the undistorted images without any scene structure assumption.&#10;Experiments on both simulated points and synthetic RS images demonstrate the&#10;effectiveness of our algorithm in relative pose estimation.&#10;" />
-          <attvalue for="2" value="&#10;With the advantage of low-cost and simplicity in design, more and more rolling-shutter (RS) CMOS cameras \cite{Meingast_Geometric_2005} have been used in various real-world computer vision applications. Different from their global shutter (GS) counterparts that expose all pixels simultaneously, RS cameras capture the pixels of each row (or column) at consecutive times. Therefore, when there is instantaneous motion between the scene and the camera, the captured image will show rolling shutter effects such as skew or wobble, and the level of rolling shutter effect varies with respect to the camera motion and the object motion.&#10;&#10;Currently, most of the geometric algorithms in computer vision are designed under the perspective camera model (i.e. GS camera model). However, this model fails to handle the RS images due to the significant difference between RS and GS cameras. Therefore, geometric algorithms for RS cameras have attracted more and more attention. In \cite{Meingast_Geometric_2005}, Meingast et al introduced the geometric model for RS cameras. Along this direction, many algorithms have been proposed to solve the corresponding multi-view geometric problems. Purkait et al. \cite{Purkait_manhattan_2017} performed pose estimation under the Manhattan world assumption.&#10;Oliver et al. \cite{Saurer_RS_stereo_2013} proposed an algorithm based on plane sweeping to recover the depth map and 3D scene for multi-view RS images. Fraundorfer et al. \cite{Fraundorfer_Minimal_2010} fused information from external sensors like IMU to estimate the relative motion while Lee et al \cite{Lee_Gyroscope_2019} simplified the pose estimation problem by using the gyroscope.&#10;Dai et al. \cite{Dai_epipolar_2016} derived a 20-point linear algorithm and a 44-point linear algorithm under different RS models based on generalized epipolar geometry.&#10;&#10;It should be noted that all of these works are designed to solve the relative pose of a monocular RS camera. In practise, stereo cameras provide extra benefits over its monocular counterpart, i.e., more accurate motion estimation, depth estimation and the availability of global scale. Existing relative pose estimation methods for stereo cameras such as \cite{Vakhitov_2018_line_point} are all designed for GS cameras.&#10;Surprisingly, no previous attempt has been reported on solving the {6 DoF relative pose} problem with stereo RS cameras. &#10;Motivated by relative pose estimation for monocular camera \cite{Dai_epipolar_2016}, we exploit the epipolar geometry to solve the relative pose from consecutive RS frames and present the first relative pose estimation algorithm for stereo RS cameras as shown in Fig.1. &#10;Specifically, we figure out a linear solver for 6 DoF relative pose from consecutive stereo images under a continuous motion model. To solve the pose, 9 pairs of correspondences from the left and right cameras respectively are needed. The pose of each scanline on each frame can be solved under the assumption of a constant velocity motion model.&#10;&#10;The estimated 6 DoF camera motion also enables the correction of the RS effects, which generally requires the underlying scene structures and the camera motions between scanlines or between views. Existing work with monocular cameras depend on strong assumptions to estimate the scene structure. &#10;Lao et al. \cite{Lao_2018_Robust} used at least 4 curves to estimate the camera's rotation speed to eliminate the RS effect. &#10;Zhuang et al.\cite{Zhuang_Rolling_shutter_aware_2017} introduced optical flow based on RS successive frames, which proposed an 8-point algorithm for constant motion, and 9-point algorithm for constant acceleration motion, then they achieved SfM as well as image rectification.&#10;Vasu et al. \cite{Vasu_Occlusion_2018} realized that the RS distortion effect is depth-dependent, and proposed an algorithm that can recover 3D scene from a set of RS distorted images. &#10;&#10;Under our stereo RS camera configuration, we are able to estimate the scene structure without assumptions. Specifically, we show that for stereo RS images, using the semi-global matching (SGM) algorithm which is based on stereo GS image can also accurately estimate the depth map, then the depth will be used in the RS image correction to obtain a smoothly corrected image.&#10;Experimental results on both simulated points and synthetic RS images prove the effectiveness of our relative pose algorithm and its application in RS image correction,&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Physics, Image Processing, Camera Motion Estimation, Computer Science, Robotics, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="cs/0503076" label="cs/0503076">
-        <attvalues>
-          <attvalue for="0" value="Geometric Models of Rolling-Shutter Cameras" />
-          <attvalue for="1" value="  Cameras with rolling shutters are becoming more common as low-power, low-cost&#10;CMOS sensors are being used more frequently in cameras. The rolling shutter&#10;means that not all scanlines are exposed over the same time interval. The&#10;effects of a rolling shutter are noticeable when either the camera or objects&#10;in the scene are moving and can lead to systematic biases in projection&#10;estimation. We develop a general projection equation for a rolling shutter&#10;camera and show how it is affected by different types of camera motion. In the&#10;case of fronto-parallel motion, we show how that camera can be modeled as an&#10;X-slit camera. We also develop approximate projection equations for a non-zero&#10;angular velocity about the optical axis and approximate the projection equation&#10;for a constant velocity screw motion. We demonstrate how the rolling shutter&#10;effects the projective geometry of the camera and in turn the&#10;structure-from-motion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06770" label="1904.06770">
-        <attvalues>
-          <attvalue for="0" value="Gyroscope-aided Relative Pose Estimation for Rolling Shutter Cameras" />
-          <attvalue for="1" value="  The rolling shutter camera has received great attention due to its low cost&#10;imaging capability, however, the estimation of relative pose between rolling&#10;shutter cameras still remains a difficult problem owing to its line-by-line&#10;image capturing characteristics. To alleviate this problem, we exploit&#10;gyroscope measurements, angular velocity, along with image measurement to&#10;compute the relative pose between rolling shutter cameras. The gyroscope&#10;measurements provide the information about instantaneous motion that causes the&#10;rolling shutter distortion. Having gyroscope measurements in one hand, we&#10;simplify the relative pose estimation problem and find a minimal solution for&#10;the problem based on the Grobner basis polynomial solver. The proposed method&#10;requires only five points to compute relative pose between rolling shutter&#10;cameras, whereas previous methods require 20 or 44 corresponding points for&#10;linear and uniform rolling shutter geometry models, respectively. Experimental&#10;results on synthetic and real data verify the superiority of the proposed&#10;method over existing relative pose estimation methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.00475" label="1605.00475">
-        <attvalues>
-          <attvalue for="0" value="Rolling Shutter Camera Relative Pose: Generalized Epipolar Geometry" />
-          <attvalue for="1" value="  The vast majority of modern consumer-grade cameras employ a rolling shutter&#10;mechanism. In dynamic geometric computer vision applications such as visual&#10;SLAM, the so-called rolling shutter effect therefore needs to be properly taken&#10;into account. A dedicated relative pose solver appears to be the first problem&#10;to solve, as it is of eminent importance to bootstrap any derivation of&#10;multi-view geometry. However, despite its significance, it has received&#10;inadequate attention to date.&#10;  This paper presents a detailed investigation of the geometry of the rolling&#10;shutter relative pose problem. We introduce the rolling shutter essential&#10;matrix, and establish its link to existing models such as the push-broom&#10;cameras, summarized in a clean hierarchy of multi-perspective cameras. The&#10;generalization of well-established concepts from epipolar geometry is completed&#10;by a definition of the Sampson distance in the rolling shutter case. The work&#10;is concluded with a careful investigation of the introduced epipolar geometry&#10;for rolling shutter cameras on several dedicated benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.00276" label="1907.00276">
-        <attvalues>
-          <attvalue for="0" value="Stereo relative pose from line and point feature triplets" />
-          <attvalue for="1" value="  Stereo relative pose problem lies at the core of stereo visual odometry&#10;systems that are used in many applications. In this work, we present two&#10;minimal solvers for the stereo relative pose. We specifically consider the case&#10;when a minimal set consists of three point or line features and each of them&#10;has three known projections on two stereo cameras. We validate the importance&#10;of this formulation for practical purposes in our experiments with motion&#10;estimation. We then present a complete classification of minimal cases with&#10;three point or line correspondences each having three projections, and present&#10;two new solvers that can handle all such cases. We demonstrate a considerable&#10;effect from the integration of the new solvers into a visual SLAM system.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.03943" label="1903.03943">
-        <attvalues>
-          <attvalue for="0" value="Rolling-Shutter-Aware Differential SfM and Image Rectification" />
-          <attvalue for="1" value="  In this paper, we develop a modified differential Structure from Motion (SfM)&#10;algorithm that can estimate relative pose from two consecutive frames despite&#10;of Rolling Shutter (RS) artifacts. In particular, we show that under constant&#10;velocity assumption, the errors induced by the rolling shutter effect can be&#10;easily rectified by a linear scaling operation on each optical flow. We further&#10;propose a 9-point algorithm to recover the relative pose of a rolling shutter&#10;camera that undergoes constant acceleration motion. We demonstrate that the&#10;dense depth maps recovered from the relative pose of the RS camera can be used&#10;in a RS-aware warping for image rectification to recover high-quality Global&#10;Shutter (GS) images. Experiments on both synthetic and real RS images show that&#10;our RS-aware differential SfM algorithm produces more accurate results on&#10;relative pose estimation and 3D reconstruction from images distorted by RS&#10;effect compared to standard SfM algorithms that assume a GS camera model. We&#10;also demonstrate that our RS-aware warping for image rectification method&#10;outperforms state-of-the-art commercial software products, i.e. Adobe After&#10;Effects and Apple Imovie, at removing RS artifacts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.09178" label="2402.09178">
-        <attvalues>
-          <attvalue for="0" value="Generalized Portrait Quality Assessment" />
-          <attvalue for="1" value="  Automated and robust portrait quality assessment (PQA) is of paramount&#10;importance in high-impact applications such as smartphone photography. This&#10;paper presents FHIQA, a learning-based approach to PQA that introduces a simple&#10;but effective quality score rescaling method based on image semantics, to&#10;enhance the precision of fine-grained image quality metrics while ensuring&#10;robust generalization to various scene settings beyond the training dataset.&#10;The proposed approach is validated by extensive experiments on the PIQ23&#10;benchmark and comparisons with the current state of the art. The source code of&#10;FHIQA will be made publicly available on the PIQ23 GitHub repository at&#10;https://github.com/DXOMARK-Research/PIQ2023.&#10;" />
-          <attvalue for="2" value="&#10;&#10;S{martphones} have significantly altered the landscape of photographic practices, with a notable emphasis on portrait photography. As the consumer base becomes increasingly discerning, there is a clear escalation in expectations regarding the quality of portrait images. This trend pushes a corresponding advancement in camera technology driving manufacturers to focus on improving image quality, which usually involves developing costly image quality assessment (IQA) protocols for optimizing camera performance. Consequently, automated IQA methods are employed to cut the cost of smartphone camera tuning. &#10;&#10;Traditional IQA \cite{larson2010most, ponomarenko2015image, ponomarenko2009tid2008, sheikh2006statistical} offers relevant insights about image quality but often doesn't capture the complexities of modern camera systems and weakly correlates with human perception \cite{fang2020perceptual}. Deep learning-based IQA has emerged as a promising alternative leveraging the widespread availability of modern smartphone images and image quality datasets. Popular datasets like LIVE \cite{sheikh2006statistical}, CSIQ \cite{larson2010most}, and TID \cite{ponomarenko2015image, ponomarenko2009tid2008} present a large amount of synthetically distorted images, but they don't encapsulate the multifaceted nature of modern smartphone camera systems. ``In-the-wild&quot; collections, including LIVE Challenge \cite{ghadiyaram2015massive}, KonIQ-10k \cite{hosu2020koniq}, and PaQ-2-PiQ \cite{ying2020patches}, offer a better representation of real-world conditions, but their uncontrolled data collection and subjective labeling make them less suited for rigorous digital camera evaluations. Recently, datasets such as PIQ23 \cite{chahine2023image} have emerged, offering a setting tailored to portrait photography including diverse scenes, each independently annotated by image quality experts.&#10;The PIQ23 dataset is crafted to achieve high precision in annotations by employing two key strategies: a) utilizing pairwise comparisons for image annotation, a method known for enhancing consistency in IQA experiments \cite{mantiuk2012comparison, perez2019pairwise}; and b) categorizing images by content into distinct scenes to harness the human visual system's precision when evaluating images with shared content. Notably, the PIQ23 dataset distinguishes itself with a comprehensive array of portrait images sourced from over 100 smartphone models, encompassing 50 varied scenarios that represent a wide range of photographic conditions.&#10;&#10;Blind IQA (BIQA) has gained significant interest in recent years, as it offers universal metrics for quality assessment without the need for pristine reference images, often scarce in practical photography scenarios. BIQA methods, based on convolutional neural networks (CNN) \cite{kang2014convolutional, kim2016fully, zhang2018blind}, have shown marked improvements over their classical counterparts, due to their ability to extract perceptual quality information \cite{ghadiyaram2017perceptual, mittal2012no, moorthy2011blind, saad2012blind, ye2012unsupervised}. &#10;However, many existing BIQA methods exhibit limitations in capturing scene-specific semantics, often treating diverse scenes with a one-size-fits-all approach. &#10;Given that image quality is inherently subjective and varies under different conditions, numerous studies underscore the significance of integrating semantic information into the assessment of image quality, tackling a challenge known as domain shift \cite{sun2021blind, zerman2017extensive, zhang2020learning, zhang2021uncertainty}. This is often achieved through a semantics-aware multitasking framework \cite{chahine2023image, fang2020perceptual, feng2023learning, huang2020multi, su2020blindly}.&#10;Moreover, the variability of quality scales across IQA datasets introduces ambiguities, complicating the aggregation of IQA insights. As a result, most BIQA methods struggle to generalize to new conditions, underscoring a critical challenge in cross-domain image quality evaluations.&#10;&#10;In tackling the limitations of current BIQA methodologies, such as combining IQA knowledge and limitations on generalization, this work presents FULL-HyperIQA (FHIQA), an advancement over prior HyperNetwork models (\cite{su2020blindly, chahine2023image}). FHIQA not only considers scene-specific semantics in predicting image quality but also merges knowledge across various scenes for improved generalization. This approach offers a context-aware IQA method, better suited to adapting to new, unfamiliar conditions.&#10;&#10;Our contribution is a new BIQA model, FHIQA, focusing on, a) precision in quality metrics, meaning, the capacity to generate highly granular results thus enabling the differentiation of closely matched cameras; and b) generality by extending the scope of previous work to tackle image content (scenes) not encountered during training. Our comprehensive tests on a newly introduced PIQ23 test split demonstrate FHIQA's enhanced generalization capabilities over existing BIQA benchmarks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Machine Learning Algorithms, Mathematics, Image Quality Assessment, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1511.02919" label="1511.02919">
-        <attvalues>
-          <attvalue for="0" value="Massive Online Crowdsourced Study of Subjective and Objective Picture&#10;  Quality" />
-          <attvalue for="1" value="  Most publicly available image quality databases have been created under&#10;highly controlled conditions by introducing graded simulated distortions onto&#10;high-quality photographs. However, images captured using typical real-world&#10;mobile camera devices are usually afflicted by complex mixtures of multiple&#10;distortions, which are not necessarily well-modeled by the synthetic&#10;distortions found in existing databases. The originators of existing legacy&#10;databases usually conducted human psychometric studies to obtain statistically&#10;meaningful sets of human opinion scores on images in a stringently controlled&#10;visual environment, resulting in small data collections relative to other kinds&#10;of image analysis databases. Towards overcoming these limitations, we designed&#10;and created a new database that we call the LIVE In the Wild Image Quality&#10;Challenge Database, which contains widely diverse authentic image distortions&#10;on a large number of images captured using a representative variety of modern&#10;mobile devices. We also designed and implemented a new online crowdsourcing&#10;system, which we have used to conduct a very large-scale, multi-month image&#10;quality assessment subjective study. Our database consists of over 350000&#10;opinion scores on 1162 images evaluated by over 7000 unique human observers.&#10;Despite the lack of control over the experimental environments of the numerous&#10;study participants, we demonstrate excellent internal consistency of the&#10;subjective dataset. We also evaluate several top-performing blind Image Quality&#10;Assessment algorithms on it and present insights on how mixtures of distortions&#10;challenge both end users as well as automatic perceptual quality prediction&#10;models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.06180" label="1910.06180">
-        <attvalues>
-          <attvalue for="0" value="KonIQ-10k: An ecologically valid database for deep learning of blind&#10;  image quality assessment" />
-          <attvalue for="1" value="  Deep learning methods for image quality assessment (IQA) are limited due to&#10;the small size of existing datasets. Extensive datasets require substantial&#10;resources both for generating publishable content and annotating it accurately.&#10;We present a systematic and scalable approach to creating KonIQ-10k, the&#10;largest IQA dataset to date, consisting of 10,073 quality scored images. It is&#10;the first in-the-wild database aiming for ecological validity, concerning the&#10;authenticity of distortions, the diversity of content, and quality-related&#10;indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable&#10;quality ratings from 1,459 crowd workers, paving the way for more general IQA&#10;models. We propose a novel, deep learning model (KonCept512), to show an&#10;excellent generalization beyond the test set (0.921 SROCC), to the current&#10;state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its&#10;core performance from the InceptionResNet architecture, being trained at a&#10;higher resolution than previous models (512x384). Correlation analysis shows&#10;that KonCept512 performs similar to having 9 subjective scores for each test&#10;image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.10088" label="1912.10088">
-        <attvalues>
-          <attvalue for="0" value="From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of&#10;  Picture Quality" />
-          <attvalue for="1" value="  Blind or no-reference (NR) perceptual picture quality prediction is a&#10;difficult, unsolved problem of great consequence to the social and streaming&#10;media industries that impacts billions of viewers daily. Unfortunately, popular&#10;NR prediction models perform poorly on real-world distorted pictures. To&#10;advance progress on this problem, we introduce the largest (by far) subjective&#10;picture quality database, containing about 40000 real-world distorted pictures&#10;and 120000 patches, on which we collected about 4M human judgments of picture&#10;quality. Using these picture and patch quality labels, we built deep&#10;region-based architectures that learn to produce state-of-the-art global&#10;picture quality predictions as well as useful local picture quality maps. Our&#10;innovations include picture quality prediction architectures that produce&#10;global-to-local inferences as well as local-to-global inferences (via&#10;feedback).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.05772" label="2304.05772">
-        <attvalues>
-          <attvalue for="0" value="An Image Quality Assessment Dataset for Portraits" />
-          <attvalue for="1" value="  Year after year, the demand for ever-better smartphone photos continues to&#10;grow, in particular in the domain of portrait photography. Manufacturers thus&#10;use perceptual quality criteria throughout the development of smartphone&#10;cameras. This costly procedure can be partially replaced by automated&#10;learning-based methods for image quality assessment (IQA). Due to its&#10;subjective nature, it is necessary to estimate and guarantee the consistency of&#10;the IQA process, a characteristic lacking in the mean opinion scores (MOS)&#10;widely used for crowdsourcing IQA. In addition, existing blind IQA (BIQA)&#10;datasets pay little attention to the difficulty of cross-content assessment,&#10;which may degrade the quality of annotations. This paper introduces PIQ23, a&#10;portrait-specific IQA dataset of 5116 images of 50 predefined scenarios&#10;acquired by 100 smartphones, covering a high variety of brands, models, and use&#10;cases. The dataset includes individuals of various genders and ethnicities who&#10;have given explicit and informed consent for their photographs to be used in&#10;public research. It is annotated by pairwise comparisons (PWC) collected from&#10;over 30 image quality experts for three image attributes: face detail&#10;preservation, face target exposure, and overall image quality. An in-depth&#10;statistical analysis of these annotations allows us to evaluate their&#10;consistency over PIQ23. Finally, we show through an extensive comparison with&#10;existing baselines that semantic information (image context) can be used to&#10;improve IQA predictions. The dataset along with the proposed statistical&#10;analysis and BIQA algorithms are available:&#10;https://github.com/DXOMARK-Research/PIQ2023&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.04757" label="1609.04757">
-        <attvalues>
-          <attvalue for="0" value="Perceptual Quality Prediction on Authentically Distorted Images Using a&#10;  Bag of Features Approach" />
-          <attvalue for="1" value="  Current top-performing blind perceptual image quality prediction models are&#10;generally trained on legacy databases of human quality opinion scores on&#10;synthetically distorted images. Therefore they learn image features that&#10;effectively predict human visual quality judgments of inauthentic, and usually&#10;isolated (single) distortions. However, real-world images usually contain&#10;complex, composite mixtures of multiple distortions. We study the perceptually&#10;relevant natural scene statistics of such authentically distorted images, in&#10;different color spaces and transform domains. We propose a bag of feature-maps&#10;approach which avoids assumptions about the type of distortion(s) contained in&#10;an image, focusing instead on capturing consistencies, or departures therefrom,&#10;of the statistics of real world images. Using a large database of authentically&#10;distorted images, human opinions of them, and bags of features computed on&#10;them, we train a regressor to conduct image quality prediction. We demonstrate&#10;the competence of the features towards improving automatic perceptual quality&#10;prediction by testing a learned algorithm using them on a benchmark legacy&#10;database as well as on a newly introduced distortion-realistic resource called&#10;the LIVE In the Wild Image Quality Challenge Database. We extensively evaluate&#10;the perceptual quality prediction model and algorithm and show that it is able&#10;to achieve good quality prediction power that is better than other leading&#10;models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.14550" label="2105.14550">
-        <attvalues>
-          <attvalue for="0" value="Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature&#10;  Fusion and Iterative Mixed Database Training" />
-          <attvalue for="1" value="  Image quality assessment (IQA) is very important for both end-users and&#10;service providers since a high-quality image can significantly improve the&#10;user's quality of experience (QoE) and also benefit lots of computer vision&#10;algorithms. Most existing blind image quality assessment (BIQA) models were&#10;developed for synthetically distorted images, however, they perform poorly on&#10;in-the-wild images, which are widely existed in various practical applications.&#10;In this paper, we propose a novel BIQA model for in-the-wild images by&#10;addressing two critical problems in this field: how to learn better&#10;quality-aware feature representation, and how to solve the problem of&#10;insufficient training samples in terms of their content and distortion&#10;diversity. Considering that perceptual visual quality is affected by both&#10;low-level visual features (e.g. distortions) and high-level semantic&#10;information (e.g. content), we first propose a staircase structure to&#10;hierarchically integrate the features from intermediate layers into the final&#10;feature representation, which enables the model to make full use of visual&#10;information from low-level to high-level. Then an iterative mixed database&#10;training (IMDT) strategy is proposed to train the BIQA model on multiple&#10;databases simultaneously, so the model can benefit from the increase in both&#10;training samples and image content and distortion diversity and can learn a&#10;more general feature representation. Experimental results show that the&#10;proposed model outperforms other state-of-the-art BIQA models on six&#10;in-the-wild IQA databases by a large margin. Moreover, the proposed model shows&#10;an excellent performance in the cross-database evaluation experiments, which&#10;further demonstrates that the learned feature representation is robust to&#10;images with diverse distortions and content. The code is available at&#10;https://github.com/sunwei925/StairIQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.12108" label="2011.12108">
-        <attvalues>
-          <attvalue for="0" value="Wide-angle Image Rectification: A Survey" />
-          <attvalue for="1" value="  Wide field-of-view (FOV) cameras, which capture a larger scene area than&#10;narrow FOV cameras, are used in many applications including 3D reconstruction,&#10;autonomous driving, and video surveillance. However, wide-angle images contain&#10;distortions that violate the assumptions underlying pinhole camera models,&#10;resulting in object distortion, difficulties in estimating scene distance,&#10;area, and direction, and preventing the use of off-the-shelf deep models&#10;trained on undistorted images for downstream computer vision tasks. Image&#10;rectification, which aims to correct these distortions, can solve these&#10;problems. In this paper, we comprehensively survey progress in wide-angle image&#10;rectification from transformation models to rectification methods.&#10;Specifically, we first present a detailed description and discussion of the&#10;camera models used in different approaches. Then, we summarize several&#10;distortion models including radial distortion and projection distortion. Next,&#10;we review both traditional geometry-based image rectification methods and deep&#10;learning-based methods, where the former formulate distortion parameter&#10;estimation as an optimization problem and the latter treat it as a regression&#10;problem by leveraging the power of deep neural networks. We evaluate the&#10;performance of state-of-the-art methods on public datasets and show that&#10;although both kinds of methods can achieve good results, these methods only&#10;work well for specific camera models and distortion types. We also provide a&#10;strong baseline model and carry out an empirical study of different distortion&#10;models on synthetic datasets and real-world wide-angle images. Finally, we&#10;discuss several potential research directions that are expected to further&#10;advance this area in the future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.12950" label="2009.12950">
-        <attvalues>
-          <attvalue for="0" value="Human-Object Interaction Detection:A Quick Survey and Examination of&#10;  Methods" />
-          <attvalue for="1" value="  Human-object interaction detection is a relatively new task in the world of&#10;computer vision and visual semantic information extraction. With the goal of&#10;machines identifying interactions that humans perform on objects, there are&#10;many real-world use cases for the research in this field. To our knowledge,&#10;this is the first general survey of the state-of-the-art and milestone works in&#10;this field. We provide a basic survey of the developments in the field of&#10;human-object interaction detection. Many works in this field use multi-stream&#10;convolutional neural network architectures, which combine features from&#10;multiple sources in the input image. Most commonly these are the humans and&#10;objects in question, as well as the spatial quality of the two. As far as we&#10;are aware, there have not been in-depth studies performed that look into the&#10;performance of each component individually. In order to provide insight to&#10;future researchers, we perform an individualized study that examines the&#10;performance of each component of a multi-stream convolutional neural network&#10;architecture for human-object interaction detection. Specifically, we examine&#10;the HORCNN architecture as it is a foundational work in the field. In addition,&#10;we provide an in-depth look at the HICO-DET dataset, a popular benchmark in the&#10;field of human-object interaction detection. Code and papers can be found at&#10;https://github.com/SHI-Labs/Human-Object-Interaction-Detection.&#10;" />
-          <attvalue for="2" value="&#10;Achieving the goal of true machine intelligence requires an agent that can observe and understand its environment just as humans are able to. There has been a significant amount of excitement and progress around machine learning and its ability to solve problems related to emulating human understanding of our natural and social environments. The field of computer vision, in particular, has recently exploded with the advent of deep learning techniques that can perform well on complex object detection problems. However, simply identifying objects in an image is not what should be considered true machine intelligence. Striving towards the idea of more intelligent machines, researchers have created models and systems that can extract richer semantic information from images and videos. As humans, we are able to recognize relationships between objects in an image. These relationships can help an intelligent machine interpret the underlying meaning of the image or scene, and therefore, take one step closer to understanding the world around us. &#10;&#10;We choose to divide the main track of AI computer vision research into two tracks, visual perception tasks, and visual understanding tasks. Visual perception tasks focus on identifying parts or features of an image. Object detection and image segmentation are examples of well known domains that fit in the purview of visual perception. Given a scene or image, our goal is to make some quick observations that are easy to see. For example, we should be able to determine the possible objects present in an image, or whether or not a sectionof the image is part of an object or the background. These properties must to be learned or hand designed, but generally are not more complex than reinforcing specific feature combinations that make up a human, while a different combination of features make up a cat. Visual understanding tasks on the other hand, require far more complex analysis of a scene. These tasks focus on the less visual and at as easily recognized features of an image. Visual understanding tasks include domains such as visual relationship detection and activity recognition, as well as our focus of human-object interaction detection. In many of these tasks, visual perceptions such as object detection, are prerequisites to obtain before moving on to identifying the finer grained features needed to complete the task. For human-object interaction detection, a model must detect the possible objects in the image, then make a decision on whether an interaction is occurring and if so what that interaction is.&#10;&#10;Human-object interaction detection is closely related to other computer vision research areas such as visual relationship detection and activity recognition. However, the foundational works and datasets differ from those used in the aforementioned tasks. We feel that there is a need for a brief and general overview for future researchers to easily obtain the baseline knowledge to create contributions in this field. One of our main contributions is a quick survey of deep learning based methods for solving human-object interaction detection, as well as a look at some of the datasets and metrics used for evaluating models. Commonly in this field, classifying the interactions is done through the use of a multi-stream convolutional neural network. We present a detailed examination of the performance of each individual stream component. This investigation can provide useful information on how to develop future models, using this multi-stream method. Using the components from HORCNN \cite{chao2018learning}, we conduct tests on their ability to correctly classify human-object interactions.&#10;&#10;We also provide a careful analysis of the HICO-DET dataset \cite{chao2018learning}. This dataset is commonly used as a baseline for human-object interaction detection. This dataset contains a large and complex set of interactions on various objects. We provide a meaningful analysis of the various components, and suggestions for improving this state-of-the-art dataset. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Neural Network Architectures, Machine Learning, Computer Vision, Artificial Intelligence, Cognitive Science, Computer Science, Human-Object Interaction" />
-        </attvalues>
-      </node>
-      <node id="2211.13509" label="2211.13509">
-        <attvalues>
-          <attvalue for="0" value="The Second-place Solution for ECCV 2022 Multiple People Tracking in&#10;  Group Dance Challenge" />
-          <attvalue for="1" value="  This is our 2nd-place solution for the ECCV 2022 Multiple People Tracking in&#10;Group Dance Challenge. Our method mainly includes two steps: online short-term&#10;tracking using our Cascaded Buffer-IoU (C-BIoU) Tracker, and, offline long-term&#10;tracking using appearance feature and hierarchical clustering. Our C-BIoU&#10;tracker adds buffers to expand the matching space of detections and tracks,&#10;which mitigates the effect of irregular motions in two aspects: one is to&#10;directly match identical but non-overlapping detections and tracks in adjacent&#10;frames, and the other is to compensate for the motion estimation bias in the&#10;matching space. In addition, to reduce the risk of overexpansion of the&#10;matching space, cascaded matching is employed: first matching alive tracks and&#10;detections with a small buffer, and then matching unmatched tracks and&#10;detections with a large buffer. After using our C-BIoU for online tracking, we&#10;applied the offline refinement introduced by ReMOTS.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Although MOT studies have been greatly developed~\cite{SORT,DeepSORT,bytetrack, Li_2022_CVPR}, a new challenge has recently attracted attention: unlike conventional MOT tasks that contain objects with distinct appearances and regular motions, MOT tasks that cover animals, group dancers, and sports players, may have indistinguishable appearances and irregular motions, which could cause existing MOT methods to fail. In particular, several MOT methods~\cite{SORT, DeepSORT, bytetrack, cao2022observation} that perform well on MOT17~\cite{MOT16}, may experience a significant performance drop on the DanceTrack~\cite{sun2022dancetrack}.&#10;&#10;We presume that tracking failures are caused by two reasons: &#10;(i) The detections and tracks of identical objects do not overlap between adjacent frames ({e.g.}, due to the fast movement) and thus the tracking fails; (ii) After track initialization, unmatched tracks ({e.g.}, occluded objects) continue to update their geometric features for multiple frames, however, if their motion estimations are inaccurate ({e.g.}, due to a sudden acceleration or turning), they miss the matching opportunity when corresponding detections are available in subsequent frames. When the appearance of objects can be distinguished, appearance features could be employed to alleviate issues (i) and (ii), by matching cross-frame detections based on their appearance similarities. Nonetheless, when irregular motions are accompanied by indistinguishable appearances, most existing MOT solutions may not be able to perform dependable tracking, so a new solution is desirable.&#10;&#10;In this study, we propose a Cascaded-Buffered Intersectionover Union (C-BIoU) tracker~\cite{CBIOU_2023_WACV} to track multiple objects that have irregular motions and indistinguishable appearances. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Tracking, Computer Science, Motion Estimation" />
-        </attvalues>
-      </node>
-      <node id="2110.06864" label="2110.06864">
-        <attvalues>
-          <attvalue for="0" value="ByteTrack: Multi-Object Tracking by Associating Every Detection Box" />
-          <attvalue for="1" value="  Multi-object tracking (MOT) aims at estimating bounding boxes and identities&#10;of objects in videos. Most methods obtain identities by associating detection&#10;boxes whose scores are higher than a threshold. The objects with low detection&#10;scores, e.g. occluded objects, are simply thrown away, which brings&#10;non-negligible true object missing and fragmented trajectories. To solve this&#10;problem, we present a simple, effective and generic association method,&#10;tracking by associating almost every detection box instead of only the high&#10;score ones. For the low score detection boxes, we utilize their similarities&#10;with tracklets to recover true objects and filter out the background&#10;detections. When applied to 9 different state-of-the-art trackers, our method&#10;achieves consistent improvement on IDF1 score ranging from 1 to 10 points. To&#10;put forwards the state-of-the-art performance of MOT, we design a simple and&#10;strong tracker, named ByteTrack. For the first time, we achieve 80.3 MOTA, 77.3&#10;IDF1 and 63.1 HOTA on the test set of MOT17 with 30 FPS running speed on a&#10;single V100 GPU. ByteTrack also achieves state-of-the-art performance on MOT20,&#10;HiEve and BDD100K tracking benchmarks. The source code, pre-trained models with&#10;deploy versions and tutorials of applying to other trackers are released at&#10;https://github.com/ifzhang/ByteTrack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.07843" label="2202.07843">
-        <attvalues>
-          <attvalue for="0" value="PCRP: Unsupervised Point Cloud Object Retrieval and Pose Estimation" />
-          <attvalue for="1" value="  An unsupervised point cloud object retrieval and pose estimation method,&#10;called PCRP, is proposed in this work. It is assumed that there exists a&#10;gallery point cloud set that contains point cloud objects with given pose&#10;orientation information. PCRP attempts to register the unknown point cloud&#10;object with those in the gallery set so as to achieve content-based object&#10;retrieval and pose estimation jointly, where the point cloud registration task&#10;is built upon an enhanced version of the unsupervised R-PointHop method.&#10;Experiments on the ModelNet40 dataset demonstrate the superior performance of&#10;PCRP in comparison with traditional and learning based methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Content-based point cloud object retrieval and category-level point&#10;cloud object pose estimation are two important tasks of point cloud&#10;processing. For the former, one can find similar objects from the&#10;gallery set, which can provide more information about the unknown&#10;object. For the latter, the goal is to estimate the 6-DOF pose of a 3D&#10;object comprising of rotation $(R \in SO(3))$ and translation $(t \in&#10;\mathbb{R}^3)$, with respect to a chosen reference. The pose&#10;information can facilitate downstream tasks such as object grasping,&#10;obstacle avoidance and path planning for robotics. In a typical scene&#10;understanding problem using data from range sensors or a depth camera,&#10;this problem would arise after a 3D detection algorithm has successfully&#10;localized and labeled the objects present in the point cloud scan. &#10;An unsupervised point cloud object retrieval and pose estimation method,&#10;called PCRP, is proposed in this work. &#10;&#10;The R-PointHop method was recently proposed by Kadam et al.&#10;\cite{kadam2021r} for point cloud registration. Although being&#10;unsupervised, it offers competitive performance with respect to other&#10;supervised learning methods. In this paper, we extend R-PointHop to the&#10;context of point cloud object retrieval and pose estimation against a&#10;gallery point cloud set, which contains point cloud objects with known&#10;pose orientation information. Point cloud retrieval has been researched&#10;for quite some time in terms of retrieving a similar object from a&#10;database or aggregating local feature descriptors for recognizing places&#10;in the wild. Yet, retrieving objects with pose variations is less&#10;investigated. Here, we show how R-PointHop features can be reused to&#10;retrieve a similar point cloud object. Registration of two similar&#10;objects (potentially with partial overlap), which was the focus of&#10;R-PointHop, has been widely studied. Although being a related problem,&#10;estimating the pose of a single object addressed in this work is less&#10;explored. We analyze several bottlenecks of R-PointHop and propose&#10;modifications to enhance its performance for pose estimation. &#10;&#10;Built upon enhanced R-PointHop, PCRP registers the unknown point cloud&#10;object with those in the gallery set to achieve content-based object&#10;retrieval and pose estimation jointly. As shown in Fig.&#10;\ref{fig:summary}, PCRP consists of ``object retrieval&quot; and ``pose&#10;estimation&quot; two functions. For object retrieval, it first aggregates the&#10;pointwise features learned from R-PointHop based on VLAD (Vector of&#10;Locally Aggregated Descriptors) \cite{jegou2010aggregating} to obtain a&#10;global point cloud feature vector and then use it to retrieve a similar&#10;pre-aligned object from the gallery set. For pose estimation, the 6-DOF&#10;pose of the query object is found by registering it with the retrieved&#10;object. Experiments on the ModelNet40 dataset demonstrate the superior&#10;performance of PCRP in comparison with traditional and learning based&#10;methods. &#10;&#10;This work has two main contributions. First, we extend R-PointHop, which&#10;was originally designed for point cloud registration, to object&#10;retrieval and pose estimation. We show how features derived from&#10;R-PointHop can be aggregated to yield a global feature descriptor for&#10;object retrieval and reuse point features for pose estimation. Second, we&#10;propose ways to modify the attribute representation in R-PointHop and&#10;make it more general. As a result, any traditional point local&#10;descriptors, such as FPFH \cite{rusu2009fast} and SHOT&#10;\cite{tombari2010unique}, can be adopted by R-PointHop. The rest of the&#10;paper is organized as follows. Related previous work is reviewed in Sec.&#10;\ref{sec:review}. The PCRP method is proposed in Sec. \ref{sec:method}.&#10;Experimental results are shown in Sec. \ref{sec:results}. Finally,&#10;concluding remarks are given in Sec. \ref{sec:conclusion}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Pose Estimation Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Geometric Computing, Computer Science, Point Cloud Analysis, Object Retrieval Methods" />
-        </attvalues>
-      </node>
-      <node id="2103.08129" label="2103.08129">
-        <attvalues>
-          <attvalue for="0" value="R-PointHop: A Green, Accurate, and Unsupervised Point Cloud Registration&#10;  Method" />
-          <attvalue for="1" value="  Inspired by the recent PointHop classification method, an unsupervised 3D&#10;point cloud registration method, called R-PointHop, is proposed in this work.&#10;R-PointHop first determines a local reference frame (LRF) for every point using&#10;its nearest neighbors and finds local attributes. Next, R-PointHop obtains&#10;local-to-global hierarchical features by point downsampling, neighborhood&#10;expansion, attribute construction and dimensionality reduction steps. Thus,&#10;point correspondences are built in hierarchical feature space using the nearest&#10;neighbor rule. Afterwards, a subset of salient points with good correspondence&#10;is selected to estimate the 3D transformation. The use of the LRF allows for&#10;invariance of the hierarchical features of points with respect to rotation and&#10;translation, thus making R-PointHop more robust at building point&#10;correspondence, even when the rotation angles are large. Experiments are&#10;conducted on the 3DMatch, ModelNet40, and Stanford Bunny datasets, which&#10;demonstrate the effectiveness of R-PointHop for 3D point cloud registration.&#10;R-PointHop's model size and training time are an order of magnitude smaller&#10;than those of deep learning methods, and its registration errors are smaller,&#10;making it a green and accurate solution. Our codes are available on GitHub.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.05275" label="2004.05275">
-        <attvalues>
-          <attvalue for="0" value="Multi-View Matching (MVM): Facilitating Multi-Person 3D Pose Estimation&#10;  Learning with Action-Frozen People Video" />
-          <attvalue for="1" value="  To tackle the challeging problem of multi-person 3D pose estimation from a&#10;single image, we propose a multi-view matching (MVM) method in this work. The&#10;MVM method generates reliable 3D human poses from a large-scale video dataset,&#10;called the Mannequin dataset, that contains action-frozen people immitating&#10;mannequins. With a large amount of in-the-wild video data labeled by 3D&#10;supervisions automatically generated by MVM, we are able to train a neural&#10;network that takes a single image as the input for multi-person 3D pose&#10;estimation. The core technology of MVM lies in effective alignment of 2D poses&#10;obtained from multiple views of a static scene that has a strong geometric&#10;constraint. Our objective is to maximize mutual consistency of 2D poses&#10;estimated in multiple frames, where geometric constraints as well as appearance&#10;similarities are taken into account simultaneously. To demonstrate the&#10;effectiveness of 3D supervisions provided by the MVM method, we conduct&#10;experiments on the 3DPW and the MSCOCO datasets and show that our proposed&#10;solution offers the state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Human pose estimation is a long-standing problem in computer vision&#10;research. It has numerous applications such as sports, augmented&#10;reality, motion analysis, visual avatar creation, etc. In the past ten&#10;years, a major advance has been made in building large-scale human pose&#10;datasets and developing deep-learning-based models for human pose&#10;estimation. As a result, estimating multi-person 2D poses and/or a&#10;single-person 3D pose in complicated scenes become mature. Yet,&#10;multi-person 3D pose estimation is still a challeging problem due to the&#10;lack of large-scale high-quality datasets for this application. This is&#10;further hindered by inadequate depth estimation algorithms in monocular&#10;images. &#10;&#10;It is nontrivial to acquire high quality 3D supervisions in dynamic&#10;scenes. Depth sensors (e.g., Kinect) can provide useful data, yet the&#10;acquisition is typically limited to indoor environments. Furthermore, it&#10;often demands a large amount of manual work in capturing and processing.&#10;As an alternative, Li {et al.} \cite{li2019learning} attempted to&#10;estimate dense depth maps using the multi-view stereo (MVS) method from&#10;video clips captured in stationary scenes. &#10;&#10;The main objective of our current research is to obtain high quality 3D&#10;skeletons from multiple 2D poses obtained in each frame of action-frozen&#10;people video. Such a dataset was collected by focusing on a special&#10;kind of Youtube video. That is, people imitate mannequins and freeze in&#10;elaborate and natural poses in a scene while a hand-held camera tours&#10;the scene to create the desired video. It is called the Mannequin&#10;dataset. This type of video is suitable for human pose estimation since&#10;it can provide diverse poses from people of different ages and genders&#10;with different scales in a wide variety of outdoor scenes. We propose a&#10;multi-view matching (MVM) method to achieve this goal. With a large&#10;amount of in-the-wild video data labeled by 3D supervisions&#10;automatically generated by MVM, we are able to train a neural network&#10;that takes a single image as the input and generate the associated&#10;multi-person 3D pose estimation as the output. &#10;&#10;Our approach has several advantages in multi-person 3D pose&#10;estimation. First, the scene does not have to be an indoor and/or lab&#10;environment. Second, there are a considerable number of video clips&#10;available online. If needed, it is possible to make more action-frozen&#10;people video at a low cost. Third, we can get high quality 3D skeleton&#10;information by exploiting the static scene assumption. To the best of&#10;our knowledge, there is only one in-the-wild 3D human pose dataset,&#10;called MPII-3DPW \cite{von2018recovering}. It relies on Inertial Measurement Unit&#10;(IMU) sensors&#10;attached to a few actors or actresses. As compared to MPII-3DPW, our&#10;approach is more scalable. It contains more diverse contents in terms of&#10;subjects and environments. Some exemplary video clips in the Mannequin&#10;dataset are shown in Fig. \ref{fig:diversity}. With the help of such&#10;video clips, we can obtain diverse 3D human poses in a wide range of&#10;scenes in daily life. &#10;&#10;Estimating 3D skeletons from predicted 2D poses of multiple views has&#10;been studied previously in several settings, e.g. the synchronized&#10;multi-camera lab environment \cite{dong2019fast,joo2017panoptic}, the&#10;synthesized sports video \cite{bridgeman2019multi}. However, existing&#10;methods do not work well in the current setting (namely, video of&#10;action-frozen people captured by a hand-held camera) for the following&#10;reason. In the action-frozen video clip, each frame can be essentially&#10;regarded as a single view of the scene. A typical 10-second video&#10;sequence sampled at 25fps will yield 250 views of the scene from 250&#10;viewing angles. As observed in \cite{dong2019fast}, the main challenge&#10;is to build the correspondence of predicted 2D poses in different&#10;frames. The optimization method in \cite{dong2019fast} was developed to&#10;minimize the cycle consistency loss in a multi-camera lab environment&#10;with a small number of views (e.g. five cameras). The solution in&#10;\cite{dong2019fast} is computationally intractable in our current case,&#10;which has a larger number of views and more people in the scene. &#10;&#10;Besides, pose tracking methods like \cite{andriluka2018posetrack} can be a potential option when we need to determine which 2D poses in different frames corresponds to the same person. However, we argue that they are also not efficient enough in our case. The reason is that, when identifying the correspondence of multiple 2D poses, the 3D geometric constraints plays a more important role compared to the visual similarity clue that is usually what the pose tracking methods rely on, which is further confirmed by our empirical experiments. As it is not common for pose tracking methods to make assumption of static scenes, the 3D geometric constraints are not fully utilized. &#10;&#10;The pipeline of the proposed MVM method is shown in Fig.&#10;\ref{fig:framework}. First, we use the OpenPifPaf method&#10;\cite{kreiss2019pifpaf} to estimate multi-person 2D poses at each&#10;frame. Second, we find the correspondence of 2D poses across multiple&#10;frames. This is achieved by adopting an approximation to the&#10;optimization objective in \cite{dong2019fast}, where both geometric&#10;constraints and appearance similarities are taken into account. Third,&#10;we apply triangulation to groups of matched 2D poses and conduct bundle&#10;adjustment to recover multi-person 3D poses. We provide both&#10;qualitative and quantative results to show the quality of the&#10;multi-person 3D poses in Youtube video. &#10;&#10;In practice, we are interested in multi-person 3D pose estimation&#10;from a single image. To accomplish this goal, we train a monocular&#10;multi-person 3D pose network, which is a modification of the network&#10;proposed in \cite{kreiss2019pifpaf}, from multiple frames using the 3D&#10;supervisions obtained by MVM. To demonstrate the effectiveness of 3D&#10;supervisions provided by the MVM method, we conduct experiments on the&#10;3DPW and the MSCOCO datasets. Evaluation shows performance improvement&#10;in the 3D human pose estimation accuracy for the 3DPW dataset and the 2D&#10;human pose estimation accuracy for the MSCOCO dataset. &#10;&#10;There are four main contributions of our current research.&#10;\begin{itemize}&#10;\item We develop an efficient method, called the MVM method, that&#10;levarages geometric constraints and appearance similarities existing in&#10;video clips of static scenes. for reliable 3D human poses estimation. &#10;\item We collect a large-scale Youtube video clips containing&#10;action-frozen people with in-the-wild scenes and call it the Mannequin&#10;dataset. It can be used as a training dataset for multi-person 3D&#10;pose estimation. &#10;\item Although there is no groundtruth, we use the MVM method to generate&#10;weak 3D supervisions for the Mannequin dataset so that the dataset can&#10;be used to train a multi-person 3D neural network that is applied to a&#10;single image. &#10;\item We use the Mannequin dataset as the training dataset and show the&#10;effectiveness of the weak 3D supervisions provided by MVM through&#10;extensive experimental results conducted for the 3DPW dataset and the 2D&#10;MSCOCO dataset. &#10;\end{itemize}&#10;&#10;The rest of this paper is organized as follows. Related work is reviewed&#10;in Sec. \ref{sec:review}. The Mannequin dataset is introduced in Sec.&#10;\ref{sec:dataset}. The MVM method is proposed in Sec.&#10;\ref{sec:method-1}. The solution to the multi-person 3D pose estimation&#10;problem from a single image is discussed in Sec. \ref{sec:method-2}.&#10;Experimental results are shown in Sec. \ref{sec:experiments}. Finally,&#10;concluding remarks are given in Sec. \ref{sec:conclusion}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Human Pose Estimation, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1901.04111" label="1901.04111">
-        <attvalues>
-          <attvalue for="0" value="Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views" />
-          <attvalue for="1" value="  This paper addresses the problem of 3D pose estimation for multiple people in&#10;a few calibrated camera views. The main challenge of this problem is to find&#10;the cross-view correspondences among noisy and incomplete 2D pose predictions.&#10;Most previous methods address this challenge by directly reasoning in 3D using&#10;a pictorial structure model, which is inefficient due to the huge state space.&#10;We propose a fast and robust approach to solve this problem. Our key idea is to&#10;use a multi-way matching algorithm to cluster the detected 2D poses in all&#10;views. Each resulting cluster encodes 2D poses of the same person across&#10;different views and consistent correspondences across the keypoints, from which&#10;the 3D pose of each person can be effectively inferred. The proposed convex&#10;optimization based multi-way matching algorithm is efficient and robust against&#10;missing and false detections, without knowing the number of people in the&#10;scene. Moreover, we propose to combine geometric and appearance cues for&#10;cross-view matching. The proposed approach achieves significant performance&#10;gains from the state-of-the-art (96.3% vs. 90.6% and 96.9% vs. 88% on the&#10;Campus and Shelf datasets, respectively), while being efficient for real-time&#10;applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.03153" label="1612.03153">
-        <attvalues>
-          <attvalue for="0" value="Panoptic Studio: A Massively Multiview System for Social Interaction&#10;  Capture" />
-          <attvalue for="1" value="  We present an approach to capture the 3D motion of a group of people engaged&#10;in a social interaction. The core challenges in capturing social interactions&#10;are: (1) occlusion is functional and frequent; (2) subtle motion needs to be&#10;measured over a space large enough to host a social group; (3) human appearance&#10;and configuration variation is immense; and (4) attaching markers to the body&#10;may prime the nature of interactions. The Panoptic Studio is a system organized&#10;around the thesis that social interactions should be measured through the&#10;integration of perceptual analyses over a large variety of view points. We&#10;present a modularized system designed around this principle, consisting of&#10;integrated structural, hardware, and software innovations. The system takes, as&#10;input, 480 synchronized video streams of multiple people engaged in social&#10;activities, and produces, as output, the labeled time-varying 3D structure of&#10;anatomical landmarks on individuals in the space. Our algorithm is designed to&#10;fuse the &quot;weak&quot; perceptual processes in the large number of views by&#10;progressively generating skeletal proposals from low-level appearance cues, and&#10;a framework for temporal refinement is also presented by associating body parts&#10;to reconstructed dense 3D trajectory stream. Our system and method are the&#10;first in reconstructing full body motion of more than five people engaged in&#10;social interactions without using markers. We also empirically demonstrate the&#10;impact of the number of views in achieving this goal.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.10000" label="1710.10000">
-        <attvalues>
-          <attvalue for="0" value="PoseTrack: A Benchmark for Human Pose Estimation and Tracking" />
-          <attvalue for="1" value="  Human poses and motions are important cues for analysis of videos with people&#10;and there is strong evidence that representations based on body pose are highly&#10;effective for a variety of tasks such as activity recognition, content&#10;retrieval and social signal processing. In this work, we aim to further advance&#10;the state of the art by establishing &quot;PoseTrack&quot;, a new large-scale benchmark&#10;for video-based human pose estimation and articulated tracking, and bringing&#10;together the community of researchers working on visual human analysis. The&#10;benchmark encompasses three competition tracks focusing on i) single-frame&#10;multi-person pose estimation, ii) multi-person pose estimation in videos, and&#10;iii) multi-person articulated tracking. To facilitate the benchmark and&#10;challenge we collect, annotate and release a new %large-scale benchmark dataset&#10;that features videos with multiple people labeled with person tracks and&#10;articulated pose. A centralized evaluation server is provided to allow&#10;participants to evaluate on a held-out test set. We envision that the proposed&#10;benchmark will stimulate productive research both by providing a large and&#10;representative training dataset as well as providing a platform to objectively&#10;evaluate and compare the proposed methods. The benchmark is freely accessible&#10;at https://posetrack.net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.06692" label="2402.06692">
-        <attvalues>
-          <attvalue for="0" value="HistoHDR-Net: Histogram Equalization for Single LDR to HDR Image&#10;  Translation" />
-          <attvalue for="1" value="  High Dynamic Range (HDR) imaging aims to replicate the high visual quality&#10;and clarity of real-world scenes. Due to the high costs associated with HDR&#10;imaging, the literature offers various data-driven methods for HDR image&#10;reconstruction from Low Dynamic Range (LDR) counterparts. A common limitation&#10;of these approaches is missing details in regions of the reconstructed HDR&#10;images, which are over- or under-exposed in the input LDR images. To this end,&#10;we propose a simple and effective method, HistoHDR-Net, to recover the fine&#10;details (e.g., color, contrast, saturation, and brightness) of HDR images via a&#10;fusion-based approach utilizing histogram-equalized LDR images along with&#10;self-attention guidance. Our experiments demonstrate the efficacy of the&#10;proposed approach over the state-of-art methods.&#10;" />
-          <attvalue for="2" value="&#10;High Dynamic Range (HDR) imaging~\cite{wang2021deep, han2023high,chen2023improving} has been a topic of much focus in the vision community for the last several years.&#10;Many applications~\cite{chen2021new,wang2021deep} require images as detailed as real-world scenes, which cannot be captured by standard Low Dynamic Range (LDR) cameras\mycomment{ (much cheaper than HDR-enabled camera systems)}.&#10;The general camera pipeline~\cite{liu2020single} captures real-world scenes with a high dynamic range of intensity values and maps them to a low dynamic range by performing quantization, non-linear mapping, and dynamic range clipping.&#10;Therefore, the focus in the community has been to reconstruct HDR images from their LDR counterparts using sophisticated data-driven methods that aim to approximate the reverse of the camera pipeline process.&#10;&#10;Most methods proposed in the literature utilize Convolutional Neural Networks and Generative Adversarial Networks~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}, where a single LDR image is used as input to reconstruct the HDR image. &#10;Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}.&#10;Some methods~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR} utilize multi-exposed LDR images (generally $2$ or $3$) as input but often produce many artifacts in the resultant HDR images~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR}. &#10;In contrast, ArtHDR-Net~\cite{barua2023arthdr} uses features from multi-exposed LDR images to reconstruct the final HDR images without artifacts, but it still lacks naturality in contrast and saturation.&#10;Recent methods employ Neural Radiance Fields~\cite{mildenhall2021nerf,huang2022hdr} drawing inspiration from physics/optics to replicate implicit color and radiance fields involved in the general camera pipeline and enable photo-realistic HDR view synthesis~\cite{mildenhall2022nerf, huang2022hdr}.&#10;Dalal \etal~\cite{dalal2023single} used Diffusion Models and exposure loss to replicate the reverse of the camera pipeline process through a conditional diffusion architecture with a noise induction process without requiring any classifier. These advanced methods can reconstruct high-quality HDR images with increased clarity of both foreground and background.&#10;However, further improvements are still required especially for images with highly contrasting and saturated regions.&#10;&#10;There is limited study on the impact of histograms~\cite{yang2023lightingnet} in HDR image reconstruction.&#10;Jang \etal~\cite{jang2020dynamic} proposed a method using histogram and color difference~\cite{coltuc2006exact} between LDR/HDR image pairs to reconstruct HDR images.&#10;Other methods explored image fusion techniques using multi-resolution~\cite{han2023highnew} and/or multi-exposed~\cite{guo2023single} images to enhance contrast levels and remove artifacts in both the HDR and tone-mapped LDR images.&#10;Attention-based methods have also been explored~\cite{nemoto2015visual,li2019hdrnet,zhang2019self,yan2019attention,chen2023improving}.&#10;Li and Fang~\cite{li2019hdrnet} proposed a way to tackle the problem of color quantization using a multi-scale Convolutional Neural Network with an attention mechanism.&#10;The method is capable of reducing the loss of information in over- and under-exposed regions.&#10;In this work, we propose a technique that combines the capabilities of histogram equalization and data fusion to reconstruct high-quality HDR images with better saturation and contrast levels. Fig.~\ref{fig:teaser} depicts the clarity of the images generated by the proposed method compared to the recent state-of-the-art~\cite{barua2023arthdr}.&#10;The contributions of our work, HistoHDR-Net, are:&#10;&#10;\begin{itemize}&#10;\item We design a simple ResNet50~\cite{shin2018cnn} based pipeline to extract features from ground truth LDR and histogram-equalized LDR images using two parallel ResNet50 blocks and perform fusion on them. We further perform self-attention on the fused feature maps before employing them for HDR image reconstruction (see Sections~\ref{sec:fusion_and_attention} and \ref{sec:atten}).&#10;\item We design a novel loss function based on Weber's law~\cite{jiaweber}, Multi-Scale Structural Similarity Index Measure~\cite{wang2009mean,wang2004image,zhao2016loss}, and Color~\cite{sharma2005ciede2000,brainard2003color} to guide the network for better and reliable HDR image reconstruction (see Section~\ref{sec:loss}). &#10;\item We perform a thorough analysis of the proposed loss function, different combinations of fusion and attention mechanisms, and feature extraction backbones, to establish the contribution of each of the components (see Section~\ref{sec:results}).&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Image Reconstruction, Image Processing, Physics, Computer Science, High Dynamic Range" />
-        </attvalues>
-      </node>
-      <node id="2110.10394" label="2110.10394">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning for HDR Imaging: State-of-the-Art and Future Trends" />
-          <attvalue for="1" value="  High dynamic range (HDR) imaging is a technique that allows an extensive&#10;dynamic range of exposures, which is important in image processing, computer&#10;graphics, and computer vision. In recent years, there has been a significant&#10;advancement in HDR imaging using deep learning (DL). This study conducts a&#10;comprehensive and insightful survey and analysis of recent developments in deep&#10;HDR imaging methodologies. We hierarchically and structurally group existing&#10;deep HDR imaging methods into five categories based on (1) number/domain of&#10;input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel&#10;learning strategies, and (5) applications. Importantly, we provide a&#10;constructive discussion on each category regarding its potential and&#10;challenges. Moreover, we review some crucial aspects of deep HDR imaging, such&#10;as datasets and evaluation metrics. Finally, we highlight some open problems&#10;and point out future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.07978" label="2108.07978">
-        <attvalues>
-          <attvalue for="0" value="A New Journey from SDRTV to HDRTV" />
-          <attvalue for="1" value="  Nowadays modern displays are capable to render video content with high&#10;dynamic range (HDR) and wide color gamut (WCG). However, most available&#10;resources are still in standard dynamic range (SDR). Therefore, there is an&#10;urgent demand to transform existing SDR-TV contents into their HDR-TV versions.&#10;In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the&#10;formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step&#10;solution pipeline including adaptive global color mapping, local enhancement&#10;and highlight generation. Moreover, the above analysis inspires us to present a&#10;lightweight network that utilizes global statistics as guidance to conduct&#10;image-adaptive color mapping. In addition, we construct a dataset using HDR&#10;videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate&#10;the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results&#10;achieve state-of-the-art performance in quantitative comparisons and visual&#10;quality. The code and dataset are available at&#10;https://github.com/chxy95/HDRTVNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.01179" label="2004.01179">
-        <attvalues>
-          <attvalue for="0" value="Single-Image HDR Reconstruction by Learning to Reverse the Camera&#10;  Pipeline" />
-          <attvalue for="1" value="  Recovering a high dynamic range (HDR) image from a single low dynamic range&#10;(LDR) input image is challenging due to missing details in under-/over-exposed&#10;regions caused by quantization and saturation of camera sensors. In contrast to&#10;existing learning-based methods, our core idea is to incorporate the domain&#10;knowledge of the LDR image formation pipeline into our model. We model the&#10;HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2)&#10;non-linear mapping from a camera response function, and (3) quantization. We&#10;then propose to learn three specialized CNNs to reverse these steps. By&#10;decomposing the problem into specific sub-tasks, we impose effective physical&#10;constraints to facilitate the training of individual sub-networks. Finally, we&#10;jointly fine-tune the entire model end-to-end to reduce error accumulation.&#10;With extensive quantitative and qualitative experiments on diverse image&#10;datasets, we demonstrate that the proposed method performs favorably against&#10;state-of-the-art single-image HDR reconstruction algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.07335" label="2005.07335">
-        <attvalues>
-          <attvalue for="0" value="Single Image HDR Reconstruction Using a CNN with Masked Features and&#10;  Perceptual Loss" />
-          <attvalue for="1" value="  Digital cameras can only capture a limited range of real-world scenes'&#10;luminance, producing images with saturated pixels. Existing single image high&#10;dynamic range (HDR) reconstruction methods attempt to expand the range of&#10;luminance, but are not able to hallucinate plausible textures, producing&#10;results with artifacts in the saturated areas. In this paper, we present a&#10;novel learning-based approach to reconstruct an HDR image by recovering the&#10;saturated pixels of an input LDR image in a visually pleasing way. Previous&#10;deep learning-based methods apply the same convolutional filters on&#10;well-exposed and saturated pixels, creating ambiguity during training and&#10;leading to checkerboard and halo artifacts. To overcome this problem, we&#10;propose a feature masking mechanism that reduces the contribution of the&#10;features from the saturated areas. Moreover, we adapt the VGG-based perceptual&#10;loss function to our application to be able to synthesize visually pleasing&#10;textures. Since the number of HDR images for training is limited, we propose to&#10;train our system in two stages. Specifically, we first train our system on a&#10;large number of images for image inpainting task and then fine-tune it on HDR&#10;reconstruction. Since most of the HDR examples contain smooth regions that are&#10;simple to reconstruct, we propose a sampling strategy to select challenging&#10;training patches during the HDR fine-tuning stage. We demonstrate through&#10;experimental results that our approach can reconstruct visually pleasing HDR&#10;results, better than the current state of the art on a wide range of scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.07480" label="1710.07480">
-        <attvalues>
-          <attvalue for="0" value="HDR image reconstruction from a single exposure using deep CNNs" />
-          <attvalue for="1" value="  Camera sensors can only capture a limited range of luminance simultaneously,&#10;and in order to create high dynamic range (HDR) images a set of different&#10;exposures are typically combined. In this paper we address the problem of&#10;predicting information that have been lost in saturated image areas, in order&#10;to enable HDR reconstruction from a single exposure. We show that this problem&#10;is well-suited for deep learning algorithms, and propose a deep convolutional&#10;neural network (CNN) that is specifically designed taking into account the&#10;challenges in predicting HDR values. To train the CNN we gather a large dataset&#10;of HDR images, which we augment by simulating sensor saturation for a range of&#10;cameras. To further boost robustness, we pre-train the CNN on a simulated HDR&#10;dataset created from a subset of the MIT Places database. We demonstrate that&#10;our approach can reconstruct high-resolution visually convincing HDR results in&#10;a wide range of situations, and that it generalizes well to reconstruction of&#10;images captured with arbitrary and low-end cameras that use unknown camera&#10;response functions and post-processing. Furthermore, we compare to existing&#10;methods for HDR expansion, and show high quality results also for image based&#10;lighting. Finally, we evaluate the results in a subjective experiment performed&#10;on an HDR display. This shows that the reconstructed HDR images are visually&#10;convincing, with large improvements as compared to existing methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.15897" label="2210.15897">
-        <attvalues>
-          <attvalue for="0" value="Single-Image HDR Reconstruction by Multi-Exposure Generation" />
-          <attvalue for="1" value="  High dynamic range (HDR) imaging is an indispensable technique in modern&#10;photography. Traditional methods focus on HDR reconstruction from multiple&#10;images, solving the core problems of image alignment, fusion, and tone mapping,&#10;yet having a perfect solution due to ghosting and other visual artifacts in the&#10;reconstruction. Recent attempts at single-image HDR reconstruction show a&#10;promising alternative: by learning to map pixel values to their irradiance&#10;using a neural network, one can bypass the align-and-merge pipeline completely&#10;yet still obtain a high-quality HDR image. In this work, we propose a weakly&#10;supervised learning method that inverts the physical image formation process&#10;for HDR reconstruction via learning to generate multiple exposures from a&#10;single image. Our neural network can invert the camera response to reconstruct&#10;pixel irradiance before synthesizing multiple exposures and hallucinating&#10;details in under- and over-exposed regions from a single input image. To train&#10;the network, we propose a representation loss, a reconstruction loss, and a&#10;perceptual loss applied on pairs of under- and over-exposure images and thus do&#10;not require HDR images for training. Our experiments show that our proposed&#10;model can effectively reconstruct HDR images. Our qualitative and quantitative&#10;results show that our method achieves state-of-the-art performance on the DrTMO&#10;dataset. Our code is available at&#10;https://github.com/VinAIResearch/single_image_hdr.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.02020" label="2309.02020">
-        <attvalues>
-          <attvalue for="0" value="RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image" />
-          <attvalue for="1" value="  High dynamic range (HDR) images capture much more intensity levels than&#10;standard ones. Current methods predominantly generate HDR images from 8-bit low&#10;dynamic range (LDR) sRGB images that have been degraded by the camera&#10;processing pipeline. However, it becomes a formidable task to retrieve&#10;extremely high dynamic range scenes from such limited bit-depth data. Unlike&#10;existing methods, the core idea of this work is to incorporate more informative&#10;Raw sensor data to generate HDR images, aiming to recover scene information in&#10;hard regions (the darkest and brightest areas of an HDR scene). To this end, we&#10;propose a model tailor-made for Raw images, harnessing the unique features of&#10;Raw data to facilitate the Raw-to-HDR mapping. Specifically, we learn exposure&#10;masks to separate the hard and easy regions of a high dynamic scene. Then, we&#10;introduce two important guidances, dual intensity guidance, which guides less&#10;informative channels with more informative ones, and global spatial guidance,&#10;which extrapolates scene specifics over an extended spatial domain. To verify&#10;our Raw-to-HDR approach, we collect a large Raw/HDR paired dataset for both&#10;training and testing. Our empirical evaluations validate the superiority of the&#10;proposed Raw-to-HDR reconstruction model, as well as our newly captured dataset&#10;in the experiments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.11270" label="2211.11270">
-        <attvalues>
-          <attvalue for="0" value="LHDR: HDR Reconstruction for Legacy Content using a Lightweight DNN" />
-          <attvalue for="1" value="  High dynamic range (HDR) image is widely-used in graphics and photography due&#10;to the rich information it contains. Recently the community has started using&#10;deep neural network (DNN) to reconstruct standard dynamic range (SDR) images&#10;into HDR. Albeit the superiority of current DNN-based methods, their&#10;application scenario is still limited: (1) heavy model impedes real-time&#10;processing, and (2) inapplicable to legacy SDR content with more degradation&#10;types. Therefore, we propose a lightweight DNN-based method trained to tackle&#10;legacy SDR. For better design, we reform the problem modeling and emphasize&#10;degradation model. Experiments show that our method reached appealing&#10;performance with minimal computational cost compared with others.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.01628" label="2007.01628">
-        <attvalues>
-          <attvalue for="0" value="HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with&#10;  Large Motions" />
-          <attvalue for="1" value="  Synthesizing high dynamic range (HDR) images from multiple low-dynamic range&#10;(LDR) exposures in dynamic scenes is challenging. There are two major problems&#10;caused by the large motions of foreground objects. One is the severe&#10;misalignment among the LDR images. The other is the missing content due to the&#10;over-/under-saturated regions caused by the moving objects, which may not be&#10;easily compensated for by the multiple LDR exposures. Thus, it requires the HDR&#10;generation model to be able to properly fuse the LDR images and restore the&#10;missing details without introducing artifacts. To address these two problems,&#10;we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR&#10;images from multi-exposed LDR images. To our best knowledge, this work is the&#10;first GAN-based approach for fusing multi-exposed LDR images for HDR&#10;reconstruction. By incorporating adversarial learning, our method is able to&#10;produce faithful information in the regions with missing content. In addition,&#10;we also propose a novel generator network, with a reference-based residual&#10;merging block for aligning large object motions in the feature domain, and a&#10;deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR&#10;images. Experimental results demonstrate that our model achieves&#10;state-of-the-art reconstruction performance over the prior HDR methods on&#10;diverse scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.01840" label="2310.01840">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in&#10;  Dynamic Scenes" />
-          <attvalue for="1" value="  Merging multi-exposure images is a common approach for obtaining high dynamic&#10;range (HDR) images, with the primary challenge being the avoidance of ghosting&#10;artifacts in dynamic scenes. Recent methods have proposed using deep neural&#10;networks for deghosting. However, the methods typically rely on sufficient data&#10;with HDR ground-truths, which are difficult and costly to collect. In this&#10;work, to eliminate the need for labeled data, we propose SelfHDR, a&#10;self-supervised HDR reconstruction method that only requires dynamic&#10;multi-exposure images during training. Specifically, SelfHDR learns a&#10;reconstruction network under the supervision of two complementary components,&#10;which can be constructed from multi-exposure images and focus on HDR color as&#10;well as structure, respectively. The color component is estimated from aligned&#10;multi-exposure images, while the structure one is generated through a&#10;structure-focused network that is supervised by the color component and an&#10;input reference (\eg, medium-exposure) image. During testing, the learned&#10;reconstruction network is directly deployed to predict an HDR image.&#10;Experiments on real-world images demonstrate our SelfHDR achieves superior&#10;results against the state-of-the-art self-supervised methods, and comparable&#10;performance to supervised ones. Codes are available at&#10;https://github.com/cszhilu1998/SelfHDR&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.03827" label="2309.03827">
-        <attvalues>
-          <attvalue for="0" value="ArtHDR-Net: Perceptually Realistic and Accurate HDR Content Creation" />
-          <attvalue for="1" value="  High Dynamic Range (HDR) content creation has become an important topic for&#10;modern media and entertainment sectors, gaming and Augmented/Virtual Reality&#10;industries. Many methods have been proposed to recreate the HDR counterparts of&#10;input Low Dynamic Range (LDR) images/videos given a single exposure or&#10;multi-exposure LDRs. The state-of-the-art methods focus primarily on the&#10;preservation of the reconstruction's structural similarity and the pixel-wise&#10;accuracy. However, these conventional approaches do not emphasize preserving&#10;the artistic intent of the images in terms of human visual perception, which is&#10;an essential element in media, entertainment and gaming. In this paper, we&#10;attempt to study and fill this gap. We propose an architecture called&#10;ArtHDR-Net based on a Convolutional Neural Network that uses multi-exposed LDR&#10;features as input. Experimental results show that ArtHDR-Net can achieve&#10;state-of-the-art performance in terms of the HDR-VDP-2 score (i.e., mean&#10;opinion score index) while reaching competitive performance in terms of PSNR&#10;and SSIM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14451" label="2111.14451">
-        <attvalues>
-          <attvalue for="0" value="HDR-NeRF: High Dynamic Range Neural Radiance Fields" />
-          <attvalue for="1" value="  We present High Dynamic Range Neural Radiance Fields (HDR-NeRF) to recover an&#10;HDR radiance field from a set of low dynamic range (LDR) views with different&#10;exposures. Using the HDR-NeRF, we are able to generate both novel HDR views and&#10;novel LDR views under different exposures. The key to our method is to model&#10;the physical imaging process, which dictates that the radiance of a scene point&#10;transforms to a pixel value in the LDR image with two implicit functions: a&#10;radiance field and a tone mapper. The radiance field encodes the scene radiance&#10;(values vary from 0 to +infty), which outputs the density and radiance of a ray&#10;by giving corresponding ray origin and ray direction. The tone mapper models&#10;the mapping process that a ray hitting on the camera sensor becomes a pixel&#10;value. The color of the ray is predicted by feeding the radiance and the&#10;corresponding exposure time into the tone mapper. We use the classic volume&#10;rendering technique to project the output radiance, colors, and densities into&#10;HDR and LDR images, while only the input LDR images are used as the&#10;supervision. We collect a new forward-facing HDR dataset to evaluate the&#10;proposed method. Experimental results on synthetic and real-world scenes&#10;validate that our method can not only accurately control the exposures of&#10;synthesized views but also render views with a high dynamic range.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Novel View Synthesis. Novel view synthesis aims to generate novel images from a new viewpoint using a set of input views. It is a typical application of image-based rendering technique ~\cite{shum2000review}, such as rendering novel views using depth \cite{zitnick2004high, chaurasia2013depth, zhou2013plane, cayon2015bayesian, overbeck2018system} or explicit geometry information \cite{debevec1996modeling, yu20143d, hedman2017casual, hedman2018instant}. Many classic IBR methods estimate radiance of input images using HDR imaging methods to render novel HDR views \cite{ruckert2021adop, lechlek2019interactive, sharma2012parameterized, lu2010multi}. The estimated radiance using HDR imaging methods is always image-wise. It may be hard to preserve the view consistency in challenging scenes. On the other hand, light field rendering methods interpolate views based on implicit soft geometry estimates derived from densely sampled images \cite{mcmillan1995plenoptic, levoy1996light, gortler1996lumigraph, buehler2001unstructured, davis2012unstructured}.&#10;&#10;In recent years, deep learning techniques have been applied to novel view synthesis to get high-quality photorealistic views. These learning-based approaches can be classified into three categories according to scene representation models. The first category aims to combine the convolutional neural network (CNN) with traditional voxel grid representation \cite{lombardi2019neural,sitzmann2019deepvoxels,chen2020neural}, such that Sitzmann et al. \cite{sitzmann2019deepvoxels} use a CNN to compensate the discretization artifacts from low resolution voxel grids. Lombardi et al. \cite{lombardi2019neural} control the predicted voxel grids based on the input time of dynamic scene. Inspired by the layered depth images \cite{shade1998layered}, other learning-based methods focus on training a CNN to predict a multi-plane images representation from a set of input images and render novel views using alpha-compositing \cite{zhou2018stereo, choi2019extreme, mildenhall2019local,flynn2019deepview}. These methods predict multi-planes images to synthesize views for specific applications, such as light-field rendering \cite{mildenhall2019local} and baseline magnification\cite{zhou2018stereo}. The third category is the NeRF family which represents a scene with a neural radiance field \cite{mildenhall2020nerf,yu2021pixelnerf,martin2021nerf,barron2021mip,boss2021nerd,li2021neural,ma2021deblur,chen2021hallucinated}. Although these recent methods achieved high-quality of rendered novel views, none of them has tackled the task of synthesizing a novel view with {high dynamic range}.&#10;&#10;Neural Implicit Representation. Recently, there has been a surge in representing 3D scenes in implicit functions via a neural network. Compared to traditional explicit representations, such as point cloud \cite{pumarola2020c}, voxels \cite{girdhar2016learning} and octrees \cite{wang2017cnn}, neural implicit representations have shown high-quality view synthesis results such as continuous and high-fidelity. We focus on the neural radiance fields representation that implicitly models the volume densities and colors of the scenes with MLPs \cite{mildenhall2020nerf}. NeRF approximates a continuous 3D function by mapping from an input 5D location to scene properties. Recently, NeRF has been explored for novel view relighting \cite{boss2021nerd,srinivasan2021nerv}, view synthesis for dynamic scenes \cite{li2021neural,xian2021space,pumarola2021d,du2021neural,park2021nerfies,li2021neural3d}, scene editing \cite{guo2020object,zhang2021editable,Yang_2021_ICCV,martin2021nerf}. Particularly, Martin-Brualla et al. \cite{martin2021nerf} propose NeRF-W to build NeRF from internet photo collections with different photometric variations and occlusions. They learn a per-image latent embedding to capture photometric appearance variations in training images, which enable them to modify the lighting and appearance of a rendering. Although various extensions have been explored to NeRF, which enables them to effectively represent the scene radiance captured by cameras. However, all the NeRF based methods ignore the physics process from radiance to pixel values, which hinders them from representing the radiance in the real world.&#10;&#10;High Dynamic Range Imaging. Traditional multiple exposures-based HDR imaging methods reconstruct HDR images by calibrating the CRF from an exposure stack that a series of LDR images under different exposures with a same pose \cite{debevec1997recovering} or directly merge the LDR images into an HDR image \cite{mertens2007exposure}. To overcome the limitations of traditional methods, such as ghosting in the HDR results when LDR images are captured by a hand-held camera or on a dynamic scene, some methods are proposed to detect the motion regions in the LDR images and then remove these regions in the fusion \cite{grosch2006fast,jacobs2008automatic}. In contrast, alignment-based methods align the input multiple LDR images by estimating optical flow then merge the aligned images \cite{tursun2015state,kalantari2017deep,yan2019robust}. Depending on the great potential of deep learning, some methods try to reconstruct an HDR image from a single LDR image \cite{eilertsen2017hdr,khan2019fhdr,kim2021end}. However, most HDR imaging methods require the given LDR images with a fixed or quasi-fixed camera pose. Besides, these methods can only synthesize HDR images with original poses and require ground truth HDR images to supervise.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2111.13679" label="2111.13679">
-        <attvalues>
-          <attvalue for="0" value="NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw&#10;  Images" />
-          <attvalue for="1" value="  Neural Radiance Fields (NeRF) is a technique for high quality novel view&#10;synthesis from a collection of posed input images. Like most view synthesis&#10;methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images&#10;have been processed by a lossy camera pipeline that smooths detail, clips&#10;highlights, and distorts the simple noise distribution of raw sensor data. We&#10;modify NeRF to instead train directly on linear raw images, preserving the&#10;scene's full dynamic range. By rendering raw output images from the resulting&#10;NeRF, we can perform novel high dynamic range (HDR) view synthesis tasks. In&#10;addition to changing the camera viewpoint, we can manipulate focus, exposure,&#10;and tonemapping after the fact. Although a single raw image appears&#10;significantly more noisy than a postprocessed one, we show that NeRF is highly&#10;robust to the zero-mean distribution of raw noise. When optimized over many&#10;noisy raw inputs (25-200), NeRF produces a scene representation so accurate&#10;that its rendered novel views outperform dedicated single and multi-image deep&#10;raw denoisers run on the same wide baseline input images. As a result, our&#10;method, which we call RawNeRF, can reconstruct scenes from extremely noisy&#10;images captured in near-darkness.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.02814" label="2307.02814">
-        <attvalues>
-          <attvalue for="0" value="Single Image LDR to HDR Conversion using Conditional Diffusion" />
-          <attvalue for="1" value="  Digital imaging aims to replicate realistic scenes, but Low Dynamic Range&#10;(LDR) cameras cannot represent the wide dynamic range of real scenes, resulting&#10;in under-/overexposed images. This paper presents a deep learning-based&#10;approach for recovering intricate details from shadows and highlights while&#10;reconstructing High Dynamic Range (HDR) images. We formulate the problem as an&#10;image-to-image (I2I) translation task and propose a conditional Denoising&#10;Diffusion Probabilistic Model (DDPM) based framework using classifier-free&#10;guidance. We incorporate a deep CNN-based autoencoder in our proposed framework&#10;to enhance the quality of the latent representation of the input LDR image used&#10;for conditioning. Moreover, we introduce a new loss function for LDR-HDR&#10;translation tasks, termed Exposure Loss. This loss helps direct gradients in&#10;the opposite direction of the saturation, further improving the results'&#10;quality. By conducting comprehensive quantitative and qualitative experiments,&#10;we have effectively demonstrated the proficiency of our proposed method. The&#10;results indicate that a simple conditional diffusion-based method can replace&#10;the complex camera pipeline-based architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.08318" label="1805.08318">
-        <attvalues>
-          <attvalue for="0" value="Self-Attention Generative Adversarial Networks" />
-          <attvalue for="1" value="  In this paper, we propose the Self-Attention Generative Adversarial Network&#10;(SAGAN) which allows attention-driven, long-range dependency modeling for image&#10;generation tasks. Traditional convolutional GANs generate high-resolution&#10;details as a function of only spatially local points in lower-resolution&#10;feature maps. In SAGAN, details can be generated using cues from all feature&#10;locations. Moreover, the discriminator can check that highly detailed features&#10;in distant portions of the image are consistent with each other. Furthermore,&#10;recent work has shown that generator conditioning affects GAN performance.&#10;Leveraging this insight, we apply spectral normalization to the GAN generator&#10;and find that this improves training dynamics. The proposed SAGAN achieves the&#10;state-of-the-art results, boosting the best published Inception score from 36.8&#10;to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on the&#10;challenging ImageNet dataset. Visualization of the attention layers shows that&#10;the generator leverages neighborhoods that correspond to object shapes rather&#10;than local regions of fixed shape.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.10293" label="1904.10293">
-        <attvalues>
-          <attvalue for="0" value="Attention-guided Network for Ghost-free High Dynamic Range Imaging" />
-          <attvalue for="1" value="  Ghosting artifacts caused by moving objects or misalignments is a key&#10;challenge in high dynamic range (HDR) imaging for dynamic scenes. Previous&#10;methods first register the input low dynamic range (LDR) images using optical&#10;flow before merging them, which are error-prone and cause ghosts in results. A&#10;very recent work tries to bypass optical flows via a deep network with&#10;skip-connections, however, which still suffers from ghosting artifacts for&#10;severe movement. To avoid the ghosting from the source, we propose a novel&#10;attention-guided end-to-end deep neural network (AHDRNet) to produce&#10;high-quality ghost-free HDR images. Unlike previous methods directly stacking&#10;the LDR images or features for merging, we use attention modules to guide the&#10;merging according to the reference image. The attention modules automatically&#10;suppress undesired components caused by misalignments and saturation and&#10;enhance desirable fine details in the non-reference images. In addition to the&#10;attention model, we use dilated residual dense block (DRDB) to make full use of&#10;the hierarchical features and increase the receptive field for hallucinating&#10;the missing details. The proposed AHDRNet is a non-flow-based method, which can&#10;also avoid the artifacts generated by optical-flow estimation error.&#10;Experiments on different datasets show that the proposed AHDRNet can achieve&#10;state-of-the-art quantitative and qualitative results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.08131" label="1910.08131">
-        <attvalues>
-          <attvalue for="0" value="A Dataset of Multi-Illumination Images in the Wild" />
-          <attvalue for="1" value="  Collections of images under a single, uncontrolled illumination have enabled&#10;the rapid advancement of core computer vision tasks like classification,&#10;detection, and segmentation. But even with modern learning techniques, many&#10;inverse problems involving lighting and material understanding remain too&#10;severely ill-posed to be solved with single-illumination datasets. To fill this&#10;gap, we introduce a new multi-illumination dataset of more than 1000 real&#10;scenes, each captured under 25 lighting conditions. We demonstrate the richness&#10;of this dataset by training state-of-the-art models for three challenging&#10;applications: single-image illumination estimation, image relighting, and&#10;mixed-illuminant white balance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Optics, Lighting Estimation, Physics, Image Processing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2407.06247" label="2407.06247">
-        <attvalues>
-          <attvalue for="0" value="Context Propagation from Proposals for Semantic Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  In this paper, we propose a novel approach to learning semantic contextual&#10;relationships in videos for semantic object segmentation. Our algorithm derives&#10;the semantic contexts from video object proposals which encode the key&#10;evolution of objects and the relationship among objects over the&#10;spatio-temporal domain. This semantic contexts are propagated across the video&#10;to estimate the pairwise contexts between all pairs of local superpixels which&#10;are integrated into a conditional random field in the form of pairwise&#10;potentials and infers the per-superpixel semantic labels. The experiments&#10;demonstrate that our contexts learning and propagation model effectively&#10;improves the robustness of resolving visual ambiguities in semantic video&#10;object segmentation compared with the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Semantic video object segmentation, which aims to assign a semantic label to every pixel in video frames, is an essential step in various computer vision and multimedia analysis tasks. Recent years have witnessed significant attention and progress toward this problem. However, in addition to fast motion, appearance variations, pose change, and occlusions, the difficulty in resolving the inherent semantic ambiguities still plagues the robustness and accuracy of such a large scale semantic labeling problem. &#10;&#10;Recently, weakly supervised methods utilizing robust detection and tracking approach have been proposed to address the semantic ambiguity issue. In the learning process, object-relevant instances \ie object proposals sharing the same semantic tags are usually selected from video frames to learn category or instance specific models for semantic labeling. Early work trained classifiers to incorporate scene topology and semantics into pixel-level object detection and localization \cite{taylor2013semantic}. Both object detector and tracker were employed to either impose spatio-temporal coherence \cite{ZhangCLWX15,drayer2016object} or learn an appearance model \cite{wang2016semi} for encoding the appearance variation of semantic objects. Lately, video object proposal selection algorithm has also been proposed to integrate longer-range object reasoning with superpixel labeling \cite{wang2017a}. Despite of significant advances that have been made by the above methods, global contextual relationships between semantic video objects remain under-explored. Yet, contextual relationships are ubiquitous and provide important&#10;cues for scene understanding related tasks. &#10;&#10;Pairwise contextual relationships have been investigated in image segmentation \cite{lin2016efficient} and object detection \cite{gidaris2015object,bai2017multiple} tasks, whose importance has been highlighted in effectively resolving semantic ambiguities. However, these methods model the contextual relationships in terms of co-occurrence of higher-level statistics of object categories, which favors frequently appeared classes in the training data to enforce rigid semantic labelling. Small objects are more likely to be omitted due to the sensitivity to the number of pixels that objects occupy.&#10;&#10;Graphical models have become powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, providing a versatile framework for modeling contextual relationships. These approaches exploit the inherent structure and relationships within images \cite{wang2015robust,tinghuai2016method,xing2021learning} and videos \cite{wang2010video,wang2014wide,chen2020fine}, facilitating more sophisticated and context-aware analysis. By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}.&#10;&#10;In this work, we propose a novel model to exploit and propagate contextual relationships among video object proposals without relying on training data. Such a way of modeling spatio-temporal object contextual relationships has not been well studied. Our model is able to capture the intra- and inter- category contextual relationships by considering the content of an input video in a non-parametric approach. This context model is comprised of a set of spatial-temporal context exemplars which provide a novel interpretation of contextual relationships in a link view which formulates the problem of learning contextual relationships as the label propagation problem on a similarity graph. This similarity graph naturally reflects the intrinsic and extrinsic relationship between semantic objects in the spatial-temporal domain. Due to the sparsity of this similarity graph, the learning process can be very efficient. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Semantic Context Learning, Mathematics, Artificial Intelligence, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1608.03066" label="1608.03066">
-        <attvalues>
-          <attvalue for="0" value="Object Detection, Tracking, and Motion Segmentation for Object-level&#10;  Video Segmentation" />
-          <attvalue for="1" value="  We present an approach for object segmentation in videos that combines&#10;frame-level object detection with concepts from object tracking and motion&#10;segmentation. The approach extracts temporally consistent object tubes based on&#10;an off-the-shelf detector. Besides the class label for each tube, this provides&#10;a location prior that is independent of motion. For the final video&#10;segmentation, we combine this information with motion cues. The method&#10;overcomes the typical problems of weakly supervised/unsupervised video&#10;segmentation, such as scenes with no motion, dominant camera motion, and&#10;objects that move as a unit. In contrast to most tracking methods, it provides&#10;an accurate, temporally consistent segmentation of each object. We report&#10;results on four video segmentation datasets: YouTube Objects, SegTrackv2,&#10;egoMotion, and FBMS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1504.01013" label="1504.01013">
-        <attvalues>
-          <attvalue for="0" value="Efficient piecewise training of deep structured models for semantic&#10;  segmentation" />
-          <attvalue for="1" value="  Recent advances in semantic image segmentation have mostly been achieved by&#10;training deep convolutional neural networks (CNNs). We show how to improve&#10;semantic segmentation through the use of contextual information; specifically,&#10;we explore `patch-patch' context between image regions, and `patch-background'&#10;context. For learning from the patch-patch context, we formulate Conditional&#10;Random Fields (CRFs) with CNN-based pairwise potential functions to capture&#10;semantic correlations between neighboring patches. Efficient piecewise training&#10;of the proposed deep structured model is then applied to avoid repeated&#10;expensive CRF inference for back propagation. For capturing the&#10;patch-background context, we show that a network design with traditional&#10;multi-scale image input and sliding pyramid pooling is effective for improving&#10;performance. Our experimental results set new state-of-the-art performance on a&#10;number of popular semantic segmentation datasets, including NYUDv2, PASCAL VOC&#10;2012, PASCAL-Context, and SIFT-flow. In particular, we achieve an&#10;intersection-over-union score of 78.0 on the challenging PASCAL VOC 2012&#10;dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.00138" label="1704.00138">
-        <attvalues>
-          <attvalue for="0" value="Multiple Instance Detection Network with Online Instance Classifier&#10;  Refinement" />
-          <attvalue for="1" value="  Of late, weakly supervised object detection is with great importance in&#10;object recognition. Based on deep learning, weakly supervised detectors have&#10;achieved many promising results. However, compared with fully supervised&#10;detection, it is more challenging to train deep network based detectors in a&#10;weakly supervised manner. Here we formulate weakly supervised detection as a&#10;Multiple Instance Learning (MIL) problem, where instance classifiers (object&#10;detectors) are put into the network as hidden nodes. We propose a novel online&#10;instance classifier refinement algorithm to integrate MIL and the instance&#10;classifier refinement procedure into a single deep network, and train the&#10;network end-to-end with only image-level supervision, i.e., without object&#10;location information. More precisely, instance labels inferred from weak&#10;supervision are propagated to their spatially overlapped instances to refine&#10;instance classifier online. The iterative instance classifier refinement&#10;procedure is implemented using multiple streams in deep network, where each&#10;stream supervises its latter stream. Weakly supervised object detection&#10;experiments are carried out on the challenging PASCAL VOC 2007 and 2012&#10;benchmarks. We obtain 47% mAP on VOC 2007 that significantly outperforms the&#10;previous state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.13992" label="2110.13992">
-        <attvalues>
-          <attvalue for="0" value="Leveraging Local Temporal Information for Multimodal Scene&#10;  Classification" />
-          <attvalue for="1" value="  Robust video scene classification models should capture the spatial&#10;(pixel-wise) and temporal (frame-wise) characteristics of a video effectively.&#10;Transformer models with self-attention which are designed to get contextualized&#10;representations for individual tokens given a sequence of tokens, are becoming&#10;increasingly popular in many computer vision tasks. However, the use of&#10;Transformer based models for video understanding is still relatively&#10;unexplored. Moreover, these models fail to exploit the strong temporal&#10;relationships between the neighboring video frames to get potent frame-level&#10;representations. In this paper, we propose a novel self-attention block that&#10;leverages both local and global temporal relationships between the video frames&#10;to obtain better contextualized representations for the individual frames. This&#10;enables the model to understand the video at various granularities. We&#10;illustrate the performance of our models on the large scale YoutTube-8M data&#10;set on the task of video categorization and further analyze the results to&#10;showcase improvement.&#10;" />
-          <attvalue for="2" value="&#10;The goal of video classification is to understand the visual and audio features to assign one or more relevant tags to the video. Such a task is relevant for applications such as content retrieval, mitigate missing video metadata, user profiling and content recommendation. 3D convolutional neural-nets (CNNs) have been used extensively for video classification tasks~\cite{feichtenhofer2020x3d, tran2019video, lin2019tsm}. However, their computational inefficiency has led the researchers to look for alternatives such as self-attention based Transformer architectures~\cite{vaswani2017attention}. Recently, ~\cite{dosovitskiy2020image} showed that Transformer based models train faster than their CNN counterparts, leading to enhanced scalability. Transformer based models also capture long-range dependencies which benefits many natural language processing (NLP) tasks ~\cite{xia2019tied, li2019beyond}. Capturing long-term dependencies across frames is crucial for video understanding as has been demonstrated by~\cite{wang2018non, wu2019long}. This has led to growing interest in using Transformer based models for video classification. &#10;&#10;Chen et al.~\cite{chen20182} proposed increasing the efficacy of convolutional blocks for action classification by using a two-stage attention mechanism to collect relevant features from different patches of video frames at various instants. Kmiec et al.~\cite{kmiec2018learnable} used Transformer encoder along with NetVLAD blocks to get effective multimodal feature representations for large scale video understanding. Girdhar et al.~\cite{girdhar2019video} proposed ‘action Transformers’ where features extracted using 3D CNN are aggregated and fed to a self-attention block to leverage the spatio-temporal information around a person for action localization. Bertasius et al.~\cite{bertasius2021space} extended the idea of~\cite{dosovitskiy2020image} to propose Timesformer for video classification by using self-attention blocks to obtain spatiotemporal contextualized feature representations from a sequence of frame-level patches. These works show the ability of Transformer based models to generate video representations capturing the global context pertinent for the task at hand. While some of them exploit the spatial correlation between the pixels, none of them leverage the strong correlation between consecutive frames to obtain the contextualized output representations.&#10;&#10;We hypothesize that utilizing the strong temporal correlation existing within consecutive frames is crucial for video understanding. By focusing attention on just a portion of a video at any time, one can obtain more fine-grained information which can help with it's classification. Owing to the sequential nature of text, the importance of local attention maps in Transformer based models have been studied in NLP domain.~\cite{pande2020importance, child2019generating, beltagy2020longformer} have explored the importance of local attention maps for various language understanding tasks and proposed computationally efficient Transformer models. The local attention maps are created by enforcing a token to attend to other tokens only in a small neighbourhood around it by masking out tokens outside the neighborhood. In this paper, we explore various masks to compute different types local attention maps useful for video classification. We further propose to combine the global and local attention maps using gating mechanisms~\cite{shazeer2017outrageously}. Recent works have explored gating in Transformer based models~\cite{chai2020highway, fedus2021switch}, but they are not motivated by gating in attention space. Finally, we perform large scale experiments on the Youtube8M dataset~\cite{abu2016youtube} and show the advantage of our model over the baseline self-attention based model for video classification. We show that our approach can alleviate the effects of noisy frames in a video, by adding the information from local attention maps and correcting the baseline model's prediction.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Transformer Models, Video Scene Classification, Mathematics, Artificial Intelligence, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2004.04730" label="2004.04730">
-        <attvalues>
-          <attvalue for="0" value="X3D: Expanding Architectures for Efficient Video Recognition" />
-          <attvalue for="1" value="  This paper presents X3D, a family of efficient video networks that&#10;progressively expand a tiny 2D image classification architecture along multiple&#10;network axes, in space, time, width and depth. Inspired by feature selection&#10;methods in machine learning, a simple stepwise network expansion approach is&#10;employed that expands a single axis in each step, such that good accuracy to&#10;complexity trade-off is achieved. To expand X3D to a specific target&#10;complexity, we perform progressive forward expansion followed by backward&#10;contraction. X3D achieves state-of-the-art performance while requiring 4.8x and&#10;5.5x fewer multiply-adds and parameters for similar accuracy as previous work.&#10;Our most surprising finding is that networks with high spatiotemporal&#10;resolution can perform well, while being extremely light in terms of network&#10;width and parameters. We report competitive accuracy at unprecedented&#10;efficiency on video classification and detection benchmarks. Code will be&#10;available at: https://github.com/facebookresearch/SlowFast&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.00530" label="1810.00530">
-        <attvalues>
-          <attvalue for="0" value="Learnable Pooling Methods for Video Classification" />
-          <attvalue for="1" value="  We introduce modifications to state-of-the-art approaches to aggregating&#10;local video descriptors by using attention mechanisms and function&#10;approximations. Rather than using ensembles of existing architectures, we&#10;provide an insight on creating new architectures. We demonstrate our solutions&#10;in the &quot;The 2nd YouTube-8M Video Understanding Challenge&quot;, by using frame-level&#10;video and audio descriptors. We obtain testing accuracy similar to the state of&#10;the art, while meeting budget constraints, and touch upon strategies to improve&#10;the state of the art. Model implementations are available in&#10;https://github.com/pomonam/LearnablePoolingMethods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.02707" label="1812.02707">
-        <attvalues>
-          <attvalue for="0" value="Video Action Transformer Network" />
-          <attvalue for="1" value="  We introduce the Action Transformer model for recognizing and localizing&#10;human actions in video clips. We repurpose a Transformer-style architecture to&#10;aggregate features from the spatiotemporal context around the person whose&#10;actions we are trying to classify. We show that by using high-resolution,&#10;person-specific, class-agnostic queries, the model spontaneously learns to&#10;track individual people and to pick up on semantic context from the actions of&#10;others. Additionally its attention mechanism learns to emphasize hands and&#10;faces, which are often crucial to discriminate an action - all without explicit&#10;supervision other than boxes and class labels. We train and test our Action&#10;Transformer network on the Atomic Visual Actions (AVA) dataset, outperforming&#10;the state-of-the-art by a significant margin using only raw RGB frames as&#10;input.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.05828" label="2008.05828">
-        <attvalues>
-          <attvalue for="0" value="On the Importance of Local Information in Transformer Based Models" />
-          <attvalue for="1" value="  The self-attention module is a key component of Transformer-based models,&#10;wherein each token pays attention to every other token. Recent studies have&#10;shown that these heads exhibit syntactic, semantic, or local behaviour. Some&#10;studies have also identified promise in restricting this attention to be local,&#10;i.e., a token attending to other tokens only in a small neighbourhood around&#10;it. However, no conclusive evidence exists that such local attention alone is&#10;sufficient to achieve high accuracy on multiple NLP tasks. In this work, we&#10;systematically analyse the role of locality information in learnt models and&#10;contrast it with the role of syntactic information. More specifically, we first&#10;do a sensitivity analysis and show that, at every layer, the representation of&#10;a token is much more sensitive to tokens in a small neighborhood around it than&#10;to tokens which are syntactically related to it. We then define an attention&#10;bias metric to determine whether a head pays more attention to local tokens or&#10;to syntactically related tokens. We show that a larger fraction of heads have a&#10;locality bias as compared to a syntactic bias. Having established the&#10;importance of local attention heads, we train and evaluate models where varying&#10;fractions of the attention heads are constrained to be local. Such models would&#10;be more efficient as they would have fewer computations in the attention layer.&#10;We evaluate these models on 4 GLUE datasets (QQP, SST-2, MRPC, QNLI) and 2 MT&#10;datasets (En-De, En-Ru) and clearly demonstrate that such constrained models&#10;have comparable performance to the unconstrained models. Through this&#10;systematic evaluation we establish that attention in Transformer-based models&#10;can be constrained to be local without affecting performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.06538" label="1701.06538">
-        <attvalues>
-          <attvalue for="0" value="Outrageously Large Neural Networks: The Sparsely-Gated&#10;  Mixture-of-Experts Layer" />
-          <attvalue for="1" value="  The capacity of a neural network to absorb information is limited by its&#10;number of parameters. Conditional computation, where parts of the network are&#10;active on a per-example basis, has been proposed in theory as a way of&#10;dramatically increasing model capacity without a proportional increase in&#10;computation. In practice, however, there are significant algorithmic and&#10;performance challenges. In this work, we address these challenges and finally&#10;realize the promise of conditional computation, achieving greater than 1000x&#10;improvements in model capacity with only minor losses in computational&#10;efficiency on modern GPU clusters. We introduce a Sparsely-Gated&#10;Mixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward&#10;sub-networks. A trainable gating network determines a sparse combination of&#10;these experts to use for each example. We apply the MoE to the tasks of&#10;language modeling and machine translation, where model capacity is critical for&#10;absorbing the vast quantities of knowledge available in the training corpora.&#10;We present model architectures in which a MoE with up to 137 billion parameters&#10;is applied convolutionally between stacked LSTM layers. On large language&#10;modeling and machine translation benchmarks, these models achieve significantly&#10;better results than state-of-the-art at lower computational cost.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.08178" label="2004.08178">
-        <attvalues>
-          <attvalue for="0" value="Highway Transformer: Self-Gating Enhanced Self-Attentive Networks" />
-          <attvalue for="1" value="  Self-attention mechanisms have made striking state-of-the-art (SOTA) progress&#10;in various sequence learning tasks, standing on the multi-headed dot product&#10;attention by attending to all the global contexts at different locations.&#10;Through a pseudo information highway, we introduce a gated component&#10;self-dependency units (SDU) that incorporates LSTM-styled gating units to&#10;replenish internal semantic importance within the multi-dimensional latent&#10;space of individual representations. The subsidiary content-based SDU gates&#10;allow for the information flow of modulated latent embeddings through skipped&#10;connections, leading to a clear margin of convergence speed with gradient&#10;descent algorithms. We may unveil the role of gating mechanism to aid in the&#10;context-based Transformer modules, with hypothesizing that SDU gates,&#10;especially on shallow layers, could push it faster to step towards suboptimal&#10;points during the optimization process.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.08675" label="1609.08675">
-        <attvalues>
-          <attvalue for="0" value="YouTube-8M: A Large-Scale Video Classification Benchmark" />
-          <attvalue for="1" value="  Many recent advancements in Computer Vision are attributed to large datasets.&#10;Open-source software packages for Machine Learning and inexpensive commodity&#10;hardware have reduced the barrier of entry for exploring novel approaches at&#10;scale. It is possible to train models over millions of examples within a few&#10;days. Although large-scale datasets exist for image understanding, such as&#10;ImageNet, there are no comparable size video classification datasets.&#10;  In this paper, we introduce YouTube-8M, the largest multi-label video&#10;classification dataset, composed of ~8 million videos (500K hours of video),&#10;annotated with a vocabulary of 4800 visual entities. To get the videos and&#10;their labels, we used a YouTube video annotation system, which labels videos&#10;with their main topics. While the labels are machine-generated, they have&#10;high-precision and are derived from a variety of human-based signals including&#10;metadata and query click signals. We filtered the video labels (Knowledge Graph&#10;entities) using both automated and manual curation strategies, including asking&#10;human raters if the labels are visually recognizable. Then, we decoded each&#10;video at one-frame-per-second, and used a Deep CNN pre-trained on ImageNet to&#10;extract the hidden representation immediately prior to the classification&#10;layer. Finally, we compressed the frame features and make both the features and&#10;video-level labels available for download.&#10;  We trained various (modest) classification models on the dataset, evaluated&#10;them using popular evaluation metrics, and report them as baselines. Despite&#10;the size of the dataset, some of our models train to convergence in less than a&#10;day on a single machine using TensorFlow. We plan to release code for training&#10;a TensorFlow model and for computing metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04253" label="2003.04253">
-        <attvalues>
-          <attvalue for="0" value="Motion-Attentive Transition for Zero-Shot Video Object Segmentation" />
-          <attvalue for="1" value="  In this paper, we present a novel Motion-Attentive Transition Network&#10;(MATNet) for zero-shot video object segmentation, which provides a new way of&#10;leveraging motion information to reinforce spatio-temporal object&#10;representation. An asymmetric attention block, called Motion-Attentive&#10;Transition (MAT), is designed within a two-stream encoder, which transforms&#10;appearance features into motion-attentive representations at each convolutional&#10;stage. In this way, the encoder becomes deeply interleaved, allowing for&#10;closely hierarchical interactions between object motion and appearance. This is&#10;superior to the typical two-stream architecture, which treats motion and&#10;appearance separately in each stream and often suffers from overfitting to&#10;appearance information. Additionally, a bridge network is proposed to obtain a&#10;compact, discriminative and scale-sensitive representation for multi-level&#10;encoder features, which is further fed into a decoder to achieve segmentation&#10;results. Extensive experiments on three challenging public benchmarks (i.e.&#10;DAVIS-16, FBMS and Youtube-Objects) show that our model achieves compelling&#10;performance against the state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Automatic Video Object Segmentation.&#10;Automatic, or unsupervised, video object segmentation aims to segment conspicuous and eye-catching objects without any human intervention. Traditional methods require no training data and typically design heuristic assumptions (e.g. motion boundary~\cite{papazoglou2013fast}, objectness~\cite{faktor2014video,zhou2016video,li2017learning}, saliency~\cite{wang2015saliency} and long-term point trajectories~\cite{ochs2013segmentation,keuper2015motion,ochs2011object}) for segmentation. In recent years, benefitting from the establishment of large datasets~\cite{perazzi2016benchmark,xu2018youtube}, many approaches~\cite{jain2017fusionseg,tokmakov2017learningmotion,li2018unsupervised,fan2018salient,lu2019see,hu2018unsupervised,Wang_2019_CVPR,ventura2019rvos,tokmakov2019learning,li2018instance,li2018unsupervised,song2018pyramid,faisal2019exploiting,fan2019shifting} propose to solve this task with zero-shot solutions and improve the performance greatly.&#10;&#10;Among them, a large number of approaches utilize motion because of its complementary role to object appearance. They typically adopt heuristic methods to fuse motion and appearance cues~\cite{tokmakov2017learningmotion,li2018unsupervised} or use two-stream networks~\cite{jain2017fusionseg,tokmakov2017learning} to learn spatio-temporal representations in an end-to-end fashion. However, a major drawback of these approaches is that they fail to consider the importance of deep interactions between appearance and motion in learning rich spatio-temporal features. To address this issue, we propose a deep interleaved two-stream encoder, in which a motion transition module is leveraged for more effective representation learning.&#10;&#10;Neural Attention.&#10;Neural attention has been widely used in recent neural networks for various tasks, such as object recognition~\cite{hu2018squeeze,woo2018cbam,xie2019attentive}, re-identification~\cite{zhou2018aware}, visual saliency~\cite{wang2019salient} and medical imaging~\cite{zhou2019collaborative}. It allows the networks to focus on the most informative parts of the inputs. In this work, neural attention is used in two ways: first, in the encoder network, soft attention is applied independently to intermediate appearance or motion feature maps, and motion attention is further transferred to enhance the appearance attention. Second, in the bridge network, a scale-sensitive attention module is designed to obtain more compact features.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Architectures, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2306.11414" label="2306.11414">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale Occ: 4th Place Solution for CVPR 2023 3D Occupancy&#10;  Prediction Challenge" />
-          <attvalue for="1" value="  In this report, we present the 4th place solution for CVPR 2023 3D occupancy&#10;prediction challenge. We propose a simple method called Multi-Scale Occ for&#10;occupancy prediction based on lift-splat-shoot framework, which introduces&#10;multi-scale image features for generating better multi-scale 3D voxel features&#10;with temporal fusion of multiple past frames. Post-processing including model&#10;ensemble, test-time augmentation, and class-wise thresh are adopted to further&#10;boost the final performance. As shown on the leaderboard, our proposed&#10;occupancy prediction method ranks the 4th place with 49.36 mIoU.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Recently, 3D occupancy prediction has attracted extensive attention as it is crucial for autonomous driving systems to understand geometric and semantic information in 3D scenes. 3D occupancy is more accurate and suitable for describing objects in arbitrary shape and undefined classes at a fine-grained level. 3D occupancy prediction may be performed using various modality input (e.g. LiDAR, Radar, Image). In the challenge, we focus on predicting 3D occupancy from pure multi-camera images which still remains as a challenging problem.&#10;&#10;Based on previous perception tasks including 3D object detection and semantic map segmentation, current multi-camera 3D perception methods mainly lies in two types: 1) lift-splat-shoot(LSS)-based&#10; \cite{huang2022bevdet4d,huang2021bevdet,philion2020lift}, lifting 2D image features to plausible 3D volume space via implicit or explicit depth estimation. 2) Transformer-based&#10; \cite{wei2023surroundocc,li2023voxformer,li2022bevformer}, which define 3D queries in 3D volume space and use transformer attention mechanism to query corresponding 2D image features. We build our method based on LSS-style framework, improved with long-term temporal stereo matching and multi-scale 3D features fusion to learn spatial and temporal details simultaneously. Besides, we use decoupled head to perform occupancy and semantic prediction separately in order to relieve the extreme imbalanced distribution between unoccupied voxel grids and those grids occupied by semantic classes.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Temporal Fusion Techniques, Mathematics, Artificial Intelligence, Computer Science, Multi-Scale Image Features" />
-        </attvalues>
-      </node>
-      <node id="2203.17054" label="2203.17054">
-        <attvalues>
-          <attvalue for="0" value="BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection" />
-          <attvalue for="1" value="  Single frame data contains finite information which limits the performance of&#10;the existing vision-based multi-camera 3D object detection paradigms. For&#10;fundamentally pushing the performance boundary in this area, a novel paradigm&#10;dubbed BEVDet4D is proposed to lift the scalable BEVDet paradigm from the&#10;spatial-only 3D space to the spatial-temporal 4D space. We upgrade the naive&#10;BEVDet framework with a few modifications just for fusing the feature from the&#10;previous frame with the corresponding one in the current frame. In this way,&#10;with negligible additional computing budget, we enable BEVDet4D to access the&#10;temporal cues by querying and comparing the two candidate features. Beyond&#10;this, we simplify the task of velocity prediction by removing the factors of&#10;ego-motion and time in the learning target. As a result, BEVDet4D with robust&#10;generalization performance reduces the velocity error by up to -62.9%. This&#10;makes the vision-based methods, for the first time, become comparable with&#10;those relied on LiDAR or radar in this aspect. On challenge benchmark nuScenes,&#10;we report a new record of 54.5% NDS with the high-performance configuration&#10;dubbed BEVDet4D-Base, which surpasses the previous leading method BEVDet-Base&#10;by +7.3% NDS. The source code is publicly available for further research at&#10;https://github.com/HuangJunJie2017/BEVDet .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.11790" label="2112.11790">
-        <attvalues>
-          <attvalue for="0" value="BEVDet: High-performance Multi-camera 3D Object Detection in&#10;  Bird-Eye-View" />
-          <attvalue for="1" value="  Autonomous driving perceives its surroundings for decision making, which is&#10;one of the most complex scenarios in visual perception. The success of paradigm&#10;innovation in solving the 2D object detection task inspires us to seek an&#10;elegant, feasible, and scalable paradigm for fundamentally pushing the&#10;performance boundary in this area. To this end, we contribute the BEVDet&#10;paradigm in this paper. BEVDet performs 3D object detection in Bird-Eye-View&#10;(BEV), where most target values are defined and route planning can be handily&#10;performed. We merely reuse existing modules to build its framework but&#10;substantially develop its performance by constructing an exclusive data&#10;augmentation strategy and upgrading the Non-Maximum Suppression strategy. In&#10;the experiment, BEVDet offers an excellent trade-off between accuracy and&#10;time-efficiency. As a fast version, BEVDet-Tiny scores 31.2% mAP and 39.2% NDS&#10;on the nuScenes val set. It is comparable with FCOS3D, but requires just 11%&#10;computational budget of 215.3 GFLOPs and runs 9.2 times faster at 15.6 FPS.&#10;Another high-precision version dubbed BEVDet-Base scores 39.3% mAP and 47.2%&#10;NDS, significantly exceeding all published results. With a comparable inference&#10;speed, it surpasses FCOS3D by a large margin of +9.8% mAP and +10.0% NDS. The&#10;source code is publicly available for further research at&#10;https://github.com/HuangJunJie2017/BEVDet .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.05711" label="2008.05711">
-        <attvalues>
-          <attvalue for="0" value="Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by&#10;  Implicitly Unprojecting to 3D" />
-          <attvalue for="1" value="  The goal of perception for autonomous vehicles is to extract semantic&#10;representations from multiple sensors and fuse these representations into a&#10;single &quot;bird's-eye-view&quot; coordinate frame for consumption by motion planning.&#10;We propose a new end-to-end architecture that directly extracts a&#10;bird's-eye-view representation of a scene given image data from an arbitrary&#10;number of cameras. The core idea behind our approach is to &quot;lift&quot; each image&#10;individually into a frustum of features for each camera, then &quot;splat&quot; all&#10;frustums into a rasterized bird's-eye-view grid. By training on the entire&#10;camera rig, we provide evidence that our model is able to learn not only how to&#10;represent images but how to fuse predictions from all cameras into a single&#10;cohesive representation of the scene while being robust to calibration error.&#10;On standard bird's-eye-view tasks such as object segmentation and map&#10;segmentation, our model outperforms all baselines and prior work. In pursuit of&#10;the goal of learning dense representations for motion planning, we show that&#10;the representations inferred by our model enable interpretable end-to-end&#10;motion planning by &quot;shooting&quot; template trajectories into a bird's-eye-view cost&#10;map output by our network. We benchmark our approach against models that use&#10;oracle depth from lidar. Project page with code:&#10;https://nv-tlabs.github.io/lift-splat-shoot .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.09551" label="2303.09551">
-        <attvalues>
-          <attvalue for="0" value="SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving" />
-          <attvalue for="1" value="  3D scene understanding plays a vital role in vision-based autonomous driving.&#10;While most existing methods focus on 3D object detection, they have difficulty&#10;describing real-world objects of arbitrary shapes and infinite classes. Towards&#10;a more comprehensive perception of a 3D scene, in this paper, we propose a&#10;SurroundOcc method to predict the 3D occupancy with multi-camera images. We&#10;first extract multi-scale features for each image and adopt spatial 2D-3D&#10;attention to lift them to the 3D volume space. Then we apply 3D convolutions to&#10;progressively upsample the volume features and impose supervision on multiple&#10;levels. To obtain dense occupancy prediction, we design a pipeline to generate&#10;dense occupancy ground truth without expansive occupancy annotations.&#10;Specifically, we fuse multi-frame LiDAR scans of dynamic objects and static&#10;scenes separately. Then we adopt Poisson Reconstruction to fill the holes and&#10;voxelize the mesh to get dense occupancy labels. Extensive experiments on&#10;nuScenes and SemanticKITTI datasets demonstrate the superiority of our method.&#10;Code and dataset are available at https://github.com/weiyithu/SurroundOcc&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.12251" label="2302.12251">
-        <attvalues>
-          <attvalue for="0" value="VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene&#10;  Completion" />
-          <attvalue for="1" value="  Humans can easily imagine the complete 3D geometry of occluded objects and&#10;scenes. This appealing ability is vital for recognition and understanding. To&#10;enable such capability in AI systems, we propose VoxFormer, a Transformer-based&#10;semantic scene completion framework that can output complete 3D volumetric&#10;semantics from only 2D images. Our framework adopts a two-stage design where we&#10;start from a sparse set of visible and occupied voxel queries from depth&#10;estimation, followed by a densification stage that generates dense 3D voxels&#10;from the sparse ones. A key idea of this design is that the visual features on&#10;2D images correspond only to the visible scene structures rather than the&#10;occluded or empty spaces. Therefore, starting with the featurization and&#10;prediction of the visible structures is more reliable. Once we obtain the set&#10;of sparse queries, we apply a masked autoencoder design to propagate the&#10;information to all the voxels by self-attention. Experiments on SemanticKITTI&#10;show that VoxFormer outperforms the state of the art with a relative&#10;improvement of 20.0% in geometry and 18.1% in semantics and reduces GPU memory&#10;during training to less than 16GB. Our code is available on&#10;https://github.com/NVlabs/VoxFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.11251" label="2101.11251">
-        <attvalues>
-          <attvalue for="0" value="e-ACJ: Accurate Junction Extraction For Event Cameras" />
-          <attvalue for="1" value="  Junctions reflect the important geometrical structure information of the&#10;image, and are of primary significance to applications such as image matching&#10;and motion analysis. Previous event-based feature extraction methods are mainly&#10;focused on corners, which mainly find their locations, however, ignoring the&#10;geometrical structure information like orientations and scales of edges. This&#10;paper adapts the frame-based a-contrario junction detector(ACJ) to event data,&#10;proposing the event-based a-contrario junction detector(e-ACJ), which yields&#10;junctions' locations while giving the scales and orientations of their&#10;branches. The proposed method relies on an a-contrario model and can operate on&#10;asynchronous events directly without generating synthesized event frames. We&#10;evaluate the performance on public event datasets. The result shows our method&#10;successfully finds the orientations and scales of branches, while maintaining&#10;high accuracy in junction's location.&#10;" />
-          <attvalue for="2" value="&#10;Junctions are of primary importance for visual perception and scene understanding. They are usually classified into L-,Y-(or T-) and X-junctions. Junctions are usually used as important keys for various computer vision tasks, such as Simultaneous Localisation And Mapping (SLAM) \cite{slam}, image segmentation, figure/ground separation \cite{ren2006figure} \cite{dimiccoli2009exploiting}, and many others \cite{lin2007empirical} \cite{shen2000uncertainty}.&#10;&#10;The detection of junction features in frame-based images has been studied for many years \cite{maire2008using} \cite{puspoki2015design}. The scaling of junctions for recognition purposes has also been studied \cite{mikolajczyk2004scale}. These methods determine the scale of the points of interest in a localized area, which significantly reduces the accuracy. In addition, these methods focus mainly on the localization and scale of corner points and ignore the differences between different types of junctions.&#10;To overcome these shortcomings, \cite{xia2014accurate} proposed the ACJ detector to detect and characterize different junctions with non-linearly scaled points. In the ACJ detector, to determine the position and bifurcation of junctions, the a-contrario are applied to eliminate the effect of noise and to obtain accurate results. &#10;On the other hand, the typical junction detection process in a conventional camera is not always optimal for robot tasks, because many images contain the same redundant information when they are not moving, resulting in a large waste of computational resources. Furthermore, robots are challenged to detect junctions in the dark.&#10;To overcome the challenges of high speed, dynamic range, and latency, event cameras provide a wealth of information encoded in event streams for AR/VR and robotics. Unlike traditional frame-based cameras, event cameras respond to pixel-by-pixel brightness changes with high temporal resolution and transmit events asynchronously with very low latency. They also have a high dynamic range and are more resistant to motion blur. Applying the event camera as a detection junction to a robotic task also solves the problem of wasting computational resources, since the event camera does not generate new data for the same image (when there is no change in brightness).&#10;&#10;We proposed a method combining event camera and frame-based ACJ detector ({e-ACJ}). In the meantime, the detection method of ACJ is improved for the special nature of event camera, and Arc$^*$ algorithm \cite{alzugaray2018asynchronous} is introduced as well to accelerate the original ACJ detection speed. We also applies techniques such as a-contrario and binarization to the event camera data. The results show that the accuracy of e-ACJ detection is better than most event camera corner detection algorithms, such as eHarris \cite{eharris}, eFAST \cite{efast} and FA-Harris \cite{li2019fa}.&#10;In the following section, we address the ACJ detector algorithm based on event camera data (e-ACJ). Section \ref{sec:algorithm} presents the details and implementation of the proposed e-ACJ detector. Section \ref{sec:experiment} gives the method for the collection of ground truth event-junctions, the evaluational methods and gives the experimental results. Finally, conclusions are drawn in section\ref{sec:conclusions}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Geometrical Structure Analysis, Computer Science, Image Feature Extraction, Event-Based Vision, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1906.10925" label="1906.10925">
-        <attvalues>
-          <attvalue for="0" value="FA-Harris: A Fast and Asynchronous Corner Detector for Event Cameras" />
-          <attvalue for="1" value="  Recently, the emerging bio-inspired event cameras have demonstrated&#10;potentials for a wide range of robotic applications in dynamic environments. In&#10;this paper, we propose a novel fast and asynchronous event-based corner&#10;detection method which is called FA-Harris. FA-Harris consists of several&#10;components, including an event filter, a Global Surface of Active Events&#10;(G-SAE) maintaining unit, a corner candidate selecting unit, and a corner&#10;candidate refining unit. The proposed G-SAE maintenance algorithm and corner&#10;candidate selection algorithm greatly enhance the real-time performance for&#10;corner detection, while the corner candidate refinement algorithm maintains the&#10;accuracy of performance by using an improved event-based Harris detector.&#10;Additionally, FA-Harris does not require artificially synthesized event-frames&#10;and can operate on asynchronous events directly. We implement the proposed&#10;method in C++ and evaluate it on public Event Camera Datasets. The results show&#10;that our method achieves approximately 8x speed-up when compared with&#10;previously reported event-based Harris detector, and with no compromise on the&#10;accuracy of performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.16431" label="2408.16431">
-        <attvalues>
-          <attvalue for="0" value="Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th&#10;  LSVOS" />
-          <attvalue for="1" value="  Video object segmentation (VOS) is a crucial task in computer vision, but&#10;current VOS methods struggle with complex scenes and prolonged object motions.&#10;To address these challenges, the MOSE dataset aims to enhance object&#10;recognition and differentiation in complex environments, while the LVOS dataset&#10;focuses on segmenting objects exhibiting long-term, intricate movements. This&#10;report introduces a discriminative spatial-temporal VOS model that utilizes&#10;discriminative object features as query representations. The semantic&#10;understanding of spatial-semantic modules enables it to recognize object parts,&#10;while salient features highlight more distinctive object characteristics. Our&#10;model, trained on extensive VOS datasets, achieved first place&#10;(\textbf{80.90\%} $\mathcal{J \&amp; F}$) on the test set of the 6th LSVOS&#10;challenge in the VOS Track, demonstrating its effectiveness in tackling the&#10;aforementioned challenges. The code will be available at&#10;\href{https://github.com/yahooo-m/VOS-Solution}{code}.&#10;" />
-          <attvalue for="2" value="&#10;Video Object Segmentation (VOS) aims to track and segment target objects in a video sequence, starting from mask annotations in the initial frame~\cite{ytvos2018, davis2017, feelvos, aot}. &#10;With the increasing volume of video data, VOS technology has broad applications in fields like autonomous driving, augmented reality, and interactive video editing~\cite{perclip, bao2018cnnmrf}. &#10;However, VOS tasks face numerous challenges, including drastic changes in object appearance, occlusions, and identity confusion caused by similar objects and background elements. &#10;These issues are particularly pronounced in long-term videos, further complicating the task.&#10;&#10;VOS methods~\cite{STM, stcn} typically perform video object segmentation by comparing the test frame with previous frames. &#10;An association model is initially used to generate pixel-wise correlated features between the test frame and target templates. &#10;These correlated features are then utilized to accurately predict the target masks~\cite{SimVOS}. &#10;To handle the issue of target appearance changes over time, some methods~\cite{aot, xmem} incorporate a memory module that stores samples of the targets to capture their evolving appearances. &#10;Additionally, recent approaches~\cite{ISVOS, cutie} introduce object queries to differentiate various target objects, thereby reducing identity confusion.&#10;&#10;Two datasets are proposed to evaluate the performance of current methods in realistic and complex environments involving complex or long-term motions. &#10;On one hand, the MOSE~\cite{mose} (coMplex video Object SEgmentation) dataset is a comprehensive collection designed to provide challenging scenarios with occluded and crowded objects. &#10;Unlike existing datasets that primarily feature relatively isolated and salient objects, MOSE focuses on realistic environments, highlighting the limitations of current VOS methods and encouraging the development of more robust algorithms. &#10;LVOS~\cite{Hong_2023_ICCV} (Long-term Video Object Segmentation) is the first densely annotated long-term video object segmentation dataset, which aims to provide a benchmark for the development and evaluation of long-term VOS models. &#10;The long-term videos in LVOS encompass multiple challenges, particularly attributes unique to long-term videos such as frequent reappearances and long-term similar object confusion.&#10;&#10;Current methods face multiple challenges when handling the MOSE and LVOS datasets. &#10;First, when targets exhibit multiple complex or independent parts due to occlusion, background clutter, and shape complexity, existing methods tend to produce incomplete prediction masks. &#10;This is mainly because these methods rely too much on pixel-level correlation, focusing on pixel similarity while overlooking semantic information. &#10;Although object query representations~\cite{ISVOS, cutie} improve ID association accuracy, they perform poorly in sequences with significant target appearance changes and small targets. &#10;The current query propagation strategy, which updates using the entire online predicted target sample, easily introduces noise and accumulates errors, leading to tracking failures such as missing targets and ID switches, particularly when tracking small objects. &#10;Additionally, most existing methods are primarily designed for short-term VOS and struggle to handle long-term scenarios where understanding complex motions becomes increasingly challenging. &#10;These methods are prone to long-term disappearance and error accumulation, which can result in poor performance and even out-of-memory crashes due to the continuously expanding memory bank required for long videos.&#10;&#10;To address the aforementioned issues, we propose a VOS framework to learn both the semantic prior, spatial dependence and discriminative query representation.&#10;Specifically, we design a block that efficiently learns both semantic and detailed information, which can extract rich semantic features from a pre-trained Vision Transformer (ViT) without the need to train all feature extraction parameters.&#10;To better model the query representation of the target, we design a more discriminative filtering mechanism to generate discriminative queries.&#10;&#10;The main designs in this project include: &#10;\begin{itemize}&#10;\item We propose a spatial-semantic block to incorporate the semantic information from the ViT feature for Video Object Segmentation. &#10;We use the cls token from a pretrained ViT backbone to extract the semantic prior and global average polling of the image patches to extract the global information of the current frames. &#10;Furthermore, we design a local fusion to leverage the spatial information from the ViT features. The proposed framework shows robustness when facing multiple complex sequences.&#10;\item We develop a discriminative feature filtering module to model the discriminative query representation which achieves great improvement when dealing with small objects.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science, Computer Vision Challenges, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1809.03327" label="1809.03327">
-        <attvalues>
-          <attvalue for="0" value="YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark" />
-          <attvalue for="1" value="  Learning long-term spatial-temporal features are critical for many video&#10;analysis tasks. However, existing video segmentation methods predominantly rely&#10;on static image segmentation techniques, and methods capturing temporal&#10;dependency for segmentation have to depend on pretrained optical flow models,&#10;leading to suboptimal solutions for the problem. End-to-end sequential learning&#10;to explore spatialtemporal features for video segmentation is largely limited&#10;by the scale of available video segmentation datasets, i.e., even the largest&#10;video segmentation dataset only contains 90 short video clips. To solve this&#10;problem, we build a new large-scale video object segmentation dataset called&#10;YouTube Video Object Segmentation dataset (YouTube-VOS). Our dataset contains&#10;4,453 YouTube video clips and 94 object categories. This is by far the largest&#10;video object segmentation dataset to our knowledge and has been released at&#10;http://youtube-vos.org. We further evaluate several existing state-of-the-art&#10;video object segmentation algorithms on this dataset which aims to establish&#10;baselines for the development of new algorithms in the future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.01924" label="2208.01924">
-        <attvalues>
-          <attvalue for="0" value="Per-Clip Video Object Segmentation" />
-          <attvalue for="1" value="  Recently, memory-based approaches show promising results on semi-supervised&#10;video object segmentation. These methods predict object masks frame-by-frame&#10;with the help of frequently updated memory of the previous mask. Different from&#10;this per-frame inference, we investigate an alternative perspective by treating&#10;video object segmentation as clip-wise mask propagation. In this per-clip&#10;inference scheme, we update the memory with an interval and simultaneously&#10;process a set of consecutive frames (i.e. clip) between the memory updates. The&#10;scheme provides two potential benefits: accuracy gain by clip-level&#10;optimization and efficiency gain by parallel computation of multiple frames. To&#10;this end, we propose a new method tailored for the per-clip inference.&#10;Specifically, we first introduce a clip-wise operation to refine the features&#10;based on intra-clip correlation. In addition, we employ a progressive matching&#10;mechanism for efficient information-passing within a clip. With the synergy of&#10;two modules and a newly proposed per-clip based training, our network achieves&#10;state-of-the-art performance on Youtube-VOS 2018/2019 val (84.6% and 84.6%) and&#10;DAVIS 2016/2017 val (91.9% and 86.1%). Furthermore, our model shows a great&#10;speed-accuracy trade-off with varying memory update intervals, which leads to&#10;huge flexibility.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.09903" label="2308.09903">
-        <attvalues>
-          <attvalue for="0" value="Scalable Video Object Segmentation with Simplified Framework" />
-          <attvalue for="1" value="  The current popular methods for video object segmentation (VOS) implement&#10;feature matching through several hand-crafted modules that separately perform&#10;feature extraction and matching. However, the above hand-crafted designs&#10;empirically cause insufficient target interaction, thus limiting the dynamic&#10;target-aware feature learning in VOS. To tackle these limitations, this paper&#10;presents a scalable Simplified VOS (SimVOS) framework to perform joint feature&#10;extraction and matching by leveraging a single transformer backbone.&#10;Specifically, SimVOS employs a scalable ViT backbone for simultaneous feature&#10;extraction and matching between query and reference features. This design&#10;enables SimVOS to learn better target-ware features for accurate mask&#10;prediction. More importantly, SimVOS could directly apply well-pretrained ViT&#10;backbones (e.g., MAE) for VOS, which bridges the gap between VOS and&#10;large-scale self-supervised pre-training. To achieve a better performance-speed&#10;trade-off, we further explore within-frame attention and propose a new token&#10;refinement module to improve the running speed and save computational cost.&#10;Experimentally, our SimVOS achieves state-of-the-art results on popular video&#10;object segmentation benchmarks, i.e., DAVIS-2017 (88.0% J&amp;F), DAVIS-2016 (92.9%&#10;J&amp;F) and YouTube-VOS 2019 (84.2% J&amp;F), without applying any synthetic video or&#10;BL30K pre-training used in previous VOS approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Video Object Segmentation. Traditional VOS methods follow the basic idea of online fine-tuning at test time to adapt to online tracked objects. Typical works \yty{include} OSVOS \cite{osvos}, OnAVIS \cite{OnaVOS}, MoNet \cite{monet}, MaskTrack \cite{MaskTrack} and PReMVOS \cite{PReMVOS}. However, the time-consuming fine-tuning step limits their applicability to real-time applications, and meanwhile, the limited number of online training samples still degrades online fine-tuning. To improve the inference efficiency, OSMN \cite{OSMN} employs a meta neural network to guide mask prediction and uses a single forward pass to adapt the network to a specific test video. PML \cite{PML} formulates VOS as a pixel-wise retrieval task in the learned feature embedding space and uses a nearest-neighbor approach for real-time pixel-wise classification.&#10;&#10;The typical matching-based approach STM \cite{STM} employs an offline-learned matching network, and treats past frame predictions as memory frames for the current frame matching. CFBI \cite{CFBI} and FEELVOS \cite{Feelvos} further improve the matching mechanism by leveraging foreground-background integration and local-global matching. AOT \cite{AOT} employs a long short-term transformer and an identification mechanism for the simultaneously multi-object association. STCN \cite{STCN} uses the L2 similarity to replace the dot product used in STM and establishes correspondences only on images to further improve the inference speed. To efficiently encode spatiotemporal cues, SSTVOS \cite{SSTVOS} uses a sparse spatiotemporal transformer. XMEM \cite{XMEM} further proposes an Atkinson-Shiffrin memory model to enable STM-like approaches to perform long-term VOS. &#10;Although successes have been achieved by these matching-based approaches, their performance is still limited due to the separate feature extraction and interaction designs. In this work, we show that the above hand-crafted designs can be effectively replaced with a general-purpose transformer architecture, i.e., a vanilla ViT \cite{ViT} backbone with joint feature extraction and interaction, which can greatly benefit from existing large-scale pre-trained models (e.g., MAE \cite{mae}) and thus improve the state-of-the-art VOS performance \cite{davis16,davis17}.&#10;&#10;Large-scale self-supervised pre-training. Large-scale self-supervised pre-training has achieved significant progress in recent years. Traditional approaches mainly focus on designing various pretext tasks for unsupervised representation learning, e.g., solving jigsaw puzzle \cite{jigsaw}, coloring images \cite{coloring} and predicting future frame \cite{future_prediction_1,future_prediction_2} or rotation angle \cite{rotation_pred}. Recent advances \cite{simclr,moco,membank,CMC} show that more discriminative unsupervised feature representations can be learned in a contrastive paradigm. However, these approaches may ignore modeling of local image structures, thus being limited in some fine-grained vision tasks, e.g., segmentation. The generative MAE \cite{mae} approach further improves on the contrastive learning-based methods by learning more fine-grained local structures, which are beneficial for localization or segmentation-based downstream tasks. In this work, our proposed SimVOS can directly apply the pre-trained models learned by existing self-supervised methods, which effectively bridges the gap between the VOS and the self-supervised learning communities. We also show that MAE can serve as a strong pre-trained model for the VOS task.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2212.06826" label="2212.06826">
-        <attvalues>
-          <attvalue for="0" value="Look Before You Match: Instance Understanding Matters in Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  Exploring dense matching between the current frame and past frames for&#10;long-range context modeling, memory-based methods have demonstrated impressive&#10;results in video object segmentation (VOS) recently. Nevertheless, due to the&#10;lack of instance understanding ability, the above approaches are oftentimes&#10;brittle to large appearance variations or viewpoint changes resulted from the&#10;movement of objects and cameras. In this paper, we argue that instance&#10;understanding matters in VOS, and integrating it with memory-based matching can&#10;enjoy the synergy, which is intuitively sensible from the definition of VOS&#10;task, \ie, identifying and segmenting object instances within the video.&#10;Towards this goal, we present a two-branch network for VOS, where the&#10;query-based instance segmentation (IS) branch delves into the instance details&#10;of the current frame and the VOS branch performs spatial-temporal matching with&#10;the memory bank. We employ the well-learned object queries from IS branch to&#10;inject instance-specific information into the query key, with which the&#10;instance-augmented matching is further performed. In addition, we introduce a&#10;multi-path fusion block to effectively combine the memory readout with&#10;multi-scale features from the instance segmentation decoder, which incorporates&#10;high-resolution instance-aware features to produce final segmentation results.&#10;Our method achieves state-of-the-art performance on DAVIS 2016/2017 val (92.6%&#10;and 87.1%), DAVIS 2017 test-dev (82.8%), and YouTube-VOS 2018/2019 val (86.3%&#10;and 86.3%), outperforming alternative methods by clear margins.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10; Propagation-based VOS. Propagation-based VOS methods~\cite{tsai2016video,cheng2017segflow,li2018video,oh2018fast,xu2018dynamic,yang2018efficient,duke2021sstvos,xu2022accelerating} take advantage of temporal correlations between adjacent frames to iteratively propagate the segmentation masks from the previous frame to the current frame. Early approaches~\cite{perazzi2017learning,caelles2017one,hu2017maskrnn} typically follow an online learning manner by finetuning models at test-time, which therefore suffer from limited inference efficiency. To mitigate this issue, the following studies shift attention to offline learning by utilizing optical flow~\cite{tsai2016video,cheng2017segflow,xu2018dynamic} as guidance to deliver the temporal information smoothly. Despite the promising results achieved, these methods are oftentimes vulnerable to the error accumulation brought by occlusion or drifting. &#10;&#10;Matching-based VOS. In order to model the spatial-temporal context over longer distances, matching-based models~\cite{hu2018videomatch,oh2019video,cheng2021mivos,cheng2021stcn} typically calculate the correspondence between the current frame and the reference frame~\cite{hu2018videomatch,chen2018blazingly,voigtlaender2019feelvos,yang2020collaborative} and even a feature memory~\cite{hu2021learning,liang2020video,lu2020video,seong2020kernelized,seong2021hierarchical,xie2021efficient,cheng2022xmem} to identify target objects. In addition, several studies focus on designing novel memory construction~\cite{li2020fast,liang2020video} or matching~\cite{wang2021swiftnet} strategies to improve the inference efficiency of VOS models. However, the widely adopted dense matching between reference features will inevitably fail on the objects with significant appearance variations or viewpoint changes. In this paper, we propose to integrate instance understanding into VOS, which is neglected in existing matching-based methods.&#10;&#10;Instance Segmentation. Built upon powerful object detectors, two-stage instance segmentation models~\cite{he2017mask,li2017fully,liu2018path,huang2019mask,bolya2019yolact,chen2020blendmask} predict bounding boxes first and then extract the instance mask in each region of interest (ROI). These methods require complicated procedures, \eg, region proposal generation~\cite{ren2015faster} and ROIAlign~\cite{he2017mask}, which motivates the following work to develop one-stage box-free models~\cite{arnab2017pixelwise,newell2017associative,liu2017sgn,kong2018recurrent,gao2019ssap,neven2019instance,cheng2020panoptic,wang2020solo,wang2020solov2}. Recently, the success of DETR~\cite{carion2020end} inspires a series of query-based models~\cite{fang2021instances,cheng2021per,cheng2022masked} to reformulate the instance segmentation from a novel ``set prediction'' perspective, which achieve state-of-the-art performance on standard benchmarks~\cite{lin2014microsoft}. In this work, we build our VOS model with an auxiliary query-based instance segmentation model~\cite{cheng2022masked} to make the intermediate feature instance-aware rather than explicitly use the output segmentation mask. It enables us to perform instance-augmented matching between the instance-aware features of the current frame and memory. Note that our approach is generic and can be employed to improve various memory-based VOS methods.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2302.01872" label="2302.01872">
-        <attvalues>
-          <attvalue for="0" value="MOSE: A New Dataset for Video Object Segmentation in Complex Scenes" />
-          <attvalue for="1" value="  Video object segmentation (VOS) aims at segmenting a particular object&#10;throughout the entire video clip sequence. The state-of-the-art VOS methods&#10;have achieved excellent performance (e.g., 90+% J&amp;F) on existing datasets.&#10;However, since the target objects in these existing datasets are usually&#10;relatively salient, dominant, and isolated, VOS under complex scenes has rarely&#10;been studied. To revisit VOS and make it more applicable in the real world, we&#10;collect a new VOS dataset called coMplex video Object SEgmentation (MOSE) to&#10;study the tracking and segmenting objects in complex environments. MOSE&#10;contains 2,149 video clips and 5,200 objects from 36 categories, with 431,725&#10;high-quality object segmentation masks. The most notable feature of MOSE&#10;dataset is complex scenes with crowded and occluded objects. The target objects&#10;in the videos are commonly occluded by others and disappear in some frames. To&#10;analyze the proposed MOSE dataset, we benchmark 18 existing VOS methods under 4&#10;different settings on the proposed MOSE dataset and conduct comprehensive&#10;comparisons. The experiments show that current VOS algorithms cannot well&#10;perceive objects in complex scenes. For example, under the semi-supervised VOS&#10;setting, the highest J&amp;F by existing state-of-the-art VOS methods is only 59.4%&#10;on MOSE, much lower than their ~90% J&amp;F performance on DAVIS. The results&#10;reveal that although excellent performance has been achieved on existing&#10;benchmarks, there are unresolved challenges under complex scenes and more&#10;efforts are desired to explore these challenges in the future. The proposed&#10;MOSE dataset has been released at https://henghuiding.github.io/MOSE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.10181" label="2211.10181">
-        <attvalues>
-          <attvalue for="0" value="LVOS: A Benchmark for Long-term Video Object Segmentation" />
-          <attvalue for="1" value="  Existing video object segmentation (VOS) benchmarks focus on short-term&#10;videos which just last about 3-5 seconds and where objects are visible most of&#10;the time. These videos are poorly representative of practical applications, and&#10;the absence of long-term datasets restricts further investigation of VOS on the&#10;application in realistic scenarios. So, in this paper, we present a new&#10;benchmark dataset named \textbf{LVOS}, which consists of 220 videos with a&#10;total duration of 421 minutes. To the best of our knowledge, LVOS is the first&#10;densely annotated long-term VOS dataset. The videos in our LVOS last 1.59&#10;minutes on average, which is 20 times longer than videos in existing VOS&#10;datasets. Each video includes various attributes, especially challenges&#10;deriving from the wild, such as long-term reappearing and cross-temporal&#10;similar objeccts.Based on LVOS, we assess existing video object segmentation&#10;algorithms and propose a Diverse Dynamic Memory network (DDMemory) that&#10;consists of three complementary memory banks to exploit temporal information&#10;adequately. The experimental results demonstrate the strength and weaknesses of&#10;prior methods, pointing promising directions for further study. Data and code&#10;are available at https://lingyihongfd.github.io/lvos.github.io/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.13489" label="2111.13489">
-        <attvalues>
-          <attvalue for="0" value="SurfEmb: Dense and Continuous Correspondence Distributions for Object&#10;  Pose Estimation with Learnt Surface Embeddings" />
-          <attvalue for="1" value="  We present an approach to learn dense, continuous 2D-3D correspondence&#10;distributions over the surface of objects from data with no prior knowledge of&#10;visual ambiguities like symmetry. We also present a new method for 6D pose&#10;estimation of rigid objects using the learnt distributions to sample, score and&#10;refine pose hypotheses. The correspondence distributions are learnt with a&#10;contrastive loss, represented in object-specific latent spaces by an&#10;encoder-decoder query model and a small fully connected key model. Our method&#10;is unsupervised with respect to visual ambiguities, yet we show that the query-&#10;and key models learn to represent accurate multi-modal surface distributions.&#10;Our pose estimation method improves the state-of-the-art significantly on the&#10;comprehensive BOP Challenge, trained purely on synthetic data, even compared&#10;with methods trained on real data. The project site is at&#10;https://surfemb.github.io/ .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;There exists a wide range of pose estimation approaches.&#10;High-performing classical methods \cite{drost, vidal} and hybrids \cite{konig} rely on depth information and use point-pair based voting.&#10;&#10;Some learning based approaches \cite{posecnn, deepim, cosypose} regress a representation of the pose directly.&#10;PoseCNN~\cite{posecnn} regresses depth, the projected 2D center and a quaternion for each region-of-interest in a custom detection pipeline.&#10;DeepIM~\cite{deepim} proposes iterative refinement by regressing a pose difference between a render of the pose hypothesis and the input image.&#10;CosyPose~\cite{cosypose} builds on DeepIM, adding several improvements, including &#10;a continuous rotation parameterization,&#10;handling symmetries explicitly,&#10;and using more recent architectures.&#10;\cite{implicit-orientation-learning, poserbpf} train autoencoders to capture rotation information in a latent representation. &#10;After training, they build a latent codebook representing a large set of rotations,&#10;and during inference, they either choose the nearest neighbour in the codebook \cite{implicit-orientation-learning} or design a distribution \cite{poserbpf} over rotations.&#10;\cite{multipath} shows that the autoencoder can be shared across multiple objects and generalize to unseen objects.&#10;The latent representation in \cite{implicit-orientation-learning, poserbpf, multipath} handles symmetries implicitly, but assumes that the object frame's origin lies on the planes or axes of symmetries, and does not represent positional ambiguity.&#10;Our method does not assume anything about the placement of the object frame and is able to represent both positional and rotational ambiguities by the distributions of correspondences.&#10;&#10;Other learning based approaches are based on establishing 2D-3D correspondences \cite{bb8, pvnet, pix2pose, cdpn, epos, sopose, dpodv2} followed by a variant of PnP-RANSAC.&#10;BB8~\cite{bb8} regresses the 2D coordinates of projected 3D bounding box corners and handles symmetries explicitly by limiting the ground truth pose space based on the object symmetry.&#10;PVNet~\cite{pvnet} regresses vector fields toward the 2D projections of a set of fixed 3D key points and handles symmetries like BB8.&#10;Pix2Pose~\cite{pix2pose} employs an encoder-decoder network to regress dense object coordinates of the object surface.&#10;To handle symmetry, they calculate the loss with respect to the pose that has the smallest loss while being valid under symmetry, and further use a Generative Adversarial Network to lock on to a plausible mode.&#10;In addition to dense correspondences, SO-Pose~\cite{sopose} also estimates self-occlusion maps to constrain their pose hypotheses.&#10;CDPN~\cite{cdpn} regresses dense correspondences but only use them to estimate the rotation, &#10;and instead uses a separate head to regress the translation.&#10;DPODv2~\cite{dpodv2} also regresses dense correspondences. They obtain an initial pose with PnP-RANSAC and refine the pose based on differentiable rendering and the estimated dense correspondences. They handle symmetry similar to BB8 and PVNet.&#10;\cite{local-surface-embeddings} regresses dense hand-crafted 3D features&#10;to establish 2D-3D correspondences, &#10;even for objects not seen during training.&#10;While their feature definition handles symmetries implicitly, &#10;they are by definition limited to local, rotation invariant 3D information.&#10;EPOS~\cite{epos} learns a dense 2D-3D correspondence distribution and handles symmetries implicitly. &#10;They do so by discretizing the surface into a set of fragments and predicting a probability distribution over fragments per pixel.&#10;They regress a coordinate offset per fragment to increase correspondence accuracy.&#10;&#10;EPOS is the most related pose estimation method, being the only one representing a distribution of correspondences.&#10;In contrast to all pose estimation methods, we estimate a continuous distribution over the object surface, &#10;and we use the distributions instead of correspondences for pose scoring and refinement.&#10;&#10;There are also works outside object pose estimation, that aim to establish correspondences.&#10;\cite{dense-object-nets} learns embeddings in a self-supervised fashion with a contrastive loss&#10;to establish 2D-2D correspondences across images for object manipulation.&#10;\cite{continuous} proposes to establish dense 2D-3D correspondences using continuous surface embeddings, and their framework for learning correspondences is similar to ours with some important differences.&#10;Most importantly, they only establish a single correspondence per pixel and do not discuss or show the ability to represent distributions or in other ways handle ambiguities.&#10;Nor do they use the obtained correspondences for object pose estimation.&#10;There are more differences, including that&#10;their key model embeddings are explicitly regularized to be continuous, &#10;whereas the continuity of our embeddings are primarily driven by visual ambiguity, &#10;and that they use a fixed set of object vertices during training, &#10;whereas we use a contrastive loss, sampling from the surface during training.&#10; " />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Object Pose Estimation, Unsupervised Learning, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1908.09101" label="1908.09101">
-        <attvalues>
-          <attvalue for="0" value="Where Is My Mirror?" />
-          <attvalue for="1" value="  Mirrors are everywhere in our daily lives. Existing computer vision systems&#10;do not consider mirrors, and hence may get confused by the reflected content&#10;inside a mirror, resulting in a severe performance degradation. However,&#10;separating the real content outside a mirror from the reflected content inside&#10;it is non-trivial. The key challenge is that mirrors typically reflect contents&#10;similar to their surroundings, making it very difficult to differentiate the&#10;two. In this paper, we present a novel method to segment mirrors from an input&#10;image. To the best of our knowledge, this is the first work to address the&#10;mirror segmentation problem with a computational approach. We make the&#10;following contributions. First, we construct a large-scale mirror dataset that&#10;contains mirror images with corresponding manually annotated masks. This&#10;dataset covers a variety of daily life scenes, and will be made publicly&#10;available for future research. Second, we propose a novel network, called&#10;MirrorNet, for mirror segmentation, by modeling both semantical and low-level&#10;color/texture discontinuities between the contents inside and outside of the&#10;mirrors. Third, we conduct extensive experiments to evaluate the proposed&#10;method, and show that it outperforms the carefully chosen baselines from the&#10;state-of-the-art detection and segmentation methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In this section, we briefly review state-of-the-art methods from relevant fields, including semantic/instance segmentation, saliency/shadow detection, as well as mirror detection works from the 3D community.&#10;&#10;{Semantic segmentation.} It aims to assign per-pixel predictions of object categories to the input image.&#10;Based on the fully convolutional encoder-decoder structure~\cite{long2015fully}, state-of-the-art semantic segmentation approaches typically leverage multi-scale (level) context aggregation to learn discriminative features for recognizing the objects and delineating their boundaries.&#10;Specifically, low-level encoder features are combined with their corresponding decoder features by feeding recorded pooling indices~\cite{Badrinarayanan2017SegNet} or concatenation~\cite{ronneberger2015u}. Dilated convolutions are used in~\cite{chen2015semantic,yu2015multi} to expand the receptive fields to compensate for the lost details in the encoder part. PSPNet~\cite{zhao2017pyramid} leverages pyramid pooling to obtain multi-scale representations in order to differentiate objects of similar appearances.&#10;Zhang~\etal~\cite{Zhang_2018_ECCV} propose to fuse the low-/high-level features so as to take advantages of both high resolution spatial and rich semantic information in the encoder part.&#10;Zhang~\etal~\cite{zhang2018context} propose to explicitly predict the objects in the scene and use this prediction to selectively highlight the semantic features.&#10;Ding~\etal~\cite{ding2018context} propose to learn contextual contrasted features to boost the segmentation performance of small objects.&#10;&#10;However, applying existing segmentation methods for mirror segmentation (\ie, treating mirrors as one of the object categories) cannot solve the fundamental problem of mirror segmentation, which is that the reflected content of a mirror&#10;can also be segmented too. In this paper, we focus on the mirror segmentation problem and formulate it as a binary classification problem (\ie, mirror or non-mirror).&#10;&#10;{Instance segmentation.} It aims to simultaneously recognize, localize and segment out objects while differentiating individual instances of the same category.&#10;State-of-the-art detection based instance segmentation methods extend object detection methods, \eg, Faster-RCNN~\cite{ren2017faster} and FPN~\cite{lin2017feature}, to obtain instance maps. Mask RCNN~\cite{he2017mask} uses one additional branch to predict instance segmentation masks from the box predictions of Faster-RCNN~\cite{ren2017faster}. PANet~\cite{liu2018path} further proposes to add bottom-up paths to facilitate feature propagation in Mask RCNN \cite{he2017mask} and aggregates multi-level features for detection and segmentation. MaskLab \cite{Chen_2018_CVPR} adopts Faster-RCNN~\cite{ren2017faster} to locate objects and combines semantic segmentation with pixel-direction (to its instance center) prediction for instance segmentation. Another line of works first use a segmentation method to obtain per-pixel labels, and then a clustering method to group the pixels into instances, via depth estimation~\cite{Zhang_2015_ICCV}, spectral clustering~\cite{liang2018proposal}, and neural networks~\cite{wu2016bridging,Liu_2017_ICCV}.&#10;&#10;Similar to semantic segmentation, instance segmentation methods cannot differentiate between the content inside a mirror and that of {outside}. As a result, they would segment objects inside the mirror too.&#10;&#10;{Salient object detection (SOD).} It aims to identify the most conspicuous object(s) in an image. While conventional SOD methods rely on low-level hand-crafted features (\eg, color and contrast), deep learning based SOD methods consider either or both bottom-up and top-down saliency inferences. Wang~\etal~\cite{Wang2015Deep} propose to integrate local pixel-wise saliency estimation and global object proposal search for salient object detection. Multi-level feature aggregation from deep networks is also explored for detecting and refining the detection \cite{lee2016deep,Zhang2017Amulet,Hou_2017_CVPR}. Recent works apply attention mechanisms for learning global and local contexts~\cite{liu2018picanet} or learning foreground/background attention maps~\cite{Chen_2018_ECCV} to help detect salient objects and eliminate non-salient objects.&#10;&#10;The content reflected by a mirror, however, may or may not be salient. Even though if it is salient, it is likely that only part of it is salient. Hence, applying existing SOD methods to detect mirrors may not address the mirror segmentation problem.&#10;&#10;{Shadow detection.} It aims to detect/remove shadows from the input images.&#10;Hu~\etal~\cite{Hu_2018_CVPR} propose to use direction-aware features to analyze the contrasts between shadow/non-shadow regions for shadow detection. Le~\etal~\cite{Le_2018_ECCV} propose to train a shadow detection network with augmented adversarial shadow examples generated from a shadow attenuation network. Zhu~\etal~\cite{Zhu_2018_ECCV} propose a bidirectional feature pyramid to leverage the spatial contexts from shallow and deep CNN layers. A conditional GAN~\cite{mirza2014conditional} is also applied to model both local features and global image semantics for shadow detection~\cite{Nguyen2017Shadow} and removal~\cite{Wang_2018_CVPR}. Qu~\etal~\cite{Qu2017CVPR} propose a multi-context network, together with a new dataset, for shadow removal.&#10;&#10;In general, shadow detection methods are largely based on detecting the intensity contrast between shadow and non-shadow regions. In {contrast}, the contents inside and outside of a mirror typically have very similar intensity, making the mirror segmentation problem more difficult to address.&#10;&#10;{Mirror detection in the 3D community.} To our knowledge, there are only two works that consider mirror segmentation in 3D reconstruction. Matterport3D~\cite{chang2017matterport3d} proposes the user to manually segment the mirrors on an iPad during scanning. Whelan~\etal~\cite{whelan2018reconstructing} attach a hardware tag (based on the AprilTag~\cite{olson2011apriltag}) to the scanner. If a tag is detected in the captured image, it signals the presence of a mirror. A total variation-based segmentation method is then used to segment the mirror based on a set of hand-crafted features (\eg, depth discontinuity and intensity variance).&#10;&#10;Instead of using any special hardware, in this paper, we propose the first automatic method for mirror segmentation and the first mirror dataset with mirror annotations.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Physics, Image Processing, Deep Learning, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2002.01154" label="2002.01154">
-        <attvalues>
-          <attvalue for="0" value="Texture Classification using Block Intensity and Gradient Difference&#10;  (BIGD) Descriptor" />
-          <attvalue for="1" value="  In this paper, we present an efficient and distinctive local descriptor,&#10;namely block intensity and gradient difference (BIGD). In an image patch, we&#10;randomly sample multi-scale block pairs and utilize the intensity and gradient&#10;differences of pairwise blocks to construct the local BIGD descriptor. The&#10;random sampling strategy and the multi-scale framework help BIGD descriptors&#10;capture the distinctive patterns of patches at different orientations and&#10;spatial granularity levels. We use vectors of locally aggregated descriptors&#10;(VLAD) or improved Fisher vector (IFV) to encode local BIGD descriptors into a&#10;full image descriptor, which is then fed into a linear support vector machine&#10;(SVM) classifier for texture classification. We compare the proposed descriptor&#10;with typical and state-of-the-art ones by evaluating their classification&#10;performance on five public texture data sets including Brodatz, CUReT,&#10;KTH-TIPS, and KTH-TIPS-2a and -2b. Experimental results show that the proposed&#10;BIGD descriptor with stronger discriminative power yields 0.12% ~ 6.43% higher&#10;classification accuracy than the state-of-the-art texture descriptor, dense&#10;microblock difference (DMD).&#10;" />
-          <attvalue for="2" value="&#10;&#10;Texture can be broadly defined as a type of visual features that characterize the surface of an object or a material. Distinctive and robust representation of texture is the key for various multimedia applications such as image representation~\cite{al2017curvelet}, texture retrieval~\cite{al2019curvelettextureretrieval}, face recognition\cite{zhao2017dynamic}, image quality assessment~\cite{temel2019perceptual,temel2016csv}, image/texture segmentation~\cite{vo2010study}, dynamic texture/scene recognition~\cite{arashloo2014dynamic,zhao2017dynamic}, texture/color style transfer~\cite{lo2016example}, and seismic interpretation\cite{long2018comparative}. Texture descriptors~\cite{bay2006surf,hu2016completed,liu2016evaluation, hu2017sselbp,liu2015fusing,mehta2016texture, allili2014texture}, which are robust against rotations and translations of images, are able to provide discriminative features. &#10;&#10;Texture representation requires texture descriptors to have two competing goals, high-quality description and low computational cost. Several research efforts have focused on extracting texture descriptors in a distinctive and efficient way. These approaches are commonly divided into several categories including covariance-, fractal-, filter-, gradient-, and binary-based descriptors. Covariance descriptors modeling the second-order statistics of images perform well on material categorization~\cite{harandi2014bregman}\cite{wang2012covariance}. However, by retaining only the second-order statistics, covariance descriptors are prone to be singular and have limited capability in modeling nonlinear, complicated feature relationships. Fractal-based descriptors~\cite{quan2014distinct,quan2014lacunarity} utilize the concept of fractal dimension and lacunarity analysis in fractal geometry to characterize the spatial distribution of local image structures in a statistical approach, guaranteeing both discriminative ability and invariance of texture descriptors. Filter-based descriptors~\cite{crosier2010using}\cite{varma2005statistical} acquire local features using filter banks. One problem of filter-based descriptors is that the design of filter banks is data dependent. Among gradient-based descriptors, scalable invariant feature transform (SIFT)~\cite{lowe2004distinctive} and speeded up robust features (SURF)~\cite{bay2006surf} as the most popular ones capture the discriminative gradient features of local patches. Binary descriptors such as local binary pattern (LBP)~\cite{ojala2002multiresolution}, binary robust independent elementary features (BRIEF)~\cite{calonder2010brief}, and local binary difference (LDB)~\cite{yang2014local}, which convert the intensity differences of neighboring pixels to binary values, are robust to monotonic illumination changes and require low computational cost. These advantages make binary descriptors more appealing for real-time applications. However, the binarization of local intensity differences leads to the loss of intensity information, which weakens the ability of discrimination.&#10;Another disadvantage of binary descriptors is their dimensions, which will grow exponentially when the number of pairwise comparisons on neighboring pixels increases. To alleviate these problems, Zhang et al.~\cite{zhang2017feature} proposed a descriptor namely normalized difference vector (NDV) and Mehta et al.~\cite{mehta2016texture} proposed a novel descriptor called dense micro-block difference (DMD). Both methods composed of real-valued intensity differences instead of binary codes of different micro-blocks in local patches. Although DMD captures non-quantized patch-based features at multiple scales and orientations, the neglect of first-order gradients may deteriorate the discriminative ability~\cite{bay2006surf}\cite{yang2014local}.&#10;&#10;In this paper, we introduce a novel local descriptor, block intensity and gradient difference (BIGD), which achieves great distinctiveness and computational efficiency. Compared with other algorithms mentioned above, our main contribution is efficiently capturing un-quantized gradient difference features in BIGD. The gradient difference captures the variations of gradients in a local patch and improves distinctiveness. Descriptors such as histogram of orientated gradients (HOG) and SIFT utilize gradient-based features to capture the orientation information. However, quantized orientations in them result in information loss. Our BIGD method extracts intensity- and gradient-difference features at multi-orientations without quantization and retains the discriminative power of features. &#10;To show the performance of BIGD, we evaluate it within a texture classification pipeline as shown in Fig.~\ref{fig:diagram}, which generally includes three major modules: a feature extraction, an image encoding, and a classifier. We utilize the our proposed descriptor in the feature extraction module. Since this paper mainly focuses on feature extraction rather than feature encoding and classifiers, we use simple feature encoding methods such as vectors of locally aggregated descriptors (VLAD)~\cite{jegou2010aggregating} or improved Fisher vectors (IFV)~\cite{perronnin2010improving}\cite{sanchez2013image} and classifiers such as a linear support vector machine (SVM) when we compare our proposed descriptor with other descriptors. The rest of this paper is organized as follows. Section~\ref{sec:BIGD_featureextraction} introduces the proposed BIGD descriptor and its application on texture classification. In Section~\ref{sec:experiments}, we evaluate the classification performance of the BIGD descriptor on five public texture databases and compare it with that of state-of-the-art methods. Section~\ref{sec:experiments} concludes the paper. Matlab codes will be available online.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Image Texture Analysis, Machine Learning Classification, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1810.05964" label="1810.05964">
-        <attvalues>
-          <attvalue for="0" value="Perceptual Image Quality Assessment through Spectral Analysis of Error&#10;  Representations" />
-          <attvalue for="1" value="  In this paper, we analyze the statistics of error signals to assess the&#10;perceived quality of images. Specifically, we focus on the magnitude spectrum&#10;of error images obtained from the difference of reference and distorted images.&#10;Analyzing spectral statistics over grayscale images partially models&#10;interference in spatial harmonic distortion exhibited by the visual system but&#10;it overlooks color information, selective and hierarchical nature of visual&#10;system. To overcome these shortcomings, we introduce an image quality&#10;assessment algorithm based on the Spectral Understanding of Multi-scale and&#10;Multi-channel Error Representations, denoted as SUMMER. We validate the quality&#10;assessment performance over 3 databases with around 30 distortion types. These&#10;distortion types are grouped into 7 main categories as compression artifact,&#10;image noise, color artifact, communication error, blur, global and local&#10;distortions. In total, we benchmark the performance of 17 algorithms along with&#10;the proposed algorithm using 5 performance metrics that measure linearity,&#10;monotonicity, accuracy, and consistency. In addition to experiments with&#10;standard performance metrics, we analyze the distribution of objective and&#10;subjective scores with histogram difference metrics and scatter plots.&#10;Moreover, we analyze the classification performance of quality assessment&#10;algorithms along with their statistical significance tests. Based on our&#10;experiments, SUMMER significantly outperforms majority of the compared methods&#10;in all benchmark categories&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.06464" label="1810.06464">
-        <attvalues>
-          <attvalue for="0" value="CSV: Image Quality Assessment Based on Color, Structure, and Visual&#10;  System" />
-          <attvalue for="1" value="  This paper presents a full-reference image quality estimator based on color,&#10;structure, and visual system characteristics denoted as CSV. In contrast to the&#10;majority of existing methods, we quantify perceptual color degradations rather&#10;than absolute pixel-wise changes. We use the CIEDE2000 color difference&#10;formulation to quantify low-level color degradations and the Earth Mover's&#10;Distance between color name descriptors to measure significant color&#10;degradations. In addition to the perceptual color difference, CSV also contains&#10;structural and perceptual differences. Structural feature maps are obtained by&#10;mean subtraction and divisive normalization, and perceptual feature maps are&#10;obtained from contrast sensitivity formulations of retinal ganglion cells. The&#10;proposed quality estimator CSV is tested on the LIVE, the Multiply Distorted&#10;LIVE, and the TID 2013 databases, and it is always among the top two performing&#10;quality estimators in terms of at least ranking, monotonic behavior or&#10;linearity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.08263" label="1812.08263">
-        <attvalues>
-          <attvalue for="0" value="A comparative study of texture attributes for characterizing subsurface&#10;  structures in seismic volumes" />
-          <attvalue for="1" value="  In this paper, we explore how to computationally characterize subsurface&#10;geological structures presented in seismic volumes using texture attributes.&#10;For this purpose, we conduct a comparative study of typical texture attributes&#10;presented in the image processing literature. We focus on spatial attributes in&#10;this study and examine them in a new application for seismic interpretation,&#10;i.e., seismic volume labeling. For this application, a data volume is&#10;automatically segmented into various structures, each assigned with its&#10;corresponding label. If the labels are assigned with reasonable accuracy, such&#10;volume labeling will help initiate an interpretation process in a more&#10;effective manner. Our investigation proves the feasibility of accomplishing&#10;this task using texture attributes. Through the study, we also identify&#10;advantages and disadvantages associated with each attribute.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.04183" label="1812.04183">
-        <attvalues>
-          <attvalue for="0" value="Completed Local Derivative Pattern for Rotation Invariant Texture&#10;  Classification" />
-          <attvalue for="1" value="  In this paper, we propose a new texture descriptor, completed local&#10;derivative pattern (CLDP). In contrast to completed local binary pattern&#10;(CLBP), which involves only local differences at each scale, CLDP encodes the&#10;directional variation of the local differences of two scales as a complementary&#10;component to local patterns in CLBP. The new component in CLDP, with regarded&#10;as the directional derivative pattern, reflects the directional smoothness of&#10;local textures without increasing computation complexity. Experimental results&#10;on the Outex database show that CLDP, as a uni-scale pattern, outperforms&#10;uni-scale state-of-the-art texture descriptors on texture classification and&#10;has comparable performance with multi-scale texture descriptors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1403.4334" label="1403.4334">
-        <attvalues>
-          <attvalue for="0" value="Bregman Divergences for Infinite Dimensional Covariance Matrices" />
-          <attvalue for="1" value="  We introduce an approach to computing and comparing Covariance Descriptors&#10;(CovDs) in infinite-dimensional spaces. CovDs have become increasingly popular&#10;to address classification problems in computer vision. While CovDs offer some&#10;robustness to measurement variations, they also throw away part of the&#10;information contained in the original data by only retaining the second-order&#10;statistics over the measurements. Here, we propose to overcome this limitation&#10;by first mapping the original data to a high-dimensional Hilbert space, and&#10;only then compute the CovDs. We show that several Bregman divergences can be&#10;computed between the resulting CovDs in Hilbert space via the use of kernels.&#10;We then exploit these divergences for classification purposes. Our experiments&#10;demonstrate the benefits of our approach on several tasks, such as material and&#10;texture recognition, person re-identification, and action recognition from&#10;motion capture data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.05735" label="2011.05735">
-        <attvalues>
-          <attvalue for="0" value="DeepSim: Semantic similarity metrics for learned image registration" />
-          <attvalue for="1" value="  We propose a semantic similarity metric for image registration. Existing&#10;metrics like euclidean distance or normalized cross-correlation focus on&#10;aligning intensity values, giving difficulties with low intensity contrast or&#10;noise. Our semantic approach learns dataset-specific features that drive the&#10;optimization of a learning-based registration model. Comparing to existing&#10;unsupervised and supervised methods across multiple image modalities and&#10;applications, we achieve consistently high registration accuracy and faster&#10;convergence than state of the art, and the learned invariance to noise gives&#10;smoother transformations on low-quality images.&#10;" />
-          <attvalue for="2" value="&#10;&#09;Deformable registration is a fundamental preprocessing tool in medical imaging, where the goal is to find anatomical correspondences between images and derive geometric transformations $\transform$ to align them. Most algorithmic and deep learning-based methods optimize alignment via a similarity measure $\dist$ and a $\lambda$-weighted regularizer $\reg$, combined in a loss function&#10;&#09;\begin{equation} &#10;&#09;\loss(\I, \J, \transform) = \dist(\I \circ \transform, \J) + \lambda \reg(\transform) \enspace.&#10;&#09;\end{equation}&#10;&#09;The similarity metric $D$ assesses the alignment and strongly influences the result.&#09;Pixel-based similarity metrics like euclidean distance ($\mse$) and patch-wise normalized cross-correlation ($\ncc$) are commonly used in both algorithmic~\cite{Avants2011, Avants2008, faisalBeg2005, Rueckert1999, thirion1998, Vercauteren2007} and deep learning based~\cite{Alven2019, balakrishnan2019voxel, dalca2018unsuperiveddiff, Dalca2019, DeVos2019, Liu2019, Lee2019a, Hu2019, Hu2019a, Yang2017, Xu2019} image registration. Typically, the similarity measure for a task is selected as the best out of a small set of metrics, with no guarantee that one of the metrics is suitable for the data.&#10;&#10;The shortcomings of pixel-based similarity metrics have been studied substantially in the image generation community \cite{Zhang2018}, where the introduction of deep similarity metrics approximating human visual perception has improved the generation of photo-realistic images \cite{Czolbe2020, hou2017deep}. As registration models are generative models \cite{dalca2018unsuperiveddiff}, we expect these similarity metrics to improve registration as well. Current attempts at using learned similarity metrics for image registration require ground truth transformations \cite{Haskins2019a} or limit the input to the registration model \cite{Lee2019a}. &#10;&#10;We propose a data-driven similarity metric for image registration based on the alignment of semantic features. We learn semantic filters of our metric on the dataset, use it to train a registration model, and validate our approach on three biomedical datasets of different image modalities and applications. Across all datasets, our method achieves consistently high registration accuracy, outperforming even metrics utilizing supervised information. Our models converge faster and learn to ignore noisy image patches, leading to smoother transformations on low-quality data.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Registration, Machine Learning, Engineering, Computer Vision, Semantic Similarity, Mathematics, Image Processing, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.15057" label="2006.15057">
-        <attvalues>
-          <attvalue for="0" value="A Loss Function for Generative Neural Networks Based on Watson's&#10;  Perceptual Model" />
-          <attvalue for="1" value="  To train Variational Autoencoders (VAEs) to generate realistic imagery&#10;requires a loss function that reflects human perception of image similarity. We&#10;propose such a loss function based on Watson's perceptual model, which computes&#10;a weighted distance in frequency space and accounts for luminance and contrast&#10;masking. We extend the model to color images, increase its robustness to&#10;translation by using the Fourier Transform, remove artifacts due to splitting&#10;the image into blocks, and make it differentiable. In experiments, VAEs trained&#10;with the new loss function generated realistic, high-quality image samples.&#10;Compared to using the Euclidean distance and the Structural Similarity Index,&#10;the images were less blurry; compared to deep neural network based losses, the&#10;new approach required less computational resources and generated images with&#10;less artifacts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04605" label="1805.04605">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration" />
-          <attvalue for="1" value="  Traditional deformable registration techniques achieve impressive results and&#10;offer a rigorous theoretical treatment, but are computationally intensive since&#10;they solve an optimization problem for each image pair. Recently,&#10;learning-based methods have facilitated fast registration by learning spatial&#10;deformation functions. However, these approaches use restricted deformation&#10;models, require supervised labels, or do not guarantee a diffeomorphic&#10;(topology-preserving) registration. Furthermore, learning-based registration&#10;tools have not been derived from a probabilistic framework that can offer&#10;uncertainty estimates. In this paper, we present a probabilistic generative&#10;model and derive an unsupervised learning-based inference algorithm that makes&#10;use of recent developments in convolutional neural networks (CNNs). We&#10;demonstrate our method on a 3D brain registration task, and provide an&#10;empirical analysis of the algorithm. Our approach results in state of the art&#10;accuracy and very fast runtimes, while providing diffeomorphic guarantees and&#10;uncertainty estimates. Our implementation is available online at&#10;http://voxelmorph.csail.mit.edu .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.09200" label="1907.09200">
-        <attvalues>
-          <attvalue for="0" value="Image-and-Spatial Transformer Networks for Structure-Guided Image&#10;  Registration" />
-          <attvalue for="1" value="  Image registration with deep neural networks has become an active field of&#10;research and exciting avenue for a long standing problem in medical imaging.&#10;The goal is to learn a complex function that maps the appearance of input image&#10;pairs to parameters of a spatial transformation in order to align corresponding&#10;anatomical structures. We argue and show that the current direct, non-iterative&#10;approaches are sub-optimal, in particular if we seek accurate alignment of&#10;Structures-of-Interest (SoI). Information about SoI is often available at&#10;training time, for example, in form of segmentations or landmarks. We introduce&#10;a novel, generic framework, Image-and-Spatial Transformer Networks (ISTNs), to&#10;leverage SoI information allowing us to learn new image representations that&#10;are optimised for the downstream registration task. Thanks to these&#10;representations we can employ a test-specific, iterative refinement over the&#10;transformation parameters which yields highly accurate registration even with&#10;very limited training data. Performance is demonstrated on pairwise 3D brain&#10;registration and illustrative synthetic data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.10269" label="2207.10269">
-        <attvalues>
-          <attvalue for="0" value="Human-centric Image Cropping with Partition-aware and Content-preserving&#10;  Features" />
-          <attvalue for="1" value="  Image cropping aims to find visually appealing crops in an image, which is an&#10;important yet challenging task. In this paper, we consider a specific and&#10;practical application: human-centric image cropping, which focuses on the&#10;depiction of a person. To this end, we propose a human-centric image cropping&#10;method with two novel feature designs for the candidate crop: partition-aware&#10;feature and content-preserving feature. For partition-aware feature, we divide&#10;the whole image into nine partitions based on the human bounding box and treat&#10;different partitions in a candidate crop differently conditioned on the human&#10;information. For content-preserving feature, we predict a heatmap indicating&#10;the important content to be included in a good crop, and extract the geometric&#10;relation between the heatmap and a candidate crop. Extensive experiments&#10;demonstrate that our method can perform favorably against state-of-the-art&#10;image cropping methods on human-centric image cropping task. Code is available&#10;at https://github.com/bcmi/Human-Centric-Image-Cropping.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Following \cite{zeng2020cropping,li2020composing}, we divide existing image cropping methods into three categories according to the criteria for evaluating candidate crops, \ie, attention-guided, aesthetics-informed, and data-driven.&#10;&#10;Attention-guided Image Cropping: Attention-guided methods \cite{lu2019end,li2019image,li2019collaborative,chen2016automatic,fang2014automatic,saliency-sun2013scale,zhang2005auto} assumed that the best crops should preserve visually important content, which is usually determined by the saliency detection methods \cite{vig2014large,hou2007saliency}. &#10;Usually, the view with the highest average saliency score is selected as the best crop. However, saliency may not accurately reflect the content of interest for human-centric images (see Figure \ref{fig:ill_important_content}). &#10;Differently, we assume that the content that appears in multiple highly scored crops is more likely to be important content, leading to more flexible and practical important content estimation. &#10;&#10;Aesthetics-informed Image Cropping: The aesthetics-informed methods evaluated candidates by comparing the overall aesthetic quality of different crops. To achieve this, earlier methods \cite{zhang2013weakly,yan2013learning,zhang2012probabilistic} usually employed hand-crafted features or composition rules. However, the simple hand-crafted features may not accurately predict the complicated image aesthetics \cite{zeng2020cropping}.&#10;&#10;Data-driven Image Cropping: Most recent methods address the task in a data-driven manner. Some methods \cite{chen2017learning,kao2017automatic,MNA-CNN-mai2016composition} trained a general aesthetic evaluator on image aesthetic datasets to facilitate image cropping.&#10;With the aid of image cropping datasets \cite{chen2017quantitative,zeng2019reliable,wei2018good}, numerous methods \cite{hong2021composing,zeng2019reliable,Tu2020ImageCW,lu2019listwise,li2020composing,wei2018good} used pairwise learning to train an end-to-end model on these datasets, which can generate crop-level scores for ranking different candidate crops.&#10;&#10;Our method is developed based on the general pipeline of the data-driven methods, but is specially tailored to human-centric image cropping with two innovations, \ie, partition-aware and content-preserving features.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Human-Computer Interaction, Cognitive Science, Computer Science, Image Cropping, Human Detection" />
-        </attvalues>
-      </node>
-      <node id="2311.02842" label="2311.02842">
-        <attvalues>
-          <attvalue for="0" value="An invariant feature extraction for multi-modal images matching" />
-          <attvalue for="1" value="  This paper aims at providing an effective multi-modal images invariant&#10;feature extraction and matching algorithm for the application of multi-source&#10;data analysis. Focusing on the differences and correlation of multi-modal&#10;images, a feature-based matching algorithm is implemented. The key technologies&#10;include phase congruency (PC) and Shi-Tomasi feature point for keypoints&#10;detection, LogGabor filter and a weighted partial main orientation map (WPMOM)&#10;for feature extraction, and a multi-scale process to deal with scale&#10;differences and optimize matching results. The experimental results on&#10;practical data from multiple sources prove that the algorithm has effective&#10;performances on multi-modal images, which achieves accurate spatial alignment,&#10;showing practical application value and good generalization.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image matching has been a classical, important, and challenging work in image processing, especially for multi-modal images. Nowadays, the focus of image matching or registration has shifted to multi-modal and multi-source techniques, which are more critical to the practical applications. Multi-modal image matching is an essential step to implement data fusion, change detection, collaborative classification, joint analysis, and other image technologies.&#10;&#10;Image matching algorithms are generally divided into area-based methods and feature-based methods according to technical means \cite{zitova2003image}, in which the feature-based ones are the most widely applied nowadays. By locating keypoints in the same positions, and accurately matching them as far as possible through similar or invariant features, the spatial transformation between images is represented in a sparse way. This type of algorithm is rather a good solution to the automatic multi-modal images matching, which has been a hot spot in recent years.&#10;&#10;Chen et al. \cite{chen2010partial} proposed partial intensity invariant feature descriptor (PIIFD) based on scale-invariant feature transform (SIFT) \cite{lowe1999object,lowe2001local,lowe2004distinctive} for multi-source retinal images, which reduces intensity distortion problems. PSO-SIFT \cite{ma2016remote} uses second order gradient and an enhanced matching method for multi-source remote sensing images. Ye et al. developed histogram of orientated phase congruency (HOPC) \cite{ye2017robust,ye2018local}, in which an extended phase congruency model is designed for feature description. Radiation-variation insensitive feature transform (RIFT) \cite{li2019rift} utilizes a maximum index map (MIM) as the feature map using LogGabor filter, which has high invariance to intensity distortion. Multi-scale histogram of local main orientation (MS-HLMO) \cite{gao2022multi} comprehensively considers the differences of multi-source images, and proposed a framework focusing on orientation feature, which achieves intensity, rotation, and scale robustness on multi-modal remote sensing images. The key problems have emerged from these studies: 1) intensity distortion, 2) rotation, 3) scale difference, which are the difficulties of multi-modal image matching and need to be solved. The feature extracted from images should be robust to the above three aspects as possible.&#10;&#10;This research focuses on analyzing the invariance feature of multi-modal images, and a feature-based image matching algorithm is proposed, which uses phase congruency (PC) and Shi-Tomasi feature point for keypoints detection, LogGabor filter and a weighted partial main orientation map (WPMOM) for feature description, and a multi-scale process to deal with scale differences and optimize matching results. The experiments on remote sensing and medical data from multiple sources indicate that the proposed method has excellent, stable performance in multi-modal image registration, achieving accurate spatial alignment, which shows practical application value and good generalization ability.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Computer Science, Image Feature Extraction, Pattern Recognition, Multi-Modal Data Analysis, Computer Vision Algorithms, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2103.16871" label="2103.16871">
-        <attvalues>
-          <attvalue for="0" value="Robust Registration of Multimodal Remote Sensing Images Based on&#10;  Structural Similarity" />
-          <attvalue for="1" value="  Automatic registration of multimodal remote sensing data (e.g., optical,&#10;LiDAR, SAR) is a challenging task due to the significant non-linear radiometric&#10;differences between these data. To address this problem, this paper proposes a&#10;novel feature descriptor named the Histogram of Orientated Phase Congruency&#10;(HOPC), which is based on the structural properties of images. Furthermore, a&#10;similarity metric named HOPCncc is defined, which uses the normalized&#10;correlation coefficient (NCC) of the HOPC descriptors for multimodal&#10;registration. In the definition of the proposed similarity metric, we first&#10;extend the phase congruency model to generate its orientation representation,&#10;and use the extended model to build HOPCncc. Then a fast template matching&#10;scheme for this metric is designed to detect the control points between images.&#10;The proposed HOPCncc aims to capture the structural similarity between images,&#10;and has been tested with a variety of optical, LiDAR, SAR and map data. The&#10;results show that HOPCncc is robust against complex non-linear radiometric&#10;differences and outperforms the state-of-the-art similarities metrics (i.e.,&#10;NCC and mutual information) in matching performance. Moreover, a robust&#10;registration method is also proposed in this paper based on HOPCncc, which is&#10;evaluated using six pairs of multimodal remote sensing images. The experimental&#10;results demonstrate the effectiveness of the proposed method for multimodal&#10;image registration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.11757" label="2302.11757">
-        <attvalues>
-          <attvalue for="0" value="Open-World Object Detection via Discriminative Class Prototype Learning" />
-          <attvalue for="1" value="  Open-world object detection (OWOD) is a challenging problem that combines&#10;object detection with incremental learning and open-set learning. Compared to&#10;standard object detection, the OWOD setting is task to: 1) detect objects seen&#10;during training while identifying unseen classes, and 2) incrementally learn&#10;the knowledge of the identified unknown objects when the corresponding&#10;annotations is available. We propose a novel and efficient OWOD solution from a&#10;prototype perspective, which we call OCPL: Open-world object detection via&#10;discriminative Class Prototype Learning, which consists of a Proposal Embedding&#10;Aggregator (PEA), an Embedding Space Compressor (ESC) and a Cosine&#10;Similarity-based Classifier (CSC). All our proposed modules aim to learn the&#10;discriminative embeddings of known classes in the feature space to minimize the&#10;overlapping distributions of known and unknown classes, which is beneficial to&#10;differentiate known and unknown classes. Extensive experiments performed on&#10;PASCAL VOC and MS-COCO benchmark demonstrate the effectiveness of our proposed&#10;method.&#10;" />
-          <attvalue for="2" value="&#10;The rapid development of deep learning in recent years has significantly improved the performance of object detection, which is to identify and localize regions of interest in an image. But almost all existing object detectors consider the close-world assumption that the test sets and training sets contain the same data categories \cite{redmon2016you,lin2017focal,ren2015faster,tian2019fcos}. In practical applications, the test set may emerge classes not seen in the training set. Therefore, &#10;open-world object detection was proposed to solve the challenge that the model not only needs to correctly detect known classes, but also recognize unknown classes \cite{joseph2021towards}. Furthermore the detector progressively learn the new knowledge once identified unknown categories are annotated. &#10;&#10;The identification of unknown classes in traditional object detection pipeline faces significant challenges. Dhamija et al. were the first to formalize the open-set object detection problem \cite{dhamija2020overlooked}. They revealed that even State-of-the-art object detectors also result in false positive detection in the open-set environment. Complementing the benchmark for open-set object detection, Joseph et al. proposed the first open-world object detection model, named ORE \cite{joseph2021towards}, based on the Faster-RCNN \cite{ren2015faster}. Since the annotations of unknown object are unavailable, ORE introduces the auto-labelling unknowns module to obtain a weakly supervised set of unknown objects. &#10;&#10;Although ORE was the first to introduce and explore the challenging OWOD paradigm, it suffers from several defects. 1) The effect of the auto-labelling mechanism is negligible. It is difficult for the model to obtain supervision of unknown samples from this module and may cause confusion between background and unknown classes. 2) Training an energy-based classifier in ORE requires a fully annotated dataset, which contains label information of known and unknown classes. Obviously, ORE violates the principle of open-world object detection, where only annotations of known classes are available during training. &#10;&#10;Motivated by the above observations, we intend to build an open-world object detection pipeline that does not require pseudo-supervised for unknown classes and illogical extra datasets for unknown classes. In fact, Region Proposal Network (RPN) can extract proposals with high objectness scores, which contain both known and unknown classes. In other words, If the overlap of the distributions of unknown and known classes can be largely avoided, we can differentiate them well. Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes. Proposal Embedding Aggregator (PEA) and Embedding Space Compressor (ESC) are proposed to make compact clusters. Furthermore, we also introduce a &#10;Cosine Similarity-Based classifier (CSC) to help form tighter clusters \cite{wang2018cosface}. We summarize our contributions as follows:&#10;&#10;1. We are the first to explore the possibility of prototype ideology in open-world object detection.&#10;&#10;2. We introduce a prototype branch with Proposal Embedding Aggregator and Embedding Space Compressor which are adopted to separate and squeeze the feature distribution of each known class respectively. In order to form more compact class clusters, we introduce a Cosine Similarity-based Classifier.&#10;&#10;3. Extensive experiments on the OWOD benchmark confirm that our proposed OCPL is satisfactory. Specifically, the proposed method outperforms the recently proposed ORE on most metrics.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Incremental Learning, Computer Science, Open-Set Classification" />
-        </attvalues>
-      </node>
-      <node id="1904.01355" label="1904.01355">
-        <attvalues>
-          <attvalue for="0" value="FCOS: Fully Convolutional One-Stage Object Detection" />
-          <attvalue for="1" value="  We propose a fully convolutional one-stage object detector (FCOS) to solve&#10;object detection in a per-pixel prediction fashion, analogue to semantic&#10;segmentation. Almost all state-of-the-art object detectors such as RetinaNet,&#10;SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast,&#10;our proposed detector FCOS is anchor box free, as well as proposal free. By&#10;eliminating the predefined set of anchor boxes, FCOS completely avoids the&#10;complicated computation related to anchor boxes such as calculating overlapping&#10;during training. More importantly, we also avoid all hyper-parameters related&#10;to anchor boxes, which are often very sensitive to the final detection&#10;performance. With the only post-processing non-maximum suppression (NMS), FCOS&#10;with ResNeXt-64x4d-101 achieves 44.7% in AP with single-model and single-scale&#10;testing, surpassing previous one-stage detectors with the advantage of being&#10;much simpler. For the first time, we demonstrate a much simpler and flexible&#10;detection framework achieving improved detection accuracy. We hope that the&#10;proposed FCOS framework can serve as a simple and strong alternative for many&#10;other instance-level tasks. Code is available at:Code is available at:&#10;https://tinyurl.com/FCOSv1&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.03438" label="1805.03438">
-        <attvalues>
-          <attvalue for="0" value="Robust Classification with Convolutional Prototype Learning" />
-          <attvalue for="1" value="  Convolutional neural networks (CNNs) have been widely used for image&#10;classification. Despite its high accuracies, CNN has been shown to be easily&#10;fooled by some adversarial examples, indicating that CNN is not robust enough&#10;for pattern classification. In this paper, we argue that the lack of robustness&#10;for CNN is caused by the softmax layer, which is a totally discriminative model&#10;and based on the assumption of closed world (i.e., with a fixed number of&#10;categories). To improve the robustness, we propose a novel learning framework&#10;called convolutional prototype learning (CPL). The advantage of using&#10;prototypes is that it can well handle the open world recognition problem and&#10;therefore improve the robustness. Under the framework of CPL, we design&#10;multiple classification criteria to train the network. Moreover, a prototype&#10;loss (PL) is proposed as a regularization to improve the intra-class&#10;compactness of the feature representation, which can be viewed as a generative&#10;model based on the Gaussian assumption of different classes. Experiments on&#10;several datasets demonstrate that CPL can achieve comparable or even better&#10;results than traditional CNN, and from the robustness perspective, CPL shows&#10;great advantages for both the rejection and incremental category learning&#10;tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.02434" label="2004.02434">
-        <attvalues>
-          <attvalue for="0" value="Class Anchor Clustering: a Loss for Distance-based Open Set Recognition" />
-          <attvalue for="1" value="  In open set recognition, deep neural networks encounter object classes that&#10;were unknown during training. Existing open set classifiers distinguish between&#10;known and unknown classes by measuring distance in a network's logit space,&#10;assuming that known classes cluster closer to the training data than unknown&#10;classes. However, this approach is applied post-hoc to networks trained with&#10;cross-entropy loss, which does not guarantee this clustering behaviour. To&#10;overcome this limitation, we introduce the Class Anchor Clustering (CAC) loss.&#10;CAC is a distance-based loss that explicitly trains known classes to form tight&#10;clusters around anchored class-dependent centres in the logit space. We show&#10;that training with CAC achieves state-of-the-art performance for distance-based&#10;open set classifiers on all six standard benchmark datasets, with a 15.2% AUROC&#10;increase on the challenging TinyImageNet, without sacrificing classification&#10;accuracy. We also show that our anchored class centres achieve higher open set&#10;performance than learnt class centres, particularly on object-based datasets&#10;and large numbers of training classes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.00953" label="2103.00953">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Reciprocal Points Learning for Open Set Recognition" />
-          <attvalue for="1" value="  Open set recognition (OSR), aiming to simultaneously classify the seen&#10;classes and identify the unseen classes as 'unknown', is essential for reliable&#10;machine learning.The key challenge of OSR is how to reduce the empirical&#10;classification risk on the labeled known data and the open space risk on the&#10;potential unknown data simultaneously. To handle the challenge, we formulate&#10;the open space risk problem from the perspective of multi-class integration,&#10;and model the unexploited extra-class space with a novel concept Reciprocal&#10;Point. Follow this, a novel learning framework, termed Adversarial Reciprocal&#10;Point Learning (ARPL), is proposed to minimize the overlap of known&#10;distribution and unknown distributions without loss of known classification&#10;accuracy. Specifically, each reciprocal point is learned by the extra-class&#10;space with the corresponding known category, and the confrontation among&#10;multiple known categories are employed to reduce the empirical classification&#10;risk. Then, an adversarial margin constraint is proposed to reduce the open&#10;space risk by limiting the latent open space constructed by reciprocal points.&#10;To further estimate the unknown distribution from open space, an instantiated&#10;adversarial enhancement method is designed to generate diverse and confusing&#10;training samples, based on the adversarial mechanism between the reciprocal&#10;points and known classes. This can effectively enhance the model&#10;distinguishability to the unknown classes. Extensive experimental results on&#10;various benchmark datasets indicate that the proposed method is significantly&#10;superior to other existing approaches and achieves state-of-the-art&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.11013" label="2110.11013">
-        <attvalues>
-          <attvalue for="0" value="Spatial Location Constraint Prototype Loss for Open Set Recognition" />
-          <attvalue for="1" value="  One of the challenges in pattern recognition is open set recognition.&#10;Compared with closed set recognition, open set recognition needs to reduce not&#10;only the empirical risk, but also the open space risk, and the reduction of&#10;these two risks corresponds to classifying the known classes and identifying&#10;the unknown classes respectively. How to reduce the open space risk is the key&#10;of open set recognition. This paper explores the origin of the open space risk&#10;by analyzing the distribution of known and unknown classes features. On this&#10;basis, the spatial location constraint prototype loss function is proposed to&#10;reduce the two risks simultaneously. Extensive experiments on multiple&#10;benchmark datasets and many visualization results indicate that our methods is&#10;superior to most existing approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.00178" label="2011.00178">
-        <attvalues>
-          <attvalue for="0" value="Learning Open Set Network with Discriminative Reciprocal Points" />
-          <attvalue for="1" value="  Open set recognition is an emerging research area that aims to simultaneously&#10;classify samples from predefined classes and identify the rest as 'unknown'. In&#10;this process, one of the key challenges is to reduce the risk of generalizing&#10;the inherent characteristics of numerous unknown samples learned from a small&#10;amount of known data. In this paper, we propose a new concept, Reciprocal&#10;Point, which is the potential representation of the extra-class space&#10;corresponding to each known category. The sample can be classified to known or&#10;unknown by the otherness with reciprocal points. To tackle the open set&#10;problem, we offer a novel open space risk regularization term. Based on the&#10;bounded space constructed by reciprocal points, the risk of unknown is reduced&#10;through multi-category interaction. The novel learning framework called&#10;Reciprocal Point Learning (RPL), which can indirectly introduce the unknown&#10;information into the learner with only known classes, so as to learn more&#10;compact and discriminative representations. Moreover, we further construct a&#10;new large-scale challenging aircraft dataset for open set recognition: Aircraft&#10;300 (Air-300). Extensive experiments on multiple benchmark datasets indicate&#10;that our framework is significantly superior to other existing approaches and&#10;achieves state-of-the-art performance on standard open set benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.09414" label="1801.09414">
-        <attvalues>
-          <attvalue for="0" value="CosFace: Large Margin Cosine Loss for Deep Face Recognition" />
-          <attvalue for="1" value="  Face recognition has made extraordinary progress owing to the advancement of&#10;deep convolutional neural networks (CNNs). The central task of face&#10;recognition, including face verification and identification, involves face&#10;feature discrimination. However, the traditional softmax loss of deep CNNs&#10;usually lacks the power of discrimination. To address this problem, recently&#10;several loss functions such as center loss, large margin softmax loss, and&#10;angular softmax loss have been proposed. All these improved losses share the&#10;same idea: maximizing inter-class variance and minimizing intra-class variance.&#10;In this paper, we propose a novel loss function, namely large margin cosine&#10;loss (LMCL), to realize this idea from a different perspective. More&#10;specifically, we reformulate the softmax loss as a cosine loss by $L_2$&#10;normalizing both features and weight vectors to remove radial variations, based&#10;on which a cosine margin term is introduced to further maximize the decision&#10;margin in the angular space. As a result, minimum intra-class variance and&#10;maximum inter-class variance are achieved by virtue of normalization and cosine&#10;decision margin maximization. We refer to our model trained with LMCL as&#10;CosFace. Extensive experimental evaluations are conducted on the most popular&#10;public-domain face recognition datasets such as MegaFace Challenge, Youtube&#10;Faces (YTF) and Labeled Face in the Wild (LFW). We achieve the state-of-the-art&#10;performance on these benchmarks, which confirms the effectiveness of our&#10;proposed approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.12706" label="1911.12706">
-        <attvalues>
-          <attvalue for="0" value="Cameras Viewing Cameras Geometry" />
-          <attvalue for="1" value="  A basic problem in computer vision is to understand the structure of a&#10;real-world scene given several images of it. Here we study several theoretical&#10;aspects of the intra multi-view geometry of calibrated cameras when all that&#10;they can reliably recognize is each other. With the proliferation of wearable&#10;cameras, autonomous vehicles and drones, the geometry of these multiple cameras&#10;is a timely and relevant problem to study.&#10;" />
-          <attvalue for="2" value="&#10;&#10;A basic problem in computer vision is to understand the structure of a real-world scene given several images of it.&#10;This goes under the rubric of multi-view geometry or SLAM, simultaneous localization and mapping.&#10;With the proliferation of wearable cameras, autonomous vehicles and drones, the geometry of these multiple cameras is a timely and relevant problem to study. &#10;Here we study several theoretical aspects of the intra multi-view geometry of calibrated cameras when all that they can reliably recognize is each other.&#10;&#10;We treat both the general 3D case as well as the restricted 2D setup as it is often an adequate, simpler and more robust model for people and vehicles restricted to a planar surface.&#10;&#10;Previous work includes using the images of other cameras to help&#10; reduce the number of required corresponding points to compute epipolar geometry, \cite{8451727,satorecovering} and the vast literature on multiview geometry and pose estimation.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Multi-View Geometry, Mathematics, Physics, Camera Calibration, Computer Science, Geometry, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1810.09496" label="1810.09496">
-        <attvalues>
-          <attvalue for="0" value="Two view constraints on the epipoles from few correspondences" />
-          <attvalue for="1" value="  In general it requires at least 7 point correspondences to compute the&#10;fundamental matrix between views. We use the cross ratio invariance between&#10;corresponding epipolar lines, stemming from epipolar line homography, to derive&#10;a simple formulation for the relationship between epipoles and corresponding&#10;points. We show how it can be used to reduce the number of required points for&#10;the epipolar geometry when some information about the epipoles is available and&#10;demonstrate this with a buddy search app.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.11104" label="2404.11104">
-        <attvalues>
-          <attvalue for="0" value="Object Remover Performance Evaluation Methods using Class-wise Object&#10;  Removal Images" />
-          <attvalue for="1" value="  Object removal refers to the process of erasing designated objects from an&#10;image while preserving the overall appearance, and it is one area where image&#10;inpainting is widely used in real-world applications. The performance of an&#10;object remover is quantitatively evaluated by measuring the quality of object&#10;removal results, similar to how the performance of an image inpainter is&#10;gauged. Current works reporting quantitative performance evaluations utilize&#10;original images as references. In this letter, to validate the current&#10;evaluation methods cannot properly evaluate the performance of an object&#10;remover, we create a dataset with object removal ground truth and compare the&#10;evaluations made by the current methods using original images to those&#10;utilizing object removal ground truth images. The disparities between two&#10;evaluation sets validate that the current methods are not suitable for&#10;measuring the performance of an object remover. Additionally, we propose new&#10;evaluation methods tailored to gauge the performance of an object remover. The&#10;proposed methods evaluate the performance through class-wise object removal&#10;results and utilize images without the target class objects as a comparison&#10;set. We confirm that the proposed methods can make judgments consistent with&#10;human evaluators in the COCO dataset, and that they can produce measurements&#10;aligning with those using object removal ground truth in the self-acquired&#10;dataset.&#10;" />
-          <attvalue for="2" value="&#10;O{bject} removal is one area where image inpainting is extensively used in real-world applications. Unlike assessing the quality of inpainted images, gauging the quality of object removal results poses considerable challenges. This arises from the fact that an original image and an object removal result differ in the presence of the removal target. Consequently, most previous works \cite{liu2018image, yi2020contextual, cao2021learning, rombach2022high, zeng2020high, suvorov2022resolution, zeng2022aggregated, 9195134} reporting object removal results present only qualitative results. Only a few, such as \cite{angah2020removal,kottler20223gan,9446636}, quantitatively evaluate the quality using full-reference (FR) methods which employ original images as references. In this letter, we demonstrate that the FR methods used in the previous works are not suitable for evaluating object removal results and introduce novel FR methods designed for evaluating the performance of an object remover.&#10;&#10;FR methods can be categorized into two groups. LPIPS \cite{zhang2018unreasonable}, PSNR \cite{wang2004image}, SSIM \cite{wang2004image}, and P-IDS \cite{zhao2021comodgan} require a pair of original and completed images, while FID \cite{heusel2017gans} and U-IDS \cite{zhao2021comodgan} can be measured using unpaired data. To validate whether the paired or unpaired data methods can properly evaluate the performance with original images, we generate a dataset with object removal ground truth (GT) using virtual environments. Then, we compare the evaluations made by the FR methods using the original images with those utilizing the object removal GT images. As shown in Fig. \ref{fig:evaluation_metric}, we confirm the disparities between the two evaluation sets, which indicates that the FR methods using original images as references cannot properly measure the quality of object removal results. We report details about the experiment in Section \ref{section:existing}. &#10;&#10;We propose new unpaired methods to properly judge which object remover performs better in the absence of object removal GT. The proposed evaluation methods assess the performance of an object remover using the class-wise object removal tasks, where all objects in an image belonging to a target class are designated as removal targets. The proposed methods utilize images that do not contain an object of a target class for the comparison. In this scenario, both activation vectors obtained from object removal results and those obtained from the comparison set do not have features from target class objects. Therefore, unpaired methods can focus on evaluating whether the object removal results are as realistic as the comparison set. We demonstrate that the proposed methods can produce evaluations consistent with those made by humans in the COCO dataset, and that they can generate measurements aligning with those using object removal GT in the self-acquired virtual environment dataset. The proposed framework has the following contributions:&#10;\begin{itemize}&#10; \item We experimentally validate that the current object remover evaluation methods, which utilize original images as references, cannot properly measure the performance. To compare the evaluations made by the FR methods using original images to those utilizing object removal ground truth images, we generate a dataset with object removal ground truth images. &#10;&#10;\item The proposed methods measure the performance of an object remover using class-wise object removal results and the comparison set composed of images without target class objects. The proposed methods can evaluate the performance of an object remover without generating object removal ground truth.&#10;&#10;\item Experiments on the images from various environments demonstrate that the proposed methods can properly evaluate the object remover performance regardless of the input images’ style. This suggests that the proposed methods can be applied to model selection during object remover training or performance assessment of off-the-shelf object removers, which enables the training or selecting models that produce superior object removal results for query images from various environments.&#10;\end{itemize}&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Image Inpainting, Object Removal Evaluation, Image Processing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1804.07723" label="1804.07723">
-        <attvalues>
-          <attvalue for="0" value="Image Inpainting for Irregular Holes Using Partial Convolutions" />
-          <attvalue for="1" value="  Existing deep learning based image inpainting methods use a standard&#10;convolutional network over the corrupted image, using convolutional filter&#10;responses conditioned on both valid pixels as well as the substitute values in&#10;the masked holes (typically the mean value). This often leads to artifacts such&#10;as color discrepancy and blurriness. Post-processing is usually used to reduce&#10;such artifacts, but are expensive and may fail. We propose the use of partial&#10;convolutions, where the convolution is masked and renormalized to be&#10;conditioned on only valid pixels. We further include a mechanism to&#10;automatically generate an updated mask for the next layer as part of the&#10;forward pass. Our model outperforms other methods for irregular masks. We show&#10;qualitative and quantitative comparisons with other methods to validate our&#10;approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.10428" label="2103.10428">
-        <attvalues>
-          <attvalue for="0" value="Large Scale Image Completion via Co-Modulated Generative Adversarial&#10;  Networks" />
-          <attvalue for="1" value="  Numerous task-specific variants of conditional generative adversarial&#10;networks have been developed for image completion. Yet, a serious limitation&#10;remains that all existing algorithms tend to fail when handling large-scale&#10;missing regions. To overcome this challenge, we propose a generic new approach&#10;that bridges the gap between image-conditional and recent modulated&#10;unconditional generative architectures via co-modulation of both conditional&#10;and stochastic style representations. Also, due to the lack of good&#10;quantitative metrics for image completion, we propose the new Paired/Unpaired&#10;Inception Discriminative Score (P-IDS/U-IDS), which robustly measures the&#10;perceptual fidelity of inpainted images compared to real images via linear&#10;separability in a feature space. Experiments demonstrate superior performance&#10;in terms of both quality and diversity over state-of-the-art methods in&#10;free-form image completion and easy generalization to image-to-image&#10;translation. Code is available at https://github.com/zsyzzsoft/co-mod-gan.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.08500" label="1706.08500">
-        <attvalues>
-          <attvalue for="0" value="GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash&#10;  Equilibrium" />
-          <attvalue for="1" value="  Generative Adversarial Networks (GANs) excel at creating realistic images&#10;with complex models for which maximum likelihood is infeasible. However, the&#10;convergence of GAN training has still not been proved. We propose a two&#10;time-scale update rule (TTUR) for training GANs with stochastic gradient&#10;descent on arbitrary GAN loss functions. TTUR has an individual learning rate&#10;for both the discriminator and the generator. Using the theory of stochastic&#10;approximation, we prove that the TTUR converges under mild assumptions to a&#10;stationary local Nash equilibrium. The convergence carries over to the popular&#10;Adam optimization, for which we prove that it follows the dynamics of a heavy&#10;ball with friction and thus prefers flat minima in the objective landscape. For&#10;the evaluation of the performance of GANs at image generation, we introduce the&#10;&quot;Fr\'echet Inception Distance&quot; (FID) which captures the similarity of generated&#10;images to real ones better than the Inception Score. In experiments, TTUR&#10;improves learning for DCGANs and Improved Wasserstein GANs (WGAN-GP)&#10;outperforming conventional GAN training on CelebA, CIFAR-10, SVHN, LSUN&#10;Bedrooms, and the One Billion Word Benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.09195" label="2209.09195">
-        <attvalues>
-          <attvalue for="0" value="Constrained Sampling for Class-Agnostic Weakly Supervised Object&#10;  Localization" />
-          <attvalue for="1" value="  Self-supervised vision transformers can generate accurate localization maps&#10;of the objects in an image. However, since they decompose the scene into&#10;multiple maps containing various objects, and they do not rely on any explicit&#10;supervisory signal, they cannot distinguish between the object of interest from&#10;other objects, as required in weakly-supervised object localization (WSOL). To&#10;address this issue, we propose leveraging the multiple maps generated by the&#10;different transformer heads to acquire pseudo-labels for training a WSOL model.&#10;In particular, a new discriminative proposals sampling method is introduced&#10;that relies on a pretrained CNN classifier to identify discriminative regions.&#10;Then, foreground and background pixels are sampled from these regions in order&#10;to train a WSOL model for generating activation maps that can accurately&#10;localize objects belonging to a specific class. Empirical results on the&#10;challenging CUB benchmark dataset indicate that our proposed approach can&#10;outperform state-of-art methods over a wide range of threshold values. Our&#10;method provides class activation maps with a better coverage of foreground&#10;object regions w.r.t. the background.&#10;" />
-          <attvalue for="2" value="&#10;Weakly supervised learning (WSL) is a paradigm that allows training machine learning models from datasets with incomplete, inexact, and ambiguous labels~\cite{zhou2018brief}. In computer vision, WSL has received much recent attention for the object localization task, where training typically requires costly pixel-wise annotations. In particularly, weakly supervised object localization (WSOL) involved training ML models to localize objects of interest using data with only image-class labels. Class Activation Mapping (CAM) methods are a popular method for WSOL, allowing to highlight image regions linked to a particular class using features from the penultimate layer~\cite{zhou2016learning}. However, CAMs provides coarse information on object location for a given input image, often resulting in poor localization performance because they tend to focus on the most discriminative regions shared among objects of the same class~\cite{negevsbelharbi2022,belharbi2022f}. Moreover, CAM methods produce continuous activation maps that may be affected by complex background regions that match the concerned (foreground) object. Therefore, optimal thresholds must be selected to produce an accurate bounding box.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Vision Transformers, Mathematics, Artificial Intelligence, Computer Science, Object Localization, Weakly Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1512.04150" label="1512.04150">
-        <attvalues>
-          <attvalue for="0" value="Learning Deep Features for Discriminative Localization" />
-          <attvalue for="1" value="  In this work, we revisit the global average pooling layer proposed in [13],&#10;and shed light on how it explicitly enables the convolutional neural network to&#10;have remarkable localization ability despite being trained on image-level&#10;labels. While this technique was previously proposed as a means for&#10;regularizing training, we find that it actually builds a generic localizable&#10;deep representation that can be applied to a variety of tasks. Despite the&#10;apparent simplicity of global average pooling, we are able to achieve 37.1%&#10;top-5 error for object localization on ILSVRC 2014, which is remarkably close&#10;to the 34.2% top-5 error achieved by a fully supervised CNN approach. We&#10;demonstrate that our network is able to localize the discriminative image&#10;regions on a variety of tasks despite not being trained for them&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.02445" label="2201.02445">
-        <attvalues>
-          <attvalue for="0" value="Negative Evidence Matters in Interpretable Histology Image&#10;  Classification" />
-          <attvalue for="1" value="  Using only global image-class labels, weakly-supervised learning methods,&#10;such as class activation mapping, allow training CNNs to jointly classify an&#10;image, and locate regions of interest associated with the predicted class.&#10;However, without any guidance at the pixel level, such methods may yield&#10;inaccurate regions. This problem is known to be more challenging with histology&#10;images than with natural ones, since objects are less salient, structures have&#10;more variations, and foreground and background regions have stronger&#10;similarities. Therefore, computer vision methods for visual interpretation of&#10;CNNs may not directly apply. In this paper, a simple yet efficient method based&#10;on a composite loss is proposed to learn information from the fully negative&#10;samples (i.e., samples without positive regions), and thereby reduce false&#10;positives/negatives. Our new loss function contains two complementary terms:&#10;the first exploits positive evidence collected from the CNN classifier, while&#10;the second leverages the fully negative samples from training data. In&#10;particular, a pre-trained CNN is equipped with a decoder that allows refining&#10;the regions of interest. The CNN is exploited to collect both positive and&#10;negative evidence at the pixel level to train the decoder. Our method called&#10;NEGEV benefits from the fully negative samples that naturally occur in the&#10;data, without any additional supervision signals beyond image-class labels.&#10;Extensive experiments show that our proposed method can substantial outperform&#10;related state-of-art methods on GlaS (public benchmark for colon cancer), and&#10;Camelyon16 (patch-based benchmark for breast cancer using three different&#10;backbones). Our results highlight the benefits of using both positive and&#10;negative evidence, the first obtained from a classifier, and the other&#10;naturally available in datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.07069" label="2109.07069">
-        <attvalues>
-          <attvalue for="0" value="F-CAM: Full Resolution Class Activation Maps via Guided Parametric&#10;  Upscaling" />
-          <attvalue for="1" value="  Class Activation Mapping (CAM) methods have recently gained much attention&#10;for weakly-supervised object localization (WSOL) tasks. They allow for CNN&#10;visualization and interpretation without training on fully annotated image&#10;datasets. CAM methods are typically integrated within off-the-shelf CNN&#10;backbones, such as ResNet50. Due to convolution and pooling operations, these&#10;backbones yield low resolution CAMs with a down-scaling factor of up to 32,&#10;contributing to inaccurate localizations. Interpolation is required to restore&#10;full size CAMs, yet it does not consider the statistical properties of objects,&#10;such as color and texture, leading to activations with inconsistent boundaries,&#10;and inaccurate localizations. As an alternative, we introduce a generic method&#10;for parametric upscaling of CAMs that allows constructing accurate full&#10;resolution CAMs (F-CAMs). In particular, we propose a trainable decoding&#10;architecture that can be connected to any CNN classifier to produce highly&#10;accurate CAM localizations. Given an original low resolution CAM, foreground&#10;and background pixels are randomly sampled to fine-tune the decoder. Additional&#10;priors such as image statistics and size constraints are also considered to&#10;expand and refine object boundaries. Extensive experiments, over three CNN&#10;backbones and six WSOL baselines on the CUB-200-2011 and OpenImages datasets,&#10;indicate that our F-CAM method yields a significant improvement in CAM&#10;localization accuracy. F-CAM performance is competitive with state-of-art WSOL&#10;methods, yet it requires fewer computations during inference.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.09989" label="2004.09989">
-        <attvalues>
-          <attvalue for="0" value="Towards Generalization of 3D Human Pose Estimation In The Wild" />
-          <attvalue for="1" value="  In this paper, we propose 3DBodyTex.Pose, a dataset that addresses the task&#10;of 3D human pose estimation in-the-wild. Generalization to in-the-wild images&#10;remains limited due to the lack of adequate datasets. Existent ones are usually&#10;collected in indoor controlled environments where motion capture systems are&#10;used to obtain the 3D ground-truth annotations of humans. 3DBodyTex.Pose offers&#10;high quality and rich data containing 405 different real subjects in various&#10;clothing and poses, and 81k image samples with ground-truth 2D and 3D pose&#10;annotations. These images are generated from 200 viewpoints among which 70&#10;challenging extreme viewpoints. This data was created starting from high&#10;resolution textured 3D body scans and by incorporating various realistic&#10;backgrounds. Retraining a state-of-the-art 3D pose estimation approach using&#10;data augmented with 3DBodyTex.Pose showed promising improvement in the overall&#10;performance, and a sensible decrease in the per joint position error when&#10;testing on challenging viewpoints. The 3DBodyTex.Pose is expected to offer the&#10;research community with new possibilities for generalizing 3D pose estimation&#10;from monocular in-the-wild images.&#10;" />
-          <attvalue for="2" value="&#10;In the past couple of years, human pose estimation has received a lot of attention from the computer vision community. The goal is to estimate the 2D or 3D position of the human body joints given an image containing a human subject. This has a significant number of applications such as sports, healthcare solutions~\cite{baptista2019home}, action recognition~\cite{baptista2019home,demisse2018pose}, and animations.&#10;&#10;Due to the recent advances in Deep Neural Networks (DNN), the task of 2D human pose estimation a great improvement in results~\cite{chu2017multi,cao2018openpose,newell2016stacked}. This has been mostly achieved thanks to the availability of large-scale datasets containing 2D annotations of humans in many different conditions, e.g., in the wild~\cite{andriluka20142d}. In contrast, advances in the task of human pose estimation in 3D remains limited. The main reasons are the ambiguity of recovering the 3D information from a single image, in addition to the lack of large-scale datasets with 3D annotations of humans, specifically considering in-the-wild conditions. Existent datasets with 3D annotations are usually collected in a controlled environment using Motion Capture (MoCap) systems~\cite{h36m_pami}. Consequently, the variations in background and camera viewpoints remain limited. In addition, DNNs~\cite{zhou2016deep} trained on such datasets have difficulties generalizing well to environments where a lot of variation is present, e.g., scenarios in the wild. &#10;&#10;Recently, many works focused on the challenging problem of 3D human pose estimation in the wild~\cite{zhou2017towards,yang20183d,pavllo20193d,rogez2019lcr}. These works differ significantly from each other but share an important aspect. They are usually evaluated on the same dataset that has been used for training. Thus, it is possible that these approaches have been over-optimized for specific datasets, leading to a lack of generalization. It becomes difficult to judge on the generalization, and more precisely for in-the-wild scenarios where variations coming from the background and camera viewpoints are always present.&#10;&#10;In order to address the aforementioned challenge, this paper presents a new dataset referred to as 3DBodyTex.Pose. It is an original dataset generated from high-resolution textured 3D body scans, similar in quality to the ones contained in the 3DBodyTex dataset introduced in~\cite{Saint20183DBodyTexT3}. 3DBodyTex.Pose is dedicated to the task of human pose estimation. Synthetic scenes are generated with ground-truth information from real 3D body scans, with a large variation in subjects, clothing, and poses (see Fig.~\ref{fig:samples_dataset}). Realistic background is incorporated to the 3D environment. Finally, 2D images are generated from different camera viewpoints, including challenging ones, by virtually changing the camera location and orientation. We distinguish extreme viewpoints as the cases where the camera is, for example, placed on top of the subject. With the information contained in 3DBodyTex.Pose, it becomes possible to better generalize the problem of the 3D human pose estimation to in-the-wild images independently of the camera viewpoint as shown experimentally on a state-of-the-art 3D pose estimation approach~\cite{zhou2017towards}.&#10;In summary, the contributions of this work are: \\&#10;(1) 3DBodyTex.Pose, a synthetic dataset with 2D and 3D annotations of human poses considering in-the-wild images, with realistic background and standard to extreme camera viewpoints. This dataset will be publicly available for the research community.\\&#10;(2) Increasing the robustness of 3D human pose estimation algorithms, specifically~\cite{zhou2017towards}, with respect to challenging camera viewpoints thanks to data augmentation with 3DBodyTex.Pose. &#10;&#10;The rest of the paper is organized as follows: Section~\ref{sec:relateddatasets} describes the related datasets for the 3D human pose estimation task. Section~\ref{sec:proposeddataet} provides details about the proposed 3DBodyTex.Pose dataset and how it addresses the challenges of in-the-wild images and extreme camera viewpoints. Then, Section~\ref{sec:experiments} shows the conducted experiments, and finally Section~\ref{sec:conclusion} concludes this work.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1609.05317" label="1609.05317">
-        <attvalues>
-          <attvalue for="0" value="Deep Kinematic Pose Regression" />
-          <attvalue for="1" value="  Learning articulated object pose is inherently difficult because the pose is&#10;high dimensional but has many structural constraints. Most existing work do not&#10;model such constraints and does not guarantee the geometric validity of their&#10;pose estimation, therefore requiring a post-processing to recover the correct&#10;geometry if desired, which is cumbersome and sub-optimal. In this work, we&#10;propose to directly embed a kinematic object model into the deep neutral&#10;network learning for general articulated object pose estimation. The kinematic&#10;function is defined on the appropriately parameterized object motion variables.&#10;It is differentiable and can be used in the gradient descent based optimization&#10;in network training. The prior knowledge on the object geometric model is fully&#10;exploited and the structure is guaranteed to be valid. We show convincing&#10;experiment results on a toy example and the 3D human pose estimation problem.&#10;For the latter we achieve state-of-the-art result on Human3.6M dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.02447" label="1704.02447">
-        <attvalues>
-          <attvalue for="0" value="Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised&#10;  Approach" />
-          <attvalue for="1" value="  In this paper, we study the task of 3D human pose estimation in the wild.&#10;This task is challenging due to lack of training data, as existing datasets are&#10;either in the wild images with 2D pose or in the lab images with 3D pose.&#10;  We propose a weakly-supervised transfer learning method that uses mixed 2D&#10;and 3D labels in a unified deep neutral network that presents two-stage&#10;cascaded structure. Our network augments a state-of-the-art 2D pose estimation&#10;sub-network with a 3D depth regression sub-network. Unlike previous two stage&#10;approaches that train the two sub-networks sequentially and separately, our&#10;training is end-to-end and fully exploits the correlation between the 2D pose&#10;and depth estimation sub-tasks. The deep features are better learnt through&#10;shared representations. In doing so, the 3D pose labels in controlled lab&#10;environments are transferred to in the wild images. In addition, we introduce a&#10;3D geometric constraint to regularize the 3D pose prediction, which is&#10;effective in the absence of ground truth depth labels. Our method achieves&#10;competitive results on both 2D and 3D benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.09722" label="1803.09722">
-        <attvalues>
-          <attvalue for="0" value="3D Human Pose Estimation in the Wild by Adversarial Learning" />
-          <attvalue for="1" value="  Recently, remarkable advances have been achieved in 3D human pose estimation&#10;from monocular images because of the powerful Deep Convolutional Neural&#10;Networks (DCNNs). Despite their success on large-scale datasets collected in&#10;the constrained lab environment, it is difficult to obtain the 3D pose&#10;annotations for in-the-wild images. Therefore, 3D human pose estimation in the&#10;wild is still a challenge. In this paper, we propose an adversarial learning&#10;framework, which distills the 3D human pose structures learned from the fully&#10;annotated dataset to in-the-wild images with only 2D pose annotations. Instead&#10;of defining hard-coded rules to constrain the pose estimation results, we&#10;design a novel multi-source discriminator to distinguish the predicted 3D poses&#10;from the ground-truth, which helps to enforce the pose estimator to generate&#10;anthropometrically valid poses even with images in the wild. We also observe&#10;that a carefully designed information source for the discriminator is essential&#10;to boost the performance. Thus, we design a geometric descriptor, which&#10;computes the pairwise relative locations and distances between body joints, as&#10;a new information source for the discriminator. The efficacy of our adversarial&#10;learning framework with the new geometric descriptor has been demonstrated&#10;through extensive experiments on widely used public benchmarks. Our approach&#10;significantly improves the performance compared with previous state-of-the-art&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.00455" label="1803.00455">
-        <attvalues>
-          <attvalue for="0" value="LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images" />
-          <attvalue for="1" value="  We propose an end-to-end architecture for joint 2D and 3D human pose&#10;estimation in natural images. Key to our approach is the generation and scoring&#10;of a number of pose proposals per image, which allows us to predict 2D and 3D&#10;poses of multiple people simultaneously. Hence, our approach does not require&#10;an approximate localization of the humans for initialization. Our&#10;Localization-Classification-Regression architecture, named LCR-Net, contains 3&#10;main components: 1) the pose proposal generator that suggests candidate poses&#10;at different locations in the image; 2) a classifier that scores the different&#10;pose proposals; and 3) a regressor that refines pose proposals both in 2D and&#10;3D. All three stages share the convolutional feature layers and are trained&#10;jointly. The final pose estimation is obtained by integrating over neighboring&#10;pose hypotheses, which is shown to improve over a standard non maximum&#10;suppression algorithm. Our method recovers full-body 2D and 3D poses,&#10;hallucinating plausible body parts when the persons are partially occluded or&#10;truncated by the image boundary. Our approach significantly outperforms the&#10;state of the art in 3D pose estimation on Human3.6M, a controlled environment.&#10;Moreover, it shows promising results on real images for both single and&#10;multi-person subsets of the MPII 2D pose benchmark and demonstrates satisfying&#10;3D pose results even for multi-person images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.10424" label="2305.10424">
-        <attvalues>
-          <attvalue for="0" value="ZeroFlow: Scalable Scene Flow via Distillation" />
-          <attvalue for="1" value="  Scene flow estimation is the task of describing the 3D motion field between&#10;temporally successive point clouds. State-of-the-art methods use strong priors&#10;and test-time optimization techniques, but require on the order of tens of&#10;seconds to process full-size point clouds, making them unusable as computer&#10;vision primitives for real-time applications such as open world object&#10;detection. Feedforward methods are considerably faster, running on the order of&#10;tens to hundreds of milliseconds for full-size point clouds, but require&#10;expensive human supervision. To address both limitations, we propose Scene Flow&#10;via Distillation, a simple, scalable distillation framework that uses a&#10;label-free optimization method to produce pseudo-labels to supervise a&#10;feedforward model. Our instantiation of this framework, ZeroFlow, achieves&#10;state-of-the-art performance on the Argoverse 2 Self-Supervised Scene Flow&#10;Challenge while using zero human labels by simply training on large-scale,&#10;diverse unlabeled data. At test-time, ZeroFlow is over 1000x faster than&#10;label-free state-of-the-art optimization-based methods on full-size point&#10;clouds (34 FPS vs 0.028 FPS) and over 1000x cheaper to train on unlabeled data&#10;compared to the cost of human annotation (\$394 vs ~\$750,000). To facilitate&#10;further research, we release our code, trained model weights, and high quality&#10;pseudo-labels for the Argoverse 2 and Waymo Open datasets at&#10;https://vedder.io/zeroflow.html&#10;" />
-          <attvalue for="2" value="&#10;Scene flow estimation is an important primitive for open-world object detection and tracking~\cite{objectdetectionmotion, Zhai2020FlowMOT3M, baur2021slim, huang2022accumulation, flowssl}. As an example, \cite{objectdetectionmotion} generates supervisory boxes for an open-world LiDAR detector via offline object extraction using high quality scene flow estimates from Neural Scene Flow Prior (NSFP) \cite{nsfp}. Although NSFP does not require human supervision, it takes tens of seconds to run on a single full-size point cloud pair. If NSFP were both high quality and real-time, its estimations could be directly used as a runtime primitive in the downstream detector instead of relegated to an offline pipeline. This runtime feature formulation is similar to \cite{Zhai2020FlowMOT3M}'s use of scene flow from FlowNet3D~\cite{flownet3d} as an input primitive for their multi-object tracking pipeline; although FlowNet3D is fast enough for online processing of subsampled point clouds, its supervised feedforward formulation requires significant in-domain human annotations. &#10;&#10;Broadly, these exemplar methods are representative of the strengths and weakness of their class of approach. Supervised feedforward methods use human annotations which are expensive to annotate. To amortize these costs, human annotations are typically done on consecutive observations, severely limiting the structural diversity of the annotated scenes (e.g.\ a 15 second sequence from an Autonomous Vehicle typically only covers a single city block); due to costs and labeling difficulty, large-scale labels are also rarely even available outside of Autonomous Vehicle domains. Test-time optimization techniques circumvent the need for human labels by relying on hand-built priors, but they are too slow for online scene flow estimation.&#10;&#10;We propose \ourpipelinefull (\ourpipeline), a simple, scalable distillation framework that uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model. \ourpipeline generates a new class of scene flow estimation methods that combine the strengths of optimization-based and feedforward methods with the power of data scale and diversity to achieve fast run-time and superior accuracy without human supervision. We instantiate this pipeline into \ourmethodfull (\ourmethod), a family of methods that, motivated by real-world applications, can process full-size point clouds while providing high quality scene flow estimates. We demonstrate the strength of \ourmethod on Argoverse~2 \cite{argoverse2} and Waymo Open \cite{waymoopen}, notably achieving state-of-the-art on the Argoverse~2 Self-Supervised Scene Flow Challenge (gref{tradeoff_curve}).&#10;&#10;Our primary contributions include: &#10;\begin{itemize}[leftmargin=*]&#10; \item We introduce a simple yet effective distillation framework, \ourpipelinefull (\ourpipeline), which uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model, allowing us to surpass the performance of slow optimization-based approaches at the speed of feedforward methods.&#10; \item Using \ourpipeline, we present \ourmethodfull (\ourmethod), a family of &#10; methods that produce fast, state-of-the-art scene flow on full-size clouds, with methods running over 1000$\times$ faster than state-of-the-art optimization methods (29.33 ms for \ourmethodonex vs 35,281.4 ms for Chodosh) on real point clouds, while being over 1000$\times$ cheaper to train compared to the cost of human annotations (\$394 vs $\sim$\$750,000).&#10; \item We release high quality flow pseudo-labels (representing 7.1 GPU months of compute) for the popular Argoverse~2~\cite{argoverse2} and Waymo Open~\cite{waymoopen} autonomous vehicle datasets, alongside our code and trained model weights, to facilitate further research.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Scene Flow Estimation, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2210.08061" label="2210.08061">
-        <attvalues>
-          <attvalue for="0" value="Motion Inspired Unsupervised Perception and Prediction in Autonomous&#10;  Driving" />
-          <attvalue for="1" value="  Learning-based perception and prediction modules in modern autonomous driving&#10;systems typically rely on expensive human annotation and are designed to&#10;perceive only a handful of predefined object categories. This closed-set&#10;paradigm is insufficient for the safety-critical autonomous driving task, where&#10;the autonomous vehicle needs to process arbitrarily many types of traffic&#10;participants and their motion behaviors in a highly dynamic world. To address&#10;this difficulty, this paper pioneers a novel and challenging direction, i.e.,&#10;training perception and prediction models to understand open-set moving&#10;objects, with no human supervision. Our proposed framework uses self-learned&#10;flow to trigger an automated meta labeling pipeline to achieve automatic&#10;supervision. 3D detection experiments on the Waymo Open Dataset show that our&#10;method significantly outperforms classical unsupervised approaches and is even&#10;competitive to the counterpart with supervised scene flow. We further show that&#10;our approach generates highly promising results in open-set 3D detection and&#10;trajectory prediction, confirming its potential in closing the safety gap of&#10;fully supervised systems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.01253" label="2111.01253">
-        <attvalues>
-          <attvalue for="0" value="Neural Scene Flow Prior" />
-          <attvalue for="1" value="  Before the deep learning revolution, many perception algorithms were based on&#10;runtime optimization in conjunction with a strong prior/regularization penalty.&#10;A prime example of this in computer vision is optical and scene flow.&#10;Supervised learning has largely displaced the need for explicit regularization.&#10;Instead, they rely on large amounts of labeled data to capture prior&#10;statistics, which are not always readily available for many problems. Although&#10;optimization is employed to learn the neural network, the weights of this&#10;network are frozen at runtime. As a result, these learning solutions are&#10;domain-specific and do not generalize well to other statistically different&#10;scenarios. This paper revisits the scene flow problem that relies predominantly&#10;on runtime optimization and strong regularization. A central innovation here is&#10;the inclusion of a neural scene flow prior, which uses the architecture of&#10;neural networks as a new type of implicit regularizer. Unlike learning-based&#10;scene flow methods, optimization occurs at runtime, and our approach needs no&#10;offline datasets -- making it ideal for deployment in new environments such as&#10;autonomous driving. We show that an architecture based exclusively on&#10;multilayer perceptrons (MLPs) can be used as a scene flow prior. Our method&#10;attains competitive -- if not better -- results on scene flow benchmarks. Also,&#10;our neural prior's implicit and continuous scene flow representation allows us&#10;to estimate dense long-term correspondences across a sequence of point clouds.&#10;The dense motion information is represented by scene flow fields where points&#10;can be propagated through time by integrating motion vectors. We demonstrate&#10;such a capability by accumulating a sequence of lidar point clouds.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.07541" label="2012.07541">
-        <attvalues>
-          <attvalue for="0" value="FlowMOT: 3D Multi-Object Tracking by Scene Flow Association" />
-          <attvalue for="1" value="  Most end-to-end Multi-Object Tracking (MOT) methods face the problems of low&#10;accuracy and poor generalization ability. Although traditional filter-based&#10;methods can achieve better results, they are difficult to be endowed with&#10;optimal hyperparameters and often fail in varying scenarios. To alleviate these&#10;drawbacks, we propose a LiDAR-based 3D MOT framework named FlowMOT, which&#10;integrates point-wise motion information with the traditional matching&#10;algorithm, enhancing the robustness of the motion prediction. We firstly&#10;utilize a scene flow estimation network to obtain implicit motion information&#10;between two adjacent frames and calculate the predicted detection for each old&#10;tracklet in the previous frame. Then we use Hungarian algorithm to generate&#10;optimal matching relations with the ID propagation strategy to finish the&#10;tracking task. Experiments on KITTI MOT dataset show that our approach&#10;outperforms recent end-to-end methods and achieves competitive performance with&#10;the state-of-the-art filter-based method. In addition, ours can work steadily&#10;in the various-speed scenarios where the filter-based methods may fail.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01411" label="1806.01411">
-        <attvalues>
-          <attvalue for="0" value="FlowNet3D: Learning Scene Flow in 3D Point Clouds" />
-          <attvalue for="1" value="  Many applications in robotics and human-computer interaction can benefit from&#10;understanding 3D motion of points in a dynamic environment, widely noted as&#10;scene flow. While most previous methods focus on stereo and RGB-D images as&#10;input, few try to estimate scene flow directly from point clouds. In this work,&#10;we propose a novel deep neural network named $FlowNet3D$ that learns scene flow&#10;from point clouds in an end-to-end fashion. Our network simultaneously learns&#10;deep hierarchical features of point clouds and flow embeddings that represent&#10;point motions, supported by two newly proposed learning layers for point sets.&#10;We evaluate the network on both challenging synthetic data from FlyingThings3D&#10;and real Lidar scans from KITTI. Trained on synthetic data only, our network&#10;successfully generalizes to real scans, outperforming various baselines and&#10;showing competitive results to the prior art. We also demonstrate two&#10;applications of our scene flow output (scan registration and motion&#10;segmentation) to show its potential wide use cases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.00493" label="2301.00493">
-        <attvalues>
-          <attvalue for="0" value="Argoverse 2: Next Generation Datasets for Self-Driving Perception and&#10;  Forecasting" />
-          <attvalue for="1" value="  We introduce Argoverse 2 (AV2) - a collection of three datasets for&#10;perception and forecasting research in the self-driving domain. The annotated&#10;Sensor Dataset contains 1,000 sequences of multimodal data, encompassing&#10;high-resolution imagery from seven ring cameras, and two stereo cameras in&#10;addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain&#10;3D cuboid annotations for 26 object categories, all of which are&#10;sufficiently-sampled to support training and evaluation of 3D perception&#10;models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point&#10;clouds and map-aligned pose. This dataset is the largest ever collection of&#10;lidar sensor data and supports self-supervised learning and the emerging task&#10;of point cloud forecasting. Finally, the Motion Forecasting Dataset contains&#10;250,000 scenarios mined for interesting and challenging interactions between&#10;the autonomous vehicle and other actors in each local scene. Models are tasked&#10;with the prediction of future motion for &quot;scored actors&quot; in each scenario and&#10;are provided with track histories that capture object location, heading,&#10;velocity, and category. In all three datasets, each scenario contains its own&#10;HD Map with 3D lane and crosswalk geometry - sourced from data captured in six&#10;distinct cities. We believe these datasets will support new and existing&#10;machine learning research problems in ways that existing datasets do not. All&#10;datasets are released under the CC BY-NC-SA 4.0 license.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.04838" label="1912.04838">
-        <attvalues>
-          <attvalue for="0" value="Scalability in Perception for Autonomous Driving: Waymo Open Dataset" />
-          <attvalue for="1" value="  The research community has increasing interest in autonomous driving&#10;research, despite the resource intensity of obtaining representative real world&#10;data. Existing self-driving datasets are limited in the scale and variation of&#10;the environments they capture, even though generalization within and between&#10;operating regions is crucial to the overall viability of the technology. In an&#10;effort to help align the research community's contributions with real-world&#10;self-driving problems, we introduce a new large scale, high quality, diverse&#10;dataset. Our new dataset consists of 1150 scenes that each span 20 seconds,&#10;consisting of well synchronized and calibrated high quality LiDAR and camera&#10;data captured across a range of urban and suburban geographies. It is 15x more&#10;diverse than the largest camera+LiDAR dataset available based on our proposed&#10;diversity metric. We exhaustively annotated this data with 2D (camera image)&#10;and 3D (LiDAR) bounding boxes, with consistent identifiers across frames.&#10;Finally, we provide strong baselines for 2D as well as 3D detection and&#10;tracking tasks. We further study the effects of dataset size and generalization&#10;across geographies on 3D detection methods. Find data, code and more up-to-date&#10;information at http://www.waymo.com/open.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.01436" label="2407.01436">
-        <attvalues>
-          <attvalue for="0" value="AdaOcc: Adaptive Forward View Transformation and Flow Modeling for 3D&#10;  Occupancy and Flow Prediction" />
-          <attvalue for="1" value="  In this technical report, we present our solution for the Vision-Centric 3D&#10;Occupancy and Flow Prediction track in the nuScenes Open-Occ Dataset Challenge&#10;at CVPR 2024. Our innovative approach involves a dual-stage framework that&#10;enhances 3D occupancy and flow predictions by incorporating adaptive forward&#10;view transformation and flow modeling. Initially, we independently train the&#10;occupancy model, followed by flow prediction using sequential frame&#10;integration. Our method combines regression with classification to address&#10;scale variations in different scenes, and leverages predicted flow to warp&#10;current voxel features to future frames, guided by future frame ground truth.&#10;Experimental results on the nuScenes dataset demonstrate significant&#10;improvements in accuracy and robustness, showcasing the effectiveness of our&#10;approach in real-world scenarios. Our single model based on Swin-Base ranks&#10;second on the public leaderboard, validating the potential of our method in&#10;advancing autonomous car perception systems.&#10;" />
-          <attvalue for="2" value="&#10;&#10;3D occupancy and flow prediction \cite{tong2023scene,wang2023openoccupancy,tian2024occ3d,liu2023fully} are critical components of autonomous driving perception systems. They involve determining the occupancy status, semantic class, and future position of each voxel in a 3D voxel space. These predictions provide rich semantic and geometric information, crucial for understanding and navigating complex driving environments. The CVPR24 occupancy and flow prediction competition \cite{tong2023scene} focuses on developing new algorithms that predict occupancy and flow solely from camera input during inference, offering a significant platform for advancing state-of-the-art 3D occupancy and flow prediction algorithms.&#10;&#10;Our approach in this competition emphasizes innovative model design. We developed a two-stage framework to separately predict occupancy and flow. In the first stage, we train the occupancy model independently. We propose an adaptive forward view transformation method to enhance the adaptability of depth-based LSS. In the second stage, we train the flow model based on the occupancy model from the first stage. We introduced a novel sequential prediction method, using adjacent frames as inputs for the flow network. We combined regression with classification to predict flow, addressing the issue of varying flow scales in different scenes. Additionally, we use the predicted flow to warp the current frame's voxel features to the future frame and supervise using the future frame's ground truth, further enhancing prediction accuracy.&#10;&#10;Our approach achieves an Occ Score of 0.453 without any post-hoc process, achieving 2nd place in this challenge.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Autonomous Car Perception, Computer Science, Computer Vision Modeling, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2306.02851" label="2306.02851">
-        <attvalues>
-          <attvalue for="0" value="Scene as Occupancy" />
-          <attvalue for="1" value="  Human driver can easily describe the complex traffic scene by visual system.&#10;Such an ability of precise perception is essential for driver's planning. To&#10;achieve this, a geometry-aware representation that quantizes the physical 3D&#10;scene into structured grid map with semantic labels per cell, termed as 3D&#10;Occupancy, would be desirable. Compared to the form of bounding box, a key&#10;insight behind occupancy is that it could capture the fine-grained details of&#10;critical obstacles in the scene, and thereby facilitate subsequent tasks. Prior&#10;or concurrent literature mainly concentrate on a single scene completion task,&#10;where we might argue that the potential of this occupancy representation might&#10;obsess broader impact. In this paper, we propose OccNet, a multi-view&#10;vision-centric pipeline with a cascade and temporal voxel decoder to&#10;reconstruct 3D occupancy. At the core of OccNet is a general occupancy&#10;embedding to represent 3D physical world. Such a descriptor could be applied&#10;towards a wide span of driving tasks, including detection, segmentation and&#10;planning. To validate the effectiveness of this new representation and our&#10;proposed algorithm, we propose OpenOcc, the first dense high-quality 3D&#10;occupancy benchmark built on top of nuScenes. Empirical experiments show that&#10;there are evident performance gain across multiple tasks, e.g., motion planning&#10;could witness a collision rate reduction by 15%-58%, demonstrating the&#10;superiority of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;3D object detection &#10;\cite{Shi2020PVRCNNPF,wang2021fcos3d,li2022bevformer,liu2022bevfusion} adopts &#10;3D boxes as the objective of perception in AD &#10;since the box-form is well structured for downstream rule-based approaches.&#10;Such a representation abstracts 3D objects with different shapes into standardized cuboids, hence only cares about foreground objects and oversimplifies object shape.&#10;In contrast, 3D occupancy is a fine-grained description of the physical world &#10;and can differentiate objects with various shapes. &#10;&#10;LiDAR segmentation &#10;\cite{zhu2021cylindrical,milioto2019rangenet++} is tasked as point-level 3D scene understanding.&#10;It requires point cloud as input, which is expensive and less portable.&#10;Since LiDAR inherently suffers from limited sensing range and sparsity in 3D scene description, it is not friendly to holistic 3D scene semantic understanding~\cite{song2016ssc} using such a pipeline.&#10;&#10;3D reconstruction and rendering.&#10;Inferring the 3D geometry of objects or scenes from 2D images \cite{Hartley2004,mildenhall2020nerf} is prevailing yet challenging for many years&#10;in computer vision.&#10;Most approaches in this domain~\cite{mueller2022instant,cao2022scenerf,Tancik2022blocknerf} cope with a single object or scene. For AD application, &#10;this is not feasible since it requires strong generalization ability.&#10;Note that 3D reconstruction and rendering concentrates more on the quality of the scene geometry and visual appearance. It pays less attention to model efficiency and semantic understanding.&#10;&#10;Semantic Scene Completion. The definition of occupancy prediction discussed in this work shares the most resemblance with SSC \cite{song2016ssc}.&#10;MonoScene~\cite{cao2022monoscene} first adopts U-Net to infer from a single monocular RGB image the dense 3D occupancy with semantic labels. There is a burst of related works released in arXiv recently. We deem them as concurrent and briefly discuss below.&#10;VoxFormer \cite{li2023voxformer} utilizes the depth estimation to set voxel queries in a two-stage framework.&#10;OccDepth~\cite{miao2023occdepth} also adopts a depth-aware spirit in a stereo setting with distillation to predict semantic occupancy.&#10;TPVFormer~\cite{huang2023tri} employs LiDAR-based sparse 3D occupancy as the supervision and proposes a tri-perspective view representation to obtain features.&#10;Wang et al. \cite{wang2023openoccupancy} provides a well human-crafted occupancy benchmark that could facilitate the community.&#10;&#10;Despite different settings from ours with work conducted on&#10;Semantic-KITTI~\cite{behley2019semantickitti} and NYUv2~\cite{Silberman2012nyuv2} &#10;(monocular or RGB-D), &#10;prior or concurrent literature unanimously neglect the adoption of temporal context. Utilizing history voxel features is straightforward; it is verified by Tesla \cite{tesla_ai_day}. Yet there is no technical details or report to the public. Moreover, we position our work to be the first to investigate occupancy as a general descriptor that could enhance multiple tasks {beyond} detection. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2303.03991" label="2303.03991">
-        <attvalues>
-          <attvalue for="0" value="OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic&#10;  Occupancy Perception" />
-          <attvalue for="1" value="  Semantic occupancy perception is essential for autonomous driving, as&#10;automated vehicles require a fine-grained perception of the 3D urban&#10;structures. However, existing relevant benchmarks lack diversity in urban&#10;scenes, and they only evaluate front-view predictions. Towards a comprehensive&#10;benchmarking of surrounding perception algorithms, we propose OpenOccupancy,&#10;which is the first surrounding semantic occupancy perception benchmark. In the&#10;OpenOccupancy benchmark, we extend the large-scale nuScenes dataset with dense&#10;semantic occupancy annotations. Previous annotations rely on LiDAR points&#10;superimposition, where some occupancy labels are missed due to sparse LiDAR&#10;channels. To mitigate the problem, we introduce the Augmenting And Purifying&#10;(AAP) pipeline to ~2x densify the annotations, where ~4000 human hours are&#10;involved in the labeling process. Besides, camera-based, LiDAR-based and&#10;multi-modal baselines are established for the OpenOccupancy benchmark.&#10;Furthermore, considering the complexity of surrounding occupancy perception&#10;lies in the computational burden of high-resolution 3D predictions, we propose&#10;the Cascade Occupancy Network (CONet) to refine the coarse prediction, which&#10;relatively enhances the performance by ~30% than the baseline. We hope the&#10;OpenOccupancy benchmark will boost the development of surrounding occupancy&#10;perception algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.14365" label="2304.14365">
-        <attvalues>
-          <attvalue for="0" value="Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous&#10;  Driving" />
-          <attvalue for="1" value="  Robotic perception requires the modeling of both 3D geometry and semantics.&#10;Existing methods typically focus on estimating 3D bounding boxes, neglecting&#10;finer geometric details and struggling to handle general, out-of-vocabulary&#10;objects. 3D occupancy prediction, which estimates the detailed occupancy states&#10;and semantics of a scene, is an emerging task to overcome these limitations. To&#10;support 3D occupancy prediction, we develop a label generation pipeline that&#10;produces dense, visibility-aware labels for any given scene. This pipeline&#10;comprises three stages: voxel densification, occlusion reasoning, and&#10;image-guided voxel refinement. We establish two benchmarks, derived from the&#10;Waymo Open Dataset and the nuScenes Dataset, namely Occ3D-Waymo and&#10;Occ3D-nuScenes benchmarks. Furthermore, we provide an extensive analysis of the&#10;proposed dataset with various baseline models. Lastly, we propose a new model,&#10;dubbed Coarse-to-Fine Occupancy (CTF-Occ) network, which demonstrates superior&#10;performance on the Occ3D benchmarks. The code, data, and benchmarks are&#10;released at https://tsinghua-mars-lab.github.io/Occ3D/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.06349" label="2010.06349">
-        <attvalues>
-          <attvalue for="0" value="Collaborative Video Object Segmentation by Multi-Scale&#10;  Foreground-Background Integration" />
-          <attvalue for="1" value="  This paper investigates the principles of embedding learning to tackle the&#10;challenging semi-supervised video object segmentation. Unlike previous&#10;practices that focus on exploring the embedding learning of foreground object&#10;(s), we consider background should be equally treated. Thus, we propose a&#10;Collaborative video object segmentation by Foreground-Background Integration&#10;(CFBI) approach. CFBI separates the feature embedding into the foreground&#10;object region and its corresponding background region, implicitly promoting&#10;them to be more contrastive and improving the segmentation results accordingly.&#10;Moreover, CFBI performs both pixel-level matching processes and instance-level&#10;attention mechanisms between the reference and the predicted sequence, making&#10;CFBI robust to various object scales. Based on CFBI, we introduce a multi-scale&#10;matching structure and propose an Atrous Matching strategy, resulting in a more&#10;robust and efficient framework, CFBI+. We conduct extensive experiments on two&#10;popular benchmarks, i.e., DAVIS and YouTube-VOS. Without applying any simulated&#10;data for pre-training, our CFBI+ achieves the performance (J&amp;F) of 82.9% and&#10;82.8%, outperforming all the other state-of-the-art methods. Code:&#10;https://github.com/z-x-yang/CFBI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Semi-supervised Video Object Segmentation.&#10;Many previous methods for semi-supervised VOS rely on fine-tuning at test time. Among them, OSVOS~\cite{osvos} and MoNet~\cite{xiao2018monet} fine-tune the network on the first-frame ground-truth at test time. OnAVOS~\cite{onavos} extends the first-frame fine-tuning by an online adaptation mechanism, i.e., online fine-tuning. MaskTrack~\cite{masktrack} uses optical flow to propagate the segmentation mask from one frame to the next. PReMVOS~\cite{premvos} combines four different neural networks (including an optical flow network~\cite{flownet}) using extensive fine-tuning and a merging algorithm. Despite achieving promising results, all these methods are seriously slowed down by fine-tuning during inference.&#10;&#10;Some other recent works (e.g.,~\cite{osmn,favos}) aim to avoid fine-tuning and achieve a better run-time. OSMN~\cite{osmn} employs two networks to extract the instance-level information and make segmentation predictions, respectively. PML~\cite{pml} learns a pixel-wise embedding with the nearest neighbor classifier. Similar to PML, VideoMatch~\cite{videomatch} uses a soft matching layer that maps the pixels of the current frame to the first frame in a learned embedding space. Following PML and VideoMatch, FEELVOS~\cite{feelvos} extends the pixel-level matching mechanism by additionally matching between the current frame and the previous frame. Compared to fine-tuning methods, FEELVOS achieves a much higher speed, but there is still an accuracy gap. Like FEELVOS, RGMP~\cite{rgmp} and STMVOS~\cite{spacetime} does not require any fine-tuning. STMVOS, which leverages a memory network to store and read the information from past frames, outperforms all the previous methods. {However, STMVOS and its following works (EGMN~\cite{EGMN} and KMNVOS~\cite{KMN}) rely on an elaborate pre-training procedure using extensive simulated data generated from multiple datasets with pixel-level annotations. LWLVOS~\cite{LWLVOS} proposes to use an online few-shot learner during both training and testing stages. Without simulated data, LWLVOS is comparable with KMNVOS on YouTube-VOS, but generalizes worse than the above methods using simulated data on DAVIS.}&#10;&#10;{In previous practices, learning foreground feature embedding has been well explored. OSMN proposed to conduct an instance-level matching, but such a matching scheme fails to consider the feature diversity among the details of the target's appearance and results in coarse predictions. PML and FEELVOS alternatively adopt the pixel-level matching by matching each pixel of the target, which effectively takes the feature diversity into account and achieves promising performance. Nevertheless, performing pixel-level matching may bring unexpected noises in the case of some pixels from the background are with a similar appearance to the ones from the foreground (Fig.~\ref{fig:cl}).}&#10;&#10;{Thus, we propose a collaborative integration method by additionally learning background embedding. &#10;Furthermore, our CFBI utilizes both the pixel-level and instance-level embeddings to guide prediction.}&#10;&#10;Attention Mechanisms.&#10;Recent works introduce the attention mechanism into convolutional networks (e.g., ~\cite{attention_conv1,attention_conv2}). &#10;Following them, SE-Nets~\cite{senet} introduced a lightweight gating mechanism that focuses on enhancing the representational power of the convolutional network by modeling channel attention. Inspired by SE-Nets, CFBI uses an instance-level average pooling method to embed collaborative instance information from pixel-level embeddings. After that, we conduct a channel-wise attention mechanism to help guide prediction. Compared to OSMN, which employs an additional convolutional network to extract instance-level embedding, our instance-level attention method is more efficient and lightweight.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Embedding Learning Techniques, Semi-Supervised Learning, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2003.08333" label="2003.08333">
-        <attvalues>
-          <attvalue for="0" value="Collaborative Video Object Segmentation by Foreground-Background&#10;  Integration" />
-          <attvalue for="1" value="  This paper investigates the principles of embedding learning to tackle the&#10;challenging semi-supervised video object segmentation. Different from previous&#10;practices that only explore the embedding learning using pixels from foreground&#10;object (s), we consider background should be equally treated and thus propose&#10;Collaborative video object segmentation by Foreground-Background Integration&#10;(CFBI) approach. Our CFBI implicitly imposes the feature embedding from the&#10;target foreground object and its corresponding background to be contrastive,&#10;promoting the segmentation results accordingly. With the feature embedding from&#10;both foreground and background, our CFBI performs the matching process between&#10;the reference and the predicted sequence from both pixel and instance levels,&#10;making the CFBI be robust to various object scales. We conduct extensive&#10;experiments on three popular benchmarks, i.e., DAVIS 2016, DAVIS 2017, and&#10;YouTube-VOS. Our CFBI achieves the performance (J$F) of 89.4%, 81.9%, and&#10;81.4%, respectively, outperforming all the other state-of-the-art methods.&#10;Code: https://github.com/z-x-yang/CFBI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Semi-supervised Video Object Segmentation.&#10;Many previous methods for semi-supervised VOS rely on fine-tuning at test time. Among them, OSVOS~\cite{osvos} and MoNet~\cite{xiao2018monet} fine-tune the network on the first-frame ground-truth at test time. OnAVOS~\cite{onavos} extends the first-frame fine-tuning by an online adaptation mechanism, \ie, online fine-tuning. MaskTrack~\cite{masktrack} uses optical flow to propagate the segmentation mask from one frame to the next. PReMVOS~\cite{premvos} combines four different neural networks (including an optical flow network~\cite{flownet}) using extensive fine-tuning and a merging algorithm. Despite achieving promising results, all these methods are seriously slowed down by fine-tuning during inference.&#10;&#10;Some other recent works (\eg,~\cite{osmn,favos}) aim to avoid fine-tuning and achieve a better run-time. OSMN~\cite{osmn} employs two networks to extract the instance-level information and make segmentation predictions, respectively. PML~\cite{pml} learns a pixel-wise embedding with the nearest neighbor classifier. Similar to PML, VideoMatch~\cite{videomatch} uses a soft matching layer that maps the pixels of the current frame to the first frame in a learned embedding space. Following PML and VideoMatch, FEELVOS~\cite{feelvos} extends the pixel-level matching mechanism by additionally matching between the current frame and the previous frame. Compared to the methods with fine-tuning, FEELVOS achieves a much higher speed, but there is still a gap inaccuracy. Like FEELVOS, RGMP~\cite{rgmp} and STMVOS~\cite{spacetime} does not require any fine-tuning. STMVOS, which leverages a memory network to store and read the information from past frames, outperforms all the previous methods. However, STMVOS relies on an elaborate training procedure using extensive simulated data generated from multiple datasets. Moreover, the above methods do not focus on background matching.&#10;&#10;Our CFBI utilizes both the pixel-level and instance-level embeddings to guide prediction. Furthermore, we propose a collaborative integration method by additionally learning background embedding. &#10;&#10;Attention Mechanisms.&#10;Recent works introduce the attention mechanism into convolutional networks (\eg, ~\cite{attention_conv1,attention_conv2}). &#10;Following them, SE-Nets~\cite{senet} introduced a lightweight gating mechanism that focuses on enhancing the representational power of the convolutional network by modeling channel attention. Inspired by SE-Nets, CFBI uses an instance-level average pooling method to embed collaborative instance information from pixel-level embeddings. After that, we conduct a channel-wise attention mechanism to help guide prediction. Compared to OSMN, which employs an additional convolutional network to extract instance-level embedding, our instance-level attention method is more efficient and lightweight.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Embedding Learning Techniques, Semi-Supervised Learning, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2210.11170" label="2210.11170">
-        <attvalues>
-          <attvalue for="0" value="Coordinates Are NOT Lonely -- Codebook Prior Helps Implicit Neural 3D&#10;  Representations" />
-          <attvalue for="1" value="  Implicit neural 3D representation has achieved impressive results in surface&#10;or scene reconstruction and novel view synthesis, which typically uses the&#10;coordinate-based multi-layer perceptrons (MLPs) to learn a continuous scene&#10;representation. However, existing approaches, such as Neural Radiance Field&#10;(NeRF) and its variants, usually require dense input views (i.e. 50-150) to&#10;obtain decent results. To relive the over-dependence on massive calibrated&#10;images and enrich the coordinate-based feature representation, we explore&#10;injecting the prior information into the coordinate-based network and introduce&#10;a novel coordinate-based model, CoCo-INR, for implicit neural 3D&#10;representation. The cores of our method are two attention modules: codebook&#10;attention and coordinate attention. The former extracts the useful prototypes&#10;containing rich geometry and appearance information from the prior codebook,&#10;and the latter propagates such prior information into each coordinate and&#10;enriches its feature representation for a scene or object surface. With the&#10;help of the prior information, our method can render 3D views with more&#10;photo-realistic appearance and geometries than the current methods using fewer&#10;calibrated images available. Experiments on various scene reconstruction&#10;datasets, including DTU and BlendedMVS, and the full 3D head reconstruction&#10;dataset, H3DS, demonstrate the robustness under fewer input views and fine&#10;detail-preserving capability of our proposed method.&#10;" />
-          <attvalue for="2" value="&#10;&#10;3D scene reconstruction aims to recover the underlying scene geometry and appearance from a set of posed images. It is a fundamental problem in computer vision and graphics with wide usage in 3D object surface reconstruction and scene novel view synthesis. Recently, implicit neural representation (INR) with coordinated-based learning has been driving significant advantages in producing high-fidelity 3D shapes~\cite{OechsleP_2021_unisurf,Wang_2021_neus,yariv2021volsdf} and photo-realistic images~\cite{Mildenhall_2020_nerf,Martinbrualla2020nerfw,Barron2021mipnerf}. With the per-scene optimized multi-layer perceptrons (MLPs), such methods usually require a dense set of calibrated views to optimize a high-quality continuous 3D scene representation. However, their performance would degrade dramatically when the number of input views is very limited, because only spatial coordinate information is insufficient for the network to characterize a continuous 3D scene with few input views. It is practical to overcome this drawback by introducing additional high-level semantic descriptors of the context scene as the prior information, to enrich the feature representation of each coordinate and compensate for the missing views when only sparse views are provided. &#10;&#10;Fortunately, learning the prior descriptors or prototypes from a large scale dataset is a long-studied topic from the early Bag-of-Words (BoW) in image classification, scene recognition~\cite{li_2005_bow_scene_recog} and visual SLAM~\cite{Kim_2011_slam}, Vector Quantized (VQ) codebook in data compression~\cite{Perlmutter_1994_vq}, to the recent neural discrete codebook representation in image synthesis~\cite{van_2017_vqvae,van_2019_vqvae2,Esser_2021_taming}. These methods learn and maintain a representative codebook, with each codeword denoting a specific prototype from the dataset for serving as a prior to encode new images or scenes. However, how to leverage and correlate the codebook prior with the implicit neural representation is still an unreached area.&#10;&#10;This paper investigates a novel coordinate-based deep model that injects the prior codebook information into the implicit neural 3D representation. To achieve this, we first design a codebook attention module to extract the valuable prototypes from the prior codebook for each scene. These prototypes might contain rich geometry or appearance information, with parts of them being shared and others being exclusive for different views. The shared prototypes in the codebook guarantee the network to generate view-consistency results. In contrast, the exclusive prototypes for each view bring additional information for the network to enrich its feature representation and produce photo-realistic rendering results and decent geometry by leveraging fewer posed images. Then, we equip each coordinate with the extracted prototypes to improve its feature representation via a coordinate attention module. They are both the attention-based modules, as depicted in Fig.~\ref{fig:coord}. The former queries the per-scene relevant prototypes from the codebook prior via the learnable embeddings, and the latter enables each coordinate to query representative features from the prototypes. With the help of prototypes, each coordinate is not alone, resulting in a more robust representation and improving the quality of rendering images. &#10;&#10;We perform extensive experiments on different scene reconstruction benchmarks, including DTU~\cite{Jensen_2014_dtu}, BlendedMVS~\cite{Yao_2020_blendmvs} and the full head reconstruction benchmark H3DS~\cite{Ramon_2021_h3ds}. Our approach outperforms current state-of-the-art methods~\cite{Mildenhall_2020_nerf,OechsleP_2021_unisurf,yariv2021volsdf} under a sparse set of input views. We summarize the contributions as follows: 1) we propose a novel coordinate-based network that incorporates codebook prior for implicit neural representations, which builds a connection between the coordinates and the codebook prior information for the first time; 2) with the help of the codebook prior, our method could improve the feature representation of each coordinate and is more robust in producing realistic-looking novel views with fewer images; 3) extensive experiments demonstrate the effectiveness of our CoCo-INR, including the robustness in generalizing to novel views under few inputs images and the capability to produce more photo-realistic results under sparse input views. The code is available at &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Scene Reconstruction, Neural, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2104.10078" label="2104.10078">
-        <attvalues>
-          <attvalue for="0" value="UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for&#10;  Multi-View Reconstruction" />
-          <attvalue for="1" value="  Neural implicit 3D representations have emerged as a powerful paradigm for&#10;reconstructing surfaces from multi-view images and synthesizing novel views.&#10;Unfortunately, existing methods such as DVR or IDR require accurate per-pixel&#10;object masks as supervision. At the same time, neural radiance fields have&#10;revolutionized novel view synthesis. However, NeRF's estimated volume density&#10;does not admit accurate surface reconstruction. Our key insight is that&#10;implicit surface models and radiance fields can be formulated in a unified way,&#10;enabling both surface and volume rendering using the same model. This unified&#10;perspective enables novel, more efficient sampling procedures and the ability&#10;to reconstruct accurate surfaces without input masks. We compare our method on&#10;the DTU, BlendedMVS, and a synthetic indoor dataset. Our experiments&#10;demonstrate that we outperform NeRF in terms of reconstruction quality while&#10;performing on par with IDR without requiring masks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.13415" label="2103.13415">
-        <attvalues>
-          <attvalue for="0" value="Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance&#10;  Fields" />
-          <attvalue for="1" value="  The rendering procedure used by neural radiance fields (NeRF) samples a scene&#10;with a single ray per pixel and may therefore produce renderings that are&#10;excessively blurred or aliased when training or testing images observe scene&#10;content at different resolutions. The straightforward solution of supersampling&#10;by rendering with multiple rays per pixel is impractical for NeRF, because&#10;rendering each ray requires querying a multilayer perceptron hundreds of times.&#10;Our solution, which we call &quot;mip-NeRF&quot; (a la &quot;mipmap&quot;), extends NeRF to&#10;represent the scene at a continuously-valued scale. By efficiently rendering&#10;anti-aliased conical frustums instead of rays, mip-NeRF reduces objectionable&#10;aliasing artifacts and significantly improves NeRF's ability to represent fine&#10;details, while also being 7% faster than NeRF and half the size. Compared to&#10;NeRF, mip-NeRF reduces average error rates by 17% on the dataset presented with&#10;NeRF and by 60% on a challenging multiscale variant of that dataset that we&#10;present. Mip-NeRF is also able to match the accuracy of a brute-force&#10;supersampled NeRF on our multiscale dataset while being 22x faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.00446" label="1906.00446">
-        <attvalues>
-          <attvalue for="0" value="Generating Diverse High-Fidelity Images with VQ-VAE-2" />
-          <attvalue for="1" value="  We explore the use of Vector Quantized Variational AutoEncoder (VQ-VAE)&#10;models for large scale image generation. To this end, we scale and enhance the&#10;autoregressive priors used in VQ-VAE to generate synthetic samples of much&#10;higher coherence and fidelity than possible before. We use simple feed-forward&#10;encoder and decoder networks, making our model an attractive candidate for&#10;applications where the encoding and/or decoding speed is critical.&#10;Additionally, VQ-VAE requires sampling an autoregressive model only in the&#10;compressed latent space, which is an order of magnitude faster than sampling in&#10;the pixel space, especially for large images. We demonstrate that a multi-scale&#10;hierarchical organization of VQ-VAE, augmented with powerful priors over the&#10;latent codes, is able to generate samples with quality that rivals that of&#10;state of the art Generative Adversarial Networks on multifaceted datasets such&#10;as ImageNet, while not suffering from GAN's known shortcomings such as mode&#10;collapse and lack of diversity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.09841" label="2012.09841">
-        <attvalues>
-          <attvalue for="0" value="Taming Transformers for High-Resolution Image Synthesis" />
-          <attvalue for="1" value="  Designed to learn long-range interactions on sequential data, transformers&#10;continue to show state-of-the-art results on a wide variety of tasks. In&#10;contrast to CNNs, they contain no inductive bias that prioritizes local&#10;interactions. This makes them expressive, but also computationally infeasible&#10;for long sequences, such as high-resolution images. We demonstrate how&#10;combining the effectiveness of the inductive bias of CNNs with the expressivity&#10;of transformers enables them to model and thereby synthesize high-resolution&#10;images. We show how to (i) use CNNs to learn a context-rich vocabulary of image&#10;constituents, and in turn (ii) utilize transformers to efficiently model their&#10;composition within high-resolution images. Our approach is readily applied to&#10;conditional synthesis tasks, where both non-spatial information, such as object&#10;classes, and spatial information, such as segmentations, can control the&#10;generated image. In particular, we present the first results on&#10;semantically-guided synthesis of megapixel images with transformers and obtain&#10;the state of the art among autoregressive models on class-conditional ImageNet.&#10;Code and pretrained models can be found at&#10;https://github.com/CompVis/taming-transformers .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.10127" label="1911.10127">
-        <attvalues>
-          <attvalue for="0" value="BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo&#10;  Networks" />
-          <attvalue for="1" value="  While deep learning has recently achieved great success on multi-view stereo&#10;(MVS), limited training data makes the trained model hard to be generalized to&#10;unseen scenarios. Compared with other computer vision tasks, it is rather&#10;difficult to collect a large-scale MVS dataset as it requires expensive active&#10;scanners and labor-intensive process to obtain ground truth 3D structures. In&#10;this paper, we introduce BlendedMVS, a novel large-scale dataset, to provide&#10;sufficient training ground truth for learning-based MVS. To create the dataset,&#10;we apply a 3D reconstruction pipeline to recover high-quality textured meshes&#10;from images of well-selected scenes. Then, we render these mesh models to color&#10;images and depth maps. To introduce the ambient lighting information during&#10;training, the rendered color images are further blended with the input images&#10;to generate the training input. Our dataset contains over 17k high-resolution&#10;images covering a variety of scenes, including cities, architectures,&#10;sculptures and small objects. Extensive experiments demonstrate that BlendedMVS&#10;endows the trained model with significantly better generalization ability&#10;compared with other MVS datasets. The dataset and pretrained models are&#10;available at \url{https://github.com/YoYo000/BlendedMVS}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.12512" label="2107.12512">
-        <attvalues>
-          <attvalue for="0" value="H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction" />
-          <attvalue for="1" value="  Recent learning approaches that implicitly represent surface geometry using&#10;coordinate-based neural representations have shown impressive results in the&#10;problem of multi-view 3D reconstruction. The effectiveness of these techniques&#10;is, however, subject to the availability of a large number (several tens) of&#10;input views of the scene, and computationally demanding optimizations. In this&#10;paper, we tackle these limitations for the specific problem of few-shot full 3D&#10;head reconstruction, by endowing coordinate-based representations with a&#10;probabilistic shape prior that enables faster convergence and better&#10;generalization when using few input images (down to three). First, we learn a&#10;shape model of 3D heads from thousands of incomplete raw scans using implicit&#10;representations. At test time, we jointly overfit two coordinate-based neural&#10;networks to the scene, one modeling the geometry and another estimating the&#10;surface radiance, using implicit differentiable rendering. We devise a&#10;two-stage optimization strategy in which the learned prior is used to&#10;initialize and constrain the geometry during an initial optimization phase.&#10;Then, the prior is unfrozen and fine-tuned to the scene. By doing this, we&#10;achieve high-fidelity head reconstructions, including hair and shoulders, and&#10;with a high level of detail that consistently outperforms both state-of-the-art&#10;3D Morphable Models methods in the few-shot scenario, and non-parametric&#10;methods when large sets of views are available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.04055" label="2104.04055">
-        <attvalues>
-          <attvalue for="0" value="Generative Landmarks" />
-          <attvalue for="1" value="  We propose a general purpose approach to detect landmarks with improved&#10;temporal consistency, and personalization. Most sparse landmark detection&#10;methods rely on laborious, manually labelled landmarks, where inconsistency in&#10;annotations over a temporal volume leads to sub-optimal landmark learning.&#10;Further, high-quality landmarks with personalization is often hard to achieve.&#10;We pose landmark detection as an image translation problem. We capture two sets&#10;of unpaired marked (with paint) and unmarked videos. We then use a generative&#10;adversarial network and cyclic consistency to predict deformations of landmark&#10;templates that simulate markers on unmarked images until these images are&#10;indistinguishable from ground-truth marked images. Our novel method does not&#10;rely on manually labelled priors, is temporally consistent, and image class&#10;agnostic -- face, and hand landmarks detection examples are shown.&#10;" />
-          <attvalue for="2" value="&#10;Sparse landmarks detection is an important problem for face detection applications \cite{amos2016openface}, face tracking with landmarks alignment as a sub-task for 3D face model fitting~\cite{bickel2007multi, dib2021practical} or to guide video synthesis for faces ~\cite{wang2020facevid2vid}, other body parts~\cite{chan2019everybody}, among several others. These tasks rely on high-quality and temporally consistent landmarks; however, off-the-shelf landmark detection methods suffer from inconsistencies due to ambiguity in manual landmark annotations as well as temporal imperfections of frame-to-frame labeling, as landmarks are difficult to define precisely, see \cite{dong2018supervision,wu2018look} for a discussion. As a result, landmark detection models suffer from temporal jitters, and sub-optimal personalization. Wu et al.~\cite{wu2018look} approach this problem with a focus on the boundary, taking advantage of well defined face boundary lines along which the landmarks reside. Dong et. al.~\cite{dong2018supervision} note that frame-to-frame landmark detection should ideally resemble the presence of physical markers and present an approach that uses optical flow, and later a triangulation-based approach ~\cite{dong2020supervision} that exploits the temporal information inherent in video data.&#10;\\&#10;With the goal of temporally consistent and personalized landmark detection, we propose a method that involves capture of two sets of unpaired videos for a given body region: one in which semantically (e.g. eyes, nose, fingers, etc.) meaningful lines are visibly marked, and the other, unmarked. We predict the landmark deformations for a template, for each unmarked image and render them, such that, it resembles the marked images. Following Zhu et al.~\cite{zhu2017unpaired}, we pose this problem as an unpaired image translation problem, with an image generator network that translates from marked-to-unmarked images, while our novel landmark deformation prediction network performs the reverse translation. Thus, our method is capable of learning a set of predefined landmarks in an unsupervised fashion, circumventing the need for laborious and imprecise manual annotations, while providing landmarks that are inherently personalized and temporally stable. &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Translation, Machine Learning, Engineering, Computer Vision, Generative Modeling, Mathematics, Artificial Intelligence, Landmark Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2011.15126" label="2011.15126">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing" />
-          <attvalue for="1" value="  We propose a neural talking-head video synthesis model and demonstrate its&#10;application to video conferencing. Our model learns to synthesize a&#10;talking-head video using a source image containing the target person's&#10;appearance and a driving video that dictates the motion in the output. Our&#10;motion is encoded based on a novel keypoint representation, where the&#10;identity-specific and motion-related information is decomposed unsupervisedly.&#10;Extensive experimental validation shows that our model outperforms competing&#10;methods on benchmark datasets. Moreover, our compact keypoint representation&#10;enables a video conferencing system that achieves the same visual quality as&#10;the commercial H.264 standard while only using one-tenth of the bandwidth.&#10;Besides, we show our keypoint representation allows the user to rotate the head&#10;during synthesis, which is useful for simulating face-to-face video&#10;conferencing experiences.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.07371" label="1808.07371">
-        <attvalues>
-          <attvalue for="0" value="Everybody Dance Now" />
-          <attvalue for="1" value="  This paper presents a simple method for &quot;do as I do&quot; motion transfer: given a&#10;source video of a person dancing, we can transfer that performance to a novel&#10;(amateur) target after only a few minutes of the target subject performing&#10;standard moves. We approach this problem as video-to-video translation using&#10;pose as an intermediate representation. To transfer the motion, we extract&#10;poses from the source subject and apply the learned pose-to-appearance mapping&#10;to generate the target subject. We predict two consecutive frames for&#10;temporally coherent video results and introduce a separate pipeline for&#10;realistic face synthesis. Although our method is quite simple, it produces&#10;surprisingly compelling results (see video). This motivates us to also provide&#10;a forensics tool for reliable synthetic content detection, which is able to&#10;distinguish videos synthesized by our system from real data. In addition, we&#10;release a first-of-its-kind open-source dataset of videos that can be legally&#10;used for training and motion transfer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.10483" label="1805.10483">
-        <attvalues>
-          <attvalue for="0" value="Look at Boundary: A Boundary-Aware Face Alignment Algorithm" />
-          <attvalue for="1" value="  We present a novel boundary-aware face alignment algorithm by utilising&#10;boundary lines as the geometric structure of a human face to help facial&#10;landmark localisation. Unlike the conventional heatmap based method and&#10;regression based method, our approach derives face landmarks from boundary&#10;lines which remove the ambiguities in the landmark definition. Three questions&#10;are explored and answered by this work: 1. Why using boundary? 2. How to use&#10;boundary? 3. What is the relationship between boundary estimation and landmarks&#10;localisation? Our boundary- aware face alignment algorithm achieves 3.49% mean&#10;error on 300-W Fullset, which outperforms state-of-the-art methods by a large&#10;margin. Our method can also easily integrate information from other datasets.&#10;By utilising boundary information of 300-W dataset, our method achieves 3.92%&#10;mean error with 0.39% failure rate on COFW dataset, and 1.25% mean error on&#10;AFLW-Full dataset. Moreover, we propose a new dataset WFLW to unify training&#10;and testing across different factors, including poses, expressions,&#10;illuminations, makeups, occlusions, and blurriness. Dataset and model will be&#10;publicly available at https://wywu.github.io/projects/LAB/LAB.html&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.09866" label="2101.09866">
-        <attvalues>
-          <attvalue for="0" value="Supervision by Registration and Triangulation for Landmark Detection" />
-          <attvalue for="1" value="  We present Supervision by Registration and Triangulation (SRT), an&#10;unsupervised approach that utilizes unlabeled multi-view video to improve the&#10;accuracy and precision of landmark detectors. Being able to utilize unlabeled&#10;data enables our detectors to learn from massive amounts of unlabeled data&#10;freely available and not be limited by the quality and quantity of manual human&#10;annotations. To utilize unlabeled data, there are two key observations: (1) the&#10;detections of the same landmark in adjacent frames should be coherent with&#10;registration, i.e., optical flow. (2) the detections of the same landmark in&#10;multiple synchronized and geometrically calibrated views should correspond to a&#10;single 3D point, i.e., multi-view consistency. Registration and multi-view&#10;consistency are sources of supervision that do not require manual labeling,&#10;thus it can be leveraged to augment existing training data during detector&#10;training. End-to-end training is made possible by differentiable registration&#10;and 3D triangulation modules. Experiments with 11 datasets and a newly proposed&#10;metric to measure precision demonstrate accuracy and precision improvements in&#10;landmark detection on both images and video. Code is available at&#10;https://github.com/D-X-Y/landmark-detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.04628" label="2112.04628">
-        <attvalues>
-          <attvalue for="0" value="Learning Auxiliary Monocular Contexts Helps Monocular 3D Object&#10;  Detection" />
-          <attvalue for="1" value="  Monocular 3D object detection aims to localize 3D bounding boxes in an input&#10;single 2D image. It is a highly challenging problem and remains open,&#10;especially when no extra information (e.g., depth, lidar and/or multi-frames)&#10;can be leveraged in training and/or inference. This paper proposes a simple yet&#10;effective formulation for monocular 3D object detection without exploiting any&#10;extra information. It presents the MonoCon method which learns Monocular&#10;Contexts, as auxiliary tasks in training, to help monocular 3D object&#10;detection. The key idea is that with the annotated 3D bounding boxes of objects&#10;in an image, there is a rich set of well-posed projected 2D supervision signals&#10;available in training, such as the projected corner keypoints and their&#10;associated offset vectors with respect to the center of 2D bounding box, which&#10;should be exploited as auxiliary tasks in training. The proposed MonoCon is&#10;motivated by the Cramer-Wold theorem in measure theory at a high level. In&#10;implementation, it utilizes a very simple end-to-end design to justify the&#10;effectiveness of learning auxiliary monocular contexts, which consists of three&#10;components: a Deep Neural Network (DNN) based feature backbone, a number of&#10;regression head branches for learning the essential parameters used in the 3D&#10;bounding box prediction, and a number of regression head branches for learning&#10;auxiliary contexts. After training, the auxiliary context regression branches&#10;are discarded for better inference efficiency. In experiments, the proposed&#10;MonoCon is tested in the KITTI benchmark (car, pedestrain and cyclist). It&#10;outperforms all prior arts in the leaderboard on car category and obtains&#10;comparable performance on pedestrian and cyclist in terms of accuracy. Thanks&#10;to the simple design, the proposed MonoCon method obtains the fastest inference&#10;speed with 38.7 fps in comparisons&#10;" />
-          <attvalue for="2" value="&#10;&#10;3D object detection is a critical component in many computer vision applications in practice, such as autonomous driving and robot navigation. High performing methods often require more costly system setups such as Lidar sensors \cite{lidar3, lidar4, lidar1, lidar2} for precise depth measurements or stereo cameras \cite{s1, s2, s3, s4} for stereo depth estimation, and are often more computationally expensive. To alleviate those ``burden&quot; and due to the potential prospects of reduced cost and increased modular redundancy, monocular 3D object detection that aims to localize 3D object bounding boxes from an input 2D image has emerged as a promising alternative approach with much attention received in the computer vision and AI community \cite{mono3d, roi10d, monodis, gs3d, m3drpn, rarnet, task3d, fda3d, dn3d, m3dssd, nms3d, fcos3d, pgd}. In addition to potential advantages in practice, developing powerful monocular 3D object detection systems will facilitate addressing a fundamental question in computer vision, that is whether it is possible to recover 3D structures from only 2D images which have lost the depth information in the first place. &#10;&#10;This paper is interested in 3D object detection in the autonomous driving application. The objective is to estimate the 3D bounding box for each object instance such as a car in a 2D image. In the KITTI benchmark \cite{kitti}, a 3D bounding box is parameterized by: (i) the 3D center location $(\mathbf{x}, \mathbf{y}, \mathbf{z})$ in the camera 3D coordinates (in meters), (ii) the observation angle $\alpha$ of the object with respect to the camera based on the vector from the camera center to the 3D object center, and (iii) the shape dimensions $(\mathbf{h}, \mathbf{w}, \mathbf{l})$, i.e., height, width and length (in meters).&#10;Based on the extensive and insightful analyses made by the MonoDLE method~\cite{monodle} in the KITTI benchmark, &#10;one main challenge of improving the overall performance in monocular 3D object detection lies in inferring the 3D center location with high accuracy. &#10; To address the challenge, there are two main types of settings in state-of-the-art monocular 3D object detection, depending on whether there are extra information (Lidar depth measurements, monocular depth estimation results by a separately trained model or multi-frames) leveraged in training and/or inference. In practice, the 3D center location $(\mathbf{x}, \mathbf{y}, \mathbf{z})$ is often decomposed to the projected 3D center in the image plane $(x_c, y_c)$ and the object depth $\mathbf{z}$. With the camera intrinsic matrix assumed to be known in both training and inference, the 3D location can be recovered with the inferred projected 3D center and object depth. &#10;&#10;This paper focuses on end-to-end monocular 3D object detection without exploiting any extra information. &#10;It adopts the anchor-offset formulation proposed in the CenterNet~\cite{centernet} in learning the projected 3D center based on the 2D bounding box center (i.e., the anchor), and proposes a simple yet effective method that facilitates better overall performance (Fig.~\ref{fig:comparisons}). The key idea is to leverage Monocular Contexts as auxiliary learning tasks in training to improve the performance (Fig.~\ref{fig:workflow}). The underlying rationale is that with the annotated 3D bounding boxes of objects in an image, there is a rich set of well-posed projected 2D supervision signals available in training, such as the projected corner keypoints and their associated offset vectors with respect to the anchor. They should be exploited in training to induce more expressive representations for monocular 3D object detection. The proposed method is thus dubbed as MonoCon. &#10;&#10;Statistically speaking, the monocular contexts can be treated as marginal random variables in the image plane, which are projected from the 3D bounding box random variables. In measure theory, the Cram\`er-Wold theorem~\cite{cramer1936some} states that a Borel probability measure on $\mathbb{R}^{k}$ is uniquely determined by the totality of its one-dimensional projections. Motivated by the Cram\`er-Wold theorem, the proposed MonoCon method introduces monocular projections as auxiliary tasks in training to learn more effective representations for monocular 3D object detection. In the meanwhile, it seeks a minimally-simple design of the overall detection system to justify the effectivenss of the underlying rationale and the high-level motivation. &#10;&#10;In implementation, the proposed MonoCon utilizes a very simple design consisting of three components (Fig.~\ref{fig:workflow}): a Deep Neural Network (DNN) based feature backbone, a number of regression head branches for learning the essential parameters used in the 3D bounding box prediction, and a number of regression head branches for learning auxiliary contexts. After training, the auxiliary context regression branches are discarded. In experiments, the proposed MonoCon is tested in the KITTI benchmark (car, pedestrian and cyclist) \cite{kitti}. It outperforms prior arts (including methods that use lidar, depth or multi-frame extra information) in the leaderboard on the car category and obtains comparable performance on pedestrian and cyclist in terms of accuracy. Thanks to the simple design, the proposed MonoCon obtains the fastest speed with 38.7 fps (on a single NVIDIA 2080Ti GPU card) in comparisons. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1902.09738" label="1902.09738">
-        <attvalues>
-          <attvalue for="0" value="Stereo R-CNN based 3D Object Detection for Autonomous Driving" />
-          <attvalue for="1" value="  We propose a 3D object detection method for autonomous driving by fully&#10;exploiting the sparse and dense, semantic and geometry information in stereo&#10;imagery. Our method, called Stereo R-CNN, extends Faster R-CNN for stereo&#10;inputs to simultaneously detect and associate object in left and right images.&#10;We add extra branches after stereo Region Proposal Network (RPN) to predict&#10;sparse keypoints, viewpoints, and object dimensions, which are combined with 2D&#10;left-right boxes to calculate a coarse 3D object bounding box. We then recover&#10;the accurate 3D bounding box by a region-based photometric alignment using left&#10;and right RoIs. Our method does not require depth input and 3D position&#10;supervision, however, outperforms all existing fully supervised image-based&#10;methods. Experiments on the challenging KITTI dataset show that our method&#10;outperforms the state-of-the-art stereo-based method by around 30% AP on both&#10;3D detection and 3D localization tasks. Code has been released at&#10;https://github.com/HKUST-Aerial-Robotics/Stereo-RCNN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.16237" label="2103.16237">
-        <attvalues>
-          <attvalue for="0" value="Delving into Localization Errors for Monocular 3D Object Detection" />
-          <attvalue for="1" value="  Estimating 3D bounding boxes from monocular images is an essential component&#10;in autonomous driving, while accurate 3D object detection from this kind of&#10;data is very challenging. In this work, by intensive diagnosis experiments, we&#10;quantify the impact introduced by each sub-task and found the `localization&#10;error' is the vital factor in restricting monocular 3D detection. Besides, we&#10;also investigate the underlying reasons behind localization errors, analyze the&#10;issues they might bring, and propose three strategies. First, we revisit the&#10;misalignment between the center of the 2D bounding box and the projected center&#10;of the 3D object, which is a vital factor leading to low localization accuracy.&#10;Second, we observe that accurately localizing distant objects with existing&#10;technologies is almost impossible, while those samples will mislead the learned&#10;network. To this end, we propose to remove such samples from the training set&#10;for improving the overall performance of the detector. Lastly, we also propose&#10;a novel 3D IoU oriented loss for the size estimation of the object, which is&#10;not affected by `localization error'. We conduct extensive experiments on the&#10;KITTI dataset, where the proposed method achieves real-time detection and&#10;outperforms previous methods by a large margin. The code will be made available&#10;at: https://github.com/xinzhuma/monodle.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.07850" label="1904.07850">
-        <attvalues>
-          <attvalue for="0" value="Objects as Points" />
-          <attvalue for="1" value="  Detection identifies objects as axis-aligned boxes in an image. Most&#10;successful object detectors enumerate a nearly exhaustive list of potential&#10;object locations and classify each. This is wasteful, inefficient, and requires&#10;additional post-processing. In this paper, we take a different approach. We&#10;model an object as a single point --- the center point of its bounding box. Our&#10;detector uses keypoint estimation to find center points and regresses to all&#10;other object properties, such as size, 3D location, orientation, and even pose.&#10;Our center point based approach, CenterNet, is end-to-end differentiable,&#10;simpler, faster, and more accurate than corresponding bounding box based&#10;detectors. CenterNet achieves the best speed-accuracy trade-off on the MS COCO&#10;dataset, with 28.1% AP at 142 FPS, 37.4% AP at 52 FPS, and 45.1% AP with&#10;multi-scale testing at 1.4 FPS. We use the same approach to estimate 3D&#10;bounding box in the KITTI benchmark and human pose on the COCO keypoint&#10;dataset. Our method performs competitively with sophisticated multi-stage&#10;methods and runs in real-time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.03375" label="1909.03375">
-        <attvalues>
-          <attvalue for="0" value="Robust Full-FoV Depth Estimation in Tele-wide Camera System" />
-          <attvalue for="1" value="  Tele-wide camera system with different Field of View (FoV) lenses becomes&#10;very popular in recent mobile devices. Usually it is difficult to obtain&#10;full-FoV depth based on traditional stereo-matching methods. Pure Deep Neural&#10;Network (DNN) based depth estimation methods can obtain full-FoV depth, but&#10;have low robustness for scenarios which are not covered by training dataset. In&#10;this paper, to address the above problems we propose a hierarchical hourglass&#10;network for robust full-FoV depth estimation in tele-wide camera system, which&#10;combines the robustness of traditional stereo-matching methods with the&#10;accuracy of DNN. More specifically, the proposed network comprises three major&#10;modules: single image depth prediction module infers initial depth from input&#10;color image, depth propagation module propagates traditional stereo-matching&#10;tele-FoV depth to surrounding regions, and depth combination module fuses the&#10;initial depth with the propagated depth to generate final output. Each of these&#10;modules employs an hourglass model, which is a kind of encoder-decoder&#10;structure with skip connections. Experimental results compared with&#10;state-of-the-art depth estimation methods demonstrate that our method not only&#10;produces robust and better subjective depth quality on wild test images, but&#10;also obtains better quantitative results on standard datasets.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Currently tele-wide camera system with different Field of View (FoV) lenses is very popular in mobile devices, wherein each lens has different FoV, e.g. wide angle lens, tele zoom lens etc. How to obtain robust full-FoV depth for this kind of camera system becomes a challenging problem.&#10;&#10;Usually it is difficult to obtain full-FoV depth by traditional stereo-matching methods \cite{Scharstein02:IJCV, Zhan16:CSVT}. This kind of method can be formulated as a three-step pipeline including matching cost calculation \cite{Hernandez16:PCS}, cost aggregation/optimization \cite{Hirschmuller08:PAMI, Zhang07:PAMI}, and disparity refinement \cite{Barron16:ECCV, Min14:TIP}.&#10;&#10;Recently Deep Neural Network (DNN) has been successfully applied to the single image depth prediction \cite{Eigen14:NIPS, Xie16:ECCV, Laina16:3DV, Godard17:CVPR, Fu18:CVPR, Li18:CVPR} and tele-wide stereo depth estimation \cite{El-Khamy19:CVPRW}. Eigen et al. \cite{Eigen14:NIPS} combined a coarse global prediction network based on the entire image with a refinement network, and proposed a scale-invariant L2-norm loss function which is widely utilized in depth prediction. Laina et al. \cite{Laina16:3DV} developed an up-projection decoder and achieved higher accuracy. Godard et al. \cite{Godard17:CVPR} proposed an unsupervised method to enforce disparity consistency between the left and right images, achieving similar depth quality compared with supervised methods. Li et al. \cite{Li18:CVPR} presented a large depth dataset called MegaDepth, which is generated from collected multi-view internet photos, the hourglass model trained on this dataset shows good results on wild test images. EI-Khamy et al. \cite{El-Khamy19:CVPRW} proposed a stereo matching neural network for tele-wide camera system so as to estimate the full-FoV depth.&#10;&#10;Pure DNN-based methods usually have low robustness for scenarios which are not covered by training dataset \cite{Nguyen18:ECCV, Su18:ECCV} (as shown in Fig. \ref{fig:expe_wild_statue} (d, e and f)). To address the above problems, we propose a hierarchical hourglass model, which incorporates the traditional tele-FoV stereo-matching depth as input, and estimate more robust full-FoV depth on various test scenarios (as shown in Fig. \ref{fig:expe_wild_statue} (g)). More specifically, the proposed network comprises three modules: single image depth prediction module that infers initial depth from input color image, depth propagation module that propagates traditional stereo-matching depth from tele-FoV to surrounding regions, and depth combination module that fuses the initial depth with the propagated depth. All of them employ the hourglass model \cite{Chen16:NIPS}, which has an encoder-decoder structure with skip connections. Experiments demonstrate that our method can not only get robust and better subjective depth quality than state-of-the-art depth estimation on wild test images, but also obtain better objective results on standard datasets.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Electrical Engineering, Depth Estimation, Computer Science, Neural Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1610.04121" label="1610.04121">
-        <attvalues>
-          <attvalue for="0" value="Embedded real-time stereo estimation via Semi-Global Matching on the GPU" />
-          <attvalue for="1" value="  Dense, robust and real-time computation of depth information from&#10;stereo-camera systems is a computationally demanding requirement for robotics,&#10;advanced driver assistance systems (ADAS) and autonomous vehicles. Semi-Global&#10;Matching (SGM) is a widely used algorithm that propagates consistency&#10;constraints along several paths across the image. This work presents a&#10;real-time system producing reliable disparity estimation results on the new&#10;embedded energy-efficient GPU devices. Our design runs on a Tegra X1 at 42&#10;frames per second (fps) for an image size of 640x480, 128 disparity levels, and&#10;using 4 path directions for the SGM method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.03296" label="1511.03296">
-        <attvalues>
-          <attvalue for="0" value="The Fast Bilateral Solver" />
-          <attvalue for="1" value="  We present the bilateral solver, a novel algorithm for edge-aware smoothing&#10;that combines the flexibility and speed of simple filtering approaches with the&#10;accuracy of domain-specific optimization algorithms. Our technique is capable&#10;of matching or improving upon state-of-the-art results on several different&#10;computer vision tasks (stereo, depth superresolution, colorization, and&#10;semantic segmentation) while being 10-1000 times faster than competing&#10;approaches. The bilateral solver is fast, robust, straightforward to generalize&#10;to new domains, and simple to integrate into deep learning pipelines.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.00373" label="1606.00373">
-        <attvalues>
-          <attvalue for="0" value="Deeper Depth Prediction with Fully Convolutional Residual Networks" />
-          <attvalue for="1" value="  This paper addresses the problem of estimating the depth map of a scene given&#10;a single RGB image. We propose a fully convolutional architecture, encompassing&#10;residual learning, to model the ambiguous mapping between monocular images and&#10;depth maps. In order to improve the output resolution, we present a novel way&#10;to efficiently learn feature map up-sampling within the network. For&#10;optimization, we introduce the reverse Huber loss that is particularly suited&#10;for the task at hand and driven by the value distributions commonly present in&#10;depth maps. Our model is composed of a single architecture that is trained&#10;end-to-end and does not rely on post-processing techniques, such as CRFs or&#10;other additional refinement steps. As a result, it runs in real-time on images&#10;or videos. In the evaluation, we show that the proposed model contains fewer&#10;parameters and requires fewer training data than the current state of the art,&#10;while outperforming all approaches on depth estimation. Code and models are&#10;publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1412.1897" label="1412.1897">
-        <attvalues>
-          <attvalue for="0" value="Deep Neural Networks are Easily Fooled: High Confidence Predictions for&#10;  Unrecognizable Images" />
-          <attvalue for="1" value="  Deep neural networks (DNNs) have recently been achieving state-of-the-art&#10;performance on a variety of pattern-recognition tasks, most notably visual&#10;classification problems. Given that DNNs are now able to classify objects in&#10;images with near-human-level performance, questions naturally arise as to what&#10;differences remain between computer and human vision. A recent study revealed&#10;that changing an image (e.g. of a lion) in a way imperceptible to humans can&#10;cause a DNN to label the image as something else entirely (e.g. mislabeling a&#10;lion a library). Here we show a related result: it is easy to produce images&#10;that are completely unrecognizable to humans, but that state-of-the-art DNNs&#10;believe to be recognizable objects with 99.99% confidence (e.g. labeling with&#10;certainty that white noise static is a lion). Specifically, we take&#10;convolutional neural networks trained to perform well on either the ImageNet or&#10;MNIST datasets and then find images with evolutionary algorithms or gradient&#10;ascent that DNNs label with high confidence as belonging to each dataset class.&#10;It is possible to produce images totally unrecognizable to human eyes that DNNs&#10;believe with near certainty are familiar objects, which we call &quot;fooling&#10;images&quot; (more generally, fooling examples). Our results shed light on&#10;interesting differences between human vision and current DNNs, and raise&#10;questions about the generality of DNN computer vision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.04600" label="2406.04600">
-        <attvalues>
-          <attvalue for="0" value="1st Place Solution for MOSE Track in CVPR 2024 PVUW Workshop: Complex&#10;  Video Object Segmentation" />
-          <attvalue for="1" value="  Tracking and segmenting multiple objects in complex scenes has always been a&#10;challenge in the field of video object segmentation, especially in scenarios&#10;where objects are occluded and split into parts. In such cases, the definition&#10;of objects becomes very ambiguous. The motivation behind the MOSE dataset is&#10;how to clearly recognize and distinguish objects in complex scenes. In this&#10;challenge, we propose a semantic embedding video object segmentation model and&#10;use the salient features of objects as query representations. The semantic&#10;understanding helps the model to recognize parts of the objects and the salient&#10;feature captures the more discriminative features of the objects. Trained on a&#10;large-scale video object segmentation dataset, our model achieves first place&#10;(\textbf{84.45\%}) in the test set of PVUW Challenge 2024: Complex Video Object&#10;Segmentation Track.&#10;" />
-          <attvalue for="2" value="&#10;Video Object Segmentation (VOS) focuses on tracking and segmenting target objects within a video sequence, beginning with mask annotations in the initial frame~\cite{ytvos2018, davis2017, feelvos, aot}. &#10;This technique has significant potential across various applications, especially given the increasing volume of video content in areas such as autonomous driving, augmented reality, and interactive video editing~\cite{perclip, bao2018cnnmrf}.&#10;The current VOS task faces several major challenges, including significant changes in target appearance, occlusions, and identity confusion due to similar objects and background clutter. These challenges are particularly pronounced in long-term videos, making the task even more complex.&#10;&#10;VOS methods~\cite{STM, stcn} generally achieve video object segmentation by comparing the test frame with previous frames. &#10;They begin by employing an association model to generate pixel-wise correlated features between the test frame and target templates. &#10;These correlated features are then used to predict the target masks accurately~\cite{SimVOS}.&#10;To handle variations in target appearances over time, certain methods~\cite{aot, xmem} incorporates a memory module that stores samples of the targets as their appearances change.&#10;Additionally, recent approaches~\cite{ISVOS, cutie} introduce object queries to differentiate between various target objects, thus mitigating identity confusion.&#10;&#10;To evaluate the performance of current methods in realistic environments, The MOSE~\cite{mose} (coMplex video Object SEgmentation) dataset is a comprehensive collection designed to provide complex and challenging scenarios with occluded and crowded objects. &#10;Unlike existing datasets that feature relatively isolated and salient objects, MOSE focuses on realistic environments, highlighting the limitations of current VOS methods and encouraging the development of more robust algorithms. &#10;This dataset is crucial for advancing VOS technologies to perform effectively in real-world applications where object interactions and occlusions are common.&#10;&#10;Current methods struggle with the MOSE dataset due to several challenges. First, when targets have multiple complex or separate parts because of occlusion, background clutter, and shape complexity, existing methods tend to produce incomplete prediction masks. This issue arises because these methods rely heavily on pixel-level correlation, which emphasizes pixel-wise similarity while overlooking semantic information. Second, although object queries~\cite{ISVOS, cutie} improve ID association accuracy, they perform poorly in sequences with significant target appearance changes and very small targets. The current query propagation strategy, which updates using the entire online predicted target sample, is susceptible to introducing noise and accumulating errors. This can lead to tracking failures, such as missing targets and ID switches, particularly when tracking small targets.&#10;&#10;To address the aforementioned issues, we propose a VOS framework to learn both the semantic prior and discriminative query representation.&#10;Specifically, we design a block that efficiently learns both semantic and detailed information, which can extract rich semantic features from a pre-trained Vision Transformer (ViT) without the need to train all feature extraction parameters.&#10;To better model the query representation of the target, we design a more discriminative filtering mechanism to generate discriminative queries.&#10;&#10;The main designs in this project include: &#10;\begin{itemize}&#10;\item we propose a fusion block to incorporate the semantic information from the ViT feature for Video Object Segmentation. We use the cls token from a pretrained ViT backbone to extract the semantic prior and global average polling of the image patches to extract the global information of the current frames. Furthermore, we design a local fusion to leverage the spatial information from the ViT features. The proposed framework shows robustness when facing multiple complex sequences.&#10;\item We develop a discriminative feature filtering module to model the discriminative query representation which achieves great improvement when dealing with small objects.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Semantic Embedding Models, Mathematics, Artificial Intelligence, Complex Scene Analysis, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2309.16653" label="2309.16653">
-        <attvalues>
-          <attvalue for="0" value="DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content&#10;  Creation" />
-          <attvalue for="1" value="  Recent advances in 3D content creation mostly leverage optimization-based 3D&#10;generation via score distillation sampling (SDS). Though promising results have&#10;been exhibited, these methods often suffer from slow per-sample optimization,&#10;limiting their practical usage. In this paper, we propose DreamGaussian, a&#10;novel 3D content generation framework that achieves both efficiency and quality&#10;simultaneously. Our key insight is to design a generative 3D Gaussian Splatting&#10;model with companioned mesh extraction and texture refinement in UV space. In&#10;contrast to the occupancy pruning used in Neural Radiance Fields, we&#10;demonstrate that the progressive densification of 3D Gaussians converges&#10;significantly faster for 3D generative tasks. To further enhance the texture&#10;quality and facilitate downstream applications, we introduce an efficient&#10;algorithm to convert 3D Gaussians into textured meshes and apply a fine-tuning&#10;stage to refine the details. Extensive experiments demonstrate the superior&#10;efficiency and competitive generation quality of our proposed approach.&#10;Notably, DreamGaussian produces high-quality textured meshes in just 2 minutes&#10;from a single-view image, achieving approximately 10 times acceleration&#10;compared to existing methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.08529" label="2310.08529">
-        <attvalues>
-          <attvalue for="0" value="GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging&#10;  2D and 3D Diffusion Models" />
-          <attvalue for="1" value="  In recent times, the generation of 3D assets from text prompts has shown&#10;impressive results. Both 2D and 3D diffusion models can help generate decent 3D&#10;objects based on prompts. 3D diffusion models have good 3D consistency, but&#10;their quality and generalization are limited as trainable 3D data is expensive&#10;and hard to obtain. 2D diffusion models enjoy strong abilities of&#10;generalization and fine generation, but 3D consistency is hard to guarantee.&#10;This paper attempts to bridge the power from the two types of diffusion models&#10;via the recent explicit and efficient 3D Gaussian splatting representation. A&#10;fast 3D object generation framework, named as GaussianDreamer, is proposed,&#10;where the 3D diffusion model provides priors for initialization and the 2D&#10;diffusion model enriches the geometry and appearance. Operations of noisy point&#10;growing and color perturbation are introduced to enhance the initialized&#10;Gaussians. Our GaussianDreamer can generate a high-quality 3D instance or 3D&#10;avatar within 15 minutes on one GPU, much faster than previous methods, while&#10;the generated instances can be directly rendered in real time. Demos and code&#10;are available at https://taoranyi.com/gaussiandreamer/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.03203" label="2312.03203">
-        <attvalues>
-          <attvalue for="0" value="Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled&#10;  Feature Fields" />
-          <attvalue for="1" value="  3D scene representations have gained immense popularity in recent years.&#10;Methods that use Neural Radiance fields are versatile for traditional tasks&#10;such as novel view synthesis. In recent times, some work has emerged that aims&#10;to extend the functionality of NeRF beyond view synthesis, for semantically&#10;aware tasks such as editing and segmentation using 3D feature field&#10;distillation from 2D foundation models. However, these methods have two major&#10;limitations: (a) they are limited by the rendering speed of NeRF pipelines, and&#10;(b) implicitly represented feature fields suffer from continuity artifacts&#10;reducing feature quality. Recently, 3D Gaussian Splatting has shown&#10;state-of-the-art performance on real-time radiance field rendering. In this&#10;work, we go one step further: in addition to radiance field rendering, we&#10;enable 3D Gaussian splatting on arbitrary-dimension semantic features via 2D&#10;foundation model distillation. This translation is not straightforward: naively&#10;incorporating feature fields in the 3DGS framework encounters significant&#10;challenges, notably the disparities in spatial resolution and channel&#10;consistency between RGB images and feature maps. We propose architectural and&#10;training changes to efficiently avert this problem. Our proposed method is&#10;general, and our experiments showcase novel view semantic segmentation,&#10;language-guided editing and segment anything through learning feature fields&#10;from state-of-the-art 2D foundation models such as SAM and CLIP-LSeg. Across&#10;experiments, our distillation method is able to provide comparable or better&#10;results, while being significantly faster to both train and render.&#10;Additionally, to the best of our knowledge, we are the first method to enable&#10;point and bounding-box prompting for radiance field manipulation, by leveraging&#10;the SAM model. Project website at: https://feature-3dgs.github.io/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.08453" label="1908.08453">
-        <attvalues>
-          <attvalue for="0" value="Noise Flow: Noise Modeling with Conditional Normalizing Flows" />
-          <attvalue for="1" value="  Modeling and synthesizing image noise is an important aspect in many computer&#10;vision applications. The long-standing additive white Gaussian and&#10;heteroscedastic (signal-dependent) noise models widely used in the literature&#10;provide only a coarse approximation of real sensor noise. This paper introduces&#10;Noise Flow, a powerful and accurate noise model based on recent normalizing&#10;flow architectures. Noise Flow combines well-established basic parametric noise&#10;models (e.g., signal-dependent noise) with the flexibility and expressiveness&#10;of normalizing flow networks. The result is a single, comprehensive, compact&#10;noise model containing fewer than 2500 parameters yet able to represent&#10;multiple cameras and gain factors. Noise Flow dramatically outperforms existing&#10;noise models, with 0.42 nats/pixel improvement over the camera-calibrated noise&#10;level functions, which translates to 52% improvement in the likelihood of&#10;sampled noise. Noise Flow represents the first serious attempt to go beyond&#10;simple parametric models to one that leverages the power of deep learning and&#10;data-driven noise distributions.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image noise modeling, estimation, and reduction is an important and active research area (e.g.,~\cite{Foi2009ClippedDenoising, Hwang2012Difference-basedDistribution, Seybold2013TowardsNoise, Trussell2012TheCameras}) with a long-standing history in computer vision (e.g.,~\cite{Healey1994RadiometricEstimation, Kuan1985AdaptiveNoise, Liu2008AutomaticImage, Naderi1978EstimationNoise}). A primary goal of such efforts is to remove or correct for noise in an image, either for aesthetic purposes, or to help improve other downstream tasks.&#10;Towards this end, accurately modeling noise distributions is a critical step.&#10;&#10;Existing noise models are not sufficient to represent the complexity of real noise~\cite{Abdelhamed2018ACameras, Plotz2017BenchmarkingPhotographs}.&#10;For example, a univariate homoscedastic Gaussian model does not represent the fact that photon noise is signal-dependent---that is, the variance of the noise is proportional to the magnitude of the signal. In turn, the signal-dependent heteroscedastic model~\cite{Foi2009ClippedDenoising, Foi2015PracticalRaw-data, Makitalo2013OptimalNoise}, often referred to as the noise level function (NLF), does not represent the spatial non-uniformity of noise power (e.g., fixed-pattern noise) or other sources of noise and non-linearities, such as amplification noise and quantization~\cite{Holst1998CCDDISPLAYS}.&#10;See Figure \ref{fig:overview}.&#10;In spite of their well-known limitations, these models are still the most commonly used. More complex models, such as a Poisson mixture~\cite{Jin2013ApproximationsNoise, Zhang2017ImprovedNoise}, exist, but still do not capture the complex noise sources mentioned earlier. &#10;&#10;Contribution~~~We introduce Noise Flow, a new noise model that combines the insights of parametric noise models and the expressiveness of powerful generative models.&#10;Specifically, we leverage recent normalizing flow architectures~\cite{Kingma2018Glow:Convolutions} to accurately model noise distributions observed from large datasets of real noisy images. &#10;In particular, based on the recent Glow architecture \cite{Kingma2018Glow:Convolutions}, we construct a normalizing flow model which is conditioned on critical variables, such as intensity, camera type, and gain settings (i.e., ISO).&#10;The model can be shown to be a strict generalization of the camera NLF but with the ability to capture significantly more complex behaviour.&#10;The result is a single model that is compact (fewer then 2500 parameters) and considerably more accurate than existing models. See Figure~\ref{fig:teaser}. &#10;We explore different aspects of the model through a set of ablation studies. To demonstrate the effectiveness of Noise Flow, we consider the application of denoising and use Noise Flow to synthesize training data for a denoising CNN resulting in significant improvements in PSNR.&#10;Code and pre-trained models for Noise Flow are available at:.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Physics, Deep Learning Architectures, Image Noise Modeling, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1807.03039" label="1807.03039">
-        <attvalues>
-          <attvalue for="0" value="Glow: Generative Flow with Invertible 1x1 Convolutions" />
-          <attvalue for="1" value="  Flow-based generative models (Dinh et al., 2014) are conceptually attractive&#10;due to tractability of the exact log-likelihood, tractability of exact&#10;latent-variable inference, and parallelizability of both training and&#10;synthesis. In this paper we propose Glow, a simple type of generative flow&#10;using an invertible 1x1 convolution. Using our method we demonstrate a&#10;significant improvement in log-likelihood on standard benchmarks. Perhaps most&#10;strikingly, we demonstrate that a generative model optimized towards the plain&#10;log-likelihood objective is capable of efficient realistic-looking synthesis&#10;and manipulation of large images. The code for our model is available at&#10;https://github.com/openai/glow&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09692" label="2111.09692">
-        <attvalues>
-          <attvalue for="0" value="SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised&#10;  Monocular Depth Estimation" />
-          <attvalue for="1" value="  We propose SUB-Depth, a universal multi-task training framework for&#10;self-supervised monocular depth estimation (SDE). Depth models trained with&#10;SUB-Depth outperform the same models trained in a standard single-task SDE&#10;framework. By introducing an additional self-distillation task into a standard&#10;SDE training framework, SUB-Depth trains a depth network, not only to predict&#10;the depth map for an image reconstruction task, but also to distill knowledge&#10;from a trained teacher network with unlabelled data. To take advantage of this&#10;multi-task setting, we propose homoscedastic uncertainty formulations for each&#10;task to penalize areas likely to be affected by teacher network noise, or&#10;violate SDE assumptions. We present extensive evaluations on KITTI to&#10;demonstrate the improvements achieved by training a range of existing networks&#10;using the proposed framework, and we achieve state-of-the-art performance on&#10;this task. Additionally, SUB-Depth enables models to estimate uncertainty on&#10;depth output.&#10;" />
-          <attvalue for="2" value="&#10;Depth perception plays an important role in real-world applications including autonomous driving, augmented reality, 3D reconstruction and other high-level computer vision tasks. Although physical sensors such as LiDAR have been deployed widely, estimating depth from pixels is appealing due to the lower cost and compatibility where a camera is available. Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels. In contrast, self-supervised approaches require no labelled data~\cite{lee2019real,multview,Godard17,tankovich2021hitnet,kendall2017end,zhang2018activestereonet,zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr} and are performing competitively.&#10;At a high level, self-supervised depth estimation uses a depth network's output as an intermediate representation for a stereo matching problem or an image reconstruction task. For the latter, the models are trained within a standard self-supervised monocular depth estimation (SDE) framework~\cite{zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr,watson2021temporal}. To build such a system, a pose network is introduced to predict the camera pose change between two consecutive frames. Hence, in a standard SDE training framework, a depth network and a pose network are trained simultaneously for an image reconstruction task by optimising the photometric loss.&#10;Previous works~\cite{choi2020safenet,tankovich2021hitnet,liebel2019multidepth,lu_icra} have shown that training a single depth model benefits from multiple regression or classification objectives. Inspired by prior works that train a student depth network with a trained teacher network~\cite{lyu2020hr,poggi2020uncertainty}, we extend the single-task SDE framework to a multi-task setting by introducing a self-distillation scheme associated with a regression objective.&#10;Compared with other multi-task settings which introduce supervised tasks such as semantic segmentation, one of the advantages of self-distillation is that the framework remains a self-supervised regime.&#10;&#10;Performance of multi-task systems is dependent on the relative loss weighting for each task. Instead of manually tuning weights of loss terms, inspired by Kendall~\cite{kendall2018multi}, we propose two uncertainty modelling strategies to calculate uncertainty for the self-distillation task and the image reconstruction task respectively. Specifically, the self-distillation uncertainty down-weights the regression loss when a teacher network outputs noisy depth values, and the photometric uncertainty outputs higher confidence where input frames satisfy the image reconstruction tasks' assumptions, that is, static world and ego-motion. We call our system SUB-Depth, and summarise its following key contributions:&#10;\begin{itemize}&#10; \item We propose a novel multi-task training framework for self-supervised monocular depth estimation.&#10; \item Instead of manually tuning loss terms' weights, we utilize the task-dependent uncertainty idea, and experiment with several ways of uncertainty modeling.&#10; \item We conduct exhaustive experiments to show that the proposed training framework is able to boost existing models' performance significantly.&#10; \end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Uncertainty Modeling, Artificial Intelligence, Depth Estimation, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2105.14021" label="2105.14021">
-        <attvalues>
-          <attvalue for="0" value="Boosting Monocular Depth Estimation Models to High-Resolution via&#10;  Content-Adaptive Multi-Resolution Merging" />
-          <attvalue for="1" value="  Neural networks have shown great abilities in estimating depth from a single&#10;image. However, the inferred depth maps are well below one-megapixel resolution&#10;and often lack fine-grained details, which limits their practicality. Our&#10;method builds on our analysis on how the input resolution and the scene&#10;structure affects depth estimation performance. We demonstrate that there is a&#10;trade-off between a consistent scene structure and the high-frequency details,&#10;and merge low- and high-resolution estimations to take advantage of this&#10;duality using a simple depth merging network. We present a double estimation&#10;method that improves the whole-image depth estimation and a patch selection&#10;method that adds local details to the final result. We demonstrate that by&#10;merging estimations at different resolutions with changing context, we can&#10;generate multi-megapixel depth maps with a high level of detail using a&#10;pre-trained model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05822" label="1904.05822">
-        <attvalues>
-          <attvalue for="0" value="Learning Single Camera Depth Estimation using Dual-Pixels" />
-          <attvalue for="1" value="  Deep learning techniques have enabled rapid progress in monocular depth&#10;estimation, but their quality is limited by the ill-posed nature of the problem&#10;and the scarcity of high quality datasets. We estimate depth from a single&#10;camera by leveraging the dual-pixel auto-focus hardware that is increasingly&#10;common on modern camera sensors. Classic stereo algorithms and prior&#10;learning-based depth estimation techniques under-perform when applied on this&#10;dual-pixel data, the former due to too-strong assumptions about RGB image&#10;matching, and the latter due to not leveraging the understanding of optics of&#10;dual-pixel image formation. To allow learning based methods to work well on&#10;dual-pixel imagery, we identify an inherent ambiguity in the depth estimated&#10;from dual-pixel cues, and develop an approach to estimate depth up to this&#10;ambiguity. Using our approach, existing monocular depth estimation techniques&#10;can be effectively applied to dual-pixel data, and much smaller models can be&#10;constructed that still infer high quality depth. To demonstrate this, we&#10;capture a large dataset of in-the-wild 5-viewpoint RGB images paired with&#10;corresponding dual-pixel data, and show how view supervision with this data can&#10;be used to learn depth up to the unknown ambiguities. On our new task, our&#10;model is 30% more accurate than any prior work on learning-based monocular or&#10;stereoscopic depth estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.00743" label="2009.00743">
-        <attvalues>
-          <attvalue for="0" value="Bidirectional Attention Network for Monocular Depth Estimation" />
-          <attvalue for="1" value="  In this paper, we propose a Bidirectional Attention Network (BANet), an&#10;end-to-end framework for monocular depth estimation (MDE) that addresses the&#10;limitation of effectively integrating local and global information in&#10;convolutional neural networks. The structure of this mechanism derives from a&#10;strong conceptual foundation of neural machine translation, and presents a&#10;light-weight mechanism for adaptive control of computation similar to the&#10;dynamic nature of recurrent neural networks. We introduce bidirectional&#10;attention modules that utilize the feed-forward feature maps and incorporate&#10;the global context to filter out ambiguity. Extensive experiments reveal the&#10;high degree of capability of this bidirectional attention model over&#10;feed-forward baselines and other state-of-the-art methods for monocular depth&#10;estimation on two challenging datasets -- KITTI and DIODE. We show that our&#10;proposed approach either outperforms or performs at least on a par with the&#10;state-of-the-art monocular depth estimation methods with less memory and&#10;computational complexity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.12140" label="2007.12140">
-        <attvalues>
-          <attvalue for="0" value="HITNet: Hierarchical Iterative Tile Refinement Network for Real-time&#10;  Stereo Matching" />
-          <attvalue for="1" value="  This paper presents HITNet, a novel neural network architecture for real-time&#10;stereo matching. Contrary to many recent neural network approaches that operate&#10;on a full cost volume and rely on 3D convolutions, our approach does not&#10;explicitly build a volume and instead relies on a fast multi-resolution&#10;initialization step, differentiable 2D geometric propagation and warping&#10;mechanisms to infer disparity hypotheses. To achieve a high level of accuracy,&#10;our network not only geometrically reasons about disparities but also infers&#10;slanted plane hypotheses allowing to more accurately perform geometric warping&#10;and upsampling operations. Our architecture is inherently multi-resolution&#10;allowing the propagation of information across different levels. Multiple&#10;experiments prove the effectiveness of the proposed approach at a fraction of&#10;the computation required by state-of-the-art methods. At the time of writing,&#10;HITNet ranks 1st-3rd on all the metrics published on the ETH3D website for two&#10;view stereo, ranks 1st on most of the metrics among all the end-to-end learning&#10;approaches on Middlebury-v3, ranks 1st on the popular KITTI 2012 and 2015&#10;benchmarks among the published methods faster than 100ms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.04309" label="1703.04309">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Learning of Geometry and Context for Deep Stereo Regression" />
-          <attvalue for="1" value="  We propose a novel deep learning architecture for regressing disparity from a&#10;rectified pair of stereo images. We leverage knowledge of the problem's&#10;geometry to form a cost volume using deep feature representations. We learn to&#10;incorporate contextual information using 3-D convolutions over this volume.&#10;Disparity values are regressed from the cost volume using a proposed&#10;differentiable soft argmin operation, which allows us to train our method&#10;end-to-end to sub-pixel accuracy without any additional post-processing or&#10;regularization. We evaluate our method on the Scene Flow and KITTI datasets and&#10;on KITTI we set a new state-of-the-art benchmark, while being significantly&#10;faster than competing approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.06009" label="1807.06009">
-        <attvalues>
-          <attvalue for="0" value="ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo&#10;  Systems" />
-          <attvalue for="1" value="  In this paper we present ActiveStereoNet, the first deep learning solution&#10;for active stereo systems. Due to the lack of ground truth, our method is fully&#10;self-supervised, yet it produces precise depth with a subpixel precision of&#10;$1/30th$ of a pixel; it does not suffer from the common over-smoothing issues;&#10;it preserves the edges; and it explicitly handles occlusions. We introduce a&#10;novel reconstruction loss that is more robust to noise and texture-less&#10;patches, and is invariant to illumination changes. The proposed loss is&#10;optimized using a window-based cost aggregation with an adaptive support weight&#10;scheme. This cost aggregation is edge-preserving and smooths the loss function,&#10;which is key to allow the network to reach compelling results. Finally we show&#10;how the task of predicting invalid regions, such as occlusions, can be trained&#10;end-to-end without ground-truth. This component is crucial to reduce blur and&#10;particularly improves predictions along depth discontinuities. Extensive&#10;quantitatively and qualitatively evaluations on real and synthetic data&#10;demonstrate state of the art results in many challenging scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.07356" label="2012.07356">
-        <attvalues>
-          <attvalue for="0" value="HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation" />
-          <attvalue for="1" value="  Self-supervised learning shows great potential in monoculardepth estimation,&#10;using image sequences as the only source ofsupervision. Although people try to&#10;use the high-resolutionimage for depth estimation, the accuracy of prediction&#10;hasnot been significantly improved. In this work, we find thecore reason comes&#10;from the inaccurate depth estimation inlarge gradient regions, making the&#10;bilinear interpolation er-ror gradually disappear as the resolution increases.&#10;To obtainmore accurate depth estimation in large gradient regions, itis&#10;necessary to obtain high-resolution features with spatialand semantic&#10;information. Therefore, we present an improvedDepthNet, HR-Depth, with two&#10;effective strategies: (1) re-design the skip-connection in DepthNet to get&#10;better high-resolution features and (2) propose feature fusion&#10;Squeeze-and-Excitation(fSE) module to fuse feature more efficiently.Using&#10;Resnet-18 as the encoder, HR-Depth surpasses all pre-vious&#10;state-of-the-art(SoTA) methods with the least param-eters at both high and low&#10;resolution. Moreover, previousstate-of-the-art methods are based on fairly&#10;complex and deepnetworks with a mass of parameters which limits their&#10;realapplications. Thus we also construct a lightweight networkwhich uses&#10;MobileNetV3 as encoder. Experiments show thatthe lightweight network can&#10;perform on par with many largemodels like Monodepth2 at high-resolution with&#10;only20%parameters. All codes and models will be available at&#10;https://github.com/shawLyu/HR-Depth.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14540" label="2104.14540">
-        <attvalues>
-          <attvalue for="0" value="The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth" />
-          <attvalue for="1" value="  Self-supervised monocular depth estimation networks are trained to predict&#10;scene depth using nearby frames as a supervision signal during training.&#10;However, for many applications, sequence information in the form of video&#10;frames is also available at test time. The vast majority of monocular networks&#10;do not make use of this extra signal, thus ignoring valuable information that&#10;could be used to improve the predicted depth. Those that do, either use&#10;computationally expensive test-time refinement techniques or off-the-shelf&#10;recurrent networks, which only indirectly make use of the geometric information&#10;that is inherently available.&#10;  We propose ManyDepth, an adaptive approach to dense depth estimation that can&#10;make use of sequence information at test time, when it is available. Taking&#10;inspiration from multi-view stereo, we propose a deep end-to-end cost volume&#10;based approach that is trained using self-supervision only. We present a novel&#10;consistency loss that encourages the network to ignore the cost volume when it&#10;is deemed unreliable, e.g. in the case of moving objects, and an augmentation&#10;scheme to cope with static cameras. Our detailed experiments on both KITTI and&#10;Cityscapes show that we outperform all published self-supervised baselines,&#10;including those that use single or multiple frames at test time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.11111" label="1907.11111">
-        <attvalues>
-          <attvalue for="0" value="MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and&#10;  Classification" />
-          <attvalue for="1" value="  We introduce MultiDepth, a novel training strategy and convolutional neural&#10;network (CNN) architecture that allows approaching single-image depth&#10;estimation (SIDE) as a multi-task problem. SIDE is an important part of road&#10;scene understanding. It, thus, plays a vital role in advanced driver assistance&#10;systems and autonomous vehicles. Best results for the SIDE task so far have&#10;been achieved using deep CNNs. However, optimization of regression problems,&#10;such as estimating depth, is still a challenging task. For the related tasks of&#10;image classification and semantic segmentation, numerous CNN-based methods with&#10;robust training behavior have been proposed. Hence, in order to overcome the&#10;notorious instability and slow convergence of depth value regression during&#10;training, MultiDepth makes use of depth interval classification as an auxiliary&#10;task. The auxiliary task can be disabled at test-time to predict continuous&#10;depth values using the main regression branch more efficiently. We applied&#10;MultiDepth to road scenes and present results on the KITTI depth prediction&#10;dataset. In experiments, we were able to show that end-to-end multi-task&#10;learning with both, regression and classification, is able to considerably&#10;improve training and yield more accurate results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.06209" label="2005.06209">
-        <attvalues>
-          <attvalue for="0" value="On the uncertainty of self-supervised monocular depth estimation" />
-          <attvalue for="1" value="  Self-supervised paradigms for monocular depth estimation are very appealing&#10;since they do not require ground truth annotations at all. Despite the&#10;astonishing results yielded by such methodologies, learning to reason about the&#10;uncertainty of the estimated depth maps is of paramount importance for&#10;practical applications, yet uncharted in the literature. Purposely, we explore&#10;for the first time how to estimate the uncertainty for this task and how this&#10;affects depth accuracy, proposing a novel peculiar technique specifically&#10;designed for self-supervised approaches. On the standard KITTI dataset, we&#10;exhaustively assess the performance of each method with different&#10;self-supervised paradigms. Such evaluation highlights that our proposal i)&#10;always improves depth accuracy significantly and ii) yields state-of-the-art&#10;results concerning uncertainty estimation when training on sequences and&#10;competitive results uniquely deploying stereo pairs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.07115" label="1705.07115">
-        <attvalues>
-          <attvalue for="0" value="Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry&#10;  and Semantics" />
-          <attvalue for="1" value="  Numerous deep learning applications benefit from multi-task learning with&#10;multiple regression and classification objectives. In this paper we make the&#10;observation that the performance of such systems is strongly dependent on the&#10;relative weighting between each task's loss. Tuning these weights by hand is a&#10;difficult and expensive process, making multi-task learning prohibitive in&#10;practice. We propose a principled approach to multi-task deep learning which&#10;weighs multiple loss functions by considering the homoscedastic uncertainty of&#10;each task. This allows us to simultaneously learn various quantities with&#10;different units or scales in both classification and regression settings. We&#10;demonstrate our model learning per-pixel depth regression, semantic and&#10;instance segmentation from a monocular input image. Perhaps surprisingly, we&#10;show our model can learn multi-task weightings and outperform separate models&#10;trained individually on each task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.00882" label="1910.00882">
-        <attvalues>
-          <attvalue for="0" value="Pose Estimation for Omni-directional Cameras using Sinusoid Fitting" />
-          <attvalue for="1" value="  We propose a novel pose estimation method for geometric vision of&#10;omni-directional cameras. On the basis of the regularity of the pixel movement&#10;after camera pose changes, we formulate and prove the sinusoidal relationship&#10;between pixels movement and camera motion. We use the improved Fourier-Mellin&#10;invariant (iFMI) algorithm to find the motion of pixels, which was shown to be&#10;more accurate and robust than the feature-based methods. While iFMI works only&#10;on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our&#10;method works on panoramic images and estimates the full 6 DoF 3D transform, up&#10;to an unknown scale factor. For that we fit the motion of the pixels in the&#10;panoramic images, as determined by iFMI, to two sinusoidal functions. The&#10;offsets, amplitudes and phase-shifts of the two functions then represent the 3D&#10;rotation and translation of the camera between the two images. We perform&#10;experiments for 3D rotation, which show that our algorithm outperforms the&#10;feature-based methods in accuracy and robustness. We leave the more complex 3D&#10;translation experiments for future work.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Omni-directional cameras have been widely used in mobile robots. Visual cues from panorama images help the robot to achieve homing in \cite{argyros2005robot}, which uses omni-cameras' advantage of the large field of view (FOV). In \cite{lemaire2007slam}, panorama images are exploited to implement a bearings-only SLAM system, which can provide rich feature points. In addition, several meaningful applications with panorama cameras are mentioned in \cite{benosman2000panoramic}. In the above cases, pose estimation is one of the important topics. Optimization and geometric methods are two common ways to achieve this task. The former is usually used in feature-based methods \cite{davison2007monoslam}, while the latter is one of the basic parts in direct methods \cite{engel2014lsd}. These two approaches are usually combined together to achieve better performance. For example, Engel et. al combined both methods to realize real-time robust visual odometry in \cite{engel2018direct}; \cite{mur2015orb} uses geometric methods for the front-end of a SLAM system and the optimization method for the back-end. In addition, recently deep learning gained attention in pose estimation \cite{handa2016gvnn,costante2016exploring}. Since the deep learning methods heavily depend on training images, which may be difficult to adapt to different environments within limited training datasets, we do not take the deep learning method into account.&#10;&#10;Optimization methods are usually used to minimize the photometric and geometry errors. In direct methods based on photometric consistency, the motion between different frames is estimated by minimizing the photometric error, as used in \cite{engel2014lsd,engel2018direct, newcombe2011dtam}. Geometry error optimization helps to boost pose estimation, such as bundle adjustment \cite{triggs1999bundle} and graph optimization \cite{kummerle2011g, IJRR2016GraphSLAM}.&#10;&#10;Geometric methods include two main groups: pose estimation by 2D to 2D correspondences and 3D to 2D correspondences. The eight-point \cite{hartley1995defence} and five-point algorithms \cite{stewenius2005minimal} are widely used to estimate the relative pose between two frames with given 2D-2D correspondences. Firstly, the essential matrix or fundamental matrix is calculated by epipolar geometry. Then relative rotation and up-to-scale translation are estimated from the matrix. Methods like \cite{zheng2013revisiting}, that estimate relative pose by 3D-2D correspondences, compensate the scaling case and calculate the absolute pose directly, because 3D points provide the real scale information.&#10;&#10;In addition to feature-based and direct methods, spectral methods are also used for motion estimation. On 2D images, the iFMI algorithm \cite{bulow2009fast, iFMI-Resolution-IAV10} uses Fast Fourier Transform (FFT) to calculate the $x$ and $y$ translation, yaw and scaling between two images. It is applied to image mosaicking \cite{JIRS-UAVmosaic-ELROB-RREE-2011} and motion detection \cite{SSRR11-UAV-MotionDetection}. A related approach is also used for registration of 3D range data \cite{PAMI2013Spectral}.&#10;&#10;If we only consider pose estimation between two frames, optimization is used with direct methods, while the geometric vision helps feature-based methods to calculate the camera's motion. Our proposed method belongs to the latter, but it differs from the two approaches mentioned above by directly exploiting the properties of catadioptric omni-directional cameras. Figure~\ref{fig:system setup} shows the omni-directional camera that we use to collect omni-images and one omni-image, mounted on a simple robot. We also show the image as captured by the smart phone and its unwrapped panorama image. From Figure~\ref{fig:omni lens}, we can see that the catadioptric panorama image has the disadvantage of low resolution, as mentioned in \cite{benosman2000panoramic}, which will increase the difficulty for feature-based methods. Thus it is important to process these images with more robust algorithms. To meet this requirement, we combined the improved Fourier-Mellin invariant (iFMI) algorithm with our proposed method, since it proves to be more robust than SIFT.&#10;Our main contributions are summarized as:&#10;\begin{enumerate}&#10;&#09;\item proposing a novel relative pose estimation method based on geometric vision and fitting of pixel displacement values to sinusoidal functions;&#10;&#09;\item exploiting the 2D frequency-based algorithm to estimate 3D pose of the omni-camera;&#10;&#09;\item comparing our algorithm with commonly used epipolar geometry methods together with different features.&#10;\end{enumerate}&#10;&#10;The rest of this paper is organized as follows: the theoretical foundation of our proposed method is introduced in Sec~\ref{sec:problem_formulation}, including motion model, proof and parameter estimation; then we explain out implementation detail in Sec~\ref{sec:implementation}; the experiment and analysis is displayed in Sec~\ref{sec:experiments}; finally, we conclude our work in Sec~\ref{sec:conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Omni-directional Imaging, Mathematics, Physics, Computer Science, Geometric Vision Algorithms, Camera Pose Estimation, Robotics, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1607.07405" label="1607.07405">
-        <attvalues>
-          <attvalue for="0" value="gvnn: Neural Network Library for Geometric Computer Vision" />
-          <attvalue for="1" value="  We introduce gvnn, a neural network library in Torch aimed towards bridging&#10;the gap between classic geometric computer vision and deep learning. Inspired&#10;by the recent success of Spatial Transformer Networks, we propose several new&#10;layers which are often used as parametric transformations on the data in&#10;geometric computer vision. These layers can be inserted within a neural network&#10;much in the spirit of the original spatial transformers and allow&#10;backpropagation to enable end-to-end learning of a network involving any domain&#10;knowledge in geometric computer vision. This opens up applications in learning&#10;invariance to 3D geometric transformation for place recognition, end-to-end&#10;visual odometry, depth estimation and unsupervised learning through warping&#10;with a parametric transformation for image reconstruction error.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.03622" label="2109.03622">
-        <attvalues>
-          <attvalue for="0" value="Learning Local-Global Contextual Adaptation for Multi-Person Pose&#10;  Estimation" />
-          <attvalue for="1" value="  This paper studies the problem of multi-person pose estimation in a bottom-up&#10;fashion. With a new and strong observation that the localization issue of the&#10;center-offset formulation can be remedied in a local-window search scheme in an&#10;ideal situation, we propose a multi-person pose estimation approach, dubbed as&#10;LOGO-CAP, by learning the LOcal-GlObal Contextual Adaptation for human Pose.&#10;Specifically, our approach learns the keypoint attraction maps (KAMs) from the&#10;local keypoints expansion maps (KEMs) in small local windows in the first step,&#10;which are subsequently treated as dynamic convolutional kernels on the&#10;keypoints-focused global heatmaps for contextual adaptation, achieving accurate&#10;multi-person pose estimation. Our method is end-to-end trainable with near&#10;real-time inference speed in a single forward pass, obtaining state-of-the-art&#10;performance on the COCO keypoint benchmark for bottom-up human pose estimation.&#10;With the COCO trained model, our method also outperforms prior arts by a large&#10;margin on the challenging OCHuman dataset.&#10;" />
-          <attvalue for="2" value="&#10;2D human pose estimation is a classical computer vision problem that aims to parsing articulated structures of human parts from natural images. With rich and longstanding studies, we have witnessed great successes on single-person pose estimation~\cite{SHG,HRNet} by convolutional neural networks. Therefore, it is of great interest in pushing the pose estimation from the single-person to multi-person configuration.&#10;&#10;The problem of multi-person pose estimation from images has been extensively studied in top-down paradigms~\cite{HRNet,SimpleBaseLine,SHG} that formulate the problem as single-person pose estimation with an off-the-shelf person detector with an impressively high AP (e.g, 56\% on the COCO-2017 validation dataset~\cite{SimpleBaseLine}). &#10;Although the top-down paradigm has been dramatically pushed into a high-performing stages, it remains several problems in both aspects of efficiency and accuracy due to the dependency of detecting person bounding boxes. Motivated by this, we are interested in studying the problem of multi-person pose estimation without incurring extra priors of bounding boxes, which is conventionally termed as bottom-up pose estimation.&#10;&#10;Bottom-up pose estimation approaches pay much attention on estimating pose parameters from any given multi-person image instead of using the cropped single-person ones, which poses several challenges on accurately identifying the learned bottom-level keypoints as person parts and grouping them into different individuals by learning part affinity fields~\cite{paf,openpose}, part association fields~\cite{pifpaf}, associative embedding~\cite{asso_embedding}. Those grouping approaches are accurate but sophisticated due to the necessity of ad-hoc grouping/decoding schemes. &#10;Recently, many researchers attempted to learning center-offsets~\cite{centernet-zhou,CenterNet-duan,ART-pose,DEKR-pose} for pose estimation as its intrinsic simplicity and efficiency. However, most of the center-offset approaches are suffering from the main challenge of localization inaccuracy due to the large structural variations of human pose, thus leading to inferior performance than the grouping ones.&#10;&#10;In this paper, we are interested in studying the bottom-up pose estimation using the center-offset formulation for its simplicity and efficiency. We directly address the aforementioned main challenges for center-offset based multi-person pose estimation. Our proposed approach is motivated by a surprisingly strong empirical observation via analyzing what the fundamental issues are with the vanilla center-offset pose estimation network.&#10;&#10;A Surprisingly Strong Observation. &#10;Our analyses are based on results in the fully-annotated subset of the COCO val-2017 dataset. &#10;The vanilla center-offset regression method utilizes the HRNet-W32~\cite{HRNet} as the feature backbone to directly predict keypoints center heatmap and the offset vectors. As shown in \ref{fig:speed-accuracy}, it obtains $60.1$ average precision (AP), which is not great, but reasonably good. It clearly shows that the pose keypoints center and the offset vectors can be learned reasonably well. We ask: How exactly bad are the center-offset estimation? We want to know (i) whether some of keypoints are truly bad being far away from the underlying ground-truth (GT) locations, or (ii) whether most of them are already in the close proximity of the GT ones? We observe that the latter is true. To quantitatively characterize the close proximity, instead of directly utilizing the learned offset vectors for human pose estimation, we treat them as human pose keypoint initialization and do a local window search to compute the empirical upper-bound of performance. More specifically, based on the initially predicted human poses, by introducing a local window (e.g., $11\times 11$) centered at each detected key point and by computing the single keypoint similarity with the ground-truth keypoint, an empirical upper-bound of $88.9$ AP is obtained, which is significantly higher than the state of the art and shows the potential of improving the vanilla center-offset regression paradigm.&#10;&#10;A Direct Solution Leveraging the Observation. The implication of the observation is significant: It reveals that the fundamental challenge of improving the center-offset approach for pose estimation is to resolve the local misplacement. &#10;To that end, a straightforward way is just to learn a local heatmap (e.g., $11\times 11$) for each human pose keypoint based on the learned center and offset vectors, and then to compute the refined keypoints by taking $\arg \max$ within the local heatmap. Although appealing, this does not work as observed during our development of the LOGO-CAP. The underlying reason is also straightforward: if this can work, the original offset vector regression should work at the first place since no additional information is introduced through learning the local heatmap.&#10;&#10;We hypothesize that on the one hand, in addition to the local heatmap, the structural relationship between different pose keypoints needs to be taken into account, and on the other hand, the intrinsic uncertainty of the local information in a local heatmap needs to be resolved. The former is the key challenge of structured output prediction problems. Many message passing algorithms have been developed in the literature. The latter can not be addressed by simply increasing the local window size. It entails learning stronger local-global information interaction and adaptation.&#10;&#10;To verify the two hypotheses, the proposed LOGO-CAP lifts the initial keypoints via the center-offset prediction to keypoint expansion maps (KEMs) to counter their lack of localization accuracy in two modules (Section~\ref{sec:LOGO-CAP}). The KEMs extend the star-structured representation of the center-offset formulation to the pictorial structure representation~\cite{fischler1973representation,felzenszwalb2005pictorial}. As shown in \ref{fig:teaser}, in our LOGO-CAP, one module computes local KEMs and learns to account for the structured output prediction nature of the human pose estimation problem, resulting in the keypoint attraction maps (KAMs), i.e., the local filters in \ref{fig:teaser}. Another module computes global KEMs and learns to refine the global KEMs by integrating the KAMs. &#10;&#10;Our LOGO-CAP is a fully end-to-end bottom-up human pose estimation method with near real-time inference speed. It obtains $70.0$ AP in the fully-annotated subset of the COCO val-2017 dataset, which is an absolute increase of $9.9$ AP compared to the vanilla center-offset method, making a significant step forward. Fig.~\ref{fig:speed-accuracy} shows the advantage of the proposed LOGO-CAP in terms of overall speed-accuracy comparisons between our LOGO-CAP and prior arts. Meanwhile, we should notice that there is still a significant gap towards the empirical upper bound in \ref{fig:speed-accuracy}, which encourages more work to be investigated. &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1904.08189" label="1904.08189">
-        <attvalues>
-          <attvalue for="0" value="CenterNet: Keypoint Triplets for Object Detection" />
-          <attvalue for="1" value="  In object detection, keypoint-based approaches often suffer a large number of&#10;incorrect object bounding boxes, arguably due to the lack of an additional look&#10;into the cropped regions. This paper presents an efficient solution which&#10;explores the visual patterns within each cropped region with minimal costs. We&#10;build our framework upon a representative one-stage keypoint-based detector&#10;named CornerNet. Our approach, named CenterNet, detects each object as a&#10;triplet, rather than a pair, of keypoints, which improves both precision and&#10;recall. Accordingly, we design two customized modules named cascade corner&#10;pooling and center pooling, which play the roles of enriching information&#10;collected by both top-left and bottom-right corners and providing more&#10;recognizable information at the central regions, respectively. On the MS-COCO&#10;dataset, CenterNet achieves an AP of 47.0%, which outperforms all existing&#10;one-stage detectors by at least 4.9%. Meanwhile, with a faster inference speed,&#10;CenterNet demonstrates quite comparable performance to the top-ranked two-stage&#10;detectors. Code is available at https://github.com/Duankaiwen/CenterNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.15480" label="2006.15480">
-        <attvalues>
-          <attvalue for="0" value="Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive&#10;  Keypoint Estimates" />
-          <attvalue for="1" value="  The typical bottom-up human pose estimation framework includes two stages,&#10;keypoint detection and grouping. Most existing works focus on developing&#10;grouping algorithms, e.g., associative embedding, and pixel-wise keypoint&#10;regression that we adopt in our approach. We present several schemes that are&#10;rarely or unthoroughly studied before for improving keypoint detection and&#10;grouping (keypoint regression) performance. First, we exploit the keypoint&#10;heatmaps for pixel-wise keypoint regression instead of separating them for&#10;improving keypoint regression. Second, we adopt a pixel-wise spatial&#10;transformer network to learn adaptive representations for handling the scale&#10;and orientation variance to further improve keypoint regression quality. Last,&#10;we present a joint shape and heatvalue scoring scheme to promote the estimated&#10;poses that are more likely to be true poses. Together with the tradeoff heatmap&#10;estimation loss for balancing the background and keypoint pixels and thus&#10;improving heatmap estimation quality, we get the state-of-the-art bottom-up&#10;human pose estimation result. Code is available at&#10;https://github.com/HRNet/HRNet-Bottom-up-Pose-Estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.02300" label="2104.02300">
-        <attvalues>
-          <attvalue for="0" value="Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" />
-          <attvalue for="1" value="  In this paper, we are interested in the bottom-up paradigm of estimating&#10;human poses from an image. We study the dense keypoint regression framework&#10;that is previously inferior to the keypoint detection and grouping framework.&#10;Our motivation is that regressing keypoint positions accurately needs to learn&#10;representations that focus on the keypoint regions.&#10;  We present a simple yet effective approach, named disentangled keypoint&#10;regression (DEKR). We adopt adaptive convolutions through pixel-wise spatial&#10;transformer to activate the pixels in the keypoint regions and accordingly&#10;learn representations from them. We use a multi-branch structure for separate&#10;regression: each branch learns a representation with dedicated adaptive&#10;convolutions and regresses one keypoint. The resulting disentangled&#10;representations are able to attend to the keypoint regions, respectively, and&#10;thus the keypoint regression is spatially more accurate. We empirically show&#10;that the proposed direct regression method outperforms keypoint detection and&#10;grouping methods and achieves superior bottom-up pose estimation results on two&#10;benchmark datasets, COCO and CrowdPose. The code and models are available at&#10;https://github.com/HRNet/DEKR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.07141" label="2303.07141">
-        <attvalues>
-          <attvalue for="0" value="An Improved Baseline Framework for Pose Estimation Challenge at ECCV&#10;  2022 Visual Perception for Navigation in Human Environments Workshop" />
-          <attvalue for="1" value="  This technical report describes our first-place solution to the pose&#10;estimation challenge at ECCV 2022 Visual Perception for Navigation in Human&#10;Environments Workshop. In this challenge, we aim to estimate human poses from&#10;in-the-wild stitched panoramic images. Our method is built based on Faster&#10;R-CNN for human detection, and HRNet for human pose estimation. We describe&#10;technical details for the JRDB-Pose dataset, together with some experimental&#10;results. In the competition, we achieved 0.303 $\text{OSPA}_{\text{IOU}}$ and&#10;64.047\% $\text{AP}_{\text{0.5}}$ on the test set of JRDB-Pose.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pose estimation plays an important role in computer vision, which serves as a fundamental task for high-level human action analysis, such as action recognition \cite{iqbal2017pose}, action assessment \cite{xiaohan2015joint}, and human-robot interaction \cite{song2012continuous}. In recent years, there has been a growing interest in developing pose estimation models that can accurately detect and estimate human poses in wild scenarios. Although many large-scale datasets \cite{lin2014microsoft,sun2020scalability,doering2022posetrack21} have been introduced in recent years, no dataset primarily targets robotic perception tasks in social navigation environments, and thus rarely reflects the specific challenges found in human-robot interaction and robot navigation in crowded human environments. The JRDB-Pose dataset \cite{vendrow2022jrdb} aims at bridging this gap and sets up a new benchmark for pose estimation.&#10;&#10;This technical report focuses on adopting image-based pose estimation, where bottom-up and top-down methods are two commonly used approaches. In bottom-up methods, the pose of individual body parts is first estimated and then grouped to form a complete pose. On the other hand, top-down methods first detect the entire body and then estimate the pose of each body part. A main strength of top-down methods is their ability to incorporate high-level information about the body, such as prior knowledge about human anatomy and shape. This makes them more robust to occlusion and deformation of body parts, which can be a challenge for bottom-up methods. Moreover, panoramic images contain lots of people with different scales. Compared with bottom-up methods, top-down methods are more robust to multi-scale targets. Therefore, we adopt a classical top-down method-HRNet \cite{sun2019deep} for human pose estimation. Details are given in the remainder of this report.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="2210.11940" label="2210.11940">
-        <attvalues>
-          <attvalue for="0" value="JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and&#10;  Tracking" />
-          <attvalue for="1" value="  Autonomous robotic systems operating in human environments must understand&#10;their surroundings to make accurate and safe decisions. In crowded human scenes&#10;with close-up human-robot interaction and robot navigation, a deep&#10;understanding requires reasoning about human motion and body dynamics over time&#10;with human body pose estimation and tracking. However, existing datasets either&#10;do not provide pose annotations or include scene types unrelated to robotic&#10;applications. Many datasets also lack the diversity of poses and occlusions&#10;found in crowded human scenes. To address this limitation we introduce&#10;JRDB-Pose, a large-scale dataset and benchmark for multi-person pose estimation&#10;and tracking using videos captured from a social navigation robot. The dataset&#10;contains challenge scenes with crowded indoor and outdoor locations and a&#10;diverse range of scales and occlusion types. JRDB-Pose provides human pose&#10;annotations with per-keypoint occlusion labels and track IDs consistent across&#10;the scene. A public evaluation server is made available for fair evaluation on&#10;a held-out test set. JRDB-Pose is available at https://jrdb.erc.monash.edu/ .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.12072" label="2009.12072">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results" />
-          <attvalue for="1" value="  This paper introduces the real image Super-Resolution (SR) challenge that was&#10;part of the Advances in Image Manipulation (AIM) workshop, held in conjunction&#10;with ECCV 2020. This challenge involves three tracks to super-resolve an input&#10;image for $\times$2, $\times$3 and $\times$4 scaling factors, respectively. The&#10;goal is to attract more attention to realistic image degradation for the SR&#10;task, which is much more complicated and challenging, and contributes to&#10;real-world image super-resolution applications. 452 participants were&#10;registered for three tracks in total, and 24 teams submitted their results.&#10;They gauge the state-of-the-art approaches for real image SR in terms of PSNR&#10;and SSIM.&#10;" />
-          <attvalue for="2" value="&#10;Single image super-resolution (SR) reconstructs high-resolution (HR) images from low-resolution (LR) counterparts with image quality degradations \cite{glasner2009super}\cite{yang2010image}. Instead of imposing higher requirements on hardware devices and sensors, it could be applicable to many practical scenarios, such as video surveillance, satellite, medical imaging, etc. As a fundamental research topic, SR has attracted a long-standing and considerable attention in computer vision community.&#10;&#10;With the emergence of deep learning, convolutional neural network (CNN) based SR methods (e.g., SRCNN \cite{SRCNN}, SRGAN \cite{SRRESNET}, EDSR \cite{EDSR}, ESRGAN \cite{ESRGAN} and RCAN \cite{RCAN}) inherit the powerful capacity of deep learning and have achieved remarkable performance improvements. Nevertheless, so far, the remarkable progress of SR is mainly driven by the supervised learning of models from LR images and their HR counterparts. While the bicubic downsampling is usually adopted to simulate the LR images, the learned deep SR model performs much less effective for real-world SR applications since the image degradation in real-world is much more complicated.&#10;&#10;To mitigate this issue, several real SR datasets have been recently built, City 100~\cite{city100} and SR-RAW~\cite{zoomlearn}. The images in City100 were captured for the printed postcards in the indoor environment, which are limited in capturing the complicated image and degradation characteristics of natural scenes. The images in SR-RAW were collected in the real world and a contextual bilateral loss was proposed to address the misalignment problem in the dataset. Besides, Cai et al. \cite{cai2019toward} released another real image SR dataset, named RealSR, which was captured from two DSLR cameras. They proposed the LP-KPN method in a Laplacian pyramid framework. Considering the complex image degradation across different scenes and devices, a large-scale diverse real SR dataset, named DRealSR~\cite{CDC}, was released to further promote the research on real-world image SR. Images of DRealSR were captured by five different DSLR cameras and posed more challenging image degradation. In \cite{CDC}, the proposed component divide-and-conquer model (CDC) built a baseline, hourglass SR network (HGSR), in a stacked architecture, explored different reconstruction difficulties in terms of three low-level image components inspired by corner point detection, i.e, the flat, edges and corner points, and trained the model with a mediate supervision strategy. Besides, its proposed gradient-weighted (GW) loss also drives the model to adapt learning objectives to the reconstruction difficulties of three image components and has a flexibility of the application to any SR model.&#10;&#10;Jointly with the Advances in Image Manipulation (AIM) 2020 workshop, we organize the AIM Challenge on Real-world Image Super-Resolution. Specifically, this challenge concerns the real-world SISR, which poses two challenging issues \cite{CDC}: (1) more complex degradation against bicubic downsampling, and (2) diverse degradation processes among devices, aiming to learn a generic model to super-resolve LR images captured in practical scenarios. To achieve this goal, paired LR and HR images are captured by various DSLR cameras and provided for training. They are randomly selected from the DRealSR dataset. Images for training, validation and testing are captured in the same way with the same set of cameras.&#10;&#10;This challenge is one of the AIM 2020 associated challenges on:&#10;scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Real World Applications, Mathematics, Image Processing, Image Super Resolution, Computer Science, Computer Vision Challenges" />
-        </attvalues>
-      </node>
-      <node id="1807.02758" label="1807.02758">
-        <attvalues>
-          <attvalue for="0" value="Image Super-Resolution Using Very Deep Residual Channel Attention&#10;  Networks" />
-          <attvalue for="1" value="  Convolutional neural network (CNN) depth is of crucial importance for image&#10;super-resolution (SR). However, we observe that deeper networks for image SR&#10;are more difficult to train. The low-resolution inputs and features contain&#10;abundant low-frequency information, which is treated equally across channels,&#10;hence hindering the representational ability of CNNs. To solve these problems,&#10;we propose the very deep residual channel attention networks (RCAN).&#10;Specifically, we propose a residual in residual (RIR) structure to form very&#10;deep network, which consists of several residual groups with long skip&#10;connections. Each residual group contains some residual blocks with short skip&#10;connections. Meanwhile, RIR allows abundant low-frequency information to be&#10;bypassed through multiple skip connections, making the main network focus on&#10;learning high-frequency information. Furthermore, we propose a channel&#10;attention mechanism to adaptively rescale channel-wise features by considering&#10;interdependencies among channels. Extensive experiments show that our RCAN&#10;achieves better accuracy and visual improvements against state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03378" label="1904.03378">
-        <attvalues>
-          <attvalue for="0" value="Camera Lens Super-Resolution" />
-          <attvalue for="1" value="  Existing methods for single image super-resolution (SR) are typically&#10;evaluated with synthetic degradation models such as bicubic or Gaussian&#10;downsampling. In this paper, we investigate SR from the perspective of camera&#10;lenses, named as CameraSR, which aims to alleviate the intrinsic tradeoff&#10;between resolution (R) and field-of-view (V) in realistic imaging systems.&#10;Specifically, we view the R-V degradation as a latent model in the SR process&#10;and learn to reverse it with realistic low- and high-resolution image pairs. To&#10;obtain the paired images, we propose two novel data acquisition strategies for&#10;two representative imaging systems (i.e., DSLR and smartphone cameras),&#10;respectively. Based on the obtained City100 dataset, we quantitatively analyze&#10;the performance of commonly-used synthetic degradation models, and demonstrate&#10;the superiority of CameraSR as a practical solution to boost the performance of&#10;existing SR methods. Moreover, CameraSR can be readily generalized to different&#10;content and devices, which serves as an advanced digital zoom tool in realistic&#10;imaging systems. Codes and datasets are available at&#10;https://github.com/ngchc/CameraSR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.05169" label="1905.05169">
-        <attvalues>
-          <attvalue for="0" value="Zoom To Learn, Learn To Zoom" />
-          <attvalue for="1" value="  This paper shows that when applying machine learning to digital zoom for&#10;photography, it is beneficial to use real, RAW sensor data for training.&#10;Existing learning-based super-resolution methods do not use real sensor data,&#10;instead operating on RGB images. In practice, these approaches result in loss&#10;of detail and accuracy in their digitally zoomed output when zooming in on&#10;distant image regions. We also show that synthesizing sensor data by resampling&#10;high-resolution RGB images is an oversimplified approximation of real sensor&#10;data and noise, resulting in worse image quality. The key barrier to using real&#10;sensor data for training is that ground truth high-resolution imagery is&#10;missing. We show how to obtain the ground-truth data with optically zoomed&#10;images and contribute a dataset, SR-RAW, for real-world computational zoom. We&#10;use SR-RAW to train a deep network with a novel contextual bilateral loss&#10;(CoBi) that delivers critical robustness to mild misalignment in input-output&#10;image pairs. The trained network achieves state-of-the-art performance in 4X&#10;and 8X computational zoom.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.00523" label="1904.00523">
-        <attvalues>
-          <attvalue for="0" value="Toward Real-World Single Image Super-Resolution: A New Benchmark and A&#10;  New Model" />
-          <attvalue for="1" value="  Most of the existing learning-based single image superresolution (SISR)&#10;methods are trained and evaluated on simulated datasets, where the&#10;low-resolution (LR) images are generated by applying a simple and uniform&#10;degradation (i.e., bicubic downsampling) to their high-resolution (HR)&#10;counterparts. However, the degradations in real-world LR images are far more&#10;complicated. As a consequence, the SISR models trained on simulated data become&#10;less effective when applied to practical scenarios. In this paper, we build a&#10;real-world super-resolution (RealSR) dataset where paired LR-HR images on the&#10;same scene are captured by adjusting the focal length of a digital camera. An&#10;image registration algorithm is developed to progressively align the image&#10;pairs at different resolutions. Considering that the degradation kernels are&#10;naturally non-uniform in our dataset, we present a Laplacian pyramid based&#10;kernel prediction network (LP-KPN), which efficiently learns per-pixel kernels&#10;to recover the HR image. Our extensive experiments demonstrate that SISR models&#10;trained on our RealSR dataset deliver better visual quality with sharper edges&#10;and finer textures on real-world scenes than those trained on simulated&#10;datasets. Though our RealSR dataset is built by using only two cameras (Canon&#10;5D3 and Nikon D810), the trained model generalizes well to other camera devices&#10;such as Sony a7II and mobile phones.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.01928" label="2008.01928">
-        <attvalues>
-          <attvalue for="0" value="Component Divide-and-Conquer for Real-World Image Super-Resolution" />
-          <attvalue for="1" value="  In this paper, we present a large-scale Diverse Real-world image&#10;Super-Resolution dataset, i.e., DRealSR, as well as a divide-and-conquer&#10;Super-Resolution (SR) network, exploring the utility of guiding SR model with&#10;low-level image components. DRealSR establishes a new SR benchmark with diverse&#10;real-world degradation processes, mitigating the limitations of conventional&#10;simulated image degradation. In general, the targets of SR vary with image&#10;regions with different low-level image components, e.g., smoothness preserving&#10;for flat regions, sharpening for edges, and detail enhancing for textures.&#10;Learning an SR model with conventional pixel-wise loss usually is easily&#10;dominated by flat regions and edges, and fails to infer realistic details of&#10;complex textures. We propose a Component Divide-and-Conquer (CDC) model and a&#10;Gradient-Weighted (GW) loss for SR. Our CDC parses an image with three&#10;components, employs three Component-Attentive Blocks (CABs) to learn attentive&#10;masks and intermediate SR predictions with an intermediate supervision learning&#10;strategy, and trains an SR model following a divide-and-conquer learning&#10;principle. Our GW loss also provides a feasible way to balance the difficulties&#10;of image components for SR. Extensive experiments validate the superior&#10;performance of our CDC and the challenging aspects of our DRealSR dataset&#10;related to diverse real-world scenarios. Our dataset and codes are publicly&#10;available at&#10;https://github.com/xiezw5/Component-Divide-and-Conquer-for-Real-World-Image-Super-Resolution&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.12798" label="2009.12798">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020: Scene Relighting and Illumination Estimation Challenge" />
-          <attvalue for="1" value="  We review the AIM 2020 challenge on virtual image relighting and illumination&#10;estimation. This paper presents the novel VIDIT dataset used in the challenge&#10;and the different proposed solutions and final evaluation results over the 3&#10;challenge tracks. The first track considered one-to-one relighting; the&#10;objective was to relight an input photo of a scene with a different color&#10;temperature and illuminant orientation (i.e., light source position). The goal&#10;of the second track was to estimate illumination settings, namely the color&#10;temperature and orientation, from a given image. Lastly, the third track dealt&#10;with any-to-any relighting, thus a generalization of the first track. The&#10;target color temperature and orientation, rather than being pre-determined, are&#10;instead given by a guide image. Participants were allowed to make use of their&#10;track 1 and 2 solutions for track 3. The tracks had 94, 52, and 56 registered&#10;participants, respectively, leading to 20 confirmed submissions in the final&#10;competition stage.&#10;" />
-          <attvalue for="2" value="&#10;~&#10;&#10;Deep image relighting has multiple applications both in research and in practice, and is recently witnessing increased interest. A single-image relighting method would allow aesthetic enhancement applications, such as photo montage of images taken under different illuminations, and illumination retouching without human expert work. Very importantly, in computer vision research image relighting can be leveraged for data augmentation, enabling the trained methods to be robust to changes in light source position or color temperature. It could also serve for domain adaptation, by normalizing input images to a unique set of illumination settings that the down-stream computer vision method was trained on. The relighting task contains multiple sub-tasks, namely, illumination estimation and manipulation, shadow removal or practically inpainting for hardly lit areas, and geometric understanding for shadow recasting. The combination of these tasks makes relighting very challenging. &#10;&#10;Recently, datasets limited to interior scenes~\cite{murmann2019dataset}, underexposed images enhanced by professionals~\cite{wang2019underexposed}, and rendered images with randomized light directions~\cite{xu2018deep} have been proposed, but none serve the benchmarking needs for image relighting, namely, having all $M\times N$ combinations of $M$ scenes and $N$ illumination settings. &#10;Further datasets are used in the literature on style transfer or intrinsic image decomposition. For instance, IIW~\cite{bell2014intrinsic} and SAW~\cite{kovacs2017shading} contain human-labeled reflectance and shading annotations, and BigTime~\cite{li2018learning} contains time-lapse data of scenes illuminated under varying light conditions. Multiple methods are recently being developed for relighting~\cite{sun2019single,nagano2019deep,dherse2020scene}, and the prior literature on intrinsic images, which disentangle surface reflectance from lighting, is rich~\cite{weiss2001deriving,tappen2003recovering,finlayson2004intrinsic,shen2011intrinsic,barron2012color,bell2014intrinsic}, notably for applications such as relighting~\cite{bousseau2009user} and normalization~\cite{matsushita2004illumination}.&#10;&#10;The aim of this challenge, and of the novel dataset Virtual Image Dataset for Illumination Transfer (VIDIT), is to gauge the current state-of-the-art for image relighting. The virtual dataset provides a well-controlled setup to provide full-reference evaluation, which is ideal for benchmarking purposes, and is an important step towards real-image relighting. Such virtual datasets have proven useful in multiple applications to augment even the training datasets containing real images, for instance the vKitti data~\cite{cabon2020virtual}. There could be differences relative to real images such as the distribution of textures that can vary from man-made to natural scenes~\cite{torralba2003statistics,burton1987color}, the specifics of the capturing device like chromatic aberrations~\cite{elhelou2018aam,llanos2020simultaneous,zhao2020modified}, or the presence of multiple light sources. VIDIT itself is described in the following section. The goal of the challenge is thus to provide a benchmark on this dataset for future research on image relighting.&#10;&#10;This challenge is one of the AIM 2020 associated challenges on:&#10;scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.01110" label="2010.01110">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Image Extreme Inpainting" />
-          <attvalue for="1" value="  This paper reviews the AIM 2020 challenge on extreme image inpainting. This&#10;report focuses on proposed solutions and results for two different tracks on&#10;extreme image inpainting: classical image inpainting and semantically guided&#10;image inpainting. The goal of track 1 is to inpaint considerably large part of&#10;the image using no supervision but the context. Similarly, the goal of track 2&#10;is to inpaint the image by having access to the entire semantic segmentation&#10;map of the image to inpaint. The challenge had 88 and 74 participants,&#10;respectively. 11 and 6 teams competed in the final phase of the challenge,&#10;respectively. This report gauges current solutions and set a benchmark for&#10;future extreme image inpainting methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.04994" label="2011.04994">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Learned Image Signal Processing Pipeline" />
-          <attvalue for="1" value="  This paper reviews the second AIM learned ISP challenge and provides the&#10;description of the proposed solutions and results. The participating teams were&#10;solving a real-world RAW-to-RGB mapping problem, where to goal was to map the&#10;original low-quality RAW images captured by the Huawei P20 device to the same&#10;photos obtained with the Canon 5D DSLR camera. The considered task embraced a&#10;number of complex computer vision subtasks, such as image demosaicing,&#10;denoising, white balancing, color and contrast correction, demoireing, etc. The&#10;target metric used in this challenge combined fidelity scores (PSNR and SSIM)&#10;with solutions' perceptual results measured in a user study. The proposed&#10;solutions significantly improved the baseline results, defining the&#10;state-of-the-art for practical image signal processing pipeline modeling.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.04988" label="2011.04988">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Rendering Realistic Bokeh" />
-          <attvalue for="1" value="  This paper reviews the second AIM realistic bokeh effect rendering challenge&#10;and provides the description of the proposed solutions and results. The&#10;participating teams were solving a real-world bokeh simulation problem, where&#10;the goal was to learn a realistic shallow focus technique using a large-scale&#10;EBB! bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs&#10;captured using the Canon 7D DSLR camera. The participants had to render bokeh&#10;effect based on only one single frame without any additional data from other&#10;cameras or sensors. The target metric used in this challenge combined the&#10;runtime and the perceptual quality of the solutions measured in the user study.&#10;To ensure the efficiency of the submitted models, we measured their runtime on&#10;standard desktop CPUs as well as were running the models on smartphone GPUs.&#10;The proposed solutions significantly improved the baseline results, defining&#10;the state-of-the-art for practical bokeh effect rendering problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.06943" label="2009.06943">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results" />
-          <attvalue for="1" value="  This paper reviews the AIM 2020 challenge on efficient single image&#10;super-resolution with focus on the proposed solutions and results. The&#10;challenge task was to super-resolve an input image with a magnification factor&#10;x4 based on a set of prior examples of low and corresponding high resolution&#10;images. The goal is to devise a network that reduces one or several aspects&#10;such as runtime, parameter count, FLOPs, activations, and memory consumption&#10;while at least maintaining PSNR of MSRResNet. The track had 150 registered&#10;participants, and 25 teams submitted the final results. They gauge the&#10;state-of-the-art in efficient single image super-resolution.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.12987" label="2009.12987">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Video Temporal Super-Resolution" />
-          <attvalue for="1" value="  Videos in the real-world contain various dynamics and motions that may look&#10;unnaturally discontinuous in time when the recordedframe rate is low. This&#10;paper reports the second AIM challenge on Video Temporal Super-Resolution&#10;(VTSR), a.k.a. frame interpolation, with a focus on the proposed solutions,&#10;results, and analysis. From low-frame-rate (15 fps) videos, the challenge&#10;participants are required to submit higher-frame-rate (30 and 60 fps) sequences&#10;by estimating temporally intermediate frames. To simulate realistic and&#10;challenging dynamics in the real-world, we employ the REDS_VTSR dataset derived&#10;from diverse videos captured in a hand-held camera for training and evaluation&#10;purposes. There have been 68 registered participants in the competition, and 5&#10;teams (one withdrawn) have competed in the final testing phase. The winning&#10;team proposes the enhanced quadratic video interpolation method and achieves&#10;state-of-the-art on the VTSR task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;While some approaches synthesize intermediate frames directly~\cite{long2016learning,choi2020channel}, most methods embed motion estimation modules into neural network architectures. &#10;Roughly, the motion modeling could be categorized into phase, kernel, and flow-based approaches. Several methods try to further improve inference accuracy adaptively to the input.&#10;&#10;Phase-based, Kernel-based methods:&#10;In an early work of Meyer et al.~\cite{Meyer_2015_CVPR}, the temporal change of frames is modeled as phase shifts.&#10;The PhaseNet~\cite{Meyer_2018_CVPR} model has utilized steerable pyramid filters for encoding larger dynamics with a phase shift.&#10;On the other hand, kernel-based approaches use convolutional kernels to shift the input frames to an intermediate position.&#10;The AdaConv~\cite{Niklaus_2017_CVPR} model estimates spatially adaptive kernels, and the SepConv~\cite{Niklaus_2018_CVPR} improves computational efficiency by factorizing the 2D spatial kernel into 1D kernels.&#10;&#10;Flow-based approaches:&#10;Meanwhile, optical flow-based piece-wise linear models have grown popular.&#10;The DVF~\cite{Liu_2017_ICCV} and SuperSloMo~\cite{Jiang_2018_CVPR} models estimate optical flow between two input frames and warp them to the target intermediate time.&#10;Later, the accuracy of SuperSloMo is improved by training with cycle consistency loss~\cite{Reda_2019_ICCV}.&#10;The TOF~\cite{xue2019video}, CtxSyn~\cite{Niklaus_2018_CVPR}, and BMBC~\cite{Park_2020_ECCV} models additionally apply image synthesis modules on the warped frames to produce the intermediate frame.&#10;To interpolate high-resolution videos in fast speed, the IM-Net~\cite{Peleg_2019_CVPR} uses block-level horizontal/vertical motion vectors in multi-scale.&#10;&#10;It is essential to handle the occluded area when warping with the optical flow as multiple pixels could overlap in the warped location.&#10;Jiang~et al.~\cite{Jiang_2018_CVPR} and Yuan~et al.~\cite{Yuan_2019_CVPR} estimate occlusion maps to exclude invalid contributions.&#10;Additional information is explored, such as depth~\cite{Bao_2019_CVPR} and scene context~\cite{Niklaus_2018_CVPR} to determine proper local warping weights.&#10;Niklaus~et al.~\cite{Niklaus_2020_CVPR} suggests forward warping with the softmax splatting method, while many other approaches use backward warping with sampling.&#10;&#10;Several works tried mixed motion representation of kernel-based and flow-based models.&#10;The DAIN~\cite{Bao_2019_CVPR} and MEMC-Net~\cite{MEMC-Net} models use both the kernel and optical flow to apply adaptive warping.&#10;The AdaCof~\cite{Lee_2020_CVPR} unifies the combined representation via an adaptive collaboration of flows in a similar formulation as deformable convolution~\cite{dai2017deformable}.&#10;To enhance the degree of freedom in the representation, the local weight values differ by position. &#10;&#10;Despite the success of motion estimation with optical flow, complex motions in the real world are hard to be modeled due to the simplicity of optical flow.&#10;Most of the methods using the optical flow between the two frames are limited to locally linear models.&#10;To improve the degree of freedom in the motion model, higher-order representations are proposed.&#10;Quadratic~\cite{qvi_nips19,qvi_iccvw19} and cubic~\cite{Chi_2020_ECCV} motion flow is estimated from multiple input frames.&#10;Such higher-order polynomial models could reflect accelerations and nonlinear trajectories.&#10;In AIM 2019 Challenge on Video Temporal Super-Resolution, the quadratic video interpolation method~\cite{qvi_nips19,qvi_iccvw19} achieved the best accuracy. &#10;&#10;Refinement methods:&#10;On top of the restored frames, some methods try to improve the initial estimation with various techniques.&#10;Gui~et al.~\cite{Gui_2020_CVPR} propose a 2-stage estimation method.&#10;Initial interpolation is obtained with a structure guide in the first stage and the second stage refines the texture.&#10;Choi~et al.~\cite{Choi_2020_CVPR} perform test-time adaptation of the model via meta-learning.&#10;Motivated by classical pyramid energy minimization, Zhang~et al.~\cite{Zhang_2020_ECCV} propose a recurrent residual pyramid network. Residual displacements are refined via recurrence.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2009.06290" label="2009.06290">
-        <attvalues>
-          <attvalue for="0" value="AIM 2020 Challenge on Video Extreme Super-Resolution: Methods and&#10;  Results" />
-          <attvalue for="1" value="  This paper reviews the video extreme super-resolution challenge associated&#10;with the AIM 2020 workshop at ECCV 2020. Common scaling factors for learned&#10;video super-resolution (VSR) do not go beyond factor 4. Missing information can&#10;be restored well in this region, especially in HR videos, where the&#10;high-frequency content mostly consists of texture details. The task in this&#10;challenge is to upscale videos with an extreme factor of 16, which results in&#10;more serious degradations that also affect the structural integrity of the&#10;videos. A single pixel in the low-resolution (LR) domain corresponds to 256&#10;pixels in the high-resolution (HR) domain. Due to this massive information&#10;loss, it is hard to accurately restore the missing information. Track 1 is set&#10;up to gauge the state-of-the-art for such a demanding task, where fidelity to&#10;the ground truth is measured by PSNR and SSIM. Perceptually higher quality can&#10;be achieved in trade-off for fidelity by generating plausible high-frequency&#10;content. Track 2 therefore aims at generating visually pleasing results, which&#10;are ranked according to human perception, evaluated by a user study. In&#10;contrast to single image super-resolution (SISR), VSR can benefit from&#10;additional information in the temporal domain. However, this also imposes an&#10;additional requirement, as the generated frames need to be consistent along&#10;time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.12327" label="2403.12327">
-        <attvalues>
-          <attvalue for="0" value="GT-Rain Single Image Deraining Challenge Report" />
-          <attvalue for="1" value="  This report reviews the results of the GT-Rain challenge on single image&#10;deraining at the UG2+ workshop at CVPR 2023. The aim of this competition is to&#10;study the rainy weather phenomenon in real world scenarios, provide a novel&#10;real world rainy image dataset, and to spark innovative ideas that will further&#10;the development of single image deraining methods on real images. Submissions&#10;were trained on the GT-Rain dataset and evaluated on an extension of the&#10;dataset consisting of 15 additional scenes. Scenes in GT-Rain are comprised of&#10;real rainy image and ground truth image captured moments after the rain had&#10;stopped. 275 participants were registered in the challenge and 55 competed in&#10;the final testing phase.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Natural weather conditions such as rain, fog, or snow can drastically reduce visibility, reduce contrast, and create noisy conditions that greatly degrade the quality of images of a scene. This naturally hurts the performance of many computer vision models and techniques designed and trained on clear image data~\cite{790306}. Running these models on this adverse data is a widely applicable problem, for example for self-driving cars, models that need to operate in geographical extremes, or models that must be highly robust~\cite{zhang2023perception}. Therefore, the field of single-image de-weathering, which focuses on techniques to remove weather effects from images, has grown significantly in popularity~\cite{10.1145/1360612.1360671, zhang2023weatherstream, ba2022not, wang2022uformer, Yang2017RainRemoval, valanarasu2022transweather, song2023vision, zhang2023data} After images are &quot;de-weathered&quot;, general vision models can then be applied for various downstream tasks such as object detection, semantic segmentation, and more. &#10;&#10;Unfortunately, current single-image weathering techniques are bottlenecked by a common issue: dataset quality. It is not possible to obtain ideal real ground-truth pairs of rain and clean images because the same scene cannot possibly be observed both with and without rain at the same time. While previous works attempted to solve this problem by using simulated and pseudo-real pairs, these approaches lead to limited performance as current rain simulators cannot model all complex behaviours of rain or other weather effects.&#10;&#10;For instance, a number of synthetic methods add rain streaks to clean images to generate dataset pairs, but rain does not only manifest as streaks: If raindrops are further away, the streaks meld together creating rain accumulation, or veiling effects, which are exceedingly difficult to simulate. A further challenge with generating high-quality synthetic data is that results on real test data can only be evaluated qualitatively, for no real paired ground truth exists. &#10;&#10;The GT-Rain dataset was proposed to tackle this problem by collecting high quality time multiplexed pairs of real rain and ground truth images. It consists of 31.5K time multiplexed pairs of real rainy and ground truth images captured moments apart in time. Each team must train their models using the GT-RAIN dataset but are free to use additional datasets to improve performance. Submissions are evaluated by comparing restored images against ground-truth images in the test dataset.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Real World Datasets, Image Deraining Methods, Physics, Computer Science, Computer Vision Challenges, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2112.08936" label="2112.08936">
-        <attvalues>
-          <attvalue for="0" value="Perception and Sensing for Autonomous Vehicles Under Adverse Weather&#10;  Conditions: A Survey" />
-          <attvalue for="1" value="  Automated Driving Systems (ADS) open up a new domain for the automotive&#10;industry and offer new possibilities for future transportation with higher&#10;efficiency and comfortable experiences. However, autonomous driving under&#10;adverse weather conditions has been the problem that keeps autonomous vehicles&#10;(AVs) from going to level 4 or higher autonomy for a long time. This paper&#10;assesses the influences and challenges that weather brings to ADS sensors in an&#10;analytic and statistical way, and surveys the solutions against inclement&#10;weather conditions. State-of-the-art techniques on perception enhancement with&#10;regard to each kind of weather are thoroughly reported. External auxiliary&#10;solutions, weather conditions coverage in currently available datasets,&#10;simulators, and experimental facilities with weather chambers are distinctly&#10;sorted out. Additionally, potential future ADS sensors candidates and&#10;approaches beyond common senses are provided. By looking into all kinds of&#10;major weather problems the autonomous driving field is currently facing, and&#10;reviewing both hardware and computer science solutions in recent years, this&#10;survey points out the main moving trends of adverse weather problems in&#10;autonomous driving, i.e., advanced sensor fusions, more sophisticated networks,&#10;and V2X &amp; IoT technologies; and also the limitations brought by emerging 1550&#10;nm LiDARs. In general, this work contributes a holistic overview of the&#10;obstacles and directions of ADS development in terms of adverse weather driving&#10;conditions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.08919" label="2007.08919">
-        <attvalues>
-          <attvalue for="0" value="Edge-Preserving Guided Semantic Segmentation for VIPriors Challenge" />
-          <attvalue for="1" value="  Semantic segmentation is one of the most attractive research fields in&#10;computer vision. In the VIPriors challenge, only very limited numbers of&#10;training samples are allowed, leading to that the current state-of-the-art and&#10;deep learning-based semantic segmentation techniques are hard to train well. To&#10;overcome this shortcoming, therefore, we propose edge-preserving guidance to&#10;obtain the extra prior information, to avoid the overfitting under small-scale&#10;training dataset. First, a two-channeled convolutional layer is concatenated to&#10;the last layer of the conventional semantic segmentation network. Then, an edge&#10;map is calculated from the ground truth by Sobel operation and followed by&#10;concatenating a hard-thresholding operation to indicate whether the pixel is&#10;the edge or not. Then, the two-dimensional cross-entropy loss is adopted to&#10;calculate the loss between the predicted edge map and its ground truth, termed&#10;as an edge-preserving loss. In this way, the continuity of boundaries between&#10;different instances can be forced by the proposed edge-preserving loss.&#10;Experiments demonstrate that the proposed method can achieve excellent&#10;performance under small-scale training set, compared to state-of-the-art&#10;semantic segmentation techniques.&#10;" />
-          <attvalue for="2" value="&#10;With the rapid growth of deep learning techniques, several semantic segmentation models were proposed recently. The strongest baseline for semantic segmentation, as well-known, is the DeepLabV3+ \cite{deeplab}. The encoder-decoder architecture makes the semantic information can be preserved from middle- and high-level feature maps, leading to state-of-the-art performance on various benchmarks. Since the dataset of the VIPriors challenges is a subset of cityscape \cite{cityscape}, several state-of-the-art semantic segmentation techniques are selected as the candidates, such as DeepLabV3+ \cite{deeplab}, Densely Connected Neural Architecture Search (DCNAS) \cite{dcans}, Object-Contextual Representation (OCR) \cite{hrnet}, Height-Driven Attention Networks (HANet) \cite{hanet}. Since the network architecture search (NAS) is widely used to find the optimized network architecture for various tasks. However, such a NAS-based network lacks generalization so that the performance of the different datasets may not be promising. OCR discovers the context information from objects/instances to obtain the promising feature representation to further boost the performance for semantic segmentation. Both DeepLabV3+ and OCR achieve state-of-the-art performance. However, the number of the parameters of these two models is relatively large, leading to the fact that the performance under small-scale training datasets maybe not promising. &#10;&#10;Recently, HANet \cite{hanet} introduces the extra image prior to the learning process--height-driven prior--to achieve a significant performance gain for existing semantic segmentation techniques. However, the number of parameters of HANet is still large, so that it may not be suitable for a small-scale dataset. Inspired by HANet \cite{hanet}, we propose the edge-preserving loss to introduce more images prior to making the network can be well-trained on the limited dataset. Based on HANet, we make three improvements to boost the performance under a small-scale dataset. Our contribution is three-fold:&#10;\begin{itemize}&#10; \item We introduce the edge-preserving loss to force the contours of instances to ensure the continuity of the predicted segmentation maps.&#10; \item To have better feature representation powerful, we introduce the ResNeSt \cite{resnest} to the HANet as the backbone without dramatically increasing the number of parameters.&#10; \item A simple but effective data augmentation strategy is proposed to solve the label unbalanced issue. &#10;\end{itemize}&#10;&#10;The rest of this paper is organized as follows. Section 2 introduces the proposed semantic segmentation method. Section 3 presents obtained experimental results of the mini-Cityscape. Finally, Section 4 gives the~conclusions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1802.02611" label="1802.02611">
-        <attvalues>
-          <attvalue for="0" value="Encoder-Decoder with Atrous Separable Convolution for Semantic Image&#10;  Segmentation" />
-          <attvalue for="1" value="  Spatial pyramid pooling module or encode-decoder structure are used in deep&#10;neural networks for semantic segmentation task. The former networks are able to&#10;encode multi-scale contextual information by probing the incoming features with&#10;filters or pooling operations at multiple rates and multiple effective&#10;fields-of-view, while the latter networks can capture sharper object boundaries&#10;by gradually recovering the spatial information. In this work, we propose to&#10;combine the advantages from both methods. Specifically, our proposed model,&#10;DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module&#10;to refine the segmentation results especially along object boundaries. We&#10;further explore the Xception model and apply the depthwise separable&#10;convolution to both Atrous Spatial Pyramid Pooling and decoder modules,&#10;resulting in a faster and stronger encoder-decoder network. We demonstrate the&#10;effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets,&#10;achieving the test set performance of 89.0\% and 82.1\% without any&#10;post-processing. Our paper is accompanied with a publicly available reference&#10;implementation of the proposed models in Tensorflow at&#10;\url{https://github.com/tensorflow/models/tree/master/research/deeplab}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.11883" label="2003.11883">
-        <attvalues>
-          <attvalue for="0" value="DCNAS: Densely Connected Neural Architecture Search for Semantic Image&#10;  Segmentation" />
-          <attvalue for="1" value="  Neural Architecture Search (NAS) has shown great potentials in automatically&#10;designing scalable network architectures for dense image predictions. However,&#10;existing NAS algorithms usually compromise on restricted search space and&#10;search on proxy task to meet the achievable computational demands. To allow as&#10;wide as possible network architectures and avoid the gap between target and&#10;proxy dataset, we propose a Densely Connected NAS (DCNAS) framework, which&#10;directly searches the optimal network structures for the multi-scale&#10;representations of visual information, over a large-scale target dataset.&#10;Specifically, by connecting cells with each other using learnable weights, we&#10;introduce a densely connected search space to cover an abundance of mainstream&#10;network designs. Moreover, by combining both path-level and channel-level&#10;sampling strategies, we design a fusion module to reduce the memory consumption&#10;of ample search space. We demonstrate that the architecture obtained from our&#10;DCNAS algorithm achieves state-of-the-art performances on public semantic image&#10;segmentation benchmarks, including 84.3% on Cityscapes, and 86.9% on PASCAL VOC&#10;2012. We also retain leading performances when evaluating the architecture on&#10;the more challenging ADE20K and Pascal Context dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.05128" label="2003.05128">
-        <attvalues>
-          <attvalue for="0" value="Cars Can't Fly up in the Sky: Improving Urban-Scene Segmentation via&#10;  Height-driven Attention Networks" />
-          <attvalue for="1" value="  This paper exploits the intrinsic features of urban-scene images and proposes&#10;a general add-on module, called height-driven attention networks (HANet), for&#10;improving semantic segmentation for urban-scene images. It emphasizes&#10;informative features or classes selectively according to the vertical position&#10;of a pixel. The pixel-wise class distributions are significantly different from&#10;each other among horizontally segmented sections in the urban-scene images.&#10;Likewise, urban-scene images have their own distinct characteristics, but most&#10;semantic segmentation networks do not reflect such unique attributes in the&#10;architecture. The proposed network architecture incorporates the capability&#10;exploiting the attributes to handle the urban scene dataset effectively. We&#10;validate the consistent performance (mIoU) increase of various semantic&#10;segmentation models on two datasets when HANet is adopted. This extensive&#10;quantitative analysis demonstrates that adding our module to existing models is&#10;easy and cost-effective. Our method achieves a new state-of-the-art performance&#10;on the Cityscapes benchmark with a large margin among ResNet-101 based&#10;segmentation models. Also, we show that the proposed model is coherent with the&#10;facts observed in the urban scene by visualizing and interpreting the attention&#10;map. Our code and trained models are publicly available at&#10;https://github.com/shachoi/HANet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.08955" label="2004.08955">
-        <attvalues>
-          <attvalue for="0" value="ResNeSt: Split-Attention Networks" />
-          <attvalue for="1" value="  It is well known that featuremap attention and multi-path representation are&#10;important for visual recognition. In this paper, we present a modularized&#10;architecture, which applies the channel-wise attention on different network&#10;branches to leverage their success in capturing cross-feature interactions and&#10;learning diverse representations. Our design results in a simple and unified&#10;computation block, which can be parameterized using only a few variables. Our&#10;model, named ResNeSt, outperforms EfficientNet in accuracy and latency&#10;trade-off on image classification. In addition, ResNeSt has achieved superior&#10;transfer learning results on several public benchmarks serving as the backbone,&#10;and has been adopted by the winning entries of COCO-LVIS challenge. The source&#10;code for complete system and pretrained models are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.03249" label="2110.03249">
-        <attvalues>
-          <attvalue for="0" value="Multimodal Colored Point Cloud to Image Alignment" />
-          <attvalue for="1" value="  Reconstruction of geometric structures from images using supervised learning&#10;suffers from limited available amount of accurate data. One type of such data&#10;is accurate real-world RGB-D images. A major challenge in acquiring such ground&#10;truth data is the accurate alignment between RGB images and the point cloud&#10;measured by a depth scanner. To overcome this difficulty, we consider a&#10;differential optimization method that aligns a colored point cloud with a given&#10;color image through iterative geometric and color matching. In the proposed&#10;framework, the optimization minimizes the photometric difference between the&#10;colors of the point cloud and the corresponding colors of the image pixels.&#10;Unlike other methods that try to reduce this photometric error, we analyze the&#10;computation of the gradient on the image plane and propose a different direct&#10;scheme. We assume that the colors produced by the geometric scanner camera and&#10;the color camera sensor are different and therefore characterized by different&#10;chromatic acquisition properties. Under these multimodal conditions, we find&#10;the transformation between the camera image and the point cloud colors. We&#10;alternately optimize for aligning the position of the point cloud and matching&#10;the different color spaces. The alignments produced by the proposed method are&#10;demonstrated on both synthetic data with quantitative evaluation and real&#10;scenes with qualitative results.&#10;" />
-          <attvalue for="2" value="&#10;In recent years, research in 3D shape reconstruction has made tremendous progress.&#10;Much of the research in the field focused on shape reconstruction from IR or RGB images.&#10;Multiple approaches have been proposed including amongst others, shape from stereo and monocular depth estimation.&#10;More recently, the focus of the community shifted towards supervised deep learning methods \cite{zbontar2016stereo, kendall2017end, zhang2019ga, chang2018pyramid, luo2016efficient, pang2017cascade, eigen2014depth, eigen2015predicting, fu2018deep}.&#10;Deep Learning relies heavily on large and accurate datasets.&#10;Since such datasets are difficult to obtain, most works use synthetic datasets to train and validate their models.&#10;However, these datasets are limited since they do not capture real-world properties such as distortions and noise.&#10;In addition, the use of deep learning methods may require specific training data for each camera model, as the shape reconstruction algorithms are sensitive to model properties and artifacts.&#10;&#10;One solution to acquire accurate ground-truth depth is to use precise, yet, often slow, depth measuring devices such as 3D laser scanners.&#10;Using such a device to acquire exact depth values for a desired camera model requires registering the device pose relative to the camera one.&#10;Placing the device at a fixed and calibrated position relative to the camera is often unsuitable for this task. &#10;Such a setup suffers from technical difficulties and requires constant maintenance \cite{geiger2013vision}. &#10;Accurate laser scanners usually require a long scanning time for a given scene.&#10;Consequently, in order to create a large-scale dataset, multiple images should be acquired for each depth scan.&#10;This means that a 3D Euclidean transformation must be found between each image coordinate system and the scanner coordinate system.&#10;&#10;The geometric information and color texture provided by some of these devices can be combined to produce a colored point cloud that can be used for registration.&#10;However, such registration faces the challenge of multimodality. &#10;More specifically, with the comparison of two images taken with different devices - one with the scanner and one with the camera.&#10;Different devices have different color properties, often referred to as color gamut.&#10;When two devices capture the same scene, the captured color space of each device is different.&#10;This defines the problem that our work attempts to solve.&#10;How to accurately align a colored point cloud with a colored image under multimodal conditions.&#10;&#10;The multimodal alignment task, which is critical to produce accurate and reliable geometric data, has not yet been directly explored.&#10;While a rough alignment can be easily found, a precise alignment is required for a proper geometric dataset construction.&#10;Therefore, in this paper, we focus on refining such an alignment.&#10;Direct visual odometry methods can be modified to be used for our task.&#10;Such methods have been mainly used in SLAM, Optical Flow and Color Mapping.&#10;These approaches attempt to minimize the photometric error between corresponding pixels based on the estimated scene geometry.&#10;However, unlike our task, these approaches align images acquired by the same device.&#10;&#10;In contrast to previous methods, the proposed pipeline uses a direct numerical sub-pixel scheme to approximate the gradients in the image plane.&#10;We prove that the commonly used method for computing such gradients is equivalent to evaluating the gradients on a blurred image.&#10;To overcome multimodality, we also propose a second-order polynomial color transformation between the point cloud colors and the image colors.&#10;Such an approach has rarely been used for color transformations.&#10;The proposed approach produces state-of-the-art results for multimodal colored point cloud to image alignment.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Geometric Reconstruction, Machine Learning, Computer Vision, Mathematics, Physics, Geometric Computing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2311.02572" label="2311.02572">
-        <attvalues>
-          <attvalue for="0" value="Multiple Object Tracking based on Occlusion-Aware Embedding Consistency&#10;  Learning" />
-          <attvalue for="1" value="  The Joint Detection and Embedding (JDE) framework has achieved remarkable&#10;progress for multiple object tracking. Existing methods often employ extracted&#10;embeddings to re-establish associations between new detections and previously&#10;disrupted tracks. However, the reliability of embeddings diminishes when the&#10;region of the occluded object frequently contains adjacent objects or clutters,&#10;especially in scenarios with severe occlusion. To alleviate this problem, we&#10;propose a novel multiple object tracking method based on visual embedding&#10;consistency, mainly including: 1) Occlusion Prediction Module (OPM) and 2)&#10;Occlusion-Aware Association Module (OAAM). The OPM predicts occlusion&#10;information for each true detection, facilitating the selection of valid&#10;samples for consistency learning of the track's visual embedding. The OAAM&#10;leverages occlusion cues and visual embeddings to generate two separate&#10;embeddings for each track, guaranteeing consistency in both unoccluded and&#10;occluded detections. By integrating these two modules, our method is capable of&#10;addressing track interruptions caused by occlusion in online tracking&#10;scenarios. Extensive experimental results demonstrate that our approach&#10;achieves promising performance levels in both unoccluded and occluded tracking&#10;scenarios.&#10;" />
-          <attvalue for="2" value="&#10;Online Multiple Object Tracking (MOT) plays an important role in video surveillance~\cite{oh2011large}, autonomous driving~\cite{sun2020scalability}, and many other fields~\cite{zhang2020multi}. Given the input video frames, online MOT methods locate track positions frame by frame while giving unique identifications for each of them. Though online MOT methods have made significant progress in recent decades, the persistence of tracking failures due to occlusion remains a major obstacle to the widespread practical adoption of such methods. In order to resolve the above problem, the common practice involves a bottom-up association strategy~\cite{Zhang_Sheng_Wu_Wang_Lyu_Ke_Xiong_2020, du2023strongsort}, which prioritizes track-detection association as the initial step, followed by track-track association. Within the track-track association phase, visual embeddings are extracted for each track using an external re-identification (re-ID) network, which is then employed to compute the similarity between different tracks. Although those methods can recover the loss track even if some detections are missed for a long range due to occlusion, the re-identification network's training and inference procedures are time-consuming~\cite{ Liu_Li_Bai_Wang_Wang_2021, Liu_Chu_Liu_Yu_2020, aharon2022bot}. Furthermore, the adopted association strategy necessitates the use of current data to amend the historical segment of the track, which is not suitable for online tracking. &#10;&#10;To mitigate the above issues, methods based on JDE paradigm are becoming prevalent, which learn both detection and re-ID embedding extraction in a shared model~\cite{wang2020towards}. &#10;In particular, FairMOT~\cite{zhang_wang_wang_zeng_liu_2021} achieves a good balance between tracking performance and efficiency by adopting a point-based framework. It follows MOTDT~\cite{Chen_Ai_Zhuang_Shang_2018} and uses a hierarchical track-detection association method, which combines Kalman filter, re-ID embeddings, and Intersection-over-Union (IOU) metrics to facilitate the matching of tracks with detections. &#10; OUTrack~\cite{liu_chen_chu_yuan_liu_zhang_yu_2022} optimizes the visual embedding by the unsupervised reID learning which does not require any (pseudo) identity information nor suffer from the scalability issue. What's more, it designs an occlusion estimation module which is used to recover the miss detections caused by occlusions. Although these methods have made notable progress, they depend on the assumption that the importance of visual embedding is higher than motion information, which is not applicable with severe occlusion. Moreover, these methods do not differentiate between the visual embeddings of occluded and unoccluded detections when updating the track's embeddings. &#10;&#10;To address these issues, we propose a novel multiple object tracking method based on visual embedding consistency to conduct track-detection associations in severe occlusions. The consistency means the visual embeddings of all detections belonging to the same track should be similar. During the track-detection association process, a Smooth Visual Embedding (SVE) is maintained to characterize the visual feature for each track. The SVE is iteratively refined and updated by the current matched detection. When the detection is occluded, the SVE is updated with the embeddings of occluding detections. This update disrupts the consistency of the visual embedding in the subsequent association process. In this paper, we decompose the SVE into two embeddings: long-term embedding and short-term embedding. Long-term embedding characterizes the visual feature for unoccluded detections in a track. Short-term embedding characterizes the visual feature for occluded detections in a track. The two embeddings can realize the consistency of the track's visual embedding in both unoccluded and occluded situations.&#10;&#10;Specifically, we devise two modules: Occlusion Prediction Module (OPM) and Occlusion-Aware Association Module (OAAM). The OPM predicts occlusion information for each detection, indicating the degree a detection's position is occluded. Through the information, we can select unoccluded true detections as valid samples to conduct the consistency learning of the track's visual embedding. In the OAAM, we design a two-stage track-detection association strategy. &#10;In each stage, we get the matching result by computing the similarities between tracks and detections based on their visual embeddings and bounding boxes. In stage one, the short-term embedding is treated as the track's visual embedding and updated by each matched detection's visual embedding. In stage two, the long-term embedding is treated as the track's visual embedding and only updated by the matched unoccluded detection's visual embedding. The two-stage strategy dynamically associates and updates the tracks for unocclued and occluded detections. &#10;&#10;To summarize, we make the following contributions: &#10;\begin{itemize}&#10;&#10;\item We conceptualize the track-detection association under severe occlusion as the task of preserving visual embedding consistency for both unoccluded and occluded detections, greatly simplifying the association process. &#10;&#10;\item We design long-term embedding and short-term embedding to characterize unoccluded and occluded detections, which help maintain the consistency of the track’s visual embedding in severe occluded situations. &#10;&#10;\item Based on the above embeddings, we build Occlusion Prediction Module and Occlusion-Aware Association Module to dynamically associate and update the tracks for unoccluded and&#10;occluded detections to improve the matching result under severe occlusion.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Visual Embeddings, Occlusion Handling, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1809.04427" label="1809.04427">
-        <attvalues>
-          <attvalue for="0" value="Real-time Multiple People Tracking with Deeply Learned Candidate&#10;  Selection and Person Re-Identification" />
-          <attvalue for="1" value="  Online multi-object tracking is a fundamental problem in time-critical video&#10;analysis applications. A major challenge in the popular tracking-by-detection&#10;framework is how to associate unreliable detection results with existing&#10;tracks. In this paper, we propose to handle unreliable detection by collecting&#10;candidates from outputs of both detection and tracking. The intuition behind&#10;generating redundant candidates is that detection and tracks can complement&#10;each other in different scenarios. Detection results of high confidence prevent&#10;tracking drifts in the long term, and predictions of tracks can handle noisy&#10;detection caused by occlusion. In order to apply optimal selection from a&#10;considerable amount of candidates in real-time, we present a novel scoring&#10;function based on a fully convolutional neural network, that shares most&#10;computations on the entire image. Moreover, we adopt a deeply learned&#10;appearance representation, which is trained on large-scale person&#10;re-identification datasets, to improve the identification ability of our&#10;tracker. Extensive experiments show that our tracker achieves real-time and&#10;state-of-the-art performance on a widely used people tracking benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.01297" label="2201.01297">
-        <attvalues>
-          <attvalue for="0" value="Online Multi-Object Tracking with Unsupervised Re-Identification&#10;  Learning and Occlusion Estimation" />
-          <attvalue for="1" value="  Occlusion between different objects is a typical challenge in Multi-Object&#10;Tracking (MOT), which often leads to inferior tracking results due to the&#10;missing detected objects. The common practice in multi-object tracking is&#10;re-identifying the missed objects after their reappearance. Though tracking&#10;performance can be boosted by the re-identification, the annotation of identity&#10;is required to train the model. In addition, such practice of re-identification&#10;still can not track those highly occluded objects when they are missed by the&#10;detector. In this paper, we focus on online multi-object tracking and design&#10;two novel modules, the unsupervised re-identification learning module and the&#10;occlusion estimation module, to handle these problems. Specifically, the&#10;proposed unsupervised re-identification learning module does not require any&#10;(pseudo) identity information nor suffer from the scalability issue. The&#10;proposed occlusion estimation module tries to predict the locations where&#10;occlusions happen, which are used to estimate the positions of missed objects&#10;by the detector. Our study shows that, when applied to state-of-the-art MOT&#10;methods, the proposed unsupervised re-identification learning is comparable to&#10;supervised re-identification learning, and the tracking performance is further&#10;improved by the proposed occlusion estimation module.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.12902" label="2103.12902">
-        <attvalues>
-          <attvalue for="0" value="Region Similarity Representation Learning" />
-          <attvalue for="1" value="  We present Region Similarity Representation Learning (ReSim), a new approach&#10;to self-supervised representation learning for localization-based tasks such as&#10;object detection and segmentation. While existing work has largely focused on&#10;solely learning global representations for an entire image, ReSim learns both&#10;regional representations for localization as well as semantic image-level&#10;representations. ReSim operates by sliding a fixed-sized window across the&#10;overlapping area between two views (e.g., image crops), aligning these areas&#10;with their corresponding convolutional feature map regions, and then maximizing&#10;the feature similarity across views. As a result, ReSim learns spatially and&#10;semantically consistent feature representation throughout the convolutional&#10;feature maps of a neural network. A shift or scale of an image region, e.g., a&#10;shift or scale of an object, has a corresponding change in the feature maps;&#10;this allows downstream tasks to leverage these representations for&#10;localization. Through object detection, instance segmentation, and dense pose&#10;estimation experiments, we illustrate how ReSim learns representations which&#10;significantly improve the localization and classification performance compared&#10;to a competitive MoCo-v2 baseline: $+2.7$ AP$^{\text{bb}}_{75}$ VOC, $+1.1$&#10;AP$^{\text{bb}}_{75}$ COCO, and $+1.9$ AP$^{\text{mk}}$ Cityscapes. Code and&#10;pre-trained models are released at: \url{https://github.com/Tete-Xiao/ReSim}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\@startsection{paragraph}{4}{\z@}&#10; {.5em \@plus1ex \@minus.2ex}{-.5em}{\normalfont\bfseries}{Self-supervised representation learning.}&#10;The goal of representation learning is to reveal the intrinsic qualities of data in such a way that they are informative and effective for a desired task~\cite{bengio2013representation}.&#10;Practically, this often manifests as pre-training a deep network so that it can be finetuned for a particular downstream task, see~\cite{devlin2018bert, donahue2014decaf, erhan2010does,goodfellow2016deep, he2019momentum, henaff2019data, lecun2015deep, radford2018improving, zeiler2014visualizing}. Recently, SimCLR~\cite{chen2020simple} and MoCo~\cite{chen2020improved, he2019momentum} demonstrated substantial improvements by using similar forms of instance contrastive learning where a network was trained to identify a pair of views originating from the same image when contrasted with a large set of views from other images.&#10;Following SimCLR and MoCo, later works, such as SwAV~\cite{caron2020unsupervised} and BYOL~\cite{grill2020bootstrap}, reported substantial improvements for image classification tasks, but as several follow-up works have shown~\cite{ding2021unsupervised, xie2021detco, yang2021instance}, SwAV and BYOL do not tend to lead to improvements on localization-based tasks such as object detection and segmentation. This decrease in performance indicated that strictly optimizing global, image-level representations could decrease performance for tasks which require localization. &#10;&#10;In contrast to prior work that leveraged instance discrimination to learn global, image-level representations, we propose a region-based pretext task to learn representations for tasks which require both localization and semantic classification. In earlier work, several authors proposed representation learning at the pixel or region level via color prediction~\cite{vondrick2018tracking, zhang2016colorful}, key-point encoding \cite{thewlis2017unsupervised, minderer2019unsupervised}, optical flow similarity~\cite{dosovitskiy2015flownet}, and cycle consistency or frame prediction in videos~\cite{li2019joint, wang2019learning}. Our work differs in that we build on instance discrimination pretext task and simultaneously learn semantic features from an entire image as well as regional features across multiple scales, shifts, and resolutions.&#10;&#10;\@startsection{paragraph}{4}{\z@}&#10; {.5em \@plus1ex \@minus.2ex}{-.5em}{\normalfont\bfseries}{Object detection and instance segmentation.}&#10;Object detection and segmentation localize and classify object bounding boxes or pixels within an image -- see \cite{liu2020deep} for a survey and review. Many commonly used object detection and instance segmentation techniques such as Fast-RCNN~\cite{girshick2015fast}, Faster-RCNN~\cite{ren2016faster}, and Mask-RCNN~\cite{he2017mask} operate through a two-stage process in which they extract region features from convolutional feature maps, regress the location of the region to align with a ground truth bounding box, and then classify the region. &#10;&#10;When regressing the location of the regional features, small adjustments to the region location within the convolutional feature map can have a large impact on the classification, \eg, shifting a region may cause it to overlap with different objects in the input pixel space. &#10;As a result, the regional features of the convolutional feature maps require spatial sensitivity for regression as well as semantic sensitivity for classification. Existing self-supervised methods have largely focused on learning strong semantic representations by average pooling the final convolutional layer, and they do not explicitly maintain spatial consistency throughout the convolutional layers. In this work, we propose a self-supervised learning method which learns both spatially consistent and semantically sensitive regional features.&#10;&#10;\@startsection{paragraph}{4}{\z@}&#10; {.5em \@plus1ex \@minus.2ex}{-.5em}{\normalfont\bfseries}{Pixel and grid-based contrastive learning.}&#10;Recently, several related papers emerged: \cite{pinheiro2020unsupervised} explored pixel-level contrastive learning, whereas our work examines region-level consistency across the convolutional features. Wang et al.~ \cite{wang2020dense} proposed contrastive learning using a grid of feature vectors over the feature maps. Rather than maximizing the similarity of the regions, their solution was more akin to clustering, in that they compared all combination of feature vectors from the feature grid and maximized the similarity of the most similar pair.&#10;As we show below, our method demonstrates superior performance over these models.&#10;&#10;" />
-          <attvalue for="4" value="Representation Learning, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2408.05635" label="2408.05635">
-        <attvalues>
-          <attvalue for="0" value="Visual SLAM with 3D Gaussian Primitives and Depth Priors Enabling Novel&#10;  View Synthesis" />
-          <attvalue for="1" value="  Conventional geometry-based SLAM systems lack dense 3D reconstruction&#10;capabilities since their data association usually relies on feature&#10;correspondences. Additionally, learning-based SLAM systems often fall short in&#10;terms of real-time performance and accuracy. Balancing real-time performance&#10;with dense 3D reconstruction capabilities is a challenging problem. In this&#10;paper, we propose a real-time RGB-D SLAM system that incorporates a novel view&#10;synthesis technique, 3D Gaussian Splatting, for 3D scene representation and&#10;pose estimation. This technique leverages the real-time rendering performance&#10;of 3D Gaussian Splatting with rasterization and allows for differentiable&#10;optimization in real time through CUDA implementation. We also enable mesh&#10;reconstruction from 3D Gaussians for explicit dense 3D reconstruction. To&#10;estimate accurate camera poses, we utilize a rotation-translation decoupled&#10;strategy with inverse optimization. This involves iteratively updating both in&#10;several iterations through gradient-based optimization. This process includes&#10;differentiably rendering RGB, depth, and silhouette maps and updating the&#10;camera parameters to minimize a combined loss of photometric loss, depth&#10;geometry loss, and visibility loss, given the existing 3D Gaussian map.&#10;However, 3D Gaussian Splatting (3DGS) struggles to accurately represent&#10;surfaces due to the multi-view inconsistency of 3D Gaussians, which can lead to&#10;reduced accuracy in both camera pose estimation and scene reconstruction. To&#10;address this, we utilize depth priors as additional regularization to enforce&#10;geometric constraints, thereby improving the accuracy of both pose estimation&#10;and 3D reconstruction. We also provide extensive experimental results on public&#10;benchmark datasets to demonstrate the effectiveness of our proposed methods in&#10;terms of pose accuracy, geometric accuracy, and rendering performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Simultaneous Localization and Mapping (SLAM) is a technique for estimating sensor motion and reconstructing the structure of an unknown environment. When SLAM uses cameras, it is specifically referred to as visual SLAM (vSLAM) because it relies solely on visual information. Visual SLAM can be categorized into monocular, stereo, RGB-D, and multi-camera SLAM. Among these, RGB-D SLAM, which provides a depth map as a prior, is particularly useful for many applications such as AR/VR~\cite{edi_wang}, autonomous driving perception~\cite{feng2023robotic, feng2023subsurface, zhang2020manipulator, gao2024decentralized, gao2024adaptive}, 3D reconstruction~\cite{Chen_2023_CVPR, icra23_weihan, ral_jinglun}, novel view synthesis~\cite{Li_2024_CVPR} and navigation~\cite{icia_jinglun,ral_zheng} due to its potential to generate a dense 3D map. Bayesian methods are important in SLAM for providing a framework of handling uncertainty and being able to make predictions from incomplete or noisy data~\cite{zhuang2022defending, zhuang2022robust}. The SLAM problem is often formulated in probabilistic terms, where Bayes' theorem is used to update the knowledge of the robot about its position and the map any time new sensor data becomes available.&#10;&#10;Current research, highlighted by the comprehensive investigations of Ming et al. ~\cite{ming2024benchmarking} in NeRF-based SLAM systems. They found that traditional SLAM faces challenges in matching sensor observations with 3D landmark points, making geometry estimation difficult for unobserved regions and potentially leading to tracking loss. Implicit neural representations, like NeRF, use multi-layer perceptrons (MLPs) to represent the scene. In addition, research fields including Edge computing and Reinforcement learning also integrate well on SLAM. Edge computing ~\cite{li2023predictive, zong2022cocktail} involves processing data near the source of data generation, such as on the robot itself or on a nearby device in the network. Reinforcement learning ~\cite{10406020, li2023red} can be used to develop optimal navigation strategies that allow the robot to efficiently explore unknown areas while avoiding obstacles. By learning from its environment, the robot can make decisions that balance exploration and exploitation. These approaches can complement SLAM by providing quick, localized processing while still allowing for cloud-based analysis if needed.&#10;&#10;They categorize NeRF-based methods in the SLAM domain into monocular SLAM, RGB-D SLAM, LiDAR SLAM, and multi-sensor SLAM. Additionally, they provide an overview of the NeRF-based SLAM pipeline, illustrating the framework and summarizing the various approaches used in NeRF-based SLAM methods which is detailed in Figure~\ref{fig:nerf-slam}. Figure~\ref{fig:nerf-slam} directly summarizes the current methods of applying NeRF in SLAM. &#10;Finally, they benchmark NeRF in the SLAM domain and provide comprehensive results to evaluate NeRF-based SLAM methods.&#10;&#10;In contract to NeRF-based SLAM, 3D Gaussians~\cite{kerbl3Dgaussians} recently have emerged as a promising 3D scene representation. 3D Gaussian Splatting (3DGS) leverages the sparse nature of 3D environments by iterating over primitives for rasterization instead of tracing rays, effectively capturing the details of these scenes and speeding up rendering. There have been many recent advancements and applications of 3DGS, such as in the capture of static scenes where 3DGS introduces an innovative differentiable renderer known as Fuzzy Metaballs ~\cite{keselman2022approximate}. These facilitate low-latency differential rendering on CPUs and focus on depicting shapes through depth maps and silhouettes. In various vision-related tasks, particularly dynamic scene capture ~\cite{wu20244d}, 3DGS constructs an efficient Gaussian splatting framework that accommodates changes in Gaussian motion and shape over time, significantly improving dynamic scene capture. In the realm of 3D generation ~\cite{tang2023dreamgaussian}, it also adapts 3D Gaussian splatting into generative settings for 3D content creation, significantly reducing the generation time of optimization-based 2D lifting methods&#10;&#10;In this paper, we aim to leverage the fast rendering performance and promising explicit 3D scene representation of 3DGS~\cite{kerbl3Dgaussians}, along with the accurate pose estimation strategy from Stereo-NEC~\cite{wang2024stereo}, to generate dense 3D reconstructions while maintaining accurate camera pose estimation~\cite{song2024eta}. We implement our pipeline using SplaTAM~\cite{spatam} as our foundation. &#10;The key contributions of our proposed method are as follows:&#10;\begin{itemize}&#10;\item We employ a rotation-translation decoupled strategy combined with inverse optimization to estimate accurate camera poses.&#10;\item We utilize 3D Gaussian primitives to represent scenes, facilitating dense reconstruction.&#10;\item We utilize 3D Gaussian Splatting with rasterization, allowing for real-time differentiable optimization via CUDA.&#10;\item We incorporate depth priors as additional regularization to enforce geometric constraints, thereby enhancing the accuracy of both pose estimation and 3D reconstruction.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, SLAM Systems, Mathematics, Computer Science, Computer Graphics, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2308.02670" label="2308.02670">
-        <attvalues>
-          <attvalue for="0" value="EDI: ESKF-based Disjoint Initialization for Visual-Inertial SLAM Systems" />
-          <attvalue for="1" value="  Visual-inertial initialization can be classified into joint and disjoint&#10;approaches. Joint approaches tackle both the visual and the inertial parameters&#10;together by aligning observations from feature-bearing points based on IMU&#10;integration then use a closed-form solution with visual and acceleration&#10;observations to find initial velocity and gravity. In contrast, disjoint&#10;approaches independently solve the Structure from Motion (SFM) problem and&#10;determine inertial parameters from up-to-scale camera poses obtained from pure&#10;monocular SLAM. However, previous disjoint methods have limitations, like&#10;assuming negligible acceleration bias impact or accurate rotation estimation by&#10;pure monocular SLAM. To address these issues, we propose EDI, a novel approach&#10;for fast, accurate, and robust visual-inertial initialization. Our method&#10;incorporates an Error-state Kalman Filter (ESKF) to estimate gyroscope bias and&#10;correct rotation estimates from monocular SLAM, overcoming dependence on pure&#10;monocular SLAM for rotation estimation. To estimate the scale factor without&#10;prior information, we offer a closed-form solution for initial velocity, scale,&#10;gravity, and acceleration bias estimation. To address gravity and acceleration&#10;bias coupling, we introduce weights in the linear least-squares equations,&#10;ensuring acceleration bias observability and handling outliers. Extensive&#10;evaluation on the EuRoC dataset shows that our method achieves an average scale&#10;error of 5.8% in less than 3 seconds, outperforming other state-of-the-art&#10;disjoint visual-inertial initialization approaches, even in challenging&#10;environments and with artificial noise corruption.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.01907" label="2106.01907">
-        <attvalues>
-          <attvalue for="0" value="Robotic Inspection of Underground Utilities for Construction Survey&#10;  Using a Ground Penetrating Radar" />
-          <attvalue for="1" value="  Ground Penetrating Radar (GPR) is a very useful non-destructive evaluation&#10;(NDE) device for locating and mapping underground assets prior to digging and&#10;trenching efforts in construction. This paper presents a novel robotic system&#10;to automate the GPR data collection process, localize the underground&#10;utilities, interpret and reconstruct the underground objects for better&#10;visualization allowing regular non-professional users to understand the survey&#10;results. This system is composed of three modules: 1) an Omni-directional&#10;robotic data collection platform, that carries an RGB-D camera with an Inertial&#10;Measurement Unit (IMU) and a GPR antenna to perform automatic GPR data&#10;collection, and tag each GPR measurement with visual positioning information at&#10;every sampling step; 2) a learning-based migration module to interpret the raw&#10;GPR B-scan image into a 2D cross-section model of objects; 3) a 3D&#10;reconstruction module, i.e., GPRNet, to generate underground utility model&#10;represented as fine 3D point cloud. Comparative studies are performed on&#10;synthetic data and field GPR raw data with various incompleteness and noise.&#10;Experimental results demonstrate that our proposed method achieves a $30.0\%$&#10;higher GPR imaging accuracy in mean Intersection Over Union (IoU) than the&#10;conventional back projection (BP) migration approach and $6.9\%$-$7.2\%$ less&#10;loss in Chamfer Distance (CD) than baseline methods regarding point cloud model&#10;reconstruction. The GPR-based robotic inspection provides an effective tool for&#10;civil engineers to detect and survey underground utilities before construction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.00152" label="2304.00152">
-        <attvalues>
-          <attvalue for="0" value="Learning the Distribution of Errors in Stereo Matching for Joint&#10;  Disparity and Uncertainty Estimation" />
-          <attvalue for="1" value="  We present a new loss function for joint disparity and uncertainty estimation&#10;in deep stereo matching. Our work is motivated by the need for precise&#10;uncertainty estimates and the observation that multi-task learning often leads&#10;to improved performance in all tasks. We show that this can be achieved by&#10;requiring the distribution of uncertainty to match the distribution of&#10;disparity errors via a KL divergence term in the network's loss function. A&#10;differentiable soft-histogramming technique is used to approximate the&#10;distributions so that they can be used in the loss. We experimentally assess&#10;the effectiveness of our approach and observe significant improvements in both&#10;disparity and uncertainty prediction on large datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.16812" label="2312.16812">
-        <attvalues>
-          <attvalue for="0" value="Spacetime Gaussian Feature Splatting for Real-Time Dynamic View&#10;  Synthesis" />
-          <attvalue for="1" value="  Novel view synthesis of dynamic scenes has been an intriguing yet challenging&#10;problem. Despite recent advancements, simultaneously achieving high-resolution&#10;photorealistic results, real-time rendering, and compact storage remains a&#10;formidable task. To address these challenges, we propose Spacetime Gaussian&#10;Feature Splatting as a novel dynamic scene representation, composed of three&#10;pivotal components. First, we formulate expressive Spacetime Gaussians by&#10;enhancing 3D Gaussians with temporal opacity and parametric motion/rotation.&#10;This enables Spacetime Gaussians to capture static, dynamic, as well as&#10;transient content within a scene. Second, we introduce splatted feature&#10;rendering, which replaces spherical harmonics with neural features. These&#10;features facilitate the modeling of view- and time-dependent appearance while&#10;maintaining small size. Third, we leverage the guidance of training error and&#10;coarse depth to sample new Gaussians in areas that are challenging to converge&#10;with existing pipelines. Experiments on several established real-world datasets&#10;demonstrate that our method achieves state-of-the-art rendering quality and&#10;speed, while retaining compact storage. At 8K resolution, our lite-version&#10;model can render at 60 FPS on an Nvidia RTX 4090 GPU. Our code is available at&#10;https://github.com/oppo-us-research/SpacetimeGaussians.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.03762" label="2203.03762">
-        <attvalues>
-          <attvalue for="0" value="Defending Graph Convolutional Networks against Dynamic Graph&#10;  Perturbations via Bayesian Self-supervision" />
-          <attvalue for="1" value="  In recent years, plentiful evidence illustrates that Graph Convolutional&#10;Networks (GCNs) achieve extraordinary accomplishments on the node&#10;classification task. However, GCNs may be vulnerable to adversarial attacks on&#10;label-scarce dynamic graphs. Many existing works aim to strengthen the&#10;robustness of GCNs; for instance, adversarial training is used to shield GCNs&#10;against malicious perturbations. However, these works fail on dynamic graphs&#10;for which label scarcity is a pressing issue. To overcome label scarcity,&#10;self-training attempts to iteratively assign pseudo-labels to highly confident&#10;unlabeled nodes but such attempts may suffer serious degradation under dynamic&#10;graph perturbations. In this paper, we generalize noisy supervision as a kind&#10;of self-supervised learning method and then propose a novel Bayesian&#10;self-supervision model, namely GraphSS, to address the issue. Extensive&#10;experiments demonstrate that GraphSS can not only affirmatively alert the&#10;perturbations on dynamic graphs but also effectively recover the prediction of&#10;a node classifier when the graph is under such perturbations. These two&#10;advantages prove to be generalized over three classic GCNs across five public&#10;graph datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.05526" label="2405.05526">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking Neural Radiance Fields for Autonomous Robots: An Overview" />
-          <attvalue for="1" value="  Neural Radiance Fields (NeRF) have emerged as a powerful paradigm for 3D&#10;scene representation, offering high-fidelity renderings and reconstructions&#10;from a set of sparse and unstructured sensor data. In the context of autonomous&#10;robotics, where perception and understanding of the environment are pivotal,&#10;NeRF holds immense promise for improving performance. In this paper, we present&#10;a comprehensive survey and analysis of the state-of-the-art techniques for&#10;utilizing NeRF to enhance the capabilities of autonomous robots. We especially&#10;focus on the perception, localization and navigation, and decision-making&#10;modules of autonomous robots and delve into tasks crucial for autonomous&#10;operation, including 3D reconstruction, segmentation, pose estimation,&#10;simultaneous localization and mapping (SLAM), navigation and planning, and&#10;interaction. Our survey meticulously benchmarks existing NeRF-based methods,&#10;providing insights into their strengths and limitations. Moreover, we explore&#10;promising avenues for future research and development in this domain. Notably,&#10;we discuss the integration of advanced techniques such as 3D Gaussian splatting&#10;(3DGS), large language models (LLM), and generative AIs, envisioning enhanced&#10;reconstruction efficiency, scene understanding, decision-making capabilities.&#10;This survey serves as a roadmap for researchers seeking to leverage NeRFs to&#10;empower autonomous robots, paving the way for innovative solutions that can&#10;navigate and interact seamlessly in complex environments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.02657" label="2210.02657">
-        <attvalues>
-          <attvalue for="0" value="Predictive Edge Caching through Deep Mining of Sequential Patterns in&#10;  User Content Retrievals" />
-          <attvalue for="1" value="  Edge caching plays an increasingly important role in boosting user content&#10;retrieval performance while reducing redundant network traffic. The&#10;effectiveness of caching ultimately hinges on the accuracy of predicting&#10;content popularity in the near future. However, at the network edge, content&#10;popularity can be extremely dynamic due to diverse user content retrieval&#10;behaviors and the low-degree of user multiplexing. It's challenging for the&#10;traditional reactive caching systems to keep up with the dynamic content&#10;popularity patterns. In this paper, we propose a novel Predictive Edge Caching&#10;(PEC) system that predicts the future content popularity using fine-grained&#10;learning models that mine sequential patterns in user content retrieval&#10;behaviors, and opportunistically prefetches contents predicted to be popular in&#10;the near future using idle network bandwidth. Through extensive experiments&#10;driven by real content retrieval traces, we demonstrate that PEC can adapt to&#10;highly dynamic content popularity, and significantly improve cache hit ratio&#10;and reduce user content retrieval latency over the state-of-art caching&#10;policies. More broadly, our study demonstrates that edge caching performance&#10;can be boosted by deep mining of user content retrieval behaviors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.10606" label="2207.10606">
-        <attvalues>
-          <attvalue for="0" value="Approximate Differentiable Rendering with Algebraic Surfaces" />
-          <attvalue for="1" value="  Differentiable renderers provide a direct mathematical link between an&#10;object's 3D representation and images of that object. In this work, we develop&#10;an approximate differentiable renderer for a compact, interpretable&#10;representation, which we call Fuzzy Metaballs. Our approximate renderer focuses&#10;on rendering shapes via depth maps and silhouettes. It sacrifices fidelity for&#10;utility, producing fast runtimes and high-quality gradient information that can&#10;be used to solve vision tasks. Compared to mesh-based differentiable renderers,&#10;our method has forward passes that are 5x faster and backwards passes that are&#10;30x faster. The depth maps and silhouette images generated by our method are&#10;smooth and defined everywhere. In our evaluation of differentiable renderers&#10;for pose estimation, we show that our method is the only one comparable to&#10;classic techniques. In shape from silhouette, our method performs well using&#10;only gradient descent and a per-pixel loss, without any surrogate losses or&#10;regularization. These reconstructions work well even on natural video sequences&#10;with segmentation artifacts. Project page:&#10;https://leonidk.github.io/fuzzy-metaballs&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.08528" label="2310.08528">
-        <attvalues>
-          <attvalue for="0" value="4D Gaussian Splatting for Real-Time Dynamic Scene Rendering" />
-          <attvalue for="1" value="  Representing and rendering dynamic scenes has been an important but&#10;challenging task. Especially, to accurately model complex motions, high&#10;efficiency is usually hard to guarantee. To achieve real-time dynamic scene&#10;rendering while also enjoying high training and storage efficiency, we propose&#10;4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes&#10;rather than applying 3D-GS for each individual frame. In 4D-GS, a novel&#10;explicit representation containing both 3D Gaussians and 4D neural voxels is&#10;proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is&#10;proposed to efficiently build Gaussian features from 4D neural voxels and then&#10;a lightweight MLP is applied to predict Gaussian deformations at novel&#10;timestamps. Our 4D-GS method achieves real-time rendering under high&#10;resolutions, 82 FPS at an 800$\times$800 resolution on an RTX 3090 GPU while&#10;maintaining comparable or better quality than previous state-of-the-art&#10;methods. More demos and code are available at&#10;https://guanjunwu.github.io/4dgs/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.07225" label="2403.07225">
-        <attvalues>
-          <attvalue for="0" value="Stereo-NEC: Enhancing Stereo Visual-Inertial SLAM Initialization with&#10;  Normal Epipolar Constraints" />
-          <attvalue for="1" value="  We propose an accurate and robust initialization approach for stereo&#10;visual-inertial SLAM systems. Unlike the current state-of-the-art method, which&#10;heavily relies on the accuracy of a pure visual SLAM system to estimate&#10;inertial variables without updating camera poses, potentially compromising&#10;accuracy and robustness, our approach offers a different solution. We realize&#10;the crucial impact of precise gyroscope bias estimation on rotation accuracy.&#10;This, in turn, affects trajectory accuracy due to the accumulation of&#10;translation errors. To address this, we first independently estimate the&#10;gyroscope bias and use it to formulate a maximum a posteriori problem for&#10;further refinement. After this refinement, we proceed to update the rotation&#10;estimation by performing IMU integration with gyroscope bias removed from&#10;gyroscope measurements. We then leverage robust and accurate rotation estimates&#10;to enhance translation estimation via 3-DoF bundle adjustment. Moreover, we&#10;introduce a novel approach for determining the success of the initialization by&#10;evaluating the residual of the normal epipolar constraint. Extensive&#10;evaluations on the EuRoC dataset illustrate that our method excels in accuracy&#10;and robustness. It outperforms ORB-SLAM3, the current leading stereo&#10;visual-inertial initialization method, in terms of absolute trajectory error&#10;and relative rotation error, while maintaining competitive computational speed.&#10;Notably, even with 5 keyframes for initialization, our method consistently&#10;surpasses the state-of-the-art approach using 10 keyframes in rotation&#10;accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.04829" label="2406.04829">
-        <attvalues>
-          <attvalue for="0" value="IOR: Inversed Objects Replay for Incremental Object Detection" />
-          <attvalue for="1" value="  Existing Incremental Object Detection (IOD) methods partially alleviate&#10;catastrophic forgetting when incrementally detecting new objects in real-world&#10;scenarios. However, many of these methods rely on the assumption that unlabeled&#10;old-class objects may co-occur with labeled new-class objects in the&#10;incremental data. When unlabeled old-class objects are absent, the performance&#10;of existing methods tends to degrade. The absence can be mitigated by&#10;generating old-class samples, but it incurs high costs. This paper argues that&#10;previous generation-based IOD suffer from redundancy, both in the use of&#10;generative models, which require additional training and storage, and in the&#10;overproduction of generated samples, many of which do not contribute&#10;significantly to performance improvements. To eliminate the redundancy, we&#10;propose Inversed Object Replay (IOR). Specifically, we generate old-class&#10;samples by inversing the original detectors, thus eliminating the necessity of&#10;training and storing additional generative models. We propose augmented replay&#10;to reuse the objects in generated samples, reducing redundant generations.&#10;Moreover, we propose high-value knowledge distillation focusing on the&#10;positions of old-class objects overwhelmed by the background, which transfers&#10;the knowledge to the incremental detector. Extensive experiments conducted on&#10;MS COCO 2017 demonstrate that our method can efficiently improve detection&#10;performance in IOD scenarios with the absence of old-class objects.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Incremental learning aims to address the challenge of catastrophic forgetting \cite{van2019three} that occurs when learning from dynamic data distributions. Typically, the process of learning new distributions leads to a reduction in the model's ability to retain knowledge of previously learned distributions \cite{wang2024comprehensive}. In incremental object detection (IOD), a specific scenario exists where images contain both unlabeled old-class objects and labeled new-class objects (co-occurrence). In this scenario, the label's distribution is dynamic, while the object's distribution remains static. Some existing methods address this scenario by leveraging unlabeled old-class objects in the images. However, the absence of old-class objects in images (non co-occurrence) is more general in real-world applications. In this scenario, the performance of IOD suffers performance degradation due to the lack of old-class information. Therefore, it is crucial to study IOD under non co-occurrence scenarios.&#10;&#10;Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}. Most IOD methods are distillation-based, transferring the old-class knowledge from the original to the incremental detector using distillation. However, the performance of such methods degrades significantly in non co-occurrence scenarios \cite{yang2023pseudo}. Previous works \cite{yang2023pseudo,kim2024sddgr} introduce generative models to generate samples of absent classes, thereby ensuring knowledge transfer. However, the additional generators and generation processes introduced by these methods result in high computational costs. Although some methods are developed to effectively reduce computational complexity \cite{dai2024sketch, liu2024resource, yang2024clip, chen2024scp, chang2023hdsuper}, these methods still face challenges when deployed in resource-constrained real-world scenarios.&#10;&#10;Generative models attempt to align generated data distribution as closely as possible with real-data distribution. However, detectors only focus on certain distinctive features in the real data. The additional generative models may fail to capture the distinctive features of the detector concerns \cite{huang2024etag}. Compared with these methods, inversing \cite{yin2020dreaming} the detector to generate samples not only captures distinctive features but also eliminates the requirement for additional generative models, as shown in Fig.\ref{fig: IOR vs. Other Method}. However, we observe redundancy in inversing, as the features of the inversed samples tend to be consistent. To illustrate this phenomenon, we use t-sne \cite{van2008visualizing} to visualize the positions of the generated objects' embedding features. As shown in Fig.\ref{fig: 1vs30}, the locations of the generated objects are highly concentrated. We believe that generating only a small number of representative samples is sufficient to reflect the features of the detector concern. &#10;&#10;To tackle the above problems, we propose the Inversed Objects Replay (IOR). We argue that the extra cost stems from the redundancy of generative models and sample generations. We exclude redundant generative models by inversing the original detector, eliminating the necessity of training or saving generative models. We alleviate redundant sample generations by employing augmented replay, which facilitates the reuse of objects from the generated samples and thus reduces the need to generate massive samples. To effectively utilize the generated objects, we distill incremental data with replayed objects. However, the generated objects are overwhelmed by the background, leading to ineffective distillation. Therefore, we propose high-value knowledge distillation, focusing on distilling outputs relevant to old-class objects.&#10;&#10;Our contributions can be summarized as follows: 1) To compensate for the absent objects at low cost, we inverse the original detector to generate old-class samples and reuse the objects in generated samples by augmented replay. To our knowledge, this is the first work to complement absent objects by inversing the detector in the IOD. 2) To effectively utilize the generated objects, we propose high-value distillation to focus on the objects overwhelmed by the background. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Knowledge Distillation, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Incremental Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1904.07734" label="1904.07734">
-        <attvalues>
-          <attvalue for="0" value="Three scenarios for continual learning" />
-          <attvalue for="1" value="  Standard artificial neural networks suffer from the well-known issue of&#10;catastrophic forgetting, making continual or lifelong learning difficult for&#10;machine learning. In recent years, numerous methods have been proposed for&#10;continual learning, but due to differences in evaluation protocols it is&#10;difficult to directly compare their performance. To enable more structured&#10;comparisons, we describe three continual learning scenarios based on whether at&#10;test time task identity is provided and--in case it is not--whether it must be&#10;inferred. Any sequence of well-defined tasks can be performed according to each&#10;scenario. Using the split and permuted MNIST task protocols, for each scenario&#10;we carry out an extensive comparison of recently proposed continual learning&#10;methods. We demonstrate substantial differences between the three scenarios in&#10;terms of difficulty and in terms of how efficient different methods are. In&#10;particular, when task identity must be inferred (i.e., class incremental&#10;learning), we find that regularization-based approaches (e.g., elastic weight&#10;consolidation) fail and that replaying representations of previous experiences&#10;seems required for solving this scenario.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.00487" label="2302.00487">
-        <attvalues>
-          <attvalue for="0" value="A Comprehensive Survey of Continual Learning: Theory, Method and&#10;  Application" />
-          <attvalue for="1" value="  To cope with real-world dynamics, an intelligent system needs to&#10;incrementally acquire, update, accumulate, and exploit knowledge throughout its&#10;lifetime. This ability, known as continual learning, provides a foundation for&#10;AI systems to develop themselves adaptively. In a general sense, continual&#10;learning is explicitly limited by catastrophic forgetting, where learning a new&#10;task usually results in a dramatic performance degradation of the old tasks.&#10;Beyond this, increasingly numerous advances have emerged in recent years that&#10;largely extend the understanding and application of continual learning. The&#10;growing and widespread interest in this direction demonstrates its realistic&#10;significance as well as complexity. In this work, we present a comprehensive&#10;survey of continual learning, seeking to bridge the basic settings, theoretical&#10;foundations, representative methods, and practical applications. Based on&#10;existing theoretical and empirical results, we summarize the general objectives&#10;of continual learning as ensuring a proper stability-plasticity trade-off and&#10;an adequate intra/inter-task generalizability in the context of resource&#10;efficiency. Then we provide a state-of-the-art and elaborated taxonomy,&#10;extensively analyzing how representative methods address continual learning,&#10;and how they are adapted to particular challenges in realistic applications.&#10;Through an in-depth discussion of promising directions, we believe that such a&#10;holistic perspective can greatly facilitate subsequent exploration in this&#10;field and beyond.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.06977" label="1708.06977">
-        <attvalues>
-          <attvalue for="0" value="Incremental Learning of Object Detectors without Catastrophic Forgetting" />
-          <attvalue for="1" value="  Despite their success for object detection, convolutional neural networks are&#10;ill-equipped for incremental learning, i.e., adapting the original model&#10;trained on a set of classes to additionally detect objects of new classes, in&#10;the absence of the initial training data. They suffer from &quot;catastrophic&#10;forgetting&quot; - an abrupt degradation of performance on the original set of&#10;classes, when the training objective is adapted to the new classes. We present&#10;a method to address this issue, and learn object detectors incrementally, when&#10;neither the original training data nor annotations for the original classes in&#10;the new training set are available. The core of our proposed solution is a loss&#10;function to balance the interplay between predictions on the new classes and a&#10;new distillation loss which minimizes the discrepancy between responses for old&#10;classes from the original and the updated networks. This incremental learning&#10;can be performed multiple times, for a new set of classes in each step, with a&#10;moderate drop in performance compared to the baseline network trained on the&#10;ensemble of data. We present object detection results on the PASCAL VOC 2007&#10;and COCO datasets, along with a detailed empirical analysis of the approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.00781" label="1904.00781">
-        <attvalues>
-          <attvalue for="0" value="RILOD: Near Real-Time Incremental Learning for Object Detection at the&#10;  Edge" />
-          <attvalue for="1" value="  Object detection models shipped with camera-equipped edge devices cannot&#10;cover the objects of interest for every user. Therefore, the incremental&#10;learning capability is a critical feature for a robust and personalized object&#10;detection system that many applications would rely on. In this paper, we&#10;present an efficient yet practical system, RILOD, to incrementally train an&#10;existing object detection model such that it can detect new object classes&#10;without losing its capability to detect old classes. The key component of RILOD&#10;is a novel incremental learning algorithm that trains end-to-end for one-stage&#10;deep object detection models only using training data of new object classes.&#10;Specifically to avoid catastrophic forgetting, the algorithm distills three&#10;types of knowledge from the old model to mimic the old model's behavior on&#10;object classification, bounding box regression and feature extraction. In&#10;addition, since the training data for the new classes may not be available, a&#10;real-time dataset construction pipeline is designed to collect training images&#10;on-the-fly and automatically label the images with both category and bounding&#10;box annotations. We have implemented RILOD under both edge-cloud and edge-only&#10;setups. Experiment results show that the proposed system can learn to detect a&#10;new object class in just a few minutes, including both dataset construction and&#10;model training. In comparison, traditional fine-tuning based method may take a&#10;few hours for training, and in most cases would also need a tedious and costly&#10;manual dataset labeling step.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.15439" label="2012.15439">
-        <attvalues>
-          <attvalue for="0" value="SID: Incremental Learning for Anchor-Free Object Detection via Selective&#10;  and Inter-Related Distillation" />
-          <attvalue for="1" value="  Incremental learning requires a model to continually learn new tasks from&#10;streaming data. However, traditional fine-tuning of a well-trained deep neural&#10;network on a new task will dramatically degrade performance on the old task --&#10;a problem known as catastrophic forgetting. In this paper, we address this&#10;issue in the context of anchor-free object detection, which is a new trend in&#10;computer vision as it is simple, fast, and flexible. Simply adapting current&#10;incremental learning strategies fails on these anchor-free detectors due to&#10;lack of consideration of their specific model structures. To deal with the&#10;challenges of incremental learning on anchor-free object detectors, we propose&#10;a novel incremental learning paradigm called Selective and Inter-related&#10;Distillation (SID). In addition, a novel evaluation metric is proposed to&#10;better assess the performance of detectors under incremental learning&#10;conditions. By selective distilling at the proper locations and further&#10;transferring additional instance relation knowledge, our method demonstrates&#10;significant advantages on the benchmark datasets PASCAL VOC and COCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.02136" label="2204.02136">
-        <attvalues>
-          <attvalue for="0" value="Overcoming Catastrophic Forgetting in Incremental Object Detection via&#10;  Elastic Response Distillation" />
-          <attvalue for="1" value="  Traditional object detectors are ill-equipped for incremental learning.&#10;However, fine-tuning directly on a well-trained detection model with only new&#10;data will lead to catastrophic forgetting. Knowledge distillation is a flexible&#10;way to mitigate catastrophic forgetting. In Incremental Object Detection (IOD),&#10;previous work mainly focuses on distilling for the combination of features and&#10;responses. However, they under-explore the information that contains in&#10;responses. In this paper, we propose a response-based incremental distillation&#10;method, dubbed Elastic Response Distillation (ERD), which focuses on&#10;elastically learning responses from the classification head and the regression&#10;head. Firstly, our method transfers category knowledge while equipping student&#10;detector with the ability to retain localization information during incremental&#10;learning. In addition, we further evaluate the quality of all locations and&#10;provide valuable responses by the Elastic Response Selection (ERS) strategy.&#10;Finally, we elucidate that the knowledge from different responses should be&#10;assigned with different importance during incremental distillation. Extensive&#10;experiments conducted on MS COCO demonstrate our method achieves&#10;state-of-the-art result, which substantially narrows the performance gap&#10;towards full training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.12427" label="2307.12427">
-        <attvalues>
-          <attvalue for="0" value="Augmented Box Replay: Overcoming Foreground Shift for Incremental Object&#10;  Detection" />
-          <attvalue for="1" value="  In incremental learning, replaying stored samples from previous tasks&#10;together with current task samples is one of the most efficient approaches to&#10;address catastrophic forgetting. However, unlike incremental classification,&#10;image replay has not been successfully applied to incremental object detection&#10;(IOD). In this paper, we identify the overlooked problem of foreground shift as&#10;the main reason for this. Foreground shift only occurs when replaying images of&#10;previous tasks and refers to the fact that their background might contain&#10;foreground objects of the current task. To overcome this problem, a novel and&#10;efficient Augmented Box Replay (ABR) method is developed that only stores and&#10;replays foreground objects and thereby circumvents the foreground shift&#10;problem. In addition, we propose an innovative Attentive RoI Distillation loss&#10;that uses spatial attention from region-of-interest (RoI) features to constrain&#10;current model to focus on the most important information from old model. ABR&#10;significantly reduces forgetting of previous classes while maintaining high&#10;plasticity in current classes. Moreover, it considerably reduces the storage&#10;requirements when compared to standard image replay. Comprehensive experiments&#10;on Pascal-VOC and COCO datasets support the state-of-the-art performance of our&#10;model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.09139" label="2312.09139">
-        <attvalues>
-          <attvalue for="0" value="Class-Wise Buffer Management for Incremental Object Detection: An&#10;  Effective Buffer Training Strategy" />
-          <attvalue for="1" value="  Class incremental learning aims to solve a problem that arises when&#10;continuously adding unseen class instances to an existing model This approach&#10;has been extensively studied in the context of image classification; however&#10;its applicability to object detection is not well established yet. Existing&#10;frameworks using replay methods mainly collect replay data without considering&#10;the model being trained and tend to rely on randomness or the number of labels&#10;of each sample. Also, despite the effectiveness of the replay, it was not yet&#10;optimized for the object detection task. In this paper, we introduce an&#10;effective buffer training strategy (eBTS) that creates the optimized replay&#10;buffer on object detection. Our approach incorporates guarantee minimum and&#10;hierarchical sampling to establish the buffer customized to the trained model.&#10;%These methods can facilitate effective retrieval of prior knowledge.&#10;Furthermore, we use the circular experience replay training to optimally&#10;utilize the accumulated buffer data. Experiments on the MS COCO dataset&#10;demonstrate that our eBTS achieves state-of-the-art performance compared to the&#10;existing replay schemes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.17323" label="2402.17323">
-        <attvalues>
-          <attvalue for="0" value="SDDGR: Stable Diffusion-based Deep Generative Replay for Class&#10;  Incremental Object Detection" />
-          <attvalue for="1" value="  In the field of class incremental learning (CIL), generative replay has&#10;become increasingly prominent as a method to mitigate the catastrophic&#10;forgetting, alongside the continuous improvements in generative models.&#10;However, its application in class incremental object detection (CIOD) has been&#10;significantly limited, primarily due to the complexities of scenes involving&#10;multiple labels. In this paper, we propose a novel approach called stable&#10;diffusion deep generative replay (SDDGR) for CIOD. Our method utilizes a&#10;diffusion-based generative model with pre-trained text-to-diffusion networks to&#10;generate realistic and diverse synthetic images. SDDGR incorporates an&#10;iterative refinement strategy to produce high-quality images encompassing old&#10;classes. Additionally, we adopt an L2 knowledge distillation technique to&#10;improve the retention of prior knowledge in synthetic images. Furthermore, our&#10;approach includes pseudo-labeling for old objects within new task images,&#10;preventing misclassification as background elements. Extensive experiments on&#10;the COCO 2017 dataset demonstrate that SDDGR significantly outperforms existing&#10;algorithms, achieving a new state-of-the-art in various CIOD scenarios. The&#10;source code will be made available to the public.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.08795" label="1912.08795">
-        <attvalues>
-          <attvalue for="0" value="Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion" />
-          <attvalue for="1" value="  We introduce DeepInversion, a new method for synthesizing images from the&#10;image distribution used to train a deep neural network. We 'invert' a trained&#10;network (teacher) to synthesize class-conditional input images starting from&#10;random noise, without using any additional information about the training&#10;dataset. Keeping the teacher fixed, our method optimizes the input while&#10;regularizing the distribution of intermediate feature maps using information&#10;stored in the batch normalization layers of the teacher. Further, we improve&#10;the diversity of synthesized images using Adaptive DeepInversion, which&#10;maximizes the Jensen-Shannon divergence between the teacher and student network&#10;logits. The resulting synthesized images from networks trained on the CIFAR-10&#10;and ImageNet datasets demonstrate high fidelity and degree of realism, and help&#10;enable a new breed of data-free applications - ones that do not require any&#10;real images or labeled data. We demonstrate the applicability of our proposed&#10;method to three tasks of immense practical importance -- (i) data-free network&#10;pruning, (ii) data-free knowledge transfer, and (iii) data-free continual&#10;learning. Code is available at https://github.com/NVlabs/DeepInversion&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.05862" label="2407.05862">
-        <attvalues>
-          <attvalue for="0" value="Bringing Masked Autoencoders Explicit Contrastive Properties for Point&#10;  Cloud Self-Supervised Learning" />
-          <attvalue for="1" value="  Contrastive learning (CL) for Vision Transformers (ViTs) in image domains has&#10;achieved performance comparable to CL for traditional convolutional backbones.&#10;However, in 3D point cloud pretraining with ViTs, masked autoencoder (MAE)&#10;modeling remains dominant. This raises the question: Can we take the best of&#10;both worlds? To answer this question, we first empirically validate that&#10;integrating MAE-based point cloud pre-training with the standard contrastive&#10;learning paradigm, even with meticulous design, can lead to a decrease in&#10;performance. To address this limitation, we reintroduce CL into the MAE-based&#10;point cloud pre-training paradigm by leveraging the inherent contrastive&#10;properties of MAE. Specifically, rather than relying on extensive data&#10;augmentation as commonly used in the image domain, we randomly mask the input&#10;tokens twice to generate contrastive input pairs. Subsequently, a&#10;weight-sharing encoder and two identically structured decoders are utilized to&#10;perform masked token reconstruction. Additionally, we propose that for an input&#10;token masked by both masks simultaneously, the reconstructed features should be&#10;as similar as possible. This naturally establishes an explicit contrastive&#10;constraint within the generative MAE-based pre-training paradigm, resulting in&#10;our proposed method, Point-CMAE. Consequently, Point-CMAE effectively enhances&#10;the representation quality and transfer performance compared to its MAE&#10;counterpart. Experimental evaluations across various downstream applications,&#10;including classification, part segmentation, and few-shot learning, demonstrate&#10;the efficacy of our framework in surpassing state-of-the-art techniques under&#10;standard ViTs and single-modal settings. The source code and trained models are&#10;available at: https://github.com/Amazingren/Point-CMAE.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Understanding 3D scenes is critical for diverse applications, from autonomous vehicles navigating urban environments to robotic manipulation tasks~\cite{an2024rethinking,cai2023spsd}. Point clouds, representing objects and surroundings precisely, offer an advantage due to their easy acquisition and accurate geometry capture~\cite{mei2022overlap}. &#10;However, annotating 3D point cloud data is even more resource-intensive compared to image data, as each point requires labeling.&#10;Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data. &#10;As a result, it is widely utilized in pre-training for 3D point clouds, demonstrating comparative or even better performance compared to its supervised counterpart in downstream tasks such as classification, segmentation, and detection~\cite{PointBERT, MaskPoint,qi2023contrast,ren2023masked,liu2020grouped}.&#10;&#10;Specifically, contrastive (single/cross-modal) and generative-based (reconstruct/predict) pre-training strategies are widely employed. CL is renowned for capturing global information to improve the model's discriminative ability, and this has been demonstrated to be effective in various domains such as 2D images and point cloud pre-training methods based on convolutional neural networks (CNNs)~\cite{mei2024unsupervised}. &#10;However, deploying single-modal CL to point clouds with ViT often results in inferior performance compared to MAE-based pre-training. On the other hand, using MAE with a Chamfer constraint for point cloud reconstruction can lead to sub-optimal solutions~\cite{MaskPoint}, as ground truth points are sampled from the original point clouds, causing inconsistencies when forcing predicted points to match them.&#10;In addition, the primary concept behind MAE typically emphasizes local relationships. &#10;These challenges collectively hinder the overall performance of ViTs-based point cloud pre-training. Consequently, a natural question arises: Can we leverage the strengths of both paradigms? In other words, can we harness the power of contrastive learning to learn consistent and abstract feature representations, while also utilizing MAE to enhance the model's ability to capture local data structures? &#10;To explore this question, we propose to enrich the MAE paradigm with contrastive learning ability, where the masked and the unmasked point cloud tokens are treated as two augmentations. &#10;&#10;Based on this setting, we deploy MoCo~\cite{he2020momentum}, and BYOL~\cite{grill2020bootstrap} style CL with Point-MAE~\cite{PointMAE} to point cloud pre-training with ViTs structure. &#10;The illustration of these two kinds of MAE-based CL is shown in Fig.~\ref{fig:compare_demo} (a) and Fig.~\ref{fig:compare_demo} (b), respectively. &#10;Our experimental results, as shown in Fig.~\ref{fig:contras_fig}, confirm that this approach frequently results in the occurrence of a performance drop.&#10;This is primarily because making CL effective for ViTs-based point cloud pre-training necessitates a well-designed framework and a tailored approach to conducting separate data augmentation for point cloud data. These requirements significantly increase the complexity of the entire problem, highlighting the non-trivial nature of even bringing contrastive learning to an MAE-based paradigm.&#10;&#10;To overcome the above limitations, we propose Point-CMAE, a simple yet effective method that explicitly integrates contrastive properties into the MAE-based point cloud pre-training paradigm using a ViTs architecture. The main idea of the proposed Point-CMAE is simplified in Fig.~\ref{fig:compare_demo} (c). Specifically, for a given point cloud token, we mask it twice randomly to construct the contrastive input pairs instead of applying heavy data augmentation. We then perform MAE for each of the masked input tokens using a weight-sharing encoder and two identically structured decoders. In this design, each decoder independently recovers the masked point cloud input token, which naturally forces the encoder to learn more common and representative features that satisfy the requirements of both decoders.&#10;Though this largely improves the semantic understanding of the encoder, the reconstruction constraint, the Chamfer distance loss, is usually used to minimize the distance between the predicted masked points and the ground truth points regarding the fact the ground truth points are also just one sampling of the original point cloud, which naturally leads to a sub-optimal problem. &#10;Since there are two different masked input tokens, there is an almost certain probability that some tokens will be simultaneously double-masked. As a remedy, we propose that for a token that is masked in both instances, the recovered feature should be as close as possible in both cases. This naturally introduces an explicit constraint at the feature level, ensuring that the encoder outputs more informative features for downstream tasks. As a result, Fig.~\ref{fig:contras_fig} shows that the proposed Point-CMAE can perform a decent convergence with obvious improvement at the beginning of the fine-tuning compared to the contrastive counterpart.&#10;&#10;In summary, our main contributions are recapped as follows:&#10;\begin{enumerate}[nosep]&#10; \item We experimentally validate that integrating contrastive learning into the MAE paradigm for point cloud pre-training with ViTs architecture is non-trivial and often leads to a severe performance drop.&#10;&#10;\item To address the aforementioned issue, we propose Point-CMAE, a method that enhances the MAE paradigm by integrating the advantages of contrastive learning through a straightforward yet effective design. This significantly enhances the informativeness and representativeness of the encoder.&#10;&#10;\item Extensive experimental results on various point cloud downstream tasks such as object classification, part segmentation, and few-shot classification demonstrate that our proposed Point-CMAE achieves new state-of-the-art performance under the standard ViTs setting for single-modal data.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Vision Transformers, Mathematics, Artificial Intelligence, Point Cloud Pretraining, Contrastive Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2403.00592" label="2403.00592">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Few-shot 3D Point Cloud Semantic Segmentation" />
-          <attvalue for="1" value="  This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS),&#10;with a focus on two significant issues in the state-of-the-art: foreground&#10;leakage and sparse point distribution. The former arises from non-uniform point&#10;sampling, allowing models to distinguish the density disparities between&#10;foreground and background for easier segmentation. The latter results from&#10;sampling only 2,048 points, limiting semantic information and deviating from&#10;the real-world practice. To address these issues, we introduce a standardized&#10;FS-PCS setting, upon which a new benchmark is built. Moreover, we propose a&#10;novel FS-PCS model. While previous methods are based on feature optimization by&#10;mainly refining support features to enhance prototypes, our method is based on&#10;correlation optimization, referred to as Correlation Optimization Segmentation&#10;(COSeg). Specifically, we compute Class-specific Multi-prototypical Correlation&#10;(CMC) for each query point, representing its correlations to category&#10;prototypes. Then, we propose the Hyper Correlation Augmentation (HCA) module to&#10;enhance CMC. Furthermore, tackling the inherent property of few-shot training&#10;to incur base susceptibility for models, we propose to learn non-parametric&#10;prototypes for the base classes during training. The learned base prototypes&#10;are used to calibrate correlations for the background class through a Base&#10;Prototypes Calibration (BPC) module. Experiments on popular datasets&#10;demonstrate the superiority of COSeg over existing methods. The code is&#10;available at: https://github.com/ZhaochongAn/COSeg&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.04805" label="1810.04805">
-        <attvalues>
-          <attvalue for="0" value="BERT: Pre-training of Deep Bidirectional Transformers for Language&#10;  Understanding" />
-          <attvalue for="1" value="  We introduce a new language representation model called BERT, which stands&#10;for Bidirectional Encoder Representations from Transformers. Unlike recent&#10;language representation models, BERT is designed to pre-train deep&#10;bidirectional representations from unlabeled text by jointly conditioning on&#10;both left and right context in all layers. As a result, the pre-trained BERT&#10;model can be fine-tuned with just one additional output layer to create&#10;state-of-the-art models for a wide range of tasks, such as question answering&#10;and language inference, without substantial task-specific architecture&#10;modifications.&#10;  BERT is conceptually simple and empirically powerful. It obtains new&#10;state-of-the-art results on eleven natural language processing tasks, including&#10;pushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI&#10;accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering&#10;Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1&#10;(5.1 point absolute improvement).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.05722" label="1911.05722">
-        <attvalues>
-          <attvalue for="0" value="Momentum Contrast for Unsupervised Visual Representation Learning" />
-          <attvalue for="1" value="  We present Momentum Contrast (MoCo) for unsupervised visual representation&#10;learning. From a perspective on contrastive learning as dictionary look-up, we&#10;build a dynamic dictionary with a queue and a moving-averaged encoder. This&#10;enables building a large and consistent dictionary on-the-fly that facilitates&#10;contrastive unsupervised learning. MoCo provides competitive results under the&#10;common linear protocol on ImageNet classification. More importantly, the&#10;representations learned by MoCo transfer well to downstream tasks. MoCo can&#10;outperform its supervised pre-training counterpart in 7 detection/segmentation&#10;tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large&#10;margins. This suggests that the gap between unsupervised and supervised&#10;representation learning has been largely closed in many vision tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.10566" label="2011.10566">
-        <attvalues>
-          <attvalue for="0" value="Exploring Simple Siamese Representation Learning" />
-          <attvalue for="1" value="  Siamese networks have become a common structure in various recent models for&#10;unsupervised visual representation learning. These models maximize the&#10;similarity between two augmentations of one image, subject to certain&#10;conditions for avoiding collapsing solutions. In this paper, we report&#10;surprising empirical results that simple Siamese networks can learn meaningful&#10;representations even using none of the following: (i) negative sample pairs,&#10;(ii) large batches, (iii) momentum encoders. Our experiments show that&#10;collapsing solutions do exist for the loss and structure, but a stop-gradient&#10;operation plays an essential role in preventing collapsing. We provide a&#10;hypothesis on the implication of stop-gradient, and further show&#10;proof-of-concept experiments verifying it. Our &quot;SimSiam&quot; method achieves&#10;competitive results on ImageNet and downstream tasks. We hope this simple&#10;baseline will motivate people to rethink the roles of Siamese architectures for&#10;unsupervised representation learning. Code will be made available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.02413" label="2112.02413">
-        <attvalues>
-          <attvalue for="0" value="PointCLIP: Point Cloud Understanding by CLIP" />
-          <attvalue for="1" value="  Recently, zero-shot and few-shot learning via Contrastive Vision-Language&#10;Pre-training (CLIP) have shown inspirational performance on 2D visual&#10;recognition, which learns to match images with their corresponding texts in&#10;open-vocabulary settings. However, it remains under explored that whether CLIP,&#10;pre-trained by large-scale image-text pairs in 2D, can be generalized to 3D&#10;recognition. In this paper, we identify such a setting is feasible by proposing&#10;PointCLIP, which conducts alignment between CLIP-encoded point cloud and 3D&#10;category texts. Specifically, we encode a point cloud by projecting it into&#10;multi-view depth maps without rendering, and aggregate the view-wise zero-shot&#10;prediction to achieve knowledge transfer from 2D to 3D. On top of that, we&#10;design an inter-view adapter to better extract the global feature and&#10;adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in&#10;2D. By just fine-tuning the lightweight adapter in the few-shot settings, the&#10;performance of PointCLIP could be largely improved. In addition, we observe the&#10;complementary property between PointCLIP and classical 3D-supervised networks.&#10;By simple ensembling, PointCLIP boosts baseline's performance and even&#10;surpasses state-of-the-art models. Therefore, PointCLIP is a promising&#10;alternative for effective 3D point cloud understanding via CLIP under low&#10;resource cost and data regime. We conduct thorough experiments on&#10;widely-adopted ModelNet10, ModelNet40 and the challenging ScanObjectNN to&#10;demonstrate the effectiveness of PointCLIP. The code is released at&#10;https://github.com/ZrrSkywalker/PointCLIP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.05918" label="2102.05918">
-        <attvalues>
-          <attvalue for="0" value="Scaling Up Visual and Vision-Language Representation Learning With Noisy&#10;  Text Supervision" />
-          <attvalue for="1" value="  Pre-trained representations are becoming crucial for many NLP and perception&#10;tasks. While representation learning in NLP has transitioned to training on raw&#10;text without human annotations, visual and vision-language representations&#10;still rely heavily on curated training datasets that are expensive or require&#10;expert knowledge. For vision applications, representations are mostly learned&#10;using datasets with explicit class labels such as ImageNet or OpenImages. For&#10;vision-language, popular datasets like Conceptual Captions, MSCOCO, or CLIP all&#10;involve a non-trivial data collection (and cleaning) process. This costly&#10;curation process limits the size of datasets and hence hinders the scaling of&#10;trained models. In this paper, we leverage a noisy dataset of over one billion&#10;image alt-text pairs, obtained without expensive filtering or post-processing&#10;steps in the Conceptual Captions dataset. A simple dual-encoder architecture&#10;learns to align visual and language representations of the image and text pairs&#10;using a contrastive loss. We show that the scale of our corpus can make up for&#10;its noise and leads to state-of-the-art representations even with such a simple&#10;learning scheme. Our visual representation achieves strong performance when&#10;transferred to classification tasks such as ImageNet and VTAB. The aligned&#10;visual and language representations enables zero-shot image classification and&#10;also set new state-of-the-art results on Flickr30K and MSCOCO image-text&#10;retrieval benchmarks, even when compared with more sophisticated&#10;cross-attention models. The representations also enable cross-modality search&#10;with complex text and text + image queries.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14819" label="2111.14819">
-        <attvalues>
-          <attvalue for="0" value="Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point&#10;  Modeling" />
-          <attvalue for="1" value="  We present Point-BERT, a new paradigm for learning Transformers to generalize&#10;the concept of BERT to 3D point cloud. Inspired by BERT, we devise a Masked&#10;Point Modeling (MPM) task to pre-train point cloud Transformers. Specifically,&#10;we first divide a point cloud into several local point patches, and a point&#10;cloud Tokenizer with a discrete Variational AutoEncoder (dVAE) is designed to&#10;generate discrete point tokens containing meaningful local information. Then,&#10;we randomly mask out some patches of input point clouds and feed them into the&#10;backbone Transformers. The pre-training objective is to recover the original&#10;point tokens at the masked locations under the supervision of point tokens&#10;obtained by the Tokenizer. Extensive experiments demonstrate that the proposed&#10;BERT-style pre-training strategy significantly improves the performance of&#10;standard point cloud Transformers. Equipped with our pre-training strategy, we&#10;show that a pure Transformer architecture attains 93.8% accuracy on ModelNet40&#10;and 83.1% accuracy on the hardest setting of ScanObjectNN, surpassing carefully&#10;designed point cloud models with much fewer hand-made designs. We also&#10;demonstrate that the representations learned by Point-BERT transfer well to new&#10;tasks and domains, where our models largely advance the state-of-the-art of&#10;few-shot point cloud classification task. The code and pre-trained models are&#10;available at https://github.com/lulutang0608/Point-BERT&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.02318" label="2302.02318">
-        <attvalues>
-          <attvalue for="0" value="Contrast with Reconstruct: Contrastive 3D Representation Learning Guided&#10;  by Generative Pretraining" />
-          <attvalue for="1" value="  Mainstream 3D representation learning approaches are built upon contrastive&#10;or generative modeling pretext tasks, where great improvements in performance&#10;on various downstream tasks have been achieved. However, we find these two&#10;paradigms have different characteristics: (i) contrastive models are&#10;data-hungry that suffer from a representation over-fitting issue; (ii)&#10;generative models have a data filling issue that shows inferior data scaling&#10;capacity compared to contrastive models. This motivates us to learn 3D&#10;representations by sharing the merits of both paradigms, which is non-trivial&#10;due to the pattern difference between the two paradigms. In this paper, we&#10;propose Contrast with Reconstruct (ReCon) that unifies these two paradigms.&#10;ReCon is trained to learn from both generative modeling teachers and&#10;single/cross-modal contrastive teachers through ensemble distillation, where&#10;the generative student guides the contrastive student. An encoder-decoder style&#10;ReCon-block is proposed that transfers knowledge through cross attention with&#10;stop-gradient, which avoids pretraining over-fitting and pattern difference&#10;issues. ReCon achieves a new state-of-the-art in 3D representation learning,&#10;e.g., 91.26% accuracy on ScanObjectNN. Codes have been released at&#10;https://github.com/qizekun/ReCon.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.12551" label="2205.12551">
-        <attvalues>
-          <attvalue for="0" value="Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision&#10;  Transformers" />
-          <attvalue for="1" value="  Position Embeddings (PEs), an arguably indispensable component in Vision&#10;Transformers (ViTs), have been shown to improve the performance of ViTs on many&#10;vision tasks. However, PEs have a potentially high risk of privacy leakage&#10;since the spatial information of the input patches is exposed. This caveat&#10;naturally raises a series of interesting questions about the impact of PEs on&#10;the accuracy, privacy, prediction consistency, etc. To tackle these issues, we&#10;propose a Masked Jigsaw Puzzle (MJP) position embedding method. In particular,&#10;MJP first shuffles the selected patches via our block-wise random jigsaw puzzle&#10;shuffle algorithm, and their corresponding PEs are occluded. Meanwhile, for the&#10;non-occluded patches, the PEs remain the original ones but their spatial&#10;relation is strengthened via our dense absolute localization regressor. The&#10;experimental results reveal that 1) PEs explicitly encode the 2D spatial&#10;relationship and lead to severe privacy leakage problems under gradient&#10;inversion attack; 2) Training ViTs with the naively shuffled patches can&#10;alleviate the problem, but it harms the accuracy; 3) Under a certain shuffle&#10;ratio, the proposed MJP not only boosts the performance and robustness on&#10;large-scale datasets (i.e., ImageNet-1K and ImageNet-C, -A/O) but also improves&#10;the privacy preservation ability under typical gradient attacks by a large&#10;margin. The source code and trained models are available&#10;at~\url{https://github.com/yhlleo/MJP}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.11183" label="2203.11183">
-        <attvalues>
-          <attvalue for="0" value="Masked Discrimination for Self-Supervised Learning on Point Clouds" />
-          <attvalue for="1" value="  Masked autoencoding has achieved great success for self-supervised learning&#10;in the image and language domains. However, mask based pretraining has yet to&#10;show benefits for point cloud understanding, likely due to standard backbones&#10;like PointNet being unable to properly handle the training versus testing&#10;distribution mismatch introduced by masking during training. In this paper, we&#10;bridge this gap by proposing a discriminative mask pretraining Transformer&#10;framework, MaskPoint}, for point clouds. Our key idea is to represent the point&#10;cloud as discrete occupancy values (1 if part of the point cloud; 0 if not),&#10;and perform simple binary classification between masked object points and&#10;sampled noise points as the proxy task. In this way, our approach is robust to&#10;the point sampling variance in point clouds, and facilitates learning rich&#10;representations. We evaluate our pretrained models across several downstream&#10;tasks, including 3D shape classification, segmentation, and real-word object&#10;detection, and demonstrate state-of-the-art results while achieving a&#10;significant pretraining speedup (e.g., 4.1x on ScanNet) compared to the prior&#10;state-of-the-art Transformer baseline. Code is available at&#10;https://github.com/haotian-liu/MaskPoint.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.06604" label="2203.06604">
-        <attvalues>
-          <attvalue for="0" value="Masked Autoencoders for Point Cloud Self-supervised Learning" />
-          <attvalue for="1" value="  As a promising scheme of self-supervised learning, masked autoencoding has&#10;significantly advanced natural language processing and computer vision.&#10;Inspired by this, we propose a neat scheme of masked autoencoders for point&#10;cloud self-supervised learning, addressing the challenges posed by point&#10;cloud's properties, including leakage of location information and uneven&#10;information density. Concretely, we divide the input point cloud into irregular&#10;point patches and randomly mask them at a high ratio. Then, a standard&#10;Transformer based autoencoder, with an asymmetric design and a shifting mask&#10;tokens operation, learns high-level latent features from unmasked point&#10;patches, aiming to reconstruct the masked point patches. Extensive experiments&#10;show that our approach is efficient during pre-training and generalizes well on&#10;various downstream tasks. Specifically, our pre-trained models achieve 85.18%&#10;accuracy on ScanObjectNN and 94.04% accuracy on ModelNet40, outperforming all&#10;the other self-supervised learning methods. We show with our scheme, a simple&#10;architecture entirely based on standard Transformers can surpass dedicated&#10;Transformer models from supervised learning. Our approach also advances&#10;state-of-the-art accuracies by 1.5%-2.3% in the few-shot object classification.&#10;Furthermore, our work inspires the feasibility of applying unified&#10;architectures from languages and images to the point cloud.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.01606" label="2403.01606">
-        <attvalues>
-          <attvalue for="0" value="A Unified Model Selection Technique for Spectral Clustering Based Motion&#10;  Segmentation" />
-          <attvalue for="1" value="  Motion segmentation is a fundamental problem in computer vision and is&#10;crucial in various applications such as robotics, autonomous driving and action&#10;recognition. Recently, spectral clustering based methods have shown impressive&#10;results on motion segmentation in dynamic environments. These methods perform&#10;spectral clustering on motion affinity matrices to cluster objects or point&#10;trajectories in the scene into different motion groups. However, existing&#10;methods often need the number of motions present in the scene to be known,&#10;which significantly reduces their practicality. In this paper, we propose a&#10;unified model selection technique to automatically infer the number of motion&#10;groups for spectral clustering based motion segmentation methods by combining&#10;different existing model selection techniques together. We evaluate our method&#10;on the KT3DMoSeg dataset and achieve competitve results comparing to the&#10;baseline where the number of clusters is given as ground truth information.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The objective of motion segmentation is to divide a video frame into regions segmented by common motions. Currently, motion segmentation is still a challenging problem when a moving camera is present, due to unknown camera motion. One popular technique to solve the motion segmentation problem in such scenario is to perform spectral clustering on motion affinity matrices constructed with motion models \cite{brox_object_2010, vidal_subspace_2011, li_perspective_2013, ochs_segmentation_2014, xu_motion_2018, jiang_what_2021, xi_multi-motion_2022, huang_motion_2023, lin_multi-motion_2023}. These methods typically take manually corrected point trajectories as input and build custom motion affinity matrices using one or more types of motion cues such as geometric models, spatio-temporal similarities or optical flow. Recently, spectral clustering based methods have shown remarkable results in segmenting motions in challenging dynamic environment containing significant motion degeneracy and complex scene structures \cite{xu_motion_2018, jiang_what_2021, xi_multi-motion_2022, lin_multi-motion_2023, huang_motion_2023}, largely thanks to its ability of synergetically fusing multiple types of motion cues together. However, all of these methods cannot automatically infer the number of motions present in the scene (i.e., model selection) and rely on user input for such information. \cite{brox_object_2010, vidal_subspace_2011, li_perspective_2013, ochs_segmentation_2014} do propose model selection techniques, but those techniques are specifically suited for their respective methods, which do not perform well in complex dynamic scenes. To address this issue, we propose a general unified model selection technique by combining the strengths of multiple existing criteria, to automate the model selection process for the current spectral clustering based motion segmentation methods relying on either single or multiple types of motion affinities.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Data Analysis, Mathematics, Spectral Clustering, Computer Science, Motion Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1804.02142" label="1804.02142">
-        <attvalues>
-          <attvalue for="0" value="Motion Segmentation by Exploiting Complementary Geometric Models" />
-          <attvalue for="1" value="  Many real-world sequences cannot be conveniently categorized as general or&#10;degenerate; in such cases, imposing a false dichotomy in using the fundamental&#10;matrix or homography model for motion segmentation would lead to difficulty.&#10;Even when we are confronted with a general scene-motion, the fundamental matrix&#10;approach as a model for motion segmentation still suffers from several defects,&#10;which we discuss in this paper. The full potential of the fundamental matrix&#10;approach could only be realized if we judiciously harness information from the&#10;simpler homography model. From these considerations, we propose a multi-view&#10;spectral clustering framework that synergistically combines multiple models&#10;together. We show that the performance can be substantially improved in this&#10;way. We perform extensive testing on existing motion segmentation datasets,&#10;achieving state-of-the-art performance on all of them; we also put forth a more&#10;realistic and challenging dataset adapted from the KITTI benchmark, containing&#10;real-world effects such as strong perspectives and strong forward translations&#10;not seen in the traditional datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.02361" label="2401.02361">
-        <attvalues>
-          <attvalue for="0" value="An Open and Comprehensive Pipeline for Unified Object Grounding and&#10;  Detection" />
-          <attvalue for="1" value="  Grounding-DINO is a state-of-the-art open-set detection model that tackles&#10;multiple vision tasks including Open-Vocabulary Detection (OVD), Phrase&#10;Grounding (PG), and Referring Expression Comprehension (REC). Its effectiveness&#10;has led to its widespread adoption as a mainstream architecture for various&#10;downstream applications. However, despite its significance, the original&#10;Grounding-DINO model lacks comprehensive public technical details due to the&#10;unavailability of its training code. To bridge this gap, we present&#10;MM-Grounding-DINO, an open-source, comprehensive, and user-friendly baseline,&#10;which is built with the MMDetection toolbox. It adopts abundant vision datasets&#10;for pre-training and various detection and grounding datasets for fine-tuning.&#10;We give a comprehensive analysis of each reported result and detailed settings&#10;for reproduction. The extensive experiments on the benchmarks mentioned&#10;demonstrate that our MM-Grounding-DINO-Tiny outperforms the Grounding-DINO-Tiny&#10;baseline. We release all our models to the research community. Codes and&#10;trained models are released at&#10;https://github.com/open-mmlab/mmdetection/tree/main/configs/mm_grounding_dino.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The task of object detection typically involves inputting an image into a model to obtain proposals, which are then matched with text through multi-modal alignment, making it a key component of most state-of-the-art multi-modal understanding architectures. Presently, object detection can be subdivided into three sub-tasks according to the type of input text: Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC). &#10;&#10;Following zero-shot settings, OVD models are trained on base categories but require predicting both base and novel categories within a large-scale language vocabulary~\cite{survey1}. Phrase grounding task takes not only a category but a phrase that describes all the candidate categories as input and output corresponding boxes~\cite{radford2021learning}. The primary aim of REC task is to accurately identify the target designated by a given textual description and subsequently demarcate its position utilizing a bounding box~\cite{he2023grec}.&#10;&#10;In recent years, numerous vision grounding and detection models have been explored to solve the tasks above. Among these grounding models, Grounding-DINO~\cite{liu2023grounding} has been made as a main-stream architecture with superior performance. Based on a closed-set detector DINO~\cite{zhang2022dino}, Grounding-DINO-Large achieves state-of-the-art zero-shot performance on COCO~\cite{lin2015microsoft}(mAP 52.5) without any COCO training data. Grounding-DINO executes the integration of vision and language modality at various stages, encompassing feature enhancer, query selection module, and decoder. This profound fusion approach significantly enhances the detection of objects in an open-set context and DETR-based structure makes it an end-to-end network without any hard-crafted module.&#10;&#10;Given that Grounding-DINO has demonstrated superior precision across the aforementioned three downstream tasks, yet is not entirely open-source (with only test and demo codes available), we rebuild the Grounding-DINO model utilizing the MMDetection toolbox~\cite{mmdetection} within the OpenMMLab project, adhering to the official test codes of Grounding-DINO. The structure of the model remains almost unchanged except for the modifications during initialization. Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO. Since the Cap4M dataset~\cite{radford2021learning} used by Grounding-DINO is not open-source, we have opted for the GRIT and V3Det datasets as substitutes in our study.&#10;&#10;We further extend all available benchmarks for OVD, PG and REC evaluation, including COCO, LVIS~\cite{gupta2019lvis}, RefCOCO/+/g, Flickr30k Entities, ODinW13/35~\cite{li2022elevater}, gRefCOCO~\cite{liu2023gres} and Description Detection Dataset($D^3$) ~\cite{xie2023described}. To our knowledge, we are the first to implement a framework that facilitates systematic evaluation across such an extensive array of datasets. All evaluation metrics are readily available in MMDetection. Pretrained with a large number of data, MM-Grounding-DINO-Tiny achieves zero-shot 50.6 mAP on COCO, 41.4 mAP on LVIS mini, and comprehensively surpasses Grounding-DINO-Tiny in REC task, detailed results are shown in Section \ref{sec:experiment}. We hope that our pipeline will serve as a valuable resource for further investigations in OVD, PG, and REC tasks.&#10;&#10;The contributions of our paper are as follows:&#10;&#10;\begin{enumerate}&#10; \item We propose MM-Grounding-DINO, a comprehensive and open-sourced grounding pipeline based on Grounding-DINO and pretrained with abundant vision datasets, which comprehensively address OVD, PG, and REC tasks.&#10; \item We take the lead in extending all available benchmarks for OVD, PG, and REC evaluation, including COCO, LVIS, RefCOCO/+/g, Flickr30K Entities, ODinW13/35, gRefCOCO and $D^3$. &#10; All evaluation metrics are readily available in MMDetection.&#10; \item We extensively evaluate the transfer ability of our models by fine-tuning our model through a multitude of external special datasets.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2306.15880" label="2306.15880">
-        <attvalues>
-          <attvalue for="0" value="Towards Open Vocabulary Learning: A Survey" />
-          <attvalue for="1" value="  In the field of visual scene understanding, deep neural networks have made&#10;impressive advancements in various core tasks like segmentation, tracking, and&#10;detection. However, most approaches operate on the close-set assumption,&#10;meaning that the model can only identify pre-defined categories that are&#10;present in the training set. Recently, open vocabulary settings were proposed&#10;due to the rapid progress of vision language pre-training. These new approaches&#10;seek to locate and recognize categories beyond the annotated label space. The&#10;open vocabulary approach is more general, practical, and effective compared to&#10;weakly supervised and zero-shot settings. This paper provides a thorough review&#10;of open vocabulary learning, summarizing and analyzing recent developments in&#10;the field. In particular, we begin by comparing it to related concepts such as&#10;zero-shot learning, open-set recognition, and out-of-distribution detection.&#10;Then, we review several closely related tasks in the case of segmentation and&#10;detection, including long-tail problems, few-shot, and zero-shot settings. For&#10;the method survey, we first present the basic knowledge of detection and&#10;segmentation in close-set as the preliminary knowledge. Next, we examine&#10;various scenarios in which open vocabulary learning is used, identifying common&#10;design elements and core ideas. Then, we compare the recent detection and&#10;segmentation approaches in commonly used datasets and benchmarks. Finally, we&#10;conclude with insights, issues, and discussions regarding future research&#10;directions. To our knowledge, this is the first comprehensive literature review&#10;of open vocabulary learning. We keep tracing related works at&#10;https://github.com/jianzongwu/Awesome-Open-Vocabulary.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.16182" label="2308.16182">
-        <attvalues>
-          <attvalue for="0" value="GREC: Generalized Referring Expression Comprehension" />
-          <attvalue for="1" value="  The objective of Classic Referring Expression Comprehension (REC) is to&#10;produce a bounding box corresponding to the object mentioned in a given textual&#10;description. Commonly, existing datasets and techniques in classic REC are&#10;tailored for expressions that pertain to a single target, meaning a sole&#10;expression is linked to one specific object. Expressions that refer to multiple&#10;targets or involve no specific target have not been taken into account. This&#10;constraint hinders the practical applicability of REC. This study introduces a&#10;new benchmark termed as Generalized Referring Expression Comprehension (GREC).&#10;This benchmark extends the classic REC by permitting expressions to describe&#10;any number of target objects. To achieve this goal, we have built the first&#10;large-scale GREC dataset named gRefCOCO. This dataset encompasses a range of&#10;expressions: those referring to multiple targets, expressions with no specific&#10;target, and the single-target expressions. The design of GREC and gRefCOCO&#10;ensures smooth compatibility with classic REC. The proposed gRefCOCO dataset, a&#10;GREC method implementation code, and GREC evaluation code are available at&#10;https://github.com/henghuiding/gRefCOCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.03605" label="2203.03605">
-        <attvalues>
-          <attvalue for="0" value="DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object&#10;  Detection" />
-          <attvalue for="1" value="  We present DINO (\textbf{D}ETR with \textbf{I}mproved de\textbf{N}oising&#10;anch\textbf{O}r boxes), a state-of-the-art end-to-end object detector. % in&#10;this paper. DINO improves over previous DETR-like models in performance and&#10;efficiency by using a contrastive way for denoising training, a mixed query&#10;selection method for anchor initialization, and a look forward twice scheme for&#10;box prediction. DINO achieves $49.4$AP in $12$ epochs and $51.3$AP in $24$&#10;epochs on COCO with a ResNet-50 backbone and multi-scale features, yielding a&#10;significant improvement of $\textbf{+6.0}$\textbf{AP} and&#10;$\textbf{+2.7}$\textbf{AP}, respectively, compared to DN-DETR, the previous&#10;best DETR-like model. DINO scales well in both model size and data size.&#10;Without bells and whistles, after pre-training on the Objects365 dataset with a&#10;SwinL backbone, DINO obtains the best results on both COCO \texttt{val2017}&#10;($\textbf{63.2}$\textbf{AP}) and \texttt{test-dev}&#10;(\textbf{$\textbf{63.3}$AP}). Compared to other models on the leaderboard, DINO&#10;significantly reduces its model size and pre-training data size while achieving&#10;better results. Our code will be available at&#10;\url{https://github.com/IDEACVR/DINO}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.14824" label="2306.14824">
-        <attvalues>
-          <attvalue for="0" value="Kosmos-2: Grounding Multimodal Large Language Models to the World" />
-          <attvalue for="1" value="  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new&#10;capabilities of perceiving object descriptions (e.g., bounding boxes) and&#10;grounding text to the visual world. Specifically, we represent refer&#10;expressions as links in Markdown, i.e., ``[text span](bounding boxes)'', where&#10;object descriptions are sequences of location tokens. Together with multimodal&#10;corpora, we construct large-scale data of grounded image-text pairs (called&#10;GrIT) to train the model. In addition to the existing capabilities of MLLMs&#10;(e.g., perceiving general modalities, following instructions, and performing&#10;in-context learning), Kosmos-2 integrates the grounding capability into&#10;downstream applications. We evaluate Kosmos-2 on a wide range of tasks,&#10;including (i) multimodal grounding, such as referring expression comprehension,&#10;and phrase grounding, (ii) multimodal referring, such as referring expression&#10;generation, (iii) perception-language tasks, and (iv) language understanding&#10;and generation. This work lays out the foundation for the development of&#10;Embodiment AI and sheds light on the big convergence of language, multimodal&#10;perception, action, and world modeling, which is a key step toward artificial&#10;general intelligence. Code and pretrained models are available at&#10;https://aka.ms/kosmos-2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.03752" label="2304.03752">
-        <attvalues>
-          <attvalue for="0" value="V3Det: Vast Vocabulary Visual Detection Dataset" />
-          <attvalue for="1" value="  Recent advances in detecting arbitrary objects in the real world are trained&#10;and evaluated on object detection datasets with a relatively restricted&#10;vocabulary. To facilitate the development of more general visual object&#10;detection, we propose V3Det, a vast vocabulary visual detection dataset with&#10;precisely annotated bounding boxes on massive images. V3Det has several&#10;appealing properties: 1) Vast Vocabulary: It contains bounding boxes of objects&#10;from 13,204 categories on real-world images, which is 10 times larger than the&#10;existing large vocabulary object detection dataset, e.g., LVIS. 2) Hierarchical&#10;Category Organization: The vast vocabulary of V3Det is organized by a&#10;hierarchical category tree which annotates the inclusion relationship among&#10;categories, encouraging the exploration of category relationships in vast and&#10;open vocabulary object detection. 3) Rich Annotations: V3Det comprises&#10;precisely annotated objects in 243k images and professional descriptions of&#10;each category written by human experts and a powerful chatbot. By offering a&#10;vast exploration space, V3Det enables extensive benchmarks on both vast and&#10;open vocabulary object detection, leading to new observations, practices, and&#10;insights for future research. It has the potential to serve as a cornerstone&#10;dataset for developing more general visual perception systems. V3Det is&#10;available at https://v3det.openxlab.org.cn/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.00272" label="1608.00272">
-        <attvalues>
-          <attvalue for="0" value="Modeling Context in Referring Expressions" />
-          <attvalue for="1" value="  Humans refer to objects in their environments all the time, especially in&#10;dialogue with other people. We explore generating and comprehending natural&#10;language referring expressions for objects in images. In particular, we focus&#10;on incorporating better measures of visual context into referring expression&#10;models and find that visual comparison to other objects within an image helps&#10;improve performance significantly. We also develop methods to tie the language&#10;generation process together, so that we generate expressions for all objects of&#10;a particular category jointly. Evaluation on three recent datasets - RefCOCO,&#10;RefCOCO+, and RefCOCOg, shows the advantages of our methods for both referring&#10;expression generation and comprehension.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.02283" label="1511.02283">
-        <attvalues>
-          <attvalue for="0" value="Generation and Comprehension of Unambiguous Object Descriptions" />
-          <attvalue for="1" value="  We propose a method that can generate an unambiguous description (known as a&#10;referring expression) of a specific object or region in an image, and which can&#10;also comprehend or interpret such an expression to infer which object is being&#10;described. We show that our method outperforms previous methods that generate&#10;descriptions of objects without taking into account other potentially ambiguous&#10;objects in the scene. Our model is inspired by recent successes of deep&#10;learning methods for image captioning, but while image captioning is difficult&#10;to evaluate, our task allows for easy objective evaluation. We also present a&#10;new large-scale dataset for referring expressions, based on MS-COCO. We have&#10;released the dataset and a toolbox for visualization and evaluation, see&#10;https://github.com/mjhucla/Google_Refexp_toolbox&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.09506" label="1902.09506">
-        <attvalues>
-          <attvalue for="0" value="GQA: A New Dataset for Real-World Visual Reasoning and Compositional&#10;  Question Answering" />
-          <attvalue for="1" value="  We introduce GQA, a new dataset for real-world visual reasoning and&#10;compositional question answering, seeking to address key shortcomings of&#10;previous VQA datasets. We have developed a strong and robust question engine&#10;that leverages scene graph structures to create 22M diverse reasoning&#10;questions, all come with functional programs that represent their semantics. We&#10;use the programs to gain tight control over the answer distribution and present&#10;a new tunable smoothing technique to mitigate question biases. Accompanying the&#10;dataset is a suite of new metrics that evaluate essential qualities such as&#10;consistency, grounding and plausibility. An extensive analysis is performed for&#10;baselines as well as state-of-the-art models, providing fine-grained results&#10;for different question types and topologies. Whereas a blind LSTM obtains mere&#10;42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%,&#10;offering ample opportunity for new research to explore. We strongly hope GQA&#10;will provide an enabling resource for the next generation of models with&#10;enhanced robustness, improved consistency, and deeper semantic understanding&#10;for images and language.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.04870" label="1505.04870">
-        <attvalues>
-          <attvalue for="0" value="Flickr30k Entities: Collecting Region-to-Phrase Correspondences for&#10;  Richer Image-to-Sentence Models" />
-          <attvalue for="1" value="  The Flickr30k dataset has become a standard benchmark for sentence-based&#10;image description. This paper presents Flickr30k Entities, which augments the&#10;158k captions from Flickr30k with 244k coreference chains, linking mentions of&#10;the same entities across different captions for the same image, and associating&#10;them with 276k manually annotated bounding boxes. Such annotations are&#10;essential for continued progress in automatic image description and grounded&#10;language understanding. They enable us to define a new benchmark for&#10;localization of textual entity mentions in an image. We present a strong&#10;baseline for this task that combines an image-text embedding, detectors for&#10;common objects, a color classifier, and a bias towards selecting larger&#10;objects. While our baseline rivals in accuracy more complex state-of-the-art&#10;models, we show that its gains cannot be easily parlayed into improvements on&#10;such tasks as image-sentence retrieval, thus underlining the limitations of&#10;current methods and the need for further research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.12763" label="2104.12763">
-        <attvalues>
-          <attvalue for="0" value="MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding" />
-          <attvalue for="1" value="  Multi-modal reasoning systems rely on a pre-trained object detector to&#10;extract regions of interest from the image. However, this crucial module is&#10;typically used as a black box, trained independently of the downstream task and&#10;on a fixed vocabulary of objects and attributes. This makes it challenging for&#10;such systems to capture the long tail of visual concepts expressed in free form&#10;text. In this paper we propose MDETR, an end-to-end modulated detector that&#10;detects objects in an image conditioned on a raw text query, like a caption or&#10;a question. We use a transformer-based architecture to reason jointly over text&#10;and image by fusing the two modalities at an early stage of the model. We&#10;pre-train the network on 1.3M text-image pairs, mined from pre-existing&#10;multi-modal datasets having explicit alignment between phrases in text and&#10;objects in the image. We then fine-tune on several downstream tasks such as&#10;phrase grounding, referring expression comprehension and segmentation,&#10;achieving state-of-the-art results on popular benchmarks. We also investigate&#10;the utility of our model as an object detector on a given label set when&#10;fine-tuned in a few-shot setting. We show that our pre-training approach&#10;provides a way to handle the long tail of object categories which have very few&#10;labelled instances. Our approach can be easily extended for visual question&#10;answering, achieving competitive performance on GQA and CLEVR. The code and&#10;models are available at https://github.com/ashkamath/mdetr.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.03195" label="1908.03195">
-        <attvalues>
-          <attvalue for="0" value="LVIS: A Dataset for Large Vocabulary Instance Segmentation" />
-          <attvalue for="1" value="  Progress on object detection is enabled by datasets that focus the research&#10;community's attention on open challenges. This process led us from simple&#10;images to complex scenes and from bounding boxes to segmentation masks. In this&#10;work, we introduce LVIS (pronounced `el-vis'): a new dataset for Large&#10;Vocabulary Instance Segmentation. We plan to collect ~2 million high-quality&#10;instance segmentation masks for over 1000 entry-level object categories in 164k&#10;images. Due to the Zipfian distribution of categories in natural images, LVIS&#10;naturally has a long tail of categories with few training samples. Given that&#10;state-of-the-art deep learning methods for object detection perform poorly in&#10;the low-sample regime, we believe that our dataset poses an important and&#10;exciting new scientific challenge. LVIS is available at&#10;http://www.lvisdataset.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08790" label="2204.08790">
-        <attvalues>
-          <attvalue for="0" value="ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented&#10;  Visual Models" />
-          <attvalue for="1" value="  Learning visual representations from natural language supervision has&#10;recently shown great promise in a number of pioneering works. In general, these&#10;language-augmented visual models demonstrate strong transferability to a&#10;variety of datasets and tasks. However, it remains challenging to evaluate the&#10;transferablity of these models due to the lack of easy-to-use evaluation&#10;toolkits and public benchmarks. To tackle this, we build ELEVATER (Evaluation&#10;of Language-augmented Visual Task-level Transfer), the first benchmark and&#10;toolkit for evaluating(pre-trained) language-augmented visual models. ELEVATER&#10;is composed of three components. (i) Datasets. As downstream evaluation suites,&#10;it consists of 20 image classification datasets and 35 object detection&#10;datasets, each of which is augmented with external knowledge. (ii) Toolkit. An&#10;automatic hyper-parameter tuning toolkit is developed to facilitate model&#10;evaluation on downstream tasks. (iii) Metrics. A variety of evaluation metrics&#10;are used to measure sample-efficiency (zero-shot and few-shot) and&#10;parameter-efficiency (linear probing and full model fine-tuning). ELEVATER is a&#10;platform for Computer Vision in the Wild (CVinW), and is publicly released at&#10;at https://computer-vision-in-the-wild.github.io/ELEVATER/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.00968" label="2306.00968">
-        <attvalues>
-          <attvalue for="0" value="GRES: Generalized Referring Expression Segmentation" />
-          <attvalue for="1" value="  Referring Expression Segmentation (RES) aims to generate a segmentation mask&#10;for the object described by a given language expression. Existing classic RES&#10;datasets and methods commonly support single-target expressions only, i.e., one&#10;expression refers to one target object. Multi-target and no-target expressions&#10;are not considered. This limits the usage of RES in practice. In this paper, we&#10;introduce a new benchmark called Generalized Referring Expression Segmentation&#10;(GRES), which extends the classic RES to allow expressions to refer to an&#10;arbitrary number of target objects. Towards this, we construct the first&#10;large-scale GRES dataset called gRefCOCO that contains multi-target, no-target,&#10;and single-target expressions. GRES and gRefCOCO are designed to be&#10;well-compatible with RES, facilitating extensive experiments to study the&#10;performance gap of the existing RES methods on the GRES task. In the&#10;experimental study, we find that one of the big challenges of GRES is complex&#10;relationship modeling. Based on this, we propose a region-based GRES baseline&#10;ReLA that adaptively divides the image into regions with sub-instance clues,&#10;and explicitly models the region-region and region-language dependencies. The&#10;proposed approach ReLA achieves new state-of-the-art performance on the both&#10;newly proposed GRES and classic RES tasks. The proposed gRefCOCO dataset and&#10;method are available at https://henghuiding.github.io/GRES.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.12813" label="2307.12813">
-        <attvalues>
-          <attvalue for="0" value="Described Object Detection: Liberating Object Detection with Flexible&#10;  Expressions" />
-          <attvalue for="1" value="  Detecting objects based on language information is a popular task that&#10;includes Open-Vocabulary object Detection (OVD) and Referring Expression&#10;Comprehension (REC). In this paper, we advance them to a more practical setting&#10;called Described Object Detection (DOD) by expanding category names to flexible&#10;language expressions for OVD and overcoming the limitation of REC only&#10;grounding the pre-existing object. We establish the research foundation for DOD&#10;by constructing a Description Detection Dataset ($D^3$). This dataset features&#10;flexible language expressions, whether short category names or long&#10;descriptions, and annotating all described objects on all images without&#10;omission. By evaluating previous SOTA methods on $D^3$, we find some&#10;troublemakers that fail current REC, OVD, and bi-functional methods. REC&#10;methods struggle with confidence scores, rejecting negative instances, and&#10;multi-target scenarios, while OVD methods face constraints with long and&#10;complex descriptions. Recent bi-functional methods also do not work well on DOD&#10;due to their separated training procedures and inference strategies for REC and&#10;OVD tasks. Building upon the aforementioned findings, we propose a baseline&#10;that largely improves REC methods by reconstructing the training data and&#10;introducing a binary classification sub-task, outperforming existing methods.&#10;Data and code are available at https://github.com/shikras/d-cube and related&#10;works are tracked in&#10;https://github.com/Charles-Xie/awesome-described-object-detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.01307" label="2102.01307">
-        <attvalues>
-          <attvalue for="0" value="Human-Machine Collaborative Video Coding Through Cuboidal Partitioning" />
-          <attvalue for="1" value="  Video coding algorithms encode and decode an entire video frame while feature&#10;coding techniques only preserve and communicate the most critical information&#10;needed for a given application. This is because video coding targets human&#10;perception, while feature coding aims for machine vision tasks. Recently,&#10;attempts are being made to bridge the gap between these two domains. In this&#10;work, we propose a video coding framework by leveraging on to the commonality&#10;that exists between human vision and machine vision applications using cuboids.&#10;This is because cuboids, estimated rectangular regions over a video frame, are&#10;computationally efficient, has a compact representation and object centric.&#10;Such properties are already shown to add value to traditional video coding&#10;systems. Herein cuboidal feature descriptors are extracted from the current&#10;frame and then employed for accomplishing a machine vision task in the form of&#10;object detection. Experimental results show that a trained classifier yields&#10;superior average precision when equipped with cuboidal features oriented&#10;representation of the current test frame. Additionally, this representation&#10;costs $7\%$ less in bit rate if the captured frames are need be communicated to&#10;a receiver.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Many video analysis systems employ a client-server architecture where video signals are captured at the front-end devices and the analyses are carried out at cloud-end servers. In such systems, video data need be communicated from the front end to the server. Prior to this transmission, captured video signal is encoded using a video coding paradigm and at the receiving end the obtained signal is decoded before performing any analysis task. An alternative approach is to extract features from the video data at the client end and then communicate the encoded features rather than the video signal itself. Since capturing of the signal and feature extraction are conducted at the front end, the server becomes less clumsy pertaining to large scale applications and can dedicate most of its resources for the analysis work. However, as the estimated features are devised for specific tasks only, they can be difficult to generalize to a broad spectrum of machine vision tasks in the cloud end.&#10;&#10;Traditional video coding standards like H.264/AVC~\cite{AVC} and HEVC~\cite{HEVC} are block-based. They are also pixel and frame centric. In order to model the commonality that exists within a video sequence, the frame that need be coded, known as the current frame, is artificially partitioned into square or rectangular shape blocks. These blocks are formed by grouping together neighboring pixels. After this partitioning, for each current frame block, commonality modeling is employed to form a prediction of it either using the already coded neighboring blocks belonging to the current frame (intra-prediction) or using motion estimation and compensation within a co-located neighborhood region in the set of already coded reference frame(s) (inter-prediction), by minimizing a rate-distortion criterion.&#10;&#10;Due to the requirements for communicating feature descriptors, coding standards like compact descriptors for visual search (CDVS)~\cite{CDVS} and compact descriptors for video analysis (CDVA)~\cite{CDVA} were developed. In CDVS, to represent the visual characteristics of images, local and global descriptors are designed. Deep learning features are employed in CDVA to further augment the video analysis performance. Although these features showed excellent performance for machine vision tasks, it is not possible to reconstruct full resolution videos for human vision from such features. This results into two successive stages of analysis and compression for machine and human vision~\cite{VCM}.&#10;&#10;As modern video coding standards like HEVC focus on human vision by reconstructing full resolution frames from the coded bitstream and standards like CDVS and CDVA focus on machine vision tasks and incapable of reconstructing full resolution pictures from the coded feature descriptors; there is a need for human-machine collaborative coding that can leverage on the advances made on the frontiers of video coding and feature coding as well as bridge the gap between these two technologies. For example, in autonomous driving use case, some decisions can be made by the machine itself and for some other decisions humans would interact with machine. Such collaborative coding problem is known as video coding for machines (VCM)~\cite{VCM}. In this direction of work, it was proposed in~\cite{VCM-image-coding} to extract features in the form of edge maps from a given image which is used to perform machine vision tasks. After that a generative model was employed to perform image reconstruction based on the edge maps and additional reference pixels.&#10;&#10;Although the objectives of human vision and machine vision are different, human-machine collaborative coding can benefit from exploring the significant overlap that exists between these two domains. One important similarity is that human vision is mostly hierarchical. For example, an image of a study room can be described by humans using annotations like - over the floor carpet (global context) there is a desk (local detail) and a laptop (finer detail) is on the desk; on the wall (global context) a clock (local detail) is mounted, etc. Machine vision tasks like object detection can also be modeled in this kind of hierarchical way. For instance, in~\cite{hierarchical} a method is proposed for performing hierarchical object detection in images through deep reinforcement learning~\cite{mnih2015}. Using scale-space theory~\cite{Scale-space} and deep reinforcement learning, another work~\cite{CT} explored optimal search strategies for locating anatomical structures, based on image information at multiple scales.&#10;&#10;Murshed et al. proposed the hierarchical cuboidal partitioning of image data (CuPID) algorithm in~\cite{cuboid-dicta1}. In CuPID framework, an entire image is initially partitioned into two rectangular shaped regions, known as cuboids, by finding a hyperplane orthogonal to one of the axes. A greedy optimization heuristic, equipped with sum of the information entropy in the split-pair cuboids as the objective function, is employed to find the minimizing hyperplane. This ensures that the obtained split-pair cuboids are maximum dissimilar in terms of image pixel intensity distribution. Next, each cuboid is recursively partitioned by solving the same greedy optimization problem. It was shown that cuboids have excellent global commonality modeling capabilities, are object centric and computationally efficient~\cite{cuboid-dicta2,cuboid-csvt,ashek-icassp,ashek-icassp2021}.&#10;&#10;The aforementioned properties of cuboids were leveraged on to traditional video coding in the work of~\cite{ashek-mmsp,ashek-icip2021}; wherein a reference frame is generated using cuboidal approximation of the target frame and incorporating this additional reference frame into a modified HEVC encoder outperformed the rate-distortion performance of a standalone HEVC encoder. The obtained gain in delta PSNR and savings in bit rate can be attributed to the fact that HEVC employs a partitioning scheme that begins at a rigid fixed size of $64\times64$ pixels level and during this process does not take into account the structural properties of the scene need be coded. This sub-optimal partitioning was improved by incorporating cuboids, which are estimated considering the entire frame's pixel intensity distribution and therefore more object-centric.&#10;&#10;Building on the work~\cite{ashek-mmsp}, in this paper we investigate the applicability of cuboidal features in video coding targeting machines. Based on the CuPID algorithm, (i) features are extracted from a given video frame for the machine vision task of object detection. These features costs less bits to encode. Addition to this, (ii) it is possible to reconstruct a full-resolution frame from the obtained feature descriptors alone; unlike the approach in~\cite{VCM-image-coding} that require additional reference pixels along with edge based feature descriptors in this regard. The reconstructed frame is a coarse representation of the original frame and is capable of preserving important structural properties of the original frame. Finally, (iii) the object detection performance of the cuboidal descriptors set are put to test across varying number of cuboids.&#10;&#10;The rest of the paper is organized as follows: in sectionII, we briefly describe the cuboidal feature extraction process from a given video frame and the reconstruction process of a full-resolution frame from those estimated feature descriptors. Section III, describes the performance of the cuboidal features over an object detection task. Finally, in sectionIV, we present our conclusions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Vision Tasks, Video Coding Algorithms, Artificial Intelligence, Cognitive Science, Computer Science, Feature Coding Techniques, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1704.08141" label="1704.08141">
-        <attvalues>
-          <attvalue for="0" value="Compact Descriptors for Video Analysis: the Emerging MPEG Standard" />
-          <attvalue for="1" value="  This paper provides an overview of the on-going compact descriptors for video&#10;analysis standard (CDVA) from the ISO/IEC moving pictures experts group (MPEG).&#10;MPEG-CDVA targets at defining a standardized bitstream syntax to enable&#10;interoperability in the context of video analysis applications. During the&#10;developments of MPEGCDVA, a series of techniques aiming to reduce the&#10;descriptor size and improve the video representation ability have been&#10;proposed. This article describes the new standard that is being developed and&#10;reports the performance of these key technical contributions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.03569" label="2001.03569">
-        <attvalues>
-          <attvalue for="0" value="Video Coding for Machines: A Paradigm of Collaborative Compression and&#10;  Intelligent Analytics" />
-          <attvalue for="1" value="  Video coding, which targets to compress and reconstruct the whole frame, and&#10;feature compression, which only preserves and transmits the most critical&#10;information, stand at two ends of the scale. That is, one is with compactness&#10;and efficiency to serve for machine vision, and the other is with full&#10;fidelity, bowing to human perception. The recent endeavors in imminent trends&#10;of video compression, e.g. deep learning based coding tools and end-to-end&#10;image/video coding, and MPEG-7 compact feature descriptor standards, i.e.&#10;Compact Descriptors for Visual Search and Compact Descriptors for Video&#10;Analysis, promote the sustainable and fast development in their own directions,&#10;respectively. In this paper, thanks to booming AI technology, e.g. prediction&#10;and generation models, we carry out exploration in the new area, Video Coding&#10;for Machines (VCM), arising from the emerging MPEG standardization efforts1.&#10;Towards collaborative compression and intelligent analytics, VCM attempts to&#10;bridge the gap between feature coding for machine vision and video coding for&#10;human vision. Aligning with the rising Analyze then Compress instance Digital&#10;Retina, the definition, formulation, and paradigm of VCM are given first.&#10;Meanwhile, we systematically review state-of-the-art techniques in video&#10;compression and feature compression from the unique perspective of MPEG&#10;standardization, which provides the academic and industrial evidence to realize&#10;the collaborative compression of video and feature streams in a broad range of&#10;AI applications. Finally, we come up with potential VCM solutions, and the&#10;preliminary results have demonstrated the performance and efficiency gains.&#10;Further direction is discussed as well.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.02915" label="2001.02915">
-        <attvalues>
-          <attvalue for="0" value="Towards Coding for Human and Machine Vision: A Scalable Image Coding&#10;  Approach" />
-          <attvalue for="1" value="  The past decades have witnessed the rapid development of image and video&#10;coding techniques in the era of big data. However, the signal fidelity-driven&#10;coding pipeline design limits the capability of the existing image/video coding&#10;frameworks to fulfill the needs of both machine and human vision. In this&#10;paper, we come up with a novel image coding framework by leveraging both the&#10;compressive and the generative models, to support machine vision and human&#10;perception tasks jointly. Given an input image, the feature analysis is first&#10;applied, and then the generative model is employed to perform image&#10;reconstruction with features and additional reference pixels, in which compact&#10;edge maps are extracted in this work to connect both kinds of vision in a&#10;scalable way. The compact edge map serves as the basic layer for machine vision&#10;tasks, and the reference pixels act as a sort of enhanced layer to guarantee&#10;signal fidelity for human vision. By introducing advanced generative models, we&#10;train a flexible network to reconstruct images from compact feature&#10;representations and the reference pixels. Experimental results demonstrate the&#10;superiority of our framework in both human visual quality and facial landmark&#10;detection, which provide useful evidence on the emerging standardization&#10;efforts on MPEG VCM (Video Coding for Machine).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.03718" label="1611.03718">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Object Detection with Deep Reinforcement Learning" />
-          <attvalue for="1" value="  We present a method for performing hierarchical object detection in images&#10;guided by a deep reinforcement learning agent. The key idea is to focus on&#10;those parts of the image that contain richer information and zoom on them. We&#10;train an intelligent agent that, given an image window, is capable of deciding&#10;where to focus the attention among five different predefined region candidates&#10;(smaller windows). This procedure is iterated providing a hierarchical image&#10;analysis.We compare two different candidate proposal strategies to guide the&#10;object search: with and without overlap. Moreover, our work compares two&#10;different strategies to extract features from a convolutional neural network&#10;for each region proposal: a first one that computes new feature maps for each&#10;region proposal, and a second one that computes the feature maps for the whole&#10;image to later generate crops for each region proposal. Experiments indicate&#10;better results for the overlapping candidate proposal strategy and a loss of&#10;performance for the cropped image features due to the loss of spatial&#10;resolution. We argue that, while this loss seems unavoidable when working with&#10;large amounts of object candidates, the much more reduced amount of region&#10;proposals generated by our reinforcement learning agent allows considering to&#10;extract features for each location without sharing convolutional computation&#10;among regions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.02298" label="1801.02298">
-        <attvalues>
-          <attvalue for="0" value="Depth Sequence Coding with Hierarchical Partitioning and Spatial-domain&#10;  Quantisation" />
-          <attvalue for="1" value="  Depth coding in 3D-HEVC for the multiview video plus depth (MVD) architecture&#10;(i) deforms object shapes due to block-level edge-approximation; (ii) misses an&#10;opportunity for high compressibility at near-lossless quality by failing to&#10;exploit strong homogeneity (clustering tendency) in depth syntax, motion vector&#10;components, and residuals at frame-level; and (iii) restricts interactivity and&#10;limits responsiveness of independent use of depth information for &quot;non-viewing&quot;&#10;applications due to texture-depth coding dependency. This paper presents a&#10;standalone depth sequence coder, which operates in the lossless to&#10;near-lossless quality range while compressing depth data superior to lossy&#10;3D-HEVC. It preserves edges implicitly by limiting quantisation to the&#10;spatial-domain and exploits clustering tendency efficiently at frame-level with&#10;a novel binary tree based decomposition (BTBD) technique. For mono-view coding&#10;of standard MVD test sequences, on average, (i) lossless BTBD achieved $\times&#10;42.2$ compression-ratio and $-60.0\%$ coding gain against the pseudo-lossless&#10;3D-HEVC, using the lowest quantisation parameter $QP = 1$, and (ii)&#10;near-lossless BTBD achieved $-79.4\%$ and $6.98$ dB Bj{\o}ntegaard delta&#10;bitrate (BD-BR) and distortion (BD-PSNR), respectively, against 3D-HEVC. In&#10;view-synthesis applications, decoded depth maps from BTBD rendered superior&#10;quality synthetic-views, compared to 3D-HEVC, with $-18.9\%$ depth BD-BR and&#10;$0.43$ dB synthetic-texture BD-PSNR on average.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.15736" label="2006.15736">
-        <attvalues>
-          <attvalue for="0" value="Roweisposes, Including Eigenposes, Supervised Eigenposes, and&#10;  Fisherposes, for 3D Action Recognition" />
-          <attvalue for="1" value="  Human action recognition is one of the important fields of computer vision&#10;and machine learning. Although various methods have been proposed for 3D action&#10;recognition, some of which are basic and some use deep learning, the need of&#10;basic methods based on generalized eigenvalue problem is sensed for action&#10;recognition. This need is especially sensed because of having similar basic&#10;methods in the field of face recognition such as eigenfaces and Fisherfaces. In&#10;this paper, we propose Roweisposes which uses Roweis discriminant analysis for&#10;generalized subspace learning. This method includes Fisherposes, eigenposes,&#10;supervised eigenposes, and double supervised eigenposes as its special cases.&#10;Roweisposes is a family of infinite number of action recongition methods which&#10;learn a discriminative subspace for embedding the body poses. Experiments on&#10;the TST, UTKinect, and UCFKinect datasets verify the effectiveness of the&#10;proposed method for action recognition.&#10;" />
-          <attvalue for="2" value="&#10;&#10;3D Action recognition uses the information of 3D coordinates of joints \cite{presti20163d}. The methods of action recognition in the literature can be divided into two main categories \cite{khezeli2019time}. The first category deals with the time series of joints and extracts features, such as Dynamic&#10;Time Warping (DTW), from these time series \cite{ghodsi2018simultaneous}.&#10;The second category classifies every frame into some predefined body poses and uses probabilistic approach to recognize the sequence as an action \cite{ghojogh2018fisherposes}.&#10;&#10;Some of the action recognition methods are based on convolutional networks \cite{ji20123d,wang2018action}. &#10;Some other methods, however, have a manifold-based or subspace-based approaches. In these approaches, the actions lie on subspaces of data \cite{hong2019variant}. Riemannian manifolds \cite{devanne20143} and Grassmann manifolds \cite{slama2015accurate} have been extensively used for action recognition. &#10;Some of the methods embed actions in sparse manifolds \cite{mohammadzade2020sparsness} to make use of betting on sparsity principle \cite{hastie2009elements}. &#10;&#10;Some of the action recognition methods use basic subspace learning methods for recognizing poses or actions. For example, histograms of 3D joint locations (HOJ3D) utilizes Linear Discriminant Analysis (LDA) which is equivalent to Fisher Discriminant Analysis (FDA) \cite{ghojogh2019linear}. LDA is also used in \cite{mokari2020recognizing} for recognizing involuntary actions. &#10;Fisherposes is a method which uses FDA \cite{hastie2009elements,fisher1936use} for embedding the body poses and their recognition \cite{ghojogh2018fisherposes}.&#10;The reader can refer to \cite{ghojogh2018fisherposes} and the references therein to see papers in action recognition which use LDA or FDA. FDA is very useful for pose embedding and action recognition because it maximizes and minimizes the inter-class and intra-class scatters of body poses, respectively \cite{ghojogh2019fisher}. &#10;&#10;There exist many subspace learning methods which are based on generalized eigenvalue problem \cite{ghojogh2019eigenvalue}.&#10;Some examples of these methods are Principal Component Analysis (PCA) \cite{jolliffe2011principal,ghojogh2019unsupervised}, Supervised PCA (SPCA) \cite{barshan2011supervised}, and FDA \cite{ghojogh2019fisher}. The subspace of PCA maximizes the scatter of projected data while SPCA has a subspace which maximizes the statistical dependence of projected data and the class labels. &#10;Roweis Discriminant Analysis (RDA) \cite{ghojogh2019roweis,ghojogh2020roweis} is a family of infinite number of subspace learning methods based on generalized eigenvalue problem. It generalizes PCA, SPCA, and FDA and also includes Double Supervised Discriminant Analysis (DSDA) \cite{ghojogh2020roweis}. &#10;&#10;Basic subspace learning methods, such as PCA, SPCA, and FDA, can be used for 3D action recognition for embedding poses and classifying actions. There have appeared different face recognition methods using basic subspace learning approaches \cite{abate20072d}. Some examples are eigenfaces \cite{turk1991face}, Fisherfaces \cite{belhumeur1997eigenfaces}, supervised eigenfaces \cite{barshan2011supervised,ghojogh2019unsupervised}, and double supervised eigenfaces \cite{ghojogh2020roweis}. &#10;Similar need to methods based on basic subspace learning is sensed for action recognition literature. &#10;&#10;In this paper, we propose Roweisposes for action recognition. This method is based on basic subspace learning approaches which make use of generalized eigenvalue problem \cite{ghojogh2019eigenvalue}. This method, which uses RDA \cite{ghojogh2020roweis}, generalizes the Fisherposes method \cite{ghojogh2018fisherposes}. Some of the special cases of Roweisposes are based on PCA, SPCA, and DSDA and we name them eigenposes, supervised eigenposes, and double supervised eigenposes, respectively. &#10;The Roweisposes method includes infinite number of subspace learning methods for embedding the body poses useful for action recognition. &#10;&#10;The remainder of this paper is organized as follows. &#10;Section \ref{section_Fisherposes} reviews FDA and how it can be restated using the total scatter. Moreover, the Fisherposes method is reviewed in this section. Section \ref{section_RDA} reviews the theory of PCA and SPCA as well as RDA and the Roweis map. The Roweisposes method is proposed in Section \ref{section_Roweisposes} where pre-processing, training the Roweisposes subspace, pose recognition, windowing, training hidden Markov model, and action recognition are detailed. The experiments are reported in Section \ref{section_experiments} to show the effectiveness of the proposed method. Finally, Section \ref{section_conclusion} concludes the paper and enumerates the future work. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Action Recognition, Computer Science, Pattern Recognition" />
-        </attvalues>
-      </node>
-      <node id="1711.01589" label="1711.01589">
-        <attvalues>
-          <attvalue for="0" value="Simultaneous Joint and Object Trajectory Templates for Human Activity&#10;  Recognition from 3-D Data" />
-          <attvalue for="1" value="  The availability of low-cost range sensors and the development of relatively&#10;robust algorithms for the extraction of skeleton joint locations have inspired&#10;many researchers to develop human activity recognition methods using the 3-D&#10;data. In this paper, an effective method for the recognition of human&#10;activities from the normalized joint trajectories is proposed. We represent the&#10;actions as multidimensional signals and introduce a novel method for generating&#10;action templates by averaging the samples in a &quot;dynamic time&quot; sense. Then in&#10;order to deal with the variations in the speed and style of performing actions,&#10;we warp the samples to the action templates by an efficient algorithm and&#10;employ wavelet filters to extract meaningful spatiotemporal features. The&#10;proposed method is also capable of modeling the human-object interactions, by&#10;performing the template generation and temporal warping procedure via the joint&#10;and object trajectories simultaneously. The experimental evaluation on several&#10;challenging datasets demonstrates the effectiveness of our method compared to&#10;the state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.09401" label="1612.09401">
-        <attvalues>
-          <attvalue for="0" value="Action Recognition Based on Joint Trajectory Maps with Convolutional&#10;  Neural Networks" />
-          <attvalue for="1" value="  Convolutional Neural Networks (ConvNets) have recently shown promising&#10;performance in many computer vision tasks, especially image-based recognition.&#10;How to effectively apply ConvNets to sequence-based data is still an open&#10;problem. This paper proposes an effective yet simple method to represent&#10;spatio-temporal information carried in $3D$ skeleton sequences into three $2D$&#10;images by encoding the joint trajectories and their dynamics into color&#10;distribution in the images, referred to as Joint Trajectory Maps (JTM), and&#10;adopts ConvNets to learn the discriminative features for human action&#10;recognition. Such an image-based representation enables us to fine-tune&#10;existing ConvNets models for the classification of skeleton sequences without&#10;training the networks afresh. The three JTMs are generated in three orthogonal&#10;planes and provide complimentary information to each other. The final&#10;recognition is further improved through multiply score fusion of the three&#10;JTMs. The proposed method was evaluated on four public benchmark datasets, the&#10;large NTU RGB+D Dataset, MSRC-12 Kinect Gesture Dataset (MSRC-12), G3D Dataset&#10;and UTD Multimodal Human Action Dataset (UTD-MHAD) and achieved the&#10;state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.02590" label="1906.02590">
-        <attvalues>
-          <attvalue for="0" value="Linear and Quadratic Discriminant Analysis: Tutorial" />
-          <attvalue for="1" value="  This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic&#10;Discriminant Analysis (QDA) as two fundamental classification methods in&#10;statistical and probabilistic learning. We start with the optimization of&#10;decision boundary on which the posteriors are equal. Then, LDA and QDA are&#10;derived for binary and multiple classes. The estimation of parameters in LDA&#10;and QDA are also covered. Then, we explain how LDA and QDA are related to&#10;metric learning, kernel principal component analysis, Mahalanobis distance,&#10;logistic regression, Bayes optimal classifier, Gaussian naive Bayes, and&#10;likelihood ratio test. We also prove that LDA and Fisher discriminant analysis&#10;are equivalent. We finally clarify some of the theoretical concepts with&#10;simulations we provide.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.06227" label="1708.06227">
-        <attvalues>
-          <attvalue for="0" value="Recognizing Involuntary Actions from 3D Skeleton Data Using Body States" />
-          <attvalue for="1" value="  Human action recognition has been one of the most active fields of research&#10;in computer vision for last years. Two dimensional action recognition methods&#10;are facing serious challenges such as occlusion and missing the third dimension&#10;of data. Development of depth sensors has made it feasible to track positions&#10;of human body joints over time. This paper proposes a novel method of action&#10;recognition which uses temporal 3D skeletal Kinect data. This method introduces&#10;the definition of body states and then every action is modeled as a sequence of&#10;these states. The learning stage uses Fisher Linear Discriminant Analysis (LDA)&#10;to construct discriminant feature space for discriminating the body states.&#10;Moreover, this paper suggests the use of the Mahalonobis distance as an&#10;appropriate distance metric for the classification of the states of involuntary&#10;actions. Hidden Markov Model (HMM) is then used to model the temporal&#10;transition between the body states in each action. According to the results,&#10;this method significantly outperforms other popular methods, with recognition&#10;rate of 88.64% for eight different actions and up to 96.18% for classifying&#10;fall actions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.09436" label="1906.09436">
-        <attvalues>
-          <attvalue for="0" value="Fisher and Kernel Fisher Discriminant Analysis: Tutorial" />
-          <attvalue for="1" value="  This is a detailed tutorial paper which explains the Fisher discriminant&#10;Analysis (FDA) and kernel FDA. We start with projection and reconstruction.&#10;Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two-&#10;and then multi-classes are explained in FDA. Then, we discuss on the rank of&#10;the scatters and the dimensionality of the subspace. A real-life example is&#10;also provided for interpreting FDA. Then, possible singularity of the scatter&#10;is discussed to introduce robust FDA. PCA and FDA directions are also compared.&#10;We also prove that FDA and linear discriminant analysis are equivalent. Fisher&#10;forest is also introduced as an ensemble of fisher subspaces useful for&#10;handling data with different features and dimensionality. Afterwards, kernel&#10;FDA is explained for both one- and multi-dimensional subspaces with both two-&#10;and multi-classes. Finally, some simulations are performed on AT&amp;T face dataset&#10;to illustrate FDA and compare it with PCA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.11240" label="1903.11240">
-        <attvalues>
-          <attvalue for="0" value="Eigenvalue and Generalized Eigenvalue Problems: Tutorial" />
-          <attvalue for="1" value="  This paper is a tutorial for eigenvalue and generalized eigenvalue problems.&#10;We first introduce eigenvalue problem, eigen-decomposition (spectral&#10;decomposition), and generalized eigenvalue problem. Then, we mention the&#10;optimization problems which yield to the eigenvalue and generalized eigenvalue&#10;problems. We also provide examples from machine learning, including principal&#10;component analysis, kernel supervised principal component analysis, and Fisher&#10;discriminant analysis, which result in eigenvalue and generalized eigenvalue&#10;problems. Finally, we introduce the solutions to both eigenvalue and&#10;generalized eigenvalue problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.03148" label="1906.03148">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised and Supervised Principal Component Analysis: Tutorial" />
-          <attvalue for="1" value="  This is a detailed tutorial paper which explains the Principal Component&#10;Analysis (PCA), Supervised PCA (SPCA), kernel PCA, and kernel SPCA. We start&#10;with projection, PCA with eigen-decomposition, PCA with one and multiple&#10;projection directions, properties of the projection matrix, reconstruction&#10;error minimization, and we connect to autoencoder. Then, PCA with singular&#10;value decomposition, dual PCA, and kernel PCA are covered. SPCA using both&#10;scoring and Hilbert-Schmidt independence criterion are explained. Kernel SPCA&#10;using both direct and dual approaches are then introduced. We cover all cases&#10;of projection and reconstruction of training and out-of-sample data. Finally,&#10;some simulations are provided on Frey and AT&amp;T face datasets for verifying the&#10;theory in practice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.05437" label="1910.05437">
-        <attvalues>
-          <attvalue for="0" value="Roweis Discriminant Analysis: A Generalized Subspace Learning Method" />
-          <attvalue for="1" value="  We present a new method which generalizes subspace learning based on&#10;eigenvalue and generalized eigenvalue problems. This method, Roweis&#10;Discriminant Analysis (RDA), is named after Sam Roweis to whom the field of&#10;subspace learning owes significantly. RDA is a family of infinite number of&#10;algorithms where Principal Component Analysis (PCA), Supervised PCA (SPCA), and&#10;Fisher Discriminant Analysis (FDA) are special cases. One of the extreme&#10;special cases, which we name Double Supervised Discriminant Analysis (DSDA),&#10;uses the labels twice; it is novel and has not appeared elsewhere. We propose a&#10;dual for RDA for some special cases. We also propose kernel RDA, generalizing&#10;kernel PCA, kernel SPCA, and kernel FDA, using both dual RDA and representation&#10;theory. Our theoretical analysis explains previously known facts such as why&#10;SPCA can use regression but FDA cannot, why PCA and SPCA have duals but FDA&#10;does not, why kernel PCA and kernel SPCA use kernel trick but kernel FDA does&#10;not, and why PCA is the best linear method for reconstruction. Roweisfaces and&#10;kernel Roweisfaces are also proposed generalizing eigenfaces, Fisherfaces,&#10;supervised eigenfaces, and their kernel variants. We also report experiments&#10;showing the effectiveness of RDA and kernel RDA on some benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.06756" label="2212.06756">
-        <attvalues>
-          <attvalue for="0" value="Connectivity-constrained Interactive Panoptic Segmentation" />
-          <attvalue for="1" value="  We address interactive panoptic annotation, where one segment all object and&#10;stuff regions in an image. We investigate two graph-based segmentation&#10;algorithms that both enforce connectivity of each region, with a notable&#10;class-aware Integer Linear Programming (ILP) formulation that ensures global&#10;optimum. Both algorithms can take RGB, or utilize the feature maps from any&#10;DCNN, whether trained on the target dataset or not, as input. We then propose&#10;an interactive, scribble-based annotation framework.&#10;" />
-          <attvalue for="2" value="&#10;D{eep} Convolutional Neural Networks (DCNNs) excel at a wide range of image recognition tasks~\cite{7780459, 7485869, Shelhamer2017}, such as semantic segmentation~\cite{7913730, Shelhamer2017, zhao2017pspnet} and panoptic segmentation~\cite{2018arXiv180100868K,2019arXiv190205093Y, 2019arXiv190103784X, 2019arXiv190102446K}. Semantic segmentation studies the tasks of assigning a class label to each pixel of an image, while instance segmentation~\cite{Chen2018MaskLabIS} detects and segment each object instance. Panoptic segmentation unifies both tasks that investigate to segment both things (such as person, cars) and {stuff} (such as road, sky) classes.&#10;&#10;While DCNNs show outstanding results for semantic and panoptic segmentation, they have two conceptional problems. First, they require huge amounts of annotated data. Annotating image segmentation masks is a very time consuming and labor extensive task. For example, annotating a semantic image mask took “more than 1.5h on average” on the Cityscapes dataset~\cite{Cordts2016Cityscapes}. &#10;In addition, DCNNs rely on their implicitly learned generalization probability and most of the state-of-the-art architectures do not make use of any domain specific knowledge, such as neighborhood relations and connectivity priors for segments. On the contrary, classical graph based segmentation models~\cite{969114, Yedidia:2003} do not require any learning data and can incorporate specific domain knowledge. Their major drawback is that they rely on human-designed similarity features and require complex optimization algorithms or solvers, which are mainly CPU based and non-suitable for real time applications.&#10;&#10;In this work, we explore the combination of DCNNs and graph-based algorithms for annotating ground truth segmentation. We investigate two interactive algorithms that both enforce the connectivity prior (to be more precise in Sec.~\ref{prerequisite}).&#10;Specifically, we design a heuristic region growing method based on the Potts model~\cite{potts_1952}, as well as an integer linear programming (ILP) formulation of the markov random field (MRF)~\cite{698673}. We utilize scribble based annotations from human annotators as initialized or iterative hard constraints for our algorithms, which is typical in a human-in-the-loop (HITL) annotation process.&#10;We explore two different scenarios. In the first scenario, we assume that for the segmentation task, there is already a pre-trained DCNN with the same class mapping. Thus, we could make full use of DCNN's probability map and scribbles as input to our algorithms.&#10;In the second scenario, we assume that no pre-trained DCNN for the same objective is available. This is true for a lot of existing datasets, e.g., Cityscapes does not contain any class labels for lane marking, which is crucial information for an autonomous vehicle. In this case, we cannot use the class specific probability map, but the more generic low level features of a DCNN can be utilized as feature description for the algorithm. &#10;&#10;To investigate the general purpose of low level DCNN features, we compare DCNN trained on ImageNet~\cite{ILSVRC15} and COCO~\cite{10.100748} with and without fine tuning on the target dataset. Our experiments show that incorporating the connectivity prior as well as the DCNN features greatly improves the algorithm performance. We present competitive semantic (and panoptic) segmentation results on the PASCAL VOC 2012~\cite{Everingham15} and Cityscapes dataset. To the best of our knowledge, our method is the first ``non-DCNN'' panoptic segmentation algorithm on Cityscapes with competing results, which shows the potential improvement gained by a combined approach. See Fig.~\ref{front_image} for a visualization that uses initialized scribbles as input.&#10;&#10;We also prototype our algorithms for the task of interactive annotation. Based on initial scribbles and a baseline DCNN's probability map, our algorithms acheive $90.9\%$ mIoU on VOC validation set, with just $3$ rounds ($1$ scribbles per image each round) of correction scribbles.&#10;&#10;Summarized, our key contributions are&#10;\begin{itemize}&#10;&#09;\item a fast and good-performing heuristic algorithm and a ``global'' ILP formulation with connectivity prior, &#10;&#09;\item an in depth analysis of a combination of DCNN and graph-based segmentation algorithms,&#10;&#09;\item extensive evaluation of the scribble based interactive algorithms for semantic and panoptic segmentations on two challenging datasets.&#10;\end{itemize}&#10;&#10;Our proposed algorithms have multiple use cases in annotating datasets for segmentation. First, &#10;they can be used inside any HITL annotation tool, as the annotator interacts with the image in forms of scribbles until satisfaction. Second, they can be used inside any weakly or semi-supervised learning framework for semantic (and panoptic) segmentation, which only requires initial scribbles~\cite{Lin2016ScribbleSupSC}. Third, semi-supervised learning in forms of scribbles can be combined with active learning framework~\cite{8215750}, to better utilize the huge amount of unlabeled data.&#10;&#10;Related Work.&#10;&#10;The procedure of annotating per pixel segmentation masks is similar to interactive image segmentation, which is widely studied in the past decade. The method using bounding boxes~\cite{Rother:2004} is suitable for instance segmentation, which requires the user to draw the box as tight as possible. Recently, $4$ extreme points~\cite{Man+18, extremeclick} clicking is used as an alternative which shows superior result. &#10;On the other hand, polygon based methods~\cite{Russell2008} require users to carefully click the extreme points of things and stuff, and the accuracy heavily depends on the number of clicks. Among others, scribbles are recognized as a more user-friendly way~\cite{937505, 4359322}. Moreover, it is also natural to annotate stuff classes using scribbles. &#10;&#10;Modern annotation tools often adopt interactive deep learning based methods, including Polygon-RNN++~\cite{acuna2018efficient} and Curve-GCN~\cite{2019arXiv190306874L}, which allows the annotator to click on the boundaries. Notably, a recent approach~\cite{agustsson2019interactive} enables both $4$ extreme points clicking and scribbles correction. Besides, they can also take advantage of ensemble learning~\cite{Zhou2009}, to combine several inference results to produce a better segmentation. However, these methods all require an existing ground truth dataset for DCNNs to learn on the first hand, which may not be available when unknown domains or new classes are introduced.&#10;&#10;As a cheaper alternative, weakly supervised learning has drawn a lot of attention recently. ~\cite{Dai2015BoxSupEB, khoreva_CVPR17, Li_2018_ECCV} claim that weakly iteratively trained by just bounding boxes and image tags, the DCNN can achieve $95\%$ segmentation score compared to fully supervised on VOC. Since this method emphasizes on thing classes, it has worse score (or even none) on stuff classes. Instead, ~\cite{Lin2016ScribbleSupSC, 20.500} claim that iteratively training a DCNN by scribble annotations suffers only a small degradation in performance on both thing and stuff classes.&#10;&#10;For graph-based methods, the (discrete) Potts model~\cite{potts_1952} is widely used for denoising and segmentation. The authors of~\cite{DBLP-1803-07351} formulate the problem as an ILP and try to solve the global optimum, but only to a reduced image size. The authors of~\cite{7410389} proposed an efficient region-fusion-based heuristic algorithm, while~\cite{rshen_ilp2017} extends the work to allow scribbles as input.&#10;The MAP-MRF (maximizing a posterior in an MRF) has been well studied for image segmentation. Previous methods focus on local priors \cite{Boykov2006, NIPS2011_4296}, and efficient approximate algorithms exist, e.g., graph cut~\cite{969114} and belief propagation~\cite{Yedidia:2003}. MRF with connectivity prior has been studied in~\cite{Rempfler2016TheMC, 5828, 4587440} and formulated as an ILP, which can be solved by any ILP solver~\cite{doi:10.1080}. ILP with root or scribble nodes connectivity was studied in~\cite{rshen_ilp2017, Rempfler2016TheMC, rshen2018}, where for each class at least one node is fixed.&#10;Since solving an ILP exactly is in general $\mathcal{NP}$-hard~\cite{Land60anautomatic}, they either solve a relaxation of the ILP, or focus on the binary MRF case.&#10;&#10;Our interactive (in forms of scribble) multi-label annotation algorithms are graph based, with global connectivity prior, i.e., they enforce pixels of the same label to be connected, which allows the annotator to better control over the global segmentation.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Graph-Based Algorithms, Machine Learning, Engineering, Computer Vision, Interactive Annotation, Mathematics, Artificial Intelligence, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1902.05093" label="1902.05093">
-        <attvalues>
-          <attvalue for="0" value="DeeperLab: Single-Shot Image Parser" />
-          <attvalue for="1" value="  We present a single-shot, bottom-up approach for whole image parsing. Whole&#10;image parsing, also known as Panoptic Segmentation, generalizes the tasks of&#10;semantic segmentation for 'stuff' classes and instance segmentation for 'thing'&#10;classes, assigning both semantic and instance labels to every pixel in an&#10;image. Recent approaches to whole image parsing typically employ separate&#10;standalone modules for the constituent semantic and instance segmentation tasks&#10;and require multiple passes of inference. Instead, the proposed DeeperLab image&#10;parser performs whole image parsing with a significantly simpler, fully&#10;convolutional approach that jointly addresses the semantic and instance&#10;segmentation tasks in a single-shot manner, resulting in a streamlined system&#10;that better lends itself to fast processing. For quantitative evaluation, we&#10;use both the instance-based Panoptic Quality (PQ) metric and the proposed&#10;region-based Parsing Covering (PC) metric, which better captures the image&#10;parsing quality on 'stuff' classes and larger object instances. We report&#10;experimental results on the challenging Mapillary Vistas dataset, in which our&#10;single model achieves 31.95% (val) / 31.6% PQ (test) and 55.26% PC (val) with 3&#10;frames per second (fps) on GPU or near real-time speed (22.6 fps on GPU) with&#10;reduced accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.04837" label="1712.04837">
-        <attvalues>
-          <attvalue for="0" value="MaskLab: Instance Segmentation by Refining Object Detection with&#10;  Semantic and Direction Features" />
-          <attvalue for="1" value="  In this work, we tackle the problem of instance segmentation, the task of&#10;simultaneously solving object detection and semantic segmentation. Towards this&#10;goal, we present a model, called MaskLab, which produces three outputs: box&#10;detection, semantic segmentation, and direction prediction. Building on top of&#10;the Faster-RCNN object detector, the predicted boxes provide accurate&#10;localization of object instances. Within each region of interest, MaskLab&#10;performs foreground/background segmentation by combining semantic and direction&#10;prediction. Semantic segmentation assists the model in distinguishing between&#10;objects of different semantic classes including background, while the direction&#10;prediction, estimating each pixel's direction towards its corresponding center,&#10;allows separating instances of the same semantic class. Moreover, we explore&#10;the effect of incorporating recent successful methods from both segmentation&#10;and detection (i.e. atrous convolution and hypercolumn). Our proposed model is&#10;evaluated on the COCO instance segmentation benchmark and shows comparable&#10;performance with other state-of-art models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.09081" label="1711.09081">
-        <attvalues>
-          <attvalue for="0" value="Deep Extreme Cut: From Extreme Points to Object Segmentation" />
-          <attvalue for="1" value="  This paper explores the use of extreme points in an object (left-most,&#10;right-most, top, bottom pixels) as input to obtain precise object segmentation&#10;for images and videos. We do so by adding an extra channel to the image in the&#10;input of a convolutional neural network (CNN), which contains a Gaussian&#10;centered in each of the extreme points. The CNN learns to transform this&#10;information into a segmentation of an object that matches those extreme points.&#10;We demonstrate the usefulness of this approach for guided segmentation&#10;(grabcut-style), interactive segmentation, video object segmentation, and dense&#10;segmentation annotation. We show that we obtain the most precise results to&#10;date, also with less user input, in an extensive and varied selection of&#10;benchmarks and datasets. All our models and code are publicly available on&#10;http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.09693" label="1803.09693">
-        <attvalues>
-          <attvalue for="0" value="Efficient Interactive Annotation of Segmentation Datasets with&#10;  Polygon-RNN++" />
-          <attvalue for="1" value="  Manually labeling datasets with object masks is extremely time consuming. In&#10;this work, we follow the idea of Polygon-RNN to produce polygonal annotations&#10;of objects interactively using humans-in-the-loop. We introduce several&#10;important improvements to the model: 1) we design a new CNN encoder&#10;architecture, 2) show how to effectively train the model with Reinforcement&#10;Learning, and 3) significantly increase the output resolution using a Graph&#10;Neural Network, allowing the model to accurately annotate high-resolution&#10;objects in images. Extensive evaluation on the Cityscapes dataset shows that&#10;our model, which we refer to as Polygon-RNN++, significantly outperforms the&#10;original model in both automatic (10% absolute and 16% relative improvement in&#10;mean IoU) and interactive modes (requiring 50% fewer clicks by annotators). We&#10;further analyze the cross-domain scenario in which our model is trained on one&#10;dataset, and used out of the box on datasets from varying domains. The results&#10;show that Polygon-RNN++ exhibits powerful generalization capabilities,&#10;achieving significant improvements over existing pixel-wise methods. Using&#10;simple online fine-tuning we further achieve a high reduction in annotation&#10;time for new datasets, moving a step closer towards an interactive annotation&#10;tool to be used in practice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06874" label="1903.06874">
-        <attvalues>
-          <attvalue for="0" value="Fast Interactive Object Annotation with Curve-GCN" />
-          <attvalue for="1" value="  Manually labeling objects by tracing their boundaries is a laborious process.&#10;In Polygon-RNN++ the authors proposed Polygon-RNN that produces polygonal&#10;annotations in a recurrent manner using a CNN-RNN architecture, allowing&#10;interactive correction via humans-in-the-loop. We propose a new framework that&#10;alleviates the sequential nature of Polygon-RNN, by predicting all vertices&#10;simultaneously using a Graph Convolutional Network (GCN). Our model is trained&#10;end-to-end. It supports object annotation by either polygons or splines,&#10;facilitating labeling efficiency for both line-based and curved objects. We&#10;show that Curve-GCN outperforms all existing approaches in automatic mode,&#10;including the powerful PSP-DeepLab and is significantly more efficient in&#10;interactive mode than Polygon-RNN++. Our model runs at 29.3ms in automatic, and&#10;2.6ms in interactive mode, making it 10x and 100x faster than Polygon-RNN++.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.01888" label="1812.01888">
-        <attvalues>
-          <attvalue for="0" value="Interactive Full Image Segmentation by Considering All Regions Jointly" />
-          <attvalue for="1" value="  We address interactive full image annotation, where the goal is to accurately&#10;segment all object and stuff regions in an image. We propose an interactive,&#10;scribble-based annotation framework which operates on the whole image to&#10;produce segmentations for all regions. This enables sharing scribble&#10;corrections across regions, and allows the annotator to focus on the largest&#10;errors made by the machine across the whole image. To realize this, we adapt&#10;Mask-RCNN into a fast interactive segmentation framework and introduce an&#10;instance-aware loss measured at the pixel-level in the full image canvas, which&#10;lets predictions for nearby regions properly compete for space. Finally, we&#10;compare to interactive single object segmentation on the COCO panoptic dataset.&#10;We demonstrate that our interactive full image segmentation approach leads to a&#10;5% IoU gain, reaching 90% IoU at a budget of four extreme clicks and four&#10;corrective scribbles per region.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07351" label="1803.07351">
-        <attvalues>
-          <attvalue for="0" value="Discrete Potts Model for Generating Superpixels on Noisy Images" />
-          <attvalue for="1" value="  Many computer vision applications, such as object recognition and&#10;segmentation, increasingly build on superpixels. However, there have been so&#10;far few superpixel algorithms that systematically deal with noisy images. We&#10;propose to first decompose the image into equal-sized rectangular patches,&#10;which also sets the maximum superpixel size. Within each patch, a Potts model&#10;for simultaneous segmentation and denoising is applied, that guarantees&#10;connected and non-overlapping superpixels and also produces a denoised image.&#10;The corresponding optimization problem is formulated as a mixed integer linear&#10;program (MILP), and solved by a commercial solver. Extensive experiments on the&#10;BSDS500 dataset images with noises are compared with other state-of-the-art&#10;superpixel methods. Our method achieves the best result in terms of a combined&#10;score (OP) composed of the under-segmentation error, boundary recall and&#10;compactness.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.06135" label="1606.06135">
-        <attvalues>
-          <attvalue for="0" value="The Minimum Cost Connected Subgraph Problem in Medical Image Analysis" />
-          <attvalue for="1" value="  Several important tasks in medical image analysis can be stated in the form&#10;of an optimization problem whose feasible solutions are connected subgraphs.&#10;Examples include the reconstruction of neural or vascular structures under&#10;connectedness constraints. We discuss the minimum cost connected subgraph&#10;(MCCS) problem and its approximations from the perspective of medical&#10;applications. We propose a) objective-dependent constraints and b) novel&#10;constraint generation schemes to solve this optimization problem exactly by&#10;means of a branch-and-cut algorithm. These are shown to improve scalability and&#10;allow us to solve instances of two medical benchmark datasets to optimality for&#10;the first time. This enables us to perform a quantitative comparison between&#10;exact and approximative algorithms, where we identify the geodesic tree&#10;algorithm as an excellent alternative to exact inference on the examined&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.01187" label="2307.01187">
-        <attvalues>
-          <attvalue for="0" value="SAMAug: Point Prompt Augmentation for Segment Anything Model" />
-          <attvalue for="1" value="  This paper introduces SAMAug, a novel visual point augmentation method for&#10;the Segment Anything Model (SAM) that enhances interactive image segmentation&#10;performance. SAMAug generates augmented point prompts to provide more&#10;information about the user's intention to SAM. Starting with an initial point&#10;prompt, SAM produces an initial mask, which is then fed into our proposed&#10;SAMAug to generate augmented point prompts. By incorporating these extra&#10;points, SAM can generate augmented segmentation masks based on both the&#10;augmented point prompts and the initial prompt, resulting in improved&#10;segmentation performance. We conducted evaluations using four different point&#10;augmentation strategies: random sampling, sampling based on maximum difference&#10;entropy, maximum distance, and saliency. Experiment results on the COCO,&#10;Fundus, COVID QUEx, and ISIC2018 datasets show that SAMAug can boost SAM's&#10;segmentation results, especially using the maximum distance and saliency.&#10;SAMAug demonstrates the potential of visual prompt augmentation for computer&#10;vision. Codes of SAMAug are available at github.com/yhydhx/SAMAug&#10;" />
-          <attvalue for="2" value=" &#10;Recent advancement in large pre-trained models \cite{liu2023summary,zhao2023brain} inspired a series of developments of foundational models in computer vision \cite{willemink2022toward,zhou2023comprehensive}. Among these, the Segment Anything Model (SAM) \cite{kirillov2023segment} stands out as a novel interactive model specifically designed for image segmentation tasks and subsequent downstream applications.&#10;&#10;SAM represents the paradigm shift in artificial intelligence towards large-scale, general-purposed &quot;foundation&quot; models \cite{yuan2023power,zhang2023biomedgpt}. The SAM model demonstrates the feasibility of performing multi-task image segmentation in a zero-shot learning approach. It also allows the use of &quot;prompts&quot; in the form of user-provided points, bounding boxes, or dense masks to perform the segmentation via an in-context-learning scheme. However, simple prompts such as a single point will lead to ambiguities in the SAM model, which is also discussed in the original paper of SAM \cite{kirillov2023segment}. While the multi-mask prediction and confidence score-based result selection scheme adopted by SAM can solve some of the ambiguities, we observed that it is also possible to generate additional point prompts automatically to improve the performance of SAM. In this work, we propose SAMAug, a novel visual point augmentation method for generating additional segmentation masks using SAM. SAMAug performs prompt (point or bounding-box) augmentation based on the initial segmentation results from SAM using a single point or box prompt, then incorporates them into the prompt-based segmentation by SAM. We evaluate four sampling approaches for point prompts: random, maximum difference entropy, maximum distance, and saliency-based. We also investigated the inner- and outer-bounding box augmentation for box prompts. &#10;&#10;With the experiments on the COCO, Fundus, COVIDQU-Ex, and ISIC2018 datasets, we demonstrate that SAMAug can improve SAM's performance, especially when using the maximum distance and saliency model point selection approaches. Our results showcase more possibilities of incorporating human inputs in a visual prompt setting. SAMAug represents an important step towards prompt-based augmentation methods for computer vision that can reduce user inputs and improve model performance.&#10;&#10;This study makes the following key contributions:&#10;\begin{itemize}&#10; \item We develop a novel visual point augmentation framework, SAMAug, for generating additional prompts without extra manual operations for SAM.&#10; \item We propose a visual prompt augmentation theory based on the invariance in prompt selection.&#10; \item We tested different techniques for visual prompt augmentation with experiments on one general-domain and three medical-domain datasets and identified the most effective augmentation techniques (maximum distance and saliency).&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Computer Science, Point Augmentation, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2303.15935" label="2303.15935">
-        <attvalues>
-          <attvalue for="0" value="When Brain-inspired AI Meets AGI" />
-          <attvalue for="1" value="  Artificial General Intelligence (AGI) has been a long-standing goal of&#10;humanity, with the aim of creating machines capable of performing any&#10;intellectual task that humans can do. To achieve this, AGI researchers draw&#10;inspiration from the human brain and seek to replicate its principles in&#10;intelligent machines. Brain-inspired artificial intelligence is a field that&#10;has emerged from this endeavor, combining insights from neuroscience,&#10;psychology, and computer science to develop more efficient and powerful AI&#10;systems. In this article, we provide a comprehensive overview of brain-inspired&#10;AI from the perspective of AGI. We begin with the current progress in&#10;brain-inspired AI and its extensive connection with AGI. We then cover the&#10;important characteristics for both human intelligence and AGI (e.g., scaling,&#10;multimodality, and reasoning). We discuss important technologies toward&#10;achieving AGI in current AI systems, such as in-context learning and prompt&#10;tuning. We also investigate the evolution of AGI systems from both algorithmic&#10;and infrastructural perspectives. Finally, we explore the limitations and&#10;future of AGI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.09419" label="2302.09419">
-        <attvalues>
-          <attvalue for="0" value="A Comprehensive Survey on Pretrained Foundation Models: A History from&#10;  BERT to ChatGPT" />
-          <attvalue for="1" value="  Pretrained Foundation Models (PFMs) are regarded as the foundation for&#10;various downstream tasks with different data modalities. A PFM (e.g., BERT,&#10;ChatGPT, and GPT-4) is trained on large-scale data which provides a reasonable&#10;parameter initialization for a wide range of downstream applications. BERT&#10;learns bidirectional encoder representations from Transformers, which are&#10;trained on large datasets as contextual language models. Similarly, the&#10;generative pretrained transformer (GPT) method employs Transformers as the&#10;feature extractor and is trained using an autoregressive paradigm on large&#10;datasets. Recently, ChatGPT shows promising success on large language models,&#10;which applies an autoregressive language model with zero shot or few shot&#10;prompting. The remarkable achievements of PFM have brought significant&#10;breakthroughs to various fields of AI. Numerous studies have proposed different&#10;methods, raising the demand for an updated survey. This study provides a&#10;comprehensive review of recent research advancements, challenges, and&#10;opportunities for PFMs in text, image, graph, as well as other data modalities.&#10;The review covers the basic components and existing pretraining methods used in&#10;natural language processing, computer vision, and graph learning. Additionally,&#10;it explores advanced PFMs used for different data modalities and unified PFMs&#10;that consider data quality and quantity. The review also discusses research&#10;related to the fundamentals of PFMs, such as model efficiency and compression,&#10;security, and privacy. Finally, the study provides key implications, future&#10;research directions, challenges, and open problems in the field of PFMs.&#10;Overall, this survey aims to shed light on the research of the PFMs on&#10;scalability, security, logical reasoning ability, cross-domain learning&#10;ability, and the user-friendly interactive ability for artificial general&#10;intelligence.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.16327" label="2211.16327">
-        <attvalues>
-          <attvalue for="0" value="On the Power of Foundation Models" />
-          <attvalue for="1" value="  With infinitely many high-quality data points, infinite computational power,&#10;an infinitely large foundation model with a perfect training algorithm and&#10;guaranteed zero generalization error on the pretext task, can the model be used&#10;for everything? This question cannot be answered by the existing theory of&#10;representation, optimization or generalization, because the issues they mainly&#10;investigate are assumed to be nonexistent here. In this paper, we show that&#10;category theory provides powerful machinery to answer this question. We have&#10;proved three results. The first one limits the power of prompt-based learning,&#10;saying that the model can solve a downstream task with prompts if and only if&#10;the task is representable. The second one says fine tuning does not have this&#10;limit, as a foundation model with the minimum required power (up to symmetry)&#10;can theoretically solve downstream tasks for the category defined by pretext&#10;task, with fine tuning and enough resources. Our final result can be seen as a&#10;new type of generalization theorem, showing that the foundation model can&#10;generate unseen objects from the target category (e.g., images) using the&#10;structural information from the source category (e.g., texts). Along the way,&#10;we provide a categorical framework for supervised and self-supervised learning,&#10;which might be of independent interest.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.15460" label="2012.15460">
-        <attvalues>
-          <attvalue for="0" value="TransTrack: Multiple Object Tracking with Transformer" />
-          <attvalue for="1" value="  In this work, we propose TransTrack, a simple but efficient scheme to solve&#10;the multiple object tracking problems. TransTrack leverages the transformer&#10;architecture, which is an attention-based query-key mechanism. It applies&#10;object features from the previous frame as a query of the current frame and&#10;introduces a set of learned object queries to enable detecting new-coming&#10;objects. It builds up a novel joint-detection-and-tracking paradigm by&#10;accomplishing object detection and object association in a single shot,&#10;simplifying complicated multi-step settings in tracking-by-detection methods.&#10;On MOT17 and MOT20 benchmark, TransTrack achieves 74.5\% and 64.5\% MOTA,&#10;respectively, competitive to the state-of-the-art methods. We expect TransTrack&#10;to provide a novel perspective for multiple object tracking. The code is&#10;available at: \url{https://github.com/PeizeSun/TransTrack}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In this section, we first review previous transformer applications in vision tasks. Then we introduce the two main MOT paradigms, namely tracking-by-detection and joint-detection-and-tracking methods.&#10;&#10;{ Transformer in vision tasks.} Recently, there is a popularity of using transformer architecture~\cite{vaswani2017attention} in vision tasks, where it has been proven powerful and inspiring. As a special query-key mechanism, the transformer heavily relies on the attention mechanism to process extracted deep features. It first shows great efficiency in natural language processing~\cite{vaswani2017attention} and later migrated to visual perception tasks~\cite{DETR} achieving remarkable success. Transformer appeals to the vision community with elegant structure and good performance. It has shown great potential in detection~\cite{DETR, deformdetr}, segmentation~\cite{zheng2020rethinking}, 3D data processing~\cite{zhao2020point} and even backbone construction~\cite{dosovitskiy2020image}. Lately, the good efforts of using a transformer in processing sequential visual data also make remarkable shots in video segmentation~\cite{wang2020endtoend}. With the natural strength of passing features along the temporal dimension, the transformer shows the ability to contribute to diverse temporal-spatial processing tasks on visual data and even replaces the role of traditional RNN models ~\cite{hochreiter1997long}. However, to the best of our knowledge, there are still no published transformer-based solutions for object tracking while it is intuitive to leverage its demonstrated good capacity in visual perception and temporal processing there. Hence, in this paper, we follow the insight to propose a transformer-based model for MOT. It shows convincingly high performance on the popular MOT benchmark.&#10;&#10;{ Tracking-by-detection.} State-of-the-art multiple object trackers are mostly dominated by the tracking-by-detection paradigm. It firstly uses the object detectors~\cite{lin2018focal, ren2016faster, FPN} to localize all objects of interest, then associates these detected objects according to their Re-ID features and/or other information, e.g., Intersectionover Unions (IoU) between each other. SORT~\cite{Bewley2016_sort} tracks bounding boxes using the Kalman Filter~\cite{welch1995introduction} and associates to the current frame by the Hungarian algorithm~\cite{kuhn1955hungarian}. DeepSORT~\cite{wojke2017simple} replaces the association cost in SORT with the appearance features from deep convolutional networks. POI~\cite{yu2016poi} achieves state-of-the-art tracking performance based on the high-performance detection and deep learning-based appearance feature. Lifted-Multicut~\cite{Tang_2017_CVPR} combines the deep representations and body pose feature obtained by the pose estimation model. STRN~\cite{xu2019spatialtemporal} presents a similarity learning framework between tracks and objects, which encodes various Spatial-Temporal relations. Tracking-by-detection pipeline achieves leading performance, but its model complexity and computational cost are not satisfying. &#10;&#10;{ Joint-detection-and-tracking.} The joint-detection-and-tracking pipeline aims to achieve detection and tracking simultaneously in a single stage. D\&amp;T~\cite{feichtenhofer2018detect} proposes a multi-task architecture for frame-based object detection and across-frame track regression. Integrated-Detection~\cite{zhang2018integrated} boosts the detection performance by combining the detection bounding boxes in the current frame and tracks in previous frames. More recently, Tracktor~\cite{bergmann2019tracking} directly uses the previous frame tracking boxes as region proposals and then applies the bounding box regression to provide tracking boxes on the current step, thus eliminating the box association procedure. JDE~\cite{wang2019towards} and FairMOT~\cite{fairmot} learn the object detection task and appearance embedding task from a shared backbone. CenterTrack~\cite{zhou2020tracking} localizes objects by tracking-conditioned detection and predicts their offsets to the previous frame. ChainedTracker~\cite{peng2020chained} chains paired bounding boxes estimated from overlapping nodes, in which each node covers two adjacent frames.&#10;Our proposed TransTrack falls into the joint-detection-and-tracking category. Previous works adopt anchor-based~\cite{ren2016faster} or point-based~\cite{zhou2019objects} detection framework. Instead, we build the pipeline based on a query-key mechanism and the tracked object feature is used as the query. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Computer Science, Transformer Architecture" />
-        </attvalues>
-      </node>
-      <node id="2104.03501" label="2104.03501">
-        <attvalues>
-          <attvalue for="0" value="DeepI2P: Image-to-Point Cloud Registration via Deep Classification" />
-          <attvalue for="1" value="  This paper presents DeepI2P: a novel approach for cross-modality registration&#10;between an image and a point cloud. Given an image (e.g. from a rgb-camera) and&#10;a general point cloud (e.g. from a 3D Lidar scanner) captured at different&#10;locations in the same scene, our method estimates the relative rigid&#10;transformation between the coordinate frames of the camera and Lidar. Learning&#10;common feature descriptors to establish correspondences for the registration is&#10;inherently challenging due to the lack of appearance and geometric correlations&#10;across the two modalities. We circumvent the difficulty by converting the&#10;registration problem into a classification and inverse camera projection&#10;optimization problem. A classification neural network is designed to label&#10;whether the projection of each point in the point cloud is within or beyond the&#10;camera frustum. These labeled points are subsequently passed into a novel&#10;inverse camera projection solver to estimate the relative pose. Extensive&#10;experimental results on Oxford Robotcar and KITTI datasets demonstrate the&#10;feasibility of our approach. Our source code is available at&#10;https://github.com/lijx10/DeepI2P&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Image-to-Image Registration.&#10;Images-to-image registrations \cite{shavit2019introduction, sattler2012improving} are done in the $\mathbb{P}^2$ space because of the lack of depth information. This is usually the first step to the computation of the projective transformation or SfM. Typical methods are usually based on feature matching. A set of features such as SIFT \cite{lowe1999object} or ORB \cite{rublee2011orb} are extracted from both source and target images. Correspondences are then established based on the extracted features, which can be used to solve for the rotation, translation using Bundle Adjustment \cite{triggs1999bundle,hartley2003multiple}, Perspective-n-Point solvers \cite{fischler1981random}, etc. Such techniques have been applied in modern SLAM systems \cite{engel2014lsd,mur2015orb,engel2017direct}. However, such methods are based on feature descriptors in the image modality to establish correspondences, and do not work for our general image-to-point cloud registration task.&#10;&#10;Point Cloud-to-Point Cloud Registration.&#10;The availability of 3D information enables direct registration between point clouds without establishing feature correspondences. Methods like ICP \cite{besl1992method,chen1992object}, NDT \cite{biber2003normal} work well with proper initial guess, and global optimization approaches such as Go-ICP \cite{yang2013go} work without initialization requirements. These methods are widely used in point cloud based SLAM algorithms like LOAM \cite{zhang2014loam}, Cartographer \cite{hess2016real}, etc. Recently data driven methods like DeepICP \cite{lu2019deepicp}, DeepClosestPoint \cite{wang2019deep}, RPM-Net \cite{yew2020-RPMNet}, etc, are also proposed. &#10;Although these approaches do not require feature correspondences, they still rely heavily on the geometrical details of the point structures in the same modality to work well. Consequently, these approaches cannot be applied to our task on cross-modal registration. &#10;Another group of common approaches is the two-step feature-based registration. Classical point cloud feature detectors \cite{tombari2013performance,zhong2009intrinsic,rusu20113d,dorai1997cosmos} and descriptors \cite{tombari2010unique,rusu2009fast} usually suffer from noise and clutter environments. Recently deep learning based feature detectors like USIP \cite{li2019usip}, 3DFeatNet \cite{yew20183dfeat}, and descriptors like 3DMatch \cite{zeng20173dmatch}, PPF-Net \cite{deng2018ppfnet}, PPF-FoldNet \cite{deng2018ppf}, PerfectMatch \cite{gojcic2019perfect}, have demonstrated improved performances in point cloud-based registration. Similar to image-to-image registration, these approaches require feature descriptors that are challenging to obtain in cross-modality registration.&#10;&#10;Image-to-Point Cloud Registration.&#10;To the best of our knowledge, 2D3D-MatchNet \cite{feng20192d3d} is the only prior work for general image-point cloud registration. It extracts images keypoints with SIFT \cite{lowe1999object}, and point cloud keypoints with ISS \cite{zhong2009intrinsic}. The image and point cloud patches around the keypoints are fed into each branch of a Siamese-like network and trained with triplet loss to extract cross-modal descriptors. At inference, it is a standard pipeline that consists of RANSAC-based descriptor matching and EPnP \cite{lepetit2009epnp} solver. Despite its greatly simplified experimental settings where the point clouds and images are captured at nearby timestamps with almost zero relative rotation, the low inlier rate of correspondences reveals the struggle for a deep network to learn common features across the drastically different modalities.&#10;Another work \cite{yu2020monocular} establishes 2D-3D line correspondences between images and prior Lidar maps, but they requires accurate initialization, e.g., from a SLAM/Odometry system. In contrast, the general image-to-point cloud registration, including 2D3D-MatchNet \cite{feng20192d3d} and our DeepI2P do not rely on another accurate localization system. Some other works \cite{pham2020lcd,cattaneo2020global} focus on image-to-point cloud place recognition / retrieval without estimating the relative rotation and translation.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Point Cloud Registration, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2206.03431" label="2206.03431">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Domain Adaptation in Crowd Counting" />
-          <attvalue for="1" value="  Self-training crowd counting has not been attentively explored though it is&#10;one of the important challenges in computer vision. In practice, the fully&#10;supervised methods usually require an intensive resource of manual annotation.&#10;In order to address this challenge, this work introduces a new approach to&#10;utilize existing datasets with ground truth to produce more robust predictions&#10;on unlabeled datasets, named domain adaptation, in crowd counting. While the&#10;network is trained with labeled data, samples without labels from the target&#10;domain are also added to the training process. In this process, the entropy map&#10;is computed and minimized in addition to the adversarial training process&#10;designed in parallel. Experiments on Shanghaitech, UCF_CC_50, and UCF-QNRF&#10;datasets prove a more generalized improvement of our method over the other&#10;state-of-the-arts in the cross-domain setting.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Crowd counting has recently been one of the popular tasks in computer vision. Recent developed methods \cite{wang2020DMCount, wang2021uniformity, song2021rethinking} and datasets \cite{7780439, 6619173, 10.1007/978-3-030-01216-8_33} have been introduced to tackle the counting task with thousands of targets. However, in real-world scenarios, these supervised methods usually learn to count through a training process that requires an extensive annotation of densely populated points in thousands of images. Directly employing models that are trained on existing datasets to a new dataset suffers from a significant performance decrease due to the domain gap.&#10;&#10;Therefore, in addition to semantic scene understanding \cite{le2018segmentation} and video temporal modeling \cite{duong2019learning, duong2029cvpr_automatic, Quach_2021_CVPR, Truong_2022_CVPR}, some self-training methods appear to utilize existing datasets with labels, i.e. source domain, and perform counting on more open-set scenarios, i.e. target domain, \cite{8578662,liu2020unsupervised} by transfer learning and domain adaptation techniques. Liu et al. \cite{liu2020unsupervised} enable knowledge distillation between both regression-based and detection-based models by formulating the mutual transformation of outputs. Xu et al. \cite{10.1109ICCV.2019.00847} enhance the generalization over density variance by categorizing image patches into several density levels. While general self learning methods improve the generalization capability by attempting to estimate pseudo ground-truths or distillation learning from a teacher network, a few approaches investigate a new direction to narrow the domain shift from entropy feedback of the target domain, especially in the semantic segmentation task \cite{vu2018advent}.&#10;&#10;In this paper, we introduce a new training approach to the crowd counting task toward a domain adaptation setting where the crowd counter utilizes the entropy minimization and adversarial learning to alleviate the distributional discrepancy between the source domain and the target domain.&#10;Particularly, our contributions can be summarized as follows:&#10;\begin{itemize}&#10; \item Reformulate the crowd counting problem from normally estimating density map to directly predicting target points in images, inspired by anchor-based and offset-based approaches.&#10;&#10;\item Utilize the Shannon entropy formula as a loss objective function to maximize the prediction certainty.&#10;&#10;\item Design an adversarial learning scheme to motivate the network to produce similar distributional predictions over the source domain and the target domain.&#10;&#10;\item Evaluate the proposed method with cross-domain settings to demonstrate its substantial generalization compared against the previous crowd counting methods and further perform estimating on a new chicken counting dataset.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Crowd Counting, Mathematics, Artificial Intelligence, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2009.13077" label="2009.13077">
-        <attvalues>
-          <attvalue for="0" value="Distribution Matching for Crowd Counting" />
-          <attvalue for="1" value="  In crowd counting, each training image contains multiple people, where each&#10;person is annotated by a dot. Existing crowd counting methods need to use a&#10;Gaussian to smooth each annotated dot or to estimate the likelihood of every&#10;pixel given the annotated point. In this paper, we show that imposing Gaussians&#10;to annotations hurts generalization performance. Instead, we propose to use&#10;Distribution Matching for crowd COUNTing (DM-Count). In DM-Count, we use&#10;Optimal Transport (OT) to measure the similarity between the normalized&#10;predicted density map and the normalized ground truth density map. To stabilize&#10;OT computation, we include a Total Variation loss in our model. We show that&#10;the generalization error bound of DM-Count is tighter than that of the Gaussian&#10;smoothed methods. In terms of Mean Absolute Error, DM-Count outperforms the&#10;previous state-of-the-art methods by a large margin on two large-scale counting&#10;datasets, UCF-QNRF and NWPU, and achieves the state-of-the-art results on the&#10;ShanghaiTech and UCF-CC50 datasets. DM-Count reduced the error of the&#10;state-of-the-art published result by approximately 16%. Code is available at&#10;https://github.com/cvlab-stonybrook/DM-Count.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.03593" label="1704.03593">
-        <attvalues>
-          <attvalue for="0" value="Reformulating Level Sets as Deep Recurrent Neural Network Approach to&#10;  Semantic Segmentation" />
-          <attvalue for="1" value="  Variational Level Set (LS) has been a widely used method in medical&#10;segmentation. However, it is limited when dealing with multi-instance objects&#10;in the real world. In addition, its segmentation results are quite sensitive to&#10;initial settings and highly depend on the number of iterations. To address&#10;these issues and boost the classic variational LS methods to a new level of the&#10;learnable deep learning approaches, we propose a novel definition of contour&#10;evolution named Recurrent Level Set (RLS)} to employ Gated Recurrent Unit under&#10;the energy minimization of a variational LS functional. The curve deformation&#10;process in RLS is formed as a hidden state evolution procedure and updated by&#10;minimizing an energy functional composed of fitting forces and contour length.&#10;By sharing the convolutional features in a fully end-to-end trainable&#10;framework, we extend RLS to Contextual RLS (CRLS) to address semantic&#10;segmentation in the wild. The experimental results have shown that our proposed&#10;RLS improves both computational time and segmentation accuracy against the&#10;classic variations LS-based method, whereas the fully end-to-end system CRLS&#10;achieves competitive performance compared to the state-of-the-art semantic&#10;segmentation approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.05383" label="2008.05383">
-        <attvalues>
-          <attvalue for="0" value="Towards Unsupervised Crowd Counting via Regression-Detection&#10;  Bi-knowledge Transfer" />
-          <attvalue for="1" value="  Unsupervised crowd counting is a challenging yet not largely explored task.&#10;In this paper, we explore it in a transfer learning setting where we learn to&#10;detect and count persons in an unlabeled target set by transferring&#10;bi-knowledge learnt from regression- and detection-based models in a labeled&#10;source set. The dual source knowledge of the two models is heterogeneous and&#10;complementary as they capture different modalities of the crowd distribution.&#10;We formulate the mutual transformations between the outputs of regression- and&#10;detection-based models as two scene-agnostic transformers which enable&#10;knowledge distillation between the two models. Given the regression- and&#10;detection-based models and their mutual transformers learnt in the source, we&#10;introduce an iterative self-supervised learning scheme with&#10;regression-detection bi-knowledge transfer in the target. Extensive experiments&#10;on standard crowd counting benchmarks, ShanghaiTech, UCF\_CC\_50, and UCF\_QNRF&#10;demonstrate a substantial improvement of our method over other&#10;state-of-the-arts in the transfer learning setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.12833" label="1811.12833">
-        <attvalues>
-          <attvalue for="0" value="ADVENT: Adversarial Entropy Minimization for Domain Adaptation in&#10;  Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation is a key problem for many computer vision tasks. While&#10;approaches based on convolutional neural networks constantly break new records&#10;on different benchmarks, generalizing well to diverse testing environments&#10;remains a major challenge. In numerous real world applications, there is indeed&#10;a large gap between data distributions in train and test domains, which results&#10;in severe performance loss at run-time. In this work, we address the task of&#10;unsupervised domain adaptation in semantic segmentation with losses based on&#10;the entropy of the pixel-wise predictions. To this end, we propose two novel,&#10;complementary methods using (i) entropy loss and (ii) adversarial loss&#10;respectively. We demonstrate state-of-the-art performance in semantic&#10;segmentation on two challenging &quot;synthetic-2-real&quot; set-ups and show that the&#10;approach can also be used for detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.07107" label="2109.07107">
-        <attvalues>
-          <attvalue for="0" value="Anchor DETR: Query Design for Transformer-Based Object Detection" />
-          <attvalue for="1" value="  In this paper, we propose a novel query design for the transformer-based&#10;object detection. In previous transformer-based detectors, the object queries&#10;are a set of learned embeddings. However, each learned embedding does not have&#10;an explicit physical meaning and we cannot explain where it will focus on. It&#10;is difficult to optimize as the prediction slot of each object query does not&#10;have a specific mode. In other words, each object query will not focus on a&#10;specific region. To solved these problems, in our query design, object queries&#10;are based on anchor points, which are widely used in CNN-based detectors. So&#10;each object query focuses on the objects near the anchor point. Moreover, our&#10;query design can predict multiple objects at one position to solve the&#10;difficulty: &quot;one region, multiple objects&quot;. In addition, we design an attention&#10;variant, which can reduce the memory cost while achieving similar or better&#10;performance than the standard attention in DETR. Thanks to the query design and&#10;the attention variant, the proposed detector that we called Anchor DETR, can&#10;achieve better performance and run faster than the DETR with 10$\times$ fewer&#10;training epochs. For example, it achieves 44.2 AP with 19 FPS on the MSCOCO&#10;dataset when using the ResNet50-DC5 feature for training 50 epochs. Extensive&#10;experiments on the MSCOCO benchmark prove the effectiveness of the proposed&#10;methods. Code is available at&#10;\url{https://github.com/megvii-research/AnchorDETR}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The object detection task is to predict a bounding box and a category for each object of interest in an image.&#10;In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}.&#10;Recently, Carion et al.~\cite{detr} propose the DETR which is a new paradigm of object detection based on the transformer.&#10;It uses a set of learned object queries to reason about the relations of the objects and the global image context to output the final predictions set.&#10;However, the learned object query is very hard to explain.&#10;It does not have an explicit physical meaning and the corresponding prediction slots of each object query do not have a specific mode.&#10;As shown in Figure~\ref{fig:query slots distribution}(a), the predictions of each object query in DETR are related to different areas and each object query will be in charge of a very large area.&#10;This positional ambiguity, i.e., the object query does not focus on a specific region, makes it hard to optimize. &#10;&#10;Reviewing the detectors based on CNN, the anchors are highly related to the position and contain interpretable physical meanings.&#10;Inspired by this motivation, we propose a novel query design based on the anchor points, i.e., we encode the anchor points as the object queries.&#10;The object queries are the encodings of anchor points' coordinates so that each object query has an explicit physical meaning.&#10;However, this solution will encounter difficulty: there will be multiple objects appearing in one position.&#10;In this situation, only one object query in this position cannot predict multiple objects, so that the object queries from other positions have to predict these objects collaboratively.&#10;It will cause each object query to be in charge of a larger area.&#10;Thus, we improve the object query design by adding multiple patterns to each anchor point so that each anchor point can predict multiple objects. &#10;As shown in Figure~\ref{fig:query slots distribution}(b), &#10;all the predictions of the three patterns of each object query are distributed around the corresponding anchor point.&#10;In other words, it demonstrates that each object query only focuses on the objects near the corresponding anchor point.&#10;So the proposed object query can be easy to explain.&#10;As the object query has a specific mode and does not need to predict the object far away from the corresponding position, it can easier to optimize.&#10;&#10;Besides the query design, we also design an attention variant that we call Row-Column Decouple Attention (RCDA).&#10;It decouples the 2D key feature into the 1D row feature and the 1D column feature, then conducts the row attention and column attention successively.&#10;The RCDA can reduce the memory cost while achieving similar or better performance than the standard attention in DETR.&#10;We believe it can be a good alternative to the standard attention in DETR. &#10;&#10;As shown in Table~\ref{tab:detr}, thanks to the novel query design based on the anchor point and the attention variant, the proposed detector Anchor DETR can achieve better performance and run faster than the original DETR with even 10$\times$ fewer training epochs when using the same single-level feature.&#10;Compared with other DETR-like detectors with 10$\times$ fewer training epochs, the proposed detector achieves the best performance among them.&#10;The proposed detector can achieve 44.2 AP with 19 FPS when using a single ResNet50-DC5~\cite{resnet} feature for training 50 epochs.&#10;&#10;The main contributions can be summarized as:&#10;\begin{itemize}&#10;&#09;\item &#10;&#09;We propose a novel query design based on anchor points for the transformer-based detectors.&#10;&#09;Moreover, we attach multiple patterns to each anchor point so that it can predict multiple objects for each position to deal with the difficulty: ``one region, multiple objects''.&#10;&#09;The proposed query based on anchor point is more explainable and easier to optimize than the learned embedding.&#10;&#09;Thanks to the effectiveness of the proposed query design, our detector can achieve better performance with 10$\times$ fewer training epochs than the DETR. &#10;&#09;\item We design an attention variant that we called Row-Column Decoupled Attention. It can reduce the memory cost while achieving similar or better performance than the standard attention in DETR, which can be a good alternative to standard attention.&#10;&#09;\item Extensive experiments are conducted to prove the effectiveness of each component. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Computer Science, Transformer Architecture" />
-        </attvalues>
-      </node>
-      <node id="1912.02424" label="1912.02424">
-        <attvalues>
-          <attvalue for="0" value="Bridging the Gap Between Anchor-based and Anchor-free Detection via&#10;  Adaptive Training Sample Selection" />
-          <attvalue for="1" value="  Object detection has been dominated by anchor-based detectors for several&#10;years. Recently, anchor-free detectors have become popular due to the proposal&#10;of FPN and Focal Loss. In this paper, we first point out that the essential&#10;difference between anchor-based and anchor-free detection is actually how to&#10;define positive and negative training samples, which leads to the performance&#10;gap between them. If they adopt the same definition of positive and negative&#10;samples during training, there is no obvious difference in the final&#10;performance, no matter regressing from a box or a point. This shows that how to&#10;select positive and negative training samples is important for current object&#10;detectors. Then, we propose an Adaptive Training Sample Selection (ATSS) to&#10;automatically select positive and negative samples according to statistical&#10;characteristics of object. It significantly improves the performance of&#10;anchor-based and anchor-free detectors and bridges the gap between them.&#10;Finally, we discuss the necessity of tiling multiple anchors per location on&#10;the image to detect objects. Extensive experiments conducted on MS COCO support&#10;our aforementioned analysis and conclusions. With the newly introduced ATSS, we&#10;improve state-of-the-art detectors by a large margin to $50.7\%$ AP without&#10;introducing any overhead. The code is available at&#10;https://github.com/sfzhang15/ATSS&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.02334" label="2006.02334">
-        <attvalues>
-          <attvalue for="0" value="DetectoRS: Detecting Objects with Recursive Feature Pyramid and&#10;  Switchable Atrous Convolution" />
-          <attvalue for="1" value="  Many modern object detectors demonstrate outstanding performances by using&#10;the mechanism of looking and thinking twice. In this paper, we explore this&#10;mechanism in the backbone design for object detection. At the macro level, we&#10;propose Recursive Feature Pyramid, which incorporates extra feedback&#10;connections from Feature Pyramid Networks into the bottom-up backbone layers.&#10;At the micro level, we propose Switchable Atrous Convolution, which convolves&#10;the features with different atrous rates and gathers the results using switch&#10;functions. Combining them results in DetectoRS, which significantly improves&#10;the performances of object detection. On COCO test-dev, DetectoRS achieves&#10;state-of-the-art 55.7% box AP for object detection, 48.5% mask AP for instance&#10;segmentation, and 50.0% PQ for panoptic segmentation. The code is made publicly&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.09460" label="2103.09460">
-        <attvalues>
-          <attvalue for="0" value="You Only Look One-level Feature" />
-          <attvalue for="1" value="  This paper revisits feature pyramids networks (FPN) for one-stage detectors&#10;and points out that the success of FPN is due to its divide-and-conquer&#10;solution to the optimization problem in object detection rather than&#10;multi-scale feature fusion. From the perspective of optimization, we introduce&#10;an alternative way to address the problem instead of adopting the complex&#10;feature pyramids - {\em utilizing only one-level feature for detection}. Based&#10;on the simple and efficient solution, we present You Only Look One-level&#10;Feature (YOLOF). In our method, two key components, Dilated Encoder and Uniform&#10;Matching, are proposed and bring considerable improvements. Extensive&#10;experiments on the COCO benchmark prove the effectiveness of the proposed&#10;model. Our YOLOF achieves comparable results with its feature pyramids&#10;counterpart RetinaNet while being $2.5\times$ faster. Without transformer&#10;layers, YOLOF can match the performance of DETR in a single-level feature&#10;manner with $7\times$ less training epochs. With an image size of&#10;$608\times608$, YOLOF achieves 44.3 mAP running at 60 fps on 2080Ti, which is&#10;$13\%$ faster than YOLOv4. Code is available at&#10;\url{https://github.com/megvii-model/YOLOF}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.05509" label="2107.05509">
-        <attvalues>
-          <attvalue for="0" value="Multi-view Image-based Hand Geometry Refinement using Differentiable&#10;  Monte Carlo Ray Tracing" />
-          <attvalue for="1" value="  The amount and quality of datasets and tools available in the research field&#10;of hand pose and shape estimation act as evidence to the significant progress&#10;that has been made.However, even the datasets of the highest quality, reported&#10;to date, have shortcomings in annotation. We propose a refinement approach,&#10;based on differentiable ray tracing,and demonstrate how a high-quality publicly&#10;available, multi-camera dataset of hands(InterHand2.6M) can become an even&#10;better dataset, with respect to annotation quality. Differentiable ray tracing&#10;has not been employed so far to relevant problems and is hereby shown to be&#10;superior to the approximative alternatives that have been employed in the past.&#10;To tackle the lack of reliable ground truth, as far as quantitative evaluation&#10;is concerned, we resort to realistic synthetic data, to show that the&#10;improvement we induce is indeed significant. The same becomes evident in real&#10;data through visual evaluation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Estimating the pose and shape of a human hand in 3D from RGB images is an important problem with numerous applications in human-computer interaction, augmented reality, robotics and more. State-of-the-art solutions have shown impressive results in real-world settings (see \ref{sec:literature}) with their remarkable performance heavily relying on advances in deep learning architectures \cite{simonyan2014very,he2016deep} &#10;and the availability of large datasets for training. Synthetic datasets have perfect ground truth information, but their lack of realism hinders generalization to the real domain. Real datasets model the real world directly, but come with noisy manual or automatically generated pose/shape annotations, that are nevertheless considered to be ground truth (see \ref{fig:concept}). As a result, state-of-the-art 3D hand reconstructions from visual data exhibit mediocre image-to-model alignment. Increasing the amount of training data will improve the results only to the degree that the quality of the accompanied annotations is high. The ability to improve upon imperfect hand pose and shape estimates provided as image annotations or as the output of a neural network during training or testing can greatly benefit the performance of hand reconstruction methods. &#10;&#10;In this work, we propose a method for refining an imperfect estimate of the geometry of a hand using calibrated \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{m} m ultiview images as input. We follow a render-and-compare approach, and optimize the shape, pose and global orientation of the hand so that the discrepancy between the rendered and the observed hand, across multiple views, is minimized. The effectiveness of our method lies&#10;\begin{inparaenum}[(a)]&#10; \item in the density of the information exploited as constraints to the problem, \ie\ all the pixels in all images that correspond to the hand, as well as&#10; \item the appropriateness of sampling in rendering, taken into account in the forward and the backward step of the optimization.&#10;\end{inparaenum}&#10;&#10;No previous work addresses unsupervised hand geometry refinement. However, the constancy of the appearance across images has been exploited in the context of hand or human pose estimation&#10;in recent works~\cite{panteleris2017back,Moon_2020_ECCV_DeepHandMesh,bogo2014faust}.&#10;Most rendering components invoked in the literature can be mapped to Pytorch3D~\cite{ravi2020pytorch3d}, which implements Neural Rendering~\cite{kato2018renderer} and Soft Rasterization~\cite{liu2020general}. Contrary to what has been employed so far, we make use of \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{d} d ifferentiable \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{r} r ay tracing \cite{Li:2018:DMC, nimier2019mitsuba}, and show that it delivers improved results, in comparison. For an impression of the gap between rasterization and \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{r} r ay tracing, and sampling correctness the reader is referred to \cite{glassner1989introduction}. We employ the differentiable Monte Carlo \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{r} r ay tracer by Li \etal \cite{Li:2018:DMC}, which provides a solution to gradient discontinuities at the edges. The latter practically implies that the benefits of \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{r} r ay tracing can also be exploited in the backward pass, instead of just the forward pass.&#10;&#10;We showcase our method on a subset of the recently proposed real and large-scale dataset InterHand2.6M~\cite{Moon_2020_ECCV_InterHand2.6M}. InterHand2.6M contains imperfect ground truth annotations (see \ref{fig:concept}) comprising 3D joint locations, derived in a semi-automatic way, and associated hand shape and pose descriptions, expressed via the MANO hand model, with a reported error of around $5 mm$. To circumvent the lack of ground truth data that are reliable enough for quantitative performance assessment, we resort to synthetic experimentation that closely resembles the setup in InterHand2.6M. We are able to successfully refine human hand estimates corresponding to a large variety of hand poses, shapes and appearances.&#10;&#10;Our contributions can be summarized as follows.&#10;\begin{itemize}&#10; \item We propose an effective unsupervised method for hand pose and shape refinement from \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{m} m ultiview images, given an initial estimate of the 3D hand geometry and, optionally, noisy target 3D joint locations. We complement the sparse noisy information about the 3D joint locations with rich image cues related to the appearance of the hand.&#10; \item We employ the proposed method to improve on a real dataset that is already on par with the state of the art with respect to annotation quality, while being richer in other aspects (number of samples, diversity in hand shape, pose, appearance, etc), yielding a dataset that is overall of even better quality.&#10; \item We employ, for the first time, a differentiable \ifnum\ifhmode\spacefactor2000&gt;1000 \uppercase{r} r ay tracer to the problem of estimating the 3D pose and shape of hands, that is shown to be theoretically and practically superior to the rasterization methods that have been employed so far. &#10; \item We provide a suite of synthetic experimentation to connect the degree of color consistency, as a function of geometry and appearance, with the improvement in 3D estimation of the hand, for the described scenario.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Dataset Annotation, Computer Graphics, Hand Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1705.05301" label="1705.05301">
-        <attvalues>
-          <attvalue for="0" value="Back to RGB: 3D tracking of hands and hand-object interactions based on&#10;  short-baseline stereo" />
-          <attvalue for="1" value="  We present a novel solution to the problem of 3D tracking of the articulated&#10;motion of human hand(s), possibly in interaction with other objects. The vast&#10;majority of contemporary relevant work capitalizes on depth information&#10;provided by RGBD cameras. In this work, we show that accurate and efficient 3D&#10;hand tracking is possible, even for the case of RGB stereo. A straightforward&#10;approach for solving the problem based on such input would be to first recover&#10;depth and then apply a state of the art depth-based 3D hand tracking method.&#10;Unfortunately, this does not work well in practice because the stereo-based,&#10;dense 3D reconstruction of hands is far less accurate than the one obtained by&#10;RGBD cameras. Our approach bypasses 3D reconstruction and follows a completely&#10;different route: 3D hand tracking is formulated as an optimization problem&#10;whose solution is the hand configuration that maximizes the color consistency&#10;between the two views of the hand. We demonstrate the applicability of our&#10;method for real time tracking of a single hand, of a hand manipulating an&#10;object and of two interacting hands. The method has been evaluated&#10;quantitatively on standard datasets and in comparison to relevant, state of the&#10;art RGBD-based approaches. The obtained results demonstrate that the proposed&#10;stereo-based method performs equally well to its RGBD-based competitors, and in&#10;some cases, it even outperforms them.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.08213" label="2008.08213">
-        <attvalues>
-          <attvalue for="0" value="DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for&#10;  High-fidelity Hand Mesh Modeling" />
-          <attvalue for="1" value="  Human hands play a central role in interacting with other people and objects.&#10;For realistic replication of such hand motions, high-fidelity hand meshes have&#10;to be reconstructed. In this study, we firstly propose DeepHandMesh, a&#10;weakly-supervised deep encoder-decoder framework for high-fidelity hand mesh&#10;modeling. We design our system to be trained in an end-to-end and&#10;weakly-supervised manner; therefore, it does not require groundtruth meshes.&#10;Instead, it relies on weaker supervisions such as 3D joint coordinates and&#10;multi-view depth maps, which are easier to get than groundtruth meshes and do&#10;not dependent on the mesh topology. Although the proposed DeepHandMesh is&#10;trained in a weakly-supervised way, it provides significantly more realistic&#10;hand mesh than previous fully-supervised hand models. Our newly introduced&#10;penetration avoidance loss further improves results by replicating physical&#10;interaction between hand parts. Finally, we demonstrate that our system can&#10;also be applied successfully to the 3D hand mesh estimation from general&#10;images. Our hand model, dataset, and codes are publicly available at&#10;https://mks0601.github.io/DeepHandMesh/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.08501" label="2007.08501">
-        <attvalues>
-          <attvalue for="0" value="Accelerating 3D Deep Learning with PyTorch3D" />
-          <attvalue for="1" value="  Deep learning has significantly improved 2D image recognition. Extending into&#10;3D may advance many new applications including autonomous vehicles, virtual and&#10;augmented reality, authoring 3D content, and even improving 2D recognition.&#10;However despite growing interest, 3D deep learning remains relatively&#10;underexplored. We believe that some of this disparity is due to the engineering&#10;challenges involved in 3D deep learning, such as efficiently processing&#10;heterogeneous data and reframing graphics operations to be differentiable. We&#10;address these challenges by introducing PyTorch3D, a library of modular,&#10;efficient, and differentiable operators for 3D deep learning. It includes a&#10;fast, modular differentiable renderer for meshes and point clouds, enabling&#10;analysis-by-synthesis approaches. Compared with other differentiable renderers,&#10;PyTorch3D is more modular and efficient, allowing users to more easily extend&#10;it while also gracefully scaling to large meshes and images. We compare the&#10;PyTorch3D operators and renderer with other implementations and demonstrate&#10;significant speed and memory improvements. We also use PyTorch3D to improve the&#10;state-of-the-art for unsupervised 3D mesh and point cloud prediction from 2D&#10;images on ShapeNet. PyTorch3D is open-source and we hope it will help&#10;accelerate research in 3D deep learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07566" label="1711.07566">
-        <attvalues>
-          <attvalue for="0" value="Neural 3D Mesh Renderer" />
-          <attvalue for="1" value="  For modeling the 3D world behind 2D images, which 3D representation is most&#10;appropriate? A polygon mesh is a promising candidate for its compactness and&#10;geometric properties. However, it is not straightforward to model a polygon&#10;mesh from 2D images using neural networks because the conversion from a mesh to&#10;an image, or rendering, involves a discrete operation called rasterization,&#10;which prevents back-propagation. Therefore, in this work, we propose an&#10;approximate gradient for rasterization that enables the integration of&#10;rendering into neural networks. Using this renderer, we perform single-image 3D&#10;mesh reconstruction with silhouette image supervision and our system&#10;outperforms the existing voxel-based approach. Additionally, we perform&#10;gradient-based 3D mesh editing operations, such as 2D-to-3D style transfer and&#10;3D DeepDream, with 2D supervision for the first time. These applications&#10;demonstrate the potential of the integration of a mesh renderer into neural&#10;networks and the effectiveness of our proposed renderer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.04323" label="2112.04323">
-        <attvalues>
-          <attvalue for="0" value="Contrastive Learning with Large Memory Bank and Negative Embedding&#10;  Subtraction for Accurate Copy Detection" />
-          <attvalue for="1" value="  Copy detection, which is a task to determine whether an image is a modified&#10;copy of any image in a database, is an unsolved problem. Thus, we addressed&#10;copy detection by training convolutional neural networks (CNNs) with&#10;contrastive learning. Training with a large memory-bank and hard data&#10;augmentation enables the CNNs to obtain more discriminative representation. Our&#10;proposed negative embedding subtraction further boosts the copy detection&#10;accuracy. Using our methods, we achieved 1st place in the Facebook AI Image&#10;Similarity Challenge: Descriptor Track. Our code is publicly available here:&#10;\url{https://github.com/lyakaap/ISC21-Descriptor-Track-1st}&#10;" />
-          <attvalue for="2" value="&#10;In recent years, with the development of social media, the problems of plagiarism and unauthorized copy have become more serious. Accordingly, attention to research on copy detection, which is a task to determine whether an image is a modified copy of any image in a database, has increased~\cite{ISC2021,Jiang2014VCDBALCopy,Christlein2012AnEOCopy}.&#10;&#10;In this study, we propose a strong copy detection pipeline that consists of EfficientNetV2~\cite{EfficientNetV2} trained with contrastive learning and a post-process that effectively utilizes negative embeddings.&#10;Our training pipeline has multiple steps inspired by progressive learning~\cite{EfficientNetV2}, which is a model training technique that increases input image resolution and regularization as a training step proceeds.&#10;In each step, models are trained by contrastive learning with hard data augmentation to obtain discriminative representation.&#10;Embeddings extracted by our model are processed employing our proposed negative embedding subtraction, which improves the copy detection performance by isolating a target sample from similar negative samples.&#10;Using our methods, accurate copy detection is realized even in difficult samples as shown in Figure \ref{fig:teaser}.&#10;&#10;Our contribution is summarized as follows: (1) multi-step training with contrastive learning and large memory-bank, (2) carefully designed data augmentation strategy that precisely reproduces manipulated images from the dataset, (3) novel post-process method that enhances embeddings utilizing negative samples, (4) results that significantly outperform baseline methods on a challenging dataset, enabling to win the Facebook AI Image Similarity Challenge: Descriptor Track at NeurIPS’21.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Convolutional Neural Networks, Mathematics, Artificial Intelligence, Contrastive Learning, Computer Science, Image Copy Detection" />
-        </attvalues>
-      </node>
-      <node id="2106.09672" label="2106.09672">
-        <attvalues>
-          <attvalue for="0" value="The 2021 Image Similarity Dataset and Challenge" />
-          <attvalue for="1" value="  This paper introduces a new benchmark for large-scale image similarity&#10;detection. This benchmark is used for the Image Similarity Challenge at&#10;NeurIPS'21 (ISC2021). The goal is to determine whether a query image is a&#10;modified copy of any image in a reference corpus of size 1~million. The&#10;benchmark features a variety of image transformations such as automated&#10;transformations, hand-crafted image edits and machine-learning based&#10;manipulations. This mimics real-life cases appearing in social media, for&#10;example for integrity-related problems dealing with misinformation and&#10;objectionable content. The strength of the image manipulations, and therefore&#10;the difficulty of the benchmark, is calibrated according to the performance of&#10;a set of baseline approaches. Both the query and reference set contain a&#10;majority of &quot;distractor&quot; images that do not match, which corresponds to a&#10;real-life needle-in-haystack setting, and the evaluation metric reflects that.&#10;We expect the DISC21 benchmark to promote image copy detection as an important&#10;and challenging computer vision task and refresh the state of the art. Code and&#10;data are available at https://github.com/facebookresearch/isc2021&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1208.3665" label="1208.3665">
-        <attvalues>
-          <attvalue for="0" value="An Evaluation of Popular Copy-Move Forgery Detection Approaches" />
-          <attvalue for="1" value="  A copy-move forgery is created by copying and pasting content within the same&#10;image, and potentially post-processing it. In recent years, the detection of&#10;copy-move forgeries has become one of the most actively researched topics in&#10;blind image forensics. A considerable number of different algorithms have been&#10;proposed focusing on different types of postprocessed copies. In this paper, we&#10;aim to answer which copy-move forgery detection algorithms and processing steps&#10;(e.g., matching, filtering, outlier detection, affine transformation&#10;estimation) perform best in various postprocessing scenarios. The focus of our&#10;analysis is to evaluate the performance of previously proposed feature sets. We&#10;achieve this by casting existing algorithms in a common pipeline. In this&#10;paper, we examined the 15 most prominent feature sets. We analyzed the&#10;detection performance on a per-image basis and on a per-pixel basis. We created&#10;a challenging real-world copy-move dataset, and a software framework for&#10;systematic image manipulation. Experiments show, that the keypoint-based&#10;features SIFT and SURF, as well as the block-based DCT, DWT, KPCA, PCA and&#10;Zernike features perform very well. These feature sets exhibit the best&#10;robustness against various noise sources and downsampling, while reliably&#10;identifying the copied regions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00298" label="2104.00298">
-        <attvalues>
-          <attvalue for="0" value="EfficientNetV2: Smaller Models and Faster Training" />
-          <attvalue for="1" value="  This paper introduces EfficientNetV2, a new family of convolutional networks&#10;that have faster training speed and better parameter efficiency than previous&#10;models. To develop this family of models, we use a combination of&#10;training-aware neural architecture search and scaling, to jointly optimize&#10;training speed and parameter efficiency. The models were searched from the&#10;search space enriched with new ops such as Fused-MBConv. Our experiments show&#10;that EfficientNetV2 models train much faster than state-of-the-art models while&#10;being up to 6.8x smaller.&#10;  Our training can be further sped up by progressively increasing the image&#10;size during training, but it often causes a drop in accuracy. To compensate for&#10;this accuracy drop, we propose to adaptively adjust regularization (e.g.,&#10;dropout and data augmentation) as well, such that we can achieve both fast&#10;training and good accuracy.&#10;  With progressive learning, our EfficientNetV2 significantly outperforms&#10;previous models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining on&#10;the same ImageNet21k, our EfficientNetV2 achieves 87.3% top-1 accuracy on&#10;ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while&#10;training 5x-11x faster using the same computing resources. Code will be&#10;available at https://github.com/google/automl/tree/master/efficientnetv2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.07238" label="2409.07238">
-        <attvalues>
-          <attvalue for="0" value="Diff-VPS: Video Polyp Segmentation via a Multi-task Diffusion Network&#10;  with Adversarial Temporal Reasoning" />
-          <attvalue for="1" value="  Diffusion Probabilistic Models have recently attracted significant attention&#10;in the community of computer vision due to their outstanding performance.&#10;However, while a substantial amount of diffusion-based research has focused on&#10;generative tasks, no work introduces diffusion models to advance the results of&#10;polyp segmentation in videos, which is frequently challenged by polyps' high&#10;camouflage and redundant temporal cues.In this paper, we present a novel&#10;diffusion-based network for video polyp segmentation task, dubbed as Diff-VPS.&#10;We incorporate multi-task supervision into diffusion models to promote the&#10;discrimination of diffusion models on pixel-by-pixel segmentation. This&#10;integrates the contextual high-level information achieved by the joint&#10;classification and detection tasks. To explore the temporal dependency,&#10;Temporal Reasoning Module (TRM) is devised via reasoning and reconstructing the&#10;target frame from the previous frames. We further equip TRM with a generative&#10;adversarial self-supervised strategy to produce more realistic frames and thus&#10;capture better dynamic cues. Extensive experiments are conducted on SUN-SEG,&#10;and the results indicate that our proposed Diff-VPS significantly achieves&#10;state-of-the-art performance. Code is available at&#10;https://github.com/lydia-yllu/Diff-VPS.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Colorectal cancer (CRC) has become the second leading cause of cancer-related deaths worldwide, accounting for approximately 10\% of all cancer cases. By 2040 the burden of colorectal cancer will increase to 3.2 million new cases per year (an increase of 63\%) and 1.6 million deaths per year (an increase of 73\%)~\cite{bernal2012towards,wu2024colorectal}. Colonoscopy plays a crucial role in detecting polyps and serves as a screening tool for CRC~\cite{xing2024segmamba,xing2022nestedformer}. However, there still exists a high missing rate in the diagnosis and treatment process. &#10;Recently, emerging methods have utilized deep learning techniques to automatically detect polyps in video, from popular convolutional neural networks to attention mechanism~\cite{tajbakhsh2015automated,puyal2020endoscopic,ji2021progressively,ji2022video,yang2023mammodg,shaharabany2023autosam,ali2024objective,yang2024vivim}. &#10;Nevertheless, video polyp segmentation (VPS) always poses the ill-posed bottleneck, \ie, the high camouflage of polyps in the camera, more specifically, the intrinsic complexity of polyps (shape and color variability, fuzzy boundaries) and the external shooting conditions (low boundary contrast, specular reflection, artifacts). &#10;Also, the temporal cues in colonoscopy videos require to be finely explored, which refers to the effective utilization of long-term temporal characteristics within a video dataset. &#10;While parts of the above methods achieved significant progress in segmenting polyps in videos, there still exists great potential to be explored considering their failure cases and limited discriminative ability.&#10;&#10;Diffusion models have shown impressive performance in many vision tasks, such as image deblurring~\cite{kawar2022denoising}, super-resolution~\cite{li2022srdiff,gao2023implicit}, and anomaly detection~\cite{wyatt2022anoddpm,chen2023diffusiondet}. The denoising diffusion probabilistic model (DDPM)~\cite{ho2020denoising} is a generative model based on a Markov chain. Simulating a diffusion process, DDPM captures the data distribution as it guides the input data towards a predefined target distribution, i.e. Gaussian distribution. &#10;By representing the diffusion process of image intensity values over successive iterations, diffusion models adeptly grasp the intrinsic structure and texture within images, facilitating the distinct segregation of regions of interest from the background~\cite{amit2023annotator}. &#10;While several pioneers~\cite{wu2024medsegdiff,xing2023diff,wolleb2022diffusion,yang2023diffmic} applied DDPM to medical image classification and segmentation, there is no diffusion-based work on medical video lesion segmentation.&#10;&#10;Motivated by the nature of diffusion models~\cite{ji2023ddp}, in this paper, 1) we present the first diffusion network, dubbed as Diff-VPS, for the VPS task. Polyps can be diagnosed into different categories (hyperplastic polyp, invasive carcinoma, \etc) depending on the size and morphological features~\cite{ji2022video}. &#10;Inspired by this, 2) we design the Multi-task Diffusion Model where classification and detection tasks are performed simultaneously to improve the discrimination on the segmentation task. Such supervision significantly advances the discriminative and generalizable ability of diffusion models by high-level semantic information. &#10;Furthermore, to capture the dynamic appearance and maintain the temporal coherence in segmenting polyps, 3) we devise a Temporal Reasoning Module by reconstructing the target frame from the previous temporal information and employing an adversarial self-supervised strategy. &#10;Extensive experiments are conducted on the SUN-SEG dataset. The superior performance on seen and unseen videos validates the effectiveness and versatility of our Diff-VPS.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Diffusion Models, Medical Imaging, Machine Learning, Computer Vision, Medicine, Mathematics, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2401.11734" label="2401.11734">
-        <attvalues>
-          <attvalue for="0" value="Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive&#10;  Survey" />
-          <attvalue for="1" value="  Colorectal polyp segmentation (CPS), an essential problem in medical image&#10;analysis, has garnered growing research attention. Recently, the deep&#10;learning-based model completely overwhelmed traditional methods in the field of&#10;CPS, and more and more deep CPS methods have emerged, bringing the CPS into the&#10;deep learning era. To help the researchers quickly grasp the main techniques,&#10;datasets, evaluation metrics, challenges, and trending of deep CPS, this paper&#10;presents a systematic and comprehensive review of deep-learning-based CPS&#10;methods from 2014 to 2023, a total of 115 technical papers. In particular, we&#10;first provide a comprehensive review of the current deep CPS with a novel&#10;taxonomy, including network architectures, level of supervision, and learning&#10;paradigm. More specifically, network architectures include eight subcategories,&#10;the level of supervision comprises six subcategories, and the learning paradigm&#10;encompasses 12 subcategories, totaling 26 subcategories. Then, we provided a&#10;comprehensive analysis the characteristics of each dataset, including the&#10;number of datasets, annotation types, image resolution, polyp size, contrast&#10;values, and polyp location. Following that, we summarized CPS's commonly used&#10;evaluation metrics and conducted a detailed analysis of 40 deep SOTA models,&#10;including out-of-distribution generalization and attribute-based performance&#10;analysis. Finally, we discussed deep learning-based CPS methods' main&#10;challenges and opportunities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.13560" label="2401.13560">
-        <attvalues>
-          <attvalue for="0" value="SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image&#10;  Segmentation" />
-          <attvalue for="1" value="  The Transformer architecture has shown a remarkable ability in modeling&#10;global relationships. However, it poses a significant computational challenge&#10;when processing high-dimensional medical images. This hinders its development&#10;and widespread adoption in this task. Mamba, as a State Space Model (SSM),&#10;recently emerged as a notable manner for long-range dependencies in sequential&#10;modeling, excelling in natural language processing filed with its remarkable&#10;memory efficiency and computational speed. Inspired by its success, we&#10;introduce SegMamba, a novel 3D medical image \textbf{Seg}mentation&#10;\textbf{Mamba} model, designed to effectively capture long-range dependencies&#10;within whole volume features at every scale. Our SegMamba, in contrast to&#10;Transformer-based methods, excels in whole volume feature modeling from a state&#10;space model standpoint, maintaining superior processing speed, even with volume&#10;features at a resolution of {$64\times 64\times 64$}. Comprehensive experiments&#10;on the BraTS2023 dataset demonstrate the effectiveness and efficiency of our&#10;SegMamba. The code for SegMamba is available at:&#10;https://github.com/ge-xing/SegMamba&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.14876" label="2208.14876">
-        <attvalues>
-          <attvalue for="0" value="NestedFormer: Nested Modality-Aware Transformer for Brain Tumor&#10;  Segmentation" />
-          <attvalue for="1" value="  Multi-modal MR imaging is routinely used in clinical practice to diagnose and&#10;investigate brain tumors by providing rich complementary information. Previous&#10;multi-modal MRI segmentation methods usually perform modal fusion by&#10;concatenating multi-modal MRIs at an early/middle stage of the network, which&#10;hardly explores non-linear dependencies between modalities. In this work, we&#10;propose a novel Nested Modality-Aware Transformer (NestedFormer) to explicitly&#10;explore the intra-modality and inter-modality relationships of multi-modal MRIs&#10;for brain tumor segmentation. Built on the transformer-based multi-encoder and&#10;single-decoder structure, we perform nested multi-modal fusion for high-level&#10;representations of different modalities and apply modality-sensitive gating&#10;(MSG) at lower scales for more effective skip connections. Specifically, the&#10;multi-modal fusion is conducted in our proposed Nested Modality-aware Feature&#10;Aggregation (NMaFA) module, which enhances long-term dependencies within&#10;individual modalities via a tri-orientated spatial-attention transformer, and&#10;further complements key contextual information among modalities via a&#10;cross-modality attention transformer. Extensive experiments on BraTS2020&#10;benchmark and a private meningiomas segmentation (MeniSeg) dataset show that&#10;the NestedFormer clearly outperforms the state-of-the-arts. The code is&#10;available at https://github.com/920232796/NestedFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.01057" label="2308.01057">
-        <attvalues>
-          <attvalue for="0" value="MammoDG: Generalisable Deep Learning Breaks the Limits of Cross-Domain&#10;  Multi-Center Breast Cancer Screening" />
-          <attvalue for="1" value="  Breast cancer is a major cause of cancer death among women, emphasising the&#10;importance of early detection for improved treatment outcomes and quality of&#10;life. Mammography, the primary diagnostic imaging test, poses challenges due to&#10;the high variability and patterns in mammograms. Double reading of mammograms&#10;is recommended in many screening programs to improve diagnostic accuracy but&#10;increases radiologists' workload. Researchers explore Machine Learning models&#10;to support expert decision-making. Stand-alone models have shown comparable or&#10;superior performance to radiologists, but some studies note decreased&#10;sensitivity with multiple datasets, indicating the need for high generalisation&#10;and robustness models. This work devises MammoDG, a novel deep-learning&#10;framework for generalisable and reliable analysis of cross-domain multi-center&#10;mammography data. MammoDG leverages multi-view mammograms and a novel&#10;contrastive mechanism to enhance generalisation capabilities. Extensive&#10;validation demonstrates MammoDG's superiority, highlighting the critical&#10;importance of domain generalisation for trustworthy mammography analysis in&#10;imaging protocol variations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.06370" label="2306.06370">
-        <attvalues>
-          <attvalue for="0" value="AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt&#10;  Encoder" />
-          <attvalue for="1" value="  The recently introduced Segment Anything Model (SAM) combines a clever&#10;architecture and large quantities of training data to obtain remarkable image&#10;segmentation capabilities. However, it fails to reproduce such results for&#10;Out-Of-Distribution (OOD) domains such as medical images. Moreover, while SAM&#10;is conditioned on either a mask or a set of points, it may be desirable to have&#10;a fully automatic solution. In this work, we replace SAM's conditioning with an&#10;encoder that operates on the same input image. By adding this encoder and&#10;without further fine-tuning SAM, we obtain state-of-the-art results on multiple&#10;medical images and video benchmarks. This new encoder is trained via gradients&#10;provided by a frozen SAM. For inspecting the knowledge within it, and providing&#10;a lightweight segmentation solution, we also learn to decode it into a mask by&#10;a shallow deconvolution network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.15753" label="2401.15753">
-        <attvalues>
-          <attvalue for="0" value="An objective comparison of methods for augmented reality in laparoscopic&#10;  liver resection by preoperative-to-intraoperative image fusion" />
-          <attvalue for="1" value="  Augmented reality for laparoscopic liver resection is a visualisation mode&#10;that allows a surgeon to localise tumours and vessels embedded within the liver&#10;by projecting them on top of a laparoscopic image. Preoperative 3D models&#10;extracted from CT or MRI data are registered to the intraoperative laparoscopic&#10;images during this process. In terms of 3D-2D fusion, most of the algorithms&#10;make use of anatomical landmarks to guide registration. These landmarks include&#10;the liver's inferior ridge, the falciform ligament, and the occluding contours.&#10;They are usually marked by hand in both the laparoscopic image and the 3D&#10;model, which is time-consuming and may contain errors if done by a&#10;non-experienced user. Therefore, there is a need to automate this process so&#10;that augmented reality can be used effectively in the operating room. We&#10;present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge&#10;(P2ILF), held during the Medical Imaging and Computer Assisted Interventions&#10;(MICCAI 2022) conference, which investigates the possibilities of detecting&#10;these landmarks automatically and using them in registration. The challenge was&#10;divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D&#10;registration task. The teams were provided with training data consisting of 167&#10;laparoscopic images and 9 preoperative 3D models from 9 patients, with the&#10;corresponding 2D and 3D landmark annotations. A total of 6 teams from 4&#10;countries participated, whose proposed methods were evaluated on 16 images and&#10;two preoperative 3D models from two patients. All the teams proposed deep&#10;learning-based methods for the 2D and 3D landmark segmentation tasks and&#10;differentiable rendering-based methods for the registration task. Based on the&#10;experimental outcomes, we propose three key hypotheses that determine current&#10;limitations and future directions for research in this domain.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.16491" label="2303.16491">
-        <attvalues>
-          <attvalue for="0" value="Implicit Diffusion Models for Continuous Super-Resolution" />
-          <attvalue for="1" value="  Image super-resolution (SR) has attracted increasing attention due to its&#10;wide applications. However, current SR methods generally suffer from&#10;over-smoothing and artifacts, and most work only with fixed magnifications.&#10;This paper introduces an Implicit Diffusion Model (IDM) for high-fidelity&#10;continuous image super-resolution. IDM integrates an implicit neural&#10;representation and a denoising diffusion model in a unified end-to-end&#10;framework, where the implicit neural representation is adopted in the decoding&#10;process to learn continuous-resolution representation. Furthermore, we design a&#10;scale-controllable conditioning mechanism that consists of a low-resolution&#10;(LR) conditioning network and a scaling factor. The scaling factor regulates&#10;the resolution and accordingly modulates the proportion of the LR information&#10;and generated features in the final output, which enables the model to&#10;accommodate the continuous-resolution requirement. Extensive experiments&#10;validate the effectiveness of our IDM and demonstrate its superior performance&#10;over prior arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.09788" label="2211.09788">
-        <attvalues>
-          <attvalue for="0" value="DiffusionDet: Diffusion Model for Object Detection" />
-          <attvalue for="1" value="  We propose DiffusionDet, a new framework that formulates object detection as&#10;a denoising diffusion process from noisy boxes to object boxes. During the&#10;training stage, object boxes diffuse from ground-truth boxes to random&#10;distribution, and the model learns to reverse this noising process. In&#10;inference, the model refines a set of randomly generated boxes to the output&#10;results in a progressive way. Our work possesses an appealing property of&#10;flexibility, which enables the dynamic number of boxes and iterative&#10;evaluation. The extensive experiments on the standard benchmarks show that&#10;DiffusionDet achieves favorable performance compared to previous&#10;well-established detectors. For example, DiffusionDet achieves 5.3 AP and 4.8&#10;AP gains when evaluated with more boxes and iteration steps, under a zero-shot&#10;transfer setting from COCO to CrowdHuman. Our code is available at&#10;https://github.com/ShoufaChen/DiffusionDet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.09004" label="2306.09004">
-        <attvalues>
-          <attvalue for="0" value="Annotator Consensus Prediction for Medical Image Segmentation with&#10;  Diffusion Models" />
-          <attvalue for="1" value="  A major challenge in the segmentation of medical images is the large inter-&#10;and intra-observer variability in annotations provided by multiple experts. To&#10;address this challenge, we propose a novel method for multi-expert prediction&#10;using diffusion models. Our method leverages the diffusion-based approach to&#10;incorporate information from multiple annotations and fuse it into a unified&#10;segmentation map that reflects the consensus of multiple experts. We evaluate&#10;the performance of our method on several datasets of medical segmentation&#10;annotated by multiple experts and compare it with state-of-the-art methods. Our&#10;results demonstrate the effectiveness and robustness of the proposed method.&#10;Our code is publicly available at&#10;https://github.com/tomeramit/Annotator-Consensus-Prediction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.00611" label="2211.00611">
-        <attvalues>
-          <attvalue for="0" value="MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic&#10;  Model" />
-          <attvalue for="1" value="  Diffusion probabilistic model (DPM) recently becomes one of the hottest topic&#10;in computer vision. Its image generation application such as Imagen, Latent&#10;Diffusion Models and Stable Diffusion have shown impressive generation&#10;capabilities, which aroused extensive discussion in the community. Many recent&#10;studies also found it is useful in many other vision tasks, like image&#10;deblurring, super-resolution and anomaly detection. Inspired by the success of&#10;DPM, we propose the first DPM based model toward general medical image&#10;segmentation tasks, which we named MedSegDiff. In order to enhance the&#10;step-wise regional attention in DPM for the medical image segmentation, we&#10;propose dynamic conditional encoding, which establishes the state-adaptive&#10;conditions for each sampling step. We further propose Feature Frequency Parser&#10;(FF-Parser), to eliminate the negative effect of high-frequency noise component&#10;in this process. We verify MedSegDiff on three medical segmentation tasks with&#10;different image modalities, which are optic cup segmentation over fundus&#10;images, brain tumor segmentation over MRI images and thyroid nodule&#10;segmentation over ultrasound images. The experimental results show that&#10;MedSegDiff outperforms state-of-the-art (SOTA) methods with considerable&#10;performance gap, indicating the generalization and effectiveness of the&#10;proposed model. Our code is released at https://github.com/WuJunde/MedSegDiff.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.10326" label="2303.10326">
-        <attvalues>
-          <attvalue for="0" value="Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentation" />
-          <attvalue for="1" value="  In recent years, Denoising Diffusion Models have demonstrated remarkable&#10;success in generating semantically valuable pixel-wise representations for&#10;image generative modeling. In this study, we propose a novel end-to-end&#10;framework, called Diff-UNet, for medical volumetric segmentation. Our approach&#10;integrates the diffusion model into a standard U-shaped architecture to extract&#10;semantic information from the input volume effectively, resulting in excellent&#10;pixel-level representations for medical volumetric segmentation. To enhance the&#10;robustness of the diffusion model's prediction results, we also introduce a&#10;Step-Uncertainty based Fusion (SUF) module during inference to combine the&#10;outputs of the diffusion models at each step. We evaluate our method on three&#10;datasets, including multimodal brain tumors in MRI, liver tumors, and&#10;multi-organ CT volumes, and demonstrate that Diff-UNet outperforms other&#10;state-of-the-art methods significantly. Our experimental results also indicate&#10;the universality and effectiveness of the proposed model. The proposed&#10;framework has the potential to facilitate the accurate diagnosis and treatment&#10;of medical conditions by enabling more precise segmentation of anatomical&#10;structures. The codes of Diff-UNet are available at&#10;https://github.com/ge-xing/Diff-UNet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.03145" label="2112.03145">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Models for Implicit Image Segmentation Ensembles" />
-          <attvalue for="1" value="  Diffusion models have shown impressive performance for generative modelling&#10;of images. In this paper, we present a novel semantic segmentation method based&#10;on diffusion models. By modifying the training and sampling scheme, we show&#10;that diffusion models can perform lesion segmentation of medical images. To&#10;generate an image specific segmentation, we train the model on the ground truth&#10;segmentation, and use the image as a prior during training and in every step&#10;during the sampling process. With the given stochastic sampling process, we can&#10;generate a distribution of segmentation masks. This property allows us to&#10;compute pixel-wise uncertainty maps of the segmentation, and allows an implicit&#10;ensemble of segmentations that increases the segmentation performance. We&#10;evaluate our method on the BRATS2020 dataset for brain tumor segmentation.&#10;Compared to state-of-the-art segmentation models, our approach yields good&#10;segmentation results and, additionally, detailed uncertainty maps.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.10610" label="2303.10610">
-        <attvalues>
-          <attvalue for="0" value="DiffMIC: Dual-Guidance Diffusion Network for Medical Image&#10;  Classification" />
-          <attvalue for="1" value="  Diffusion Probabilistic Models have recently shown remarkable performance in&#10;generative image modeling, attracting significant attention in the computer&#10;vision community. However, while a substantial amount of diffusion-based&#10;research has focused on generative tasks, few studies have applied diffusion&#10;models to general medical image classification. In this paper, we propose the&#10;first diffusion-based model (named DiffMIC) to address general medical image&#10;classification by eliminating unexpected noise and perturbations in medical&#10;images and robustly capturing semantic representation. To achieve this goal, we&#10;devise a dual conditional guidance strategy that conditions each diffusion step&#10;with multiple granularities to improve step-wise regional attention.&#10;Furthermore, we propose learning the mutual information in each granularity by&#10;enforcing Maximum-Mean Discrepancy regularization during the diffusion forward&#10;process. We evaluate the effectiveness of our DiffMIC on three medical&#10;classification tasks with different image modalities, including placental&#10;maturity grading on ultrasound images, skin lesion classification using&#10;dermatoscopic images, and diabetic retinopathy grading using fundus images. Our&#10;experimental results demonstrate that DiffMIC outperforms state-of-the-art&#10;methods by a significant margin, indicating the universality and effectiveness&#10;of the proposed model. Our code will be publicly available at&#10;https://github.com/scott-yjyang/DiffMIC.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.17559" label="2303.17559">
-        <attvalues>
-          <attvalue for="0" value="DDP: Diffusion Model for Dense Visual Prediction" />
-          <attvalue for="1" value="  We propose a simple, efficient, yet powerful framework for dense visual&#10;predictions based on the conditional diffusion pipeline. Our approach follows a&#10;&quot;noise-to-map&quot; generative paradigm for prediction by progressively removing&#10;noise from a random Gaussian distribution, guided by the image. The method,&#10;called DDP, efficiently extends the denoising diffusion process into the modern&#10;perception pipeline. Without task-specific design and architecture&#10;customization, DDP is easy to generalize to most dense prediction tasks, e.g.,&#10;semantic segmentation and depth estimation. In addition, DDP shows attractive&#10;properties such as dynamic inference and uncertainty awareness, in contrast to&#10;previous single-step discriminative methods. We show top results on three&#10;representative tasks with six diverse benchmarks, without tricks, DDP achieves&#10;state-of-the-art or competitive performance on each task compared to the&#10;specialist counterparts. For example, semantic segmentation (83.9 mIoU on&#10;Cityscapes), BEV map segmentation (70.6 mIoU on nuScenes), and depth estimation&#10;(0.05 REL on KITTI). We hope that our approach will serve as a solid baseline&#10;and facilitate future research&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.12729" label="2312.12729">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything Model Meets Image Harmonization" />
-          <attvalue for="1" value="  Image harmonization is a crucial technique in image composition that aims to&#10;seamlessly match the background by adjusting the foreground of composite&#10;images. Current methods adopt either global-level or pixel-level feature&#10;matching. Global-level feature matching ignores the proximity prior, treating&#10;foreground and background as separate entities. On the other hand, pixel-level&#10;feature matching loses contextual information. Therefore, it is necessary to&#10;use the information from semantic maps that describe different objects to guide&#10;harmonization. In this paper, we propose Semantic-guided Region-aware Instance&#10;Normalization (SRIN) that can utilize the semantic segmentation maps output by&#10;a pre-trained Segment Anything Model (SAM) to guide the visual consistency&#10;learning of foreground and background features. Abundant experiments&#10;demonstrate the superiority of our method for image harmonization over&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image synthesis~\cite{anydoor,DiffUTE} aims to synthesize foreground objects from one image into another image to produce realistic, visually consistent synthesized images. However, due to differences in lighting, climate, and capture devices, the foreground and background in a synthesized image may have inconsistent appearances. To address this challenge, image harmonization methods~\cite{Mat2016,RAIN,Bargainnet,MM23_HDNet,DCCF} ensure the compatibility of foreground objects with the background by adjusting their appearance. &#10;&#10;While traditional image harmonization methods~\cite{Mat2016} are mainly based on low-level color and statistical illumination&#10;information, the development of deep learning techniques has made deep image harmonization methods mainstream and achieve better results~\cite{RAIN,Bargainnet,MM23_HDNet,DCCF}. Existing methods can be roughly categorized into two types: local translation and region matching. The former employs an encoder-decoder framework to learn pixel-to-pixel translation~\cite{DIH,SSAM}. However, due to the limitation of shallow neural networks in capturing a limited amount of background, the available background reference for harmonizing the foreground is insufficient. The latter region matching methods treat foreground and background as two distinct style regions. They address the harmonization problem as a matching problem by aligning statistical features or using discriminators on these two regions. Although these methods harmonize images with finer pixel-level features, they completely disregard spatial variations within the two regions. In fact, not all pixels in the background contribute to the harmonization of the foreground, and pixel-level matching may result in the loss of contextual information.&#10;&#10;To solve this problem, we rethink the essential semantic priors in image harmonization, i.e., when we paste an object into a background, the color or lighting should be semantically related, and should be referenced based on the semantics of object. Inspired by this observation, we propose a novel semantic-guided region-aware instance normalization (SRIN) that utilizes semantic priors to align the feature distributions of the foreground and background. Specifically, to avoid training an additional segmentation network, we leverage the powerful Segment Anything Model (SAM)~\cite{SAM} to obtain semantic priors. Our method achieves state-of-the-art results on iHarmony4 datasets, which is demonstrated by extensive experiments in both quantitative and qualitative evaluations. As shown in Fig.~\ref{fig1}, the proposed framework demonstrates its efficiency and effectiveness when compared to existing image harmonization models. Notably, our method achieves higher performance while utilizing fewer parameters.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Science, Semantic Segmentation, Computer Vision, Mathematics, Artificial Intelligence, Feature Matching, Image Processing, Computer Science, Image Composition" />
-        </attvalues>
-      </node>
-      <node id="2307.09481" label="2307.09481">
-        <attvalues>
-          <attvalue for="0" value="AnyDoor: Zero-shot Object-level Image Customization" />
-          <attvalue for="1" value="  This work presents AnyDoor, a diffusion-based image generator with the power&#10;to teleport target objects to new scenes at user-specified locations in a&#10;harmonious way. Instead of tuning parameters for each object, our model is&#10;trained only once and effortlessly generalizes to diverse object-scene&#10;combinations at the inference stage. Such a challenging zero-shot setting&#10;requires an adequate characterization of a certain object. To this end, we&#10;complement the commonly used identity feature with detail features, which are&#10;carefully designed to maintain texture details yet allow versatile local&#10;variations (e.g., lighting, orientation, posture, etc.), supporting the object&#10;in favorably blending with different surroundings. We further propose to borrow&#10;knowledge from video datasets, where we can observe various forms (i.e., along&#10;the time axis) of a single object, leading to stronger model generalizability&#10;and robustness. Extensive experiments demonstrate the superiority of our&#10;approach over existing alternatives as well as its great potential in&#10;real-world applications, such as virtual try-on and object moving. Project page&#10;is https://damo-vilab.github.io/AnyDoor-Page/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.10825" label="2305.10825">
-        <attvalues>
-          <attvalue for="0" value="DiffUTE: Universal Text Editing Diffusion Model" />
-          <attvalue for="1" value="  Diffusion model based language-guided image editing has achieved great&#10;success recently. However, existing state-of-the-art diffusion models struggle&#10;with rendering correct text and text style during generation. To tackle this&#10;problem, we propose a universal self-supervised text editing diffusion model&#10;(DiffUTE), which aims to replace or modify words in the source image with&#10;another one while maintaining its realistic appearance. Specifically, we build&#10;our model on a diffusion model and carefully modify the network structure to&#10;enable the model for drawing multilingual characters with the help of glyph and&#10;position information. Moreover, we design a self-supervised learning framework&#10;to leverage large amounts of web data to improve the representation ability of&#10;the model. Experimental results show that our method achieves an impressive&#10;performance and enables controllable editing on in-the-wild images with high&#10;fidelity. Our code will be avaliable in&#10;\url{https://github.com/chenhaoxing/DiffUTE}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02853" label="2106.02853">
-        <attvalues>
-          <attvalue for="0" value="Region-aware Adaptive Instance Normalization for Image Harmonization" />
-          <attvalue for="1" value="  Image composition plays a common but important role in photo editing. To&#10;acquire photo-realistic composite images, one must adjust the appearance and&#10;visual style of the foreground to be compatible with the background. Existing&#10;deep learning methods for harmonizing composite images directly learn an image&#10;mapping network from the composite to the real one, without explicit&#10;exploration on visual style consistency between the background and the&#10;foreground images. To ensure the visual style consistency between the&#10;foreground and the background, in this paper, we treat image harmonization as a&#10;style transfer problem. In particular, we propose a simple yet effective&#10;Region-aware Adaptive Instance Normalization (RAIN) module, which explicitly&#10;formulates the visual style from the background and adaptively applies them to&#10;the foreground. With our settings, our RAIN module can be used as a drop-in&#10;module for existing image harmonization networks and is able to bring&#10;significant improvements. Extensive experiments on the existing image&#10;harmonization benchmark datasets show the superior capability of the proposed&#10;method. Code is available at {https://github.com/junleen/RainNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.09169" label="2009.09169">
-        <attvalues>
-          <attvalue for="0" value="BargainNet: Background-Guided Domain Translation for Image Harmonization" />
-          <attvalue for="1" value="  Image composition is a fundamental operation in image editing field. However,&#10;unharmonious foreground and background downgrade the quality of composite&#10;image. Image harmonization, which adjusts the foreground to improve the&#10;consistency, is an essential yet challenging task. Previous deep learning based&#10;methods mainly focus on directly learning the mapping from composite image to&#10;real image, while ignoring the crucial guidance role that background plays. In&#10;this work, with the assumption that the foreground needs to be translated to&#10;the same domain as background, we formulate image harmonization task as&#10;background-guided domain translation. Therefore, we propose an image&#10;harmonization network with a novel domain code extractor and well-tailored&#10;triplet losses, which could capture the background domain information to guide&#10;the foreground harmonization. Extensive experiments on the existing image&#10;harmonization benchmark demonstrate the effectiveness of our proposed method.&#10;Code is available at https://github.com/bcmi/BargainNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.04788" label="2207.04788">
-        <attvalues>
-          <attvalue for="0" value="DCCF: Deep Comprehensible Color Filter Learning Framework for&#10;  High-Resolution Image Harmonization" />
-          <attvalue for="1" value="  Image color harmonization algorithm aims to automatically match the color&#10;distribution of foreground and background images captured in different&#10;conditions. Previous deep learning based models neglect two issues that are&#10;critical for practical applications, namely high resolution (HR) image&#10;processing and model comprehensibility. In this paper, we propose a novel Deep&#10;Comprehensible Color Filter (DCCF) learning framework for high-resolution image&#10;harmonization. Specifically, DCCF first downsamples the original input image to&#10;its low-resolution (LR) counter-part, then learns four human comprehensible&#10;neural filters (i.e. hue, saturation, value and attentive rendering filters) in&#10;an end-to-end manner, finally applies these filters to the original input image&#10;to get the harmonized result. Benefiting from the comprehensible neural&#10;filters, we could provide a simple yet efficient handler for users to cooperate&#10;with deep model to get the desired results with very little effort when&#10;necessary. Extensive experiments demonstrate the effectiveness of DCCF learning&#10;framework and it outperforms state-of-the-art post-processing method on&#10;iHarmony4 dataset on images' full-resolutions by achieving 7.63% and 1.69%&#10;relative improvements on MSE and PSNR respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.00069" label="1703.00069">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Harmonization" />
-          <attvalue for="1" value="  Compositing is one of the most common operations in photo editing. To&#10;generate realistic composites, the appearances of foreground and background&#10;need to be adjusted to make them compatible. Previous approaches to harmonize&#10;composites have focused on learning statistical relationships between&#10;hand-crafted appearance features of the foreground and background, which is&#10;unreliable especially when the contents in the two layers are vastly different.&#10;In this work, we propose an end-to-end deep convolutional neural network for&#10;image harmonization, which can capture both the context and semantic&#10;information of the composite images during harmonization. We also introduce an&#10;efficient way to collect large-scale and high-quality training data that can&#10;facilitate the training process. Experiments on the synthesized dataset and&#10;real composite images show that the proposed network outperforms previous&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.06406" label="1907.06406">
-        <attvalues>
-          <attvalue for="0" value="Improving the Harmony of the Composite Image by Spatial-Separated&#10;  Attention Module" />
-          <attvalue for="1" value="  Image composition is one of the most important applications in image&#10;processing. However, the inharmonious appearance between the spliced region and&#10;background degrade the quality of the image. Thus, we address the problem of&#10;Image Harmonization: Given a spliced image and the mask of the spliced region,&#10;we try to harmonize the &quot;style&quot; of the pasted region with the background&#10;(non-spliced region). Previous approaches have been focusing on learning&#10;directly by the neural network. In this work, we start from an empirical&#10;observation: the differences can only be found in the spliced region between&#10;the spliced image and the harmonized result while they share the same semantic&#10;information and the appearance in the non-spliced region. Thus, in order to&#10;learn the feature map in the masked region and the others individually, we&#10;propose a novel attention module named Spatial-Separated Attention Module&#10;(S2AM). Furthermore, we design a novel image harmonization framework by&#10;inserting the S2AM in the coarser low-level features of the Unet structure in&#10;two different ways. Besides image harmonization, we make a big step for&#10;harmonizing the composite image without the specific mask under previous&#10;observation. The experiments show that the proposed S2AM performs better than&#10;other state-of-the-art attention modules in our task. Moreover, we demonstrate&#10;the advantages of our model against other state-of-the-art image harmonization&#10;methods via criteria from multiple points of view. Code is available at&#10;https://github.com/vinthony/s2am&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;We do a brief review of the recent development of image harmonization and image-to-image transformation tasks. Importantly, we review the recently proposed attention mechanisms for the comparison of the proposed S$^2$AM.&#10;&#10;Image Harmonization&#10;Recently, because of the efficient and impressive results, the neural network based methods have drawn much attention in image editing. However, in image harmonization field, there are still limited research works to solve this problem. Zhu et al.~\cite{Zhu:2015tl} propose a fine-tuned model for predicting the realism of the composite images. They also fix the parameters of the model and adjust the color of the masked region of the input image by optimizing the predicted realism score. It can also be regarded as an image harmonization task. \cite{Tsai:2016id} proposes a method for exchange the sky of images while this method only works for the sky. An end-to-end learning based approach of image harmonization has been proposed by \cite{Tsai:2017kv}, they trained an end-to-end neural network with the ground truth image and segmentation mask. However, their method just learns everything from the power of the neural network. Another related work is GP-GAN~\cite{Wu:2017wq}. It utilizes the generator in GAN to synthesize the low-resolution images, and then, the high-resolution output is synthesized by the input image and the low-resolution outputs using traditional Gaussian-Poisson Equation. Their method focuses more on blending the edge of the two different images rather than the style transform in the images. Thus, this method also changes the appearance of the background while the image harmonization only focus on the spliced region. More recently, video harmonization methods have been proposed by \cite{Huang:2018ti} and \cite{Lee:2019th}. Their methods harmonize the video and insert the video to another video using spatial and temporal information respectively.&#10;&#10;Attention Mechanisms &#10;Attention mechanisms draw the attention of researchers because these modules allocate the most informative components according to analysis the features themselves. Squeeze-and-Excitation Network~\cite{Woo:2018wr} insert a channel attention module in the network to make the network itself more focus on the essential features by weighting the channels with global feature analysis and Convolutional Block Attention Module~\cite{Hu:2017tf} extend this method by adding a spatial attention module for weighting the features in spatial space. For choosing the feature kernel with different size, Selective Kernel Networks~\cite{Li:2019vy} is proposed. As for learning the specific region in spatial space, \cite{Liu:2018ui} propose Partial Convolution for focusing on the masked region and \cite{Yu:2018uw} design Contextual Attention to borrow the background information to hole filling. \cite{Yu:2018hf} use Gated Convolution, which can also be regarded as the spatial attention mechanism for each convolution block, to update the parameters of the network and mask~(or attention map ) softly. However, these methods are designed for image in-painting task specifically.&#10;Another noticeable direction of attention mechanism is self-attention. Self-Attention aims to capture the long-range dependencies in the image context. Non-local Network~\cite{Wang:2017ut} extend the self-attention~\cite{Vaswani:2017ul} as a form of non-local means~\cite{Buades:2005vp} to capture long-range dependencies for video classification and \cite{Chen:2018tu} extends this method in GAN-based structure for generating divergence samples. Most recently, \cite{Cao:2019wb} model the Non-Local Network and Squeeze-and-Excitation Network in a new form called Global Content Network for various computer vision tasks. However, these attention mechanisms are focus on learning the representation on the high level computer vision task other than digital image processing with low-level features change.&#10;&#10;Image-to-Image Transformation &#10;Image harmonization can also be regarded as an image-to-image transformation task which focuses on the changes the appearance between the input and target. Chen et al.~\cite{Chen:2017ta} propose a neural network for fast image processing with the Dilated Convolution~\cite{Yu:2016uc} to capture a larger reception field without reducing the feature map. By the excellent performance of guided filter~\cite{He:2013je} in edge preserving, \cite{Gharbi:2017ks} and \cite{Chen:2016ck} model this filter by the neural network. Taking the benefit of \cite{Chen:2017ta} and \cite{He:2013je}, \cite{Wu:2018uk} propose the deep guided filter for fast image processing. When the content of the image changes significantly, pix2pix~\cite{Isola:2016tp} train the network for alignment datasets by GAN~\cite{Goodfellow:2014wp} while CycleGAN~\cite{Zhu:2017ua} is proposed for unsupervised learning. By combining the image-to-image transformation task with the attention network, Another noticeable direction of unsupervised image-to-image transformation is attention-based methods. \cite{Chen:2018wl, Mejjati:2018wu} extend CycleGAN with the attention network as an additional block and learning the foreground specifically. These methods generate a global attention mask for image-to-image transformation for specific objects only.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2312.12080" label="2312.12080">
-        <attvalues>
-          <attvalue for="0" value="Learning Subject-Aware Cropping by Outpainting Professional Photos" />
-          <attvalue for="1" value="  How to frame (or crop) a photo often depends on the image subject and its&#10;context; e.g., a human portrait. Recent works have defined the subject-aware&#10;image cropping task as a nuanced and practical version of image cropping. We&#10;propose a weakly-supervised approach (GenCrop) to learn what makes a&#10;high-quality, subject-aware crop from professional stock images. Unlike&#10;supervised prior work, GenCrop requires no new manual annotations beyond the&#10;existing stock image collection. The key challenge in learning from this data,&#10;however, is that the images are already cropped and we do not know what regions&#10;were removed. Our insight is to combine a library of stock images with a&#10;modern, pre-trained text-to-image diffusion model. The stock image collection&#10;provides diversity and its images serve as pseudo-labels for a good crop, while&#10;the text-image diffusion model is used to out-paint (i.e., outward inpainting)&#10;realistic uncropped images. Using this procedure, we are able to automatically&#10;generate a large dataset of cropped-uncropped training pairs to train a&#10;cropping model. Despite being weakly-supervised, GenCrop is competitive with&#10;state-of-the-art supervised methods and significantly better than comparable&#10;weakly-supervised baselines on quantitative and qualitative evaluation metrics.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Framing a photo is compositional skill that professional photographers hone with years of experience, and cropping is a key way to adjust framing or experiment with alternative compositions after capture.&#10;Framing and cropping decide what elements of a scene to include or exclude from the image, and how to frame or crop an image is influenced by the subject that one wishes to portray.&#10;Subject-aware cropping takes a notion of a subject in addition to pixels and has been studied in recent work on data-driven approaches~\cite{sacd} and in the context of human-centric images~\cite{hcic}.&#10;High-quality solutions to this problem are based on supervised learning, from large datasets of manual annotations created specifically for cropping~\cite{gaic,cpc,sacd}.&#10;&#10;We explore an alternative approach to the subject-aware cropping problem that is only weakly-supervised.&#10;We observe that millions of professional images are easily accessible online in the form of stock image collections and that these collections cover a wide range of subject-matter that people want to capture --- e.g., portraits of people.&#10;We then ask, {to crop better portraits, can one seek out a relevant set of professional portraits from these collections and teach a model to replicate that distribution?}&#10;The key challenge is that, although every professional image provides an expert label (i.e., a good crop), the original uncropped photo is unknown and cannot be recovered from the crop.&#10;Unlike typical weakly-supervised computer vision tasks where input images are plentiful but labels are scarce and/or unreliable, in our setting this assumption is reversed.&#10;&#10;Our proposed method, {GenCrop}, addresses this challenge by combining a readily available dataset of stock images with powerful, pre-trained image generation models to synthesize the required inputs.&#10;Specifically, we use text-to-image diffusion to ``out-paint'' (i.e., outward pixel generate or outward inpaint) stock images and generate plausible uncropped-and-cropped pairs (Fig.~\ref{fig:teaser}).&#10;By scaling this automatic process, we can generate a large and diverse set of images to train our subject-aware cropping model.&#10;The key advantage of {GenCrop} is that it is weakly-supervised, requiring no new manual crop or scoring annotations beyond access to the original professional image collection.&#10;&#10;To demonstrate the effectiveness of {GenCrop}, we evaluate it on the human-centric (portrait) and the subject-aware cropping tasks proposed by~\cite{hcic} and~\cite{sacd}.&#10;We show that {GenCrop} yields competitive results against fully supervised approaches~\cite{hcic} on the existing datasets~\cite{flms,fcdb,sacd} (under quantitative metrics such as Intersection-over-Union and boundary displacement~\cite{hcic}), while being superior to the best weakly/unsupervised method~\cite{vfn}.&#10;We also evaluate {GenCrop} on additional subject categories such as cats, dogs, etc. to test the generalization of our approach beyond just humans.&#10;On qualitative evaluation, {GenCrop} is comparable to or better than supervised prior work on the rate of cropping errors, while prior weakly-supervised/unsupervised baselines fall substantially short.&#10;&#10;Lastly, we conduct additional analysis and ablations to assess the effectiveness and limitations of learning to crop from our generated data.&#10;The code for our data generation pipeline and cropping models is publicly available.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science, Image Cropping, Weakly Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1904.04441" label="1904.04441">
-        <attvalues>
-          <attvalue for="0" value="Reliable and Efficient Image Cropping: A Grid Anchor based Approach" />
-          <attvalue for="1" value="  Image cropping aims to improve the composition as well as aesthetic quality&#10;of an image by removing extraneous content from it. Existing image cropping&#10;databases provide only one or several human-annotated bounding boxes as the&#10;groundtruth, which cannot reflect the non-uniqueness and flexibility of image&#10;cropping in practice. The employed evaluation metrics such as&#10;intersection-over-union cannot reliably reflect the real performance of&#10;cropping models, either. This work revisits the problem of image cropping, and&#10;presents a grid anchor based formulation by considering the special properties&#10;and requirements (e.g., local redundancy, content preservation, aspect ratio)&#10;of image cropping. Our formulation reduces the searching space of candidate&#10;crops from millions to less than one hundred. Consequently, a grid anchor based&#10;cropping benchmark is constructed, where all crops of each image are annotated&#10;and more reliable evaluation metrics are defined. We also design an effective&#10;and lightweight network module, which simultaneously considers the region of&#10;interest and region of discard for more accurate image cropping. Our model can&#10;stably output visually pleasing crops for images of different scenes and run at&#10;a speed of 125 FPS. Code and dataset are available at:&#10;https://github.com/HuiZeng/Grid-Anchor-based-Image-Cropping.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.01480" label="1701.01480">
-        <attvalues>
-          <attvalue for="0" value="Quantitative Analysis of Automatic Image Cropping Algorithms: A Dataset&#10;  and Comparative Study" />
-          <attvalue for="1" value="  Automatic photo cropping is an important tool for improving visual quality of&#10;digital photos without resorting to tedious manual selection. Traditionally,&#10;photo cropping is accomplished by determining the best proposal window through&#10;visual quality assessment or saliency detection. In essence, the performance of&#10;an image cropper highly depends on the ability to correctly rank a number of&#10;visually similar proposal windows. Despite the ranking nature of automatic&#10;photo cropping, little attention has been paid to learning-to-rank algorithms&#10;in tackling such a problem. In this work, we conduct an extensive study on&#10;traditional approaches as well as ranking-based croppers trained on various&#10;image features. In addition, a new dataset consisting of high quality cropping&#10;and pairwise ranking annotations is presented to evaluate the performance of&#10;various baselines. The experimental results on the new dataset provide useful&#10;insights into the design of better photo cropping algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.00503" label="1702.00503">
-        <attvalues>
-          <attvalue for="0" value="Learning to Compose with Professional Photographs on the Web" />
-          <attvalue for="1" value="  Photo composition is an important factor affecting the aesthetics in&#10;photography. However, it is a highly challenging task to model the aesthetic&#10;properties of good compositions due to the lack of globally applicable rules to&#10;the wide variety of photographic styles. Inspired by the thinking process of&#10;photo taking, we formulate the photo composition problem as a view finding&#10;process which successively examines pairs of views and determines their&#10;aesthetic preferences. We further exploit the rich professional photographs on&#10;the web to mine unlimited high-quality ranking samples and demonstrate that an&#10;aesthetics-aware deep ranking network can be trained without explicitly&#10;modeling any photographic rules. The resulting model is simple and effective in&#10;terms of its architectural design and data sampling method. It is also generic&#10;since it naturally learns any photographic rules implicitly encoded in&#10;professional photographs. The experiments show that the proposed view finding&#10;network achieves state-of-the-art performance with sliding window search&#10;strategy on two image cropping datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.06957" label="2003.06957">
-        <attvalues>
-          <attvalue for="0" value="Frustratingly Simple Few-Shot Object Detection" />
-          <attvalue for="1" value="  Detecting rare objects from a few examples is an emerging problem. Prior&#10;works show meta-learning is a promising approach. But, fine-tuning techniques&#10;have drawn scant attention. We find that fine-tuning only the last layer of&#10;existing detectors on rare classes is crucial to the few-shot object detection&#10;task. Such a simple approach outperforms the meta-learning methods by roughly&#10;2~20 points on current benchmarks and sometimes even doubles the accuracy of&#10;the prior methods. However, the high variance in the few samples often leads to&#10;the unreliability of existing benchmarks. We revise the evaluation protocols by&#10;sampling multiple groups of training examples to obtain stable comparisons and&#10;build new benchmarks based on three datasets: PASCAL VOC, COCO and LVIS. Again,&#10;our fine-tuning approach establishes a new state of the art on the revised&#10;benchmarks. The code as well as the pretrained models are available at&#10;https://github.com/ucbdrive/few-shot-object-detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.09384" label="2007.09384">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale Positive Sample Refinement for Few-Shot Object Detection" />
-          <attvalue for="1" value="  Few-shot object detection (FSOD) helps detectors adapt to unseen classes with&#10;few training instances, and is useful when manual annotation is time-consuming&#10;or data acquisition is limited. Unlike previous attempts that exploit few-shot&#10;classification techniques to facilitate FSOD, this work highlights the&#10;necessity of handling the problem of scale variations, which is challenging due&#10;to the unique sample distribution. To this end, we propose a Multi-scale&#10;Positive Sample Refinement (MPSR) approach to enrich object scales in FSOD. It&#10;generates multi-scale positive samples as object pyramids and refines the&#10;prediction at various scales. We demonstrate its advantage by integrating it as&#10;an auxiliary branch to the popular architecture of Faster R-CNN with FPN,&#10;delivering a strong FSOD solution. Several experiments are conducted on PASCAL&#10;VOC and MS COCO, and the proposed approach achieves state of the art results&#10;and significantly outperforms other counterparts, which shows its&#10;effectiveness. Code is available at https://github.com/jiaxi-wu/MPSR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.04550" label="2106.04550">
-        <attvalues>
-          <attvalue for="0" value="DETReg: Unsupervised Pretraining with Region Priors for Object Detection" />
-          <attvalue for="1" value="  Recent self-supervised pretraining methods for object detection largely focus&#10;on pretraining the backbone of the object detector, neglecting key parts of&#10;detection architecture. Instead, we introduce DETReg, a new self-supervised&#10;method that pretrains the entire object detection network, including the object&#10;localization and embedding components. During pretraining, DETReg predicts&#10;object localizations to match the localizations from an unsupervised region&#10;proposal generator and simultaneously aligns the corresponding feature&#10;embeddings with embeddings from a self-supervised image encoder. We implement&#10;DETReg using the DETR family of detectors and show that it improves over&#10;competitive baselines when finetuned on COCO, PASCAL VOC, and Airbus Ship&#10;benchmarks. In low-data regimes DETReg achieves improved performance, e.g.,&#10;when training with only 1% of the labels and in the few-shot learning settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Self-supervised pretraining. Recent work~\cite{goyal2019scaling, chen2020simple, henaff2021efficient,he2019momentum,he2020momentum,caron2020unsupervised,chen2020improved,misra2020self,gidaris2020learning} has shown that self-supervised pretraining can generate powerful representations for transfer learning, even outperforming its supervised counterparts on challenging vision benchmarks~\cite{wu2018unsupervised, chen2020simple}. &#10;The learned representations transfer well to image classification but the improvement is less significant for instance-level tasks, such as object detection and instance segmentation~\cite{he2019rethinking,zhou2020cheaper,he2020momentum, reed2021self}.&#10;&#10;More recently, a number of works~\cite{roh2021spatially,henaff2021efficient, xiao2021region,xie2021detco} focused on learning backbones that can transfer to object detection. In contrast to these works, we pretrain the entire detection network. As we show, pretraining the backbone with an image-patch-based task does not necessarily empower the model to learn what and where an object is, and adding weak supervision from the region priors proves beneficial.&#10;&#10;Our approach is also different from semi-supervised object detection approaches~\cite{jeong2019consistency,sohn2020simple,liu2021unbiased,xu2021end} and few-shot detection approaches~\cite{chen2018lstd,karlinsky2019repmet,kang2019few,yan2019meta,wu2020meta,wu2020multi,fan2020few,xiao2020few,wang2020frustratingly,li2021transformation,fan2021generalized,chen2021should,zhang2021meta} as we initialize the detector from a pretrained DETReg model without further modifying the architecture. Therefore, these approaches can be viewed as complementary to DETReg.&#10;&#10;End-to-end object detection. Detection with transformers (DETR)~\cite{carion2020end} builds&#10;the first fully end-to-end object detector and eliminates the need for components such as anchor generation and&#10;non-maximum suppression (NMS) post-processing.&#10;This model has quickly gained traction in the machine vision community.&#10;However, the original DETR suffers from slow&#10;convergence and limited sample efficiency. Deformable~DETR~\cite{zhu2020deformable} introduced a deformable attention module to attend to a sparsely sampled small set of prominent key elements, and achieved better performance compared to DETR with reduced training epochs. We therefore use Deformable~DETR as our base detection architecture. &#10;&#10;Both DETR and Deformable~DETR adopt &#10;the supervised pretrained backbone (ResNet~\cite{he2016deep}) on ImageNet. UP-DETR~\cite{dai2020up} pretrains DETR in a self-supervised way by detecting and reconstructing the random patches from the input image. &#10;Instead, we additionally adopt region priors from unsupervised region proposal algorithms to provide weak supervision for pretraining, which has an explicit notion of objects rather than the random patches used by UP-DETR.&#10;&#10;Region proposals. A rich study&#10;of region proposals methods exists in the object detection literature~\cite{alexe2010object,van2011segmentation,carreira2011cpmc,endres2013category, arbelaez2014multiscale,zitnick2014edge,cheng2014bing,krahenbuhl2014geodesic}.&#10;Grouping based method, Selective Search~\cite{van2011segmentation}, and window scoring based approach, Objectness~\cite{alexe2010object} are two early and well known proposal methods, which have been widely adopted and supported in major software libraries (e.g., OpenCV~\cite{opencv_library}).&#10;Selective Search greedily merges superpixels to generate proposals. Objectness relies on visual cues such as multi-scale saliency, color contrast, edge density and superpixel straddling to identify likely regions. &#10;&#10;While the field has largely shifted to learning-based approaches, the key benefit of these models is that they require little or no training data, and can produce region proposals at a high recall~\cite{uijlings2013selective, arbelaez2014multiscale, arbelaez2014multiscale, cheng2014bing, cheng2014global}. This provides a cheap, albeit noisy, source of supervision. Hosang~\etal~\cite{hosang2014good,hosang2015makes} offer a comprehensive analysis over the various region proposals methods, and Selective Search is among the top performing approaches in terms of recall. Here, we seek weak supervision from the region proposals generated by Selective Search, which has been widely adopted and proven successful in the well-known detectors such as R-CNN~\cite{girshick2014rich} and Fast R-CNN~\cite{girshick2015fast}. However, our approach is not limited to Selective Search and can employ other proposal methods. &#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2106.02637" label="2106.02637">
-        <attvalues>
-          <attvalue for="0" value="Aligning Pretraining for Detection via Object-Level Contrastive Learning" />
-          <attvalue for="1" value="  Image-level contrastive representation learning has proven to be highly&#10;effective as a generic model for transfer learning. Such generality for&#10;transfer learning, however, sacrifices specificity if we are interested in a&#10;certain downstream task. We argue that this could be sub-optimal and thus&#10;advocate a design principle which encourages alignment between the&#10;self-supervised pretext task and the downstream task. In this paper, we follow&#10;this principle with a pretraining method specifically designed for the task of&#10;object detection. We attain alignment in the following three aspects: 1)&#10;object-level representations are introduced via selective search bounding boxes&#10;as object proposals; 2) the pretraining network architecture incorporates the&#10;same dedicated modules used in the detection pipeline (e.g. FPN); 3) the&#10;pretraining is equipped with object detection properties such as object-level&#10;translation invariance and scale invariance. Our method, called Selective&#10;Object COntrastive learning (SoCo), achieves state-of-the-art results for&#10;transfer performance on COCO detection using a Mask R-CNN framework. Code is&#10;available at https://github.com/hologerry/SoCo.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Unsupervised feature learning has a long history in training deep neural networks.&#10;Auto-encoders~\cite{vincent2010stacked} and Deep Boltzmann Machines~\cite{salakhutdinov2009deep} learn layers of representations by reconstructing image pixels.&#10;Unsupervised pretraining is often used as a method for initialization, which is followed by supervised training on the same dataset such as handwritten digits for classification.&#10;&#10;Recent advances in unsupervised learning, especially self-supervised learning, tend to formulate the problem in a transfer learning scenario, where pretraining and finetuning are trained on different datasets with different purposes.&#10;Pretext tasks such as colorization~\cite{larsson2017colorization}, context prediction~\cite{doersch2015unsupervised}, inpainting~\cite{pathak2016context} and rotation prediction~\cite{gidaris2018unsupervised} force the network to learn semantic information in order to solve the pretext tasks.&#10;Contrastive methods based on the instance discrimination pretext task~\cite{wu2018unsupervised} learn to map augmented views of the same instance to similar embeddings.&#10;Such approaches~\cite{henaff2020data,he2020momentum,chen2020improved,chen2020simple,zbontar2021barlow,caron2021emerging} have shown strong transfer ability for a number of downstream tasks, sometimes even outperforming the supervised counterpart by large margins~\cite{he2020momentum}.&#10;&#10;With new technical innovations such as learning clustering assignments~\cite{caron2020unsupervised}, large-scale contrastive learning has been successfully applied on non-curated data sets~\cite{caron2019unsupervised,goyal2021self}. &#10;While the linear evaluation result on ImageNet classification has improved significantly~\cite{grill2020bootstrap}, the progress on transfer performance for dense prediction tasks has been limited.&#10;Due to this, a growing number of works investigate pretraining specifically for object detection and semantic segmentation.&#10;The idea is to shift image-level representations to pixel-level or region-level representations.&#10;VADer~\cite{pinheiro2020unsupervised}, PixPro~\cite{xie2020propagate} and DenseCL~\cite{wang2020dense} propose to learn pixel-level representations by matching point features of the same physical location under different views.&#10;InsLoc~\cite{yang2021instance} learns to match region-level features from composited imagery.&#10;DetCon~\cite{henaff2021efficient} additionally uses the bottom-up segmentation of MCG~\cite{arbelaez2014multiscale} for supervising intra-segment pixel variations. UP-DETR~\cite{dai2021up} proposes a pretext task named random query patch detection to pretrain DETR detector. Self-EMD~\cite{liu2020self} explores self-supervised representation learning for object detection without using Imagenet images.&#10;&#10;Our work is also motivated and inspired by object detection methods which advocate architectures and training schemes invariant to translation and scale transformations~\cite{lin2017feature,singh2018analysis}. We find that incorporating them in the pretraining stage also benefits object detection, which has largely been overlooked in previous self-supervised learning works.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2102.08318" label="2102.08318">
-        <attvalues>
-          <attvalue for="0" value="Instance Localization for Self-supervised Detection Pretraining" />
-          <attvalue for="1" value="  Prior research on self-supervised learning has led to considerable progress&#10;on image classification, but often with degraded transfer performance on object&#10;detection. The objective of this paper is to advance self-supervised pretrained&#10;models specifically for object detection. Based on the inherent difference&#10;between classification and detection, we propose a new self-supervised pretext&#10;task, called instance localization. Image instances are pasted at various&#10;locations and scales onto background images. The pretext task is to predict the&#10;instance category given the composited images as well as the foreground&#10;bounding boxes. We show that integration of bounding boxes into pretraining&#10;promotes better task alignment and architecture alignment for transfer&#10;learning. In addition, we propose an augmentation method on the bounding boxes&#10;to further enhance the feature alignment. As a result, our model becomes weaker&#10;at Imagenet semantic classification but stronger at image patch localization,&#10;with an overall stronger pretrained model for object detection. Experimental&#10;results demonstrate that our approach yields state-of-the-art transfer learning&#10;results for object detection on PASCAL VOC and MSCOCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.09785" label="2106.09785">
-        <attvalues>
-          <attvalue for="0" value="Efficient Self-supervised Vision Transformers for Representation&#10;  Learning" />
-          <attvalue for="1" value="  This paper investigates two techniques for developing efficient&#10;self-supervised vision transformers (EsViT) for visual representation learning.&#10;First, we show through a comprehensive empirical study that multi-stage&#10;architectures with sparse self-attentions can significantly reduce modeling&#10;complexity but with a cost of losing the ability to capture fine-grained&#10;correspondences between image regions. Second, we propose a new pre-training&#10;task of region matching which allows the model to capture fine-grained region&#10;dependencies and as a result significantly improves the quality of the learned&#10;vision representations. Our results show that combining the two techniques,&#10;EsViT achieves 81.3% top-1 on the ImageNet linear probe evaluation,&#10;outperforming prior arts with around an order magnitude of higher throughput.&#10;When transferring to downstream linear classification tasks, EsViT outperforms&#10;its supervised counterpart on 17 out of 18 datasets. The code and models are&#10;publicly available: https://github.com/microsoft/esvit&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.05499" label="2011.05499">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Dense Visual Representations" />
-          <attvalue for="1" value="  Contrastive self-supervised learning has emerged as a promising approach to&#10;unsupervised visual representation learning. In general, these methods learn&#10;global (image-level) representations that are invariant to different views&#10;(i.e., compositions of data augmentation) of the same image. However, many&#10;visual understanding tasks require dense (pixel-level) representations. In this&#10;paper, we propose View-Agnostic Dense Representation (VADeR) for unsupervised&#10;learning of dense representations. VADeR learns pixelwise representations by&#10;forcing local features to remain constant over different viewing conditions.&#10;Specifically, this is achieved through pixel-level contrastive learning:&#10;matching features (that is, features that describes the same location of the&#10;scene on different views) should be close in an embedding space, while&#10;non-matching features should be apart. VADeR provides a natural representation&#10;for dense prediction tasks and transfers well to downstream tasks. Our method&#10;outperforms ImageNet supervised pretraining (and strong unsupervised baselines)&#10;in multiple dense prediction tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.09157" label="2011.09157">
-        <attvalues>
-          <attvalue for="0" value="Dense Contrastive Learning for Self-Supervised Visual Pre-Training" />
-          <attvalue for="1" value="  To date, most existing self-supervised learning methods are designed and&#10;optimized for image classification. These pre-trained models can be sub-optimal&#10;for dense prediction tasks due to the discrepancy between image-level&#10;prediction and pixel-level prediction. To fill this gap, we aim to design an&#10;effective, dense self-supervised learning method that directly works at the&#10;level of pixels (or local features) by taking into account the correspondence&#10;between local features. We present dense contrastive learning, which implements&#10;self-supervised learning by optimizing a pairwise contrastive (dis)similarity&#10;loss at the pixel level between two views of input images. Compared to the&#10;baseline method MoCo-v2, our method introduces negligible computation overhead&#10;(only &lt;1% slower), but demonstrates consistently superior performance when&#10;transferring to downstream dense prediction tasks including object detection,&#10;semantic segmentation and instance segmentation; and outperforms the&#10;state-of-the-art methods by a large margin. Specifically, over the strong&#10;MoCo-v2 baseline, our method achieves significant improvements of 2.0% AP on&#10;PASCAL VOC object detection, 1.1% AP on COCO object detection, 0.9% AP on COCO&#10;instance segmentation, 3.0% mIoU on PASCAL VOC semantic segmentation and 1.8%&#10;mIoU on Cityscapes semantic segmentation. Code is available at:&#10;https://git.io/AdelaiDet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.09094" label="2011.09094">
-        <attvalues>
-          <attvalue for="0" value="UP-DETR: Unsupervised Pre-training for Object Detection with&#10;  Transformers" />
-          <attvalue for="1" value="  DEtection TRansformer (DETR) for object detection reaches competitive&#10;performance compared with Faster R-CNN via a transformer encoder-decoder&#10;architecture. However, trained with scratch transformers, DETR needs&#10;large-scale training data and an extreme long training schedule even on COCO&#10;dataset. Inspired by the great success of pre-training transformers in natural&#10;language processing, we propose a novel pretext task named random query patch&#10;detection in Unsupervised Pre-training DETR (UP-DETR). Specifically, we&#10;randomly crop patches from the given image and then feed them as queries to the&#10;decoder. The model is pre-trained to detect these query patches from the input&#10;image. During the pre-training, we address two critical issues: multi-task&#10;learning and multi-query localization. (1) To trade off classification and&#10;localization preferences in the pretext task, we find that freezing the CNN&#10;backbone is the prerequisite for the success of pre-training transformers. (2)&#10;To perform multi-query localization, we develop UP-DETR with multi-query patch&#10;detection with attention mask. Besides, UP-DETR also provides a unified&#10;perspective for fine-tuning object detection and one-shot detection tasks. In&#10;our experiments, UP-DETR significantly boosts the performance of DETR with&#10;faster convergence and higher average precision on object detection, one-shot&#10;detection and panoptic segmentation. Code and pre-training models:&#10;https://github.com/dddzg/up-detr.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.01300" label="2403.01300">
-        <attvalues>
-          <attvalue for="0" value="Causal Mode Multiplexer: A Novel Framework for Unbiased Multispectral&#10;  Pedestrian Detection" />
-          <attvalue for="1" value="  RGBT multispectral pedestrian detection has emerged as a promising solution&#10;for safety-critical applications that require day/night operations. However,&#10;the modality bias problem remains unsolved as multispectral pedestrian&#10;detectors learn the statistical bias in datasets. Specifically, datasets in&#10;multispectral pedestrian detection mainly distribute between ROTO (day) and&#10;RXTO (night) data; the majority of the pedestrian labels statistically co-occur&#10;with their thermal features. As a result, multispectral pedestrian detectors&#10;show poor generalization ability on examples beyond this statistical&#10;correlation, such as ROTX data. To address this problem, we propose a novel&#10;Causal Mode Multiplexer (CMM) framework that effectively learns the causalities&#10;between multispectral inputs and predictions. Moreover, we construct a new&#10;dataset (ROTX-MP) to evaluate modality bias in multispectral pedestrian&#10;detection. ROTX-MP mainly includes ROTX examples not presented in previous&#10;datasets. Extensive experiments demonstrate that our proposed CMM framework&#10;generalizes well on existing datasets (KAIST, CVC-14, FLIR) and the new&#10;ROTX-MP. We will release our new dataset to the public for future research.&#10;" />
-          <attvalue for="2" value="&#10; Multispectral pedestrian detection plays a critical role in many real-world applications that require both day/night operations, such as smart surveillance cameras (CCTVs), search and rescue (SAR) autopilots, and autonomous vehicles (AVs)~\cite{kim2022defending, hwang2015multispectral,choi2018kaist, c:25,gonzalez2016pedestrian, 9419080, cao2021handcrafted, qingyun2022crossmodality, zhou2020improving, kim2022map, zhang2019weakly,kim2024mscotdet}. Despite its notable progress, an overlooked factor in multispectral pedestrian detection is the modality bias problem that occurs in multimodal models (e.g., Visual Question Answering) ~\cite{niu2021counterfactual, cadene2019rubi,zhang2023layout, kv2020reducing, wen2021debiased}. Multimodal models are known to often leverage spurious correlations between a certain modality and answers due to statistical biases in datasets~\cite{yang2021causal, zang2023discovering, nan2021interventional, liu2022show, huang2022deconfounded}. For example, a Visual Question Answering (VQA) model may blindly answer “tennis” for the question “What sports?” just referring to the most co-occurring textual QA pairs (i.e., linguistic bias) in the train data~\cite{ramakrishnan2018overcoming, niu2021counterfactual}. Models exploiting statistical bias in datasets often demonstrate poor generalization ability to out-of-distribution data, rarely providing proper multimodal evidence for prediction~\cite{wu2022characterizing, gat2020removing}. \\&#10;&#10;As modality biases can significantly impact the reliability of multimodal models, it is essential to investigate statistical bias in multispectral pedestrian datasets~\cite{hwang2015multispectral, gonzalez2016pedestrian, c:25, jia2021llvip}. Analyzing their distribution, we find an ever-overlooked statistical bias. Illustrated in Fig.\ref{fig:1} (a), most data are distributed between ROTO (daytime images) or RXTO (nighttime images). In these data, pedestrian labels always statistically co-occur with their thermal features, as thermal sensors can generally capture robust pedestrian silhouettes all day/night. On the other hand, there are rarely ROTX images that can be captured in thermal obscured situations in the daytime e.g., firefighters wearing heat-insulation cloth as in Fig.\ref{fig:1} (c). From this data distribution, we hypothesize that models may learn the statistical co-occurrence between the pedestrian label and the presence of thermal features. If so, there is a potential risk that multispectral pedestrian detectors may fail in scenes in which such statistical correlation does not hold, e.g., ROTX images in which thermal features are obscured.\\&#10; In this paper, we test multispectral pedestrian detection models under ROTX data and validate our hypothesis. Since existing datasets~\cite{hwang2015multispectral, gonzalez2016pedestrian, c:25, jia2021llvip} do not include ROTX data, we first collect ROTX pedestrian images based on practical scenarios. This includes search and rescue (SAR) on firefighters and surveillance of pedestrians over a window. In this situation, pedestrians are visible in only RGB but obscured in thermal as the intermediate obstacle (i.e., heat-insulating cloth or window) blocks the thermal radiation emitted by the pedestrian from reaching the thermal sensors. Fig.\ref{fig:1} (d) demonstrates that a multispectral pedestrian detection model fails prediction on ROTX, though the firefighter is obviously visible in RGB. As seen, multispectral pedestrian detectors make predictions on ROTX just based on the absence of the thermal feature, motivating us to formulate this phenomenon as a modality bias problem.\\&#10; To address this problem, we propose a novel Causal Mode Multiplexer (CMM) framework that performs unbiased inference from statistically biased multispectral pedestrian training data. We adopt this problem setting instead of explicitly changing the training priors (e.g., adding ROTX in the train set) to verify that multispectral pedestrian detectors can disentangle the learned multimodal knowledge and memorized priors by learning causality. Specifically, the CMM framework learns causality based on different cause-and-effects between ROTO, RXTO, and ROTX inputs and predictions. For ROTO data, we guide the model to learn the total effect~\cite{pearl2018book} in the common mode learning scheme. Next, for ROTX and RXTO data, we utilize the tools of counterfactual intervention to eliminate the direct effect of thermal by subtracting it from the total effect. To this end, we modify the training objective from maximizing the posterior probability likelihood to maximizing the total indirect effect~\cite{pearl2018book} in the differential mode learning scheme. Our design requires combining two different learning schemes; therefore, we propose a Causal Mode Multiplexing (CMM) Loss to optimize the interchange.\\ &#10; Moreover, we evaluate the modality bias in multispectral pedestrian detection with our new dataset: ROTX Multispectral Pedestrian (ROTX-MP) dataset. Different from existing multispectral pedestrian datasets~\cite{hwang2015multispectral, gonzalez2016pedestrian, c:25} which are constrained to ROTO (day) and RXTO (night) data, ROTX-MP mainly includes ROTX pedestrians comprised of 1000 test image pairs. Our experimental results demonstrate that our CMM framework generalizes well under ROTX-MP even with biased training and also performs robustly on existing datasets~\cite{hwang2015multispectral, c:25, gonzalez2016pedestrian}. \\&#10;The main contributions of our paper are:&#10;\begin{enumerate}&#10; \item We propose a Causal Mode Multiplexer (CMM) framework that learns different causality between ROTO, RXTO, and ROTX inputs and pedestrian labels in multispectral pedestrian detection.&#10; \item We propose a Causal Mode Multiplexing (CMM) Loss to optimize the interchange between learning different causal representations.&#10; \item To evaluate modality bias in multispectral pedestrian detection, we conduct a new dataset: ROTX-MP. &#10; \item Extensive experiments demonstrate that our CMM framework generalizes well under ROTX test data with even biased training data - ROTO, RXTO.&#10;\end{enumerate}&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Modality Bias Mitigation, Machine Learning, Computer Vision, Computer Vision Frameworks, Artificial Intelligence, Electrical Engineering, Computer Science, Statistics, Multispectral Pedestrian Detection" />
-        </attvalues>
-      </node>
-      <node id="2403.15209" label="2403.15209">
-        <attvalues>
-          <attvalue for="0" value="MSCoTDet: Language-driven Multi-modal Fusion for Improved Multispectral&#10;  Pedestrian Detection" />
-          <attvalue for="1" value="  Multispectral pedestrian detection is attractive for around-the-clock&#10;applications due to the complementary information between RGB and thermal&#10;modalities. However, current models often fail to detect pedestrians in certain&#10;cases (e.g., thermal-obscured pedestrians), particularly due to the modality&#10;bias learned from statistically biased datasets. In this paper, we investigate&#10;how to mitigate modality bias in multispectral pedestrian detection using Large&#10;Language Models (LLMs). Accordingly, we design a Multispectral Chain-of-Thought&#10;(MSCoT) prompting strategy, which prompts the LLM to perform multispectral&#10;pedestrian detection. Moreover, we propose a novel Multispectral&#10;Chain-of-Thought Detection (MSCoTDet) framework that integrates MSCoT prompting&#10;into multispectral pedestrian detection. To this end, we design a&#10;Language-driven Multi-modal Fusion (LMF) strategy that enables fusing the&#10;outputs of MSCoT prompting with the detection results of vision-based&#10;multispectral pedestrian detection models. Extensive experiments validate that&#10;MSCoTDet effectively mitigates modality biases and improves multispectral&#10;pedestrian detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.04315" label="2006.04315">
-        <attvalues>
-          <attvalue for="0" value="Counterfactual VQA: A Cause-Effect Look at Language Bias" />
-          <attvalue for="1" value="  VQA models may tend to rely on language bias as a shortcut and thus fail to&#10;sufficiently learn the multi-modal knowledge from both vision and language.&#10;Recent debiasing methods proposed to exclude the language prior during&#10;inference. However, they fail to disentangle the &quot;good&quot; language context and&#10;&quot;bad&quot; language bias from the whole. In this paper, we investigate how to&#10;mitigate language bias in VQA. Motivated by causal effects, we proposed a novel&#10;counterfactual inference framework, which enables us to capture the language&#10;bias as the direct causal effect of questions on answers and reduce the&#10;language bias by subtracting the direct language effect from the total causal&#10;effect. Experiments demonstrate that our proposed counterfactual inference&#10;framework 1) is general to various VQA backbones and fusion strategies, 2)&#10;achieves competitive performance on the language-bias sensitive VQA-CP dataset&#10;while performs robustly on the balanced VQA v2 dataset without any augmented&#10;data. The code is available at https://github.com/yuleiniu/cfvqa.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.03493" label="2103.03493">
-        <attvalues>
-          <attvalue for="0" value="Causal Attention for Vision-Language Tasks" />
-          <attvalue for="1" value="  We present a novel attention mechanism: Causal Attention (CATT), to remove&#10;the ever-elusive confounding effect in existing attention-based vision-language&#10;models. This effect causes harmful bias that misleads the attention module to&#10;focus on the spurious correlations in training data, damaging the model&#10;generalization. As the confounder is unobserved in general, we use the&#10;front-door adjustment to realize the causal intervention, which does not&#10;require any knowledge on the confounder. Specifically, CATT is implemented as a&#10;combination of 1) In-Sample Attention (IS-ATT) and 2) Cross-Sample Attention&#10;(CS-ATT), where the latter forcibly brings other samples into every IS-ATT,&#10;mimicking the causal intervention. CATT abides by the Q-K-V convention and&#10;hence can replace any attention module such as top-down attention and&#10;self-attention in Transformers. CATT improves various popular attention-based&#10;vision-language models by considerable margins. In particular, we show that&#10;CATT has great potential in large-scale pre-training, e.g., it can promote the&#10;lighter LXMERT~\cite{tan2019lxmert}, which uses fewer data and less&#10;computational power, comparable to the heavier UNITER~\cite{chen2020uniter}.&#10;Code is published in \url{https://github.com/yangxuntu/catt}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.03649" label="1810.03649">
-        <attvalues>
-          <attvalue for="0" value="Overcoming Language Priors in Visual Question Answering with Adversarial&#10;  Regularization" />
-          <attvalue for="1" value="  Modern Visual Question Answering (VQA) models have been shown to rely heavily&#10;on superficial correlations between question and answer words learned during&#10;training such as overwhelmingly reporting the type of room as kitchen or the&#10;sport being played as tennis, irrespective of the image. Most alarmingly, this&#10;shortcoming is often not well reflected during evaluation because the same&#10;strong priors exist in test distributions; however, a VQA system that fails to&#10;ground questions in image content would likely perform poorly in real-world&#10;settings. In this work, we present a novel regularization scheme for VQA that&#10;reduces this effect. We introduce a question-only model that takes as input the&#10;question encoding from the VQA model and must leverage language biases in order&#10;to succeed. We then pose training as an adversarial game between the VQA model&#10;and this question-only adversary -- discouraging the VQA model from capturing&#10;language biases in its question encoding. Further,we leverage this&#10;question-only model to estimate the increase in model confidence after&#10;considering the image, which we maximize explicitly to encourage visual&#10;grounding. Our approach is a model agnostic training procedure and simple to&#10;implement. We show empirically that it can improve performance significantly on&#10;a bias-sensitive split of the VQA dataset for multiple base models -- achieving&#10;state-of-the-art on this task. Further, on standard VQA tasks, our approach&#10;shows significantly less drop in accuracy compared to existing bias-reducing&#10;VQA models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.05306" label="2202.05306">
-        <attvalues>
-          <attvalue for="0" value="Characterizing and overcoming the greedy nature of learning in&#10;  multi-modal deep neural networks" />
-          <attvalue for="1" value="  We hypothesize that due to the greedy nature of learning in multi-modal deep&#10;neural networks, these models tend to rely on just one modality while&#10;under-fitting the other modalities. Such behavior is counter-intuitive and&#10;hurts the models' generalization, as we observe empirically. To estimate the&#10;model's dependence on each modality, we compute the gain on the accuracy when&#10;the model has access to it in addition to another modality. We refer to this&#10;gain as the conditional utilization rate. In the experiments, we consistently&#10;observe an imbalance in conditional utilization rates between modalities,&#10;across multiple tasks and architectures. Since conditional utilization rate&#10;cannot be computed efficiently during training, we introduce a proxy for it&#10;based on the pace at which the model learns from each modality, which we refer&#10;to as the conditional learning speed. We propose an algorithm to balance the&#10;conditional learning speeds between modalities during training and demonstrate&#10;that it indeed addresses the issue of greedy learning. The proposed algorithm&#10;improves the model's generalization on three datasets: Colored MNIST,&#10;ModelNet40, and NVIDIA Dynamic Hand Gesture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.03923" label="1910.03923">
-        <attvalues>
-          <attvalue for="0" value="Multiple Kernel Fisher Discriminant Metric Learning for Person&#10;  Re-identification" />
-          <attvalue for="1" value="  Person re-identification addresses the problem of matching pedestrian images&#10;across disjoint camera views. Design of feature descriptor and distance metric&#10;learning are the two fundamental tasks in person re-identification. In this&#10;paper, we propose a metric learning framework for person re-identification,&#10;where the discriminative metric space is learned using Kernel Fisher&#10;Discriminant Analysis (KFDA), to simultaneously maximize the inter-class&#10;variance as well as minimize the intra-class variance. We derive a Mahalanobis&#10;metric induced by KFDA and argue that KFDA is efficient to be applied for&#10;metric learning in person re-identification. We also show how the efficiency of&#10;KFDA in metric learning can be further enhanced for person re-identification by&#10;using two simple yet efficient multiple kernel learning methods. We conduct&#10;extensive experiments on three benchmark datasets for person re-identification&#10;and demonstrate that the proposed approaches have competitive performance with&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;Person re-identification (re-ID) addresses the problem of matching pedestrian images across disjoint camera views. Person re-ID is of growing interest due to its wide application in video surveillance, security, bio-metrics and forensics. It is a very challenging problem since images of the same person in distinct cameras look very different due to the large variations in illumination, pose, viewpoint, camera characteristics and background clutter (see Fig. \ref{fig:re-ID}). The low resolution makes the image quality insufficient to distinguish the identities based on their physical attributes. Moreover, the costume of distinct pedestrians can be very similar making them more indistinguishable.&#10;&#10;Person re-ID consists of generally two stages: design of robust feature descriptors and distance metric learning. The feature descriptors are designed to capture the similarity of same person images and the dissimilarity of distinct persons. Various feature descriptors ELF \cite{ELF}, SDALF \cite{SDALF}, WHOS \cite{LisantiPAMI14}, LOMO \cite{LOMO}, GOG \cite{GOG} &#10;have been proposed in the literature. However, due to large variations in the camera and scene characteristics, the accuracy of the feature descriptors in discrimination is limited. Therefore distance metric learning \cite{PRDC,LFDA:CVPR,SSSVM,rPcca,KISSME,LOMO,MLAPG,Zheng:nfst,NK3ML}&#10; is used as a second stage in person re-ID systems to improve the discrimination. They learn a distance function which minimizes the within class variance and maximizes the between class variance to effectively improve the accuracy of person re-ID systems.&#10;&#10;There exist re-ranking methods \cite{Symantic,song:scalableManifold,SHaPE,Re-ranking:k-reciprocal} to be applied in the testing phase of a trained person re-ID system, to further improve the accuracy. They refine the ranked results by learning the similarity or manifold structure of the whole test data. However, they are computationally intensive and unsuitable for real-time applications. Moreover, the accuracy of re-ranking methods largely depend on the efficiency of the preceding metric learning method. Hence, efficient metric learning methods are very crucial for such person re-ID systems also.&#10; Deep learning has also been applied for person re-ID and shown good results on large training datasets \cite{Beyond:triplet_loss, Zhun:BodyLatentParts, Zhou:PointtoSetSimilarity, SpindleNet}. However, the availability of large labeled data for every new camera network is practically very expensive in the context of person re-ID. Regardless, metric learning methods have shown impressive performance even with small training data. Therefore, developing efficient metric learning methods is very important for person re-ID and hence is the main focus of this paper.&#10;&#10;In this paper, we propose a metric learning framework for person re-ID, where the discriminative metric space is learned using Kernel Fisher Discriminant Analysis (KFDA) \cite{kfda_Mika}. We derive a Mahalanobis distance metric, induced by KFDA, which simultaneously maximizes the inter-class variance as well as minimizes the intra-class variance. We first argue that, without the need of any supplementary method, KFDA has strong potential and is efficient to be applied for metric learning in person re-ID. In order to further enhance the efficiency of the learned metric in discrimination, we also propose to use two simple yet efficient multiple kernel learning frameworks. Multiple kernel learning is used to remove the bias in using a single kernel and to utilize the information from all the given kernels effectively.&#10;We use kernels for two main reasons: (i) The training computation in most metric learning methods largely depends on the dimensionality of the feature descriptor, which is very high, typically in ten thousands. To remove the computational overhead, most methods use unsupervised dimensionality reduction, which removes the important discriminative information. However, use of kernels help reduce the computations to the order of the number of samples, without the need of any dimensionality reduction and thus providing significant reduction in the training time without losing discriminative information. &#10;(ii) Usage of appropriate kernel helps to get a richer representation of the data in the high dimensional feature space for better discrimination. Kernels facilitate to model the high non-linearity in the person appearance across cameras for more efficient person re-ID. &#10;&#10;Our main contributions in this paper are the following: &#10;\begin{itemize}&#10;\item We first identify that, without the need of any supplementary method, KFDA is a strong candidate to be applied in metric learning for person re-ID.&#10;\item We show that the performance of KFDA for person re-ID can be further enhanced using two multiple kernel learning frameworks. &#10;\item We perform extensive experiments on three datasets and show that the proposed frameworks attain competitive performance with many state-of-the-art methods.&#10;\end{itemize} &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Metric Learning, Computer Science, Pattern Recognition, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="1406.4216" label="1406.4216">
-        <attvalues>
-          <attvalue for="0" value="Person Re-identification by Local Maximal Occurrence Representation and&#10;  Metric Learning" />
-          <attvalue for="1" value="  Person re-identification is an important technique towards automatic search&#10;of a person's presence in a surveillance video. Two fundamental problems are&#10;critical for person re-identification, feature representation and metric&#10;learning. An effective feature representation should be robust to illumination&#10;and viewpoint changes, and a discriminant metric should be learned to match&#10;various person images. In this paper, we propose an effective feature&#10;representation called Local Maximal Occurrence (LOMO), and a subspace and&#10;metric learning method called Cross-view Quadratic Discriminant Analysis&#10;(XQDA). The LOMO feature analyzes the horizontal occurrence of local features,&#10;and maximizes the occurrence to make a stable representation against viewpoint&#10;changes. Besides, to handle illumination variations, we apply the Retinex&#10;transform and a scale invariant texture operator. To learn a discriminant&#10;metric, we propose to learn a discriminant low dimensional subspace by&#10;cross-view quadratic discriminant analysis, and simultaneously, a QDA metric is&#10;learned on the derived subspace. We also present a practical computation method&#10;for XQDA, as well as its regularization. Experiments on four challenging person&#10;re-identification databases, VIPeR, QMUL GRID, CUHK Campus, and CUHK03, show&#10;that the proposed method improves the state-of-the-art rank-1 identification&#10;rates by 2.2%, 4.88%, 28.91%, and 31.55% on the four databases, respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.02139" label="1603.02139">
-        <attvalues>
-          <attvalue for="0" value="Learning a Discriminative Null Space for Person Re-identification" />
-          <attvalue for="1" value="  Most existing person re-identification (re-id) methods focus on learning the&#10;optimal distance metrics across camera views. Typically a person's appearance&#10;is represented using features of thousands of dimensions, whilst only hundreds&#10;of training samples are available due to the difficulties in collecting matched&#10;training images. With the number of training samples much smaller than the&#10;feature dimension, the existing methods thus face the classic small sample size&#10;(SSS) problem and have to resort to dimensionality reduction techniques and/or&#10;matrix regularisation, which lead to loss of discriminative power. In this&#10;work, we propose to overcome the SSS problem in re-id distance metric learning&#10;by matching people in a discriminative null space of the training data. In this&#10;null space, images of the same person are collapsed into a single point thus&#10;minimising the within-class scatter to the extreme and maximising the relative&#10;between-class separation simultaneously. Importantly, it has a fixed dimension,&#10;a closed-form solution and is very efficient to compute. Extensive experiments&#10;carried out on five person re-identification benchmarks including VIPeR,&#10;PRID2011, CUHK01, CUHK03 and Market1501 show that such a simple approach beats&#10;the state-of-the-art alternatives, often by a big margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.10908" label="1807.10908">
-        <attvalues>
-          <attvalue for="0" value="Maximum Margin Metric Learning Over Discriminative Nullspace for Person&#10;  Re-identification" />
-          <attvalue for="1" value="  In this paper we propose a novel metric learning framework called Nullspace&#10;Kernel Maximum Margin Metric Learning (NK3ML) which efficiently addresses the&#10;small sample size (SSS) problem inherent in person re-identification and offers&#10;a significant performance gain over existing state-of-the-art methods. Taking&#10;advantage of the very high dimensionality of the feature space, the metric is&#10;learned using a maximum margin criterion (MMC) over a discriminative nullspace&#10;where all training sample points of a given class map onto a single point,&#10;minimizing the within class scatter. A kernel version of MMC is used to obtain&#10;a better between class separation. Extensive experiments on four challenging&#10;benchmark datasets for person re-identification demonstrate that the proposed&#10;algorithm outperforms all existing methods. We obtain 99.8% rank-1 accuracy on&#10;the most widely accepted and challenging dataset VIPeR, compared to the&#10;previous state of the art being only 63.92%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.03725" label="1706.03725">
-        <attvalues>
-          <attvalue for="0" value="Transferring a Semantic Representation for Person Re-Identification and&#10;  Search" />
-          <attvalue for="1" value="  Learning semantic attributes for person re-identification and&#10;description-based person search has gained increasing interest due to&#10;attributes' great potential as a pose and view-invariant representation.&#10;However, existing attribute-centric approaches have thus far underperformed&#10;state-of-the-art conventional approaches. This is due to their non-scalable&#10;need for extensive domain (camera) specific annotation. In this paper we&#10;present a new semantic attribute learning approach for person re-identification&#10;and search. Our model is trained on existing fashion photography datasets --&#10;either weakly or strongly labelled. It can then be transferred and adapted to&#10;provide a powerful semantic description of surveillance person detections,&#10;without requiring any surveillance domain supervision. The resulting&#10;representation is useful for both unsupervised and supervised person&#10;re-identification, achieving state-of-the-art and near state-of-the-art&#10;performance respectively. Furthermore, as a semantic representation it allows&#10;description-based person search to be integrated within the same framework.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.08359" label="1703.08359">
-        <attvalues>
-          <attvalue for="0" value="Scalable Person Re-identification on Supervised Smoothed Manifold" />
-          <attvalue for="1" value="  Most existing person re-identification algorithms either extract robust&#10;visual features or learn discriminative metrics for person images. However, the&#10;underlying manifold which those images reside on is rarely investigated. That&#10;raises a problem that the learned metric is not smooth with respect to the&#10;local geometry structure of the data manifold.&#10;  In this paper, we study person re-identification with manifold-based affinity&#10;learning, which did not receive enough attention from this area. An&#10;unconventional manifold-preserving algorithm is proposed, which can 1) make the&#10;best use of supervision from training data, whose label information is given as&#10;pairwise constraints; 2) scale up to large repositories with low on-line time&#10;complexity; and 3) be plunged into most existing algorithms, serving as a&#10;generic postprocessing procedure to further boost the identification&#10;accuracies. Extensive experimental results on five popular person&#10;re-identification benchmarks consistently demonstrate the effectiveness of our&#10;method. Especially, on the largest CUHK03 and Market-1501, our method&#10;outperforms the state-of-the-art alternatives by a large margin with high&#10;efficiency, which is more appropriate for practical applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.08398" label="1701.08398">
-        <attvalues>
-          <attvalue for="0" value="Re-ranking Person Re-identification with k-reciprocal Encoding" />
-          <attvalue for="1" value="  When considering person re-identification (re-ID) as a retrieval process,&#10;re-ranking is a critical step to improve its accuracy. Yet in the re-ID&#10;community, limited effort has been devoted to re-ranking, especially those&#10;fully automatic, unsupervised solutions. In this paper, we propose a&#10;k-reciprocal encoding method to re-rank the re-ID results. Our hypothesis is&#10;that if a gallery image is similar to the probe in the k-reciprocal nearest&#10;neighbors, it is more likely to be a true match. Specifically, given an image,&#10;a k-reciprocal feature is calculated by encoding its k-reciprocal nearest&#10;neighbors into a single vector, which is used for re-ranking under the Jaccard&#10;distance. The final distance is computed as the combination of the original&#10;distance and the Jaccard distance. Our re-ranking method does not require any&#10;human interaction or any labeled data, so it is applicable to large-scale&#10;datasets. Experiments on the large-scale Market-1501, CUHK03, MARS, and PRW&#10;datasets confirm the effectiveness of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.01719" label="1704.01719">
-        <attvalues>
-          <attvalue for="0" value="Beyond triplet loss: a deep quadruplet network for person&#10;  re-identification" />
-          <attvalue for="1" value="  Person re-identification (ReID) is an important task in wide area video&#10;surveillance which focuses on identifying people across different cameras.&#10;Recently, deep learning networks with a triplet loss become a common framework&#10;for person ReID. However, the triplet loss pays main attentions on obtaining&#10;correct orders on the training set. It still suffers from a weaker&#10;generalization capability from the training set to the testing set, thus&#10;resulting in inferior performance. In this paper, we design a quadruplet loss,&#10;which can lead to the model output with a larger inter-class variation and a&#10;smaller intra-class variation compared to the triplet loss. As a result, our&#10;model has a better generalization ability and can achieve a higher performance&#10;on the testing set. In particular, a quadruplet deep network using a&#10;margin-based online hard negative mining is proposed based on the quadruplet&#10;loss for the person ReID. In extensive experiments, the proposed network&#10;outperforms most of the state-of-the-art algorithms on representative datasets&#10;which clearly demonstrates the effectiveness of our proposed method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.00801" label="2102.00801">
-        <attvalues>
-          <attvalue for="0" value="Few-shot Image Classification with Multi-Facet Prototypes" />
-          <attvalue for="1" value="  The aim of few-shot learning (FSL) is to learn how to recognize image&#10;categories from a small number of training examples. A central challenge is&#10;that the available training examples are normally insufficient to determine&#10;which visual features are most characteristic of the considered categories. To&#10;address this challenge, we organize these visual features into facets, which&#10;intuitively group features of the same kind (e.g. features that are relevant to&#10;shape, color, or texture). This is motivated from the assumption that (i) the&#10;importance of each facet differs from category to category and (ii) it is&#10;possible to predict facet importance from a pre-trained embedding of the&#10;category names. In particular, we propose an adaptive similarity measure,&#10;relying on predicted facet importance weights for a given set of categories.&#10;This measure can be used in combination with a wide array of existing&#10;metric-based methods. Experiments on miniImageNet and CUB show that our&#10;approach improves the state-of-the-art in metric-based FSL.&#10;" />
-          <attvalue for="2" value="&#10;Few-shot learning aims to recognize new image categories given only a few labeled examples. Most existing methods either fall into the meta-learning based~\cite{optimization-as-model,MAML,Meta-SGD} or the metric-based&#10;~\cite{relationnet,protonet,feat,match-net} paradigms. &#10;In this paper, we focus in particular on metric-based few-shot learning methods, which have received a lot of attention due to their simplicity, flexibility and effectiveness. These methods aim to learn an embedding space in which the similarity of images can be evaluated using a pre-defined metric, such as Euclidean distance. &#10;&#10;Early work on one-shot learning used Siamese networks \cite{siamese} to learn appropriate embedding spaces. Beyond one-shot learning, \cite{match-net} introduced an episode-based training strategy and proposed Matching Network, an FSL model which combines a weighted nearest-neighbor classifier with an attention mechanism. ProtoNet~\cite{protonet} first constructs a prototype for each category, by taking the mean of all the corresponding training examples, and then classifies test images based on the nearest prototype. Instead of fixing a pre-defined metric, Relation Network~\cite{relationnet} learns a deep distance metric to compare each query-support image pair. The recently introduced FEAT model~\cite{feat} applies a transformer~\cite{DBLP:conf/nips/VaswaniSPUJGKP17} to contextualize the image features relative to the support set in a given task. &#10;&#10;The aforementioned methods only rely on visual features. However, in addition to the training examples, we usually also have access to the names of the image categories to be learned. AM3~\cite{AM3} uses a pre-trained embedding of these category names, using the GloVe word embedding model \cite{DBLP:conf/emnlp/PenningtonSM14}, to predict visual prototypes. TRAML~\cite{traml} also uses GloVe vectors, but instead uses them to get prior information about which categories may be most difficult to separate. More precisely, TRAML uses the word vectors to implement an adaptive margin-based model.&#10;&#10;In this paper, we propose a third way in which word vectors may be used to improve metric based FSL models. Our starting point is that we can intuitively think of the visual features as capturing different facets, such as shape, texture, color, etc. The relative importance of these facets moreover strongly depends on the considered image categories.&#10;For example, color-related facets are likely to be important to classify image from the `desert' category, whereas shape-related facets may be more important for images from the `cat' category. Therefore, we propose a method to predict the importance of the different facets for a given category, based on a pre-trained embedding of the category name. In addition to using GloVe vectors for representing the category names, we also experiment with embeddings that are obtained using the BERT language model \cite{DBLP:conf/naacl/DevlinCLT19}. The facets themselves are generated by clustering the coordinates of the visual feature vectors, using a similarity metric that is based on which visual classes can be discriminated well by the different coordinates. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Few-Shot Learning, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science, Visual Feature Analysis, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1707.09835" label="1707.09835">
-        <attvalues>
-          <attvalue for="0" value="Meta-SGD: Learning to Learn Quickly for Few-Shot Learning" />
-          <attvalue for="1" value="  Few-shot learning is challenging for learning algorithms that learn each task&#10;in isolation and from scratch. In contrast, meta-learning learns from many&#10;related tasks a meta-learner that can learn a new task more accurately and&#10;faster with fewer examples, where the choice of meta-learners is crucial. In&#10;this paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learner&#10;that can initialize and adapt any differentiable learner in just one step, on&#10;both supervised learning and reinforcement learning. Compared to the popular&#10;meta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, and&#10;can be learned more efficiently. Compared to the latest meta-learner MAML,&#10;Meta-SGD has a much higher capacity by learning to learn not just the learner&#10;initialization, but also the learner update direction and learning rate, all in&#10;a single meta-learning process. Meta-SGD shows highly competitive performance&#10;for few-shot learning on regression, classification, and reinforcement&#10;learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.03664" label="1812.03664">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions" />
-          <attvalue for="1" value="  Learning with limited data is a key challenge for visual recognition. Many&#10;few-shot learning methods address this challenge by learning an instance&#10;embedding function from seen classes and apply the function to instances from&#10;unseen classes with limited labels. This style of transfer learning is&#10;task-agnostic: the embedding function is not learned optimally discriminative&#10;with respect to the unseen classes, where discerning among them leads to the&#10;target task. In this paper, we propose a novel approach to adapt the instance&#10;embeddings to the target classification task with a set-to-set function,&#10;yielding embeddings that are task-specific and are discriminative. We&#10;empirically investigated various instantiations of such set-to-set functions&#10;and observed the Transformer is most effective -- as it naturally satisfies key&#10;properties of our desired model. We denote this model as FEAT (few-shot&#10;embedding adaptation w/ Transformer) and validate it on both the standard&#10;few-shot classification benchmark and four extended few-shot learning settings&#10;with essential use cases, i.e., cross-domain, transductive, generalized&#10;few-shot learning, and low-shot learning. It archived consistent improvements&#10;over baseline models as well as previous methods and established the new&#10;state-of-the-art results on two benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.04080" label="1606.04080">
-        <attvalues>
-          <attvalue for="0" value="Matching Networks for One Shot Learning" />
-          <attvalue for="1" value="  Learning from a few examples remains a key challenge in machine learning.&#10;Despite recent advances in important domains such as vision and language, the&#10;standard supervised deep learning paradigm does not offer a satisfactory&#10;solution for learning new concepts rapidly from little data. In this work, we&#10;employ ideas from metric learning based on deep neural features and from recent&#10;advances that augment neural networks with external memories. Our framework&#10;learns a network that maps a small labelled support set and an unlabelled&#10;example to its label, obviating the need for fine-tuning to adapt to new class&#10;types. We then define one-shot learning problems on vision (using Omniglot,&#10;ImageNet) and language tasks. Our algorithm improves one-shot accuracy on&#10;ImageNet from 87.6% to 93.2% and from 88.0% to 93.8% on Omniglot compared to&#10;competing approaches. We also demonstrate the usefulness of the same model on&#10;language modeling by introducing a one-shot task on the Penn Treebank.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.07104" label="1902.07104">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Cross-Modal Few-Shot Learning" />
-          <attvalue for="1" value="  Metric-based meta-learning techniques have successfully been applied to&#10;few-shot classification problems. In this paper, we propose to leverage&#10;cross-modal information to enhance metric-based few-shot learning methods.&#10;Visual and semantic feature spaces have different structures by definition. For&#10;certain concepts, visual features might be richer and more discriminative than&#10;text ones. While for others, the inverse might be true. Moreover, when the&#10;support from visual information is limited in image classification, semantic&#10;representations (learned from unsupervised text corpora) can provide strong&#10;prior knowledge and context to help learning. Based on these two intuitions, we&#10;propose a mechanism that can adaptively combine information from both&#10;modalities according to new image categories to be learned. Through a series of&#10;experiments, we show that by this adaptive combination of the two modalities,&#10;our model outperforms current uni-modality few-shot learning methods and&#10;modality-alignment methods by a large margin on all benchmarks and few-shot&#10;scenarios tested. Experiments also show that our model can effectively adjust&#10;its focus on the two modalities. The improvement in performance is particularly&#10;large when the number of shots is very small.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.13826" label="2005.13826">
-        <attvalues>
-          <attvalue for="0" value="Boosting Few-Shot Learning With Adaptive Margin Loss" />
-          <attvalue for="1" value="  Few-shot learning (FSL) has attracted increasing attention in recent years&#10;but remains challenging, due to the intrinsic difficulty in learning to&#10;generalize from a few examples. This paper proposes an adaptive margin&#10;principle to improve the generalization ability of metric-based meta-learning&#10;approaches for few-shot learning problems. Specifically, we first develop a&#10;class-relevant additive margin loss, where semantic similarity between each&#10;pair of classes is considered to separate samples in the feature embedding&#10;space from similar classes. Further, we incorporate the semantic context among&#10;all classes in a sampled training task and develop a task-relevant additive&#10;margin loss to better distinguish samples from different classes. Our adaptive&#10;margin method can be easily extended to a more realistic generalized FSL&#10;setting. Extensive experiments demonstrate that the proposed method can boost&#10;the performance of current metric-based meta-learning approaches, under both&#10;the standard FSL and generalized FSL settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.11549" label="2304.11549">
-        <attvalues>
-          <attvalue for="0" value="Spectral Sensitivity Estimation Without a Camera" />
-          <attvalue for="1" value="  A number of problems in computer vision and related fields would be mitigated&#10;if camera spectral sensitivities were known. As consumer cameras are not&#10;designed for high-precision visual tasks, manufacturers do not disclose&#10;spectral sensitivities. Their estimation requires a costly optical setup, which&#10;triggered researchers to come up with numerous indirect methods that aim to&#10;lower cost and complexity by using color targets. However, the use of color&#10;targets gives rise to new complications that make the estimation more&#10;difficult, and consequently, there currently exists no simple, low-cost, robust&#10;go-to method for spectral sensitivity estimation. Furthermore, even if not&#10;limited by hardware or cost, researchers frequently work with imagery from&#10;multiple cameras that they do not have in their possession. To provide a&#10;practical solution to this problem, we propose a framework for spectral&#10;sensitivity estimation that not only does not require any hardware, but also&#10;does not require physical access to the camera itself. Similar to other work,&#10;we formulate an optimization problem that minimizes a two-term objective&#10;function: a camera-specific term from a system of equations, and a universal&#10;term that bounds the solution space. Different than other work, we use publicly&#10;available high-quality calibration data to construct both terms. We use the&#10;colorimetric mapping matrices provided by the Adobe DNG Converter to formulate&#10;the camera-specific system of equations, and constrain the solutions using an&#10;autoencoder trained on a database of ground-truth curves. On average, we&#10;achieve reconstruction errors as low as those that can arise due to&#10;manufacturing imperfections between two copies of the same camera. We provide&#10;our code and predicted sensitivities for 1,000+ cameras, and discuss which&#10;tasks can become trivial when camera responses are available.&#10;" />
-          <attvalue for="2" value="&#10; C{onsumer} digital cameras (i.e., point-and-shoot, DSLR, mirrorless cameras and smartphones) are not tools designed for scientific imaging, i.e., they are not scientific light-measuring devices~\cite{burggraaff2019standardized,akkaynak2014use,kim2012new}. Yet their outputs\textemdash digital photos and videos\textemdash constitute major sources of data for image processing, colorimetry, computational photography, computer vision, and machine learning. Typically, research utilizing consumer camera imagery focuses on the development of filters or algorithms that alter the visual appearance of images~\cite{paris2009fast, guo2020zero,zhu2017unpaired,bychkovsky2011learning}, or on the understanding of scene content~\cite{deng2009imagenet,lowe1999object,xiao2010sun} and structure~\cite{mildenhall2021nerf,lee2009geometric} with downstream goals like recognizing, tracking, or counting objects. For many of these goals, successfully recovering scene reflectance and/or illumination is key (and often the main goal itself), but these tasks are complicated by the fact that consumer cameras do not capture colors in a standardized way~\cite{nguyen2014raw,chakrabarti2009empirical}.&#10;&#10;Each camera captures colors in its own, device-specific color space. This is because manufacturers make different design choices for each sensor, which materialize within the limitations inherent to silicon-based semiconductor technology~\cite{bhandari2022computational}. As a result, each make and model of camera is characterized by its unique set of spectral sensitivities that define how that camera responds to light. Effectively, this means that different cameras will register a different set of RGB values for a given scene\textemdash even if the scene is imaged under identical conditions~\cite{nguyen2014raw,karaimer2018improving}. If the camera's color bias could be eliminated, the recovery of scene reflectance (e.g., for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc.) and/or scene illumination (e.g., for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc.) could be achieved simpler and faster, and with more consistent results across cameras. Figure~\ref{fig:fig1} gives some examples of tasks that could become possible or easier if camera spectral sensitivities were known.&#10;&#10;Unfortunately, however, manufacturers of consumer cameras rarely make spectral sensitivity curves available, and the burden falls on the researchers to derive them. Their derivation (detailed in~\ref{sec:background}) requires the stimulation of the camera's sensor with monochromatic light across the entire wavelength range to which it is sensitive. Such light can be obtained through the use of a monochromator or a spectrally-tuneable light source, but these optical instruments are generally too specialized and expensive to be found in most research labs. A seemingly lower cost and complexity alternative is to use photos of calibrated color targets, but with this indirect option comes a hidden cost: the implicit requirement that the illumination under which the color target is imaged is also known. In practice, this either means that the light source is calibrated, or its spectrum is measured with a spectrophotometer; necessitating, again, that the user has access to non-standard hardware (\ref{tab:survey}). &#10;&#10;Even when the light spectrum is known, photos of color charts do not guarantee a ``good enough'' estimate of curves because there are too many unknowns in camera spectral sensitivities and not enough color chart patches whose reflectances are linearly independent~\cite{alsam2002recovering,hardeberg2002spectral,parkkinen1989characteristic}. The ill-posed nature of the problem introduces new challenges due to the need to constrain the solutions to a ``realistic'' space (\ref{tab:survey}). Furthermore, the commonly used image formation model does not take into account the optoelectronics of the camera~\cite{martinez2002calculation}, making it difficult to relate the captured RGB values to the predicted ones. Finally, even if not limited by hardware or cost, it is not uncommon that researchers work with imagery captured by cameras that they actually do not have in their possession. Consequently, the missing knowledge regarding the spectral responses of consumer cameras hampers the otherwise fruitful utilization of a consumer electronic device in academic research.&#10;&#10;Our own research in underwater computer vision has been held back because we did not have the spectral sensitivities of the ever-evolving set of consumer cameras that produce the imagery we work with, and this motivated us to search for a solution that did \underline{not} require: 1) any specialized hardware, 2) a color chart photo taken by the camera, or 3) physical access to the camera itself. Unlike how it may seem at first glance, this is not an impossible task\textemdash one simply has to be aware of the wealth of publicly available data regarding colorimetric mapping of consumer digital cameras. Here, we describe a method that satisfies these requirements with reconstruction errors that are, on average, as low as those that can arise due to manufacturing imperfections between two copies of the same camera.&#10;&#10;Our key observation is the following: Among the tags specific to Adobe's Digital Negative (DNG) format~\cite{AdobeDNGSpec}, are color transformation matrices prefixed ColorMatrix. These 3x3 matrices map camera-specific colors to the standard CIE XYZ color space under a known illuminant. Calibrated by the manufacturer, Adobe, or third-party companies and appended to image metadata by the Adobe DNG Converter, each DNG file includes at least one ColorMatrix tag, whereas frequently, two matrices per camera can be found. In the common case when two matrices are available per camera, they are generally calibrated under illuminant A (tungsten, CCT$\approx$2856K) and D65 (daylight, CCT$\approx$6504K).&#10;We use these matrices to construct a linear system of equations which contains the sought spectral sensitivities in its solution space; and this solution space can be further constrained using an autoencoder trained on database of ground-truth spectral sensitivities from the literature. Using this novel approach we make the following contributions:&#10; \begin{enumerate}&#10; \item a zero-cost method with which to obtain reasonable estimates of camera spectral sensitivities, and&#10; \item predicted spectral sensitivities of more than $1,000$ makes and models of cameras that are currently in the market.&#10; \end{enumerate}&#10; In addition, we provide a thorough survey of indirect spectral sensitivity estimation methods that have evolved over the last three decades, corresponding with the advent of consumer cameras as prevalent research tools.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Optics, Optical Estimation Methods, Mathematics, Physics, Computer Science, Computer Vision Challenges, Camera Spectral Sensitivity, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1906.04155" label="1906.04155">
-        <attvalues>
-          <attvalue for="0" value="Standardized spectral and radiometric calibration of consumer cameras" />
-          <attvalue for="1" value="  Consumer cameras, particularly onboard smartphones and UAVs, are now commonly&#10;used as scientific instruments. However, their data processing pipelines are&#10;not optimized for quantitative radiometry and their calibration is more complex&#10;than that of scientific cameras. The lack of a standardized calibration&#10;methodology limits the interoperability between devices and, in the&#10;ever-changing market, ultimately the lifespan of projects using them. We&#10;present a standardized methodology and database (SPECTACLE) for spectral and&#10;radiometric calibrations of consumer cameras, including linearity, bias&#10;variations, read-out noise, dark current, ISO speed and gain, flat-field, and&#10;RGB spectral response. This includes golden standard ground-truth methods and&#10;do-it-yourself methods suitable for non-experts. Applying this methodology to&#10;seven popular cameras, we found high linearity in RAW but not JPEG data,&#10;inter-pixel gain variations &gt;400% correlated with large-scale bias and read-out&#10;noise patterns, non-trivial ISO speed normalization functions, flat-field&#10;correction factors varying by up to 2.79 over the field of view, and both&#10;similarities and differences in spectral response. Moreover, these results&#10;differed wildly between camera models, highlighting the importance of&#10;standardization and a centralized database.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.09470" label="1703.09470">
-        <attvalues>
-          <attvalue for="0" value="Learned Spectral Super-Resolution" />
-          <attvalue for="1" value="  We describe a novel method for blind, single-image spectral super-resolution.&#10;While conventional super-resolution aims to increase the spatial resolution of&#10;an input image, our goal is to spectrally enhance the input, i.e., generate an&#10;image with the same spatial resolution, but a greatly increased number of&#10;narrow (hyper-spectral) wave-length bands. Just like the spatial statistics of&#10;natural images has rich structure, which one can exploit as prior to predict&#10;high-frequency content from a low resolution image, the same is also true in&#10;the spectral domain: the materials and lighting conditions of the observed&#10;world induce structure in the spectrum of wavelengths observed at a given&#10;pixel. Surprisingly, very little work exists that attempts to use this&#10;diagnosis and achieve blind spectral super-resolution from single images. We&#10;start from the conjecture that, just like in the spatial domain, we can learn&#10;the statistics of natural image spectra, and with its help generate finely&#10;resolved hyper-spectral images from RGB input. Technically, we follow the&#10;current best practice and implement a convolutional neural network (CNN), which&#10;is trained to carry out the end-to-end mapping from an entire RGB image to the&#10;corresponding hyperspectral image of equal size. We demonstrate spectral&#10;super-resolution both for conventional RGB images and for multi-spectral&#10;satellite data, outperforming the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.04989" label="2106.04989">
-        <attvalues>
-          <attvalue for="0" value="CLCC: Contrastive Learning for Color Constancy" />
-          <attvalue for="1" value="  In this paper, we present CLCC, a novel contrastive learning framework for&#10;color constancy. Contrastive learning has been applied for learning&#10;high-quality visual representations for image classification. One key aspect to&#10;yield useful representations for image classification is to design illuminant&#10;invariant augmentations. However, the illuminant invariant assumption conflicts&#10;with the nature of the color constancy task, which aims to estimate the&#10;illuminant given a raw image. Therefore, we construct effective contrastive&#10;pairs for learning better illuminant-dependent features via a novel raw-domain&#10;color augmentation. On the NUS-8 dataset, our method provides $17.5\%$ relative&#10;improvements over a strong baseline, reaching state-of-the-art performance&#10;without increasing model complexity. Furthermore, our method achieves&#10;competitive performance on the Gehler dataset with $3\times$ fewer parameters&#10;compared to top-ranking deep learning methods. More importantly, we show that&#10;our model is more robust to different scenes under close proximity of&#10;illuminants, significantly reducing $28.7\%$ worst-case error in data-sparse&#10;regions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.13365" label="2104.13365">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2021 Depth Guided Image Relighting Challenge" />
-          <attvalue for="1" value="  Image relighting is attracting increasing interest due to its various&#10;applications. From a research perspective, image relighting can be exploited to&#10;conduct both image normalization for domain adaptation, and also for data&#10;augmentation. It also has multiple direct uses for photo montage and aesthetic&#10;enhancement. In this paper, we review the NTIRE 2021 depth guided image&#10;relighting challenge.&#10;  We rely on the VIDIT dataset for each of our two challenge tracks, including&#10;depth information. The first track is on one-to-one relighting where the goal&#10;is to transform the illumination setup of an input image (color temperature and&#10;light source position) to the target illumination setup. In the second track,&#10;the any-to-any relighting challenge, the objective is to transform the&#10;illumination settings of the input image to match those of another guide image,&#10;similar to style transfer. In both tracks, participants were given depth&#10;information about the captured scenes. We had nearly 250 registered&#10;participants, leading to 18 confirmed team submissions in the final competition&#10;stage. The competitions, methods, and final results are presented in this&#10;paper.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.11890" label="2011.11890">
-        <attvalues>
-          <attvalue for="0" value="Cross-Camera Convolutional Color Constancy" />
-          <attvalue for="1" value="  We present &quot;Cross-Camera Convolutional Color Constancy&quot; (C5), a&#10;learning-based method, trained on images from multiple cameras, that accurately&#10;estimates a scene's illuminant color from raw images captured by a new camera&#10;previously unseen during training. C5 is a hypernetwork-like extension of the&#10;convolutional color constancy (CCC) approach: C5 learns to generate the weights&#10;of a CCC model that is then evaluated on the input image, with the CCC weights&#10;dynamically adapted to different input content. Unlike prior cross-camera color&#10;constancy models, which are usually designed to be agnostic to the spectral&#10;properties of test-set images from unobserved cameras, C5 approaches this&#10;problem through the lens of transductive inference: additional unlabeled images&#10;are provided as input to the model at test time, which allows the model to&#10;calibrate itself to the spectral properties of the test-set camera during&#10;inference. C5 achieves state-of-the-art accuracy for cross-camera color&#10;constancy on several datasets, is fast to evaluate (~7 and ~90 ms per image on&#10;a GPU or CPU, respectively), and requires little memory (~2 MB), and thus is a&#10;practical solution to the problem of calibration-free automatic white balance&#10;for mobile photography.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.12709" label="2006.12709">
-        <attvalues>
-          <attvalue for="0" value="CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks" />
-          <attvalue for="1" value="  Cameras currently allow access to two image states: (i) a minimally processed&#10;linear raw-RGB image state (i.e., raw sensor data) or (ii) a highly-processed&#10;nonlinear image state (e.g., sRGB). There are many computer vision tasks that&#10;work best with a linear image state, such as image deblurring and image&#10;dehazing. Unfortunately, the vast majority of images are saved in the nonlinear&#10;image state. Because of this, a number of methods have been proposed to&#10;&quot;unprocess&quot; nonlinear images back to a raw-RGB state. However, existing&#10;unprocessing methods have a drawback because raw-RGB images are&#10;sensor-specific. As a result, it is necessary to know which camera produced the&#10;sRGB output and use a method or network tailored for that sensor to properly&#10;unprocess it. This paper addresses this limitation by exploiting another camera&#10;image state that is not available as an output, but it is available inside the&#10;camera pipeline. In particular, cameras apply a colorimetric conversion step to&#10;convert the raw-RGB image to a device-independent space based on the CIE XYZ&#10;color space before they apply the nonlinear photo-finishing. Leveraging this&#10;canonical image state, we propose a deep learning framework, CIE XYZ Net, that&#10;can unprocess a nonlinear image back to the canonical CIE XYZ image. This image&#10;can then be processed by any low-level computer vision operator and re-rendered&#10;back to the nonlinear image. We demonstrate the usefulness of the CIE XYZ Net&#10;on several low-level vision tasks and show significant gains that can be&#10;obtained by this processing framework. Code and dataset are publicly available&#10;at https://github.com/mahmoudnafifi/CIE_XYZ_NET.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.14313" label="2005.14313">
-        <attvalues>
-          <attvalue for="0" value="Self-Attention Dense Depth Estimation Network for Unrectified Video&#10;  Sequences" />
-          <attvalue for="1" value="  The dense depth estimation of a 3D scene has numerous applications, mainly in&#10;robotics and surveillance. LiDAR and radar sensors are the hardware solution&#10;for real-time depth estimation, but these sensors produce sparse depth maps and&#10;are sometimes unreliable. In recent years research aimed at tackling depth&#10;estimation using single 2D image has received a lot of attention. The deep&#10;learning based self-supervised depth estimation methods from the rectified&#10;stereo and monocular video frames have shown promising results. We propose a&#10;self-attention based depth and ego-motion network for unrectified images. We&#10;also introduce non-differentiable distortion of the camera into the training&#10;pipeline. Our approach performs competitively when compared to other&#10;established approaches that used rectified images for depth estimation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Several prior works for depth estimation from monocular images doesn't work with raw images straight out of the camera, which would be distorted and unrectified. Instead, these raw images are corrected with image processing techniques before being fed into the network to predict an undistorted rectified depth map. We propose a novel fully differentiable architecture which allows estimating the distorted unrectified depth directly from a raw image without the need for any pre-processing techniques to correct the image. This pipeline in turn saves time and computation power, and can be used directly in real-time scenarios without any prerequisites. The parameters which define distortion vary slightly with the environment in which the camera is present. Our model exploits the fact that more or less the amount of distortion in an image is fixed as long as we use the same camera. This allows us to pre-define a transformation flowfield for undistorting the image and using it in the training pipeline. Image rectification on-the-fly with prerecorded camera parameters has an overhead of around 75msec for a typical 30 FPS (frames per second) camera in one second.&#10;Our proposed model for unrectified distorted images doesn't have any overhead at inference when compared to models based on rectified images as it uses a neural network with the same number of trainable parameters. Image rectification also leads to information loss from the raw image. Higher the camera distortion, the higher the loss of pixel information from the raw data. In the KITTI dataset due to the distortion effect of the camera, the images have been rectified and cropped to 1242 $\times$ 375 such that the size of the rectified undistorted images is smaller than the raw unrectified distorted image size of 1392 $\times$ 512 pixels. Image rectification in KITTI resulted in the pixel loss of 10.77\% and 27.34\% along the width and height of the image, respectively. This pixel loss becomes more prominent at higher resolutions. We address the above issues by proposing the following contributions:&#10;&#10;\begin{enumerate}&#10;\itemsep -0.5em &#10;\item Introduced an end to end fully differentiable novel pipeline to tackle monocular depth estimation on distorted and unrectified images.&#10;\item Proposed a novel self-attention depth network to handle long range dependencies in the feature map leading to sharper depth maps.&#10;\item Incorporated instance normalisation throughout model architecture to handle style variation in the input data.&#10;\end{enumerate}&#10;&#10;Some of the prior self-supervised monocular depth estimation works are briefed here.&#10;Zhou et al., \cite{zhou2017unsupervised} proposed an unsupervised depth estimation end-to-end framework from video sequence where view synthesis act as the supervisory signal.&#10;Mahjourian et al., \cite{mahjourian2018unsupervised} introduced a 3D loss to enforce consistency of the estimated 3D point cloud.&#10;Yin et al., \cite{yin2018geonet} jointly learned monocular depth, optical flow and ego motion from videos in unsupervised manner.&#10;Luo et al., \cite{luo2018every} also jointly learned monocular depth, optical flow and ego motion from video but used these result to produce motion mask, occlusion mask, 3D motion map for rigid background and dynamic objects.&#10;Godard et al., \cite{godard2019digging} achieved state of the art results in KITTI rectified dataset with minimum reprojection loss, full-resolution mutli-scale sampling and a mask that ignores training pixels that violate camera motion assumptions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Physics, Depth Estimation, Deep Learning, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2304.01382" label="2304.01382">
-        <attvalues>
-          <attvalue for="0" value="PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching" />
-          <attvalue for="1" value="  Estimating the pose of an unseen object is the goal of the challenging&#10;one-shot pose estimation task. Previous methods have heavily relied on feature&#10;matching with great success. However, these methods are often inefficient and&#10;limited by their reliance on pre-trained models that have not be designed&#10;specifically for pose estimation. In this paper we propose PoseMatcher, an&#10;accurate model free one-shot object pose estimator that overcomes these&#10;limitations. We create a new training pipeline for object to image matching&#10;based on a three-view system: a query with a positive and negative templates.&#10;This simple yet effective approach emulates test time scenarios by cheaply&#10;constructing an approximation of the full object point cloud during training.&#10;To enable PoseMatcher to attend to distinct input modalities, an image and a&#10;pointcloud, we introduce IO-Layer, a new attention layer that efficiently&#10;accommodates self and cross attention between the inputs. Moreover, we propose&#10;a pruning strategy where we iteratively remove redundant regions of the target&#10;object to further reduce the complexity and noise of the network while&#10;maintaining accuracy. Finally we redesign commonly used pose refinement&#10;strategies, zoom and 2D offset refinements, and adapt them to the one-shot&#10;paradigm. We outperform all prior one-shot pose estimation methods on the&#10;Linemod and YCB-V datasets as well achieve results rivaling recent&#10;instance-level methods. The source code and models are available at&#10;https://github.com/PedroCastro/PoseMatcher.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Accurately retrieving the relative position and orientation of an object is the first step for any task that requires interaction with objects in the real world. Estimating the pose of an object is an indispensable step for robotic manipulation as well as in VR/AR applications. It is imperative for the pose estimation to be accurate and robust to external obstacles such as occlusion, illumination and symmetries. Existing methods excel at retrieving the pose of known objects to a very high accuracy standard \cite{gdr, sopose, SC6D}. Some methods can even produce pose estimation at high throughput \cite{gdr, crt6d}, be specially robust to symmetries \cite{surfemb} and can even surpass synthetic to real domain gaps \cite{self6d, juil}. However, most share a very large limitation: the target objects must be known. This constraint is severely limiting as it necessitates model retraining for each object addition. Retraining new objects on existing models might lead to catastrophic forgetting if not handled properly~\cite{catastrophic}. Category level approaches~\cite{posefromshape, nocs} try to generalize up to a category, where each target object can be obtained with a simple deformation of a canonical model and all share semantic keypoints (ex. handle on a mug or the cap of a bottle). Nonetheless, much of the same one-shot domain problems still remain: objects category must have been seen before and the target object must not lie outside of the scope of the category for the estimation to be possible. In order to overcome these problems we must tackle object pose estimation in the one-shot paradigm.&#10;&#10;By one-shot pose estimation, we refer to estimating the pose of an novel object based on information seen only at test-time, without the object,or its category, being present in the training dataset. Older methods have some very hard constraints such as needing masking and depth maps at test-time \cite{ppf}, a full colored 3D model of the target object \cite{deepim}, need to be fine-tuned on images of the same dataset for difficulties of overcoming domain gap~\cite{gen6d} or rely on an extremely expensive test-time optimization through renderers~\cite{inerf, nerfpose, latentfusion}.&#10;More recently, feature matching methods have been shown to achieve impressive results. Particularly, OnePose~\cite{onepose} and OnePose++~\cite{onepose++} make use of existing state of the art pre-trained descriptor extractors on top of which a pose estimation pipeline is built. However, by relying on a fixed pre-trained model to extract descriptive keypoints from templates it fails to capture the optimal keypoint object descriptions. Ideally, the template extraction model should be the same as the query model and should be jointly optimized.&#10;&#10;We rethink the approach to the problem and introduce a three-view pipeline that allows us to jointly train the template and query extractors. Just by redesigning the training pipeline, we can improve OnePose++\cite{onepose++} without additional changes to its methodology. We also introduce a new efficient image to object attention layer which we call IO-Layer, which reduces both parameters when compared to the modules used by OnePose++~\cite{onepose++} and also separates the two input modalities, an image and a pointcloud, allowing the model to optimize weights specifically for either image or pointcloud keypoints. We also found that working with a full object at all stages of the feature matching is redundant and reduces pose estimation accuracy which lead us to introduce a novel iterative matching based pointcloud prunning. On top of these changes, we employ a 3D-refinement technique based on zoom refinement used on instance-level pose estimation~\cite{deepim,crt6d,cosypose}.&#10;&#10;In summary our contributions are as follows:&#10;&#10;\begin{itemize}&#10; \item We redesign the training pipeline for one-shot object-to-image matching. Our three-view training approach allows us to train from scratch PoseMatcher, a pose estimation method leveraging detection free keypoint matching.&#10; \item We introduce a more efficient image to object attention layer we call IO-Layer, specifically built to accommodate the two input modalities, the image and the object pointcloud. &#10; \item We propose a layered pruning of the target object point cloud. We improve runtime by reducing the amount of attented keypoints while reducing noisy matching, leading to an improvement in accuracy.&#10; \item We create a fine level 3D based refinement where we directly estimate relative 2D and depth positions, replacing the 2D keypoint refinement used by prior one-shot approaches.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning Architectures, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2102.12145" label="2102.12145">
-        <attvalues>
-          <attvalue for="0" value="GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D&#10;  Object Pose Estimation" />
-          <attvalue for="1" value="  6D pose estimation from a single RGB image is a fundamental task in computer&#10;vision. The current top-performing deep learning-based methods rely on an&#10;indirect strategy, i.e., first establishing 2D-3D correspondences between the&#10;coordinates in the image plane and object coordinate system, and then applying&#10;a variant of the P$n$P/RANSAC algorithm. However, this two-stage pipeline is&#10;not end-to-end trainable, thus is hard to be employed for many tasks requiring&#10;differentiable poses. On the other hand, methods based on direct regression are&#10;currently inferior to geometry-based methods. In this work, we perform an&#10;in-depth investigation on both direct and indirect methods, and propose a&#10;simple yet effective Geometry-guided Direct Regression Network (GDR-Net) to&#10;learn the 6D pose in an end-to-end manner from dense correspondence-based&#10;intermediate geometric representations. Extensive experiments show that our&#10;approach remarkably outperforms state-of-the-art methods on LM, LM-O and YCB-V&#10;datasets. Code is available at https://git.io/GDR-Net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;While methods based on depth data used to dominate the field of 6D pose estimation~\cite{Hinterstoisser2012,michel2016global,hinterstoisser2016going,zeng2017multi,vidal2018method},&#10;Deep Learning-based methods have recently demonstrated promising results for the task at hand~\cite{hodan2018bop,hodan_bop20}.&#10;In this section, we review several commonly employed strategies for monocular 6D pose estimation.&#10;&#10;Indirect Methods.&#10;The most popular approach is to establish 2D-3D correspondences, which are then leveraged to solve for the 6D pose using a variant of the RANSAC-based P$n$P algorithm.&#10;For instance,~\cite{rad2017bb8} and~\cite{tekin18_yolo6d} compute the 2D projections of a set of fixed control points (\eg the 3D corners of the encapsulating bounding box).&#10;To enhance the robustness,~\cite{hu2019segpose} and~\cite{peng2019pvnet} additionally conduct segmentation coupled with voting for each correspondence.&#10;However, the recent trend goes towards predicting dense rather than sparse correspondences~\cite{zakharov2019dpod,li2019cdpn}.&#10;Moreover, while~\cite{park2019pix2pose} leverages a GAN on top of dense correspondences to increase stability,~\cite{hodan2020epos} makes use of fragments in order to account for ambiguities in pose.&#10;&#10;Another orthogonal line of works aims at learning a latent embedding of pose which can be utilized for retrieval during inference.&#10;These embeddings are commonly either grounded on metric learning employing a triplet loss~\cite{wohlhart2015learning},&#10;or via training of an Auto-Encoder~\cite{sundermeyer2018AAE,Sundermeyer_2020_CVPR}.&#10;&#10;Direct Methods.&#10;Although indirect methods, leveraging 2D-3D correspondences, are currently performing better,&#10;they cannot be directly employed in many tasks, which require the pose estimation to be differentiable~\cite{wang2020self6d}.&#10;Hence, some methods directly regress the 6D pose, either leveraging a point matching loss~\cite{xiang2017posecnn,li2019deepim,labbe2020CosyPose}&#10;or employing separate loss terms for each component~\cite{manhardt2019ambiguity,lienet_do2018}.&#10;Other methods discretize the pose space and conduct classification rather than regression~\cite{kehl2017ssd6d}.&#10;A few methods also try to solve a proxy task during optimization.&#10;Thereby,~\cite{manhardt2018deep} proposes to employ an edge-alignment loss using the distance transform,&#10;while ~\cite{wang2020self6d} harnesses differentiable rendering to allow training on unlabeled samples.&#10;&#10;Differentiable Indirect Methods.&#10;Recently, a few works attempt to make P$n$P/RANSAC differentiable. In ~\cite{brachmann2017dsac,brachmann2018DSACpp,brachmann2019NG_RANSAC} the authors introduce a novel differentiable way to apply RANSAC via sharing of hypotheses based on the predicted distribution. &#10;Nonetheless, these approaches require a complex training strategy, as they expect a good initialization for the scene coordinates. &#10;As for P$n$P,&#10;~\cite{chen2020BPnP} employs the Implicit Function Theorem~\cite{krantz2012IFT} to enable the computation of analytical gradients \wrt the pose loss. Yet, it is computationally expensive especially given too many correspondences since P$n$P/RANSAC is still needed for both training and inference. &#10;Instead, ~\cite{hu2020single_stage} attempts to learn the P$n$P stage with a PointNet-based architecture ~\cite{qi2017pointnet} which learns to infer the 6D pose from a fixed set of sparse 2D-3D correspondences.&#10;&#10;Beyond Instance-level 6D Pose Estimation.&#10;Noteworthy, a few methods are recently trying to go beyond the instance-level scenario, even estimating the pose~\cite{wang2019nocs},&#10;sometimes paired with shape~\cite{chen2020learning,manhardt2020cpspp,park2020latentfusion}, for previously unseen objects.&#10; " />
-        </attvalues>
-      </node>
-      <node id="1906.05105" label="1906.05105">
-        <attvalues>
-          <attvalue for="0" value="Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects" />
-          <attvalue for="1" value="  Most deep pose estimation methods need to be trained for specific object&#10;instances or categories. In this work we propose a completely generic deep pose&#10;estimation approach, which does not require the network to have been trained on&#10;relevant categories, nor objects in a category to have a canonical pose. We&#10;believe this is a crucial step to design robotic systems that can interact with&#10;new objects in the wild not belonging to a predefined category. Our main&#10;insight is to dynamically condition pose estimation with a representation of&#10;the 3D shape of the target object. More precisely, we train a Convolutional&#10;Neural Network that takes as input both a test image and a 3D model, and&#10;outputs the relative 3D pose of the object in the input image with respect to&#10;the 3D model. We demonstrate that our method boosts performances for supervised&#10;category pose estimation on standard benchmarks, namely Pascal3D+, ObjectNet3D&#10;and Pix3D, on which we provide results superior to the state of the art. More&#10;importantly, we show that our network trained on everyday man-made objects from&#10;ShapeNet generalizes without any additional training to completely new types of&#10;3D objects by providing results on the LINEMOD dataset as well as on natural&#10;entities such as animals from ImageNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.04061" label="1711.04061">
-        <attvalues>
-          <attvalue for="0" value="Going Further with Point Pair Features" />
-          <attvalue for="1" value="  Point Pair Features is a widely used method to detect 3D objects in point&#10;clouds, however they are prone to fail in presence of sensor noise and&#10;background clutter. We introduce novel sampling and voting schemes that&#10;significantly reduces the influence of clutter and sensor noise. Our&#10;experiments show that with our improvements, PPFs become competitive against&#10;state-of-the-art methods as it outperforms them on several objects from&#10;challenging benchmarks, at a low computational cost.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.00175" label="1804.00175">
-        <attvalues>
-          <attvalue for="0" value="DeepIM: Deep Iterative Matching for 6D Pose Estimation" />
-          <attvalue for="1" value="  Estimating the 6D pose of objects from images is an important problem in&#10;various applications such as robot manipulation and virtual reality. While&#10;direct regression of images to object poses has limited accuracy, matching&#10;rendered images of an object against the observed image can produce accurate&#10;results. In this work, we propose a novel deep neural network for 6D pose&#10;matching named DeepIM. Given an initial pose estimation, our network is able to&#10;iteratively refine the pose by matching the rendered image against the observed&#10;image. The network is trained to predict a relative pose transformation using&#10;an untangled representation of 3D location and 3D orientation and an iterative&#10;training process. Experiments on two commonly used benchmarks for 6D pose&#10;estimation demonstrate that DeepIM achieves large improvements over&#10;state-of-the-art methods. We furthermore show that DeepIM is able to match&#10;previously unseen objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;We review representative works on 6D pose estimation in the literature.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2205.12257" label="2205.12257">
-        <attvalues>
-          <attvalue for="0" value="OnePose: One-Shot Object Pose Estimation without CAD Models" />
-          <attvalue for="1" value="  We propose a new method named OnePose for object pose estimation. Unlike&#10;existing instance-level or category-level methods, OnePose does not rely on CAD&#10;models and can handle objects in arbitrary categories without instance- or&#10;category-specific network training. OnePose draws the idea from visual&#10;localization and only requires a simple RGB video scan of the object to build a&#10;sparse SfM model of the object. Then, this model is registered to new query&#10;images with a generic feature matching network. To mitigate the slow runtime of&#10;existing visual localization methods, we propose a new graph attention network&#10;that directly matches 2D interest points in the query image with the 3D points&#10;in the SfM model, resulting in efficient and robust pose estimation. Combined&#10;with a feature-based pose tracker, OnePose is able to stably detect and track&#10;6D poses of everyday household objects in real-time. We also collected a&#10;large-scale dataset that consists of 450 sequences of 150 objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.07673" label="2301.07673">
-        <attvalues>
-          <attvalue for="0" value="OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD&#10;  Models" />
-          <attvalue for="1" value="  We propose a new method for object pose estimation without CAD models. The&#10;previous feature-matching-based method OnePose has shown promising results&#10;under a one-shot setting which eliminates the need for CAD models or&#10;object-specific training. However, OnePose relies on detecting repeatable image&#10;keypoints and is thus prone to failure on low-textured objects. We propose a&#10;keypoint-free pose estimation pipeline to remove the need for repeatable&#10;keypoint detection. Built upon the detector-free feature matching method LoFTR,&#10;we devise a new keypoint-free SfM method to reconstruct a semi-dense&#10;point-cloud model for the object. Given a query image for object pose&#10;estimation, a 2D-3D matching network directly establishes 2D-3D correspondences&#10;between the query image and the reconstructed point-cloud model without first&#10;detecting keypoints in the image. Experiments show that the proposed pipeline&#10;outperforms existing one-shot CAD-model-free methods by a large margin and is&#10;comparable to CAD-model-based methods on LINEMOD even for low-textured objects.&#10;We also collect a new dataset composed of 80 sequences of 40 low-textured&#10;objects to facilitate future research on one-shot object pose estimation. The&#10;supplementary material, code and dataset are available on the project page:&#10;https://zju3dv.github.io/onepose_plus_plus/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.11718" label="2210.11718">
-        <attvalues>
-          <attvalue for="0" value="CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement&#10;  Transformers" />
-          <attvalue for="1" value="  Learning based 6D object pose estimation methods rely on computing large&#10;intermediate pose representations and/or iteratively refining an initial&#10;estimation with a slow render-compare pipeline. This paper introduces a novel&#10;method we call Cascaded Pose Refinement Transformers, or CRT-6D. We replace the&#10;commonly used dense intermediate representation with a sparse set of features&#10;sampled from the feature pyramid we call OSKFs(Object Surface Keypoint&#10;Features) where each element corresponds to an object keypoint. We employ&#10;lightweight deformable transformers and chain them together to iteratively&#10;refine proposed poses over the sampled OSKFs. We achieve inference runtimes 2x&#10;faster than the closest real-time state of the art methods while supporting up&#10;to 21 objects on a single model. We demonstrate the effectiveness of CRT-6D by&#10;performing extensive experiments on the LM-O and YCBV datasets. Compared to&#10;real-time methods, we achieve state of the art on LM-O and YCB-V, falling&#10;slightly behind methods with inference runtimes one order of magnitude higher.&#10;The source code is available at: https://github.com/PedroCastro/CRT-6D&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Keypoint Detection. Object pose estimation can be seen as the inverse of camera pose estimation. One can extract 6D pose by solving the PnP problem which means we can detect the pixel position of keypoints, creating the necessary 2D-3D correspondence set. Early works started by choosing the 3D bounding box of the objects as keypoints \cite{yolo6d, bb8, heatmaps}. However, the projected 3D bounding box keypoints usually lie outside the silhouette of the objects, which potentially reduces the local information extraction. This shortfall was noticed by PVNet \cite{pvnet}, which suggests the use of the surface&#10;region to find suitable &#10;keypoints. &#10;&#10;Dense Object Coordinate Estimation. &#10;Instead of pre-selecting a few keypoints, NOCS~\cite{nocs} was proposed where for every pixel in the silhouette of the object is used to estimate the coordinate of the surface of the object (in normalized space) projected at that pixel. In other words, every point in the surface would become a keypoint and could be used for the 2D-3D correspondence set to solve PnP. Inspired by NOCS, Pix2Pose \cite{pix2pose} proposed the use of a GAN to solve issues with occlusion. DPOD \cite{dpod, epos, gdr, zebrapose} suggested using UV maps and object regions instead of a 3D coordinate system, ensuring that every point estimated lied within the object's surface.&#10;Each of these methods has an increasingly more complex model, and while performance has been improved by each method, runtime has been overlooked.&#10;&#10;Direct Pose Estimation. &#10;Posenet\cite{posenet} proposed learning quaternions to predict rotation on camera pose estimation tasks. In 6D object pose estimation field, PoseCNN \cite{posecnn} used Lie algebra instead. SSD-6D \cite{ssd6d} discretized the viewpoint space and learned to classify it, while using the mask to regress the distance to the camera. &#10;These methods are more susceptible to noise and occlusion due to their holistic approach. Moreover, these usually require an extra step to solve the ambiguity caused by egocentric orientations. Some methods learn a mapping from an intermediate representation to emulate a PnP solver, making them differentiable w.r.t. the final pose \cite{gdr, sopose, singlestage, bpnp}. This step can be used to reduce the need for symmetry hacks on dense methods. &#10;&#10;Pose Estimation Refinement&#10;If depth is available, then Iterative Closest Point (ICP) is the most commonly used algorithm \cite{icp}. The ICP algorithm finds the correspondence between points by iteratively refining the pose that takes to align them. However, it is heavily dependent on initial pose and might converge in a local minimum. Recent learned approaches mostly rely a render-compare pipeline, with slight variations among these methods \cite{deepim,dpod,cosypose}. More recently, Repose \cite{repose} introduced a fast iterative refinement algorithm however it requires a great initialization.&#10;&#10;Transformers in Pose Estimation. Given the rising effectiveness of transformers in computer vision tasks, there have been attempts to use transformers to improve human~\cite{human_transformer1, human_transformer2}, hand~\cite{hand_transformer1} and object~\cite{DProST, yolopose, 6dvit, osop, handobjecttransformer} pose estimation. For object pose, such approaches are aimed at improving results~\cite{DProST, yolopose}, category level estimation~\cite{6dvit, osop} or hand-object interaction~\cite{handobjecttransformer}. However, these improvements come at the cost of runtime, making them unsuitable for real-time applications. Our novel approach, not only improves results, but also decreases the runtime when compared to prior object pose estimation methods.&#10; " />
-        </attvalues>
-      </node>
-      <node id="2008.08465" label="2008.08465">
-        <attvalues>
-          <attvalue for="0" value="CosyPose: Consistent multi-view multi-object 6D pose estimation" />
-          <attvalue for="1" value="  We introduce an approach for recovering the 6D pose of multiple known objects&#10;in a scene captured by a set of input images with unknown camera viewpoints.&#10;First, we present a single-view single-object 6D pose estimation method, which&#10;we use to generate 6D object pose hypotheses. Second, we develop a robust&#10;method for matching individual 6D object pose hypotheses across different input&#10;images in order to jointly estimate camera viewpoints and 6D poses of all&#10;objects in a single consistent scene. Our approach explicitly handles object&#10;symmetries, does not require depth measurements, is robust to missing or&#10;incorrect object hypotheses, and automatically recovers the number of objects&#10;in the scene. Third, we develop a method for global scene refinement given&#10;multiple object hypotheses and their correspondences across views. This is&#10;achieved by solving an object-level bundle adjustment problem that refines the&#10;poses of cameras and objects to minimize the reprojection error in all views.&#10;We demonstrate that the proposed method, dubbed CosyPose, outperforms current&#10;state-of-the-art results for single-view and multi-view 6D object pose&#10;estimation by a large margin on two challenging benchmarks: the YCB-Video and&#10;T-LESS datasets. Code and pre-trained models are available on the project&#10;webpage https://www.di.ens.fr/willow/research/cosypose/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Our work builds on results in single-view and multi-view object 6D pose estimation&#10;from RGB images and object-level SLAM.&#10;&#10;Single-view single-object 6D pose estimation. &#10;The object pose estimation problem~\cite{Collet2010-zj,Collet2011-lj} has been approached either by estimating the pose from 2D-3D correspondences using local invariant features~\cite{Lowe1999-bf,Bay2006-hj}, or directly by estimating the object pose using template-matching~\cite{Hinterstoisser2011-es}. However, local features do not work well for texture-less objects and global templates often fail to detect partially occluded objects. &#10;Both of these approaches (feature-based and template&#10;matching) have been revisited using deep neural networks. &#10;A convolutional neural network (CNN) can be used to detect object&#10;features in 2D~\cite{Rad2017-de,Kehl2017-ek,Tremblay2018-bd,Tekin2017-hp,Xiang2018-dv} or to&#10;directly find 2D-to-3D correspondences~\cite{Park2019-od,Zakharov2019-fx,Peng2018-ev,Pitteri2019-sn}.&#10;Deep approaches have also been used to match implicit pose features, which can&#10;be learned without requiring ground truth pose annotations~\cite{Sundermeyer2018-es}. &#10;The estimated 6D pose of the objects can be further refined~\cite{Rad2017-de,Li2018-fp} using an iterative procedure that effectively moves the camera around the object&#10;so that the rendered image of the object best matches the input image. Such a refinement step provides important performance improvements and is becoming common practice~\cite{Zakharov2019-fx,Wang2019-bb} as a final stage of the estimation&#10;process. Our single-view single-object pose estimation described in Section~\ref{sec:single-view} builds on DeepIM&#10;\cite{Li2018-fp}.&#10;The performance of 6D pose estimation can be further improved using depth sensors~\cite{Li2018-fp,Xiang2018-dv,Wang2019-bb}, but in this work we focus on the most challenging scenario where only RGB images are available. &#10;&#10;Multi-view single-object 6D pose estimation.&#10;Multiple views of an object can be used to resolve depth ambiguities and&#10;gain robustness with respect to occlusions. Prior work using local invariant features includes~\cite{Collet2010-zj,Collet2011-lj,Grossberg2001-oh,Pless2003-ae} and involves some form of feature matching to establish correspondences across views to aggregate information from multiple viewpoints. &#10;More recently, the multi-view&#10;single-object pose estimation problem has been revisited with a deep neural network&#10;that predicts an object pose candidate in each view~\cite{Li2018-vl} and aggregates information from multiple views assuming known camera poses.&#10;In contrast, our work does not assume the camera poses to be known.&#10;We experimentally demonstrate that our approach outperforms~\cite{Li2018-vl} despite requiring less information. &#10;&#10;Multi-view multi-object 6D pose estimation. Other works consider all&#10;objects in a scene together in order to jointly estimate the state of the scene&#10;in the form of a compact representation of the object and camera poses in a common coordinate system. This problem is known as object-level&#10;SLAM~\cite{Salas-Moreno2013-mt} where a depth-based object pose estimation method~\cite{Drost2010-zr} is used to recognize&#10;objects from a database in individual images and estimate their poses. The individual objects are tracked across frames using depth measurements, assuming the motion of the sensor is continuous.&#10;Consecutive depth measurements also enable to produce hypotheses for camera poses using ICP \cite{zhang1994iterative} and the poses of objects and cameras are finally refined in a joint optimization procedure.Another approach \cite{doumanoglou2016recovering} uses local RGBD patches to generate object hypotheses and find the best view of a scene. All of these methods, however, strongly rely on depth sensors to estimate the 3D structure of the scene while our method only exploits RGB images. In addition, they assume temporal continuity between the views, which is also not required by our approach.&#10;&#10;Other works have considered monocular RGB only object-level SLAM \cite{bao2011semantic,pillai2015monocular,yang2019cubeslam}.&#10; Related is also \cite{bachmann2019motion} where semantic 2D keypoint correspondences across multiple views and local features are used to jointly estimate the pose&#10; of a single human and the positions of the observing cameras. All of these works rely on local images features to estimate camera poses.&#10; In contrast, our work exploits 6D pose hypotheses generated by a neural&#10; network which allows to recover camera poses in situations where feature-based registration fails, as is the case for example for the complex texture-less images of the T-LESS dataset.&#10; In addition, \cite{pillai2015monocular,yang2019cubeslam} do not consider full 6D pose of objects, and&#10; \cite{bachmann2019motion,Li2018-vl} only consider scenes with a single instance of each object. In contrast, our method is able to handle scenes with multiple instances of the same object.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2211.03594" label="2211.03594">
-        <attvalues>
-          <attvalue for="0" value="Group DETR v2: Strong Object Detector with Encoder-Decoder Pretraining" />
-          <attvalue for="1" value="  We present a strong object detector with encoder-decoder pretraining and&#10;finetuning. Our method, called Group DETR v2, is built upon a vision&#10;transformer encoder ViT-Huge~\cite{dosovitskiy2020image}, a DETR variant&#10;DINO~\cite{zhang2022dino}, and an efficient DETR training method Group&#10;DETR~\cite{chen2022group}. The training process consists of self-supervised&#10;pretraining and finetuning a ViT-Huge encoder on ImageNet-1K, pretraining the&#10;detector on Object365, and finally finetuning it on COCO. Group DETR v2&#10;achieves $\textbf{64.5}$ mAP on COCO test-dev, and establishes a new SoTA on&#10;the COCO leaderboard https://paperswithcode.com/sota/object-detection-on-coco&#10;" />
-          <attvalue for="2" value=" &#10;&#10;Recent studies show the effectiveness of transformer models at scale~\cite{dosovitskiy2020image,zhai2022scaling,liu2022swin}.&#10;With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results &#10;on various vision tasks, including object detection. &#10;With supervised encoder-decoder pretraining on a large-scale dataset, Object365~\cite{shao2019objects365}, DINO~\cite{zhang2022dino} achieves a state-of-the-art result on COCO~\cite{lin2014microsoft}.&#10;&#10;Our method, {Group DETR v2}, is built upon ViT-Huge, DINO, and Group DETR. &#10;We adopt an encoder-decoder pretraining and finetuning pipeline: &#10;pretraining and then&#10;finetuning a ViT-Huge encoder on ImageNet-1K~\cite{deng2009imagenet}, &#10;pretraining the detector, both the encoder and the decoder, on Object365, &#10;and finally finetuning it on COCO. &#10;Group DETR v2 achieves $64.5$ mAP on COCO test-dev~\cite{lin2014microsoft} (Table~\ref{tab:leaderboard} and Table~\ref{tab:coco_dev}), setting a new record for COCO object detection. &#10;We expect that the results will be further improved with more data and larger models. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2104.10972" label="2104.10972">
-        <attvalues>
-          <attvalue for="0" value="ImageNet-21K Pretraining for the Masses" />
-          <attvalue for="1" value="  ImageNet-1K serves as the primary dataset for pretraining deep learning&#10;models for computer vision tasks. ImageNet-21K dataset, which is bigger and&#10;more diverse, is used less frequently for pretraining, mainly due to its&#10;complexity, low accessibility, and underestimation of its added value. This&#10;paper aims to close this gap, and make high-quality efficient pretraining on&#10;ImageNet-21K available for everyone. Via a dedicated preprocessing stage,&#10;utilization of WordNet hierarchical structure, and a novel training scheme&#10;called semantic softmax, we show that various models significantly benefit from&#10;ImageNet-21K pretraining on numerous datasets and tasks, including small&#10;mobile-oriented models. We also show that we outperform previous ImageNet-21K&#10;pretraining schemes for prominent new models like ViT and Mixer. Our proposed&#10;pretraining pipeline is efficient, accessible, and leads to SoTA reproducible&#10;results, from a publicly available dataset. The training code and pretrained&#10;models are available at: https://github.com/Alibaba-MIIL/ImageNet21K&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.08254" label="2106.08254">
-        <attvalues>
-          <attvalue for="0" value="BEiT: BERT Pre-Training of Image Transformers" />
-          <attvalue for="1" value="  We introduce a self-supervised vision representation model BEiT, which stands&#10;for Bidirectional Encoder representation from Image Transformers. Following&#10;BERT developed in the natural language processing area, we propose a masked&#10;image modeling task to pretrain vision Transformers. Specifically, each image&#10;has two views in our pre-training, i.e, image patches (such as 16x16 pixels),&#10;and visual tokens (i.e., discrete tokens). We first &quot;tokenize&quot; the original&#10;image into visual tokens. Then we randomly mask some image patches and fed them&#10;into the backbone Transformer. The pre-training objective is to recover the&#10;original visual tokens based on the corrupted image patches. After pre-training&#10;BEiT, we directly fine-tune the model parameters on downstream tasks by&#10;appending task layers upon the pretrained encoder. Experimental results on&#10;image classification and semantic segmentation show that our model achieves&#10;competitive results with previous pre-training methods. For example, base-size&#10;BEiT achieves 83.2% top-1 accuracy on ImageNet-1K, significantly outperforming&#10;from-scratch DeiT training (81.8%) with the same setup. Moreover, large-size&#10;BEiT obtains 86.3% only using ImageNet-1K, even outperforming ViT-L with&#10;supervised pre-training on ImageNet-22K (85.2%). The code and pretrained models&#10;are available at https://aka.ms/beit.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09886" label="2111.09886">
-        <attvalues>
-          <attvalue for="0" value="SimMIM: A Simple Framework for Masked Image Modeling" />
-          <attvalue for="1" value="  This paper presents SimMIM, a simple framework for masked image modeling. We&#10;simplify recently proposed related approaches without special designs such as&#10;block-wise masking and tokenization via discrete VAE or clustering. To study&#10;what let the masked image modeling task learn good representations, we&#10;systematically study the major components in our framework, and find that&#10;simple designs of each component have revealed very strong representation&#10;learning performance: 1) random masking of the input image with a moderately&#10;large masked patch size (e.g., 32) makes a strong pre-text task; 2) predicting&#10;raw pixels of RGB values by direct regression performs no worse than the patch&#10;classification approaches with complex designs; 3) the prediction head can be&#10;as light as a linear layer, with no worse performance than heavier ones. Using&#10;ViT-B, our approach achieves 83.8% top-1 fine-tuning accuracy on ImageNet-1K by&#10;pre-training also on this dataset, surpassing previous best approach by +0.6%.&#10;When applied on a larger model of about 650 million parameters, SwinV2-H, it&#10;achieves 87.1% top-1 accuracy on ImageNet-1K using only ImageNet-1K data. We&#10;also leverage this approach to facilitate the training of a 3B model&#10;(SwinV2-G), that by $40\times$ less data than that in previous practice, we&#10;achieve the state-of-the-art on four representative vision benchmarks. The code&#10;and models will be publicly available at https://github.com/microsoft/SimMIM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.03026" label="2202.03026">
-        <attvalues>
-          <attvalue for="0" value="Context Autoencoder for Self-Supervised Representation Learning" />
-          <attvalue for="1" value="  We present a novel masked image modeling (MIM) approach, context autoencoder&#10;(CAE), for self-supervised representation pretraining. We pretrain an encoder&#10;by making predictions in the encoded representation space. The pretraining&#10;tasks include two tasks: masked representation prediction - predict the&#10;representations for the masked patches, and masked patch reconstruction -&#10;reconstruct the masked patches. The network is an encoder-regressor-decoder&#10;architecture: the encoder takes the visible patches as input; the regressor&#10;predicts the representations of the masked patches, which are expected to be&#10;aligned with the representations computed from the encoder, using the&#10;representations of visible patches and the positions of visible and masked&#10;patches; the decoder reconstructs the masked patches from the predicted encoded&#10;representations. The CAE design encourages the separation of learning the&#10;encoder (representation) from completing the pertaining tasks: masked&#10;representation prediction and masked patch reconstruction tasks, and making&#10;predictions in the encoded representation space empirically shows the benefit&#10;to representation learning. We demonstrate the effectiveness of our CAE through&#10;superior transfer performance in downstream tasks: semantic segmentation,&#10;object detection and instance segmentation, and classification. The code will&#10;be available at https://github.com/Atten4Vis/CAE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.06366" label="2208.06366">
-        <attvalues>
-          <attvalue for="0" value="BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers" />
-          <attvalue for="1" value="  Masked image modeling (MIM) has demonstrated impressive results in&#10;self-supervised representation learning by recovering corrupted image patches.&#10;However, most existing studies operate on low-level image pixels, which hinders&#10;the exploitation of high-level semantics for representation models. In this&#10;work, we propose to use a semantic-rich visual tokenizer as the reconstruction&#10;target for masked prediction, providing a systematic way to promote MIM from&#10;pixel-level to semantic-level. Specifically, we propose vector-quantized&#10;knowledge distillation to train the tokenizer, which discretizes a continuous&#10;semantic space to compact codes. We then pretrain vision Transformers by&#10;predicting the original visual tokens for the masked image patches.&#10;Furthermore, we introduce a patch aggregation strategy which associates&#10;discrete image patches to enhance global semantic representation. Experiments&#10;on image classification and semantic segmentation show that BEiT v2 outperforms&#10;all compared MIM methods. On ImageNet-1K (224 size), the base-size BEiT v2&#10;achieves 85.5% top-1 accuracy for fine-tuning and 80.1% top-1 accuracy for&#10;linear probing. The large-size BEiT v2 obtains 87.3% top-1 accuracy for&#10;ImageNet-1K (224 size) fine-tuning, and 56.7% mIoU on ADE20K for semantic&#10;segmentation. The code and pretrained models are available at&#10;https://aka.ms/beitv2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.10442" label="2208.10442">
-        <attvalues>
-          <attvalue for="0" value="Image as a Foreign Language: BEiT Pretraining for All Vision and&#10;  Vision-Language Tasks" />
-          <attvalue for="1" value="  A big convergence of language, vision, and multimodal pretraining is&#10;emerging. In this work, we introduce a general-purpose multimodal foundation&#10;model BEiT-3, which achieves state-of-the-art transfer performance on both&#10;vision and vision-language tasks. Specifically, we advance the big convergence&#10;from three aspects: backbone architecture, pretraining task, and model scaling&#10;up. We introduce Multiway Transformers for general-purpose modeling, where the&#10;modular architecture enables both deep fusion and modality-specific encoding.&#10;Based on the shared backbone, we perform masked &quot;language&quot; modeling on images&#10;(Imglish), texts (English), and image-text pairs (&quot;parallel sentences&quot;) in a&#10;unified manner. Experimental results show that BEiT-3 obtains state-of-the-art&#10;performance on object detection (COCO), semantic segmentation (ADE20K), image&#10;classification (ImageNet), visual reasoning (NLVR2), visual question answering&#10;(VQAv2), image captioning (COCO), and cross-modal retrieval (Flickr30K, COCO).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.14284" label="2302.14284">
-        <attvalues>
-          <attvalue for="0" value="Rethink Long-tailed Recognition with Vision Transformers" />
-          <attvalue for="1" value="  In the real world, data tends to follow long-tailed distributions w.r.t.&#10;class or attribution, motivating the challenging Long-Tailed Recognition (LTR)&#10;problem. In this paper, we revisit recent LTR methods with promising Vision&#10;Transformers (ViT). We figure out that 1) ViT is hard to train with long-tailed&#10;data. 2) ViT learns generalized features in an unsupervised manner, like mask&#10;generative training, either on long-tailed or balanced datasets. Hence, we&#10;propose to adopt unsupervised learning to utilize long-tailed data.&#10;Furthermore, we propose the Predictive Distribution Calibration (PDC) as a&#10;novel metric for LTR, where the model tends to simply classify inputs into&#10;common classes. Our PDC can measure the model calibration of predictive&#10;preferences quantitatively. On this basis, we find many LTR approaches&#10;alleviate it slightly, despite the accuracy improvement. Extensive experiments&#10;on benchmark datasets validate that PDC reflects the model's predictive&#10;preference precisely, which is consistent with the visualization.&#10;" />
-          <attvalue for="2" value="&#10;With rapid advances in visual classification, deep models tend to depend on balanced large-scale datasets more seriously \cite{Imagenet, COCO}. However, the number of instances in real-world data usually follows a Long-Tailed (LT) distribution w.r.t. class. Many tail classes are associated with limited samples, while a few head categories occupy most of the instances \cite{CB-loss, OLTR, PriorLT, icasspunbalance}. The model supervised by long-tailed data tends to bias toward the head classes and ignore the tail ones. The tail data paucity makes the model hard to train with satisfying generalization. It is still a challenging task to overcome Long Tailed Recognition (LTR) and utilize real-world data effectively.&#10;&#10;Recent literature mainly adopt two approaches to tackle LT data, i.e., feature re-sampling and class-wise re-weighting. The re-sampling methods balanced select the training data by over-sampling the tail or under-sampling the head. Some effective proposals replenish the tail samples via generation or optimization with the help of head instances\cite{M2m, Bagoftricks}. The re-weighting ones punish different categories with data number relevant weight or logit bias\cite{BalCE, LADE-loss}. Although the aforementioned methods have greatly mitigated the LT problem, the conclusions hold on the ResNet-based backbones \cite{MiSLAS, IB-loss}.&#10;&#10;In recent years, many transformer-based backbones \cite{vit} have surpassed the performance of CNN. DeiT\cite{deit3} proposes an effective receipt to train ViT with limited data, and MAE\cite{MAE} adopts a masked autoencoder to pre-train the ViT. However, there is limited research on how ViTs perform on LTR. Motivated by this, we rethink the previous LT works with ViT. We figure out that it is hard to train ViTs with long-tailed data while the unsupervised pretraining manner ameliorates it by a large margin. The unsupervised pretraining ViTs will learn meaningful feature (c.f. Figure \ref{fig:mae-pretrain}) and generalize well on downstream tasks (c.f. Table \ref{tab:recipe}), either on long-tailed or balanced datasets.&#10;&#10;Numerous studies have demonstrated that the model supervised by the LT dataset will inevitably exhibit prediction bias to the head\cite{PriorLT, LA, LADE, icassphead}. The predictor will simply classify the inquiry image to the head to attain a low misclassification error. The previous metrics, like accuracy on the validation dataset, are difficult to evaluate the model's predictive preference directly. The same accuracy may come at the cost of a different number of predictions (c.f. Figure \ref{fig:teaser}). Although some works show models' prediction distribution by visualization qualitatively\cite{PriorLT, MiSLAS, NCL}, a metric is required to evaluate it quantitatively. In this paper, we propose Prediction Distribution Calibration (PDC) to fill this gap. Specifically, if we view the prediction number and target instance number of each class as probability distributions, we can measure the distance between the two probability distributions. Considering the imbalance degree of training samples, we take the training label into account as well. To summarize, our main contributions are:&#10;&#10;1) We figure out that it is difficult to train ViT with long-tailed data, which can be tackled with unsupervised pretraining.&#10;&#10;2) We propose PDC to provide a quantitative view to measure how the proposal ameliorates the model predictive preference.&#10;&#10;3) We conduct extensive experiments to analyze LTR proposals' performance on ViT with our proposed PDC, which will accurately indicate the model's predictive bias and is consistent with the visualization results.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Vision Transformers, Predictive Distribution Calibration, Mathematics, Artificial Intelligence, Computer Science, Long-Tailed Recognition, Statistics" />
-        </attvalues>
-      </node>
-      <node id="1901.05555" label="1901.05555">
-        <attvalues>
-          <attvalue for="0" value="Class-Balanced Loss Based on Effective Number of Samples" />
-          <attvalue for="1" value="  With the rapid increase of large-scale, real-world datasets, it becomes&#10;critical to address the problem of long-tailed data distribution (i.e., a few&#10;classes account for most of the data, while most classes are&#10;under-represented). Existing solutions typically adopt class re-balancing&#10;strategies such as re-sampling and re-weighting based on the number of&#10;observations for each class. In this work, we argue that as the number of&#10;samples increases, the additional benefit of a newly added data point will&#10;diminish. We introduce a novel theoretical framework to measure data overlap by&#10;associating with each sample a small neighboring region rather than a single&#10;point. The effective number of samples is defined as the volume of samples and&#10;can be calculated by a simple formula $(1-\beta^{n})/(1-\beta)$, where $n$ is&#10;the number of samples and $\beta \in [0,1)$ is a hyperparameter. We design a&#10;re-weighting scheme that uses the effective number of samples for each class to&#10;re-balance the loss, thereby yielding a class-balanced loss. Comprehensive&#10;experiments are conducted on artificially induced long-tailed CIFAR datasets&#10;and large-scale datasets including ImageNet and iNaturalist. Our results show&#10;that when trained with the proposed class-balanced loss, the network is able to&#10;achieve significant performance gains on long-tailed datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00431" label="2004.00431">
-        <attvalues>
-          <attvalue for="0" value="M2m: Imbalanced Classification via Major-to-minor Translation" />
-          <attvalue for="1" value="  In most real-world scenarios, labeled training datasets are highly&#10;class-imbalanced, where deep neural networks suffer from generalizing to a&#10;balanced testing criterion. In this paper, we explore a novel yet simple way to&#10;alleviate this issue by augmenting less-frequent classes via translating&#10;samples (e.g., images) from more-frequent classes. This simple approach enables&#10;a classifier to learn more generalizable features of minority classes, by&#10;transferring and leveraging the diversity of the majority information. Our&#10;experimental results on a variety of class-imbalanced datasets show that the&#10;proposed method improves the generalization on minority classes significantly&#10;compared to other existing re-sampling or re-weighting methods. The performance&#10;of our method even surpasses those of previous state-of-the-art methods for the&#10;imbalanced classification.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.10740" label="2007.10740">
-        <attvalues>
-          <attvalue for="0" value="Balanced Meta-Softmax for Long-Tailed Visual Recognition" />
-          <attvalue for="1" value="  Deep classifiers have achieved great success in visual recognition. However,&#10;real-world data is long-tailed by nature, leading to the mismatch between&#10;training and testing distributions. In this paper, we show that the Softmax&#10;function, though used in most classification tasks, gives a biased gradient&#10;estimation under the long-tailed setup. This paper presents Balanced Softmax,&#10;an elegant unbiased extension of Softmax, to accommodate the label distribution&#10;shift between training and testing. Theoretically, we derive the generalization&#10;bound for multiclass Softmax regression and show our loss minimizes the bound.&#10;In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta&#10;Sampler to estimate the optimal class sample rate and further improve&#10;long-tailed learning. In our experiments, we demonstrate that Balanced&#10;Meta-Softmax outperforms state-of-the-art long-tailed classification solutions&#10;on both visual recognition and instance segmentation tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00466" label="2104.00466">
-        <attvalues>
-          <attvalue for="0" value="Improving Calibration for Long-Tailed Recognition" />
-          <attvalue for="1" value="  Deep neural networks may perform poorly when training datasets are heavily&#10;class-imbalanced. Recently, two-stage methods decouple representation learning&#10;and classifier learning to improve performance. But there is still the vital&#10;issue of miscalibration. To address it, we design two methods to improve&#10;calibration and performance in such scenarios. Motivated by the fact that&#10;predicted probability distributions of classes are highly related to the&#10;numbers of class instances, we propose label-aware smoothing to deal with&#10;different degrees of over-confidence for classes and improve classifier&#10;learning. For dataset bias between these two stages due to different samplers,&#10;we further propose shifted batch normalization in the decoupling framework. Our&#10;proposed methods set new records on multiple popular long-tailed recognition&#10;benchmark datasets, including CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT,&#10;Places-LT, and iNaturalist 2018. Code will be available at&#10;https://github.com/Jia-Research-Lab/MiSLAS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.07118" label="2204.07118">
-        <attvalues>
-          <attvalue for="0" value="DeiT III: Revenge of the ViT" />
-          <attvalue for="1" value="  A Vision Transformer (ViT) is a simple neural architecture amenable to serve&#10;several computer vision tasks. It has limited built-in architectural priors, in&#10;contrast to more recent architectures that incorporate priors either about the&#10;input data or of specific tasks. Recent works show that ViTs benefit from&#10;self-supervised pre-training, in particular BerT-like pre-training like BeiT.&#10;In this paper, we revisit the supervised training of ViTs. Our procedure builds&#10;upon and simplifies a recipe introduced for training ResNet-50. It includes a&#10;new simple data-augmentation procedure with only 3 augmentations, closer to the&#10;practice in self-supervised learning. Our evaluations on Image classification&#10;(ImageNet-1k with and without pre-training on ImageNet-21k), transfer learning&#10;and semantic segmentation show that our procedure outperforms by a large margin&#10;previous fully supervised training recipes for ViT. It also reveals that the&#10;performance of our ViT trained with supervision is comparable to that of more&#10;recent architectures. Our results could serve as better baselines for recent&#10;self-supervised approaches demonstrated on ViT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.03874" label="2111.03874">
-        <attvalues>
-          <attvalue for="0" value="Towards Calibrated Model for Long-Tailed Visual Recognition from Prior&#10;  Perspective" />
-          <attvalue for="1" value="  Real-world data universally confronts a severe class-imbalance problem and&#10;exhibits a long-tailed distribution, i.e., most labels are associated with&#10;limited instances. The na\&quot;ive models supervised by such datasets would prefer&#10;dominant labels, encounter a serious generalization challenge and become poorly&#10;calibrated. We propose two novel methods from the prior perspective to&#10;alleviate this dilemma. First, we deduce a balance-oriented data augmentation&#10;named Uniform Mixup (UniMix) to promote mixup in long-tailed scenarios, which&#10;adopts advanced mixing factor and sampler in favor of the minority. Second,&#10;motivated by the Bayesian theory, we figure out the Bayes Bias (Bayias), an&#10;inherent bias caused by the inconsistency of prior, and compensate it as a&#10;modification on standard cross-entropy loss. We further prove that both the&#10;proposed methods ensure the classification calibration theoretically and&#10;empirically. Extensive experiments verify that our strategies contribute to a&#10;better-calibrated model, and their combination achieves state-of-the-art&#10;performance on CIFAR-LT, ImageNet-LT, and iNaturalist 2018.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.11201" label="2308.11201">
-        <attvalues>
-          <attvalue for="0" value="Masked Cross-image Encoding for Few-shot Segmentation" />
-          <attvalue for="1" value="  Few-shot segmentation (FSS) is a dense prediction task that aims to infer the&#10;pixel-wise labels of unseen classes using only a limited number of annotated&#10;images. The key challenge in FSS is to classify the labels of query pixels&#10;using class prototypes learned from the few labeled support exemplars. Prior&#10;approaches to FSS have typically focused on learning class-wise descriptors&#10;independently from support images, thereby ignoring the rich contextual&#10;information and mutual dependencies among support-query features. To address&#10;this limitation, we propose a joint learning method termed Masked Cross-Image&#10;Encoding (MCE), which is designed to capture common visual properties that&#10;describe object details and to learn bidirectional inter-image dependencies&#10;that enhance feature interaction. MCE is more than a visual representation&#10;enrichment module; it also considers cross-image mutual dependencies and&#10;implicit guidance. Experiments on FSS benchmarks PASCAL-$5^i$ and COCO-$20^i$&#10;demonstrate the advanced meta-learning ability of the proposed method.&#10;" />
-          <attvalue for="2" value="&#10;Semantic segmentation is a pixel-wise classification problem that aims to predict the class label of each pixel in an image, a task that typically requires a large number of finely labeled images for supervised learning. However, obtaining sufficient image data to train a reliable segmentation model is both expensive and labor-intensive. To address this challenge and reduce the annotation workload, few-shot segmentation methods \cite{RN4, MLC, Part-aware-prototype} have been proposed to learn segmenting previously unseen classes with only one or a few labeled training images, a problem commonly referred to as one-shot or Few-Shot Segmentation (FSS).&#10;&#10;FSS learning methods generally follow the ``learning to learn'' paradigm, also known as meta-learning \cite{Siamese, tian2020rethinking}, to obtain generalized prototypes that can describe the classes of interest. &#10;These methods typically apply a two-branch framework~\cite{RN4, RN5,RN10} with a shared frozen pre-trained backbone, where a support branch is used to learn the object prototypes from very few labeled images (support images) and a query branch is utilized to make predictions for a query image on conditioned of the support prototypes. &#10;Under this framework, masked average pooling (MAP) \cite{ RN5, PANet, RN10} and clustering-based aggregation methods \cite{RN6, ASGNet, Part-aware-prototype} are two popular approaches to learning the representations for support images.While both prototype-based and metric-based approaches have shown promise for few-shot segmentation (FSS), they primarily focus on capturing the overall visual properties of a class and do not effectively incorporate contextual information from a spatial perspective. This can result in a failure to guide features that correspond to the local context of support images, leading to decreased segmentation performance. To overcome this limitation, alignment schemes for FSS employ &quot;features matching&quot; and &quot;decoder optimization&quot; strategies to learn the metric similarities between support and query features extracted from the backbone. Specifically, methods such as \cite{PFENet, ASGNet, MLC} calculate prior maps, or similarity metrics, based on the query and support features to define their correlations. Other methods, including \cite{CWTans, CyCTR, HSNet}, feed support and query features into a decoder (e.g., 4D convolution or transformer) to perform local region matching between query and support images. However, as support and query features are generated independently through a fixed backbone, these alignment methods can still suffer from underdeveloped contextual information mining for support-query pairs that contain the same objects.&#10;&#10;It is widely acknowledged that deep learning features extracted from different images of identical objects share similar representations compared to features of other objects. This property makes it more effective to encode an image with features from others that contain the same object when only a limited number of samples are provided. This approach enables better exploration of the semantic representation among features across different image sources and preserves more detailed local contexts that aid in identifying subtle targets. Notably, the FSS two-branch framework naturally provides different image sources containing the same object, namely, the support images and the query image, whose visual features were extracted separately in prior methods. Building on this insight, we observe that self-attention in the Vision Transformer (ViT) \cite{ViT} and cross-attention in \cite{crossattention} can be utilized to capture contextual information of images during token dependency construction. Consequently, we adopt a similar approach to consider joint learning between query and support features for FSS. Specifically, we model cross-image object semantic encoding to identify discriminative local regions, as illustrated in Figure \ref{fig:fig1} (b).&#10;&#10;The overall framework of the proposed method is illustrated in Figure \ref{fig:fig2}. The method aims to capture the object semantic mutual relations across support and query images. Unlike CyCTR \cite{CyCTR} which employs Transformer blocks to pass support features to the query decoder, our approach emphasizes the importance of consistent mutual relations between query and support features. To achieve this, we propose a symmetric cross-attention structure called Masked Cross-Image Encoding (MCE), which is designed to assemble bidirectional inter-image relations on multi-level features. The MCE incorporates the support segmentation mask to restrict attention within the localized features of the target objects, thus enhancing the ability to distinguish objects from backgrounds. Additionally, we utilize multi-level features of the support-query images to calculate similarity score matrices, which provide a comprehensive understanding of the correspondence between each position of query features and the support object. Thus, these similarity score matrices help refine the pixel-wise classification accuracy in FSS.&#10;&#10;We evaluated the meta-learning ability of our model on two public FSS benchmarks, PASCAL-$5^i$ \cite{VOC} and COCO-$20^i$ \cite{COCO}. The experimental results show that the proposed masked cross-attention encoding helps enrich query features by attending support object regions mutually and, therefore, obtains a strong meta-learning ability, surpassing the prior counterpart methods. In summary, the main contributions of this work are as follows:&#10;\begin{itemize}&#10;\item We propose a masked cross-image encoding method to discover shared visual representations of the target objects in support and query features. By using a symmetric cross-attention structure, MCE can attend to bidirectional inter-image relations on multi-level features, which not only enriches the query features with information from the support object regions but also enhances the support-query interaction, leading to a more favorable meta-learning capability for FSS.&#10;\item We propose to calculate support-query similarity score matrices that reflect the likelihood of a pixel in query features belonging to the foreground. These matrices are then incorporated into our model along with multi-level cross-image features to facilitate final segmentation.&#10;\item Extensive experiments on PASCAL-$5^i$ and COCO-$20^i$ benchmarks under 1-shot and 5-shot settings demonstrate the effectiveness of the proposed MCE and similarity score matrices. The proposed model achieves superior meta-learning performance across all compared state-of-the-art methods. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1903.02351" label="1903.02351">
-        <attvalues>
-          <attvalue for="0" value="CANet: Class-Agnostic Segmentation Networks with Iterative Refinement&#10;  and Attentive Few-Shot Learning" />
-          <attvalue for="1" value="  Recent progress in semantic segmentation is driven by deep Convolutional&#10;Neural Networks and large-scale labeled image datasets. However, data labeling&#10;for pixel-wise segmentation is tedious and costly. Moreover, a trained model&#10;can only make predictions within a set of pre-defined classes. In this paper,&#10;we present CANet, a class-agnostic segmentation network that performs few-shot&#10;segmentation on new classes with only a few annotated images available. Our&#10;network consists of a two-branch dense comparison module which performs&#10;multi-level feature comparison between the support image and the query image,&#10;and an iterative optimization module which iteratively refines the predicted&#10;results. Furthermore, we introduce an attention mechanism to effectively fuse&#10;information from multiple support examples under the setting of k-shot&#10;learning. Experiments on PASCAL VOC 2012 show that our method achieves a mean&#10;Intersection-over-Union score of 55.4% for 1-shot segmentation and 57.1% for&#10;5-shot segmentation, outperforming state-of-the-art methods by a large margin&#10;of 14.6% and 13.2%, respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.03898" label="2008.03898">
-        <attvalues>
-          <attvalue for="0" value="Prototype Mixture Models for Few-shot Semantic Segmentation" />
-          <attvalue for="1" value="  Few-shot segmentation is challenging because objects within the support and&#10;query images could significantly differ in appearance and pose. Using a single&#10;prototype acquired directly from the support image to segment the query image&#10;causes semantic ambiguity. In this paper, we propose prototype mixture models&#10;(PMMs), which correlate diverse image regions with multiple prototypes to&#10;enforce the prototype-based semantic representation. Estimated by an&#10;Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and&#10;spatial semantics from limited support images. Utilized as representations as&#10;well as classifiers, PMMs fully leverage the semantics to activate objects in&#10;the query image while depressing background regions in a duplex manner.&#10;Extensive experiments on Pascal VOC and MS-COCO datasets show that PMMs&#10;significantly improve upon state-of-the-arts. Particularly, PMMs improve 5-shot&#10;segmentation performance on MS-COCO by up to 5.82\% with only a moderate cost&#10;for model size and inference speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Semantic Segmentation.&#10;Semantic segmentation, which performs per-pixel classification of a class of objects, has been extensively investigated. State-of-the-art methods, such as UNet~\cite{UNet}, PSPNet~\cite{PSPNet}, DeepLab~\cite{DeepLabV1,DeepLabV2,DeepLabV3}, are based on fully convolutional networks (FCNs)~\cite{FCN}. Semantic segmentation has been updated to instance segmentation~\cite{MaskRCNN-PAMI2020} and panoptic segmentation ~\cite{PanopticFPN-CVPR2019}, which shared useful modules, $e.g.$, Atrous Spatial Pyramid Pooling (ASPP)~\cite{DeepLabV2} and multi-scale feature aggregation~\cite{PSPNet}, with few-shot segmentation. The clustering method used in SegSort~\cite{SegSort}, which partitioned objects into parts using a divide-and-conquer strategy, provides an insight for this study.&#10;&#10;Few-shot Learning.&#10;Existing methods can be broadly categorized as either: metric learning~\cite{MatchNetwork16,Compare2018,CollectSelect19}, meta-learning ~\cite{LearningToLearn16,Optimization17,MAML17,TaskAgnosticMeta19}, or data argumentation.&#10;Metric learning based methods train networks to predict whether two images/regions belong to the same category. Meta-learning based approaches specify optimization or loss functions which force faster adaptation of the parameters to new categories with few examples. The data argumentation methods learn to generate additional examples for unseen categories~\cite{Hallucinating17,Imaginary18}. &#10;&#10;In the metric learning framework, the effect of prototypes for few-shot learning has been demonstrated. With a simple prototype, $e.g.$, a linear layer learned on top of a frozen CNN~\cite{CloserLook19}, state-of-the-art results can be achieved based on a simple baseline. This provides reason for applying prototypes to capture representative and discriminative features.&#10;&#10;Few-shot Segmentation. &#10;Existing few-shot segmentation approaches largely followed the metric learning framework, $e.g.$, learning knowledge using a prototype vector, from a set of support images, and then feed learned knowledge to a metric module to segment query images~\cite{PANet}.&#10;&#10;In OSLSM~\cite{OSLSM}, a two-branch network consisting of a support branch and a query branch was proposed for few-shot segmentation. The support branch is devoted to generating a model from the support set, which is then used to tune the segmentation process of an image in the query branch.&#10;In PL~\cite{BMVC18Prototype}, the idea of prototypical networks was employed to tackle few-shot segmentation using metric learning.&#10;SG-One~\cite{SG-One} also used a prototype vector to guide semantic segmentation procedure. To obtain the squeezed representation of the support image, a masked average pooling strategy is designed to produce the prototype vector. A cosine similarity metric is then applied to build the relationship between the guidance features and features of pixels from the query image. &#10;PANet~\cite{PANet} further introduced a prototype alignment regularization between support and query branches to fully exploit knowledge from support images for better generalization.&#10;CANet ~\cite{CaNet} introduced a dense comparison module, which effectively exploits multiple levels of feature discriminativeness from CNNs to make dense feature comparison. With this approach comes an iterative optimization module which refines segmentation masks.&#10;The FWB approach~\cite{FWB-ICCV2019} focused on discriminativeness of prototype vectors (support features) by leveraging foreground-background feature differences of support images. It also used an ensemble of prototypes and similarity maps to handle the diversity of object appearances.&#10;&#10;As a core of metric learning in few-shot segmentation, the prototype vector was commonly calculated by global average pooling. &#10;However, such a strategy typically disregards the spatial extent of objects, which tends to mix semantics from various parts. This unintended mixing seriously deteriorates the diversity of prototype vectors and feature representation capacity. &#10;Recent approaches alleviated this problem using iterative mask refinement~\cite{CaNet} or model ensemble~\cite{FWB-ICCV2019}. &#10;However, issues remain when using single prototypes to represent object regions and the semantic ambiguity problem remains unsolved. &#10;&#10;Our research is inspired by the prototypical network~\cite{PrototypicalNet17}, which learns a metric space where classification is performed using distances to the prototype of each class. The essential differences are twofold: (1) A prototype in prototypical network~\cite{PrototypicalNet17} represents a class of samples while a prototype in our approach represents an object part; (2) The prototypical network does not involve mixing prototypes for a single sample or a class of samples.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2008.01449" label="2008.01449">
-        <attvalues>
-          <attvalue for="0" value="Prior Guided Feature Enrichment Network for Few-Shot Segmentation" />
-          <attvalue for="1" value="  State-of-the-art semantic segmentation methods require sufficient labeled&#10;data to achieve good results and hardly work on unseen classes without&#10;fine-tuning. Few-shot segmentation is thus proposed to tackle this problem by&#10;learning a model that quickly adapts to new classes with a few labeled support&#10;samples. Theses frameworks still face the challenge of generalization ability&#10;reduction on unseen classes due to inappropriate use of high-level semantic&#10;information of training classes and spatial inconsistency between query and&#10;support targets. To alleviate these issues, we propose the Prior Guided Feature&#10;Enrichment Network (PFENet). It consists of novel designs of (1) a&#10;training-free prior mask generation method that not only retains generalization&#10;power but also improves model performance and (2) Feature Enrichment Module&#10;(FEM) that overcomes spatial inconsistency by adaptively enriching query&#10;features with support features and prior masks. Extensive experiments on&#10;PASCAL-5$^i$ and COCO prove that the proposed prior generation method and FEM&#10;both improve the baseline method significantly. Our PFENet also outperforms&#10;state-of-the-art methods by a large margin without efficiency loss. It is&#10;surprising that our model even generalizes to cases without labeled support&#10;samples. Our code is available at https://github.com/Jia-Research-Lab/PFENet/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.03032" label="2108.03032">
-        <attvalues>
-          <attvalue for="0" value="Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight&#10;  Transformer" />
-          <attvalue for="1" value="  A few-shot semantic segmentation model is typically composed of a CNN&#10;encoder, a CNN decoder and a simple classifier (separating foreground and&#10;background pixels). Most existing methods meta-learn all three model components&#10;for fast adaptation to a new class. However, given that as few as a single&#10;support set image is available, effective model adaption of all three&#10;components to the new class is extremely challenging. In this work we propose&#10;to simplify the meta-learning task by focusing solely on the simplest&#10;component, the classifier, whilst leaving the encoder and decoder to&#10;pre-training. We hypothesize that if we pre-train an off-the-shelf segmentation&#10;model over a set of diverse training classes with sufficient annotations, the&#10;encoder and decoder can capture rich discriminative features applicable for any&#10;unseen classes, rendering the subsequent meta-learning stage unnecessary. For&#10;the classifier meta-learning, we introduce a Classifier Weight Transformer&#10;(CWT) designed to dynamically adapt the supportset trained classifier's weights&#10;to each query image in an inductive way. Extensive experiments on two standard&#10;benchmarks show that despite its simplicity, our method outperforms the&#10;state-of-the-art alternatives, often by a large margin.Code is available on&#10;https://github.com/zhiheLu/CWT-for-FSS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.00784" label="2207.00784">
-        <attvalues>
-          <attvalue for="0" value="Learning Cross-Image Object Semantic Relation in Transformer for&#10;  Few-Shot Fine-Grained Image Classification" />
-          <attvalue for="1" value="  Few-shot fine-grained learning aims to classify a query image into one of a&#10;set of support categories with fine-grained differences. Although learning&#10;different objects' local differences via Deep Neural Networks has achieved&#10;success, how to exploit the query-support cross-image object semantic relations&#10;in Transformer-based architecture remains under-explored in the few-shot&#10;fine-grained scenario. In this work, we propose a Transformer-based&#10;double-helix model, namely HelixFormer, to achieve the cross-image object&#10;semantic relation mining in a bidirectional and symmetrical manner. The&#10;HelixFormer consists of two steps: 1) Relation Mining Process (RMP) across&#10;different branches, and 2) Representation Enhancement Process (REP) within each&#10;individual branch. By the designed RMP, each branch can extract fine-grained&#10;object-level Cross-image Semantic Relation Maps (CSRMs) using information from&#10;the other branch, ensuring better cross-image interaction in semantically&#10;related local object regions. Further, with the aid of CSRMs, the developed REP&#10;can strengthen the extracted features for those discovered semantically-related&#10;local regions in each branch, boosting the model's ability to distinguish&#10;subtle feature differences of fine-grained objects. Extensive experiments&#10;conducted on five public fine-grained benchmarks demonstrate that HelixFormer&#10;can effectively enhance the cross-image object semantic relation matching for&#10;recognizing fine-grained objects, achieving much better performance over most&#10;state-of-the-art methods under 1-shot and 5-shot scenarios. Our code is&#10;available at: https://github.com/JiakangYuan/HelixFormer&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02320" label="2106.02320">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Segmentation via Cycle-Consistent Transformer" />
-          <attvalue for="1" value="  Few-shot segmentation aims to train a segmentation model that can fast adapt&#10;to novel classes with few exemplars. The conventional training paradigm is to&#10;learn to make predictions on query images conditioned on the features from&#10;support images. Previous methods only utilized the semantic-level prototypes of&#10;support images as conditional information. These methods cannot utilize all&#10;pixel-wise support information for the query predictions, which is however&#10;critical for the segmentation task. In this paper, we focus on utilizing&#10;pixel-wise relationships between support and query images to facilitate the&#10;few-shot segmentation task. We design a novel Cycle-Consistent TRansformer&#10;(CyCTR) module to aggregate pixel-wise support features into query ones. CyCTR&#10;performs cross-attention between features from different images, i.e. support&#10;and query images. We observe that there may exist unexpected irrelevant&#10;pixel-level support features. Directly performing cross-attention may aggregate&#10;these features from support to query and bias the query features. Thus, we&#10;propose using a novel cycle-consistent attention mechanism to filter out&#10;possible harmful support features and encourage query features to attend to the&#10;most informative pixels from support images. Experiments on all few-shot&#10;segmentation benchmarks demonstrate that our proposed CyCTR leads to remarkable&#10;improvement compared to previous state-of-the-art methods. Specifically, on&#10;Pascal-$5^i$ and COCO-$20^i$ datasets, we achieve 67.5% and 45.6% mIoU for&#10;5-shot segmentation, outperforming previous state-of-the-art methods by 5.6%&#10;and 7.1% respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.10240" label="2106.10240">
-        <attvalues>
-          <attvalue for="0" value="VSAC: Efficient and Accurate Estimator for H and F" />
-          <attvalue for="1" value="  We present VSAC, a RANSAC-type robust estimator with a number of novelties.&#10;It benefits from the introduction of the concept of independent inliers that&#10;improves significantly the efficacy of the dominant plane handling and, also,&#10;allows near error-free rejection of incorrect models, without false positives.&#10;The local optimization process and its application is improved so that it is&#10;run on average only once. Further technical improvements include adaptive&#10;sequential hypothesis verification and efficient model estimation via Gaussian&#10;elimination. Experiments on four standard datasets show that VSAC is&#10;significantly faster than all its predecessors and runs on average in 1-2 ms,&#10;on a CPU. It is two orders of magnitude faster and yet as precise as MAGSAC++,&#10;the currently most accurate estimator of two-view geometry. In the repeated&#10;runs on EVD, HPatches, PhotoTourism, and Kusvod2 datasets, it never failed.&#10;" />
-          <attvalue for="2" value="&#10;The {Ran}dom {Sa}mple {C}onsensus (RANSAC) &#10;algorithm introduced by Fischler and Bolles \cite{fischler1981random} is one of the most popular robust estimators in computer science.&#10;The method is widely used in computer vision,&#10;its applications include stereo matching \cite{torr1993outlier, torr1998robust}, image mosaicing \cite{ghosh2016survey}, motion segmentation \cite{torr1993outlier}, 3D reconstruction, detection of geometric primitives, and structure and motion estimation~\cite{conf/iccv/Nister03}.&#10;&#10;The textbook version of RANSAC proceeds as follows: random samples of minimal size sufficient to estimate the model parameters are drawn repeatedly.&#10;Model consistency with input data is evaluated, \eg, by counting the points closer than a manually set inlier-outlier threshold. If the current model is better then the \sftb, it gets stored. &#10;The procedure terminates when the probability of finding a better model falls below a user-defined level. &#10;Finally, the estimate is polished by least-squares fitting of inliers.&#10;&#10;Many modifications of the original algorithm have been proposed.&#10;Regarding sampling,&#10;PROSAC~\cite{chum2005matching} exploits an a priori predicted inlier probability rank.&#10;NAPSAC \cite{nasuto2002napsac} samples in the neighborhood of the first, randomly selected, point. &#10;Progressive NAPSAC~\cite{barath2019progressive} combines both and adds gradual convergence to uniform spatial sampling.&#10;&#10;In textbook RANSAC, the model quality is measured by its support, \ie, the number of inliers, points consistent with the model.&#10;MLESAC~\cite{torr2000mlesac} introduced a quality measure that makes it the maximum likelihood procedure.&#10;To avoid the need for a user-defined noise level, MINPRAN~\cite{stewart1995minpran} and A-contrario RANSAC~\cite{a-contrario} select the inlier-outlier threshold so that the inliers are the least likely to occur at random. &#10;Reflecting the inherent uncertainty of the threshold estimate, &#10;MAGSAC~\cite{barath2019magsac} marginalizes the quality function over a range of noise levels.&#10;MAGSAC++~\cite{Barath_2020_CVPR} proposes an iterative re-weighted least-squares optimization of the {\sftb} model with weights calculated from the inlier probability of points.&#10;The Locally Optimized RANSAC \cite{chum2003locally} refines the {\sftb} model using a non-minimal number of points, \eg, by iterated least-squares fitting. &#10;Graph-Cut RANSAC \cite{barath2018graph}, in its local optimization, exploits the fact that real-world data tend to form spatial structures.&#10;The model evaluation is usually the most time-consuming part as it depends both on the number of models generated and the number of input data points. &#10;A quasi-optimal speed-up was achieved by the Sequential Probability Ratio Test (SPRT) \cite{matas2005randomized} that randomizes the verification process itself.&#10;&#10;In many cases, points in degenerate configuration affect the estimation severely. For example, correspondences lying on a single plane is a degenerate case for $\mathbf{F}$ estimation. &#10;DEGENSAC \cite{chum2005two} detects such cases and applies the plane-and-parallax algorithm. &#10;USAC \cite{raguram2013usac} was the first framework integrating many of the mentioned techniques, including PROSAC, SPRT, DEGENSAC, and LO-RANSAC. &#10;&#10;In this paper, we present VSAC, a RANSAC-type estimator that exploits a number of novelties. It is significantly faster than all its predecessors, and yet as precise as MAGSAC++, the currently most accurate method both in our experiments and according to a recent survey~\cite{ma2021image}. The accuracy reaches, or is very near, the geometric error of the ground truth, estimated by cross-validation.&#10;For homography $\mathbf{H}$ and epipolar geometry $\mathbf{F}$ estimation,&#10;VSAC runs on average in 1-2 ms (on a CPU) on all datasets,&#10;two orders of magnitude faster than MAGSAC++. In the repeated runs on datasets EVD~\cite{mishkin2015mods}, HPatches~\cite{Balntas_2017_CVPR}, PhotoTourism~\cite{phototourism} and Kusvod2~\cite{lebeda2012fixing}, it never failed. &#10;&#10;Moreover, VSAC is able to reject non-matching image pairs, with a zero false positive rate on hundreds of random image pairs and a zero false negative rate on pairs from the above-mentioned datasets. &#10;The ability is underpinned by a novel concept of {independent random inliers} in the contrario context.&#10;We show that if dependent random inliers, \eg spatially co-located points, are not counted, the support of random models follows very closely a Poisson distribution with a single parameter $\lambda$ that is easy to estimate reliably&#10;for the given pair. &#10;The easily calculated CDF of Poisson raised to the power of the number of evaluated models provides the probability that a certain model quality was reached by chance.&#10;VSAC thus provides two confidence measures together with its result. The first is the classical one -- the probability that RANSAC returned the model with the highest support. &#10;The second is the confidence that the returned solution was not obtained by chance.&#10;&#10;The concept of independent random inliers plays a critical role in VSAC's accuracy and robustness. &#10;Experiments show that most failures of USAC-like methods for $\mathbf{F}$ estimation occur in the presence of a dominant plane, despite the DEGENSAC algorithm.&#10;In such cases, there are only few out-of-plane inliers, and due to structures in the outliers, incorrect models with high support exist. Removing the dependent structures addresses the problem. Further improvements of dominant plane handling include a heuristic guess of the calibration matrix allowing to deal with fully planar scenes and detects pure rotation. If the guess is wrong, the support reveals it and nothing but a microsecond is lost.&#10;&#10;The speed of VSAC is achieved with several technical improvements. Most significantly, we attack the problem of expensive local optimization. In the LO-RANSAC paper~\cite{chum2003locally}, the authors prove that the local optimization is run at most $\log(k)$ times, where $k$ is the number of iteration. Nevertheless, despite $ \log(k) \ll k$, the complex local optimization may end up being the efficiency bottleneck. We show that a fast local optimization combined with {a single complex final optimization} leads to a faster, yet equally precise algorithm. Moreover, by detecting the intersectionover union of {\sftb} and the current set of inliers and by not optimizing similar models, an algorithm is obtained that runs the local optimization on average about once and almost always fewer than two times. &#10;&#10;Further speed up is gained by adaptive SPRT. We use the estimated expected number of random inliers to tune SPRT~\cite{matas2005randomized} to the outlier density of the processed pair. We also measure, on the fly, the actual time of model estimation and model verification on the given hardware at the given moment, which is needed for calculating the quasi-optimal thresholds of the SPRT. &#10;&#10;To broaden its application potential, VSAC provides novel outputs. Employing the highly efficient Lindstrom method for triangulation \cite{triangulationEasyLindstrom}, it obtains the point pair exactly fitting the returned $\mathbf{F}$ that minimizes the geometric error. VSAC can be thus employed for noise filtering. VSAC returns all input points sorted by the residual, allowing the user to set his own ex-post trade-off between the density, and possibly spread, of points on the one hand, and their accuracy on the other. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Robust Estimation, Mathematics, Computer Science, Numerical Analysis, Algorithm Optimization" />
-        </attvalues>
-      </node>
-      <node id="1803.07469" label="1803.07469">
-        <attvalues>
-          <attvalue for="0" value="MAGSAC: marginalizing sample consensus" />
-          <attvalue for="1" value="  A method called, sigma-consensus, is proposed to eliminate the need for a&#10;user-defined inlier-outlier threshold in RANSAC. Instead of estimating the&#10;noise sigma, it is marginalized over a range of noise scales. The optimized&#10;model is obtained by weighted least-squares fitting where the weights come from&#10;the marginalization over sigma of the point likelihoods of being inliers. A new&#10;quality function is proposed not requiring sigma and, thus, a set of inliers to&#10;determine the model quality. Also, a new termination criterion for RANSAC is&#10;built on the proposed marginalization approach. Applying sigma-consensus,&#10;MAGSAC is proposed with no need for a user-defined sigma and improving the&#10;accuracy of robust estimation significantly. It is superior to the&#10;state-of-the-art in terms of geometric accuracy on publicly available&#10;real-world datasets for epipolar geometry (F and E) and homography estimation.&#10;In addition, applying sigma-consensus only once as a post-processing step to&#10;the RANSAC output always improved the model quality on a wide range of vision&#10;problems without noticeable deterioration in processing time, adding a few&#10;milliseconds. The source code is at https://github.com/danini/magsac.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.05909" label="1912.05909">
-        <attvalues>
-          <attvalue for="0" value="MAGSAC++, a fast, reliable and accurate robust estimator" />
-          <attvalue for="1" value="  A new method for robust estimation, MAGSAC++, is proposed. It introduces a&#10;new model quality (scoring) function that does not require the inlier-outlier&#10;decision, and a novel marginalization procedure formulated as an iteratively&#10;re-weighted least-squares approach. We also propose a new sampler, Progressive&#10;NAPSAC, for RANSAC-like robust estimators. Exploiting the fact that nearby&#10;points often originate from the same model in real-world data, it finds local&#10;structures earlier than global samplers. The progressive transition from local&#10;to global sampling does not suffer from the weaknesses of purely localized&#10;samplers. On six publicly available real-world datasets for homography and&#10;fundamental matrix fitting, MAGSAC++ produces results superior to&#10;state-of-the-art robust methods. It is faster, more geometrically accurate and&#10;fails less often.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.05939" label="1704.05939">
-        <attvalues>
-          <attvalue for="0" value="HPatches: A benchmark and evaluation of handcrafted and learned local&#10;  descriptors" />
-          <attvalue for="1" value="  In this paper, we propose a novel benchmark for evaluating local image&#10;descriptors. We demonstrate that the existing datasets and evaluation protocols&#10;do not specify unambiguously all aspects of evaluation, leading to ambiguities&#10;and inconsistencies in results reported in the literature. Furthermore, these&#10;datasets are nearly saturated due to the recent improvements in local&#10;descriptors obtained by learning them from large annotated datasets. Therefore,&#10;we introduce a new large dataset suitable for training and testing modern&#10;descriptors, together with strictly defined evaluation protocols in several&#10;tasks such as matching, retrieval and classification. This allows for more&#10;realistic, and thus more reliable comparisons in different application&#10;scenarios. We evaluate the performance of several state-of-the-art descriptors&#10;and analyse their properties. We show that a simple normalisation of&#10;traditional hand-crafted descriptors can boost their performance to the level&#10;of deep learning based descriptors within a realistic benchmarks evaluation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.10624" label="2302.10624">
-        <attvalues>
-          <attvalue for="0" value="Self-improving object detection via disagreement reconciliation" />
-          <attvalue for="1" value="  Object detectors often experience a drop in performance when new&#10;environmental conditions are insufficiently represented in the training data.&#10;This paper studies how to automatically fine-tune a pre-existing object&#10;detector while exploring and acquiring images in a new environment without&#10;relying on human intervention, i.e., in a self-supervised fashion. In our&#10;setting, an agent initially explores the environment using a pre-trained&#10;off-the-shelf detector to locate objects and associate pseudo-labels. By&#10;assuming that pseudo-labels for the same object must be consistent across&#10;different views, we devise a novel mechanism for producing refined predictions&#10;from the consensus among observations. Our approach improves the off-the-shelf&#10;object detector by 2.66% in terms of mAP and outperforms the current state of&#10;the art without relying on ground-truth annotations.&#10;" />
-          <attvalue for="2" value="&#10;The deployment of object detectors, particularly in real-world scenarios such as indoor robots, often requires fine-tuning with a custom dataset to accommodate for the domain shift between the training and test environments. While fine-tuning leads to improved performance, constructing a dataset can be a daunting task, requiring extensive time and effort in collecting and annotating thousands of images.&#10;&#10;What if, instead of relying on human annotation, we could use data collected autonomously by the robot itself to fine-tune the detector? This paper explores this possibility by utilizing an embodied agent that actively explores an environment and leverages the collected data to improve object detection models. Object detectors play a crucial role in goal-oriented exploration \cite{chaplot2020object,cartillier2020semantic,chaplot2020learning}, and recent studies have demonstrated the relationship between object detection improvements and the reliable semantic mapping of environments \cite{chaplot2020object}, making them of significant importance in the field of robotics exploration.&#10;&#10;This study draws parallels with the self-training paradigm \cite{lee2013pseudo}, where a model is trained on both labeled and unlabeled data. In self-training, a pre-trained model provides pseudo-labels for the unlabeled data, which may be refined by the introduction of additional noise \cite{arazo2019,xie2020selftraining,chen2020big}. Chaplot et al. \cite{chaplot2021seal} introduced a scenario where an agent explores the environment and collects data autonomously, proposing an approach to collect consistent pseudo-labels in the environment. While their approach effectively builds a semantic map and re-projects it onto each frame, it does not consider the inconsistency of detections as a viable training signal. Our approach, in contrast, presents a novel mechanism that leverages different points of view of the same object as a data augmentation strategy.&#10;&#10;Figure \ref{fig:fig1} summarizes our proposal. We implement our approach by equipping an agent with a pre-trained object detector, in this case MaskRCNN \cite{he2018mask}. The agent then explores an unknown environment and detects objects within the scene. The resulting predictions are used as pseudo-labels to fine-tune the detector. This process is conducted in three stages. Firstly, we enforce consistency of the predictions across multiple views through a consensus mechanism. The refined pseudo-labels are assigned to all relevant images and the consensus is used to fine-tune the detector. Secondly, using a contrastive learning approach, the model backbone is forced to map different views of the same object close together in the feature space while different instances are kept distinguishable. Lastly, we leverage the soft targets obtained by averaging multiple predictions for each instance as further feedback, to fully utilize the multiple acquired views.&#10;In our approach, we utilize a contrastive loss function \cite{facenet2015} to ensure that features of an object are similar across its different views while features of different objects are distinguishable. Our method can be considered a generalization of the cropping strategy that is commonly utilized in self-supervised learning \cite{chen2020simple}. However, it should be noted that the cropping strategy alone is not sufficient to fully capture the intricacies of a 3D object viewed from various perspectives.&#10;&#10;Our contributions are the following:&#10;\begin{itemize}[topsep=0pt,parsep=0pt,partopsep=0pt]&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Environmental Adaptation, Artificial Intelligence, Object Detection, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2007.00643" label="2007.00643">
-        <attvalues>
-          <attvalue for="0" value="Object Goal Navigation using Goal-Oriented Semantic Exploration" />
-          <attvalue for="1" value="  This work studies the problem of object goal navigation which involves&#10;navigating to an instance of the given object category in unseen environments.&#10;End-to-end learning-based navigation methods struggle at this task as they are&#10;ineffective at exploration and long-term planning. We propose a modular system&#10;called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic&#10;map and uses it to explore the environment efficiently based on the goal object&#10;category. Empirical results in visually realistic simulation environments show&#10;that the proposed model outperforms a wide range of baselines including&#10;end-to-end learning-based methods as well as modular map-based methods and led&#10;to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation&#10;analysis indicates that the proposed model learns semantic priors of the&#10;relative arrangement of objects in a scene, and uses them to explore&#10;efficiently. Domain-agnostic module design allow us to transfer our model to a&#10;mobile robot platform and achieve similar performance for object goal&#10;navigation in the real-world.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.05155" label="2004.05155">
-        <attvalues>
-          <attvalue for="0" value="Learning to Explore using Active Neural SLAM" />
-          <attvalue for="1" value="  This work presents a modular and hierarchical approach to learn policies for&#10;exploring 3D environments, called `Active Neural SLAM'. Our approach leverages&#10;the strengths of both classical and learning-based methods, by using analytical&#10;path planners with learned SLAM module, and global and local policies. The use&#10;of learning provides flexibility with respect to input modalities (in the SLAM&#10;module), leverages structural regularities of the world (in global policies),&#10;and provides robustness to errors in state estimation (in local policies). Such&#10;use of learning within each module retains its benefits, while at the same&#10;time, hierarchical decomposition and modular training allow us to sidestep the&#10;high sample complexities associated with training end-to-end policies. Our&#10;experiments in visually and physically realistic simulated 3D environments&#10;demonstrate the effectiveness of our approach over past learning and&#10;geometry-based approaches. The proposed model can also be easily transferred to&#10;the PointGoal task and was the winning entry of the CVPR 2019 Habitat PointGoal&#10;Navigation Challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.02983" label="1908.02983">
-        <attvalues>
-          <attvalue for="0" value="Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning" />
-          <attvalue for="1" value="  Semi-supervised learning, i.e. jointly learning from labeled and unlabeled&#10;samples, is an active research topic due to its key role on relaxing human&#10;supervision. In the context of image classification, recent advances to learn&#10;from unlabeled samples are mainly focused on consistency regularization methods&#10;that encourage invariant predictions for different perturbations of unlabeled&#10;samples. We, conversely, propose to learn from unlabeled data by generating&#10;soft pseudo-labels using the network predictions. We show that a naive&#10;pseudo-labeling overfits to incorrect pseudo-labels due to the so-called&#10;confirmation bias and demonstrate that mixup augmentation and setting a minimum&#10;number of labeled samples per mini-batch are effective regularization&#10;techniques for reducing it. The proposed approach achieves state-of-the-art&#10;results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler&#10;than other methods. These results demonstrate that pseudo-labeling alone can&#10;outperform consistency regularization methods, while the opposite was supposed&#10;in previous work. Source code is available at https://git.io/fjQsC.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.04252" label="1911.04252">
-        <attvalues>
-          <attvalue for="0" value="Self-training with Noisy Student improves ImageNet classification" />
-          <attvalue for="1" value="  We present Noisy Student Training, a semi-supervised learning approach that&#10;works well even when labeled data is abundant. Noisy Student Training achieves&#10;88.4% top-1 accuracy on ImageNet, which is 2.0% better than the&#10;state-of-the-art model that requires 3.5B weakly labeled Instagram images. On&#10;robustness test sets, it improves ImageNet-A top-1 accuracy from 61.0% to&#10;83.7%, reduces ImageNet-C mean corruption error from 45.7 to 28.3, and reduces&#10;ImageNet-P mean flip rate from 27.8 to 12.2.&#10;  Noisy Student Training extends the idea of self-training and distillation&#10;with the use of equal-or-larger student models and noise added to the student&#10;during learning. On ImageNet, we first train an EfficientNet model on labeled&#10;images and use it as a teacher to generate pseudo labels for 300M unlabeled&#10;images. We then train a larger EfficientNet as a student model on the&#10;combination of labeled and pseudo labeled images. We iterate this process by&#10;putting back the student as the teacher. During the learning of the student, we&#10;inject noise such as dropout, stochastic depth, and data augmentation via&#10;RandAugment to the student so that the student generalizes better than the&#10;teacher. Models are available at&#10;https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet.&#10;Code is available at https://github.com/google-research/noisystudent.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.10029" label="2006.10029">
-        <attvalues>
-          <attvalue for="0" value="Big Self-Supervised Models are Strong Semi-Supervised Learners" />
-          <attvalue for="1" value="  One paradigm for learning from few labeled examples while making best use of&#10;a large amount of unlabeled data is unsupervised pretraining followed by&#10;supervised fine-tuning. Although this paradigm uses unlabeled data in a&#10;task-agnostic way, in contrast to common approaches to semi-supervised learning&#10;for computer vision, we show that it is surprisingly effective for&#10;semi-supervised learning on ImageNet. A key ingredient of our approach is the&#10;use of big (deep and wide) networks during pretraining and fine-tuning. We find&#10;that, the fewer the labels, the more this approach (task-agnostic use of&#10;unlabeled data) benefits from a bigger network. After fine-tuning, the big&#10;network can be further improved and distilled into a much smaller one with&#10;little loss in classification accuracy by using the unlabeled examples for a&#10;second time, but in a task-specific way. The proposed semi-supervised learning&#10;algorithm can be summarized in three steps: unsupervised pretraining of a big&#10;ResNet model using SimCLRv2, supervised fine-tuning on a few labeled examples,&#10;and distillation with unlabeled examples for refining and transferring the&#10;task-specific knowledge. This procedure achieves 73.9% ImageNet top-1 accuracy&#10;with just 1% of the labels ($\le$13 labeled images per class) using ResNet-50,&#10;a $10\times$ improvement in label efficiency over the previous&#10;state-of-the-art. With 10% of labels, ResNet-50 trained with our method&#10;achieves 77.5% top-1 accuracy, outperforming standard supervised training with&#10;all of the labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.01001" label="2112.01001">
-        <attvalues>
-          <attvalue for="0" value="SEAL: Self-supervised Embodied Active Learning using Exploration and 3D&#10;  Consistency" />
-          <attvalue for="1" value="  In this paper, we explore how we can build upon the data and models of&#10;Internet images and use them to adapt to robot vision without requiring any&#10;extra labels. We present a framework called Self-supervised Embodied Active&#10;Learning (SEAL). It utilizes perception models trained on internet images to&#10;learn an active exploration policy. The observations gathered by this&#10;exploration policy are labelled using 3D consistency and used to improve the&#10;perception model. We build and utilize 3D semantic maps to learn both action&#10;and perception in a completely self-supervised manner. The semantic map is used&#10;to compute an intrinsic motivation reward for training the exploration policy&#10;and for labelling the agent observations using spatio-temporal 3D consistency&#10;and label propagation. We demonstrate that the SEAL framework can be used to&#10;close the action-perception loop: it improves object detection and instance&#10;segmentation performance of a pretrained perception model by just moving around&#10;in training environments and the improved perception model can be used to&#10;improve Object Goal Navigation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.07061" label="1909.07061">
-        <attvalues>
-          <attvalue for="0" value="Motion Guided Attention for Video Salient Object Detection" />
-          <attvalue for="1" value="  Video salient object detection aims at discovering the most visually&#10;distinctive objects in a video. How to effectively take object motion into&#10;consideration during video salient object detection is a critical issue.&#10;Existing state-of-the-art methods either do not explicitly model and harvest&#10;motion cues or ignore spatial contexts within optical flow images. In this&#10;paper, we develop a multi-task motion guided video salient object detection&#10;network, which learns to accomplish two sub-tasks using two sub-networks, one&#10;sub-network for salient object detection in still images and the other for&#10;motion saliency detection in optical flow images. We further introduce a series&#10;of novel motion guided attention modules, which utilize the motion saliency&#10;sub-network to attend and enhance the sub-network for still images. These two&#10;sub-networks learn to adapt to each other by end-to-end training. Experimental&#10;results demonstrate that the proposed method significantly outperforms existing&#10;state-of-the-art algorithms on a wide range of benchmarks. We hope our simple&#10;and effective approach will serve as a solid baseline and help ease future&#10;research in video salient object detection. Code and models will be made&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Architectures, Computer Science, Video Object Detection, Motion Analysis Techniques" />
-        </attvalues>
-      </node>
-      <node id="2202.12369" label="2202.12369">
-        <attvalues>
-          <attvalue for="0" value="On Monocular Depth Estimation and Uncertainty Quantification using&#10;  Classification Approaches for Regression" />
-          <attvalue for="1" value="  Monocular depth is important in many tasks, such as 3D reconstruction and&#10;autonomous driving. Deep learning based models achieve state-of-the-art&#10;performance in this field. A set of novel approaches for estimating monocular&#10;depth consists of transforming the regression task into a classification one.&#10;However, there is a lack of detailed descriptions and comparisons for&#10;Classification Approaches for Regression (CAR) in the community and no in-depth&#10;exploration of their potential for uncertainty estimation. To this end, this&#10;paper will introduce a taxonomy and summary of CAR approaches, a new&#10;uncertainty estimation solution for CAR, and a set of experiments on depth&#10;accuracy and uncertainty quantification for CAR-based models on KITTI dataset.&#10;The experiments reflect the differences in the portability of various CAR&#10;methods on two backbones. Meanwhile, the newly proposed method for uncertainty&#10;estimation can outperform the ensembling method with only one forward&#10;propagation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In machine learning, regression tasks predict a continuous output based on a given input. Yet, if the ground truth (prediction target) is within a specific range, e.g., in the case of age estimation~\cite{levi2015age}, one can quantize the ground truth and cast regression into a classification problem. We refer to these techniques as Classification Approaches for Regression (CAR), and in this paper we explore CAR techniques applied on monocular depth estimation.&#10;&#10;Monocular depth estimation (MDE), which is an ill-posed problem~\cite{sinha1993recovering}, consists in predicting the scene depth given only an RGB image as the input. Deep Neural Networks (DNNs) learn the mapping between the single RGB images and their corresponding depth maps to solve MDE, and show good performance on indoor and outdoor benchmarks~\cite{Uhrig2017THREEDV, Silberman:ECCV12}. &#10;&#10;Classification Approaches for Regression (CAR) \cite{cao2017estimating, li2018monocular, fu2018deep, diaz2019soft, yang2019inferring, bhat2021adabins} have emerged recently in the spotlight among MDE algorithms. The core idea is to transfer regression to a classification problem using quantization (or discretization) strategies.&#10;The classification models can natively provide the confidence for prediction results, which also has the potential to improve the prediction accuracy~\cite{diaz2019soft}. DNNs are prone to two kinds of uncertainty: aleatoric uncertainty and epistemic uncertainty~\cite{kendall2017uncertainties}. It is crucial to study the uncertainty of DNNs if we want to rely on their predictions. Some works proposed to estimate the uncertainty of MDE DNNs by using an auxiliary network~\cite{Poggi_CVPR_2020, yu21bmvc}, or ensembling~\cite{2017simple}. Here we gain access to the uncertainty directly using the CAR DNN.&#10;&#10;This work will investigate and show the complete picture of the CAR MDE methods. The contributions are as follows:\\&#10;1. We systematically summarize and formalize the all major CAR MDE mechanisms to the best of our knowledge;\\&#10;2. We implement these mechanisms on top of two different backbones, comparing depth prediction and uncertainty quality on various evaluation metrics;\\&#10;3. {We propose a new, effective uncertainty estimation method named Expectation of Distance for CAR MDE models.}&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Monocular Depth Estimation, Uncertainty Quantification, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1708.06500" label="1708.06500">
-        <attvalues>
-          <attvalue for="0" value="Sparsity Invariant CNNs" />
-          <attvalue for="1" value="  In this paper, we consider convolutional neural networks operating on sparse&#10;inputs with an application to depth upsampling from sparse laser scan data.&#10;First, we show that traditional convolutional networks perform poorly when&#10;applied to sparse data even when the location of missing data is provided to&#10;the network. To overcome this problem, we propose a simple yet effective sparse&#10;convolution layer which explicitly considers the location of missing data&#10;during the convolution operation. We demonstrate the benefits of the proposed&#10;network architecture in synthetic and real experiments with respect to various&#10;baseline approaches. Compared to dense baselines, the proposed sparse&#10;convolution network generalizes well to novel datasets and is invariant to the&#10;level of sparsity in the data. For our evaluation, we derive a novel dataset&#10;from the KITTI benchmark, comprising 93k depth annotated RGB images. Our&#10;dataset allows for training and evaluating depth upsampling and depth&#10;prediction techniques in challenging real-world settings and will be made&#10;available upon publication.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.04977" label="1703.04977">
-        <attvalues>
-          <attvalue for="0" value="What Uncertainties Do We Need in Bayesian Deep Learning for Computer&#10;  Vision?" />
-          <attvalue for="1" value="  There are two major types of uncertainty one can model. Aleatoric uncertainty&#10;captures noise inherent in the observations. On the other hand, epistemic&#10;uncertainty accounts for uncertainty in the model -- uncertainty which can be&#10;explained away given enough data. Traditionally it has been difficult to model&#10;epistemic uncertainty in computer vision, but with new Bayesian deep learning&#10;tools this is now possible. We study the benefits of modeling epistemic vs.&#10;aleatoric uncertainty in Bayesian deep learning models for vision tasks. For&#10;this we present a Bayesian deep learning framework combining input-dependent&#10;aleatoric uncertainty together with epistemic uncertainty. We study models&#10;under the framework with per-pixel semantic segmentation and depth regression&#10;tasks. Further, our explicit uncertainty formulation leads to new loss&#10;functions for these tasks, which can be interpreted as learned attenuation.&#10;This makes the loss more robust to noisy data, also giving new state-of-the-art&#10;results on segmentation and depth regression benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.01474" label="1612.01474">
-        <attvalues>
-          <attvalue for="0" value="Simple and Scalable Predictive Uncertainty Estimation using Deep&#10;  Ensembles" />
-          <attvalue for="1" value="  Deep neural networks (NNs) are powerful black box predictors that have&#10;recently achieved impressive performance on a wide spectrum of tasks.&#10;Quantifying predictive uncertainty in NNs is a challenging and yet unsolved&#10;problem. Bayesian NNs, which learn a distribution over weights, are currently&#10;the state-of-the-art for estimating predictive uncertainty; however these&#10;require significant modifications to the training procedure and are&#10;computationally expensive compared to standard (non-Bayesian) NNs. We propose&#10;an alternative to Bayesian NNs that is simple to implement, readily&#10;parallelizable, requires very little hyperparameter tuning, and yields high&#10;quality predictive uncertainty estimates. Through a series of experiments on&#10;classification and regression benchmarks, we demonstrate that our method&#10;produces well-calibrated uncertainty estimates which are as good or better than&#10;approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate&#10;the predictive uncertainty on test examples from known and unknown&#10;distributions, and show that our method is able to express higher uncertainty&#10;on out-of-distribution examples. We demonstrate the scalability of our method&#10;by evaluating predictive uncertainty estimates on ImageNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.00484" label="2112.00484">
-        <attvalues>
-          <attvalue for="0" value="Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic&#10;  Foggy Scene Understanding" />
-          <attvalue for="1" value="  Although considerable progress has been made in semantic scene understanding&#10;under clear weather, it is still a tough problem under adverse weather&#10;conditions, such as dense fog, due to the uncertainty caused by imperfect&#10;observations. Besides, difficulties in collecting and labeling foggy images&#10;hinder the progress of this field. Considering the success in semantic scene&#10;understanding under clear weather, we think it is reasonable to transfer&#10;knowledge learned from clear images to the foggy domain. As such, the problem&#10;becomes to bridge the domain gap between clear images and foggy images. Unlike&#10;previous methods that mainly focus on closing the domain gap caused by fog --&#10;defogging the foggy images or fogging the clear images, we propose to alleviate&#10;the domain gap by considering fog influence and style variation simultaneously.&#10;The motivation is based on our finding that the style-related gap and the&#10;fog-related gap can be divided and closed respectively, by adding an&#10;intermediate domain. Thus, we propose a new pipeline to cumulatively adapt&#10;style, fog and the dual-factor (style and fog). Specifically, we devise a&#10;unified framework to disentangle the style factor and the fog factor&#10;separately, and then the dual-factor from images in different domains.&#10;Furthermore, we collaborate the disentanglement of three factors with a novel&#10;cumulative loss to thoroughly disentangle these three factors. Our method&#10;achieves the state-of-the-art performance on three benchmarks and shows&#10;generalization ability in rainy and snowy scenes.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Semantic foggy scene understanding (SFSU) is important for autonomous driving~\cite{sakaridis2018model,sakaridis2018semantic,dai2020curriculum,2019Purely}.&#10;Although great progress has been made on semantic understanding of clear scenes, SFSU tends to have unsatisfactory performance due to visibility degradation caused by fog~\cite{narasimhan2003contrast,tan2008visibility}.&#10;Besides, unlike the abundant data and annotation under clear scenes, the lack of data and annotation under dense fog weather further complicates this problem.&#10;Therefore, handling the challenging SFSU problem often requires transferring the segmentation knowledge learned from labeled clear images to the unlabeled foggy images. &#10;&#10;Intuitively, we may tackle this problem by closing the domain gap between clear images and foggy images with state-of-the-art domain adaptation methods.&#10;However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, &#10;regardless of how the domain gap is caused. Besides, as has been validated by \cite{sakaridis2018model}, they fail to address the SFSU problem well due to the large domain gap.&#10;&#10;Consequently, the attention of SFSU has been focused on the fog factor, &#10;which is regarded as the dominating cause of the domain gap in the SFSU problem. One solution is to close this gap by defogging real foggy images, using an existing defogging method~\cite{ren2018deep,de2019learning,zhang2018densely,ren2016single,he2010single,berman2016non,ren2018gated}.&#10;Whereas, the defogging method will also introduce artifacts. They act as noise to hinder the domain adaptation to some extent~\cite{2018Does}. Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}. Nevertheless, these rendered synthetic foggy images, not as real as real foggy ones, could also widen the domain gap between clear and foggy images and yield unsatisfactory performance. Moreover, we argue that these methods over-concerned the fog factor while ignoring other factors, which may affect the domain gap in the SFSU problem.&#10;&#10;Thinking out of the box, we propose to explicitly investigate the domain gap in SFSU 1) to avoid directly treating the total domain gap; 2) without using synthetic foggy data or knowledge of defogging. We assume that the domain gap is caused by the mixed fog influence and style variation, and both of them are important to SFSU. &#10;That is, we assume there exist the style-related gap and the fog-related gap in the domain gap of SFSU, and we can decompose the mixed dual-factor gap into these two single-factor gaps by adding an intermediate domain.&#10;Next, we elaborate on why we can disentangle the style-related gap and what relationship lies between the style-related gap and the fog-related gap in the SFSU problem, using the following empirical finding.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Domain Adaptation Techniques, Mathematics, Artificial Intelligence, Computer Science, Semantic Scene Understanding, Image Processing Algorithms" />
-        </attvalues>
-      </node>
-      <node id="1808.01265" label="1808.01265">
-        <attvalues>
-          <attvalue for="0" value="Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy&#10;  Scene Understanding" />
-          <attvalue for="1" value="  This work addresses the problem of semantic scene understanding under dense&#10;fog. Although considerable progress has been made in semantic scene&#10;understanding, it is mainly related to clear-weather scenes. Extending&#10;recognition methods to adverse weather conditions such as fog is crucial for&#10;outdoor applications. In this paper, we propose a novel method, named&#10;Curriculum Model Adaptation (CMAda), which gradually adapts a semantic&#10;segmentation model from light synthetic fog to dense real fog in multiple&#10;steps, using both synthetic and real foggy data. In addition, we present three&#10;other main stand-alone contributions: 1) a novel method to add synthetic fog to&#10;real, clear-weather scenes using semantic input; 2) a new fog density&#10;estimator; 3) the Foggy Zurich dataset comprising $3808$ real foggy images,&#10;with pixel-level semantic annotations for $16$ images with dense fog. Our&#10;experiments show that 1) our fog simulation slightly outperforms a&#10;state-of-the-art competing simulation with respect to the task of semantic&#10;foggy scene understanding (SFSU); 2) CMAda improves the performance of&#10;state-of-the-art models for SFSU significantly by leveraging unlabeled real&#10;foggy data. The datasets and code are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.07819" label="1708.07819">
-        <attvalues>
-          <attvalue for="0" value="Semantic Foggy Scene Understanding with Synthetic Data" />
-          <attvalue for="1" value="  This work addresses the problem of semantic foggy scene understanding (SFSU).&#10;Although extensive research has been performed on image dehazing and on&#10;semantic scene understanding with clear-weather images, little attention has&#10;been paid to SFSU. Due to the difficulty of collecting and annotating foggy&#10;images, we choose to generate synthetic fog on real images that depict&#10;clear-weather outdoor scenes, and then leverage these partially synthetic data&#10;for SFSU by employing state-of-the-art convolutional neural networks (CNN). In&#10;particular, a complete pipeline to add synthetic fog to real, clear-weather&#10;images using incomplete depth information is developed. We apply our fog&#10;synthesis on the Cityscapes dataset and generate Foggy Cityscapes with 20550&#10;images. SFSU is tackled in two ways: 1) with typical supervised learning, and&#10;2) with a novel type of semi-supervised learning, which combines 1) with an&#10;unsupervised supervision transfer from clear-weather images to their synthetic&#10;foggy counterparts. In addition, we carefully study the usefulness of image&#10;dehazing for SFSU. For evaluation, we present Foggy Driving, a dataset with 101&#10;real-world images depicting foggy driving scenes, which come with ground truth&#10;annotations for semantic segmentation and object detection. Extensive&#10;experiments show that 1) supervised learning with our synthetic data&#10;significantly improves the performance of state-of-the-art CNN for SFSU on&#10;Foggy Driving; 2) our semi-supervised learning strategy further improves&#10;performance; and 3) image dehazing marginally advances SFSU with our learning&#10;strategy. The datasets, models and code are made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.01415" label="1901.01415">
-        <attvalues>
-          <attvalue for="0" value="Curriculum Model Adaptation with Synthetic and Real Data for Semantic&#10;  Foggy Scene Understanding" />
-          <attvalue for="1" value="  This work addresses the problem of semantic scene understanding under fog.&#10;Although marked progress has been made in semantic scene understanding, it is&#10;mainly concentrated on clear-weather scenes. Extending semantic segmentation&#10;methods to adverse weather conditions such as fog is crucial for outdoor&#10;applications. In this paper, we propose a novel method, named Curriculum Model&#10;Adaptation (CMAda), which gradually adapts a semantic segmentation model from&#10;light synthetic fog to dense real fog in multiple steps, using both labeled&#10;synthetic foggy data and unlabeled real foggy data. The method is based on the&#10;fact that the results of semantic segmentation in moderately adverse conditions&#10;(light fog) can be bootstrapped to solve the same problem in highly adverse&#10;conditions (dense fog). CMAda is extensible to other adverse conditions and&#10;provides a new paradigm for learning with synthetic data and unlabeled real&#10;data. In addition, we present three other main stand-alone contributions: 1) a&#10;novel method to add synthetic fog to real, clear-weather scenes using semantic&#10;input; 2) a new fog density estimator; 3) a novel fog densification method to&#10;densify the fog in real foggy scenes without using depth; and 4) the Foggy&#10;Zurich dataset comprising 3808 real foggy images, with pixel-level semantic&#10;annotations for 40 images under dense fog. Our experiments show that 1) our fog&#10;simulation and fog density estimator outperform their state-of-the-art&#10;counterparts with respect to the task of semantic foggy scene understanding&#10;(SFSU); 2) CMAda improves the performance of state-of-the-art models for SFSU&#10;significantly, benefiting both from our synthetic and real foggy data. The&#10;datasets and code are available at the project website.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.03997" label="1910.03997">
-        <attvalues>
-          <attvalue for="0" value="Semantic Understanding of Foggy Scenes with Purely Synthetic Data" />
-          <attvalue for="1" value="  This work addresses the problem of semantic scene understanding under foggy&#10;road conditions. Although marked progress has been made in semantic scene&#10;understanding over the recent years, it is mainly concentrated on clear weather&#10;outdoor scenes. Extending semantic segmentation methods to adverse weather&#10;conditions like fog is crucially important for outdoor applications such as&#10;self-driving cars. In this paper, we propose a novel method, which uses purely&#10;synthetic data to improve the performance on unseen real-world foggy scenes&#10;captured in the streets of Zurich and its surroundings. Our results highlight&#10;the potential and power of photo-realistic synthetic images for training and&#10;especially fine-tuning deep neural nets. Our contributions are threefold, 1) we&#10;created a purely synthetic, high-quality foggy dataset of 25,000 unique outdoor&#10;scenes, that we call Foggy Synscapes and plan to release publicly 2) we show&#10;that with this data we outperform previous approaches on real-world foggy test&#10;data 3) we show that a combination of our data and previously used data can&#10;even further improve the performance on real-world foggy data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.10349" label="1802.10349">
-        <attvalues>
-          <attvalue for="0" value="Learning to Adapt Structured Output Space for Semantic Segmentation" />
-          <attvalue for="1" value="  Convolutional neural network-based approaches for semantic segmentation rely&#10;on supervision with pixel-level ground truth, but may not generalize well to&#10;unseen image domains. As the labeling process is tedious and labor intensive,&#10;developing algorithms that can adapt source ground truth labels to the target&#10;domain is of great interest. In this paper, we propose an adversarial learning&#10;method for domain adaptation in the context of semantic segmentation.&#10;Considering semantic segmentations as structured outputs that contain spatial&#10;similarities between the source and target domains, we adopt adversarial&#10;learning in the output space. To further enhance the adapted model, we&#10;construct a multi-level adversarial network to effectively perform output space&#10;domain adaptation at different feature levels. Extensive experiments and&#10;ablation study are conducted under various domain adaptation settings,&#10;including synthetic-to-real and cross-city scenarios. We show that the proposed&#10;method performs favorably against the state-of-the-art methods in terms of&#10;accuracy and visual quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.08286" label="1804.08286">
-        <attvalues>
-          <attvalue for="0" value="Fully Convolutional Adaptation Networks for Semantic Segmentation" />
-          <attvalue for="1" value="  The recent advances in deep neural networks have convincingly demonstrated&#10;high capability in learning vision models on large datasets. Nevertheless,&#10;collecting expert labeled datasets especially with pixel-level annotations is&#10;an extremely expensive process. An appealing alternative is to render synthetic&#10;data (e.g., computer games) and generate ground truth automatically. However,&#10;simply applying the models learnt on synthetic images may lead to high&#10;generalization error on real images due to domain shift. In this paper, we&#10;facilitate this issue from the perspectives of both visual appearance-level and&#10;representation-level domain adaptation. The former adapts source-domain images&#10;to appear as if drawn from the &quot;style&quot; in the target domain and the latter&#10;attempts to learn domain-invariant representations. Specifically, we present&#10;Fully Convolutional Adaptation Networks (FCAN), a novel deep architecture for&#10;semantic segmentation which combines Appearance Adaptation Networks (AAN) and&#10;Representation Adaptation Networks (RAN). AAN learns a transformation from one&#10;domain to the other in the pixel space and RAN is optimized in an adversarial&#10;learning manner to maximally fool the domain discriminator with the learnt&#10;source and target representations. Extensive experiments are conducted on the&#10;transfer from GTA5 (game videos) to Cityscapes (urban street scenes) on&#10;semantic segmentation and our proposal achieves superior results when comparing&#10;to state-of-the-art unsupervised adaptation techniques. More remarkably, we&#10;obtain a new record: mIoU of 47.5% on BDDS (drive-cam videos) in an&#10;unsupervised setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.11556" label="1711.11556">
-        <attvalues>
-          <attvalue for="0" value="ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban&#10;  Scenes" />
-          <attvalue for="1" value="  Exploiting synthetic data to learn deep models has attracted increasing&#10;attention in recent years. However, the intrinsic domain difference between&#10;synthetic and real images usually causes a significant performance drop when&#10;applying the learned model to real world scenarios. This is mainly due to two&#10;reasons: 1) the model overfits to synthetic images, making the convolutional&#10;filters incompetent to extract informative representation for real images; 2)&#10;there is a distribution difference between synthetic and real data, which is&#10;also known as the domain adaptation problem. To this end, we propose a new&#10;reality oriented adaptation approach for urban scene semantic segmentation by&#10;learning from synthetic data. First, we propose a target guided distillation&#10;approach to learn the real image style, which is achieved by training the&#10;segmentation model to imitate a pretrained real style model using real images.&#10;Second, we further take advantage of the intrinsic spatial structure presented&#10;in urban scene images, and propose a spatial-aware adaptation scheme to&#10;effectively align the distribution of two domains. These two modules can be&#10;readily integrated with existing state-of-the-art semantic segmentation&#10;networks to improve their generalizability when adapting from synthetic to real&#10;urban scenes. We evaluate the proposed method on Cityscapes dataset by adapting&#10;from GTAV and SYNTHIA datasets, where the results demonstrate the effectiveness&#10;of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.12212" label="1903.12212">
-        <attvalues>
-          <attvalue for="0" value="All about Structure: Adapting Structural Information across Domains for&#10;  Boosting Semantic Segmentation" />
-          <attvalue for="1" value="  In this paper we tackle the problem of unsupervised domain adaptation for the&#10;task of semantic segmentation, where we attempt to transfer the knowledge&#10;learned upon synthetic datasets with ground-truth labels to real-world images&#10;without any annotation. With the hypothesis that the structural content of&#10;images is the most informative and decisive factor to semantic segmentation and&#10;can be readily shared across domains, we propose a Domain Invariant Structure&#10;Extraction (DISE) framework to disentangle images into domain-invariant&#10;structure and domain-specific texture representations, which can further&#10;realize image-translation across domains and enable label transfer to improve&#10;segmentation performance. Extensive experiments verify the effectiveness of our&#10;proposed DISE model and demonstrate its superiority over several&#10;state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.00867" label="2003.00867">
-        <attvalues>
-          <attvalue for="0" value="Learning Texture Invariant Representation for Domain Adaptation of&#10;  Semantic Segmentation" />
-          <attvalue for="1" value="  Since annotating pixel-level labels for semantic segmentation is laborious,&#10;leveraging synthetic data is an attractive solution. However, due to the domain&#10;gap between synthetic domain and real domain, it is challenging for a model&#10;trained with synthetic data to generalize to real data. In this paper,&#10;considering the fundamental difference between the two domains as the texture,&#10;we propose a method to adapt to the texture of the target domain. First, we&#10;diversity the texture of synthetic images using a style transfer algorithm. The&#10;various textures of generated images prevent a segmentation model from&#10;overfitting to one specific (synthetic) texture. Then, we fine-tune the model&#10;with self-training to get direct supervision of the target texture. Our results&#10;achieve state-of-the-art performance and we analyze the properties of the model&#10;trained on the stylized dataset with extensive experiments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.03267" label="2108.03267">
-        <attvalues>
-          <attvalue for="0" value="BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in&#10;  Semantic Scene Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation aims to predict pixel-level labels. It has become a&#10;popular task in various computer vision applications. While fully supervised&#10;segmentation methods have achieved high accuracy on large-scale vision&#10;datasets, they are unable to generalize on a new test environment or a new&#10;domain well. In this work, we first introduce a new Un-aligned Domain Score to&#10;measure the efficiency of a learned model on a new target domain in&#10;unsupervised manner. Then, we present the new Bijective Maximum&#10;Likelihood(BiMaL) loss that is a generalized form of the Adversarial Entropy&#10;Minimization without any assumption about pixel independence. We have evaluated&#10;the proposed BiMaL on two domains. The proposed BiMaL approach consistently&#10;outperforms the SOTA methods on empirical experiments on &quot;SYNTHIA to&#10;Cityscapes&quot;, &quot;GTA5 to Cityscapes&quot;, and &quot;SYNTHIA to Vistas&quot;.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.09222" label="2007.09222">
-        <attvalues>
-          <attvalue for="0" value="Classes Matter: A Fine-grained Adversarial Approach to Cross-domain&#10;  Semantic Segmentation" />
-          <attvalue for="1" value="  Despite great progress in supervised semantic segmentation,a large&#10;performance drop is usually observed when deploying the model in the wild.&#10;Domain adaptation methods tackle the issue by aligning the source domain and&#10;the target domain. However, most existing methods attempt to perform the&#10;alignment from a holistic view, ignoring the underlying class-level data&#10;structure in the target domain. To fully exploit the supervision in the source&#10;domain, we propose a fine-grained adversarial learning strategy for class-level&#10;feature alignment while preserving the internal structure of semantics across&#10;domains. We adopt a fine-grained domain discriminator that not only plays as a&#10;domain distinguisher, but also differentiates domains at class level. The&#10;traditional binary domain labels are also generalized to domain encodings as&#10;the supervision signal to guide the fine-grained feature alignment. An analysis&#10;with Class Center Distance (CCD) validates that our fine-grained adversarial&#10;strategy achieves better class-level alignment compared to other&#10;state-of-the-art methods. Our method is easy to implement and its effectiveness&#10;is evaluated on three classical domain adaptation tasks, i.e., GTA5 to&#10;Cityscapes, SYNTHIA to Cityscapes and Cityscapes to Cross-City. Large&#10;performance gains show that our method outperforms other global feature&#10;alignment based and class-wise alignment based counterparts. The code is&#10;publicly available at https://github.com/JDAI-CV/FADA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.05498" label="2004.05498">
-        <attvalues>
-          <attvalue for="0" value="FDA: Fourier Domain Adaptation for Semantic Segmentation" />
-          <attvalue for="1" value="  We describe a simple method for unsupervised domain adaptation, whereby the&#10;discrepancy between the source and target distributions is reduced by swapping&#10;the low-frequency spectrum of one with the other. We illustrate the method in&#10;semantic segmentation, where densely annotated images are aplenty in one domain&#10;(synthetic data), but difficult to obtain in another (real images). Current&#10;state-of-the-art methods are complex, some requiring adversarial optimization&#10;to render the backbone of a neural network invariant to the discrete domain&#10;selection variable. Our method does not require any training to perform the&#10;domain alignment, just a simple Fourier Transform and its inverse. Despite its&#10;simplicity, it achieves state-of-the-art performance in the current benchmarks,&#10;when integrated into a relatively standard semantic segmentation model. Our&#10;results indicate that even simple procedures can discount nuisance variability&#10;in the data that more sophisticated methods struggle to learn away.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.00589" label="1909.00589">
-        <attvalues>
-          <attvalue for="0" value="Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation&#10;  in Semantic Segmentation" />
-          <attvalue for="1" value="  Deep learning-based semantic segmentation methods have an intrinsic&#10;limitation that training a model requires a large amount of data with&#10;pixel-level annotations. To address this challenging issue, many researchers&#10;give attention to unsupervised domain adaptation for semantic segmentation.&#10;Unsupervised domain adaptation seeks to adapt the model trained on the source&#10;domain to the target domain. In this paper, we introduce a self-ensembling&#10;technique, one of the successful methods for domain adaptation in&#10;classification. However, applying self-ensembling to semantic segmentation is&#10;very difficult because heavily-tuned manual data augmentation used in&#10;self-ensembling is not useful to reduce the large domain gap in the semantic&#10;segmentation. To overcome this limitation, we propose a novel framework&#10;consisting of two components, which are complementary to each other. First, we&#10;present a data augmentation method based on Generative Adversarial Networks&#10;(GANs), which is computationally efficient and effective to facilitate domain&#10;alignment. Given those augmented images, we apply self-ensembling to enhance&#10;the performance of the segmentation network on the target domain. The proposed&#10;method outperforms state-of-the-art semantic segmentation methods on&#10;unsupervised domain adaptation benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.12197" label="2008.12197">
-        <attvalues>
-          <attvalue for="0" value="Instance Adaptive Self-Training for Unsupervised Domain Adaptation" />
-          <attvalue for="1" value="  The divergence between labeled training data and unlabeled testing data is a&#10;significant challenge for recent deep learning models. Unsupervised domain&#10;adaptation (UDA) attempts to solve such a problem. Recent works show that&#10;self-training is a powerful approach to UDA. However, existing methods have&#10;difficulty in balancing scalability and performance. In this paper, we propose&#10;an instance adaptive self-training framework for UDA on the task of semantic&#10;segmentation. To effectively improve the quality of pseudo-labels, we develop a&#10;novel pseudo-label generation strategy with an instance adaptive selector.&#10;Besides, we propose the region-guided regularization to smooth the pseudo-label&#10;region and sharpen the non-pseudo-label region. Our method is so concise and&#10;efficient that it is easy to be generalized to other unsupervised domain&#10;adaptation methods. Experiments on 'GTA5 to Cityscapes' and 'SYNTHIA to&#10;Cityscapes' demonstrate the superior performance of our approach compared with&#10;the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.08702" label="2007.08702">
-        <attvalues>
-          <attvalue for="0" value="DACS: Domain Adaptation via Cross-domain Mixed Sampling" />
-          <attvalue for="1" value="  Semantic segmentation models based on convolutional neural networks have&#10;recently displayed remarkable performance for a multitude of applications.&#10;However, these models typically do not generalize well when applied on new&#10;domains, especially when going from synthetic to real data. In this paper we&#10;address the problem of unsupervised domain adaptation (UDA), which attempts to&#10;train on labelled data from one domain (source domain), and simultaneously&#10;learn from unlabelled data in the domain of interest (target domain). Existing&#10;methods have seen success by training on pseudo-labels for these unlabelled&#10;images. Multiple techniques have been proposed to mitigate low-quality&#10;pseudo-labels arising from the domain shift, with varying degrees of success.&#10;We propose DACS: Domain Adaptation via Cross-domain mixed Sampling, which mixes&#10;images from the two domains along with the corresponding labels and&#10;pseudo-labels. These mixed samples are then trained on, in addition to the&#10;labelled data itself. We demonstrate the effectiveness of our solution by&#10;achieving state-of-the-art results for GTA5 to Cityscapes, a common&#10;synthetic-to-real semantic segmentation benchmark for UDA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.00097" label="2105.00097">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Augmentation Consistency for Adapting Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  We propose an approach to domain adaptation for semantic segmentation that is&#10;both practical and highly accurate. In contrast to previous work, we abandon&#10;the use of computationally involved adversarial objectives, network ensembles&#10;and style transfer. Instead, we employ standard data augmentation techniques&#10;$-$ photometric noise, flipping and scaling $-$ and ensure consistency of the&#10;semantic predictions across these image transformations. We develop this&#10;principle in a lightweight self-supervised framework trained on co-evolving&#10;pseudo labels without the need for cumbersome extra training rounds. Simple in&#10;training from a practitioner's standpoint, our approach is remarkably&#10;effective. We achieve significant improvements of the state-of-the-art&#10;segmentation accuracy after adaptation, consistent both across different&#10;choices of the backbone architecture and adaptation scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.05254" label="2103.05254">
-        <attvalues>
-          <attvalue for="0" value="MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised&#10;  Domain Adaptation in Semantic Segmentation" />
-          <attvalue for="1" value="  Unsupervised domain adaptation (UDA) aims to transfer the knowledge from the&#10;labeled source domain to the unlabeled target domain. Existing self-training&#10;based UDA approaches assign pseudo labels for target data and treat them as&#10;ground truth labels to fully leverage unlabeled target data for model&#10;adaptation. However, the generated pseudo labels from the model optimized on&#10;the source domain inevitably contain noise due to the domain gap. To tackle&#10;this issue, we advance a MetaCorrection framework, where a Domain-aware&#10;Meta-learning strategy is devised to benefit Loss Correction (DMLC) for UDA&#10;semantic segmentation. In particular, we model the noise distribution of pseudo&#10;labels in target domain by introducing a noise transition matrix (NTM) and&#10;construct meta data set with domain-invariant source data to guide the&#10;estimation of NTM. Through the risk minimization on the meta data set, the&#10;optimized NTM thus can correct the noisy issues in pseudo labels and enhance&#10;the generalization ability of the model on the target data. Considering the&#10;capacity gap between shallow and deep features, we further employ the proposed&#10;DMLC strategy to provide matched and compatible supervision signals for&#10;different level features, thereby ensuring deep adaptation. Extensive&#10;experimental results highlight the effectiveness of our method against existing&#10;state-of-the-art methods on three benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.09600" label="2107.09600">
-        <attvalues>
-          <attvalue for="0" value="DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Unsupervised domain adaptation (UDA) for semantic segmentation aims to adapt&#10;a segmentation model trained on the labeled source domain to the unlabeled&#10;target domain. Existing methods try to learn domain invariant features while&#10;suffering from large domain gaps that make it difficult to correctly align&#10;discrepant features, especially in the initial training phase. To address this&#10;issue, we propose a novel Dual Soft-Paste (DSP) method in this paper.&#10;Specifically, DSP selects some classes from a source domain image using a&#10;long-tail class first sampling strategy and softly pastes the corresponding&#10;image patch on both the source and target training images with a fusion weight.&#10;Technically, we adopt the mean teacher framework for domain adaptation, where&#10;the pasted source and target images go through the student network while the&#10;original target image goes through the teacher network. Output-level alignment&#10;is carried out by aligning the probability maps of the target fused image from&#10;both networks using a weighted cross-entropy loss. In addition, feature-level&#10;alignment is carried out by aligning the feature maps of the source and target&#10;images from student network using a weighted maximum mean discrepancy loss. DSP&#10;facilitates the model learning domain-invariant features from the intermediate&#10;domains, leading to faster convergence and better performance. Experiments on&#10;two challenging benchmarks demonstrate the superiority of DSP over&#10;state-of-the-art methods. Code is available at&#10;\url{https://github.com/GaoLii/DSP}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.10979" label="2101.10979">
-        <attvalues>
-          <attvalue for="0" value="Prototypical Pseudo Label Denoising and Target Structure Learning for&#10;  Domain Adaptive Semantic Segmentation" />
-          <attvalue for="1" value="  Self-training is a competitive approach in domain adaptive segmentation,&#10;which trains the network with the pseudo labels on the target domain. However&#10;inevitably, the pseudo labels are noisy and the target features are dispersed&#10;due to the discrepancy between source and target domains. In this paper, we&#10;rely on representative prototypes, the feature centroids of classes, to address&#10;the two issues for unsupervised domain adaptation. In particular, we take one&#10;step further and exploit the feature distances from prototypes that provide&#10;richer information than mere prototypes. Specifically, we use it to estimate&#10;the likelihood of pseudo labels to facilitate online correction in the course&#10;of training. Meanwhile, we align the prototypical assignments based on relative&#10;feature distances for two different views of the same target, producing a more&#10;compact target feature space. Moreover, we find that distilling the already&#10;learned knowledge to a self-supervised pretrained model further boosts the&#10;performance. Our method shows tremendous performance advantage over&#10;state-of-the-art methods. We will make the code publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.01530" label="1903.01530">
-        <attvalues>
-          <attvalue for="0" value="Learning of Image Dehazing Models for Segmentation Tasks" />
-          <attvalue for="1" value="  To evaluate their performance, existing dehazing approaches generally rely on&#10;distance measures between the generated image and its corresponding ground&#10;truth. Despite its ability to produce visually good images, using pixel-based&#10;or even perceptual metrics do not guarantee, in general, that the produced&#10;image is fit for being used as input for low-level computer vision tasks such&#10;as segmentation. To overcome this weakness, we are proposing a novel end-to-end&#10;approach for image dehazing, fit for being used as input to an image&#10;segmentation procedure, while maintaining the visual quality of the generated&#10;images. Inspired by the success of Generative Adversarial Networks (GAN), we&#10;propose to optimize the generator by introducing a discriminator network and a&#10;loss function that evaluates segmentation quality of dehazed images. In&#10;addition, we make use of a supplementary loss function that verifies that the&#10;visual and the perceptual quality of the generated image are preserved in hazy&#10;conditions. Results obtained using the proposed technique are appealing, with a&#10;favorable comparison to state-of-the-art approaches when considering the&#10;performance of segmentation algorithms on the hazy images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08396" label="1803.08396">
-        <attvalues>
-          <attvalue for="0" value="Densely Connected Pyramid Dehazing Network" />
-          <attvalue for="1" value="  We propose a new end-to-end single image dehazing method, called Densely&#10;Connected Pyramid Dehazing Network (DCPDN), which can jointly learn the&#10;transmission map, atmospheric light and dehazing all together. The end-to-end&#10;learning is achieved by directly embedding the atmospheric scattering model&#10;into the network, thereby ensuring that the proposed method strictly follows&#10;the physics-driven scattering model for dehazing. Inspired by the dense network&#10;that can maximize the information flow along features from different levels, we&#10;propose a new edge-preserving densely connected encoder-decoder structure with&#10;multi-level pyramid pooling module for estimating the transmission map. This&#10;network is optimized using a newly introduced edge-preserving loss function. To&#10;further incorporate the mutual structural information between the estimated&#10;transmission map and the dehazed result, we propose a joint-discriminator based&#10;on generative adversarial network framework to decide whether the corresponding&#10;dehazed image and the estimated transmission map are real or fake. An ablation&#10;study is conducted to demonstrate the effectiveness of each module evaluated at&#10;both estimated transmission map and dehazed result. Extensive experiments&#10;demonstrate that the proposed method achieves significant improvements over the&#10;state-of-the-art methods. Code will be made available at:&#10;https://github.com/hezhangsprinter&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.00213" label="1804.00213">
-        <attvalues>
-          <attvalue for="0" value="Gated Fusion Network for Single Image Dehazing" />
-          <attvalue for="1" value="  In this paper, we propose an efficient algorithm to directly restore a clear&#10;image from a hazy input. The proposed algorithm hinges on an end-to-end&#10;trainable neural network that consists of an encoder and a decoder. The encoder&#10;is exploited to capture the context of the derived input images, while the&#10;decoder is employed to estimate the contribution of each input to the final&#10;dehazed result using the learned representations attributed to the encoder. The&#10;constructed network adopts a novel fusion-based strategy which derives three&#10;inputs from an original hazy image by applying White Balance (WB), Contrast&#10;Enhancing (CE), and Gamma Correction (GC). We compute pixel-wise confidence&#10;maps based on the appearance differences between these different inputs to&#10;blend the information of the derived inputs and preserve the regions with&#10;pleasant visibility. The final dehazed image is yielded by gating the important&#10;features of the derived inputs. To train the network, we introduce a&#10;multi-scale approach such that the halo artifacts can be avoided. Extensive&#10;experimental results on both synthetic and real-world images demonstrate that&#10;the proposed algorithm performs favorably against the state-of-the-art&#10;algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.05716" label="1810.05716">
-        <attvalues>
-          <attvalue for="0" value="Does Haze Removal Help CNN-based Image Classification?" />
-          <attvalue for="1" value="  Hazy images are common in real scenarios and many dehazing methods have been&#10;developed to automatically remove the haze from images. Typically, the goal of&#10;image dehazing is to produce clearer images from which human vision can better&#10;identify the object and structural details present in the images. When the&#10;ground-truth haze-free image is available for a hazy image, quantitative&#10;evaluation of image dehazing is usually based on objective metrics, such as&#10;Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). However, in&#10;many applications, large-scale images are collected not for visual examination&#10;by human. Instead, they are used for many high-level vision tasks, such as&#10;automatic classification, recognition and categorization. One fundamental&#10;problem here is whether various dehazing methods can produce clearer images&#10;that can help improve the performance of the high-level tasks. In this paper,&#10;we empirically study this problem in the important task of image classification&#10;by using both synthetic and real hazy image datasets. From the experimental&#10;results, we find that the existing image-dehazing methods cannot improve much&#10;the image-classification performance and sometimes even reduce the&#10;image-classification performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.15618" label="2006.15618">
-        <attvalues>
-          <attvalue for="0" value="Analogical Image Translation for Fog Generation" />
-          <attvalue for="1" value="  Image-to-image translation is to map images from a given \emph{style} to&#10;another given \emph{style}. While exceptionally successful, current methods&#10;assume the availability of training images in both source and target domains,&#10;which does not always hold in practice. Inspired by humans' reasoning&#10;capability of analogy, we propose analogical image translation (AIT). Given&#10;images of two styles in the source domain: $\mathcal{A}$ and&#10;$\mathcal{A}^\prime$, along with images $\mathcal{B}$ of the first style in the&#10;target domain, learn a model to translate $\mathcal{B}$ to $\mathcal{B}^\prime$&#10;in the target domain, such that $\mathcal{A}:\mathcal{A}^\prime&#10;::\mathcal{B}:\mathcal{B}^\prime$. AIT is especially useful for translation&#10;scenarios in which training data of one style is hard to obtain but training&#10;data of the same two styles in another domain is available. For instance, in&#10;the case from normal conditions to extreme, rare conditions, obtaining real&#10;training images for the latter case is challenging but obtaining synthetic data&#10;for both cases is relatively easy. In this work, we are interested in adding&#10;adverse weather effects, more specifically fog effects, to images taken in&#10;clear weather. To circumvent the challenge of collecting real foggy images, AIT&#10;learns with synthetic clear-weather images, synthetic foggy images and real&#10;clear-weather images to add fog effects onto real clear-weather images without&#10;seeing any real foggy images during training. AIT achieves this zero-shot image&#10;translation capability by coupling a supervised training scheme in the&#10;synthetic domain, a cycle consistency strategy in the real domain, an&#10;adversarial training scheme between the two domains, and a novel network&#10;design. Experiments show the effectiveness of our method for zero-short image&#10;translation and its benefit for downstream tasks such as semantic foggy scene&#10;understanding.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.05571" label="2004.05571">
-        <attvalues>
-          <attvalue for="0" value="Cross-domain Correspondence Learning for Exemplar-based Image&#10;  Translation" />
-          <attvalue for="1" value="  We present a general framework for exemplar-based image translation, which&#10;synthesizes a photo-realistic image from the input in a distinct domain (e.g.,&#10;semantic segmentation mask, or edge map, or pose keypoints), given an exemplar&#10;image. The output has the style (e.g., color, texture) in consistency with the&#10;semantically corresponding objects in the exemplar. We propose to jointly learn&#10;the crossdomain correspondence and the image translation, where both tasks&#10;facilitate each other and thus can be learned with weak supervision. The images&#10;from distinct domains are first aligned to an intermediate domain where dense&#10;correspondence is established. Then, the network synthesizes images based on&#10;the appearance of semantically corresponding patches in the exemplar. We&#10;demonstrate the effectiveness of our approach in several image translation&#10;tasks. Our method is superior to state-of-the-art methods in terms of image&#10;quality significantly, with the image style faithful to the exemplar with&#10;semantic consistency. Moreover, we show the utility of our method for several&#10;applications&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Image-to-image translation~~&#10;The goal of image translation is to learn the mapping between different image domains. Most prominent contemporary approaches solve this problem through conditional generative adversarial network~\cite{mirza2014conditional} that leverages either paired data~\cite{isola2017image,wang2018high,park2019semantic} or unpaired data~\cite{zhu2017unpaired,yi2017dualgan,kim2017learning,liu2017unsupervised,royer2017xgan}. Since the mapping from one image domain to another is inherently multi-modal, following works promote the synthesis diversity by performing stochastic sampling from the latent space~\cite{zhu2017toward,huang2018multimodal,lee2018diverse}. However, none of these methods allow delicate control of the output since the latent representation is rather complex and does not have an explicit correspondence to image style. In contrast, our method supports customization of the result according to a user-given exemplar, which allows more flexible control for multi-modal generation.&#10;&#10;Exemplar-based image synthesis~~&#10;Very recently, a few works~\cite{qi2018semi,wang2019example,ma2018exemplar,riviere2019inspirational,bansal2019shapes} propose to synthesize photorealistic images from semantic layout under the guidance of exemplars. Non-parametric or semi-parametric approaches~\cite{qi2018semi,bansal2019shapes} synthesize images by compositing the image fragments retrieved from a large database. &#10;Mainstream works, however, formulate the problem as image-to-image translation. \cite{huang2018multimodal} and \cite{ma2018exemplar} propose to employ Adaptive Instance Normalization (AdaIN)~\cite{huang2017arbitrary} to transfer the style code from the exemplar to the source image. \cite{park2019semantic} learn an encoder to map the exemplar image into a vector from which the images are further synthesized. The style consistency discriminator is proposed in~\cite{wang2019example} to examine whether the image pairs exhibit a similar style. However, this method requires to constitute style consistency image pairs from video clips, which makes it unsuitable for general image translation. Unlike all of the above methods that only transfer the global style, our method transfers the fine style from a semantically corresponding region of the exemplar. Our work is inspired by the recent exemplar-based image colorization~\cite{zhang2019deep,he2018deep}, but we solve a more general problem: translating images between distinct domains.&#10;&#10;Semantic correspondence~~&#10;Early studies~\cite{lowe2004distinctive,dalal2005histograms,tola2009daisy} on semantic correspondence focus on matching hand-crafted features. With the advent of the convolutional neural network, deep features are proven powerful to represent the high-level semantics. \cite{long2014convnets} first propose to establish semantic correspondence by matching deep features extracted from a pretrained classification model. Following works further improve the correspondence quality by incorporating additional annotations~\cite{zhou2016learning,choy2016universal,ham2017proposal,han2017scnet,kim2017fcss,lee2019sfnet}, adopting coarse-to-fine strategy~\cite{liao2017visual} or retaining reliable sparse matchings~\cite{aberman2018neural}.&#10;However, all these methods can only handle the correspondence between natural images instead of cross-domain images, \eg, edge and photorealistic images. We explore this new scenario and implicitly learns the task with weak supervision.&#10;&#10;" />
-          <attvalue for="4" value="Image Translation, Machine Learning, Engineering, Computer Vision, Cross-Domain Learning, Mathematics, Artificial Intelligence, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2209.15475" label="2209.15475">
-        <attvalues>
-          <attvalue for="0" value="Point Cloud Quality Assessment using 3D Saliency Maps" />
-          <attvalue for="1" value="  Point cloud quality assessment (PCQA) has become an appealing research field&#10;in recent days. Considering the importance of saliency detection in quality&#10;assessment, we propose an effective full-reference PCQA metric which makes the&#10;first attempt to utilize the saliency information to facilitate quality&#10;prediction, called point cloud quality assessment using 3D saliency maps&#10;(PQSM). Specifically, we first propose a projection-based point cloud saliency&#10;map generation method, in which depth information is introduced to better&#10;reflect the geometric characteristics of point clouds. Then, we construct point&#10;cloud local neighborhoods to derive three structural descriptors to indicate&#10;the geometry, color and saliency discrepancies. Finally, a saliency-based&#10;pooling strategy is proposed to generate the final quality score. Extensive&#10;experiments are performed on four independent PCQA databases. The results&#10;demonstrate that the proposed PQSM shows competitive performances compared to&#10;multiple state-of-the-art PCQA metrics.&#10;" />
-          <attvalue for="2" value="&#10;&#10;W{ith} the rapid development of 3D capturing technologies, point clouds (PCs) have emerged as a popular and prominent format to represent 3D photo-realistic content. A PC is composed of a great number of disordered points, and each point is attached with the information of its 3D coordinates and additional attributes such as RGB color and normals. In recent years, we have witnessed the occurrence of many applications based on PCs (e.g., navigation \cite{Navigation}, industrial robotics\cite{robotics} and mixed reality\cite{mixedreal}, etc) which require PC data with high quality. However, a variety of distortions could be induced during the relevant processing of PCs, such as acquisition, compression \cite{Compre1}\cite{Compre2}, transmission \cite{transmiss}, and rendering, which may impair the perceptual quality of the human visual system (HVS). Therefore, point cloud quality assessment (PCQA) has become an appealing research field to ensure provide high-quality human perception service.&#10;&#10;PCQA aims to directly reflect or approximate the perception quality of HVS for specific PCs. Subjective PCQA is the most straightforward and reliable method, but also time-consuming and expensive. Thus, effective objective PCQA metrics which correlate well with HVS for practical application are highly desired. In this paper, we focus on the full-reference (FR) objective PCQA metrics. Existing FR-PCQA metrics either project 3D PCs onto 2D planes, utilizing the well-developed image quality assessment (IQA) algorithms; or inferring the perceptual quality by modeling the 3D properties. In terms of the former, projection-based metrics like \cite{SJTUPCQA} project PCs onto several perpendicular image planes and aggregate image-based features to evaluate the PC quality. In terms of the latter, point-wise metrics, such as the point-to-point\cite{p2po} and the point-to-plane\cite{p2pl}, calculate the differences of the Euclidean distance related features between the corresponding point pairs. Meanwhile, some other metrics, e.g., PointSSIM\cite{pointSSIM}, GraphSIM\cite{GraphSIM}, MS-GraphSIM\cite{MSGraphSIM} and MPED\cite{MPED}, construct local neighborhoods to involve the structural features, which usually have better performances due to the fact that HVS is more sensitive to structural information.&#10;&#10;One of the things that above methods overlooked is that they hardly consider the vital role of visual saliency in quality prediction. Specifically, the visual saliency is an important characteristic of HVS which can reflect the human attention distribution. In the domain of image quality assessment, some metrics integrated with saliency information have achieved great success, such as VSI \cite{VSI}. One reason that there are few PCQA metrics invloved with visual saliency is due to the lag of PC saliency detection. For the PC saliency detection, most methods are directly conducted on uncolored 3D PCs\cite{SalGuo}\cite{SalDing}, which is usually utilized in machine vision tasks rather than human vision tasks. &#10;&#10;Inspired by the strategy proposed in \cite{SalMaps}, we use the projection to transfer the achievement of image saliency detection to PC and propose a novel saliency guided metric, called the point cloud quality assessment using 3D saliency maps (PQSM). To our best knowledge, this is the first FR-PCQA metric utilizing the saliency information. Specifically, we first propose to incorporate depth-related maps into the projection-based saliency detection model to involve the intrinsic geometric information of PCs. We then design three descriptors to quantitively measure the distortions on geometry, color and saliency maps. After that, a saliency-based pooling strategy is adopted to generate the final score. Our method is examined by extensive experiments on four publicly accessible PCQA databases. The results show the competitive performance of our proposed model in inferring the perceived quality of distorted PCs.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Theory, Computer Vision Techniques, Computer Vision, Engineering, Mathematics, Quality Assessment Metrics, Computer Science, Point Cloud Analysis, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2006.00497" label="2006.00497">
-        <attvalues>
-          <attvalue for="0" value="Inferring Point Cloud Quality via Graph Similarity" />
-          <attvalue for="1" value="  We propose the GraphSIM -- an objective metric to accurately predict the&#10;subjective quality of point cloud with superimposed geometry and color&#10;impairments. Motivated by the facts that human vision system is more sensitive&#10;to the high spatial-frequency components (e.g., contours, edges), and weighs&#10;more to the local structural variations rather individual point intensity, we&#10;first extract geometric keypoints by resampling the reference point cloud&#10;geometry information to form the object skeleton; we then construct local&#10;graphs centered at these keypoints for both reference and distorted point&#10;clouds, followed by collectively aggregating color gradient moments (e.g.,&#10;zeroth, first, and second) that are derived between all other points and&#10;centered keypoint in the same local graph for significant feature similarity&#10;(a.k.a., local significance) measurement; Final similarity index is obtained by&#10;pooling the local graph significance across all color channels and by averaging&#10;across all graphs. Our GraphSIM is validated using two large and independent&#10;point cloud assessment datasets that involve a wide range of impairments (e.g.,&#10;re-sampling, compression, additive noise), reliably demonstrating the&#10;state-of-the-art performance for all distortions with noticeable gains in&#10;predicting the subjective mean opinion score (MOS), compared with those&#10;point-wise distance-based metrics adopted in standardization reference&#10;software. Ablation studies have further shown that GraphSIM is generalized to&#10;various scenarios with consistent performance by examining its key modules and&#10;parameters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.13899" label="2209.13899">
-        <attvalues>
-          <attvalue for="0" value="Strong Instance Segmentation Pipeline for MMSports Challenge" />
-          <attvalue for="1" value="  The goal of ACM MMSports2022 DeepSportRadar Instance Segmentation Challenge&#10;is to tackle the segmentation of individual humans including players, coaches&#10;and referees on a basketball court. And the main characteristics of this&#10;challenge are there is a high level of occlusions between players and the&#10;amount of data is quite limited. In order to address these problems, we&#10;designed a strong instance segmentation pipeline. Firstly, we employed a proper&#10;data augmentation strategy for this task mainly including photometric&#10;distortion transform and copy-paste strategy, which can generate more image&#10;instances with a wider distribution. Secondly, we employed a strong&#10;segmentation model, Hybrid Task Cascade based detector on the Swin-Base based&#10;CBNetV2 backbone, and we add MaskIoU head to HTCMaskHead that can simply and&#10;effectively improve the performance of instance segmentation. Finally, the SWA&#10;training strategy was applied to improve the performance further. Experimental&#10;results demonstrate the proposed pipeline can achieve a competitive result on&#10;the DeepSportRadar challenge, with 0.768AP@0.50:0.95 on the challenge set.&#10;Source code is available at&#10;https://github.com/YJingyu/Instanc_Segmentation_Pro.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Instance segmentation applies widely in image editing, image analysis and autonomous driving, etc. Instance segmentation is a fundamental problem in computer vision. Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}.&#10;&#10;In addition, transformers have made enormous strides in NLP\cite{devlin2018bert, radford2019language}. There are quite a bit of works applying transformers to computer vision\cite{ramachandran2019stand, Zhao_2020_CVPR, carion2020end}, because transformers can capture the non-local and relational nature of images. Especially, Swin Transformer\cite{liu2021swin} has been widely used for many computer vision tasks and achieved successful results, such as detection and segmentation tasks on COCO.&#10;&#10;In order to address the problems of data-deficient and occlusions, we employed a proper data augmentation strategy mainly including two components, photometric distortion transform and copy-paste strategy, which can increase the diversity of data distribution effectively. And then we employed a strong segmentation model, and add mask scoring~\cite{huang2019mask} to the mask head. Finally, an effective training strategy is applied to improve the model performance further. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1803.01534" label="1803.01534">
-        <attvalues>
-          <attvalue for="0" value="Path Aggregation Network for Instance Segmentation" />
-          <attvalue for="1" value="  The way that information propagates in neural networks is of great&#10;importance. In this paper, we propose Path Aggregation Network (PANet) aiming&#10;at boosting information flow in proposal-based instance segmentation framework.&#10;Specifically, we enhance the entire feature hierarchy with accurate&#10;localization signals in lower layers by bottom-up path augmentation, which&#10;shortens the information path between lower layers and topmost feature. We&#10;present adaptive feature pooling, which links feature grid and all feature&#10;levels to make useful information in each feature level propagate directly to&#10;following proposal subnetworks. A complementary branch capturing different&#10;views for each proposal is created to further improve mask prediction. These&#10;improvements are simple to implement, with subtle extra computational overhead.&#10;Our PANet reaches the 1st place in the COCO 2017 Challenge Instance&#10;Segmentation task and the 2nd place in Object Detection task without&#10;large-batch training. It is also state-of-the-art on MVD and Cityscapes. Code&#10;is available at https://github.com/ShuLiu1993/PANet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.04446" label="2004.04446">
-        <attvalues>
-          <attvalue for="0" value="CenterMask: single shot instance segmentation with point representation" />
-          <attvalue for="1" value="  In this paper, we propose a single-shot instance segmentation method, which&#10;is simple, fast and accurate. There are two main challenges for one-stage&#10;instance segmentation: object instances differentiation and pixel-wise feature&#10;alignment. Accordingly, we decompose the instance segmentation into two&#10;parallel subtasks: Local Shape prediction that separates instances even in&#10;overlapping conditions, and Global Saliency generation that segments the whole&#10;image in a pixel-to-pixel manner. The outputs of the two branches are assembled&#10;to form the final instance masks. To realize that, the local shape information&#10;is adopted from the representation of object center points. Totally trained&#10;from scratch and without any bells and whistles, the proposed CenterMask&#10;achieves 34.5 mask AP with a speed of 12.3 fps, using a single-model with&#10;single-scale training/testing on the challenging COCO dataset. The accuracy is&#10;higher than all other one-stage instance segmentation methods except the 5&#10;times slower TensorMask, which shows the effectiveness of CenterMask. Besides,&#10;our method can be easily embedded to other one-stage object detectors such as&#10;FCOS and performs well, showing the generalization of CenterMask.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Two-stage Instance Segmentation: Two-stage instance segmentation method often follows the detect-then-segment paradigm, which first performs bounding box detection and then classifies the pixels in the bounding box area to obtain the final mask. &#10;Mask R-CNN \cite{he2017mask} extends the successful Faster R-CNN \cite{ren2015faster} detector by adding a mask segmentation branch on each Region of Interest area. To preserve the exact spatial locations, it introduces the RoIAlign module to fix the pixel misalignment problem. &#10;PANet \cite{liu2018path} aims to improve the information propagation of Mask R-CNN by introducing bottom-up path augmentation, adaptive feature pooling, and fully-connected fusion.&#10;Mask Scoring R-CNN \cite{huang2019mask} proposes a mask scoring module instead of the classification score to evaluate the mask, which can improve the quality of the segmented mask.&#10;&#10;Although two-stage instance segmentation methods achieve state-of-the-art performance, these models are often complicated and slow. Advances of one-stage object detection motivate us to develop faster and simpler one-stage instance segmentation methods.&#10;&#10;One-stage Instance Segmentation: State-of-the-art one-stage instance segmentation methods can be roughly divided into two categories: global-area-based and local-area-based approaches. &#10; Global-area-based methods first generate intermediate and shared feature maps based on the whole image, then assemble the extracted features to form the final masks for each instance.&#10;&#10;InstanceFCN \cite{dai2016instance} utilizes FCN \cite{long2015fully} to generate multiple instance-sensitive score maps which contain the relative positions to objects instances, then applies an assembling module to output object instances. YOLACT \cite{bolya-iccv2019} generates multiple prototype masks of the global image, then utilizes per-instance mask coefficients to produce the instance level mask. Global-area-based methods can maintain the pixel-to-pixel alignment which makes masks precise, but performs worse when objects overlap.&#10;In contrast to these methods, local-area-based methods output instance masks on each local region directly.&#10;PolarMask \cite{xie2019polarmask} represents mask in its contour form and utilizes rays from the center to describe the contour, but the polygon surrounded by the contour can not depict the mask precisely and can not describe objects that have holes in the center. &#10;TensorMask \cite{chen2019tensormask} utilizes structured 4D tensors to represent masks over a spatial domain, it also introduces aligned representation and tensor bipyramid to recover spatial details, but these align operations make the network even slower than the two-stage Mask R-CNN \cite{he2017mask}. &#10;&#10;Different from the above approaches, CenterMask contains both a Global Saliency generation branch and a Local Shape prediction branch, and integrates them to preserve pixel alignment and separate objects simultaneously.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1912.04488" label="1912.04488">
-        <attvalues>
-          <attvalue for="0" value="SOLO: Segmenting Objects by Locations" />
-          <attvalue for="1" value="  We present a new, embarrassingly simple approach to instance segmentation in&#10;images. Compared to many other dense prediction tasks, e.g., semantic&#10;segmentation, it is the arbitrary number of instances that have made instance&#10;segmentation much more challenging. In order to predict a mask for each&#10;instance, mainstream approaches either follow the 'detect-thensegment' strategy&#10;as used by Mask R-CNN, or predict category masks first then use clustering&#10;techniques to group pixels into individual instances. We view the task of&#10;instance segmentation from a completely new perspective by introducing the&#10;notion of &quot;instance categories&quot;, which assigns categories to each pixel within&#10;an instance according to the instance's location and size, thus nicely&#10;converting instance mask segmentation into a classification-solvable problem.&#10;Now instance segmentation is decomposed into two classification tasks. We&#10;demonstrate a much simpler and flexible instance segmentation framework with&#10;strong performance, achieving on par accuracy with Mask R-CNN and outperforming&#10;recent singleshot instance segmenters in accuracy. We hope that this very&#10;simple and strong framework can serve as a baseline for many instance-level&#10;recognition tasks besides instance segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.05909" label="1906.05909">
-        <attvalues>
-          <attvalue for="0" value="Stand-Alone Self-Attention in Vision Models" />
-          <attvalue for="1" value="  Convolutions are a fundamental building block of modern computer vision&#10;systems. Recent approaches have argued for going beyond convolutions in order&#10;to capture long-range dependencies. These efforts focus on augmenting&#10;convolutional models with content-based interactions, such as self-attention&#10;and non-local means, to achieve gains on a number of vision tasks. The natural&#10;question that arises is whether attention can be a stand-alone primitive for&#10;vision models instead of serving as just an augmentation on top of&#10;convolutions. In developing and testing a pure self-attention vision model, we&#10;verify that self-attention can indeed be an effective stand-alone layer. A&#10;simple procedure of replacing all instances of spatial convolutions with a form&#10;of self-attention applied to ResNet model produces a fully self-attentional&#10;model that outperforms the baseline on ImageNet classification with 12% fewer&#10;FLOPS and 29% fewer parameters. On COCO object detection, a pure self-attention&#10;model matches the mAP of a baseline RetinaNet while having 39% fewer FLOPS and&#10;34% fewer parameters. Detailed ablation studies demonstrate that self-attention&#10;is especially impactful when used in later layers. These results establish that&#10;stand-alone self-attention is an important addition to the vision&#10;practitioner's toolbox.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.00241" label="1903.00241">
-        <attvalues>
-          <attvalue for="0" value="Mask Scoring R-CNN" />
-          <attvalue for="1" value="  Letting a deep network be aware of the quality of its own predictions is an&#10;interesting yet important problem. In the task of instance segmentation, the&#10;confidence of instance classification is used as mask quality score in most&#10;instance segmentation frameworks. However, the mask quality, quantified as the&#10;IoU between the instance mask and its ground truth, is usually not well&#10;correlated with classification score. In this paper, we study this problem and&#10;propose Mask Scoring R-CNN which contains a network block to learn the quality&#10;of the predicted instance masks. The proposed network block takes the instance&#10;feature and the corresponding predicted mask together to regress the mask IoU.&#10;The mask scoring strategy calibrates the misalignment between mask quality and&#10;mask score, and improves instance segmentation performance by prioritizing more&#10;accurate mask predictions during COCO AP evaluation. By extensive evaluations&#10;on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain&#10;with different models, and outperforms the state-of-the-art Mask R-CNN. We hope&#10;our simple and effective approach will provide a new direction for improving&#10;instance segmentation. The source code of our method is available at&#10;\url{https://github.com/zjhuang22/maskscoring_rcnn}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.06522" label="2303.06522">
-        <attvalues>
-          <attvalue for="0" value="Token Sparsification for Faster Medical Image Segmentation" />
-          <attvalue for="1" value="  Can we use sparse tokens for dense prediction, e.g., segmentation? Although&#10;token sparsification has been applied to Vision Transformers (ViT) to&#10;accelerate classification, it is still unknown how to perform segmentation from&#10;sparse tokens. To this end, we reformulate segmentation as a sparse encoding -&gt;&#10;token completion -&gt; dense decoding (SCD) pipeline. We first empirically show&#10;that naively applying existing approaches from classification token pruning and&#10;masked image modeling (MIM) leads to failure and inefficient training caused by&#10;inappropriate sampling algorithms and the low quality of the restored dense&#10;features. In this paper, we propose Soft-topK Token Pruning (STP) and&#10;Multi-layer Token Assembly (MTA) to address these problems. In sparse encoding,&#10;STP predicts token importance scores with a lightweight sub-network and samples&#10;the topK tokens. The intractable topK gradients are approximated through a&#10;continuous perturbed score distribution. In token completion, MTA restores a&#10;full token sequence by assembling both sparse output tokens and pruned&#10;multi-layer intermediate ones. The last dense decoding stage is compatible with&#10;existing segmentation decoders, e.g., UNETR. Experiments show SCD pipelines&#10;equipped with STP and MTA are much faster than baselines without token pruning&#10;in both training (up to 120% higher throughput and inference up to 60.6% higher&#10;throughput) while maintaining segmentation quality.&#10;" />
-          <attvalue for="2" value="&#10;Vision Transformers (ViT)~\cite{dosovitskiy2020image} for dense prediction~\cite{zheng2021rethinking,ranftl2021vision} have achieved impressive results in tasks including medical image segmentation~\cite{hatamizadeh2022unetr}.&#10;In general, high-resolution features~\cite{wang2020deep} preserving details are always desirable for precise segmentation. However, because of the quadratic computation complexity in self-attention~\cite{vaswani2017attention}, doubling the resolution per dimension in a 3D volume can lead to an 8$\times$ longer sequence and hence 64$\times$ more computation.&#10;This growing computing burden can quickly surpass limited computation budgets.&#10;Considering ViT's flexibility and great potential in masked image modeling~\cite{he2021masked,li2021benchmarking}, we explore acceleration algorithms based on the standard ViT.&#10;Recently, token sparsification ~\cite{rao2021dynamicvit,liang2021evit,meng2021adavit} has been proposed to accelerate inference in ViT for classification by dropping less important tokens. However, to the best of our knowledge, there are no ViT token sparsification approaches for segmentation.&#10;This leads us to ask the question: Can we use sparse tokens for dense prediction, e.g., segmentation?&#10;&#10;To answer the question, we reformulate segmentation as a \underline{s}parse encoding $arrow$ token \underline{c}ompletion $arrow$ \underline{d}ense decoding (SCD) pipeline. Unlike a standard \underline{d}ense encoding $arrow$ \underline{d}ense decoding (DD) pipeline, sparse encoding and token completion are required in SCD. Sparse encoding requires learning a sparse token representation for speed and token completion is needed to restore the full set of tokens for dense prediction.&#10;We first examine a na\&quot;ive realization of sparse encoding and token completion by applying existing approaches. Specifically, we adapt sampling methods in classification, e.g., EViT~\cite{liang2021evit} and DynamicViT~\cite{rao2021dynamicvit}, to sparse encoding, and masked image modeling (MIM)~\cite{he2021masked,bao2021beit} to token completion. However, we observe significantly inferior results in this SCD pipeline (See Table~\ref{tab:comb}). Next, we provide more insight into the problems of existing methods.&#10;\\&#10;Problems in Sparse Encoding. There are two steps in this step, i.e., token score estimation and token sampling. We show that EViT's token score estimation is inappropriate for segmentation and DynamicViT's token sampling leads to training inefficiency: i) EViT~\cite{liang2021evit} uses the attention weights between spatial tokens and the [CLS] token to estimate scores. While this is sound for classification since [CLS] is used for prediction, it is sub-optimal for segmentation because [CLS] is deprecated in the segmentation decoder. ii) DynamicViT~\cite{rao2021dynamicvit} estimates token scores with a sub-network. DynamicViT frames token sampling as a series of independent binary decisions to keep or drop tokens. This does not guarantee a fixed number of sampled tokens for each training input. To fit in batch training, DynamicViT keeps all tokens in memory and masks self-attention entries, leading to training inefficiency.&#10;\\&#10;Problems in Token Completion. Previous sparse token classification models~\cite{rao2021dynamicvit,liang2021evit} do not require token completion. Thus, we borrow the design from MIM. MIM reconstructs full tokens from a partial token sequence by padding it to full length with learnable mask tokens and then hallucinating the masked regions from their context. While MIM is useful for pre-training, it cannot accurately restore detailed information, resulting in inferior segmentation results.&#10;&#10;We propose Soft-topK Token Pruning (STP) and Multi-layer Token Assembly (MTA) to implement sparse encoding and token completion. i) In sparse encoding, STP predicts token importance scores with a sub-network, avoiding the limitation of [CLS] in segmentation. STP then samples topK-scored tokens instead of making binary decisions per token separately, accelerating training by retaining only the sampled tokens in memory and computing. Motivated by subset sampling~\cite{xie2019reparameterizable,li2021differentiable,cordonnier2021differentiable}, the intractable gradients of the topK operation are approximated through a perturbed continuous score distribution.&#10;ii) In token completion, the MTA restores a full token sequence by assembling both sparse output tokens and pruned intermediate tokens from multiple layers. &#10;Compared to MIM that fills the pruned positions with identical mask tokens, MTA produces more informative, position-specific representations. For dense decoding, the SCD pipeline is compatible with existing segmentation decoders, such as UNETR.&#10;&#10;We evaluate our method on two relatively sparse 3D medical image segmentation datasets, the CT Abdomen Multi-organ Segmentation (BTCV~\cite{landman2015miccai}, N=30) dataset and the MRI Brain Tumor Segmentation (MSD BraTS~\cite{antonelli2021medical}, N=484) dataset. On both tasks, { STP+MTA+UNETR} matches the UNETR baseline while providing significant computing savings with large token pruning ratios. On BraTS, { STP+MTA+UNETR} accelerates segmentation inference/training throughput by 60.6\%/120\% and achieves the same segmentation accuracy. On BTCV, { STP+MTA+UNETR} increases inference/training throughput by 24.1\%/97.36\% while maintaining performance. In summary, our contributions are: &#10;\begin{itemize}&#10; \item To the best of our knowledge, we are the first to use token pruning/dropping for ViT-based medical image segmentation.&#10; \item Based on subset sampling, our proposed Soft-topK Token Pruning (STP) module can be flexibly incorporated into a standard ViT to prune tokens with greater efficiency while maintaining accuracy.&#10; \item We propose Multi-layer Token Assembly (MTA) to recover a full set of tokens, i.e., a dense representation, from a sparse set. MTA preserves high-detail information for accurate segmentation.&#10; \item We show that { STP+MTA+UNETR} maintains performance compared with UNETR with much less computation on two 3D medical image datasets.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2012.15840" label="2012.15840">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective&#10;  with Transformers" />
-          <attvalue for="1" value="  Most recent semantic segmentation methods adopt a fully-convolutional network&#10;(FCN) with an encoder-decoder architecture. The encoder progressively reduces&#10;the spatial resolution and learns more abstract/semantic visual concepts with&#10;larger receptive fields. Since context modeling is critical for segmentation,&#10;the latest efforts have been focused on increasing the receptive field, through&#10;either dilated/atrous convolutions or inserting attention modules. However, the&#10;encoder-decoder based FCN architecture remains unchanged. In this paper, we aim&#10;to provide an alternative perspective by treating semantic segmentation as a&#10;sequence-to-sequence prediction task. Specifically, we deploy a pure&#10;transformer (ie, without convolution and resolution reduction) to encode an&#10;image as a sequence of patches. With the global context modeled in every layer&#10;of the transformer, this encoder can be combined with a simple decoder to&#10;provide a powerful segmentation model, termed SEgmentation TRansformer (SETR).&#10;Extensive experiments show that SETR achieves new state of the art on ADE20K&#10;(50.28% mIoU), Pascal Context (55.83% mIoU) and competitive results on&#10;Cityscapes. Particularly, we achieve the first position in the highly&#10;competitive ADE20K test server leaderboard on the day of submission.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.13413" label="2103.13413">
-        <attvalues>
-          <attvalue for="0" value="Vision Transformers for Dense Prediction" />
-          <attvalue for="1" value="  We introduce dense vision transformers, an architecture that leverages vision&#10;transformers in place of convolutional networks as a backbone for dense&#10;prediction tasks. We assemble tokens from various stages of the vision&#10;transformer into image-like representations at various resolutions and&#10;progressively combine them into full-resolution predictions using a&#10;convolutional decoder. The transformer backbone processes representations at a&#10;constant and relatively high resolution and has a global receptive field at&#10;every stage. These properties allow the dense vision transformer to provide&#10;finer-grained and more globally coherent predictions when compared to&#10;fully-convolutional networks. Our experiments show that this architecture&#10;yields substantial improvements on dense prediction tasks, especially when a&#10;large amount of training data is available. For monocular depth estimation, we&#10;observe an improvement of up to 28% in relative performance when compared to a&#10;state-of-the-art fully-convolutional network. When applied to semantic&#10;segmentation, dense vision transformers set a new state of the art on ADE20K&#10;with 49.02% mIoU. We further show that the architecture can be fine-tuned on&#10;smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets&#10;the new state of the art. Our models are available at&#10;https://github.com/intel-isl/DPT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.07919" label="1908.07919">
-        <attvalues>
-          <attvalue for="0" value="Deep High-Resolution Representation Learning for Visual Recognition" />
-          <attvalue for="1" value="  High-resolution representations are essential for position-sensitive vision&#10;problems, such as human pose estimation, semantic segmentation, and object&#10;detection. Existing state-of-the-art frameworks first encode the input image as&#10;a low-resolution representation through a subnetwork that is formed by&#10;connecting high-to-low resolution convolutions \emph{in series} (e.g., ResNet,&#10;VGGNet), and then recover the high-resolution representation from the encoded&#10;low-resolution representation. Instead, our proposed network, named as&#10;High-Resolution Network (HRNet), maintains high-resolution representations&#10;through the whole process. There are two key characteristics: (i) Connect the&#10;high-to-low resolution convolution streams \emph{in parallel}; (ii) Repeatedly&#10;exchange the information across resolutions. The benefit is that the resulting&#10;representation is semantically richer and spatially more precise. We show the&#10;superiority of the proposed HRNet in a wide range of applications, including&#10;human pose estimation, semantic segmentation, and object detection, suggesting&#10;that the HRNet is a stronger backbone for computer vision problems. All the&#10;codes are available at~{\url{https://github.com/HRNet}}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.11429" label="2111.11429">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking Detection Transfer Learning with Vision Transformers" />
-          <attvalue for="1" value="  Object detection is a central downstream task used to test if pre-trained&#10;network parameters confer benefits, such as improved accuracy or training&#10;speed. The complexity of object detection methods can make this benchmarking&#10;non-trivial when new architectures, such as Vision Transformer (ViT) models,&#10;arrive. These difficulties (e.g., architectural incompatibility, slow training,&#10;high memory consumption, unknown training formulae, etc.) have prevented recent&#10;studies from benchmarking detection transfer learning with standard ViT models.&#10;In this paper, we present training techniques that overcome these challenges,&#10;enabling the use of standard ViT models as the backbone of Mask R-CNN. These&#10;tools facilitate the primary goal of our study: we compare five ViT&#10;initializations, including recent state-of-the-art self-supervised learning&#10;methods, supervised initialization, and a strong random initialization&#10;baseline. Our results show that recent masking-based unsupervised learning&#10;methods may, for the first time, provide convincing transfer learning&#10;improvements on COCO, increasing box AP up to 4% (absolute) over supervised and&#10;prior self-supervised pre-training methods. Moreover, these masking-based&#10;initializations scale better, with the improvement growing as model size&#10;increases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.02034" label="2106.02034">
-        <attvalues>
-          <attvalue for="0" value="DynamicViT: Efficient Vision Transformers with Dynamic Token&#10;  Sparsification" />
-          <attvalue for="1" value="  Attention is sparse in vision transformers. We observe the final prediction&#10;in vision transformers is only based on a subset of most informative tokens,&#10;which is sufficient for accurate image recognition. Based on this observation,&#10;we propose a dynamic token sparsification framework to prune redundant tokens&#10;progressively and dynamically based on the input. Specifically, we devise a&#10;lightweight prediction module to estimate the importance score of each token&#10;given the current features. The module is added to different layers to prune&#10;redundant tokens hierarchically. To optimize the prediction module in an&#10;end-to-end manner, we propose an attention masking strategy to differentiably&#10;prune a token by blocking its interactions with other tokens. Benefiting from&#10;the nature of self-attention, the unstructured sparse tokens are still hardware&#10;friendly, which makes our framework easy to achieve actual speed-up. By&#10;hierarchically pruning 66% of the input tokens, our method greatly reduces&#10;31%~37% FLOPs and improves the throughput by over 40% while the drop of&#10;accuracy is within 0.5% for various vision transformers. Equipped with the&#10;dynamic token sparsification framework, DynamicViT models can achieve very&#10;competitive complexity/accuracy trade-offs compared to state-of-the-art CNNs&#10;and vision transformers on ImageNet. Code is available at&#10;https://github.com/raoyongming/DynamicViT&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.10517" label="1901.10517">
-        <attvalues>
-          <attvalue for="0" value="Reparameterizable Subset Sampling via Continuous Relaxations" />
-          <attvalue for="1" value="  Many machine learning tasks require sampling a subset of items from a&#10;collection based on a parameterized distribution. The Gumbel-softmax trick can&#10;be used to sample a single item, and allows for low-variance reparameterized&#10;gradients with respect to the parameters of the underlying distribution.&#10;However, stochastic optimization involving subset sampling is typically not&#10;reparameterizable. To overcome this limitation, we define a continuous&#10;relaxation of subset sampling that provides reparameterization gradients by&#10;generalizing the Gumbel-max trick. We use this approach to sample subsets of&#10;features in an instance-wise feature selection task for model interpretability,&#10;subsets of neighbors to implement a deep stochastic k-nearest neighbors model,&#10;and sub-sequences of neighbors to implement parametric t-SNE by directly&#10;comparing the identities of local neighbors. We improve performance in all&#10;these tasks by incorporating subset sampling in end-to-end training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.04657" label="2108.04657">
-        <attvalues>
-          <attvalue for="0" value="Differentiable Subset Pruning of Transformer Heads" />
-          <attvalue for="1" value="  Multi-head attention, a collection of several attention mechanisms that&#10;independently attend to different parts of the input, is the key ingredient in&#10;the Transformer. Recent work has shown, however, that a large proportion of the&#10;heads in a Transformer's multi-head attention mechanism can be safely pruned&#10;away without significantly harming the performance of the model; such pruning&#10;leads to models that are noticeably smaller and faster in practice. Our work&#10;introduces a new head pruning technique that we term differentiable subset&#10;pruning. Intuitively, our method learns per-head importance variables and then&#10;enforces a user-specified hard constraint on the number of unpruned heads. The&#10;importance variables are learned via stochastic gradient descent. We conduct&#10;experiments on natural language inference and machine translation; we show that&#10;differentiable subset pruning performs comparably or better than previous works&#10;while offering precise control of the sparsity level.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.03059" label="2104.03059">
-        <attvalues>
-          <attvalue for="0" value="Differentiable Patch Selection for Image Recognition" />
-          <attvalue for="1" value="  Neural Networks require large amounts of memory and compute to process high&#10;resolution images, even when only a small part of the image is actually&#10;informative for the task at hand. We propose a method based on a differentiable&#10;Top-K operator to select the most relevant parts of the input to efficiently&#10;process high resolution images. Our method may be interfaced with any&#10;downstream neural network, is able to aggregate information from different&#10;patches in a flexible way, and allows the whole model to be trained end-to-end&#10;using backpropagation. We show results for traffic sign recognition,&#10;inter-patch relationship reasoning, and fine-grained recognition without using&#10;object/part bounding box annotations during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.05735" label="2106.05735">
-        <attvalues>
-          <attvalue for="0" value="The Medical Segmentation Decathlon" />
-          <attvalue for="1" value="  International challenges have become the de facto standard for comparative&#10;assessment of image analysis algorithms given a specific task. Segmentation is&#10;so far the most widely investigated medical image processing task, but the&#10;various segmentation challenges have typically been organized in isolation,&#10;such that algorithm development was driven by the need to tackle a single&#10;specific clinical problem. We hypothesized that a method capable of performing&#10;well on multiple tasks will generalize well to a previously unseen task and&#10;potentially outperform a custom-designed solution. To investigate the&#10;hypothesis, we organized the Medical Segmentation Decathlon (MSD) - a&#10;biomedical image analysis challenge, in which algorithms compete in a multitude&#10;of both tasks and modalities. The underlying data set was designed to explore&#10;the axis of difficulties typically encountered when dealing with medical&#10;images, such as small data sets, unbalanced labels, multi-site data and small&#10;objects. The MSD challenge confirmed that algorithms with a consistent good&#10;performance on a set of tasks preserved their good average performance on a&#10;different set of previously unseen tasks. Moreover, by monitoring the MSD&#10;winner for two years, we found that this algorithm continued generalizing well&#10;to a wide range of other clinical problems, further confirming our hypothesis.&#10;Three main conclusions can be drawn from this study: (1) state-of-the-art image&#10;segmentation algorithms are mature, accurate, and generalize well when&#10;retrained on unseen tasks; (2) consistent algorithmic performance across&#10;multiple tasks is a strong surrogate of algorithmic generalizability; (3) the&#10;training of accurate AI segmentation models is now commoditized to non AI&#10;experts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.04376" label="2303.04376">
-        <attvalues>
-          <attvalue for="0" value="Tsanet: Temporal and Scale Alignment for Unsupervised Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  Unsupervised Video Object Segmentation (UVOS) refers to the challenging task&#10;of segmenting the prominent object in videos without manual guidance. In recent&#10;works, two approaches for UVOS have been discussed that can be divided into:&#10;appearance and appearance-motion-based methods, which have limitations&#10;respectively. Appearance-based methods do not consider the motion of the target&#10;object due to exploiting the correlation information between randomly paired&#10;frames. Appearance-motion-based methods have the limitation that the dependency&#10;on optical flow is dominant due to fusing the appearance with motion. In this&#10;paper, we propose a novel framework for UVOS that can address the&#10;aforementioned limitations of the two approaches in terms of both time and&#10;scale. Temporal Alignment Fusion aligns the saliency information of adjacent&#10;frames with the target frame to leverage the information of adjacent frames.&#10;Scale Alignment Decoder predicts the target object mask by aggregating&#10;multi-scale feature maps via continuous mapping with implicit neural&#10;representation. We present experimental results on public benchmark datasets,&#10;DAVIS 2016 and FBMS, which demonstrate the effectiveness of our method.&#10;Furthermore, we outperform the state-of-the-art methods on DAVIS 2016.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Video Object Segmentation (VOS) is a vital task in the field of computer vision. VOS aims to segment accurate regions of the target object by pixel unit. It has been widely applied to various fields such as surveillance, video editing, medicine, and autonomous driving. More specifically, Unsupervised VOS (UVOS) segments the most salient object without any prior knowledge at the test time. This is a challenging task in that the network should recognize a target object, which is either the most prominent moving object or has the most consistent appearance in a sequence without using a manual guidance.&#10;&#10;In recent years, several studies based on deep learning have been progressed into two streams for addressing UVOS: appearance-base methods and appearance-motion based methods. Appearance-based methods~\cite{cosnet,cfam,f2net}, use only appearance features extracted from multiple frames. These methods proposed the module that captures the saliency information based on attention mechanism between target frame and other frames. These approaches can fully use contextual information including semantic details of inter-frames, regardless of frame order. However, these approaches focus on the correlation between saliency features from different frames without considering the motion of the target object. Furthermore, these methods tend to vary in performance depending on which frame is used as a reference at test time. Appearance-motion based methods~\cite{matnet,rtnet,dtt,amc,tmo} use the target frame as well as corresponding optical flow (called &quot;motion cue&quot;), which is calculated using optical flow networks~\cite{flownet,raft}. Motion cue enables the model to capture the prominent region of the target object. By combining appearance features with motion features, it is possible to acquire distinct region of salient object whose motion is prominent. However, these methods have an intrinsic drawback that the dependency on optical flow is high. If the target object shows significant motion blur or occlusion, the optical flow quality is inferior, which makes it difficult to determine the target object accurately. This limitation has been addressed with several methods~\cite{amc,rtnet}. \cite{amc} exploits multi modality fusion module that, adaptively, fuses appearance with motion features by scoring each feature. \cite{rtnet} utilizes bidirectional optical flow to fully take advantage of the motion cues between two frames. However, one problem still remain of these methods: model cannot use the contextual information of consecutive RGB frames considering the characteristics of the video sequence.&#10;&#10;To tackle the aforementioned problems, we propose a novel framework, Temporal and Scale Alignment Network (TSANet), which can utilize both contextual and motion information from adjacent frames. Our framework consists of two modules, Temporal Alignment Fusion~(TAF) and Scale Alignment Decoder~(SAD). TAF aligns the information of adjacent frames to the target frame to propagate contextual information in multi-layer features extracted from each residual block of ResNet~\cite{resnet}. Then, SAD aggregates the features with different sizes which are aligned from TAF. In particular, we introduce scale alignment method, which is based on implicit neural representation. We generate a continuous feature map by embedding the relative coordinates and feature values to interpret the correlation information between relative coordinates when upscaling the aligned features. We demonstrate the effectiveness of our method on benchmark datasets, DAVIS 2016 and FBMS. Our proposed framework show the state-of-the-art performance in DAVIS 2016 dataset and compromising results in FBMS dataset.\\Our contributions are as follows:&#10;&#10;\begin{itemize}&#10;\item We propose Temporal Alignment Fusion (TAF) that aligns features of adjacent frames to target features for leveraging the contextual information of adjacent frames to the target frame.&#10;\item To aggregate features which have different scales, we propose Scale Alignment Decoder (SAD), introducing implicit neural representation using relative coordinate information that occurs in alignment process.&#10;\item Our framework (TSANet) shows the state-of-the performance on DAVIS 2016 with 87.4\% $\mathcal{J}\&amp;\mathcal{F}\;$Mean score.&#10;\end{itemize}&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Deep Learning Frameworks, Mathematics, Artificial Intelligence, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2001.06810" label="2001.06810">
-        <attvalues>
-          <attvalue for="0" value="See More, Know More: Unsupervised Video Object Segmentation with&#10;  Co-Attention Siamese Networks" />
-          <attvalue for="1" value="  We introduce a novel network, called CO-attention Siamese Network (COSNet),&#10;to address the unsupervised video object segmentation task from a holistic&#10;view. We emphasize the importance of inherent correlation among video frames&#10;and incorporate a global co-attention mechanism to improve further the&#10;state-of-the-art deep learning based solutions that primarily focus on learning&#10;discriminative foreground representations over appearance and motion in&#10;short-term temporal segments. The co-attention layers in our network provide&#10;efficient and competent stages for capturing global correlations and scene&#10;context by jointly computing and appending co-attention responses into a joint&#10;feature space. We train COSNet with pairs of video frames, which naturally&#10;augments training data and allows increased learning capacity. During the&#10;segmentation stage, the co-attention model encodes useful information by&#10;processing multiple reference frames together, which is leveraged to infer the&#10;frequently reappearing and salient foreground objects better. We propose a&#10;unified and end-to-end trainable framework where different co-attention&#10;variants can be derived for mining the rich context within videos. Our&#10;extensive experiments over three large benchmarks manifest that COSNet&#10;outperforms the current alternatives by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.02368" label="2111.02368">
-        <attvalues>
-          <attvalue for="0" value="Video Salient Object Detection via Contrastive Features and Attention&#10;  Modules" />
-          <attvalue for="1" value="  Video salient object detection aims to find the most visually distinctive&#10;objects in a video. To explore the temporal dependencies, existing methods&#10;usually resort to recurrent neural networks or optical flow. However, these&#10;approaches require high computational cost, and tend to accumulate inaccuracies&#10;over time. In this paper, we propose a network with attention modules to learn&#10;contrastive features for video salient object detection without the high&#10;computational temporal modeling techniques. We develop a non-local&#10;self-attention scheme to capture the global information in the video frame. A&#10;co-attention formulation is utilized to combine the low-level and high-level&#10;features. We further apply the contrastive learning to improve the feature&#10;representations, where foreground region pairs from the same video are pulled&#10;together, and foreground-background region pairs are pushed away in the latent&#10;space. The intra-frame contrastive loss helps separate the foreground and&#10;background features, and the inter-frame contrastive loss improves the temporal&#10;consistency. We conduct extensive experiments on several benchmark datasets for&#10;video salient object detection and unsupervised video object segmentation, and&#10;show that the proposed method requires less computation, and performs favorably&#10;against the state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.02534" label="2012.02534">
-        <attvalues>
-          <attvalue for="0" value="F2Net: Learning to Focus on the Foreground for Unsupervised Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  Although deep learning based methods have achieved great progress in&#10;unsupervised video object segmentation, difficult scenarios (e.g., visual&#10;similarity, occlusions, and appearance changing) are still not well-handled. To&#10;alleviate these issues, we propose a novel Focus on Foreground Network (F2Net),&#10;which delves into the intra-inter frame details for the foreground objects and&#10;thus effectively improve the segmentation performance. Specifically, our&#10;proposed network consists of three main parts: Siamese Encoder Module, Center&#10;Guiding Appearance Diffusion Module, and Dynamic Information Fusion Module.&#10;Firstly, we take a siamese encoder to extract the feature representations of&#10;paired frames (reference frame and current frame). Then, a Center Guiding&#10;Appearance Diffusion Module is designed to capture the inter-frame feature&#10;(dense correspondences between reference frame and current frame), intra-frame&#10;feature (dense correspondences in current frame), and original semantic feature&#10;of current frame. Specifically, we establish a Center Prediction Branch to&#10;predict the center location of the foreground object in current frame and&#10;leverage the center point information as spatial guidance prior to enhance the&#10;inter-frame and intra-frame feature extraction, and thus the feature&#10;representation considerably focus on the foreground objects. Finally, we&#10;propose a Dynamic Information Fusion Module to automatically select relatively&#10;important features through three aforementioned different level features.&#10;Extensive experiments on DAVIS2016, Youtube-object, and FBMS datasets show that&#10;our proposed F2Net achieves the state-of-the-art performance with significant&#10;improvement.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.03138" label="2209.03138">
-        <attvalues>
-          <attvalue for="0" value="Treating Motion as Option to Reduce Motion Dependency in Unsupervised&#10;  Video Object Segmentation" />
-          <attvalue for="1" value="  Unsupervised video object segmentation (VOS) aims to detect the most salient&#10;object in a video sequence at the pixel level. In unsupervised VOS, most&#10;state-of-the-art methods leverage motion cues obtained from optical flow maps&#10;in addition to appearance cues to exploit the property that salient objects&#10;usually have distinctive movements compared to the background. However, as they&#10;are overly dependent on motion cues, which may be unreliable in some cases,&#10;they cannot achieve stable prediction. To reduce this motion dependency of&#10;existing two-stream VOS methods, we propose a novel motion-as-option network&#10;that optionally utilizes motion cues. Additionally, to fully exploit the&#10;property of the proposed network that motion is not always required, we&#10;introduce a collaborative network learning strategy. On all the public&#10;benchmark datasets, our proposed network affords state-of-the-art performance&#10;with real-time inference speed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.01925" label="1612.01925">
-        <attvalues>
-          <attvalue for="0" value="FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks" />
-          <attvalue for="1" value="  The FlowNet demonstrated that optical flow estimation can be cast as a&#10;learning problem. However, the state of the art with regard to the quality of&#10;the flow has still been defined by traditional methods. Particularly on small&#10;displacements and real-world data, FlowNet cannot compete with variational&#10;methods. In this paper, we advance the concept of end-to-end learning of&#10;optical flow and make it work really well. The large improvements in quality&#10;and speed are caused by three major contributions: first, we focus on the&#10;training data and show that the schedule of presenting data during training is&#10;very important. Second, we develop a stacked architecture that includes warping&#10;of the second image with intermediate optical flow. Third, we elaborate on&#10;small displacements by introducing a sub-network specializing on small motions.&#10;FlowNet 2.0 is only marginally slower than the original FlowNet but decreases&#10;the estimation error by more than 50%. It performs on par with state-of-the-art&#10;methods, while running at interactive frame rates. Moreover, we present faster&#10;variants that allow optical flow computation at up to 140fps with accuracy&#10;matching the original FlowNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2311.18299" label="2311.18299">
-        <attvalues>
-          <attvalue for="0" value="Reconstructing the normal and shape at specularities in endoscopy" />
-          <attvalue for="1" value="  Specularities are numerous in endoscopic images. They occur as many white&#10;small elliptic spots, which are generally ruled out as nuisance in image&#10;analysis and computer vision methods. Instead, we propose to use specularities&#10;as cues for 3D perception. Specifically, we propose a new method to&#10;reconstruct, at each specularity, the observed tissue's normal direction (i.e.,&#10;its orientation) and shape (i.e., its curvature) from a single image. We show&#10;results on simulated and real interventional images.&#10;" />
-          <attvalue for="2" value="&#10;Specularities on smooth reflecting surfaces depend on the surface's local orientation and curvature, on the light source and on the camera position~\cite[Chapter 2]{forsyth2002computer}. In the general setting, all these contain far too many unknowns to allow one to exploit specularities effectively in 3D reconstruction. &#10;However, endoscopy is a particular case where the {light source and camera form a collocated rig}. &#10;Specularities are numerous in endoscopy, where the moist tissue causes specular reflection.&#10;However, they are generally considered as artifacts in endoscopic 3D reconstruction methods, both in classical methods such as Shape-from-Shading (SfS)~\cite{ciuti2012intra} and learning-based methods such as direct depth estimation~\cite{liu2023self}. &#10;More recently, it has been understood that specularities could form useful cues for 3D reconstruction.&#10;The study of~\cite{daher2023cyclesttn} has shown that training a learning-based reconstruction model from synthetic data which can eventually deal with real data requires one to generate specularities.&#10;Concretely, we have recently shown that the surface normal could be reconstructed at each specularity~\cite{makki2023normal, makki2023elliptical}, with a two-fold ambiguity.&#10;The reconstruction method uses the {specular isophote}, which is the curve enclosing the specularity in the image.&#10;Based on the assumption that the surface underlying the specularity is flat, we have used a mathematical model to show that the isophote is elliptic.&#10;This approach has advantages: the method runs fast and can process full high image resolution, in contrast to the learning-based approach which is currently limited to lower image resolution to keep runtime reasonable (the available datasets such as the EndoSLAM~\cite{ozyoruk2021endoslam} and UCL~\cite{rau2019implicit} ones are at lower resolution).&#10;However, it also has strong limitations.&#10;First, the reconstruction is up to a two-fold ambiguity: only one of the two directions found at each point is the correct normal.&#10;Second, the local planarity assumption is unreasonable, as specularities typically appear near highly curved surface patches.&#10;{Is an ambiguous surface normal all one can reconstruct from a specularity?}&#10;&#10;We propose to move a step ahead by using a more advanced local surface model.&#10;Specifically, we leave the restrictive flat surface to adopt a curved surface model.&#10;We show that with this model, the specular isophote is still elliptical in the image, albeit that its eccentricity is not just due to surface orientation as in past work, but to both surface orientation and curvature.&#10;We show that one can reconstruct the surface normal from just the {specularity's Brightest Point} (BP) and the surface curvature from the elliptical isophote.&#10;Specifically, we show that the principal curvature directions (the directions of minimum and maximum curvatures on the tangent plane) and the curvature ratio can be reconstructed.&#10;Therefore, {the specular isophote contains much more information} than one could think from past work.&#10;We propose an automatic reconstruction method, which, from an endoscopic image and the camera's intrinsic parameters, detects the specularities and reconstructs the local surface orientation and curvature ratio at each specularity.&#10;This reconstruction can be used to implement downstream tasks such as computer-aided navigation, to bootstrap other reconstruction methods such as SfS and to train depth estimation networks.&#10;We evaluate the method on realistic simulated colonoscopic data from Blender and surgical laparoscopy images.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Biology, Biomedical Engineering, Physics, Image Analysis, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2211.05642" label="2211.05642">
-        <attvalues>
-          <attvalue for="0" value="Normal reconstruction from specularity in the endoscopic setting" />
-          <attvalue for="1" value="  We show that for a plane imaged by an endoscope the specular isophotes are&#10;concentric circles on the scene plane, which appear as nested ellipses in the&#10;image. We show that these ellipses can be detected and used to estimate the&#10;plane's normal direction, forming a normal reconstruction method, which we&#10;validate on simulated data. In practice, the anatomical surfaces visible in&#10;endoscopic images are locally planar. We use our method to show that the&#10;surface normal can thus be reconstructed for each of the numerous specularities&#10;typically visible on moist tissues. We show results on laparoscopic and&#10;colonoscopic images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.01355" label="2209.01355">
-        <attvalues>
-          <attvalue for="0" value="Semantic Segmentation in Learned Compressed Domain" />
-          <attvalue for="1" value="  Most machine vision tasks (e.g., semantic segmentation) are based on images&#10;encoded and decoded by image compression algorithms (e.g., JPEG). However,&#10;these decoded images in the pixel domain introduce distortion, and they are&#10;optimized for human perception, making the performance of machine vision tasks&#10;suboptimal. In this paper, we propose a method based on the compressed domain&#10;to improve segmentation tasks. i) A dynamic and a static channel selection&#10;method are proposed to reduce the redundancy of compressed representations that&#10;are obtained by encoding. ii) Two different transform modules are explored and&#10;analyzed to help the compressed representation be transformed as the features&#10;in the segmentation network. The experimental results show that we can save up&#10;to 15.8\% bitrates compared with a state-of-the-art compressed domain-based&#10;work while saving up to about 83.6\% bitrates and 44.8\% inference time&#10;compared with the pixel domain-based method.&#10;" />
-          <attvalue for="2" value="&#10;With the development of deep learning, machine vision tasks have been widely implemented in the past few years and have played important roles in our lives. &#10;For example, semantic segmentation is applied to medical image analysis, autonomous vehicles, video surveillance, augmented reality, and many other areas \cite{minaee2021image}. However, many machine vision tasks are usually based on decoded images that need image compression.The distortion of decoded images deteriorate&#10; the performance of machine vision tasks.&#10;\par&#10;&#10;Classical image compression algorithms, such as, JPEG \cite{wallace1992jpeg}, and HEVC/H.265-intra \cite{sullivan2012overview} have achieved good Rate-distortion performance in human perception while some learned end-to-end image compression methods \cite{balle2018variational, cheng2020learned} also achieved comparable performance. However, all of them are designed to reduce the distortion which is perceived by human and has a gap with machine perception. Therefore, current algorithms usually lead to suboptimal semantic segmentation. &#10;&#10;\par&#10;Some methods based on the compressed domain are proposed to solve these problems. Pixel domain-based methods mean we need to input the images encoded and decoded by an image compression algorithm to the semantic segmentation network, as Fig. \ref{fig:20} shows. Different from that, compressed domain-based methods mean we directly input the representations in the compressed domain into the semantic segmentation network without the computation cost for the decoded image. A related standard named JPEG AI \cite{JPEGAI} had been proposed. Torfason $et\ al.$ \cite{torfason2018towards} had proved that the compressed domain-based method could achieve a better performance than the pixel domain-based method, and made some experiments on image classification and object detection. Choi $et\ al.$ \cite{choi2021scalable} further verified this by using an information theory method and proposed a compression domain-based approach which is verified on object detection and semantic segmentation. Bai $et\ al.$ \cite{Bai2022AAAI} utilized a transformer to improve image classification based on compressed domain. &#10;Wang $et\ al.$ \cite{wang2022learning} proposed a channel selection method and an entropy estimation method to improve the results in \cite{torfason2018towards}. However, the channels selection method can only delete the channels with the lowest entropy since the selection method is based on the variance of the compressed representations.&#10;&#10;\par&#10;In this paper, we propose a method based on the compressed domain to improve segmentation tasks.&#10;The main contributions of this paper are as follows:&#10;\begin{itemize}&#10;&#10;\item Based on a learned gate module, a dynamic and a static channel selection method are proposed to select suitable channels and reduce the redundancy of compressed representations.&#10;&#10;\item We explore and analyze two different modules transforming the representation in the compressed domain to the feature that comes from the mid-layer in the segmentation network.&#10;&#10;\item We perform experiments based on Cityscapes, and the results show that, in the same mIoU cases, we can save up to 15.8\% bitrates compared with a state-of-the-art compressed domain-based work while saving up to about 83.6\% bitrates and 44.8\% inference time compared with the pixel domain-based method.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Machine Vision Tasks, Compressed Domain Processing, Mathematics, Image Processing, Image Compression Algorithms, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2001.05566" label="2001.05566">
-        <attvalues>
-          <attvalue for="0" value="Image Segmentation Using Deep Learning: A Survey" />
-          <attvalue for="1" value="  Image segmentation is a key topic in image processing and computer vision&#10;with applications such as scene understanding, medical image analysis, robotic&#10;perception, video surveillance, augmented reality, and image compression, among&#10;many others. Various algorithms for image segmentation have been developed in&#10;the literature. Recently, due to the success of deep learning models in a wide&#10;range of vision applications, there has been a substantial amount of works&#10;aimed at developing image segmentation approaches using deep learning models.&#10;In this survey, we provide a comprehensive review of the literature at the time&#10;of this writing, covering a broad spectrum of pioneering works for semantic and&#10;instance-level segmentation, including fully convolutional pixel-labeling&#10;networks, encoder-decoder architectures, multi-scale and pyramid based&#10;approaches, recurrent networks, visual attention models, and generative models&#10;in adversarial settings. We investigate the similarity, strengths and&#10;challenges of these deep learning models, examine the most widely used&#10;datasets, report performances, and discuss promising future research directions&#10;in this area.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.06131" label="1803.06131">
-        <attvalues>
-          <attvalue for="0" value="Towards Image Understanding from Deep Compression without Decoding" />
-          <attvalue for="1" value="  Motivated by recent work on deep neural network (DNN)-based image compression&#10;methods showing potential improvements in image quality, savings in storage,&#10;and bandwidth reduction, we propose to perform image understanding tasks such&#10;as classification and segmentation directly on the compressed representations&#10;produced by these compression methods. Since the encoders and decoders in&#10;DNN-based compression methods are neural networks with feature-maps as internal&#10;representations of the images, we directly integrate these with architectures&#10;for image understanding. This bypasses decoding of the compressed&#10;representation into RGB space and reduces computational cost. Our study shows&#10;that accuracies comparable to networks that operate on compressed RGB images&#10;can be achieved while reducing the computational complexity up to $2\times$.&#10;Furthermore, we show that synergies are obtained by jointly training&#10;compression networks with classification networks on the compressed&#10;representations, improving image quality, classification accuracy, and&#10;segmentation performance. We find that inference from compressed&#10;representations is particularly advantageous compared to inference from&#10;compressed RGB images for aggressive compression rates.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.08373" label="2107.08373">
-        <attvalues>
-          <attvalue for="0" value="Scalable Image Coding for Humans and Machines" />
-          <attvalue for="1" value="  At present, and increasingly so in the future, much of the captured visual&#10;content will not be seen by humans. Instead, it will be used for automated&#10;machine vision analytics and may require occasional human viewing. Examples of&#10;such applications include traffic monitoring, visual surveillance, autonomous&#10;navigation, and industrial machine vision. To address such requirements, we&#10;develop an end-to-end learned image codec whose latent space is designed to&#10;support scalability from simpler to more complicated tasks. The simplest task&#10;is assigned to a subset of the latent space (the base layer), while more&#10;complicated tasks make use of additional subsets of the latent space, i.e.,&#10;both the base and enhancement layer(s). For the experiments, we establish a&#10;2-layer and a 3-layer model, each of which offers input reconstruction for&#10;human vision, plus machine vision task(s), and compare them with relevant&#10;benchmarks. The experiments show that our scalable codecs offer 37%-80% bitrate&#10;savings on machine vision tasks compared to best alternatives, while being&#10;comparable to state-of-the-art image codecs in terms of input reconstruction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.09300" label="2112.09300">
-        <attvalues>
-          <attvalue for="0" value="Towards End-to-End Image Compression and Analysis with Transformers" />
-          <attvalue for="1" value="  We propose an end-to-end image compression and analysis model with&#10;Transformers, targeting to the cloud-based image classification application.&#10;Instead of placing an existing Transformer-based image classification model&#10;directly after an image codec, we aim to redesign the Vision Transformer (ViT)&#10;model to perform image classification from the compressed features and&#10;facilitate image compression with the long-term information from the&#10;Transformer. Specifically, we first replace the patchify stem (i.e., image&#10;splitting and embedding) of the ViT model with a lightweight image encoder&#10;modelled by a convolutional neural network. The compressed features generated&#10;by the image encoder are injected convolutional inductive bias and are fed to&#10;the Transformer for image classification bypassing image reconstruction.&#10;Meanwhile, we propose a feature aggregation module to fuse the compressed&#10;features with the selected intermediate features of the Transformer, and feed&#10;the aggregated features to a deconvolutional neural network for image&#10;reconstruction. The aggregated features can obtain the long-term information&#10;from the self-attention mechanism of the Transformer and improve the&#10;compression performance. The rate-distortion-accuracy optimization problem is&#10;finally solved by a two-step training strategy. Experimental results&#10;demonstrate the effectiveness of the proposed model in both the image&#10;compression and the classification tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.01197" label="2205.01197">
-        <attvalues>
-          <attvalue for="0" value="Boosting Video Object Segmentation based on Scale Inconsistency" />
-          <attvalue for="1" value="  We present a refinement framework to boost the performance of pre-trained&#10;semi-supervised video object segmentation (VOS) models. Our work is based on&#10;scale inconsistency, which is motivated by the observation that existing VOS&#10;models generate inconsistent predictions from input frames with different&#10;sizes. We use the scale inconsistency as a clue to devise a pixel-level&#10;attention module that aggregates the advantages of the predictions from&#10;different-size inputs. The scale inconsistency is also used to regularize the&#10;training based on a pixel-level variance measured by an uncertainty estimation.&#10;We further present a self-supervised online adaptation, tailored for test-time&#10;optimization, that bootstraps the predictions without ground-truth masks based&#10;on the scale inconsistency. Experiments on DAVIS 16 and DAVIS 17 datasets show&#10;that our framework can be generically applied to various VOS models and improve&#10;their performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Video object segmentation (VOS) aims to divide target objects from other instances in a video sequence. In this work, we focus on a semi-supervised setting, where the ground-truth mask of the first frame is given. Semi-supervised VOS~\cite{perazzi2016benchmark,pont20172017} is challenging as the model needs to address appearance changes, similar instances, occlusions, and scale variations based on the mask of the first frame. &#10;&#10;Existing deep learning-based methods to address the aforementioned problems can be categorized into three approaches. Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time~\cite{caelles2017one,voigtlaender2017online,maninis2018video,meinhardt2020make}. Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and &#10;Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}. &#10;These methods mainly focus on designing networks to improve segmentation accuracy.&#10;&#10;We observe that existing VOS models commonly generate inconsistent predictions when the same frames with different sizes are used as input. As shown in Fig.~\ref{fig:introduction}, the predictions from the different-size inputs (Figs.~\ref{fig:introduction}(b) and~\ref{fig:introduction}(c)) show the inconsistent results. Some methods~\cite{oh2018fast,yang2019anchor} address the scale-inconsistency problem by averaging the predictions of the inputs with multiple sizes. However, the simple averaging cannot fully address the scale-inconsistency as the magnitude of the scale inconsistency varies from pixel to pixel and the amount of the inconsistencies are different for each frame in a video sequence (Fig.~\ref{fig:introduction}(d)).&#10;&#10;In this paper, we propose a model-agnostic refinement framework that improves existing VOS models by addressing the aforementioned scale-inconsistency problem. We devise a multi-scale context aggregation module that combines the predictions from different-size inputs using learnable pixel-level attention. We train this module by constructing a pixel-level variance map based on the scale inconsistency from the multi-scale predictions. The pixel-level variance map is used for regularizing a segmentation loss, i.e. a pixel with larger-scale inconsistency is more penalized during the training. To prevent the false-positive error accumulation at the test time, we further present a self-supervised online adaptation that optimizes the model parameters based on scale inconsistency. Specifically, the proposed online adaptation consists of an intra-frame adaptation, which performs pseudo-label learning based on the pixel-level variance map, and an inter-frame adaptation, which enforces the consistency between the adjacent frames using color and distance cues. The experiments on the DAVIS16~\cite{perazzi2016benchmark} and DAVIS17~\cite{pont20172017} datasets show that the proposed method improves the $\mathcal{J\&amp;F}$ measure of three representative VOS models, OSVOS~\cite{caelles2017one}, RGMP~\cite{oh2018fast}, STM~\cite{oh2019video}, by $7.0\%$, $1.1\%$, $2.1\%$ in DAVIS16 and $12.3\%$, $0.7\%$, $1.8\%$ in DAVIS17, respectively.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Deep Learning Frameworks, Mathematics, Artificial Intelligence, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1706.09364" label="1706.09364">
-        <attvalues>
-          <attvalue for="0" value="Online Adaptation of Convolutional Neural Networks for Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  We tackle the task of semi-supervised video object segmentation, i.e.&#10;segmenting the pixels belonging to an object in the video using the ground&#10;truth pixel mask for the first frame. We build on the recently introduced&#10;one-shot video object segmentation (OSVOS) approach which uses a pretrained&#10;network and fine-tunes it on the first frame. While achieving impressive&#10;performance, at test time OSVOS uses the fine-tuned network in unchanged form&#10;and is not able to adapt to large changes in object appearance. To overcome&#10;this limitation, we propose Online Adaptive Video Object Segmentation (OnAVOS)&#10;which updates the network online using training examples selected based on the&#10;confidence of the network and the spatial configuration. Additionally, we add a&#10;pretraining step based on objectness, which is learned on PASCAL. Our&#10;experiments show that both extensions are highly effective and improve the&#10;state of the art on DAVIS to an intersection-over-union score of 85.7%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.06031" label="1709.06031">
-        <attvalues>
-          <attvalue for="0" value="Video Object Segmentation Without Temporal Information" />
-          <attvalue for="1" value="  Video Object Segmentation, and video processing in general, has been&#10;historically dominated by methods that rely on the temporal consistency and&#10;redundancy in consecutive video frames. When the temporal smoothness is&#10;suddenly broken, such as when an object is occluded, or some frames are missing&#10;in a sequence, the result of these methods can deteriorate significantly or&#10;they may not even produce any result at all. This paper explores the orthogonal&#10;approach of processing each frame independently, i.e disregarding the temporal&#10;information. In particular, it tackles the task of semi-supervised video object&#10;segmentation: the separation of an object from the background in a video, given&#10;its mask in the first frame. We present Semantic One-Shot Video Object&#10;Segmentation (OSVOS-S), based on a fully-convolutional neural network&#10;architecture that is able to successively transfer generic semantic&#10;information, learned on ImageNet, to the task of foreground segmentation, and&#10;finally to learning the appearance of a single annotated object of the test&#10;sequence (hence one shot). We show that instance level semantic information,&#10;when combined effectively, can dramatically improve the results of our previous&#10;method, OSVOS. We perform experiments on two recent video segmentation&#10;databases, which show that OSVOS-S is both the fastest and most accurate method&#10;in the state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.01866" label="2012.01866">
-        <attvalues>
-          <attvalue for="0" value="Make One-Shot Video Object Segmentation Efficient Again" />
-          <attvalue for="1" value="  Video object segmentation (VOS) describes the task of segmenting a set of&#10;objects in each frame of a video. In the semi-supervised setting, the first&#10;mask of each object is provided at test time. Following the one-shot principle,&#10;fine-tuning VOS methods train a segmentation model separately on each given&#10;object mask. However, recently the VOS community has deemed such a test time&#10;optimization and its impact on the test runtime as unfeasible. To mitigate the&#10;inefficiencies of previous fine-tuning approaches, we present efficient&#10;One-Shot Video Object Segmentation (e-OSVOS). In contrast to most VOS&#10;approaches, e-OSVOS decouples the object detection task and predicts only local&#10;segmentation masks by applying a modified version of Mask R-CNN. The one-shot&#10;test runtime and performance are optimized without a laborious and handcrafted&#10;hyperparameter search. To this end, we meta learn the model initialization and&#10;learning rates for the test time optimization. To achieve optimal learning&#10;behavior, we predict individual learning rates at a neuron level. Furthermore,&#10;we apply an online adaptation to address the common performance degradation&#10;throughout a sequence by continuously fine-tuning the model on previous mask&#10;predictions supported by a frame-to-frame bounding box propagation. e-OSVOS&#10;provides state-of-the-art results on DAVIS 2016, DAVIS 2017, and YouTube-VOS&#10;for one-shot fine-tuning methods while reducing the test runtime substantially.&#10;  Code is available at https://github.com/dvl-tum/e-osvos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.02646" label="1612.02646">
-        <attvalues>
-          <attvalue for="0" value="Learning Video Object Segmentation from Static Images" />
-          <attvalue for="1" value="  Inspired by recent advances of deep learning in instance segmentation and&#10;object tracking, we introduce video object segmentation problem as a concept of&#10;guided instance segmentation. Our model proceeds on a per-frame basis, guided&#10;by the output of the previous frame towards the object of interest in the next&#10;frame. We demonstrate that highly accurate object segmentation in videos can be&#10;enabled by using a convnet trained with static images only. The key ingredient&#10;of our approach is a combination of offline and online learning strategies,&#10;where the former serves to produce a refined mask from the previous frame&#10;estimate and the latter allows to capture the appearance of the specific object&#10;instance. Our method can handle different types of input annotations: bounding&#10;boxes and segments, as well as incorporate multiple annotated frames, making&#10;the system suitable for diverse applications. We obtain competitive results on&#10;three different datasets, independently from the type of input annotation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.05478" label="1612.05478">
-        <attvalues>
-          <attvalue for="0" value="Video Propagation Networks" />
-          <attvalue for="1" value="  We propose a technique that propagates information forward through video&#10;data. The method is conceptually simple and can be applied to tasks that&#10;require the propagation of structured information, such as semantic labels,&#10;based on video content. We propose a 'Video Propagation Network' that processes&#10;video frames in an adaptive manner. The model is applied online: it propagates&#10;information forward without the need to access future frames. In particular we&#10;combine two components, a temporal bilateral network for dense and video&#10;adaptive filtering, followed by a spatial network to refine features and&#10;increased flexibility. We present experiments on video object segmentation and&#10;semantic video segmentation and show increased performance comparing to the&#10;best previous task-specific methods, while having favorable runtime.&#10;Additionally we demonstrate our approach on an example regression task of color&#10;propagation in a grayscale video.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.09554" label="1703.09554">
-        <attvalues>
-          <attvalue for="0" value="Lucid Data Dreaming for Video Object Segmentation" />
-          <attvalue for="1" value="  Convolutional networks reach top quality in pixel-level video object&#10;segmentation but require a large amount of training data (1k~100k) to deliver&#10;such results. We propose a new training strategy which achieves&#10;state-of-the-art results across three evaluation datasets while using 20x~1000x&#10;less annotated data than competing methods. Our approach is suitable for both&#10;single and multiple object segmentation. Instead of using large training sets&#10;hoping to generalize across domains, we generate in-domain training data using&#10;the provided annotation on the first frame of each video to synthesize (&quot;lucid&#10;dream&quot;) plausible future video frames. In-domain per-video training data allows&#10;us to train high quality appearance- and motion-based models, as well as tune&#10;the post-processing stage. This approach allows to reach competitive results&#10;even when training from only a single annotated frame, without ImageNet&#10;pre-training. Our results indicate that using a larger training set is not&#10;automatically better, and that for the video object segmentation task a smaller&#10;training set that is closer to the target domain is more effective. This&#10;changes the mindset regarding how many training samples and general&#10;&quot;objectness&quot; knowledge are required for the video object segmentation task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.06647" label="1908.06647">
-        <attvalues>
-          <attvalue for="0" value="RANet: Ranking Attention Network for Fast Video Object Segmentation" />
-          <attvalue for="1" value="  Despite online learning (OL) techniques have boosted the performance of&#10;semi-supervised video object segmentation (VOS) methods, the huge time costs of&#10;OL greatly restrict their practicality. Matching based and propagation based&#10;methods run at a faster speed by avoiding OL techniques. However, they are&#10;limited by sub-optimal accuracy, due to mismatching and drifting problems. In&#10;this paper, we develop a real-time yet very accurate Ranking Attention Network&#10;(RANet) for VOS. Specifically, to integrate the insights of matching based and&#10;propagation based methods, we employ an encoder-decoder framework to learn&#10;pixel-level similarity and segmentation in an end-to-end manner. To better&#10;utilize the similarity maps, we propose a novel ranking attention module, which&#10;automatically ranks and selects these maps for fine-grained VOS performance.&#10;Experiments on DAVIS-16 and DAVIS-17 datasets show that our RANet achieves the&#10;best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and&#10;J&amp;F=85.5% on DAVIS-16. With OL, our RANet reaches J&amp;F=87.1% on DAVIS-16,&#10;exceeding state-of-the-art VOS methods. The code can be found at&#10;https://github.com/Storife/RANet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;{Online learning based methods}.\&#10;OL based methods~\cite{osvos,onavos,masktrack,osvos-s,reid,lucid,premvos,premvos2,premvos3} fine-tune on the first frame of a video to extract the primary object(s), and then segment the video frame-by-frame.\ OSVOS~\cite{osvos} uses a pre-trained object segmentation network, and fine-tunes it on the first frame of the test video.\ OnAVOS~\cite{onavos} extends OSVOS with an online adaptation mechanism, and OSVOS-S~\cite{osvos-s} utilizes semantic information from an instance segmentation network.\ LucidTracker~\cite{lucid} introduces a data augmentation mechanism for online fine-tuning.\ DyeNet~\cite{reid} integrates instance re-identification and temporal propagation, and uses OL to boost the performance.\ PReMVOS~\cite{premvos,premvos2,premvos3} integrates techniques from instance segmentation~\cite{mask-rcnn}, optical flow~\cite{flownet,flownet2}, refinement, and re-identification~\cite{personsearch} together with extensive fine-tuning, and achieves satisfactory performance.\ In summary, OL is very effective for the VOS task.\ Therefore, subsequent methods~\cite{masktrack, cinm, reid} regard OL as a conventional technique to boost VOS performance.\ However, OL based methods are computationally expensive for practical applications.\ In this work, we solve the VOS problem with a very fast network that obtains a competitive accuracy at a speed of $30$ FPS on DAVIS$_{16}$, $130\sim400$ times faster than previous OL based methods~\cite{osvos, masktrack, osvos-s, onavos}.&#10;&#10;{Propagation or matching based methods}.\ Propagation based methods additionally resort to the previous frame(s) for better VOS performance.\ Masktrack\cite{masktrack} tackles VOS by combining the image and segmentation mask of the previous frame as the input.\ This strategy is also used in CINM~\cite{cinm}, OSMN~\cite{osmn} and RGMP~\cite{rgmp}.\ &#10;RGMP~\cite{rgmp} stacks the first, previous and current frames' features during propagation through a Siamese architecture network.\ In this work, we also utilize the Siamese network, but use a pixel-level matching technique instead of simply stacking, and feed the previous frame's mask into the decoder, instead of the encoder as in RGMP~\cite{rgmp}.\ OSMN~\cite{osmn} introduces a modulator to manipulate the intermediate layers of the segmentation network, by using visual and spatial guidance.\&#10;Optical flow~\cite{flownet,flownet2} is also used to guide the propagation process in many methods~\cite{sfl,masktrack,ofl,ctn}.\ However, it fails to distinguish non-rigid objects from motionless sections of the background.\ All these strategies are effective, but still, suffer from the drifting problem.\ MaskTrack~\cite{masktrack} embraces OL to remember the target object, which eliminates this problem and improves VOS performance.\ However, since OL is time-consuming, we employ more efficient matching techniques to handle this drifting problem.&#10;&#10;Matching based methods~\cite{videomatch, pml, plm, Voigtlaender2019FEELVOS} are very efficient.\ They first calculate pixel-level matching between the features of the template frame and the current frame in videos, and then segment each pixel of the current frame directly from the matching results.\ Pixel-Wise Metric Learning~\cite{pml} predicts each pixel by nearest neighbor matching in pixel space to the template frame.\ However, the point-to-point correspondence strategy~\cite{deepmatching, plm} often results in noisy predictions.\ To ease this problem, we apply a decoder to utilize the matching results as guidance.\ Hu~\etal proposed a soft matching mechanism in VideoMatch~\cite{videomatch}, which performs soft segmentation upon the averaged similarity score maps of matching features to generate smooth predictions.\ However, due to the lack of temporal information, they still suffer from the mismatching problem.\ In this work, we employ both the strategies of point-to-point correspondence matching for pixel-level object location and temporal propagation, to handle the mismatching and drifting problem.\ &#10;FEELVOS~\cite{Voigtlaender2019FEELVOS} employs global and local matching for more stable pixel-level matching, but only calculates extreme value maps for final segmentation, losing major information of the similarity maps.\ Our RAM can better utilize the similarity information.\ Moreover, for faster speed, we use a light-weight decoder and employ a standard ResNet~\cite{resnet} pre-trained on ImageNet~\cite{imagenet} as the backbone, instead of the time-consuming semantic segmentation networks~\cite{deeplab, deeplabv3, deeplabv3+, YazhaoSegICCV2019} used in previous methods~\cite{videomatch, masktrack}.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2109.11404" label="2109.11404">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Memory Matching Network for Video Object Segmentation" />
-          <attvalue for="1" value="  We present Hierarchical Memory Matching Network (HMMN) for semi-supervised&#10;video object segmentation. Based on a recent memory-based method [33], we&#10;propose two advanced memory read modules that enable us to perform memory&#10;reading in multiple scales while exploiting temporal smoothness. We first&#10;propose a kernel guided memory matching module that replaces the non-local&#10;dense memory read, commonly adopted in previous memory-based methods. The&#10;module imposes the temporal smoothness constraint in the memory read, leading&#10;to accurate memory retrieval. More importantly, we introduce a hierarchical&#10;memory matching scheme and propose a top-k guided memory matching module in&#10;which memory read on a fine-scale is guided by that on a coarse-scale. With the&#10;module, we perform memory read in multiple scales efficiently and leverage both&#10;high-level semantic and low-level fine-grained memory features to predict&#10;detailed object masks. Our network achieves state-of-the-art performance on the&#10;validation sets of DAVIS 2016/2017 (90.8% and 84.7%) and YouTube-VOS 2018/2019&#10;(82.6% and 82.5%), and test-dev set of DAVIS 2017 (78.6%). The source code and&#10;model are available online: https://github.com/Hongje/HMMN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.05210" label="2106.05210">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Space-Time Networks with Improved Memory Coverage for&#10;  Efficient Video Object Segmentation" />
-          <attvalue for="1" value="  This paper presents a simple yet effective approach to modeling space-time&#10;correspondences in the context of video object segmentation. Unlike most&#10;existing approaches, we establish correspondences directly between frames&#10;without re-encoding the mask features for every object, leading to a highly&#10;efficient and robust framework. With the correspondences, every node in the&#10;current query frame is inferred by aggregating features from the past in an&#10;associative fashion. We cast the aggregation process as a voting problem and&#10;find that the existing inner-product affinity leads to poor use of memory with&#10;a small (fixed) subset of memory nodes dominating the votes, regardless of the&#10;query. In light of this phenomenon, we propose using the negative squared&#10;Euclidean distance instead to compute the affinities. We validated that every&#10;memory node now has a chance to contribute, and experimentally showed that such&#10;diversified voting is beneficial to both memory efficiency and inference&#10;accuracy. The synergy of correspondence networks and diversified voting works&#10;exceedingly well, achieves new state-of-the-art results on both DAVIS and&#10;YouTubeVOS datasets while running significantly faster at 20+ FPS for multiple&#10;objects without bells and whistles.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.10895" label="1910.10895">
-        <attvalues>
-          <attvalue for="0" value="Anchor Diffusion for Unsupervised Video Object Segmentation" />
-          <attvalue for="1" value="  Unsupervised video object segmentation has often been tackled by methods&#10;based on recurrent neural networks and optical flow. Despite their complexity,&#10;these kinds of approaches tend to favour short-term temporal dependencies and&#10;are thus prone to accumulating inaccuracies, which cause drift over time.&#10;Moreover, simple (static) image segmentation models, alone, can perform&#10;competitively against these methods, which further suggests that the way&#10;temporal dependencies are modelled should be reconsidered. Motivated by these&#10;observations, in this paper we explore simple yet effective strategies to model&#10;long-term temporal dependencies. Inspired by the non-local operators of [70],&#10;we introduce a technique to establish dense correspondences between pixel&#10;embeddings of a reference &quot;anchor&quot; frame and the current one. This allows the&#10;learning of pairwise dependencies at arbitrarily long distances without&#10;conditioning on intermediate frames. Without online supervision, our approach&#10;can suppress the background and precisely segment the foreground object even in&#10;challenging scenarios, while maintaining consistent performance over time. With&#10;a mean IoU of $81.7\%$, our method ranks first on the DAVIS-2016 leaderboard of&#10;unsupervised methods, while still being competitive against state-of-the-art&#10;online semi-supervised approaches. We further evaluate our method on the FBMS&#10;dataset and the ViSal video saliency dataset, showing results competitive with&#10;the state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value=" &#10;The problem of video object segmentation (VOS) is tackled by the computer vision community in the unsupervised or semi-supervised settings, which are defined by the level of supervision provided at test time.&#10;&#10;Semi-supervised VOS methods are provided with a pixel-wise mask identifying the target object in the first frame of a video.&#10;When aiming at very high segmentation accuracy, methods generally perform online fine-tuning on the basis of this supervision~\cite{osvos,lucid,vsreid,premvos,osvoss,msk,onavos}, sometimes exploiting data-augmentation techniques~\cite{osvos,lucid} or self-supervision~\cite{onavos}.&#10;As online fine-tuning can take up to several minutes per video, many recently proposed methods renounce to it and instead aim at a faster online speed (\eg,~\cite{pml,favos,siammask}).&#10;These faster semi-supervised approaches come in many flavours.&#10;For instance, Chen~\etal~\cite{pml} learn a metric space for pixel embeddings, which is then used to establish associations between pixels across frames, while Cheng~\etal~\cite{favos} suggest to individually track object parts from the first frame with a visual object tracker~\cite{siamFC} and then aggregate them according to their similarity with the initialisation mask.&#10;&#10;Unsupervised VOS methods, instead, cannot rely on any supervision at test time and are often based on optical flow and RNNs.&#10;The purely optical flow-based MP-Net~\cite{mp} discards appearance modelling and casts segmentation as foreground motion prediction, an approach which poorly deals with static foreground objects.&#10;To address this problem, several methods (\eg,~LVO~\cite{gru-mp}, SegFlow~\cite{segflow}, MotAdapt~\cite{teacher-student} and MBN~\cite{mbn}) suggest to integrate appearance-based and optical flow-based features together, leading to variations of the ``two-stream model'' presenting two dedicated parallel branches.&#10;The drawbacks of these methods are threefold.&#10;First, flow estimation networks are typically trained on synthetic datasets and can thus result in poor performance when deployed in the real world.&#10;Second, while modelling long-term temporal dependencies is critical for adapting to significant online changes, the vector fields can only model short-term one-step dependencies.&#10;Targeting this issue, Tokmakow~\etal~\cite{gru-mp} proposed to extend the horizon spanned by optical flow-based features by employing a convolutional gated recurrent unit~\cite{gru}.&#10;Third, vector fields cannot distinguish foreground and background objects when they move in a synchronised fashion (\eg,~the cars in a traffic jam).&#10;Li~\etal~\cite{mbn} attempt to address this issue by employing a bilateral network for detecting the motion of background objects.&#10;Our investigations with a much simpler appearance-based approach show that optical flow may not be an essential component of unsupervised VOS systems.&#10;&#10;RNN-based models are often challenged by the problems of exploding and vanishing gradients~\cite{bengio,difficultRNN}, which limit their long-term modelling capability.&#10;Among the methods that make use of recurrent connections, Song~\etal~\cite{pdb} propose a novel convolutional long short-term memory~\cite{lstm} architecture, in which two atrous convolution~\cite{deeplabv3} layers are stacked along the forward axis and propagate features in opposite directions.&#10;&#10;Recently, it has been shown~\cite{iet,rgmp,onavos} that both recurrent and optical flow-based methods significantly suffer from a deterioration in the quality of their predictions over time.&#10;This has motivated the several approaches (including ours) that tackle video object segmentation by simply learning similarities between pixel embeddings (\eg,~\cite{pml,instanceDML,iet,mbn}).&#10;These methods first select a set of seed pixels that are most likely to belong to the foreground object and then classify all other pixels based on their similarities to these seeds, for instance by thresholding or by propagating labels between neighbours.&#10;Fathi~\etal~\cite{instanceDML} adopt this approach for semantic instance segmentation, in which the pairwise pixel similarity function measures the likelihood of two pixels belonging to the same instance.&#10;IET~\cite{iet} extends this concept to video sequences.&#10;Similarly, it selects a set of foreground and background seeds for each frame and organises them into tracks.&#10;It then segments each frame individually based on pixel similarities with the foreground and background seeds.&#10;Note that IET utilises pre-trained instance embeddings.&#10;MBN~\cite{mbn} extends IET with a bilateral filtering network that filters false-positive foreground predictions using optical flow features and an energy minimisation procedure on a graph of seeds sampled from a few consecutive frames.&#10;When segmenting frame $t$, MBN classifies each pixel by assigning it the label of the seed (sampled from frames $t{-}1$, $t$, and $t{+}1$) with which it has the smallest embedding distance.&#10;&#10;The main drawback of these methods is in the complexity involved in the procedures of seed selection, ranking and classification, critical for achieving good performance.&#10;Moreover, these algorithms also depend on multiple scores such as motion saliency and objectness that need to be carefully calibrated and combined into one final metric.&#10;&#10;Albeit our proposal is related to this last class of approaches, it is considerably simpler.&#10;Instead of separately learning individual components from image datasets and classifying pixels based on similarities with seeds, our method performs similarity learning, feature propagation and binary segmentation in a single network.&#10; " />
-        </attvalues>
-      </node>
-      <node id="2401.09252" label="2401.09252">
-        <attvalues>
-          <attvalue for="0" value="3D Scene Geometry Estimation from 360$^\circ$ Imagery: A Survey" />
-          <attvalue for="1" value="  This paper provides a comprehensive survey on pioneer and state-of-the-art 3D&#10;scene geometry estimation methodologies based on single, two, or multiple&#10;images captured under the omnidirectional optics. We first revisit the basic&#10;concepts of the spherical camera model, and review the most common acquisition&#10;technologies and representation formats suitable for omnidirectional (also&#10;called 360$^\circ$, spherical or panoramic) images and videos. We then survey&#10;monocular layout and depth inference approaches, highlighting the recent&#10;advances in learning-based solutions suited for spherical data. The classical&#10;stereo matching is then revised on the spherical domain, where methodologies&#10;for detecting and describing sparse and dense features become crucial. The&#10;stereo matching concepts are then extrapolated for multiple view camera setups,&#10;categorizing them among light fields, multi-view stereo, and structure from&#10;motion (or visual simultaneous localization and mapping). We also compile and&#10;discuss commonly adopted datasets and figures of merit indicated for each&#10;purpose and list recent results for completeness. We conclude this paper by&#10;pointing out current and future trends.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The world is three-dimensional (3D). As such, recovering 3D information about real-world objects allows the exploration of many relevant applications, including &#10;self-driving cars~\cite{Wang:ACCV:2018, Schonbein:IROS:2014}, &#10;robot navigation~\cite{Moreau:SITIS:2012,Fernandez-Labrador:LRA:2020},&#10;virtual tourism~\cite{Fangi:ISPRS:2018,Kim:JCVIU:2015}, &#10;infrastructure inspection~\cite{Pathak:IST:2016,Pathak:JCMSI:2017}, archaeological~\cite{Pagani:VAST:2011,Gava:VISAPP:2015} and architectural modeling~\cite{Fangi:ISPRS:2018,Giovanni:ECCV:2016}, &#10;city planning~\cite{Kim:IVMSPW:2013,Kuhn:GCPR:2017}, and 3D cinema~\cite{Azevedo:TCSVT:2019,Thatte:VCIP:2017}. &#10;&#10;Many sensors can be used to obtain 3D data from real objects, such as light detection and ranging &#10;\cite{Kim:IJCV:2013}, structured light~\cite{Silberman:ICCV:2011}, and time of flight &#10;\cite{Lukierski:ECMR:2015}. &#10;There is a plethora of approaches for inferring 3D information from plain color images/videos. &#10;The widespread accessibility and low-cost of consumer cameras is a strong motivation for the continued research efforts&#10;devoted to image-based 3D scene reconstruction methods~\cite{Kim:ICCV:2009}. &#10;In theory, 3D information can only be inferred from two or more&#10;captures of the scene, as in typical multi-view stereo &#10;\cite{Seitz:CVPR:2006} or structure from motion&#10; \cite{Ozyesil:AN:2017} approaches. &#10;However, recent approaches are exploring machine learning&#10;to perform single-image depth inference~\cite{Hamzah:Sensors:2016,Kumari:IJEM:2016,Scharstein:SMBV:2001}.&#10;Most methods developed so far rely on traditional perspective/pinhole-based cameras, which have a narrow field of view (FoV) and &#10;thus might &#10;require thousands of captures to model large scenes~\cite{Agarwal:CACM:2011, Pollefeys:IJCV:2008}. &#10;&#10;Unlike pinhole cameras, truly omnidirectional cameras capture the light intensities from the entire surrounding &#10;environment ($360^\circ\times180^\circ$) at once~ \cite{Im:ECCV:2016, Huang:VR:2017}. &#10; Thus,&#10;from the application point of view, only two captures might be necessary for recovering 3D information from the whole scenario (except for occlusions and disocclusions) based on triangulation-like algorithms~\cite{Oliveira:ICASSP:2019} or even a single capture might be needed when using modern deep learning approaches~\cite{Tateno:ECCV:2018}.&#10;Although catadioptric sensors date from the early 1970s~\cite{Rees:Patent:1970}, the recent release of affordable devices for capturing and visualizing &#10;full FoV media besides the novel augmented, mixed and virtual (AR/MR/VR) applications are inciting both the scientific community and the industry to invest efforts in novel technological solutions.&#10;Popular consumer-grade cameras include the &#10;Samsung Gear 360&#10;and the &#10;Ricoh Theta,&#10;whereas examples of head-mounted displays &#10;are the &#10;Samsung Gear VR&#10;and the&#10;Oculus Rift.&#10;It is worth mentioning that the existing nomenclature for 360 images is vast, &#10;and the terms &#10;omnidirectional \cite{Won:ICRA:2019, Wegner:ICIP:2018},&#10;spherical \cite{Silveira:CVPR:2019, Eder:3DV:2019},&#10;full panorama \cite{Pagani:ICCV:2011, Im:ECCV:2016}, and &#10;360\degree \cite{Zioulis:3DV:2019, Huang:VR:2017} images/videos/cameras are often used as synonyms.&#10;&#10;360\degree images &#10;are&#10;intrinsically defined on the sphere surface and &#10;present three-degrees of freedom (3-DoF) natively, &#10;associated with the rotation angles pitch, yaw, and roll.&#10;This allows, for instance, &#10;{head-mounted displays}&#10;users to look at any direction of the captured view by performing rotational head movements. &#10;However, restricting the movements only to rotations may break the immersion sense when the observer's head translates and the rendered scene mismatches, causing discomfort and sickness~\cite{Thatte:VCIP:2017}.&#10;Although most &#10;{head-mounted displays}&#10;support 6-DoF tracking~\cite{Huang:VR:2017}, the vast majority of real capture content is monoscopic~\cite{Im:ECCV:2016}, so that full immersion is mostly experienced on computer-generated imagery~\cite{Luo:TVCG:2018}.&#10;Depth perception is accomplished by presenting a stereoscopic image pair restricted to the user eye's viewport, which may reduce the discomfort~\cite{Serrano:TVCG:2019}. &#10;For allowing small head translational movements, both color (often called texture) and depth (or disparity) information must be provided for that specific virtual camera (and user) location.&#10;This still limited application scenario is called 3-DoF+~\cite{Jeong:Sensors:2018,Silveira:VR:2019}, but approximates the full 6-DoF-enabled navigation, where colored 3D information is required for every possible location within the scene~\cite{Wien:JETCAS:2019}. &#10;&#10;Estimating 3D scene geometry from 360\degree imagery allows to address all the classical problems mentioned in the first paragraph of this paper and is also pivotal for providing full immersive experiences to &#10;{head-mounted displays}&#10;users in novel AR/MR/VR-based applications.&#10;Despite the benefits of presenting full FoV, both the capture sensors (CCD, CMOS) and the representation formats (equirectangular images or cube-maps) for spherical images involve a regular rectangular 2D grid. As such, omnidirectional &#10;media &#10;present heavy &#10;{non-affine}&#10;distortions that are inherent to the spherical camera model~\cite{Eder:CVPR:2020}.&#10;Therefore, most of the image processing and computer vision algorithms developed in the past -- which were designed to work with perspective images -- tend to fail when directly applied to omnidirectional media, regardless of their objective~\cite{Cruz-Mota:IJCV:2012,Su:NIPS:2017,Tateno:ECCV:2018,Azevedo:TCSVT:2019}. &#10;In this context, there is a growing number of works trying to bridge this gap by employing geometry- and learning-based approaches for recovering 3D information from one, two, or more full panoramas.&#10;{How these applications are explored in immersive media is discussed throughout the text and summarized in the conclusions (Section~\ref{sec:conclusions}).}&#10;&#10;This survey paper aims to briefly review the spherical imaging model (along with common acquisition pipelines and representation formats), and then focus on a comprehensive analysis of representative methods that tackle the 3D geometry estimation problem using one or more 360\degree images.&#10;A secondary goal is to compile meaningful publicly available datasets up to date, as well as the most widely accepted evaluation metrics for comparing the existing techniques.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Depth Inference Methods, Machine Learning, Computer Vision, Mathematics, Spherical Camera Models, Stereo Matching Techniques, Physics, Geometric Computing, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1901.01848" label="1901.01848">
-        <attvalues>
-          <attvalue for="0" value="Visual Distortions in 360-degree Videos" />
-          <attvalue for="1" value="  Omnidirectional (or 360-degree) images and videos are emergent signals in&#10;many areas such as robotics and virtual/augmented reality. In particular, for&#10;virtual reality, they allow an immersive experience in which the user is&#10;provided with a 360-degree field of view and can navigate throughout a scene,&#10;e.g., through the use of Head Mounted Displays. Since it represents the full&#10;360-degree field of view from one point of the scene, omnidirectional content&#10;is naturally represented as spherical visual signals. Current approaches for&#10;capturing, processing, delivering, and displaying 360-degree content, however,&#10;present many open technical challenges and introduce several types of&#10;distortions in these visual signals. Some of the distortions are specific to&#10;the nature of 360-degree images, and often different from those encountered in&#10;the classical image communication framework. This paper provides a first&#10;comprehensive review of the most common visual distortions that alter&#10;360-degree signals undergoing state of the art processing in common&#10;applications. While their impact on viewers' visual perception and on the&#10;immersive experience at large is still unknown ---thus, it stays an open&#10;research topic--- this review serves the purpose of identifying the main causes&#10;of visual distortions in the end-to-end 360-degree content distribution&#10;pipeline. It is essential as a basis for benchmarking different processing&#10;techniques, allowing the effective design of new algorithms and applications.&#10;It is also necessary to the deployment of proper psychovisual studies to&#10;characterise the human perception of these new images in interactive and&#10;immersive applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.08493" label="1701.08493">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Structure from Motion" />
-          <attvalue for="1" value="  The structure from motion (SfM) problem in computer vision is the problem of&#10;recovering the three-dimensional ($3$D) structure of a stationary scene from a&#10;set of projective measurements, represented as a collection of two-dimensional&#10;($2$D) images, via estimation of motion of the cameras corresponding to these&#10;images. In essence, SfM involves the three main stages of (1) extraction of&#10;features in images (e.g., points of interest, lines, etc.) and matching these&#10;features between images, (2) camera motion estimation (e.g., using relative&#10;pairwise camera positions estimated from the extracted features), and (3)&#10;recovery of the $3$D structure using the estimated motion and features (e.g.,&#10;by minimizing the so-called reprojection error). This survey mainly focuses on&#10;relatively recent developments in the literature pertaining to stages (2) and&#10;(3). More specifically, after touching upon the early factorization-based&#10;techniques for motion and structure estimation, we provide a detailed account&#10;of some of the recent camera location estimation methods in the literature,&#10;followed by discussion of notable techniques for $3$D structure recovery. We&#10;also cover the basics of the simultaneous localization and mapping (SLAM)&#10;problem, which can be viewed as a specific case of the SfM problem. Further,&#10;our survey includes a review of the fundamentals of feature extraction and&#10;matching (i.e., stage (1) above), various recent methods for handling&#10;ambiguities in $3$D scenes, SfM techniques involving relatively uncommon camera&#10;models and image features, and popular sources of data and SfM software.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.10904" label="1902.10904">
-        <attvalues>
-          <attvalue for="0" value="SweepNet: Wide-baseline Omnidirectional Depth Estimation" />
-          <attvalue for="1" value="  Omnidirectional depth sensing has its advantage over the conventional stereo&#10;systems since it enables us to recognize the objects of interest in all&#10;directions without any blind regions. In this paper, we propose a novel&#10;wide-baseline omnidirectional stereo algorithm which computes the dense depth&#10;estimate from the fisheye images using a deep convolutional neural network. The&#10;capture system consists of multiple cameras mounted on a wide-baseline rig with&#10;ultrawide field of view (FOV) lenses, and we present the calibration algorithm&#10;for the extrinsic parameters based on the bundle adjustment. Instead of&#10;estimating depth maps from multiple sets of rectified images and stitching&#10;them, our approach directly generates one dense omnidirectional depth map with&#10;full 360-degree coverage at the rig global coordinate system. To this end, the&#10;proposed neural network is designed to output the cost volume from the warped&#10;images in the sphere sweeping method, and the final depth map is estimated by&#10;taking the minimum cost indices of the aggregated cost volume by SGM. For&#10;training the deep neural network and testing the entire system, realistic&#10;synthetic urban datasets are rendered using Blender. The experiments using the&#10;synthetic and real-world datasets show that our algorithm outperforms the&#10;conventional depth estimation methods and generate highly accurate depth maps.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.08529" label="2011.08529">
-        <attvalues>
-          <attvalue for="0" value="Slender Object Detection: Diagnoses and Improvements" />
-          <attvalue for="1" value="  In this paper, we are concerned with the detection of a particular type of&#10;objects with extreme aspect ratios, namely \textbf{slender objects}. In&#10;real-world scenarios, slender objects are actually very common and crucial to&#10;the objective of a detection system. However, this type of objects has been&#10;largely overlooked by previous object detection algorithms. Upon our&#10;investigation, for a classical object detection method, a drastic drop of&#10;$18.9\%$ mAP on COCO is observed, if solely evaluated on slender objects.&#10;Therefore, we systematically study the problem of slender object detection in&#10;this work. Accordingly, an analytical framework with carefully designed&#10;benchmark and evaluation protocols is established, in which different&#10;algorithms and modules can be inspected and compared. \New Our study reveals&#10;that effective slender object detection can be achieved ~\textbf{with none of}&#10;(1) anchor-based localization; (2) specially designed box representations.&#10;Instead, \textbf{the critical aspect of improving slender object detection is&#10;feature adaptation}. It identifies and extends the insights of existing methods&#10;that are previously underexploited. Furthermore, we propose a feature adaption&#10;strategy that achieves clear and consistent improvements over current&#10;representative object detection methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;As a fundamental task in computer vision that draws considerable research attention from the community, object detection~\cite{rcnn, fasterrcnn, YOLO} has made substantial progress in recent years.&#10;As the needs of real-world applications in a wide variety of scenarios arise~\cite{widerface, aerial}, the significance of research regarding a particular topic elevates. The works on improving specific aspects~\cite{relation, scale} of object detection, such as detecting dense objects~\cite{florence2018dense, densebox} and small objects~\cite{augmentation}, boost the practical value of object detection and consequently inspire further advances~\cite{cascade, augfpn}.&#10;&#10;While a large portion of the problems have been well investigated and numerous new ideas have been proposed, grand challenges remained in object detection. \cite{retina} propose the focal loss to tackle dense object detection and prompt it to become a common practice for classification loss in object detection.&#10;Object scale has been widely considered in model design, as various detection paradigms~\cite{rcnn}, augmentation schemes~\cite{augmentation}, and modules~\cite{sod} are proposed to improve small object detection. Such insightful works propel object detection methods to transfer from academic research to a wide variety of real-world applications~\cite{plastics, db}. Despite such progress in object detection, one significant problem has not been formally explored in previous works. &#10;&#10;This work is dedicated to studying the problem of slender object detection. From the perspective of academic research, the distinctive properties of slender objects pose special challenges, which give rise to research topics of scientific value. From the perspective of application, once slender objects can be well-handled, the practical utility of object detection systems will become higher.&#10;&#10;Inspired by previous works~\cite{diagnose,rethinking,survey}, which provide in-depth ablations, analyses, and insights regarding object detection algorithms, we start with diagnosing and analyzing existing methods for object detection.&#10;Specifically, we address issues in slenderness definition, feasible evaluation protocols, and data bias neutralization. A unified analytical framework is established, plus a standard detection pipeline, to dissect and compare different approaches in a clear and fair manner.&#10;We make it convenient to identify the key factors of previous methods, effective choices for model design, and potential directions for improvement.&#10;Key findings related to effective slender object detection, which is proven practicable without anchor-based localization nor specialized box representations, are presented in Sec.~\ref{dissections}.&#10;&#10;Beyond diagnoses and analyses, we further propose strategies to boost the detection of slender objects. In particular, a generalized feature adaption module, called self-adaption, is introduced. In addition, we show potential trade-off measures between the detection of slender and regular objects. According to the quantitative experiments (see Sec.~\ref{improvements}), the proposed feature adaptation has proven effective for slender objects while also working well for regular objects (see Tab.~\ref{tab:backbones}).&#10;&#10;In summary, the main contributions of this paper are as follows:&#10;\begin{itemize}&#10; \item We are the first to formally investigate the problem of slender object detection, which is important but largely overlooked by previous works. Issues preventing from in-depth study in slender object detection are addressed, including definition, metrics, and evaluation bias against slender objects. \item We construct an analytical framework for rigorously diagnosing different object detection methods. With this framework, a series of key insights and valuable findings, which may inspire other researchers in the field of object detection, is derived.&#10; \item We identify the feature adaption module as a key factor for the improvement of slender object detection. A generalized feature adaption module, called self-adaption, is devised. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Computer Science, Slender Object Analysis, Feature Adaptation" />
-        </attvalues>
-      </node>
-      <node id="1311.2524" label="1311.2524">
-        <attvalues>
-          <attvalue for="0" value="Rich feature hierarchies for accurate object detection and semantic&#10;  segmentation" />
-          <attvalue for="1" value="  Object detection performance, as measured on the canonical PASCAL VOC&#10;dataset, has plateaued in the last few years. The best-performing methods are&#10;complex ensemble systems that typically combine multiple low-level image&#10;features with high-level context. In this paper, we propose a simple and&#10;scalable detection algorithm that improves mean average precision (mAP) by more&#10;than 30% relative to the previous best result on VOC 2012---achieving a mAP of&#10;53.3%. Our approach combines two key insights: (1) one can apply high-capacity&#10;convolutional neural networks (CNNs) to bottom-up region proposals in order to&#10;localize and segment objects and (2) when labeled training data is scarce,&#10;supervised pre-training for an auxiliary task, followed by domain-specific&#10;fine-tuning, yields a significant performance boost. Since we combine region&#10;proposals with CNNs, we call our method R-CNN: Regions with CNN features. We&#10;also compare R-CNN to OverFeat, a recently proposed sliding-window detector&#10;based on a similar CNN architecture. We find that R-CNN outperforms OverFeat by&#10;a large margin on the 200-class ILSVRC2013 detection dataset. Source code for&#10;the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.06523" label="1511.06523">
-        <attvalues>
-          <attvalue for="0" value="WIDER FACE: A Face Detection Benchmark" />
-          <attvalue for="1" value="  Face detection is one of the most studied topics in the computer vision&#10;community. Much of the progresses have been made by the availability of face&#10;detection benchmark datasets. We show that there is a gap between current face&#10;detection performance and the real world requirements. To facilitate future&#10;face detection research, we introduce the WIDER FACE dataset, which is 10 times&#10;larger than existing datasets. The dataset contains rich annotations, including&#10;occlusions, poses, event categories, and face bounding boxes. Faces in the&#10;proposed dataset are extremely challenging due to large variations in scale,&#10;pose and occlusion, as shown in Fig. 1. Furthermore, we show that WIDER FACE&#10;dataset is an effective training source for face detection. We benchmark&#10;several representative detection systems, providing an overview of&#10;state-of-the-art performance and propose a solution to deal with large scale&#10;variation. Finally, we discuss common failure cases that worth to be further&#10;investigated. Dataset can be downloaded at:&#10;mmlab.ie.cuhk.edu.hk/projects/WIDERFace&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.08756" label="1806.08756">
-        <attvalues>
-          <attvalue for="0" value="Dense Object Nets: Learning Dense Visual Object Descriptors By and For&#10;  Robotic Manipulation" />
-          <attvalue for="1" value="  What is the right object representation for manipulation? We would like&#10;robots to visually perceive scenes and learn an understanding of the objects in&#10;them that (i) is task-agnostic and can be used as a building block for a&#10;variety of manipulation tasks, (ii) is generally applicable to both rigid and&#10;non-rigid objects, (iii) takes advantage of the strong priors provided by 3D&#10;vision, and (iv) is entirely learned from self-supervision. This is hard to&#10;achieve with previous methods: much recent work in grasping does not extend to&#10;grasping specific objects or other tasks, whereas task-specific learning may&#10;require many trials to generalize well across object configurations or other&#10;tasks. In this paper we present Dense Object Nets, which build on recent&#10;developments in self-supervised dense descriptor learning, as a consistent&#10;object representation for visual understanding and manipulation. We demonstrate&#10;they can be trained quickly (approximately 20 minutes) for a wide variety of&#10;previously unseen and potentially non-rigid objects. We additionally present&#10;novel contributions to enable multi-object descriptor learning, and show that&#10;by modifying our training procedure, we can either acquire descriptors which&#10;generalize across classes of objects, or descriptors that are distinct for each&#10;object instance. Finally, we demonstrate the novel application of learned dense&#10;descriptors to robotic manipulation. We demonstrate grasping of specific points&#10;on an object across potentially deformed object configurations, and demonstrate&#10;using class general descriptors to transfer specific grasps across objects in a&#10;class.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11172" label="1906.11172">
-        <attvalues>
-          <attvalue for="0" value="Learning Data Augmentation Strategies for Object Detection" />
-          <attvalue for="1" value="  Data augmentation is a critical component of training deep learning models.&#10;Although data augmentation has been shown to significantly improve image&#10;classification, its potential has not been thoroughly investigated for object&#10;detection. Given the additional cost for annotating images for object&#10;detection, data augmentation may be of even greater importance for this&#10;computer vision task. In this work, we study the impact of data augmentation on&#10;object detection. We first demonstrate that data augmentation operations&#10;borrowed from image classification may be helpful for training detection&#10;models, but the improvement is limited. Thus, we investigate how learned,&#10;specialized data augmentation policies improve generalization performance for&#10;detection models. Importantly, these augmentation policies only affect training&#10;and leave a trained model unchanged during evaluation. Experiments on the COCO&#10;dataset indicate that an optimized data augmentation policy improves detection&#10;accuracy by more than +2.3 mAP, and allow a single inference model to achieve a&#10;state-of-the-art accuracy of 50.7 mAP. Importantly, the best policy found on&#10;COCO may be transferred unchanged to other detection datasets and models to&#10;improve predictive accuracy. For example, the best augmentation policy&#10;identified with COCO improves a strong baseline on PASCAL-VOC by +2.7 mAP. Our&#10;results also reveal that a learned augmentation policy is superior to&#10;state-of-the-art architecture regularization methods for object detection, even&#10;when considering strong baselines. Code for training with the learned policy is&#10;available online at&#10;https://github.com/tensorflow/tpu/tree/master/models/official/detection&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.08883" label="1811.08883">
-        <attvalues>
-          <attvalue for="0" value="Rethinking ImageNet Pre-training" />
-          <attvalue for="1" value="  We report competitive results on object detection and instance segmentation&#10;on the COCO dataset using standard models trained from random initialization.&#10;The results are no worse than their ImageNet pre-training counterparts even&#10;when using the hyper-parameters of the baseline system (Mask R-CNN) that were&#10;optimized for fine-tuning pre-trained models, with the sole exception of&#10;increasing the number of training iterations so the randomly initialized models&#10;may converge. Training from random initialization is surprisingly robust; our&#10;results hold even when: (i) using only 10% of the training data, (ii) for&#10;deeper and wider models, and (iii) for multiple tasks and metrics. Experiments&#10;show that ImageNet pre-training speeds up convergence early in training, but&#10;does not necessarily provide regularization or improve final target task&#10;accuracy. To push the envelope we demonstrate 50.9 AP on COCO object detection&#10;without using any external data---a result on par with the top COCO 2017&#10;competition results that used ImageNet pre-training. These observations&#10;challenge the conventional wisdom of ImageNet pre-training for dependent tasks&#10;and we expect these discoveries will encourage people to rethink the current de&#10;facto paradigm of `pre-training and fine-tuning' in computer vision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.09408" label="1907.09408">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Deep Learning-based Object Detection" />
-          <attvalue for="1" value="  Object detection is one of the most important and challenging branches of&#10;computer vision, which has been widely applied in peoples life, such as&#10;monitoring security, autonomous driving and so on, with the purpose of locating&#10;instances of semantic objects of a certain class. With the rapid development of&#10;deep learning networks for detection tasks, the performance of object detectors&#10;has been greatly improved. In order to understand the main development status&#10;of object detection pipeline, thoroughly and deeply, in this survey, we first&#10;analyze the methods of existing typical detection models and describe the&#10;benchmark datasets. Afterwards and primarily, we provide a comprehensive&#10;overview of a variety of object detection methods in a systematic manner,&#10;covering the one-stage and two-stage detectors. Moreover, we list the&#10;traditional and new applications. Some representative branches of object&#10;detection are analyzed as well. Finally, we discuss the architecture of&#10;exploiting these object detection methods to build an effective and efficient&#10;system and point out a set of development trends to better follow the&#10;state-of-the-art algorithms and further research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.08650" label="2403.08650">
-        <attvalues>
-          <attvalue for="0" value="Data Augmentation in Human-Centric Vision" />
-          <attvalue for="1" value="  This survey presents a comprehensive analysis of data augmentation techniques&#10;in human-centric vision tasks, a first of its kind in the field. It delves into&#10;a wide range of research areas including person ReID, human parsing, human pose&#10;estimation, and pedestrian detection, addressing the significant challenges&#10;posed by overfitting and limited training data in these domains. Our work&#10;categorizes data augmentation methods into two main types: data generation and&#10;data perturbation. Data generation covers techniques like graphic engine-based&#10;generation, generative model-based generation, and data recombination, while&#10;data perturbation is divided into image-level and human-level perturbations.&#10;Each method is tailored to the unique requirements of human-centric tasks, with&#10;some applicable across multiple areas. Our contributions include an extensive&#10;literature review, providing deep insights into the influence of these&#10;augmentation techniques in human-centric vision and highlighting the nuances of&#10;each method. We also discuss open issues and future directions, such as the&#10;integration of advanced generative models like Latent Diffusion Models, for&#10;creating more realistic and diverse training data. This survey not only&#10;encapsulates the current state of data augmentation in human-centric vision but&#10;also charts a course for future research, aiming to develop more robust,&#10;accurate, and efficient human-centric vision systems.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}. Despite significant advancements, these algorithms are inherently data-intensive and frequently encounter overfitting issues~\cite{ying2019overview,bejani2021systematic,bartlett2020benign,chen2020robust,zhang2021lbcf,hosseini2020tried,zhang2022towards}, where models excel with training data but falter on unseen test data. This challenge intensifies when access to large datasets is limited, often due to privacy concerns or the need for labor-intensive and costly human annotation tasks~\cite{ashktorab2021ai}.&#10;&#10;Addressing this issue, data augmentation emerges as a practical solution, particularly in the context of high collection and annotation costs. While previous surveys~\cite{shorten2019survey,kumar2023advanced} have explored data augmentation across various computer vision tasks, they often overlook the unique aspects of human-centric vision tasks. This survey seeks to fill this gap by extensively summarizing data augmentation works specific to human-centric vision tasks, such as person ReID, human parsing, human pose estimation, and pedestrian detection. These tasks, while distinct, share commonalities in augmentation methods that leverage human-specific features, with some techniques being applicable across multiple human-centric tasks.&#10;&#10;We aim to comprehensively survey data augmentation methods in human-centric vision, including several works previously discussed in surveys~\cite{shorten2019survey,kumar2023advanced}, while primarily focusing on methods not extensively covered before, especially those concerning human body augmentation. Data augmentation techniques in this domain can be broadly categorized into two types: data generation and data perturbation. Data generation involves creating or expanding datasets by adding new examples through various techniques, such as collecting additional samples, applying transformations, or introducing variations to bolster model training and generalization. This category encompasses graphic engine-based generation, generative model-based generation, and data recombination.&#10;&#10;Complementing data generation, data perturbation is frequently employed, subdivided into image-level and human-level perturbations. Image-level perturbation involves applying transformations to the entire image, such as rotations, flips, zooms, or adjustments in brightness and contrast, to artificially enhance the training dataset's diversity. On the other hand, human-level perturbation introduces alterations at the level of individual samples, aiming to increase a model's adaptability to varied instances by simulating the diversity encountered in real scenes.&#10;&#10;Furthermore, data augmentation methods can be differentiated based on their application in specific human-centric tasks, including person ReID, human parsing, human pose estimation, and pedestrian detection. Each of these tasks employs distinct augmentation methods that perform both data perturbation and data generation, which will be elaborated upon in this survey.&#10;&#10;Our contribution can be summarized as:&#10;\begin{itemize}&#10; \item We are the first to conduct a comprehensive survey of data augmentation methods tailored for human-centric vision tasks, highlighting the unique characteristics of these methods in relation to human-centric tasks.&#10; \item We provide a comprehensive literature review on data augmentation methods for human-centric vision tasks, summarizing and categorizing methods from various perspectives. This provides an in-depth understanding of crucial factors influencing augmentation techniques in human-centric vision.&#10; \item We present a thorough discussion about open issues and potential future directions based on our investigation. Our comprehensive studies uncover the pros/cons of current methods and bring new observations and insights to the community.&#10;\end{itemize}&#10;&#10;Outline.&#10;The remainder of this paper is organized as follows. Section~\ref{sec2} introduces the existing review papers classified by the type of augmentation method.&#10;Sections~\ref{sec3} describe augmentation methods categorized by the applied human-centric tasks.&#10;Finally, Section~\ref{sec4} concludes the paper and discusses several promising future research directions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Vision Research, Human-Centric Vision, Computer Science, Data Augmentation Techniques" />
-        </attvalues>
-      </node>
-      <node id="2001.04193" label="2001.04193">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning for Person Re-identification: A Survey and Outlook" />
-          <attvalue for="1" value="  Person re-identification (Re-ID) aims at retrieving a person of interest&#10;across multiple non-overlapping cameras. With the advancement of deep neural&#10;networks and increasing demand of intelligent video surveillance, it has gained&#10;significantly increased interest in the computer vision community. By&#10;dissecting the involved components in developing a person Re-ID system, we&#10;categorize it into the closed-world and open-world settings. The widely studied&#10;closed-world setting is usually applied under various research-oriented&#10;assumptions, and has achieved inspiring success using deep learning techniques&#10;on a number of datasets. We first conduct a comprehensive overview with&#10;in-depth analysis for closed-world person Re-ID from three different&#10;perspectives, including deep feature representation learning, deep metric&#10;learning and ranking optimization. With the performance saturation under&#10;closed-world setting, the research focus for person Re-ID has recently shifted&#10;to the open-world setting, facing more challenging issues. This setting is&#10;closer to practical applications under specific scenarios. We summarize the&#10;open-world Re-ID in terms of five different aspects. By analyzing the&#10;advantages of existing methods, we design a powerful AGW baseline, achieving&#10;state-of-the-art or at least comparable performance on twelve datasets for FOUR&#10;different Re-ID tasks. Meanwhile, we introduce a new evaluation metric (mINP)&#10;for person Re-ID, indicating the cost for finding all the correct matches,&#10;which provides an additional criteria to evaluate the Re-ID system for real&#10;applications. Finally, some important yet under-investigated open issues are&#10;discussed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.02984" label="1610.02984">
-        <attvalues>
-          <attvalue for="0" value="Person Re-identification: Past, Present and Future" />
-          <attvalue for="1" value="  Person re-identification (re-ID) has become increasingly popular in the&#10;community due to its application and research significance. It aims at spotting&#10;a person of interest in other cameras. In the early days, hand-crafted&#10;algorithms and small-scale evaluation were predominantly reported. Recent years&#10;have witnessed the emergence of large-scale datasets and deep learning systems&#10;which make use of large data volumes. Considering different tasks, we classify&#10;most current re-ID methods into two classes, i.e., image-based and video-based;&#10;in both tasks, hand-crafted and deep learning systems will be reviewed.&#10;Moreover, two new re-ID tasks which are much closer to real-world applications&#10;are described and discussed, i.e., end-to-end re-ID and fast re-ID in very&#10;large galleries. This paper: 1) introduces the history of person re-ID and its&#10;relationship with image classification and instance retrieval; 2) surveys a&#10;broad selection of the hand-crafted systems and the large-scale methods in both&#10;image- and video-based re-ID; 3) describes critical future directions in&#10;end-to-end re-ID and fast retrieval in large galleries; and 4) finally briefs&#10;some important yet under-developed issues.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.02531" label="1604.02531">
-        <attvalues>
-          <attvalue for="0" value="Person Re-identification in the Wild" />
-          <attvalue for="1" value="  We present a novel large-scale dataset and comprehensive baselines for&#10;end-to-end pedestrian detection and person recognition in raw video frames. Our&#10;baselines address three issues: the performance of various combinations of&#10;detectors and recognizers, mechanisms for pedestrian detection to help improve&#10;overall re-identification accuracy and assessing the effectiveness of different&#10;detectors for re-identification. We make three distinct contributions. First, a&#10;new dataset, PRW, is introduced to evaluate Person Re-identification in the&#10;Wild, using videos acquired through six synchronized cameras. It contains 932&#10;identities and 11,816 frames in which pedestrians are annotated with their&#10;bounding box positions and identities. Extensive benchmarking results are&#10;presented on this dataset. Second, we show that pedestrian detection aids&#10;re-identification through two simple yet effective improvements: a&#10;discriminatively trained ID-discriminative Embedding (IDE) in the person&#10;subspace using convolutional neural network (CNN) features and a Confidence&#10;Weighted Similarity (CWS) metric that incorporates detection scores into&#10;similarity measurement. Third, we derive insights in evaluating detector&#10;performance for the particular scenario of accurate person re-identification.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08184" label="1711.08184">
-        <attvalues>
-          <attvalue for="0" value="AlignedReID: Surpassing Human-Level Performance in Person&#10;  Re-Identification" />
-          <attvalue for="1" value="  In this paper, we propose a novel method called AlignedReID that extracts a&#10;global feature which is jointly learned with local features. Global feature&#10;learning benefits greatly from local feature learning, which performs an&#10;alignment/matching by calculating the shortest path between two sets of local&#10;features, without requiring extra supervision. After the joint learning, we&#10;only keep the global feature to compute the similarities between images. Our&#10;method achieves rank-1 accuracy of 94.4% on Market1501 and 97.8% on CUHK03,&#10;outperforming state-of-the-art methods by a large margin. We also evaluate&#10;human-level performance and demonstrate that our method is the first to surpass&#10;human-level performance on Market1501 and CUHK03, two widely used Person ReID&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.09777" label="1910.09777">
-        <attvalues>
-          <attvalue for="0" value="Self-Correction for Human Parsing" />
-          <attvalue for="1" value="  Labeling pixel-level masks for fine-grained semantic segmentation tasks, e.g.&#10;human parsing, remains a challenging task. The ambiguous boundary between&#10;different semantic parts and those categories with similar appearance usually&#10;are confusing, leading to unexpected noises in ground truth masks. To tackle&#10;the problem of learning with label noises, this work introduces a purification&#10;strategy, called Self-Correction for Human Parsing (SCHP), to progressively&#10;promote the reliability of the supervised labels as well as the learned models.&#10;In particular, starting from a model trained with inaccurate annotations as&#10;initialization, we design a cyclically learning scheduler to infer more&#10;reliable pseudo-masks by iteratively aggregating the current learned model with&#10;the former optimal one in an online manner. Besides, those correspondingly&#10;corrected labels can in turn to further boost the model performance. In this&#10;way, the models and the labels will reciprocally become more robust and&#10;accurate during the self-correction learning cycles. Benefiting from the&#10;superiority of SCHP, we achieve the best performance on two popular&#10;single-person human parsing benchmarks, including LIP and Pascal-Person-Part&#10;datasets. Our overall system ranks 1st in CVPR2019 LIP Challenge. Code is&#10;available at https://github.com/PeikeLi/Self-Correction-Human-Parsing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.00157" label="1808.00157">
-        <attvalues>
-          <attvalue for="0" value="Instance-level Human Parsing via Part Grouping Network" />
-          <attvalue for="1" value="  Instance-level human parsing towards real-world human analysis scenarios is&#10;still under-explored due to the absence of sufficient data resources and&#10;technical difficulty in parsing multiple instances in a single pass. Several&#10;related works all follow the &quot;parsing-by-detection&quot; pipeline that heavily&#10;relies on separately trained detection models to localize instances and then&#10;performs human parsing for each instance sequentially. Nonetheless, two&#10;discrepant optimization targets of detection and parsing lead to suboptimal&#10;representation learning and error accumulation for final results. In this work,&#10;we make the first attempt to explore a detection-free Part Grouping Network&#10;(PGN) for efficiently parsing multiple people in an image in a single pass. Our&#10;PGN reformulates instance-level human parsing as two twinned sub-tasks that can&#10;be jointly learned and mutually refined via a unified network: 1) semantic part&#10;segmentation for assigning each pixel as a human part (e.g., face, arms); 2)&#10;instance-aware edge detection to group semantic parts into distinct person&#10;instances. Thus the shared intermediate representation would be endowed with&#10;capabilities in both characterizing fine-grained parts and inferring instance&#10;belongings of each part. Finally, a simple instance partition process is&#10;employed to get final results during inference. We conducted experiments on&#10;PASCAL-Person-Part dataset and our PGN outperforms all state-of-the-art&#10;methods. Furthermore, we show its superiority on a newly collected multi-person&#10;parsing dataset (CIHP) including 38,280 diverse images, which is the largest&#10;dataset so far and can facilitate more advanced human analysis. The CIHP&#10;benchmark and our source code are available at http://sysu-hcp.net/lip/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.07206" label="1705.07206">
-        <attvalues>
-          <attvalue for="0" value="Multiple-Human Parsing in the Wild" />
-          <attvalue for="1" value="  Human parsing is attracting increasing research attention. In this work, we&#10;aim to push the frontier of human parsing by introducing the problem of&#10;multi-human parsing in the wild. Existing works on human parsing mainly tackle&#10;single-person scenarios, which deviates from real-world applications where&#10;multiple persons are present simultaneously with interaction and occlusion. To&#10;address the multi-human parsing problem, we introduce a new multi-human parsing&#10;(MHP) dataset and a novel multi-human parsing model named MH-Parser. The MHP&#10;dataset contains multiple persons captured in real-world scenes with&#10;pixel-level fine-grained semantic annotations in an instance-aware setting. The&#10;MH-Parser generates global parsing maps and person instance masks&#10;simultaneously in a bottom-up fashion with the help of a new Graph-GAN model.&#10;We envision that the MHP dataset will serve as a valuable data resource to&#10;develop new multi-human parsing models, and the MH-Parser offers a strong&#10;baseline to drive future research for multi-human parsing in the wild.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04536" label="1904.04536">
-        <attvalues>
-          <attvalue for="0" value="Graphonomy: Universal Human Parsing via Graph Transfer Learning" />
-          <attvalue for="1" value="  Prior highly-tuned human parsing models tend to fit towards each dataset in a&#10;specific domain or with discrepant label granularity, and can hardly be adapted&#10;to other human parsing tasks without extensive re-training. In this paper, we&#10;aim to learn a single universal human parsing model that can tackle all kinds&#10;of human parsing needs by unifying label annotations from different domains or&#10;at various levels of granularity. This poses many fundamental learning&#10;challenges, e.g. discovering underlying semantic structures among different&#10;label granularity, performing proper transfer learning across different image&#10;domains, and identifying and utilizing label redundancies across related tasks.&#10;  To address these challenges, we propose a new universal human parsing agent,&#10;named &quot;Graphonomy&quot;, which incorporates hierarchical graph transfer learning&#10;upon the conventional parsing network to encode the underlying label semantic&#10;structures and propagate relevant semantic information. In particular,&#10;Graphonomy first learns and propagates compact high-level graph representation&#10;among the labels within one dataset via Intra-Graph Reasoning, and then&#10;transfers semantic information across multiple datasets via Inter-Graph&#10;Transfer. Various graph transfer dependencies (\eg, similarity, linguistic&#10;knowledge) between different datasets are analyzed and encoded to enhance graph&#10;transfer capability. By distilling universal semantic graph representation to&#10;each specific task, Graphonomy is able to predict all levels of parsing labels&#10;in one system without piling up the complexity. Experimental results show&#10;Graphonomy effectively achieves the state-of-the-art results on three human&#10;parsing benchmarks as well as advantageous universal human parsing performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04845" label="2003.04845">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Human Parsing with Typed Part-Relation Reasoning" />
-          <attvalue for="1" value="  Human parsing is for pixel-wise human semantic understanding. As human bodies&#10;are underlying hierarchically structured, how to model human structures is the&#10;central theme in this task. Focusing on this, we seek to simultaneously exploit&#10;the representational capacity of deep graph networks and the hierarchical human&#10;structures. In particular, we provide following two contributions. First, three&#10;kinds of part relations, i.e., decomposition, composition, and dependency, are,&#10;for the first time, completely and precisely described by three distinct&#10;relation networks. This is in stark contrast to previous parsers, which only&#10;focus on a portion of the relations and adopt a type-agnostic relation modeling&#10;strategy. More expressive relation information can be captured by explicitly&#10;imposing the parameters in the relation networks to satisfy the specific&#10;characteristics of different relations. Second, previous parsers largely ignore&#10;the need for an approximation algorithm over the loopy human hierarchy, while&#10;we instead address an iterative reasoning process, by assimilating generic&#10;message-passing networks with their edge-typed, convolutional counterparts.&#10;With these efforts, our parser lays the foundation for more sophisticated and&#10;flexible human relation patterns of reasoning. Comprehensive experiments on&#10;five datasets demonstrate that our parser sets a new state-of-the-art on each.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.05446" label="1703.05446">
-        <attvalues>
-          <attvalue for="0" value="Look into Person: Self-supervised Structure-sensitive Learning and A New&#10;  Benchmark for Human Parsing" />
-          <attvalue for="1" value="  Human parsing has recently attracted a lot of research interests due to its&#10;huge application potentials. However existing datasets have limited number of&#10;images and annotations, and lack the variety of human appearances and the&#10;coverage of challenging cases in unconstrained environment. In this paper, we&#10;introduce a new benchmark &quot;Look into Person (LIP)&quot; that makes a significant&#10;advance in terms of scalability, diversity and difficulty, a contribution that&#10;we feel is crucial for future developments in human-centric analysis. This&#10;comprehensive dataset contains over 50,000 elaborately annotated images with 19&#10;semantic part labels, which are captured from a wider range of viewpoints,&#10;occlusions and background complexity. Given these rich annotations we perform&#10;detailed analyses of the leading human parsing approaches, gaining insights&#10;into the success and failures of these methods. Furthermore, in contrast to the&#10;existing efforts on improving the feature discriminative capability, we solve&#10;human parsing by exploring a novel self-supervised structure-sensitive learning&#10;approach, which imposes human pose structures into parsing results without&#10;resorting to extra supervision (i.e., no need for specifically labeling human&#10;joints in model training). Our self-supervised learning framework can be&#10;injected into any advanced neural networks to help incorporate rich high-level&#10;knowledge regarding human joints from a global perspective and improve the&#10;parsing results. Extensive evaluations on our LIP and the public&#10;PASCAL-Person-Part dataset demonstrate the superiority of our method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.13392" label="2012.13392">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning-Based Human Pose Estimation: A Survey" />
-          <attvalue for="1" value="  Human pose estimation aims to locate the human body parts and build human&#10;body representation (e.g., body skeleton) from input data such as images and&#10;videos. It has drawn increasing attention during the past decade and has been&#10;utilized in a wide range of applications including human-computer interaction,&#10;motion analysis, augmented reality, and virtual reality. Although the recently&#10;developed deep learning-based solutions have achieved high performance in human&#10;pose estimation, there still remain challenges due to insufficient training&#10;data, depth ambiguities, and occlusion. The goal of this survey paper is to&#10;provide a comprehensive review of recent deep learning-based solutions for both&#10;2D and 3D pose estimation via a systematic analysis and comparison of these&#10;solutions based on their input data and inference procedures. More than 250&#10;research papers since 2014 are covered in this survey. Furthermore, 2D and 3D&#10;human pose estimation datasets and evaluation metrics are included.&#10;Quantitative performance comparisons of the reviewed methods on popular&#10;datasets are summarized and discussed. Finally, the challenges involved,&#10;applications, and future research directions are concluded. A regularly updated&#10;project page is provided: \url{https://github.com/zczcwh/DL-HPE}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.10455" label="2103.10455">
-        <attvalues>
-          <attvalue for="0" value="3D Human Pose Estimation with Spatial and Temporal Transformers" />
-          <attvalue for="1" value="  Transformer architectures have become the model of choice in natural language&#10;processing and are now being introduced into computer vision tasks such as&#10;image classification, object detection, and semantic segmentation. However, in&#10;the field of human pose estimation, convolutional architectures still remain&#10;dominant. In this work, we present PoseFormer, a purely transformer-based&#10;approach for 3D human pose estimation in videos without convolutional&#10;architectures involved. Inspired by recent developments in vision transformers,&#10;we design a spatial-temporal transformer structure to comprehensively model the&#10;human joint relations within each frame as well as the temporal correlations&#10;across frames, then output an accurate 3D human pose of the center frame. We&#10;quantitatively and qualitatively evaluate our method on two popular and&#10;standard benchmark datasets: Human3.6M and MPI-INF-3DHP. Extensive experiments&#10;show that PoseFormer achieves state-of-the-art performance on both datasets.&#10;Code is available at \url{https://github.com/zczcwh/PoseFormer}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.02757" label="1705.02757">
-        <attvalues>
-          <attvalue for="0" value="What Can Help Pedestrian Detection?" />
-          <attvalue for="1" value="  Aggregating extra features has been considered as an effective approach to&#10;boost traditional pedestrian detection methods. However, there is still a lack&#10;of studies on whether and how CNN-based pedestrian detectors can benefit from&#10;these extra features. The first contribution of this paper is exploring this&#10;issue by aggregating extra features into CNN-based pedestrian detection&#10;framework. Through extensive experiments, we evaluate the effects of different&#10;kinds of extra features quantitatively. Moreover, we propose a novel network&#10;architecture, namely HyperLearner, to jointly learn pedestrian detection as&#10;well as the given extra feature. By multi-task training, HyperLearner is able&#10;to utilize the information of given features and improve detection performance&#10;without extra inputs in inference. The experimental results on multiple&#10;pedestrian benchmarks validate the effectiveness of the proposed HyperLearner.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11300" label="1906.11300">
-        <attvalues>
-          <attvalue for="0" value="Benign Overfitting in Linear Regression" />
-          <attvalue for="1" value="  The phenomenon of benign overfitting is one of the key mysteries uncovered by&#10;deep learning methodology: deep neural networks seem to predict well, even with&#10;a perfect fit to noisy training data. Motivated by this phenomenon, we consider&#10;when a perfect fit to training data in linear regression is compatible with&#10;accurate prediction. We give a characterization of linear regression problems&#10;for which the minimum norm interpolating prediction rule has near-optimal&#10;prediction accuracy. The characterization is in terms of two notions of the&#10;effective rank of the data covariance. It shows that overparameterization is&#10;essential for benign overfitting in this setting: the number of directions in&#10;parameter space that are unimportant for prediction must significantly exceed&#10;the sample size. By studying examples of data covariance properties that this&#10;characterization shows are required for benign overfitting, we find an&#10;important role for finite-dimensional data: the accuracy of the minimum norm&#10;interpolating prediction rule approaches the best possible accuracy for a much&#10;narrower range of properties of the data distribution when the data lies in an&#10;infinite dimensional space versus when the data lies in a finite dimensional&#10;space whose dimension grows faster than the sample size.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.06053" label="2209.06053">
-        <attvalues>
-          <attvalue for="0" value="Towards Understanding the Overfitting Phenomenon of Deep Click-Through&#10;  Rate Prediction Models" />
-          <attvalue for="1" value="  Deep learning techniques have been applied widely in industrial&#10;recommendation systems. However, far less attention has been paid to the&#10;overfitting problem of models in recommendation systems, which, on the&#10;contrary, is recognized as a critical issue for deep neural networks. In the&#10;context of Click-Through Rate (CTR) prediction, we observe an interesting&#10;one-epoch overfitting problem: the model performance exhibits a dramatic&#10;degradation at the beginning of the second epoch. Such a phenomenon has been&#10;witnessed widely in real-world applications of CTR models. Thereby, the best&#10;performance is usually achieved by training with only one epoch. To understand&#10;the underlying factors behind the one-epoch phenomenon, we conduct extensive&#10;experiments on the production data set collected from the display advertising&#10;system of Alibaba. The results show that the model structure, the optimization&#10;algorithm with a fast convergence rate, and the feature sparsity are closely&#10;related to the one-epoch phenomenon. We also provide a likely hypothesis for&#10;explaining such a phenomenon and conduct a set of proof-of-concept experiments.&#10;We hope this work can shed light on future research on training more epochs for&#10;better performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.09057" label="1912.09057">
-        <attvalues>
-          <attvalue for="0" value="PointVoteNet: Accurate Object Detection and 6 DoF Pose Estimation in&#10;  Point Clouds" />
-          <attvalue for="1" value="  We present a learning-based method for 6 DoF pose estimation of rigid objects&#10;in point cloud data. Many recent learning-based approaches use primarily RGB&#10;information for detecting objects, in some cases with an added refinement step&#10;using depth data. Our method consumes unordered point sets with/without RGB&#10;information, from initial detection to the final transformation estimation&#10;stage. This allows us to achieve accurate pose estimates, in some cases&#10;surpassing state of the art methods trained on the same data.&#10;" />
-          <attvalue for="2" value="&#10;Deep learning has reigned supreme for almost a decade now, with applications in a number of domains, perhaps most notably computer vision. A key challenge for visual processing systems is detection and accurate localization of rigid objects, allowing for a number of applications, including robotic manipulation. The most mature branch of deep learning is arguably CNN-based models, which are now applied in a variety of contexts. For object detection, CNNs are currently providing the best building blocks for learning-based detection systems in 2D, where the relevant objects are localized by a bounding box. More recently, full 6 DoF pose estimation systems have been demonstrated using similar techniques.&#10;&#10;Some methods rely on RGB inputs only to reconstruct the full 6 DoF object pose. In \cite{brachmann2016uncertainty} an uncertainty-aware regression method provides dense predictions of 3D object coordinates in the input image. Only keypoints are detected in \cite{rad2017bb8} and \cite{tekin2018real}, in the latter case using a YOLO-like \cite{redmon2016you} backbone. In \cite{xiang2017posecnn} a prediction is made for the object center, followed by a regression towards the rotation. The method in \cite{peng2019pvnet} uses a compromising approach where a semi-dense set of keypoints are predicted over the image.&#10;&#10;Methods relying on RGB-D data often augment an initial RGB-based detection with a refinement step using the depth information. \cite{brachmann2014learning} again casts dense pixel-wise votes for 3D object coordinates, but using RGB-D inputs. In \cite{kehl2017ssd}, an SSD-like detector \cite{liu2016ssd} provides initial detections, which are refined using depth information. Initial segmentations are used in \cite{wang2019densefusion} and finally the pose is refined using PointNet \cite{qi2017pointnet}.&#10;&#10;Some of these works use point cloud data, but not in the global detection phase, rather as a component during refinement. Our work, on the other hand, uses raw, unordered 3D point cloud data from end to end, i.e. both during initial detection of candidate regions of the scene and during actual transformation estimation and refinement. Our method relies on a PointNet backbone \cite{qi2017pointnet} to achieve this, and we show that in many of the tested circumstances, this significantly increases detection rates and localization accuracy during full 6 DoF pose estimation of objects.&#10;&#10;We describe here a method for training a joint classification and segmentation to directly propose detections and perform point-to-point correspondence assignments between an observed scene point cloud and a known 3D object model. An example of these segments of an object is shown in Fig.~\ref{fig:segments}. These correspondences are passed to a voting algorithm that performs density estimates directly in the 6-dimensional pose parameter space. We also show how the addition of color coordinates to the input point cloud dramatically increases the accuracy of our method.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Learning-Based Methods, Computer Science, Object Pose Estimation, Point Cloud Analysis, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1711.00199" label="1711.00199">
-        <attvalues>
-          <attvalue for="0" value="PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in&#10;  Cluttered Scenes" />
-          <attvalue for="1" value="  Estimating the 6D pose of known objects is important for robots to interact&#10;with the real world. The problem is challenging due to the variety of objects&#10;as well as the complexity of a scene caused by clutter and occlusions between&#10;objects. In this work, we introduce PoseCNN, a new Convolutional Neural Network&#10;for 6D object pose estimation. PoseCNN estimates the 3D translation of an&#10;object by localizing its center in the image and predicting its distance from&#10;the camera. The 3D rotation of the object is estimated by regressing to a&#10;quaternion representation. We also introduce a novel loss function that enables&#10;PoseCNN to handle symmetric objects. In addition, we contribute a large scale&#10;video dataset for 6D object pose estimation named the YCB-Video dataset. Our&#10;dataset provides accurate 6D poses of 21 objects from the YCB dataset observed&#10;in 92 videos with 133,827 frames. We conduct extensive experiments on our&#10;YCB-Video dataset and the OccludedLINEMOD dataset to show that PoseCNN is&#10;highly robust to occlusions, can handle symmetric objects, and provide accurate&#10;pose estimation using only color images as input. When using depth data to&#10;further refine the poses, our approach achieves state-of-the-art results on the&#10;challenging OccludedLINEMOD dataset. Our code and dataset are available at&#10;https://rse-lab.cs.washington.edu/projects/posecnn/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.04780" label="1901.04780">
-        <attvalues>
-          <attvalue for="0" value="DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion" />
-          <attvalue for="1" value="  A key technical challenge in performing 6D object pose estimation from RGB-D&#10;image is to fully leverage the two complementary data sources. Prior works&#10;either extract information from the RGB image and depth separately or use&#10;costly post-processing steps, limiting their performances in highly cluttered&#10;scenes and real-time applications. In this work, we present DenseFusion, a&#10;generic framework for estimating 6D pose of a set of known objects from RGB-D&#10;images. DenseFusion is a heterogeneous architecture that processes the two data&#10;sources individually and uses a novel dense fusion network to extract&#10;pixel-wise dense feature embedding, from which the pose is estimated.&#10;Furthermore, we integrate an end-to-end iterative pose refinement procedure&#10;that further improves the pose estimation while achieving near real-time&#10;inference. Our experiments show that our method outperforms state-of-the-art&#10;approaches in two datasets, YCB-Video and LineMOD. We also deploy our proposed&#10;method to a real robot to grasp and manipulate objects based on the estimated&#10;pose.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05582" label="1904.05582">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Space-time Graph Neural Networks" />
-          <attvalue for="1" value="  Learning in the space-time domain remains a very challenging problem in&#10;machine learning and computer vision. Current computational models for&#10;understanding spatio-temporal visual data are heavily rooted in the classical&#10;single-image based paradigm. It is not yet well understood how to integrate&#10;information in space and time into a single, general model. We propose a neural&#10;graph model, recurrent in space and time, suitable for capturing both the local&#10;appearance and the complex higher-level interactions of different entities and&#10;objects within the changing world scene. Nodes and edges in our graph have&#10;dedicated neural networks for processing information. Nodes operate over&#10;features extracted from local parts in space and time and previous memory&#10;states. Edges process messages between connected nodes at different locations&#10;and spatial scales or between past and present time. Messages are passed&#10;iteratively in order to transmit information globally and establish long range&#10;interactions. Our model is general and could learn to recognize a variety of&#10;high level spatio-temporal concepts and be applied to different learning tasks.&#10;We demonstrate, through extensive experiments and ablation studies, that our&#10;model outperforms strong baselines and top published methods on recognizing&#10;complex activities in video. Moreover, we obtain state-of-the-art performance&#10;on the challenging Something-Something human-object interaction dataset.&#10;" />
-          <attvalue for="2" value="&#10;Video data is available almost everywhere. While image level recognition is better understood, visual learning in space and time is far from being solved. The main challenge is how to model interactions between objects and higher level concepts, within the large spatio-temporal context. For such a difficult learning task it is important to efficiently model the local appearance, the spatial relationships and the complex interactions and changes that take place over time.&#10;&#10;Often, for different learning tasks, different models are preferred, such that they capture the specific domain priors and biases of the problem~\cite{battaglia2018relational}. Convolutional neural networks (CNNs) are preferred on tasks involving strong local and stationary assumptions about the data. Recurrent models are chosen when data is sequential in nature. Fully connected models could be preferred when there is no known structure in the data. Our recurrent neural graph efficiently processes information in both space and time and can be applied to different learning tasks in video.&#10;&#10;We propose Recurrent Space-time Graph (RSTG) neural networks, in which each node receives features extracted from a specific region in space-time using a backbone deep neural network. Global processing is achieved through iterative message passing in space and time. Spatio-temporal processing is factorized, into a space processing stage and a time processing stage, which are alternated within each iteration. We aim to decouple, conceptually, the data from the computational machine that processes the data. Thus, our nodes are processing units that receive inputs from several sources: local regions in space at the present time, their neighbor spatial nodes as well as their past memory states (Fig.~\ref{fig:main_graph_figure}). &#10;&#10;Main contributions.&#10;We sum up our contributions into the following three main ideas:&#10;&#10;\begin{enumerate}&#10; \item We propose a novel computational model for learning in spatio-temporal domain. Space and time are treated differently, while they function together in complementary ways. Our model is general and could be applied to various learning problems. It could also be used as a processing block in combination with other powerful models.&#10;&#10;\item We factorize space and time and process them differently within a unified neural graph model from an unstructured video. In extensive ablation studies we show the importance of each graph component and also demonstrate that different temporal and spatial processing is crucial for learning in space-time domain. Through recurrent and factorized space-time processing&#10; our model achieves a relatively low computational complexity. &#10;&#10;\item We introduce a new synthetic dataset, with complex interactions, to analyse and evaluate different spatio-temporal models. We obtain a performance that is superior to several powerful baselines and top published methods. More importantly, we obtain state-of-the-art results on the challenging Something-Something, real world dataset.&#10;&#10;\end{enumerate}&#10;&#10;Relation to previous work:&#10;Iterative graph based methods have a long history in machine learning&#10;and are currently enjoying a fast-growing interest~\cite{battaglia2018relational, pmlr-v70-gilmer17a}. Their main paradigm is the following: at each iteration, messages are passed between nodes, information is updated at each node and the process continues until convergence or a stopping criterion is met. Such ideas trace back to work on image denoising, restoration and labeling~\cite{besag1986statistical,hummel1983foundations,geman1984stochastic,geman1986markov}, with many inference methods, graphical models and mathematical formulations being proposed over time for various tasks~\cite{lafferty2001conditional,kumar2006discriminative,pearl2014probabilistic,ravikumar2006quadratic,schaeffer2007graph,leordeanu2012unsupervised,ng2002spectral}.&#10;&#10;Current approaches combine the idea of message passing between graph nodes, from graphical models, with convolution operations. Thus, the idea of graph convolutions was born. Initial methods generalizing conv nets to the case of graph structured data~\cite{DBLP:journals/corr/BrunaZSL13_bruna_lecun_spectral,henaff2015deep,defferrard2016convolutional} learn in the spectral domain of the graph. They are approximated~\cite{kipf2017semi} by message passing based on linear operations~\cite{duvenaud2015convolutional} or MLPs~\cite{battaglia2016interaction}. Aggregation of messages needs permutation invariant operators such as max or sum, the last one being proved superior in \cite{xu2018how_powerful}, with attention mechanism~\cite{velickovic2018graph_gat} as an alternative.&#10;&#10;Recurrence in graph models has been proposed for sequential tasks~\cite{li2016gated, jain2016structural} or for iteratively processing the input~\cite{dehghani2018universal_transformer, NIPS2018_7960_transformer}. Recurrence is used in graph neural nets~\cite{li2016gated} to tackle symbolic tasks with single input and sequential language output. Different from them, we have two types of recurrent stages, with distinct functionality, one over space and the other over time.&#10;&#10;The idea of modeling complex, higher order and long range spatial relationships by the spatial recurrence relates to more classical work using pictorial structures~\cite{felzenszwalb2005pictorial} to model object parts and their relationships and perform inference through iterative optimization algorithms. The idea of combining information at different scales also relates to classic approaches in object recognition, such as the well-known spatial pyramid model~\cite{lazebnik2006beyond, he2015spatial_pyramid}.&#10;&#10;Long-range dependencies in sequential language are captured in~\cite{vaswani2017attention} with a self-attention model. It has a stack of attention layers, each with different parameters. It is improved in~\cite{dehghani2018universal_transformer} by performing operations recurrently. This is similar to our recurrent spatial processing stage. As mentioned before, our model is different by adding another complementary dimension - the temporal one.&#10;In \cite{NIPS2018_7960_transformer} new information is incorporated into the existing memory by self-attention using a temporary new node. Then each node is updated by an LSTM~\cite{hochreiter1997long_lstm}. Their method is applied on program evaluation, simulated environments used in reinforcement learning and language modeling where they do not have a spatial dimension. Their nodes act as a set of memories. Different from them, we receive new information for each node and process them in multiple interleaved iterations of our two stages.&#10;&#10;Initial node information could come from each local spatio-temporal point in convolutional feature maps~\cite{NIPS2017_7082_ralational_cuburi, wang2018non_local} or from features corresponding to entities detected by external methods, such as objects~\cite{wang2018videos_gupta2, Ghosh2018StackedSG_B, tsai2019video_C} or skeletons~\cite{yan2018spatial_A}. Also, the approach in \cite{Baradel_2018_ECCV_orn} is to extract objects and form relations between objects from pairs of time steps randomly chosen. Different from that methods, our nodes are not attached to specific volumes in time and space. Also, we do not need pre-trained higher-level detectors, our model working on unstructured videos.&#10;&#10;While the above methods need access to the whole video at test time, ours is recurrent and can function in an online, continuous manner in time. &#10; All space-time positions in the input volume are connected in~\cite{wang2018non_local, wang2018videos_gupta2,chen20182_a2nets}. In contrast, we treat space and time differently and prove the effectiveness of our choice in experiments. &#10; A 1D convolution is used in \cite{yan2018spatial_A} to temporally connect only the nodes corresponding to the same skeleton joint and recently~\cite{Ghosh2018StackedSG_B} send messages between nodes corresponding to the same entities, while we recurrently update in time the state of each node. We could see our different handling of time and space as an efficient factorization into simpler mechanisms that function together along different dimensions. The work in \cite{szegedy2016incv2, chollet2017xception} confirm our hypothesis that features could be more efficiently processed by factorization into simpler operations. The models in~\cite{sun2015human_factorised, xie2018rethinking, tran2018closer} factorize 3D convolutions into 2D spatial and 1D temporal convolutions.&#10;&#10;For spatio-temporal processing, some methods, which do not use explicit graph modeling, encode frames individually using 2D convolutions and aggregate them in different ways~\cite{karpathy2014large, yue2015beyond, donahue2015long}; others form relations as functions (MLPs) over sets of frames~\cite{zhou2018temporal_trn_torralba} or use 3D convolution inflated from existing 2D convolutional networks~\cite{carreira2017quo}.&#10;Optical flow could be used as input to a separate branch of a 2D ConvNet \cite{simonyan2014two_stream} or used as part of the model to guide the kernel of 3D convolutions~\cite{NIPS2018_7489_trajectory}. To cover both spatial and temporal dimensions simultaneously, Convolutional LSTM~\cite{Shi2015ConvLSTM} can be used, augmented with additional memory~\cite{Wang2017ST-LSTM} or self-attention in order to update LSTM hidden states~\cite{wang2018eidetic}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Spatio-Temporal Modeling, Artificial Intelligence, Cognitive Science, Computer Science, Neural Network Architecture" />
-        </attvalues>
-      </node>
-      <node id="1806.01261" label="1806.01261">
-        <attvalues>
-          <attvalue for="0" value="Relational inductive biases, deep learning, and graph networks" />
-          <attvalue for="1" value="  Artificial intelligence (AI) has undergone a renaissance recently, making&#10;major progress in key domains such as vision, language, control, and&#10;decision-making. This has been due, in part, to cheap data and cheap compute&#10;resources, which have fit the natural strengths of deep learning. However, many&#10;defining characteristics of human intelligence, which developed under much&#10;different pressures, remain out of reach for current approaches. In particular,&#10;generalizing beyond one's experiences--a hallmark of human intelligence from&#10;infancy--remains a formidable challenge for modern AI.&#10;  The following is part position paper, part review, and part unification. We&#10;argue that combinatorial generalization must be a top priority for AI to&#10;achieve human-like abilities, and that structured representations and&#10;computations are key to realizing this objective. Just as biology uses nature&#10;and nurture cooperatively, we reject the false choice between&#10;&quot;hand-engineering&quot; and &quot;end-to-end&quot; learning, and instead advocate for an&#10;approach which benefits from their complementary strengths. We explore how&#10;using relational inductive biases within deep learning architectures can&#10;facilitate learning about entities, relations, and rules for composing them. We&#10;present a new building block for the AI toolkit with a strong relational&#10;inductive bias--the graph network--which generalizes and extends various&#10;approaches for neural networks that operate on graphs, and provides a&#10;straightforward interface for manipulating structured knowledge and producing&#10;structured behaviors. We discuss how graph networks can support relational&#10;reasoning and combinatorial generalization, laying the foundation for more&#10;sophisticated, interpretable, and flexible patterns of reasoning. As a&#10;companion to this paper, we have released an open-source software library for&#10;building graph networks, with demonstrations of how to use them in practice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1312.6203" label="1312.6203">
-        <attvalues>
-          <attvalue for="0" value="Spectral Networks and Locally Connected Networks on Graphs" />
-          <attvalue for="1" value="  Convolutional Neural Networks are extremely efficient architectures in image&#10;and audio recognition tasks, thanks to their ability to exploit the local&#10;translational invariance of signal classes over their domain. In this paper we&#10;consider possible generalizations of CNNs to signals defined on more general&#10;domains without the action of a translation group. In particular, we propose&#10;two constructions, one based upon a hierarchical clustering of the domain, and&#10;another based on the spectrum of the graph Laplacian. We show through&#10;experiments that for low-dimensional graphs it is possible to learn&#10;convolutional layers with a number of parameters independent of the input size,&#10;resulting in efficient deep architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.05163" label="1506.05163">
-        <attvalues>
-          <attvalue for="0" value="Deep Convolutional Networks on Graph-Structured Data" />
-          <attvalue for="1" value="  Deep Learning's recent successes have mostly relied on Convolutional&#10;Networks, which exploit fundamental statistical properties of images, sounds&#10;and video data: the local stationarity and multi-scale compositional structure,&#10;that allows expressing long range interactions in terms of shorter, localized&#10;interactions. However, there exist other important examples, such as text&#10;documents or bioinformatic data, that may lack some or all of these strong&#10;statistical regularities.&#10;  In this paper we consider the general question of how to construct deep&#10;architectures with small learning complexity on general non-Euclidean domains,&#10;which are typically unknown and need to be estimated from the data. In&#10;particular, we develop an extension of Spectral Networks which incorporates a&#10;Graph Estimation procedure, that we test on large-scale classification&#10;problems, matching or improving over Dropout Networks with far less parameters&#10;to estimate.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.09375" label="1606.09375">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Neural Networks on Graphs with Fast Localized Spectral&#10;  Filtering" />
-          <attvalue for="1" value="  In this work, we are interested in generalizing convolutional neural networks&#10;(CNNs) from low-dimensional regular grids, where image, video and speech are&#10;represented, to high-dimensional irregular domains, such as social networks,&#10;brain connectomes or words' embedding, represented by graphs. We present a&#10;formulation of CNNs in the context of spectral graph theory, which provides the&#10;necessary mathematical background and efficient numerical schemes to design&#10;fast localized convolutional filters on graphs. Importantly, the proposed&#10;technique offers the same linear computational complexity and constant learning&#10;complexity as classical CNNs, while being universal to any graph structure.&#10;Experiments on MNIST and 20NEWS demonstrate the ability of this novel deep&#10;learning system to learn local, stationary, and compositional features on&#10;graphs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.02907" label="1609.02907">
-        <attvalues>
-          <attvalue for="0" value="Semi-Supervised Classification with Graph Convolutional Networks" />
-          <attvalue for="1" value="  We present a scalable approach for semi-supervised learning on&#10;graph-structured data that is based on an efficient variant of convolutional&#10;neural networks which operate directly on graphs. We motivate the choice of our&#10;convolutional architecture via a localized first-order approximation of&#10;spectral graph convolutions. Our model scales linearly in the number of graph&#10;edges and learns hidden layer representations that encode both local graph&#10;structure and features of nodes. In a number of experiments on citation&#10;networks and on a knowledge graph dataset we demonstrate that our approach&#10;outperforms related methods by a significant margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1509.09292" label="1509.09292">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Networks on Graphs for Learning Molecular Fingerprints" />
-          <attvalue for="1" value="  We introduce a convolutional neural network that operates directly on graphs.&#10;These networks allow end-to-end learning of prediction pipelines whose inputs&#10;are graphs of arbitrary size and shape. The architecture we present generalizes&#10;standard molecular feature extraction methods based on circular fingerprints.&#10;We show that these data-driven features are more interpretable, and have better&#10;predictive performance on a variety of tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.00222" label="1612.00222">
-        <attvalues>
-          <attvalue for="0" value="Interaction Networks for Learning about Objects, Relations and Physics" />
-          <attvalue for="1" value="  Reasoning about objects, relations, and physics is central to human&#10;intelligence, and a key goal of artificial intelligence. Here we introduce the&#10;interaction network, a model which can reason about how objects in complex&#10;systems interact, supporting dynamical predictions, as well as inferences about&#10;the abstract properties of the system. Our model takes graphs as input,&#10;performs object- and relation-centric reasoning in a way that is analogous to a&#10;simulation, and is implemented using deep neural networks. We evaluate its&#10;ability to reason about several challenging physical domains: n-body problems,&#10;rigid-body collision, and non-rigid dynamics. Our results show it can be&#10;trained to accurately simulate the physical trajectories of dozens of objects&#10;over thousands of time steps, estimate abstract quantities such as energy, and&#10;generalize automatically to systems with different numbers and configurations&#10;of objects and relations. Our interaction network implementation is the first&#10;general-purpose, learnable physics engine, and a powerful general framework for&#10;reasoning about object and relations in a wide variety of complex real-world&#10;domains.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.00826" label="1810.00826">
-        <attvalues>
-          <attvalue for="0" value="How Powerful are Graph Neural Networks?" />
-          <attvalue for="1" value="  Graph Neural Networks (GNNs) are an effective framework for representation&#10;learning of graphs. GNNs follow a neighborhood aggregation scheme, where the&#10;representation vector of a node is computed by recursively aggregating and&#10;transforming representation vectors of its neighboring nodes. Many GNN variants&#10;have been proposed and have achieved state-of-the-art results on both node and&#10;graph classification tasks. However, despite GNNs revolutionizing graph&#10;representation learning, there is limited understanding of their&#10;representational properties and limitations. Here, we present a theoretical&#10;framework for analyzing the expressive power of GNNs to capture different graph&#10;structures. Our results characterize the discriminative power of popular GNN&#10;variants, such as Graph Convolutional Networks and GraphSAGE, and show that&#10;they cannot learn to distinguish certain simple graph structures. We then&#10;develop a simple architecture that is provably the most expressive among the&#10;class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism&#10;test. We empirically validate our theoretical findings on a number of graph&#10;classification benchmarks, and demonstrate that our model achieves&#10;state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.10903" label="1710.10903">
-        <attvalues>
-          <attvalue for="0" value="Graph Attention Networks" />
-          <attvalue for="1" value="  We present graph attention networks (GATs), novel neural network&#10;architectures that operate on graph-structured data, leveraging masked&#10;self-attentional layers to address the shortcomings of prior methods based on&#10;graph convolutions or their approximations. By stacking layers in which nodes&#10;are able to attend over their neighborhoods' features, we enable (implicitly)&#10;specifying different weights to different nodes in a neighborhood, without&#10;requiring any kind of costly matrix operation (such as inversion) or depending&#10;on knowing the graph structure upfront. In this way, we address several key&#10;challenges of spectral-based graph neural networks simultaneously, and make our&#10;model readily applicable to inductive as well as transductive problems. Our GAT&#10;models have achieved or matched state-of-the-art results across four&#10;established transductive and inductive graph benchmarks: the Cora, Citeseer and&#10;Pubmed citation network datasets, as well as a protein-protein interaction&#10;dataset (wherein test graphs remain unseen during training).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.05493" label="1511.05493">
-        <attvalues>
-          <attvalue for="0" value="Gated Graph Sequence Neural Networks" />
-          <attvalue for="1" value="  Graph-structured data appears frequently in domains including chemistry,&#10;natural language semantics, social networks, and knowledge bases. In this work,&#10;we study feature learning techniques for graph-structured inputs. Our starting&#10;point is previous work on Graph Neural Networks (Scarselli et al., 2009), which&#10;we modify to use gated recurrent units and modern optimization techniques and&#10;then extend to output sequences. The result is a flexible and broadly useful&#10;class of neural network models that has favorable inductive biases relative to&#10;purely sequence-based models (e.g., LSTMs) when the problem is&#10;graph-structured. We demonstrate the capabilities on some simple AI (bAbI) and&#10;graph algorithm learning tasks. We then show it achieves state-of-the-art&#10;performance on a problem from program verification, in which subgraphs need to&#10;be matched to abstract data structures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.03819" label="1807.03819">
-        <attvalues>
-          <attvalue for="0" value="Universal Transformers" />
-          <attvalue for="1" value="  Recurrent neural networks (RNNs) sequentially process data by updating their&#10;state with each new data point, and have long been the de facto choice for&#10;sequence modeling tasks. However, their inherently sequential computation makes&#10;them slow to train. Feed-forward and convolutional architectures have recently&#10;been shown to achieve superior results on some sequence modeling tasks such as&#10;machine translation, with the added advantage that they concurrently process&#10;all inputs in the sequence, leading to easy parallelization and faster training&#10;times. Despite these successes, however, popular feed-forward sequence models&#10;like the Transformer fail to generalize in many simple tasks that recurrent&#10;models handle with ease, e.g. copying strings or even simple logical inference&#10;when the string or formula lengths exceed those observed at training time. We&#10;propose the Universal Transformer (UT), a parallel-in-time self-attentive&#10;recurrent sequence model which can be cast as a generalization of the&#10;Transformer model and which addresses these issues. UTs combine the&#10;parallelizability and global receptive field of feed-forward sequence models&#10;like the Transformer with the recurrent inductive bias of RNNs. We also add a&#10;dynamic per-position halting mechanism and find that it improves accuracy on&#10;several tasks. In contrast to the standard Transformer, under certain&#10;assumptions, UTs can be shown to be Turing-complete. Our experiments show that&#10;UTs outperform standard Transformers on a wide range of algorithmic and&#10;language understanding tasks, including the challenging LAMBADA language&#10;modeling task where UTs achieve a new state of the art, and machine translation&#10;where UTs achieve a 0.9 BLEU improvement over Transformers on the WMT14 En-De&#10;dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01822" label="1806.01822">
-        <attvalues>
-          <attvalue for="0" value="Relational recurrent neural networks" />
-          <attvalue for="1" value="  Memory-based neural networks model temporal data by leveraging an ability to&#10;remember information for long periods. It is unclear, however, whether they&#10;also have an ability to perform complex relational reasoning with the&#10;information they remember. Here, we first confirm our intuitions that standard&#10;memory architectures may struggle at tasks that heavily involve an&#10;understanding of the ways in which entities are connected -- i.e., tasks&#10;involving relational reasoning. We then improve upon these deficits by using a&#10;new memory module -- a \textit{Relational Memory Core} (RMC) -- which employs&#10;multi-head dot product attention to allow memories to interact. Finally, we&#10;test the RMC on a suite of tasks that may profit from more capable relational&#10;reasoning across sequential information, and show large gains in RL domains&#10;(e.g. Mini PacMan), program evaluation, and language modeling, achieving&#10;state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.01427" label="1706.01427">
-        <attvalues>
-          <attvalue for="0" value="A simple neural network module for relational reasoning" />
-          <attvalue for="1" value="  Relational reasoning is a central component of generally intelligent&#10;behavior, but has proven difficult for neural networks to learn. In this paper&#10;we describe how to use Relation Networks (RNs) as a simple plug-and-play module&#10;to solve problems that fundamentally hinge on relational reasoning. We tested&#10;RN-augmented networks on three tasks: visual question answering using a&#10;challenging dataset called CLEVR, on which we achieve state-of-the-art,&#10;super-human performance; text-based question answering using the bAbI suite of&#10;tasks; and complex reasoning about dynamic physical systems. Then, using a&#10;curated dataset called Sort-of-CLEVR we show that powerful convolutional&#10;networks do not have a general capacity to solve relational questions, but can&#10;gain this capacity when augmented with RNs. Our work shows how a deep learning&#10;architecture equipped with an RN module can implicitly discover and learn to&#10;reason about entities and their relations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01810" label="1806.01810">
-        <attvalues>
-          <attvalue for="0" value="Videos as Space-Time Region Graphs" />
-          <attvalue for="1" value="  How do humans recognize the action &quot;opening a book&quot; ? We argue that there are&#10;two important cues: modeling temporal shape dynamics and modeling functional&#10;relationships between humans and objects. In this paper, we propose to&#10;represent videos as space-time region graphs which capture these two important&#10;cues. Our graph nodes are defined by the object region proposals from different&#10;frames in a long range video. These nodes are connected by two types of&#10;relations: (i) similarity relations capturing the long range dependencies&#10;between correlated objects and (ii) spatial-temporal relations capturing the&#10;interactions between nearby objects. We perform reasoning on this graph&#10;representation via Graph Convolutional Networks. We achieve state-of-the-art&#10;results on both Charades and Something-Something datasets. Especially for&#10;Charades, we obtain a huge 4.4% gain when our model is applied in complex&#10;environments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.07455" label="1801.07455">
-        <attvalues>
-          <attvalue for="0" value="Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action&#10;  Recognition" />
-          <attvalue for="1" value="  Dynamics of human body skeletons convey significant information for human&#10;action recognition. Conventional approaches for modeling skeletons usually rely&#10;on hand-crafted parts or traversal rules, thus resulting in limited expressive&#10;power and difficulties of generalization. In this work, we propose a novel&#10;model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks&#10;(ST-GCN), which moves beyond the limitations of previous methods by&#10;automatically learning both the spatial and temporal patterns from data. This&#10;formulation not only leads to greater expressive power but also stronger&#10;generalization capability. On two large datasets, Kinetics and NTU-RGBD, it&#10;achieves substantial improvements over mainstream methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.06157" label="1806.06157">
-        <attvalues>
-          <attvalue for="0" value="Object Level Visual Reasoning in Videos" />
-          <attvalue for="1" value="  Human activity recognition is typically addressed by detecting key concepts&#10;like global and local motion, features related to object classes present in the&#10;scene, as well as features related to the global context. The next open&#10;challenges in activity recognition require a level of understanding that pushes&#10;beyond this and call for models with capabilities for fine distinction and&#10;detailed comprehension of interactions between actors and objects in a scene.&#10;We propose a model capable of learning to reason about semantically meaningful&#10;spatiotemporal interactions in videos. The key to our approach is a choice of&#10;performing this reasoning at the object level through the integration of state&#10;of the art object detection networks. This allows the model to learn detailed&#10;spatial interactions that exist at a semantic, object-interaction relevant&#10;level. We evaluate our method on three standard datasets (Twenty-BN&#10;Something-Something, VLOG and EPIC Kitchens) and achieve state of the art&#10;results on all of them. Finally, we show visualizations of the interactions&#10;learned by the model, which illustrate object classes and their interactions&#10;corresponding to different activity classes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.10575" label="1811.10575">
-        <attvalues>
-          <attvalue for="0" value="Stacked Spatio-Temporal Graph Convolutional Networks for Action&#10;  Segmentation" />
-          <attvalue for="1" value="  We propose novel Stacked Spatio-Temporal Graph Convolutional Networks&#10;(Stacked-STGCN) for action segmentation, i.e., predicting and localizing a&#10;sequence of actions over long videos. We extend the Spatio-Temporal Graph&#10;Convolutional Network (STGCN) originally proposed for skeleton-based action&#10;recognition to enable nodes with different characteristics (e.g., scene, actor,&#10;object, action, etc.), feature descriptors with varied lengths, and arbitrary&#10;temporal edge connections to account for large graph deformation commonly&#10;associated with complex activities. We further introduce the stacked hourglass&#10;architecture to STGCN to leverage the advantages of an encoder-decoder design&#10;for improved generalization performance and localization accuracy. We explore&#10;various descriptors such as frame-level VGG, segment-level I3D, RCNN-based&#10;object, etc. as node descriptors to enable action segmentation based on joint&#10;inference over comprehensive contextual information. We show results on CAD120&#10;(which provides pre-computed node features and edge weights for fair&#10;performance comparison across algorithms) as well as a more complex real-world&#10;activity dataset, Charades. Our Stacked-STGCN in general achieves 4.0%&#10;performance improvement over the best reported results in F1 score on CAD120&#10;and 1.3% in mAP on Charades using VGG features.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.00567" label="1512.00567">
-        <attvalues>
-          <attvalue for="0" value="Rethinking the Inception Architecture for Computer Vision" />
-          <attvalue for="1" value="  Convolutional networks are at the core of most state-of-the-art computer&#10;vision solutions for a wide variety of tasks. Since 2014 very deep&#10;convolutional networks started to become mainstream, yielding substantial gains&#10;in various benchmarks. Although increased model size and computational cost&#10;tend to translate to immediate quality gains for most tasks (as long as enough&#10;labeled data is provided for training), computational efficiency and low&#10;parameter count are still enabling factors for various use cases such as mobile&#10;vision and big-data scenarios. Here we explore ways to scale up networks in&#10;ways that aim at utilizing the added computation as efficiently as possible by&#10;suitably factorized convolutions and aggressive regularization. We benchmark&#10;our methods on the ILSVRC 2012 classification challenge validation set&#10;demonstrate substantial gains over the state of the art: 21.2% top-1 and 5.6%&#10;top-5 error for single frame evaluation using a network with a computational&#10;cost of 5 billion multiply-adds per inference and with using less than 25&#10;million parameters. With an ensemble of 4 models and multi-crop evaluation, we&#10;report 3.5% top-5 error on the validation set (3.6% error on the test set) and&#10;17.3% top-1 error on the validation set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1510.00562" label="1510.00562">
-        <attvalues>
-          <attvalue for="0" value="Human Action Recognition using Factorized Spatio-Temporal Convolutional&#10;  Networks" />
-          <attvalue for="1" value="  Human actions in video sequences are three-dimensional (3D) spatio-temporal&#10;signals characterizing both the visual appearance and motion dynamics of the&#10;involved humans and objects. Inspired by the success of convolutional neural&#10;networks (CNN) for image classification, recent attempts have been made to&#10;learn 3D CNNs for recognizing human actions in videos. However, partly due to&#10;the high complexity of training 3D convolution kernels and the need for large&#10;quantities of training videos, only limited success has been reported. This has&#10;triggered us to investigate in this paper a new deep architecture which can&#10;handle 3D signals more effectively. Specifically, we propose factorized&#10;spatio-temporal convolutional networks (FstCN) that factorize the original 3D&#10;convolution kernel learning as a sequential process of learning 2D spatial&#10;kernels in the lower layers (called spatial convolutional layers), followed by&#10;learning 1D temporal kernels in the upper layers (called temporal convolutional&#10;layers). We introduce a novel transformation and permutation operator to make&#10;factorization in FstCN possible. Moreover, to address the issue of sequence&#10;alignment, we propose an effective training and inference strategy based on&#10;sampling multiple video clips from a given action video sequence. We have&#10;tested FstCN on two commonly used benchmark datasets (UCF-101 and HMDB-51).&#10;Without using auxiliary training videos to boost the performance, FstCN&#10;outperforms existing CNN based methods and achieves comparable performance with&#10;a recent method that benefits from using auxiliary training videos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08496" label="1711.08496">
-        <attvalues>
-          <attvalue for="0" value="Temporal Relational Reasoning in Videos" />
-          <attvalue for="1" value="  Temporal relational reasoning, the ability to link meaningful transformations&#10;of objects or entities over time, is a fundamental property of intelligent&#10;species. In this paper, we introduce an effective and interpretable network&#10;module, the Temporal Relation Network (TRN), designed to learn and reason about&#10;temporal dependencies between video frames at multiple time scales. We evaluate&#10;TRN-equipped networks on activity recognition tasks using three recent video&#10;datasets - Something-Something, Jester, and Charades - which fundamentally&#10;depend on temporal relational reasoning. Our results demonstrate that the&#10;proposed TRN gives convolutional neural networks a remarkable capacity to&#10;discover temporal relations in videos. Through only sparsely sampled video&#10;frames, TRN-equipped networks can accurately predict human-object interactions&#10;in the Something-Something dataset and identify various human gestures on the&#10;Jester dataset with very competitive performance. TRN-equipped networks also&#10;outperform two-stream networks and 3D convolution networks in recognizing daily&#10;activities in the Charades dataset. Further analyses show that the models learn&#10;intuitive and interpretable visual common sense knowledge in videos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.07750" label="1705.07750">
-        <attvalues>
-          <attvalue for="0" value="Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset" />
-          <attvalue for="1" value="  The paucity of videos in current action classification datasets (UCF-101 and&#10;HMDB-51) has made it difficult to identify good video architectures, as most&#10;methods obtain similar performance on existing small-scale benchmarks. This&#10;paper re-evaluates state-of-the-art architectures in light of the new Kinetics&#10;Human Action Video dataset. Kinetics has two orders of magnitude more data,&#10;with 400 human action classes and over 400 clips per class, and is collected&#10;from realistic, challenging YouTube videos. We provide an analysis on how&#10;current architectures fare on the task of action classification on this dataset&#10;and how much performance improves on the smaller benchmark datasets after&#10;pre-training on Kinetics.&#10;  We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on&#10;2D ConvNet inflation: filters and pooling kernels of very deep image&#10;classification ConvNets are expanded into 3D, making it possible to learn&#10;seamless spatio-temporal feature extractors from video while leveraging&#10;successful ImageNet architecture designs and even their parameters. We show&#10;that, after pre-training on Kinetics, I3D models considerably improve upon the&#10;state-of-the-art in action classification, reaching 80.9% on HMDB-51 and 98.0%&#10;on UCF-101.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.2199" label="1406.2199">
-        <attvalues>
-          <attvalue for="0" value="Two-Stream Convolutional Networks for Action Recognition in Videos" />
-          <attvalue for="1" value="  We investigate architectures of discriminatively trained deep Convolutional&#10;Networks (ConvNets) for action recognition in video. The challenge is to&#10;capture the complementary information on appearance from still frames and&#10;motion between frames. We also aim to generalise the best performing&#10;hand-crafted features within a data-driven learning framework.&#10;  Our contribution is three-fold. First, we propose a two-stream ConvNet&#10;architecture which incorporates spatial and temporal networks. Second, we&#10;demonstrate that a ConvNet trained on multi-frame dense optical flow is able to&#10;achieve very good performance in spite of limited training data. Finally, we&#10;show that multi-task learning, applied to two different action classification&#10;datasets, can be used to increase the amount of training data and improve the&#10;performance on both.&#10;  Our architecture is trained and evaluated on the standard video actions&#10;benchmarks of UCF-101 and HMDB-51, where it is competitive with the state of&#10;the art. It also exceeds by a large margin previous attempts to use deep nets&#10;for video classification.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.04214" label="1506.04214">
-        <attvalues>
-          <attvalue for="0" value="Convolutional LSTM Network: A Machine Learning Approach for&#10;  Precipitation Nowcasting" />
-          <attvalue for="1" value="  The goal of precipitation nowcasting is to predict the future rainfall&#10;intensity in a local region over a relatively short period of time. Very few&#10;previous studies have examined this crucial and challenging weather forecasting&#10;problem from the machine learning perspective. In this paper, we formulate&#10;precipitation nowcasting as a spatiotemporal sequence forecasting problem in&#10;which both the input and the prediction target are spatiotemporal sequences. By&#10;extending the fully connected LSTM (FC-LSTM) to have convolutional structures&#10;in both the input-to-state and state-to-state transitions, we propose the&#10;convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model&#10;for the precipitation nowcasting problem. Experiments show that our ConvLSTM&#10;network captures spatiotemporal correlations better and consistently&#10;outperforms FC-LSTM and the state-of-the-art operational ROVER algorithm for&#10;precipitation nowcasting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.10035" label="2007.10035">
-        <attvalues>
-          <attvalue for="0" value="Improving Semantic Segmentation via Decoupled Body and Edge Supervision" />
-          <attvalue for="1" value="  Existing semantic segmentation approaches either aim to improve the object's&#10;inner consistency by modeling the global context, or refine objects detail&#10;along their boundaries by multi-scale feature fusion. In this paper, a new&#10;paradigm for semantic segmentation is proposed. Our insight is that appealing&#10;performance of semantic segmentation requires \textit{explicitly} modeling the&#10;object \textit{body} and \textit{edge}, which correspond to the high and low&#10;frequency of the image. To do so, we first warp the image feature by learning a&#10;flow field to make the object part more consistent. The resulting body feature&#10;and the residual edge feature are further optimized under decoupled supervision&#10;by explicitly sampling different parts (body or edge) pixels. We show that the&#10;proposed framework with various baselines or backbone networks leads to better&#10;object inner consistency and object boundaries. Extensive experiments on four&#10;major road scene semantic segmentation benchmarks including&#10;\textit{Cityscapes}, \textit{CamVid}, \textit{KIITI} and \textit{BDD} show that&#10;our proposed approach establishes new state of the art while retaining high&#10;efficiency in inference. In particular, we achieve 83.7 mIoU \% on Cityscape&#10;with only fine-annotated data. Code and models are made available to foster any&#10;further research (\url{https://github.com/lxtGH/DecoupleSegNets}).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Semantic segmentation. Recent approaches for semantic segmentation are predominantly based on FCNs~\cite{fcn}. Some earlier works~\cite{deeplabv1,deep_structured_seg,crf_as_rnn,seg_as_deep_parsing,crf_instance_seg,Sparse_filer_crf_} use structured prediction operators such as conditional random fields (CRFs) to refine the output boundaries. Instead of these costly DenseCRF, current state-of-the-art methods~\cite{pspnet,deeplabv3,DAnet,EMAnet} boost the segmentation performance by designing sophisticated head networks on dilated backbones~\cite{dilation} to capture contextual information. &#10;PSPNet~\cite{pspnet} proposes pyramid pooling module (PPM) to model multi-scale contexts, whilst DeepLab series~\cite{deeplabv2,deeplabv3,denseaspp} uses astrous spatial pyramid pooling (ASPP). In~\cite{DAnet,AdaptivePyramid_seg,EMAnet,annnet,ccnet,DMF_seg}, non-local operator~\cite{non_local} and self-attention mechanism~\cite{transformer} are adopted to harvest pixel-wise context from the whole image. Meanwhile, graph convolution networks~\cite{graph_cnn,beyond_grids,spg_net,DGM_net} are used to propagate information over the whole image by projecting features into an interaction space. Different from previous approaches, our method learns a flow field generated by the network itself to warp features towards object inner parts. &#10;DCN~\cite{deformable} uses predicted offset to aggregate information in kernel and SPN~\cite{spn_nips2017} proposes to propagate information through affinity pairs. &#10;Different from both work, our module aims to align pixels towards object inner according to the learned offset field to form body feature which is learned with specific loss supervision.&#10;Ding et al~\cite{BAFNet} models unidirectional acyclic graphs to propagate information within the object guided by the boundary. However, it is not efficient due to the usage of the RNN structure between pixels. Our module is light-weighted and can be plugged into the state-of-the-art methods~\cite{deeplabv3p,pspnet} to improve their performance with negligible extra cost, which also proves its efficiency and orthogonality. &#10;&#10;Boundary processing. Several prior works obtain better boundary localization by structure modeling, such as boundary neural fields~\cite{boundaries_network_fields}, affinity field~\cite{aaf}, random walk~\cite{cnn_random_wark}. The work~\cite{Task_edge_detection,PGN_net} uses edge information to refine network output by predicting edge maps from intermediate CNN layers. However, these approaches have some drawbacks, such as the potential error propagation from wrong edge estimation since both tasks are not orthogonal. Also overfitting edges brings noise and leads to inferior final segmentation results. Zhu et al.~\cite{video_propagation} proposes boundary relation loss to utilize coarse predicted segmentation labels for data augmentation. Inspired by the idea of label relaxation~\cite{video_propagation}, we supervise the edge and the body parts respectively. The relaxation body avoids the noise from the edge supervision with the relaxation loss. Experimental results demonstrate both higher model accuracy.&#10;&#10;Multi task learning. Serveral works have proved the effectiveness of combining networks for complementary tasks learning~\cite{cross_stitch_net,ubernet}. The works of previous unified architectures that learn a shared representation using multi-task losses. There are some works~\cite{gated-scnn,PAD-net} using learned segmentation and boundary detection network simultaneously and the learned boundaries as an intermediate representation to aid segmentation. GSCNN~\cite{gated-scnn} designs a two-stream network by merging shape information into feature maps explicitly and introduces a dual-task loss to refine both semantic masks and boundary prediction. Different from these works, our goal is to improve the final segmentation results by explicitly optimizing two decoupled feature maps, and we design a specific framework by decoupling semantic body and boundaries into two orthogonal parts with corresponding loss functions and merge them back into final representation for segmentation task.&#10; &#09;" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Image Feature Modeling, Mathematics, Artificial Intelligence, Deep Learning Architectures, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1908.07433" label="1908.07433">
-        <attvalues>
-          <attvalue for="0" value="Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose&#10;  Estimation" />
-          <attvalue for="1" value="  Estimating the 6D pose of objects using only RGB images remains challenging&#10;because of problems such as occlusion and symmetries. It is also difficult to&#10;construct 3D models with precise texture without expert knowledge or&#10;specialized scanning devices. To address these problems, we propose a novel&#10;pose estimation method, Pix2Pose, that predicts the 3D coordinates of each&#10;object pixel without textured models. An auto-encoder architecture is designed&#10;to estimate the 3D coordinates and expected errors per pixel. These pixel-wise&#10;predictions are then used in multiple stages to form 2D-3D correspondences to&#10;directly compute poses with the PnP algorithm with RANSAC iterations. Our&#10;method is robust to occlusion by leveraging recent achievements in generative&#10;adversarial training to precisely recover occluded parts. Furthermore, a novel&#10;loss function, the transformer loss, is proposed to handle symmetric objects by&#10;guiding predictions to the closest symmetric pose. Evaluations on three&#10;different benchmark datasets containing symmetric and occluded objects show our&#10;method outperforms the state of the art using only RGB images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value=" &#10;This sectiongives a brief summary of previous work related to pose estimation using RGB images. Three different approaches for pose estimation using CNNs are discussed and the recent advances of generative models are reviewed. &#10;&#10;CNN based pose estimation The first, and simplest, method to estimate the pose of an object using a CNN is to predict a representation of a pose directly such as the locations of projected points of 3D bounding boxes~\cite{rad2017bb8,Tekin_2018_CVPR}, classified view points~\cite{kehl2017ssd}, unit quaternions and translations~\cite{xiang2017posecnn}, or the Lie algebra representation, $so(3)$, with the translation of $z$-axis~\cite{Do2018LieNetRM}. Except for methods that predict projected points of the 3D bounding box, which requires further computations for the PnP algorithm, the direct regression is computationally efficient since it does not require additional computation for the pose. The drawback of these methods, however, is the lack of correspondences that can be useful to generate multiple pose hypotheses for the robust estimation of occluded objects. Furthermore, symmetric objects are usually handled by limiting the range of viewpoints, which sometimes requires additional treatments, e.g., training a CNN for classifying view ranges~\cite{rad2017bb8}. Xiang~\etal~\cite{xiang2017posecnn} propose a loss function that computes the average distance to the nearest points of transformed models in an estimated pose and an annotated pose. However, searching for the nearest 3D points is time consuming and makes the training process inefficient. &#10;&#10;The second method is to match features to find the nearest pose template and use the pose information of the template as an initial guess~\cite{linemode_hinterstoisser2012}. Recently, Sundermeyer~\etal~\cite{Sundermeyer_2018_ECCV_implicit} propose an auto-encoder network to train implicit representations of poses without supervision using RGB images only. Manual handling of symmetric objects is not necessary for this work since the implicit representation can be close to any symmetric view. However, it is difficult to specify 3D translations using rendered templates that only give a good estimation of rotations. The size of the 2D bounding box is used to compute the $z$-component of 3D translation, which is too sensitive to small errors of 2D bounding boxes that are given from a 2D detection method.&#10;&#10;The last method is to predict 3D locations of pixels or local shapes in the object space~\cite{cnn_pose:brachmann2016uncertainty_only_rgb,kehl2016deep,Oberweger_2018_ECCV_heatmap}. Brachmann~\etal~\cite{cnn_pose:brachmann2016uncertainty_only_rgb} regress 3D coordinates and predict a class for each pixel using the auto-context random forest. Oberwerger~\etal~\cite{Oberweger_2018_ECCV_heatmap} predict multiple heat-maps to localize the 2D projections of 3D points of objects using local patches. These methods are robust to occlusion because they focus on local information only. However, additional computation is required to derive the best result among pose hypotheses, which makes these methods slow. &#10;&#10;The method proposed in this paper belongs to the last category that predicts 3D locations of pixels in the object frame as in ~\cite{brachmann2014learning_occlusion,cnn_pose:brachmann2016uncertainty_only_rgb}. Instead of detecting an object using local patches from sliding windows, an independent 2D detection network is employed to provide areas of interest for target objects as performed in~\cite{Sundermeyer_2018_ECCV_implicit}.&#10;&#10;Generative models &#10;Generative models using auto-encoders have been used to de-noise~\cite{vincent2010stacked_denoising} or recover the missing parts of images~\cite{NIPS2012_4686_denoising_inpainting}. Recently, using Generative Adversarial Network~(GAN)~\cite{goodfellow2014generative} improves the quality of generated images that are less blurry and more realistic, which are used for the image-to-image translation ~\cite{Isola_2017_CVPR_pix2pix}, image in-painting and de-noising~\cite{iizuka2017globally_inpaintGAN,pathak2016context_inpaintGAN} tasks. Zakharov~\etal~\cite{zakharov2018keepit} propose a GAN based framework to convert a real depth image to a synthetic depth image without noise and background for classification and pose estimation. &#10;&#10;Inspired by previous work, we train an auto-encoder architecture with GAN to convert color images to coordinate values accurately as in the image-to-image translation task while recovering values of occluded parts as in the image in-painting task. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2206.02573" label="2206.02573">
-        <attvalues>
-          <attvalue for="0" value="Team VI-I2R Technical Report on EPIC-KITCHENS-100 Unsupervised Domain&#10;  Adaptation Challenge for Action Recognition 2021" />
-          <attvalue for="1" value="  In this report, we present the technical details of our approach to the&#10;EPIC-KITCHENS-100 Unsupervised Domain Adaptation (UDA) Challenge for Action&#10;Recognition. The EPIC-KITCHENS-100 dataset consists of daily kitchen activities&#10;focusing on the interaction between human hands and their surrounding objects.&#10;It is very challenging to accurately recognize these fine-grained activities,&#10;due to the presence of distracting objects and visually similar action classes,&#10;especially in the unlabelled target domain. Based on an existing method for&#10;video domain adaptation, i.e., TA3N, we propose to learn hand-centric features&#10;by leveraging the hand bounding box information for UDA on fine-grained action&#10;recognition. This helps reduce the distraction from background as well as&#10;facilitate the learning of domain-invariant features. To achieve high quality&#10;hand localization, we adopt an uncertainty-aware domain adaptation network,&#10;i.e., MEAA, to train a domain-adaptive hand detector, which only uses very&#10;limited hand bounding box annotations in the source domain but can generalize&#10;well to the unlabelled target domain. Our submission achieved the 1st place in&#10;terms of top-1 action recognition accuracy, using only RGB and optical flow&#10;modalities as input.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The EPIC-KITCHENS-100 dataset is a large-scale video benchmark, capturing daily cooking activities from egocentric perspective~\cite{damen2020rescaling}. It mainly contains fine-grained actions which reflect the interaction between human hands and their surrounding objects, and each action class is defined by a verb and a noun class. The EPIC-KITCHENS-100 Unsupervised Domain Adaptation (UDA) Challenge for Action Recognition aims to adapt an action recognition model trained on a labelled source domain to an unlabelled target domain. In this challenge, the source domain contains egocentric videos captured in 2018, while the target domain contains egocentric videos captured in 2020 with the same subjects but different cameras and potential change of kitchens. This is a challenging task, as the source and target domains have different data distributions, due to the changes of environments and camera settings. Solutions successfully addressing this challenge can help save much time and efforts when applying the model trained on an existing labelled dataset to a newly collected dataset without annotation.&#10;&#10;In Fig~\ref{figure:intro}, we present some samples from the EPIC-KITCHENS-100 dataset. As shown in the figure, activities in this dataset mainly focus on the interactions between human hands and their surrounding objects. This brings two challenges for fine-grained action recognition. The first challenge is the presence of distracting objects in the scene, which makes it difficult to identify the active objects. Based on the observation that active objects are generally located around the hand regions, we find that it is important to enhance the features around the hand regions~\cite{fan2021understanding}. The other challenge is that some action classes are very similar visually, where capturing the subtle difference among hand motions are essential to accurately recognize the target actions. Therefore, enhancing the features around hand regions can help to reduce the distraction from background and thus improving the action recognition accuracy. Moreover, this may provide further benefits in the context of UDA for fine-grained action recognition by facilitating the learning of domain-invariant features. To the best of our knowledge, MM-SADA~\cite{munro2020multi} is the first attempt on UDA for fine-grained action recognition. It leverages the multi-modal nature of video data to adapt fine-grained action recognition models to unlabelled target domain, which provides the first benchmark on this task. However, it does not consider the aforementioned challenges.&#10;&#10;To address these two challenges, we propose to learn hand-centric features by leveraging the hand bounding box information for UDA on fine-grained action recognition. Specifically, we adapt the TA3N~\cite{chen2019temporal}, an existing method for video domain adaptation, to learn a fine-grained action recognition model that can be adapted to the unlabeled target domain. To achieve high quality hand localization, we apply an uncertainty-aware domain adaptation network, \ie, MEAA, to train a hand detector, which only uses very limited hand bounding box annotations in the source domain but can generalize well to the unlabelled target domain. The experimental results on the EPIC-KITCHENS-100 dataset demonstrate the effectiveness of our approach for UDA on fine-grained action recognition.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Mathematics, Action Recognition, Artificial Intelligence, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.13256" label="2006.13256">
-        <attvalues>
-          <attvalue for="0" value="Rescaling Egocentric Vision" />
-          <attvalue for="1" value="  This paper introduces the pipeline to extend the largest dataset in&#10;egocentric vision, EPIC-KITCHENS. The effort culminates in EPIC-KITCHENS-100, a&#10;collection of 100 hours, 20M frames, 90K actions in 700 variable-length videos,&#10;capturing long-term unscripted activities in 45 environments, using&#10;head-mounted cameras. Compared to its previous version, EPIC-KITCHENS-100 has&#10;been annotated using a novel pipeline that allows denser (54% more actions per&#10;minute) and more complete annotations of fine-grained actions (+128% more&#10;action segments). This collection enables new challenges such as action&#10;detection and evaluating the &quot;test of time&quot; - i.e. whether models trained on&#10;data collected in 2018 can generalise to new footage collected two years later.&#10;The dataset is aligned with 6 challenges: action recognition (full and weak&#10;supervision), action detection, action anticipation, cross-modal retrieval&#10;(from captions), as well as unsupervised domain adaptation for action&#10;recognition. For each challenge, we define the task, provide baselines and&#10;evaluation metrics&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.09691" label="2001.09691">
-        <attvalues>
-          <attvalue for="0" value="Multi-Modal Domain Adaptation for Fine-Grained Action Recognition" />
-          <attvalue for="1" value="  Fine-grained action recognition datasets exhibit environmental bias, where&#10;multiple video sequences are captured from a limited number of environments.&#10;Training a model in one environment and deploying in another results in a drop&#10;in performance due to an unavoidable domain shift. Unsupervised Domain&#10;Adaptation (UDA) approaches have frequently utilised adversarial training&#10;between the source and target domains. However, these approaches have not&#10;explored the multi-modal nature of video within each domain. In this work we&#10;exploit the correspondence of modalities as a self-supervised alignment&#10;approach for UDA in addition to adversarial alignment.&#10;  We test our approach on three kitchens from our large-scale dataset,&#10;EPIC-Kitchens, using two modalities commonly employed for action recognition:&#10;RGB and Optical Flow. We show that multi-modal self-supervision alone improves&#10;the performance over source-only training by 2.4% on average. We then combine&#10;adversarial training with multi-modal self-supervision, showing that our&#10;approach outperforms other UDA methods by 3%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.12743" label="1907.12743">
-        <attvalues>
-          <attvalue for="0" value="Temporal Attentive Alignment for Large-Scale Video Domain Adaptation" />
-          <attvalue for="1" value="  Although various image-based domain adaptation (DA) techniques have been&#10;proposed in recent years, domain shift in videos is still not well-explored.&#10;Most previous works only evaluate performance on small-scale datasets which are&#10;saturated. Therefore, we first propose two large-scale video DA datasets with&#10;much larger domain discrepancy: UCF-HMDB_full and Kinetics-Gameplay. Second, we&#10;investigate different DA integration methods for videos, and show that&#10;simultaneously aligning and learning temporal dynamics achieves effective&#10;alignment even without sophisticated DA methods. Finally, we propose Temporal&#10;Attentive Adversarial Adaptation Network (TA3N), which explicitly attends to&#10;the temporal dynamics using domain discrepancy for more effective domain&#10;alignment, achieving state-of-the-art performance on four video DA datasets&#10;(e.g. 7.9% accuracy gain over &quot;Source only&quot; from 73.9% to 81.8% on &quot;HMDB --&gt;&#10;UCF&quot;, and 10.3% gain on &quot;Kinetics --&gt; Gameplay&quot;). The code and data are&#10;released at http://github.com/cmhungsteve/TA3N.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.02715" label="2304.02715">
-        <attvalues>
-          <attvalue for="0" value="Learning Knowledge-Rich Sequential Model for Planar Homography&#10;  Estimation in Aerial Video" />
-          <attvalue for="1" value="  This paper presents an unsupervised approach that leverages raw aerial videos&#10;to learn to estimate planar homographic transformation between consecutive&#10;video frames. Previous learning-based estimators work on pairs of images to&#10;estimate their planar homographic transformations but suffer from severe&#10;over-fitting issues, especially when applying over aerial videos. To address&#10;this concern, we develop a sequential estimator that directly processes a&#10;sequence of video frames and estimates their pairwise planar homographic&#10;transformations in batches. We also incorporate a set of spatial-temporal&#10;knowledge to regularize the learning of such a sequence-to-sequence model. We&#10;collect a set of challenging aerial videos and compare the proposed method to&#10;the alternative algorithms. Empirical studies suggest that our sequential model&#10;achieves significant improvement over alternative image-based methods and the&#10;knowledge-rich regularization further boosts our system performance. Our codes&#10;and dataset could be found at https://github.com/Paul-LiPu/DeepVideoHomography&#10;" />
-          <attvalue for="2" value="&#10;&#10;Estimating planar homographic transformations between consecutive video frames is a fundamental image task and is an essential part of many multimedia applications, including video management, robot navigation, content-based video retrieval, intelligent drones, self-driving vehicles, etc. The estimated homography matrix, for example, can be used for image stitching~\cite{brown2007automatic}, image completion~\cite{huang2014image}, monocular SLAM~\cite{mur2015orb}, camera calibration~\cite{zhang2000flexible}, 3d scene reconstruction~\cite{zhang19963d}, and camera tracking~\cite{simon2000markerless}. The matrix can also be used to approximate camera poses in aerial videos where facing-downward cameras are positioned far away from the ground and objects on the ground could be reasonably assumed to be from the same planar.&#10;&#10;In the past literature, classical estimators usually build feature-level~\cite{lowe2004distinctive, rublee2011orb} or pixel-level~\cite{lucas1981iterative, baker2004lucas, evangelidis2008parametric} correspondences between camera views and employ perspective geometry to recover their planar transformations. Each feature or pixel is represented as a feature descriptor (e.g., histograms) and is matched across camera views. Robust matching methods, e.g., RANSAC~\cite{fischler1981random}, might be used for eliminating incorrect or noisy correspondences. The feature-based methods are robust to some extent because the detected feature points and feature descriptors are invariant to rotation and scaling, and the RANSAC algorithm can potentially prune outlier correspondences. However, such a stage-wise pipeline involves many hand-crafted parameters, including thresholds for detecting feature points, choices of designing feature descriptors, and algorithms for matching feature descriptors. Those hyper-parameters could be tricky to calibrate in real-world multimedia applications. &#10;&#10;A recent research stream aims to take advantage of the representative power of deep neural networks to develop end-to-end learning-based estimators. Detone et al.~\cite{detone2016deep} propose to regress planar homography parameters from a pair of input images. A multi-layer convolutional neural network (CNN) is trained using synthetic data which includes multiple pairs of images transformed with randomly generated homography matrices. &#10;Nguyen et al.~\cite{nguyen2018unsupervised} introduce an unsupervised method that does not need labels of homography matrices to train the deep networks, as reviewed in Section~\ref{sect:one}. Such an unsupervised method has better adaptability and stronger performance than the supervised one~\cite{detone2016deep} but may suffer from severe over-fitting issues. &#10;&#10;To address the above concerns, in this work, we develop a knowledge-rich sequence-to-sequence model to regress homography parameters for aerial videos. &#10;Figure~\ref{fig:fig1} shows a typical result of the proposed method, which takes a sequence of video frames as inputs and generates a stitched image by a sequence of estimated homography matrices between consecutive video frames. For each pair of input images, our model employs a deep neural network to extract the feature representation of video frame pairs, regresses homography parameters, and warps one input image into the other image. A Long Short-Term Memory (LSTM) model is also employed to directly incorporate the temporal dependencies while estimating the sequence of homography matrices. Our method does not require any annotations and can be trained from raw video frames using standard gradient-based methods. &#10;&#10;We also introduce a set of prior knowledge to regularize the learning of the proposed sequential estimator. These knowledge explicitly impose consistency constraints that should be satisfied while estimating the sequence of homography matrices from the input video sequence. One of the key observations is that for most aerial videos, the camera motion is smooth over time. The homographic transformations between consecutive frames are not arbitrarily different from each other and should be estimated in a coordinated fashion. This leads to a set of &#10;temporal knowledge&#10;Moreover, we might extract multiple image regions of different resolutions from video frames and estimate the homographic transformations from these image regions. Similarly the estimations cross image regions or scales in the same video frames should be consistent with each other, which leads to set of &#10;spatial and scale &#10;knowledge. Incorporating these different types of knowledge is capable of suppressing over-fitting issues while learning the deep model from raw videos. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning Techniques, Computer Science, Aerial Video Analysis, Homographic Transformation, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1709.03966" label="1709.03966">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Deep Homography: A Fast and Robust Homography Estimation&#10;  Model" />
-          <attvalue for="1" value="  Homography estimation between multiple aerial images can provide relative&#10;pose estimation for collaborative autonomous exploration and monitoring. The&#10;usage on a robotic system requires a fast and robust homography estimation&#10;algorithm. In this study, we propose an unsupervised learning algorithm that&#10;trains a Deep Convolutional Neural Network to estimate planar homographies. We&#10;compare the proposed algorithm to traditional feature-based and direct methods,&#10;as well as a corresponding supervised learning algorithm. Our empirical results&#10;demonstrate that compared to traditional approaches, the unsupervised algorithm&#10;achieves faster inference speed, while maintaining comparable or better&#10;accuracy and robustness to illumination variation. In addition, on both a&#10;synthetic dataset and representative real-world aerial dataset, our&#10;unsupervised method has superior adaptability and performance compared to the&#10;supervised deep learning method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.05149" label="2010.05149">
-        <attvalues>
-          <attvalue for="0" value="SDE-AWB: a Generic Solution for 2nd International Illumination&#10;  Estimation Challenge" />
-          <attvalue for="1" value="  We propose a neural network-based solution for three different tracks of 2nd&#10;International Illumination Estimation Challenge (chromaticity.iitp.ru). Our&#10;method is built on pre-trained Squeeze-Net backbone, differential 2D chroma&#10;histogram layer and a shallow MLP utilizing Exif information. By combining&#10;semantic feature, color feature and Exif metadata, the resulting method --&#10;SDE-AWB -- obtains 1st place in both indoor and two-illuminant tracks and 2nd&#10;place in general track.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Auto-white balance, in short AWB, is a necessary processing step in the image signal processor (ISP) in most digital cameras. This function originates from the human eye's property -- color constancy providing object coloration recognition independent of the casting scene illumination. This problem has been heavily explored over decades \cite{gijsenij2011computational}.&#10;&#10;Starting from the classic Gray World algorithm \cite{buchsbaum1980spatial} assuming the global averaged vector of the captured scene is achromatic, a large selection of AWB methods have been proposed, including but not limited to Gray Edge \cite{van2007edge}, Gray Pixels \cite{yanlin2019vissap, yanlin2019arxiv}, Random Forest-based AWB \cite{cheng2015effective}, Corrected-Moment \cite{finlayson2013corrected} and Convolutional Color Constancy \cite{barron2015convolutional, barron2017fourier}. In the era of deep convolutional neural network (CNN), there are also a list of works that challenges the conventional works and refreshes the state-of-the-art performance from year to year. Bianco et al. \cite{bianco2015color} initially shows the potential of applying CNN for illumination estimation. Then more effective and efficient CNN variants are designed, like AlexNet+SVM \cite{yanlin2016icpr}, DS-Net \cite{shi2016eccv}, FC4-Net \cite{hu2017cvpr}, RCC \cite{yanlin2017iccv}, BCC \cite{qian2020benchmark} and so on. &#10;&#10;Along with the rapid changes in AWB work, the mainstream datasets for assessing AWB methods are varying as well. SFU laboratory dataset \cite{barnard2002data} and SFU gray ball dataset \cite{ciurea2003large} are two datasets published over 20 years. Both have their design merits, but are less visited nowadays due to the limited number of images or poor image resolution. In recent years, Gehler et al. \cite{gehler2008bayesian, shi2010re} released Gehler-Shi dataset and Cheng et al. \cite{Cheng14} published the NUS 8-camera dataset. Compared to the mentioned SFU data, both datasets contains more varying-scene images (568 for Gehler-Shi, ) are captured using DSLR cameras (its high resolution also allows cropping to generate more samples). The up-to-date works C4-Net \cite{yu2020aaai}, ICDF \cite{xu2020end} achieved nearly-saturated results over these two datasets. There exists a great need of a new challenging large-scale dataset. &#10;The 2nd illumination estimation challenge, which this paper is submitted for, brings such a dataset for AWB study. For convenience, we call this challenge dataset as IITP dataset. The main attributes of IITP datasets are: in total it has nearly five thousand DSLR images, each of which annotated with two illuminant vectors and Exif tag; All images are divided into three tracks (general, indoor and two-illuminant cases), with regarding to the captured scenes and the discrepancy of the measured two illuminant vectors. &#10;&#10;For the challenge, we propose a net, called SDE-AWB, for all three tracks. To allow the method to output single illuminant vector or two, SDE-AWB itself is a flexible combination of three different learnable modules: Squeeze-Net backbone, Differential 2D chroma histogram layer and a shallow Exif MLP. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Computer Vision Applications, Mathematics, Physics, Computer Science, Image Processing Techniques, Neural Network Architecture, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1803.08326" label="1803.08326">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Gray Pixel for Statistical Illumination Estimation" />
-          <attvalue for="1" value="  We present a statistical color constancy method that relies on novel gray&#10;pixel detection and mean shift clustering. The method, called Mean Shifted Grey&#10;Pixel -- MSGP, is based on the observation: true-gray pixels are aligned&#10;towards one single direction. Our solution is compact, easy to compute and&#10;requires no training. Experiments on two real-world benchmarks show that the&#10;proposed approach outperforms state-of-the-art methods in the camera-agnostic&#10;scenario. In the setting where the camera is known, MSGP outperforms all&#10;statistical methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1507.00410" label="1507.00410">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Color Constancy" />
-          <attvalue for="1" value="  Color constancy is the problem of inferring the color of the light that&#10;illuminated a scene, usually so that the illumination color can be removed.&#10;Because this problem is underconstrained, it is often solved by modeling the&#10;statistical regularities of the colors of natural objects and illumination. In&#10;contrast, in this paper we reformulate the problem of color constancy as a 2D&#10;spatial localization task in a log-chrominance space, thereby allowing us to&#10;apply techniques from object detection and structured prediction to the color&#10;constancy problem. By directly learning how to discriminate between correctly&#10;white-balanced images and poorly white-balanced images, our model is able to&#10;improve performance on standard benchmarks by nearly 40%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1504.04548" label="1504.04548">
-        <attvalues>
-          <attvalue for="0" value="Color Constancy Using CNNs" />
-          <attvalue for="1" value="  In this work we describe a Convolutional Neural Network (CNN) to accurately&#10;predict the scene illumination. Taking image patches as input, the CNN works in&#10;the spatial domain without using hand-crafted features that are employed by&#10;most previous methods. The network consists of one convolutional layer with max&#10;pooling, one fully connected layer and three output nodes. Within the network&#10;structure, feature learning and regression are integrated into one optimization&#10;process, which leads to a more effective model for estimating scene&#10;illumination. This approach achieves state-of-the-art performance on a standard&#10;dataset of RAW images. Preliminary experiments on images with spatially varying&#10;illumination demonstrate the stability of the local illuminant estimation&#10;ability of our CNN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1607.03856" label="1607.03856">
-        <attvalues>
-          <attvalue for="0" value="Deep Structured-Output Regression Learning for Computational Color&#10;  Constancy" />
-          <attvalue for="1" value="  Computational color constancy that requires esti- mation of illuminant colors&#10;of images is a fundamental yet active problem in computer vision, which can be&#10;formulated into a regression problem. To learn a robust regressor for color&#10;constancy, obtaining meaningful imagery features and capturing latent&#10;correlations across output variables play a vital role. In this work, we&#10;introduce a novel deep structured-output regression learning framework to&#10;achieve both goals simultaneously. By borrowing the power of deep convolutional&#10;neural networks (CNN) originally designed for visual recognition, the proposed&#10;framework can automatically discover strong features for white balancing over&#10;different illumination conditions and learn a multi-output regressor beyond&#10;underlying relationships between features and targets to find the complex&#10;interdependence of dif- ferent dimensions of target variables. Experiments on&#10;two public benchmarks demonstrate that our method achieves competitive&#10;performance in comparison with the state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.11180" label="1912.11180">
-        <attvalues>
-          <attvalue for="0" value="Cascading Convolutional Color Constancy" />
-          <attvalue for="1" value="  Regressing the illumination of a scene from the representations of object&#10;appearances is popularly adopted in computational color constancy. However,&#10;it's still challenging due to intrinsic appearance and label ambiguities caused&#10;by unknown illuminants, diverse reflection property of materials and extrinsic&#10;imaging factors (such as different camera sensors). In this paper, we introduce&#10;a novel algorithm by Cascading Convolutional Color Constancy (in short, C4) to&#10;improve robustness of regression learning and achieve stable generalization&#10;capability across datasets (different cameras and scenes) in a unique&#10;framework. The proposed C4 method ensembles a series of dependent illumination&#10;hypotheses from each cascade stage via introducing a weighted&#10;multiply-accumulate loss function, which can inherently capture different modes&#10;of illuminations and explicitly enforce coarse-to-fine network optimization.&#10;Experimental results on the public Color Checker and NUS 8-Camera benchmarks&#10;demonstrate superior performance of the proposed algorithm in comparison with&#10;the state-of-the-art methods, especially for more difficult scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.03796" label="2301.03796">
-        <attvalues>
-          <attvalue for="0" value="Enhancing Evaluation Methods for Infrared Small-Target Detection in&#10;  Real-world Scenarios" />
-          <attvalue for="1" value="  Infrared small target detection (IRSTD) poses a significant challenge in the&#10;field of computer vision. While substantial efforts have been made over the&#10;past two decades to improve the detection capabilities of IRSTD algorithms,&#10;there has been a lack of extensive investigation into the evaluation metrics&#10;used for assessing their performance. In this paper, we employ a systematic&#10;approach to address this issue by first evaluating the effectiveness of&#10;existing metrics and then proposing new metrics to overcome the limitations of&#10;conventional ones. To achieve this, we carefully analyze the necessary&#10;conditions for successful detection and identify the shortcomings of current&#10;evaluation metrics, including both pre-thresholding and post-thresholding&#10;metrics. We then introduce new metrics that are designed to align with the&#10;requirements of real-world systems. Furthermore, we utilize these newly&#10;proposed metrics to compare and evaluate the performance of five widely&#10;recognized small infrared target detection algorithms. The results demonstrate&#10;that the new metrics provide consistent and meaningful quantitative&#10;assessments, aligning with qualitative observations.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Nowadays, infrared (IR) imaging has a wide range of application from medical \cite{chudecka2022use,moradi2022infrared} and industrial diagnosis \cite{CHOUDHARY2021109196} to defense \cite{shahraki2021infrared} and remote sensing \cite{raza2021ir}. Generally, processing IR images is a challenging task \cite{zhang2022isnet} due to the specifications of IR imaging. Among all the aforementioned applications, IR small target detection (IRSTD) is a highly challenging research field because:&#10;\begin{itemize}&#10;\item Since the IR small targets are far from the imaging device, the target has low local contrast and appears as a dim spot in the image plane \cite{shahraki2022infrared}.&#10;\item The small target in IR images typically occupies handful of pixels \cite{moradi2020fast}. Thus, the region of interest (ROI) does not represent distinguished features.&#10;\item The edges of the small target are blurred due to atmospheric thermal fields \cite{khaledian2014new}. Therefore, there are not a clear boundary between background area and target pixels. &#10;\end{itemize} &#10;&#10;The block diagram of a typical IRSTD pipeline is illustrated in \ref{fig:IRST_block_diagram}. As shown in the figure, the input IR image is first process by the IRSTD algorithm to create a saliency map. Note that, while the IRSTD algorithm may refer to the end-to-end IR image processing pipeline, here, the process of construction of saliency map from input IR image is called IRSTD algorithm. The goal is to suppress the background area and enhance target pixels. An ideal saliency map should eliminate the background intensities and only preserve the target area. After saliency map reconstruction, a thresholding strategy is chosen to be applied on the saliency map. Then, true (logical one) pixels in the resulting binary image is considered as target-like objects. &#10;&#10;Considering the pipeline in the \ref{fig:IRST_block_diagram}, specific attributes are defined for images in pipeline. &#10;&#09;In the IRSTD terminology, the input image can be described using two attributes:&#10;&#09;\begin{itemize}&#10;&#10;\item $\sigma_{b,in}$: This represents the standard deviation of background pixels in the input image, which is directly related to the complexity of the background. A smaller value of $\sigma_{b,in}$ indicates a smoother background, while a larger value indicates a more complex background.&#09;&#10;&#09;\item SCR$_{in}$: This acronym stands for the signal-to-clutter ratio in the input image. SCR is defined as $\frac{\mu_t-\mu_b}{\sigma_b}$, where $\mu_t$, $\mu_b$, and $\sigma_b$ denote the mean value of target pixels, mean value of local background pixels, and standard deviation of the local background, respectively.&#10;&#09;&#09;\end{itemize} &#10;&#10;Same argument is valid for saliency map (The processed image by IRSTD algorithm). Thus, just like the input attributes, $\sigma_{b,out}$ and SCR$_{out}$ represents the background complexity and the signal to clutter ration in the saliency map. It is clear that for a typical IRSTD pipeline:&#10;\begin{equation}&#10;\sigma_{b,out} &lt; \sigma_{b,in} \quad and \quad SCR_{out} &gt; SCR_{in}&#10;\end{equation}&#10;&#10;According to \ref{eq:enhancementparam}, two performance metrics are defined for evaluation of IRSTD algorithms: Background suppression factor (BSF) and signal to clutter ration gain (SCRG) which are defined as follows \cite{bai2022floating}:&#10;\begin{equation}&#10;BSF=\frac{\sigma_{b,in}}{\sigma_{b,out}} \quad, \quad SCRG=\frac{SCR_{out}}{SCR_{in}}&#10;\end{equation}&#10;&#10;Based on \ref{eq:enhancementparam} and \ref{eq:bsfscrg}, larger values for both SCRG and BSF are desired. Note that, in case of evaluation of different IRSTD algorithms, since the input images are the same for all baseline algorithms, $SCR_{out}$ and $\frac{1}{\sigma_{b,out}}$ can be used as performance metrics, as well.&#10;&#10;The IRSTD algorithms are well-studied in the literature. Mainly, these algorithms can be categorized based on filtering method, contrast measure calculation, and data structure decomposition \cite{zhao2022single}. &#10;&#10;The filtering based methods are divided into two sub-categories. The first one is the spatial domain filtering, in which, the input infrared image is processed using local kernels to enhance the target area. Max-mean \cite{deshpande1999max}, max-median \cite{deshpande1999max}, bilateral filtering \cite{bae2014spatial}, morphological operators \cite{zhu2020balanced}, two dimensional least mean square \cite{han2019local} are some instances of this sub-category. The second one refers to processing in the transformation domain. In these techniques, the input image is transformed to a desired transformation space like as frequency \cite{wang2017infrared} and wavelet \cite{wang2012detecting} domains. Then, after processing the transformed information, the inverse transform is applied to recover true targets.&#10;&#10;Methods based on human visual systems (HVS) which lead to local contrast-based mechanism has received researchers' attention during last few years. These methods outperform filter-based methods in terms of SCRG and BSF. However, they usually have higher computational complexity compared to filter-based ones. Generally, local contrast can be constructed in either difference or ratio forms. Difference local contrast like as Laplacian of Gaussian (LoG) \cite{kim2012scale}, difference of Gaussian (DoG) \cite{wang2012infrared}, improved difference of Gabor \cite{han2016infrared}, center-surround difference measure \cite{xie2014small}, and local difference adaptive measure \cite{li2019small}. Unlike the difference form local measures, ratio-form local measures utilize enhancement factor which is the ration between the center cell and surrounding ones. Local contrast measure (LCM) \cite{chen2013local}, improved local contrast measure (ILCM) \cite{han2014robust}, relative local contrast measure (RLCM) \cite{han2018infrared}, Tri-Layer local constrast method (TLLCM) \cite{han2019localb}, novel local contrast descriptor (NLCD) \cite{qin2019infrared}, absolute directional mean difference (ADMD) \cite{moradi2020fast}, and weighted strengthened local contrast measure (WSLCM) \cite{han2020infrared} are the most effective IRSTDs in the literature. There is also a combined local measure which benefits from both difference and ratio from of local contrast measure \cite{han2022ratio}. &#10;&#10;Data structure decomposition-based methods are also a newly introduced class of IRSTDs. Sparse and low-rank matrices decomposition is the principal of these class of IRSTDs. Infrared patch image (IPI) model \cite{gao2013infrared}, weighted infrared patch image (WIPI) model \cite{dai2016infrared}, non-negative infrared patch image model based on partial sum minimization of singular values (NIPPS) \cite{dai2017non}, nonconvex rank approximation minimization (NRAM) \cite{zhang2018infrared}, and nonconvex optimization with an L$_p$ norm constraint (NOLC) \cite{zhang2019infrared} are the recent efforts of IR image decomposition-based approach.&#10;&#10;Deep learning-based methods, unlike manually designed operators, have the advantage of leveraging the abundance of data available to effectively learn the target model. The work in \cite{wang2019miss} involved training generators using a generative adversarial network to address miss detection and false alarm aspects. ACM \cite{dai2021asymmetric} devised feature fusion modules within the encoder-decoder framework to enhance the representation of low and deep semantic features, resulting in a more effective feature representation. ALCNet \cite{dai2021attentional} employs shift operations on semantic tensors to simulate local contrast, aiming to extract specific local information about the target. &#10;&#09;In \cite{li2022dense}, the authors introduce a dense nested interactive module (DNIM) to maintain the information of small targets in deep layers through progressive interaction. Additionally, a cascaded channel and spatial attention module (CSAM) is proposed to adaptively enhance multi-level features, enabling effective incorporation and exploitation of contextual information.&#10;&#09;&#09;In \cite{ying2023mapping}, the authors proposed a label evolution framework called Label Evolution with Single Point Supervision (LESPS) to address the labor-intensive nature of fully supervised training for infrared small target detection in CNNs. By leveraging the phenomenon of &quot;mapping degeneration,&quot; where CNNs initially learn to segment pixels near the targets before converging to predict ground truth point labels, LESP enables progressive expansion of the point labels using intermediate predictions.&#10;&#10;When a saliency map is generated prior to the detection task, the dream is to obtain larger BSF and SCRG values. However, having larger BSF and SCRG does not guarantee a successful detection. A high performance IRSTD algorithm should be followed by a proper thresholding strategy to detect real targets and eliminate false responses. This is why there are two more performance metrics after applying the thresholding operation to the saliency map. These two metrics which demonstrate the ability of detection true targets and eliminating false responses are called probability of detection $P_d$ and probability of false alarms $P_{fa}$, respectively. In contrast to BSF and SCRG which are measurable before applying the threshold (This is why we call them pre-thresholding attributes), these two metrics are measured on binary images and therefore we call them post-thresholding attributes (\ref{fig:IRST_block_diagram}). &#10;&#10;As mentioned in the previous paragraph, for a successful detection, both high performance IRSTD algorithm as well as the proper thresholding strategy are required. Regardless of effectiveness of the IRSTD algorithm, improper thresholding will leads to missing true targets and having false responses which could be disaster for a practical system. Hence, in this paper, after investigating various thresholding strategies, the best methods for applying threshold to the saliency map is presented. Then, current pre-thresholding as well as the post-thresholding metrics are investigated, and some new metrics which are aligned with practical considerations are proposed. The rest of this paper is organized as follows: in the next section, the role of thresholding in practical systems is deeply investigated. Then, in section3, current pre-thresholding metrics are reviewed. After demonstrating their shortages, modified metrics are proposed for IRSTD performance evaluation. In section4, same process is performed for post-thresholding metrics. In section5, The newly proposed metrics are used for performance comparison of common IRSTD algorithms. Finally, the paper is concluded in section6. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Infrared Target Detection, Mathematics, Evaluation Metric Analysis, Computer Science, Pattern Recognition, Computer Vision Algorithms, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1810.03173" label="1810.03173">
-        <attvalues>
-          <attvalue for="0" value="Fast and Robust Small Infrared Target Detection Using Absolute&#10;  Directional Mean Difference Algorithm" />
-          <attvalue for="1" value="  Infrared small target detection in an infrared search and track (IRST) system&#10;is a challenging task. This situation becomes more complicated when high&#10;gray-intensity structural backgrounds appear in the field of view (FoV) of the&#10;infrared seeker. While the majority of the infrared small target detection&#10;algorithms neglect directional information, in this paper, a directional&#10;approach is presented to suppress structural backgrounds and develop a more&#10;effective detection algorithm. To this end, a similar concept to the average&#10;absolute gray difference (AAGD) is utilized to construct a novel directional&#10;small target detection algorithm called absolute directional mean difference&#10;(ADMD). Also, an efficient implementation procedure is presented for the&#10;proposed algorithm. The proposed algorithm effectively enhances the target area&#10;and eliminates background clutter. Simulation results on real infrared images&#10;prove the significant effectiveness of the proposed algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.14530" label="2009.14530">
-        <attvalues>
-          <attvalue for="0" value="Asymmetric Contextual Modulation for Infrared Small Target Detection" />
-          <attvalue for="1" value="  Single-frame infrared small target detection remains a challenge not only due&#10;to the scarcity of intrinsic target characteristics but also because of lacking&#10;a public dataset. In this paper, we first contribute an open dataset with&#10;high-quality annotations to advance the research in this field. We also propose&#10;an asymmetric contextual modulation module specially designed for detecting&#10;infrared small targets. To better highlight small targets, besides a top-down&#10;global contextual feedback, we supplement a bottom-up modulation pathway based&#10;on point-wise channel attention for exchanging high-level semantics and subtle&#10;low-level details. We report ablation studies and comparisons to&#10;state-of-the-art methods, where we find that our approach performs&#10;significantly better. Our dataset and code are available online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.00487" label="2106.00487">
-        <attvalues>
-          <attvalue for="0" value="Dense Nested Attention Network for Infrared Small Target Detection" />
-          <attvalue for="1" value="  Single-frame infrared small target (SIRST) detection aims at separating small&#10;targets from clutter backgrounds. With the advances of deep learning, CNN-based&#10;methods have yielded promising results in generic object detection due to their&#10;powerful modeling capability. However, existing CNN-based methods cannot be&#10;directly applied for infrared small targets since pooling layers in their&#10;networks could lead to the loss of targets in deep layers. To handle this&#10;problem, we propose a dense nested attention network (DNANet) in this paper.&#10;Specifically, we design a dense nested interactive module (DNIM) to achieve&#10;progressive interaction among high-level and low-level features. With the&#10;repeated interaction in DNIM, infrared small targets in deep layers can be&#10;maintained. Based on DNIM, we further propose a cascaded channel and spatial&#10;attention module (CSAM) to adaptively enhance multi-level features. With our&#10;DNANet, contextual information of small targets can be well incorporated and&#10;fully exploited by repeated fusion and enhancement. Moreover, we develop an&#10;infrared small target dataset (namely, NUDT-SIRST) and propose a set of&#10;evaluation metrics to conduct comprehensive performance evaluation. Experiments&#10;on both public and our self-developed datasets demonstrate the effectiveness of&#10;our method. Compared to other state-of-the-art methods, our method achieves&#10;better performance in terms of probability of detection (Pd), false-alarm rate&#10;(Fa), and intersection of union (IoU).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.01484" label="2304.01484">
-        <attvalues>
-          <attvalue for="0" value="Mapping Degeneration Meets Label Evolution: Learning Infrared Small&#10;  Target Detection with Single Point Supervision" />
-          <attvalue for="1" value="  Training a convolutional neural network (CNN) to detect infrared small&#10;targets in a fully supervised manner has gained remarkable research interests&#10;in recent years, but is highly labor expensive since a large number of&#10;per-pixel annotations are required. To handle this problem, in this paper, we&#10;make the first attempt to achieve infrared small target detection with&#10;point-level supervision. Interestingly, during the training phase supervised by&#10;point labels, we discover that CNNs first learn to segment a cluster of pixels&#10;near the targets, and then gradually converge to predict groundtruth point&#10;labels. Motivated by this &quot;mapping degeneration&quot; phenomenon, we propose a label&#10;evolution framework named label evolution with single point supervision (LESPS)&#10;to progressively expand the point label by leveraging the intermediate&#10;predictions of CNNs. In this way, the network predictions can finally&#10;approximate the updated pseudo labels, and a pixel-level target mask can be&#10;obtained to train CNNs in an end-to-end manner. We conduct extensive&#10;experiments with insightful visualizations to validate the effectiveness of our&#10;method. Experimental results show that CNNs equipped with LESPS can well&#10;recover the target masks from corresponding point labels, {and can achieve over&#10;70% and 95% of their fully supervised performance in terms of pixel-level&#10;intersection over union (IoU) and object-level probability of detection (Pd),&#10;respectively. Code is available at https://github.com/XinyiYing/LESPS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.09220" label="2210.09220">
-        <attvalues>
-          <attvalue for="0" value="A Saccaded Visual Transformer for General Object Spotting" />
-          <attvalue for="1" value="  This paper presents the novel combination of a visual transformer style patch&#10;classifier with saccaded local attention. A novel optimisation paradigm for&#10;training object models is also presented, rather than the optimisation function&#10;minimising class membership probability error the network is trained to&#10;estimate the normalised distance to the centroid of labelled objects. This&#10;approach builds a degree of transnational invariance directly into the model&#10;and allows fast saccaded search with gradient ascent to find object centroids.&#10;The resulting saccaded visual transformer is demonstrated on human faces.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The task of recognising or spotting general objects in images remains combinatorically difficult.&#10;The space of visual appearance of objects is both large and scales geometrically with the number of pixels in&#10;any image of an object to be recognised. Increasing the resolution of images of an object can make the&#10;computational aspects of object recognition harder.&#10;&#10;Smaller distinguished sub-parts can be used to reduce the relevant search space for refined model matching. &#10;This style of recogniser is exemplified by finding SIFT\cite{DBLP:journals/ijcv/Lowe04} or&#10;MSER\cite{DBLP:conf/bmvc/MatasCUP02} etc. style features followed by geometric matching or bag of words indexing,&#10;reviews of discrete feature detectors include \cite{gauglitz2011evaluation}.&#10;&#10;Convolution nets have represented the state of the art in object classification \cite{726791} but deeper networks like Resnet50 \cite{ offer a greater capacity to learn larger scale objects.&#10;&#10;The Visual Transformer \cite{DBLP:journals/corr/abs-2010-11929} offers a different approach using a higher dimensional model consisting of&#10;embedded feature vectors derived from learned basis functions for sub-patches covering an extended image patch.&#10;The size of the resulting network means large training sets are required (300M images to train 90M weights) and the networks are&#10;slow to apply. Adding new models to a network requires significant training resources. The degree to which the ViT demonstrates&#10;translational invariance is not clear.&#10;&#10;This paper presents a novel optimisation paradigm for training transformer style object spotting/recognition networks&#10;that allows fast saccaded search. Visual transformers are typically trained through presenting vast numbers of larger image patches&#10;that contain single exemplars of visual categories to the training system. The attraction of this being a single recogniser&#10;can be trained to distinguish thousands of visual categories at once. In the training process a set of basis functions&#10;are learned that allow the creation of a set of linear coefficients to represent the content of sub-patches and their relative locations.&#10;Visual inspection of the basis functions in the ViT paper \cite{DBLP:journals/corr/abs-2010-11929}&#10;show them to bear at least a passing resemblance to&#10;visual receptive fields in higher animals \cite{hubel1963shape}.&#10;&#10;This paper uses a similar network structure but seeks to create translation tolerant single object recognisers through&#10;the use of a novel optimisation criteria. Rather than training on error in probability of object class the system optimises&#10;predicted scaled distance from the labeled object centroid. Additionaly sampling points may be drawn&#10;from a subset of image locations which are of high visual interest. &#10;In this paper the visual interest operator chosen is the ST-transform \cite{DBLP:journals/corr/abs-2102-02000}, returning the&#10;connected dark/light boundary chains as geometrically filterable candidate saccade points.&#10;Geometric filtering of the dark/light boundary could follow the ideas in \cite{DBLP:conf/bmvc/RichardsFJ92} and \cite{DBLP:journals/cvgip/RichardsH85}.&#10;&#10;The paper is layed out as follows: section\ref{sec:distance} details the \{bf Di\}stance to \{bf F\}eature \{bf T\}raining function,&#10;the PyTorch network definition of the model being given in appendix\ref{sec:pytorch}. &#10;Section \ref{sec:saccade} quantifies the speed benefits of only applying expensive classifiers at appropriate saccade points.&#10;Conclusions an directions for future research are given in section\ref{sec:conclusions}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Deep Learning Optimization, Object Recognition, Mathematics, Artificial Intelligence, Visual Transformers, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2102.02000" label="2102.02000">
-        <attvalues>
-          <attvalue for="0" value="A generalised feature for low level vision" />
-          <attvalue for="1" value="  This papers presents a novel quantised transform (the Sinclair-Town or ST&#10;transform for short) that subsumes the rolls of both edge-detector, MSER style&#10;region detector and corner detector. The transform is similar to the $unsharp$&#10;transform but the difference from the local mean is quantised to 3 values&#10;(dark-neutral-light). The transform naturally leads to the definition of an&#10;appropriate local scale. A range of methods for extracting shape features form&#10;the transformed image are presented. The generalized feature provides a robust&#10;basis for establishing correspondence between images. The transform readily&#10;admits more complicated kernel behaviour including multi-scale and asymmetric&#10;elements to prefer shorter scale or oriented local features.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.04821" label="2205.04821">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised regression learning using domain knowledge: Applications&#10;  to improving self-supervised denoising in imaging" />
-          <attvalue for="1" value="  Regression that predicts continuous quantity is a central part of&#10;applications using computational imaging and computer vision technologies. Yet,&#10;studying and understanding self-supervised learning for regression tasks -&#10;except for a particular regression task, image denoising - have lagged behind.&#10;This paper proposes a general self-supervised regression learning (SSRL)&#10;framework that enables learning regression neural networks with only input data&#10;(but without ground-truth target data), by using a designable pseudo-predictor&#10;that encapsulates domain knowledge of a specific application. The paper&#10;underlines the importance of using domain knowledge by showing that under&#10;different settings, the better pseudo-predictor can lead properties of SSRL&#10;closer to those of ordinary supervised learning. Numerical experiments for&#10;low-dose computational tomography denoising and camera image denoising&#10;demonstrate that proposed SSRL significantly improves the denoising quality&#10;over several existing self-supervised denoising methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Deep regression neural network (NN)-based methods that can accurately predict real- or complex-valued output have been rapidly gaining popularity in a wide range of computational imaging and computer vision applications including &#10;image denoising \cite{Vincent&amp;etal:10JMLR, Xie&amp;Xu&amp;Chen:12NIPS, Zhang&amp;etal:17TIP}, &#10;image deblurring \cite{Xu&amp;etal:14NIPS}, &#10;image super-resolution \cite{Dong&amp;etal:16TPAMI,Kim&amp;Lee&amp;Lee:16CVPR},&#10;light-field reconstruction \cite{Chun&amp;etal:20PAMI, Huang&amp;etal:20ICASSP}, object localization \cite{Szegedy&amp;Toshev&amp;Erhan:13NIPS},&#10;end-to-end autonomous driving \cite{Bojarski&amp;etal:16NIPS-DLS}.&#10;Yet, they lack a general self-supervised learning framework.&#10;&#10;In training a regression NN $f : \bbR^{N} arrow \bbR^{M}$,&#10;the most prevalent supervised learning approach minimizes the mean square error (MSE) between what $f$ predicts from an input $x \in \bbR^N$ and a ground-truth target $y \in \bbR^M$:&#10;\be{&#10;\min_{f} \bbE_{x,y} \nm{ f(x) - y }_2^2.&#10;}&#10;Learning a denoising or refining NN uses \R{sys:sup} &#10;with $M=N$ -- dubbed Noise2True -- where $x$ is a corrupted image and $y$ is a clean (i.e., ground-truth) image.&#10;However, it is challenging or even impossible to collect many clean images $y$ in many practical applications, motivating research on self-supervised learning for image denoising&#10;\cite{Ulyanov&amp;etal:18CVPR, &#10;Soltanayev&amp;Chun:18NIPS, &#10;Krull&amp;etal:19CVPR, &#10;Batson&amp;Royer:19ICML, &#10;Laine&amp;etal:19NIPS, &#10;Moran&amp;etal:20CVPR,&#10;Quan&amp;etal:20CVPR,&#10;Xu&amp;etal:20TIP,&#10;Hendriksen&amp;etal:20TCI, &#10;Xie&amp;Wang&amp;Ji:20NIPS,&#10;Krull&amp;etal:20FCS,&#10;Prakash&amp;etal:20ISBI,&#10;Prakash&amp;Krull&amp;Jug:21ICLR,&#10;Khademi&amp;etal:21WACV,&#10;Huang&amp;eta:21CVPR,Kim&amp;Ye:21NIPS}&#10;-- called self-supervised image denoising.&#10;To learn a denoiser $f$ with single noisy images,&#10;a popular self-supervised image denoising method, Noise2Self&#10;\cite{Batson&amp;Royer:19ICML} (see also the concurrent work \cite{Krull&amp;etal:19CVPR}), and its sophisticated relaxation, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, study the following MSE minimization problem:&#10;\be{&#10;\min_{f} \bbE_{x} \nm{ f(x) - x }_2^2.&#10;}&#10;These methods use some partitioning schemes in \R{sys:self-sup} to avoid that its optimal solution is just the identity mapping $\cI$. &#10;Partitioning schemes were originally proposed to train denoising NN only with one noisy observation per scene.&#10;Noise2Noise \cite{Lehtinen&amp;etal:18ICML} that learns a denoiser with pairs of two independent noisy images,&#10;is a pioneer work for self-supervised image denoising.&#10;Motivated by Noise2Noise, several self-supervised image denoising methods such as Noise2Inverse \cite{Hendriksen&amp;etal:20TCI}, Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, and \cite{Soltanayev&amp;Chun:18NIPS, Moran&amp;etal:20CVPR, Xu&amp;etal:20TIP}&#10;emulate pairs of two independent noisy images,&#10;by applying partitioning or adding simulated noise to single noisy measurements. Noise2Inverse~\cite{Hendriksen&amp;etal:20TCI} emulates two independent noisy images by dividing measurement data, e.g., sinogram in computational tomography (CT), into two and reconstructing them.&#10;Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR} emulates two independent noisy images from a single noisy image using a sub-sampling method.&#10;In addition, there exist single-image self-supervised denoising methods that fit a denoising NN to each noisy image, such as deep image prior~\cite{Ulyanov&amp;etal:18CVPR}, Noisy-As-Clean~\cite{Xu&amp;etal:20TIP} and Self2Self~\cite{Quan&amp;etal:20CVPR}.&#10;All the aforementioned methods are blind in a sense that one does not need to estimate noise parameters,&#10;whereas some works estimate statistical parameters of noise, such as noise histogram \cite{Krull&amp;etal:20FCS} and parameters of Gaussian mixture noise model \cite{Prakash&amp;Krull&amp;Jug:21ICLR}.&#10;All the aforementioned methods except for \cite{Ulyanov&amp;etal:18CVPR} have been developed based on some noise assumptions &#10;including pixel-wise independent noise&#10;\cite{Krull&amp;etal:19CVPR, Xie&amp;Wang&amp;Ji:20NIPS, Huang&amp;eta:21CVPR}, zero-mean noise \cite{Lehtinen&amp;etal:18ICML, Batson&amp;Royer:19ICML, Xie&amp;Wang&amp;Ji:20NIPS}, and \dquotes{weak} noise \cite{Xu&amp;etal:20TIP}.&#10;Yet, the existing self-supervised denoising methods lack design flexibility that might relax noise assumptions and further improve the denoising performance of NNs. &#10;&#10;This paper presents new insights on this topic.&#10;The paper proposes a general self-supervised learning framework for regression problems, which we refer to as &#10;self-supervised regression learning (SSRL).&#10;Proposed SSRL enables learning regression NNs with only input samples, by using a designable pseudo-predictor that can encapsulate domain knowledge of a specific application.&#10;Our main results show that under some mild condition(s) (e.g., in image denoising, statistical noise properties in $x$), the \dquotes{better} pseudo-predictor is designed, the prediction error of the optimal solution of SSRL is expected to become closer to that of ordinary supervised learning or&#10;the proposed SSRL loss becomes closer to the supervision counterpart. &#10;(We will define a \dquotes{good} pseudo-predictor when presenting the main results for different setups.)&#10;In addition, a pseudo-predictor designed with good domain knowledge can substantially relax noise assumptions of existing self-supervised denoising methods. &#10;Numerical experiments for image denoising in low-dose CT and camera imaging with both simulated and intrinsically noisy (i.e.,~real-world) datasets -- corrupted by only single noise realization -- demonstrate that the proposed SSRL framework significantly improves denoising quality compared to several existing self-supervised denoising methods.&#10;Put together, our findings provide new insights into how using good domain knowledge can improve self-supervised denoising, underscoring the benefits of understanding application-specific knowledge in SSRL.&#10;&#10;The remainder of this paper is organized as follows.&#10;Section~\ref{sec:theory} proposes the SSRL framework in different setups, &#10;and analyzes prediction errors of its optimal solution and SSRL loss function depending on a designable pseudo-predictor.&#10;In addition, the sectiondescribes its relations to existing self-supervised denoising works, and explains that SSRL can relax noise assumptions of existing self-supervised denoising methods. &#10;Section~\ref{sec:know} explains how to use domain knowledge for designing a pseudo-predictor in low-dose CT and camera imaging applications with practical noise models, and proposes an empirical pseudo-predictor selection approach for image denoising if domain knowledge of specific applications is unavailable. &#10;Section~\ref{sec:result} demonstrates the outperforming performance of the proposed SSRL approach over the several existing self-supervised denoising methods, Noise2Self \cite{Batson&amp;Royer:19ICML}, Noise2Inverse \cite{Hendriksen&amp;etal:20TCI} or Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, Noisy-As-Claen~\cite{Xu&amp;etal:20TIP}, and Self2Self~\cite{Quan&amp;etal:20CVPR}, with low-dose CT and camera image denoising experiments using both real and synthetic datasets.&#10;In addition, Section~\ref{sec:result} provides some discussions based on experimental results therein.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Computer Science, Regression Analysis, Self-Supervised Learning, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1501.00092" label="1501.00092">
-        <attvalues>
-          <attvalue for="0" value="Image Super-Resolution Using Deep Convolutional Networks" />
-          <attvalue for="1" value="  We propose a deep learning method for single image super-resolution (SR). Our&#10;method directly learns an end-to-end mapping between the low/high-resolution&#10;images. The mapping is represented as a deep convolutional neural network (CNN)&#10;that takes the low-resolution image as the input and outputs the&#10;high-resolution one. We further show that traditional sparse-coding-based SR&#10;methods can also be viewed as a deep convolutional network. But unlike&#10;traditional methods that handle each component separately, our method jointly&#10;optimizes all layers. Our deep CNN has a lightweight structure, yet&#10;demonstrates state-of-the-art restoration quality, and achieves fast speed for&#10;practical on-line usage. We explore different network structures and parameter&#10;settings to achieve trade-offs between performance and speed. Moreover, we&#10;extend our network to cope with three color channels simultaneously, and show&#10;better overall reconstruction quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.07316" label="1604.07316">
-        <attvalues>
-          <attvalue for="0" value="End to End Learning for Self-Driving Cars" />
-          <attvalue for="1" value="  We trained a convolutional neural network (CNN) to map raw pixels from a&#10;single front-facing camera directly to steering commands. This end-to-end&#10;approach proved surprisingly powerful. With minimum training data from humans&#10;the system learns to drive in traffic on local roads with or without lane&#10;markings and on highways. It also operates in areas with unclear visual&#10;guidance such as in parking lots and on unpaved roads.&#10;  The system automatically learns internal representations of the necessary&#10;processing steps such as detecting useful road features with only the human&#10;steering angle as the training signal. We never explicitly trained it to&#10;detect, for example, the outline of roads.&#10;  Compared to explicit decomposition of the problem, such as lane marking&#10;detection, path planning, and control, our end-to-end system optimizes all&#10;processing steps simultaneously. We argue that this will eventually lead to&#10;better performance and smaller systems. Better performance will result because&#10;the internal components self-optimize to maximize overall system performance,&#10;instead of optimizing human-selected intermediate criteria, e.g., lane&#10;detection. Such criteria understandably are selected for ease of human&#10;interpretation which doesn't automatically guarantee maximum system&#10;performance. Smaller networks are possible because the system learns to solve&#10;the problem with the minimal number of processing steps.&#10;  We used an NVIDIA DevBox and Torch 7 for training and an NVIDIA DRIVE(TM) PX&#10;self-driving car computer also running Torch 7 for determining where to drive.&#10;The system operates at 30 frames per second (FPS).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.10925" label="1711.10925">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Prior" />
-          <attvalue for="1" value="  Deep convolutional networks have become a popular tool for image generation&#10;and restoration. Generally, their excellent performance is imputed to their&#10;ability to learn realistic image priors from a large number of example images.&#10;In this paper, we show that, on the contrary, the structure of a generator&#10;network is sufficient to capture a great deal of low-level image statistics&#10;prior to any learning. In order to do so, we show that a randomly-initialized&#10;neural network can be used as a handcrafted prior with excellent results in&#10;standard inverse problems such as denoising, super-resolution, and inpainting.&#10;Furthermore, the same prior can be used to invert deep neural representations&#10;to diagnose them, and to restore images based on flash-no flash input pairs.&#10;  Apart from its diverse applications, our approach highlights the inductive&#10;bias captured by standard generator network architectures. It also bridges the&#10;gap between two very popular families of image restoration methods:&#10;learning-based methods using deep convolutional networks and learning-free&#10;methods based on handcrafted image priors such as self-similarity. Code and&#10;supplementary material are available at&#10;https://dmitryulyanov.github.io/deep_image_prior .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.11365" label="1901.11365">
-        <attvalues>
-          <attvalue for="0" value="Noise2Self: Blind Denoising by Self-Supervision" />
-          <attvalue for="1" value="  We propose a general framework for denoising high-dimensional measurements&#10;which requires no prior on the signal, no estimate of the noise, and no clean&#10;training data. The only assumption is that the noise exhibits statistical&#10;independence across different dimensions of the measurement, while the true&#10;signal exhibits some correlation. For a broad class of functions&#10;(&quot;$\mathcal{J}$-invariant&quot;), it is then possible to estimate the performance of&#10;a denoiser from noisy data alone. This allows us to calibrate&#10;$\mathcal{J}$-invariant versions of any parameterised denoising algorithm, from&#10;the single hyperparameter of a median filter to the millions of weights of a&#10;deep neural network. We demonstrate this on natural image and microscopy data,&#10;where we exploit noise independence between pixels, and on single-cell gene&#10;expression data, where we exploit independence between detections of individual&#10;molecules. This framework generalizes recent work on training neural nets from&#10;noisy images and on cross-validation for matrix factorization.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.10980" label="1811.10980">
-        <attvalues>
-          <attvalue for="0" value="Noise2Void - Learning Denoising from Single Noisy Images" />
-          <attvalue for="1" value="  The field of image denoising is currently dominated by discriminative deep&#10;learning methods that are trained on pairs of noisy input and clean target&#10;images. Recently it has been shown that such methods can also be trained&#10;without clean targets. Instead, independent pairs of noisy images can be used,&#10;in an approach known as Noise2Noise (N2N). Here, we introduce Noise2Void (N2V),&#10;a training scheme that takes this idea one step further. It does not require&#10;noisy image pairs, nor clean target images. Consequently, N2V allows us to&#10;train directly on the body of data to be denoised and can therefore be applied&#10;when other methods cannot. Especially interesting is the application to&#10;biomedical image data, where the acquisition of training targets, clean or&#10;noisy, is frequently not possible. We compare the performance of N2V to&#10;approaches that have either clean target images and/or noisy image pairs&#10;available. Intuitively, N2V cannot be expected to outperform methods that have&#10;more information available during training. Still, we observe that the&#10;denoising performance of Noise2Void drops in moderation and compares favorably&#10;to training-free denoising methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.11971" label="2010.11971">
-        <attvalues>
-          <attvalue for="0" value="Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising" />
-          <attvalue for="1" value="  Self-supervised frameworks that learn denoising models with merely individual&#10;noisy images have shown strong capability and promising performance in various&#10;image denoising tasks. Existing self-supervised denoising frameworks are mostly&#10;built upon the same theoretical foundation, where the denoising models are&#10;required to be J-invariant. However, our analyses indicate that the current&#10;theory and the J-invariance may lead to denoising models with reduced&#10;performance. In this work, we introduce Noise2Same, a novel self-supervised&#10;denoising framework. In Noise2Same, a new self-supervised loss is proposed by&#10;deriving a self-supervised upper bound of the typical supervised loss. In&#10;particular, Noise2Same requires neither J-invariance nor extra information&#10;about the noise model and can be used in a wider range of denoising&#10;applications. We analyze our proposed Noise2Same both theoretically and&#10;experimentally. The experimental results show that our Noise2Same remarkably&#10;outperforms previous self-supervised denoising methods in terms of denoising&#10;performance and training efficiency. Our code is available at&#10;https://github.com/divelab/Noise2Same.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.04189" label="1803.04189">
-        <attvalues>
-          <attvalue for="0" value="Noise2Noise: Learning Image Restoration without Clean Data" />
-          <attvalue for="1" value="  We apply basic statistical reasoning to signal reconstruction by machine&#10;learning -- learning to map corrupted observations to clean signals -- with a&#10;simple and powerful conclusion: it is possible to learn to restore images by&#10;only looking at corrupted examples, at performance at and sometimes exceeding&#10;training using clean data, without explicit image priors or likelihood models&#10;of the corruption. In practice, we show that a single model learns photographic&#10;noise removal, denoising synthetic Monte Carlo images, and reconstruction of&#10;undersampled MRI scans -- all corrupted by different processes -- based on&#10;noisy data only.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.11801" label="2001.11801">
-        <attvalues>
-          <attvalue for="0" value="Noise2Inverse: Self-supervised deep convolutional denoising for&#10;  tomography" />
-          <attvalue for="1" value="  Recovering a high-quality image from noisy indirect measurements is an&#10;important problem with many applications. For such inverse problems, supervised&#10;deep convolutional neural network (CNN)-based denoising methods have shown&#10;strong results, but the success of these supervised methods critically depends&#10;on the availability of a high-quality training dataset of similar measurements.&#10;For image denoising, methods are available that enable training without a&#10;separate training dataset by assuming that the noise in two different pixels is&#10;uncorrelated. However, this assumption does not hold for inverse problems,&#10;resulting in artifacts in the denoised images produced by existing methods.&#10;Here, we propose Noise2Inverse, a deep CNN-based denoising method for linear&#10;image reconstruction algorithms that does not require any additional clean or&#10;noisy data. Training a CNN-based denoiser is enabled by exploiting the noise&#10;model to compute multiple statistically independent reconstructions. We develop&#10;a theoretical framework which shows that such training indeed obtains a&#10;denoising CNN, assuming the measured noise is element-wise independent and&#10;zero-mean. On simulated CT datasets, Noise2Inverse demonstrates an improvement&#10;in peak signal-to-noise ratio and structural similarity index compared to&#10;state-of-the-art image denoising methods and conventional reconstruction&#10;methods, such as Total-Variation Minimization. We also demonstrate that the&#10;method is able to significantly reduce noise in challenging real-world&#10;experimental datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.02824" label="2101.02824">
-        <attvalues>
-          <attvalue for="0" value="Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images" />
-          <attvalue for="1" value="  In the last few years, image denoising has benefited a lot from the fast&#10;development of neural networks. However, the requirement of large amounts of&#10;noisy-clean image pairs for supervision limits the wide use of these models.&#10;Although there have been a few attempts in training an image denoising model&#10;with only single noisy images, existing self-supervised denoising approaches&#10;suffer from inefficient network training, loss of useful information, or&#10;dependence on noise modeling. In this paper, we present a very simple yet&#10;effective method named Neighbor2Neighbor to train an effective image denoising&#10;model with only noisy images. Firstly, a random neighbor sub-sampler is&#10;proposed for the generation of training image pairs. In detail, input and&#10;target used to train a network are images sub-sampled from the same noisy&#10;image, satisfying the requirement that paired pixels of paired images are&#10;neighbors and have very similar appearance with each other. Secondly, a&#10;denoising network is trained on sub-sampled training pairs generated in the&#10;first stage, with a proposed regularizer as additional loss for better&#10;performance. The proposed Neighbor2Neighbor framework is able to enjoy the&#10;progress of state-of-the-art supervised denoising networks in network&#10;architecture design. Moreover, it avoids heavy dependence on the assumption of&#10;the noise distribution. We explain our approach from a theoretical perspective&#10;and further validate it through extensive experiments, including synthetic&#10;experiments with different noise distributions in sRGB space and real-world&#10;experiments on a denoising benchmark dataset in raw-RGB space.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.01314" label="1803.01314">
-        <attvalues>
-          <attvalue for="0" value="Training Deep Learning Based Denoisers without Ground Truth Data" />
-          <attvalue for="1" value="  Recently developed deep-learning-based denoisers often outperform&#10;state-of-the-art conventional denoisers such as the BM3D. They are typically&#10;trained to minimize the mean squared error (MSE) between the output image of a&#10;deep neural network (DNN) and a ground truth image. Thus, it is important for&#10;deep-learning-based denoisers to use high quality noiseless ground truth data&#10;for high performance. However, it is often challenging or even infeasible to&#10;obtain noiseless images in some applications. Here, we propose a method based&#10;on Stein's unbiased risk estimator (SURE) for training DNN denoisers based only&#10;on the use of noisy images in the training data with Gaussian noise. We&#10;demonstrate that our SURE-based method, without the use of ground truth data,&#10;is able to train DNN denoisers to yield performances close to those networks&#10;trained with ground truth for both grayscale and color images. We also propose&#10;a SURE-based refining method with a noisy test image for further performance&#10;improvement. Our quick refining method outperformed conventional BM3D, deep&#10;image prior, and often the networks trained with ground truth. Potential&#10;extension of our SURE-based methods to Poisson noise model was also&#10;investigated.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.00651" label="1906.00651">
-        <attvalues>
-          <attvalue for="0" value="Probabilistic Noise2Void: Unsupervised Content-Aware Denoising" />
-          <attvalue for="1" value="  Today, Convolutional Neural Networks (CNNs) are the leading method for image&#10;denoising. They are traditionally trained on pairs of images, which are often&#10;hard to obtain for practical applications. This motivates self-supervised&#10;training methods such as Noise2Void~(N2V) that operate on single noisy images.&#10;Self-supervised methods are, unfortunately, not competitive with models trained&#10;on image pairs. Here, we present 'Probabilistic Noise2Void' (PN2V), a method to&#10;train CNNs to predict per-pixel intensity distributions. Combining these with a&#10;suitable description of the noise, we obtain a complete probabilistic model for&#10;the noisy observations and true signal in every pixel. We evaluate PN2V on&#10;publicly available microscopy datasets, under a broad range of noise regimes,&#10;and achieve competitive results with respect to supervised state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.06072" label="2006.06072">
-        <attvalues>
-          <attvalue for="0" value="Fully Unsupervised Diversity Denoising with Convolutional Variational&#10;  Autoencoders" />
-          <attvalue for="1" value="  Deep Learning based methods have emerged as the indisputable leaders for&#10;virtually all image restoration tasks. Especially in the domain of microscopy&#10;images, various content-aware image restoration (CARE) approaches are now used&#10;to improve the interpretability of acquired data. Naturally, there are&#10;limitations to what can be restored in corrupted images, and like for all&#10;inverse problems, many potential solutions exist, and one of them must be&#10;chosen. Here, we propose DivNoising, a denoising approach based on fully&#10;convolutional variational autoencoders (VAEs), overcoming the problem of having&#10;to choose a single solution by predicting a whole distribution of denoised&#10;images. First we introduce a principled way of formulating the unsupervised&#10;denoising problem within the VAE framework by explicitly incorporating imaging&#10;noise models into the decoder. Our approach is fully unsupervised, only&#10;requiring noisy images and a suitable description of the imaging noise&#10;distribution. We show that such a noise model can either be measured,&#10;bootstrapped from noisy data, or co-learned during training. If desired,&#10;consensus predictions can be inferred from a set of DivNoising predictions,&#10;leading to competitive results with other unsupervised methods and, on&#10;occasion, even with the supervised state-of-the-art. DivNoising samples from&#10;the posterior enable a plethora of useful applications. We are (i) showing&#10;denoising results for 13 datasets, (ii) discussing how optical character&#10;recognition (OCR) applications can benefit from diverse predictions, and are&#10;(iii) demonstrating how instance cell segmentation improves when using diverse&#10;DivNoising predictions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.00272" label="2403.00272">
-        <attvalues>
-          <attvalue for="0" value="Dual Pose-invariant Embeddings: Learning Category and Object-specific&#10;  Discriminative Representations for Recognition and Retrieval" />
-          <attvalue for="1" value="  In the context of pose-invariant object recognition and retrieval, we&#10;demonstrate that it is possible to achieve significant improvements in&#10;performance if both the category-based and the object-identity-based embeddings&#10;are learned simultaneously during training. In hindsight, that sounds intuitive&#10;because learning about the categories is more fundamental than learning about&#10;the individual objects that correspond to those categories. However, to the&#10;best of what we know, no prior work in pose-invariant learning has demonstrated&#10;this effect. This paper presents an attention-based dual-encoder architecture&#10;with specially designed loss functions that optimize the inter- and intra-class&#10;distances simultaneously in two different embedding spaces, one for the&#10;category embeddings and the other for the object-level embeddings. The loss&#10;functions we have proposed are pose-invariant ranking losses that are designed&#10;to minimize the intra-class distances and maximize the inter-class distances in&#10;the dual representation spaces. We demonstrate the power of our approach with&#10;three challenging multi-view datasets, ModelNet-40, ObjectPI, and FG3D. With&#10;our dual approach, for single-view object recognition, we outperform the&#10;previous best by 20.0% on ModelNet40, 2.0% on ObjectPI, and 46.5% on FG3D. On&#10;the other hand, for single-view object retrieval, we outperform the previous&#10;best by 33.7% on ModelNet40, 18.8% on ObjectPI, and 56.9% on FG3D.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pose-invariant recognition and retrieval \cite{PIE2019} is an&#10;important problem in computer vision with practical applications in&#10;robotic automation, automatic checkout systems, and inventory&#10;management. The appearance of many objects belonging to the same&#10;general category can vary significantly from different viewpoints,&#10;and, yet, humans have no difficulty in recognizing them from arbitrary&#10;viewpoints. In pose-invariant recognition and retrieval, the focus is&#10;on mapping the object images to embedding vectors such that the&#10;embeddings for the objects that belong to the same category are pulled&#10;together for all the available viewpoints in relation to the&#10;embeddings for the objects for the different categories.&#10;&#10;Our work demonstrates that the performance of&#10;pose-invariant learning as described above can be significantly&#10;improved if we disentangle the category-based learning from the&#10;object-identity-based learning.&#10;&#10;Fig. \ref{fig:intr_diag} illustrates what we mean by disentangling the&#10;category-based representation from the object-identity-based&#10;representation. Assume that an object database contains images of&#10;different types of chairs and different types of stools. We would want&#10;our network to learn the category-based embedding vectors for the&#10;chair class and for the stool class. These embeddings need to capture&#10;what is maximally discriminating between the chairs and the stools ---&#10;the presence or the absence of a back-rest. At the same time, we&#10;would want the network to learn object-identity based embeddings.&#10;These embeddings should represent what is distinctive about each&#10;object type in relation to all other objects types in the same&#10;category. For example, in the chair category, we would want the&#10;network to be able to discriminate between, say, lounge chairs and&#10;desk chairs.&#10;&#10;Prior work \cite{mvcnn15, RotationNet2018, TCL2018, ModelNet40} has&#10;employed multi-view deep networks to learn aggregated multi-view&#10;representations capturing the variability in object appearance under&#10;different pose transformations. While these methods demonstrate good&#10;performance in category-level tasks when multiple views of objects are&#10;available during inference, {their performance degrades when only&#10; a single view is available}. Since real-world applications often&#10;necessitate inference from single views, Ho et al. \cite{PIE2019}&#10;proposed a family of pose-invariant embeddings for both recognition&#10;and retrieval by imposing constraints such that the single-view&#10;embeddings of an object are clustered around its multi-view&#10;embeddings, which in turn are clustered around a proxy embedding&#10;representing the associated high-level category that the object&#10;belongs to. However, this approach does not do a good enough job of&#10;separating the embeddings for two different objects that belong to the&#10;same category (e.g., two different types of chairs, two different&#10;types of kettles, etc.), As a result, prior approaches perform well on&#10;category-level tasks but not on object-level tasks, as we will&#10;demonstrate later in our experimental results (see Tables&#10;\ref{tbl:resultsPICR}, \ref{tbl:fg3d_comp}).&#10;&#10;Here is arguably the most significant difference between the previous&#10;methods and the one being proposed in this paper: {Rather than&#10; learning representations that capture both category-specific and&#10; object-specific discriminative features within the same embedding&#10; space, we simultaneously learn them in two distinct embedding&#10; spaces}, as depicted in the lower panel in&#10;Fig. \ref{fig:intr_diag}. In one space that is devoted to&#10;category-based representations, objects from the same category can be&#10;closely embedded together, capturing shared characteristics among&#10;them, while in the other space, the one for object identity-based&#10;representations, embeddings for the different object types (within&#10;the same category or otherwise) are allowed to be as separated as&#10;dictated by the attributes that differentiate them. This strategy&#10;enables our network to learn object representations that are more&#10;discriminative overall. This should explain the superior performance&#10;of our framework in both recognition and retrieval, especially for the&#10;more difficult case when only a single-viewpoint query image is&#10;available. For single-view&#10;object recognition, we get an improvement in accuracy of 20.0\% on&#10;ModelNet40, 46.5\% on FG3D, and 2.0\% on ObjectPI. Along the same lines, for the case&#10;of single-view object retrieval, we achieve a significant mAP&#10;improvement of 33.7\% on ModelNet-40, 56.9\% on FG3D, and 18.8\% on ObjectPI datasets.&#10;&#10;In order to learn the dual embeddings simultaneously, we propose an&#10;encoder that we refer to as the Pose-invariant Attention Network&#10;(PAN). PAN uses a shared CNN backbone for capturing visual features&#10;common to both the category and the object-identity based&#10;representations from a set of images of an object recorded from&#10;different viewpoints. The visual features are then mapped to separate&#10;low-dimensional category and object-identity based embeddings using&#10;two fully connected layers. PAN also aggregates visual features of&#10;objects from different views using self-attention to generate what we&#10;call multi-view embeddings. The dual embeddings, defined in Section&#10;\ref{sec:approach}, can be used for {both category and&#10; object-level recognition and retrieval from single and multiple&#10; views.}&#10;&#10;For training the network, we propose two pose-invariant category and&#10;object-identity based losses that are jointly optimized to learn the&#10;dual embeddings. The pose-invariant category loss clusters together&#10;the instances of different objects belonging to the same category&#10;while separating apart the instances from different categories in the&#10;category embedding space. On the other hand, the pose-invariant&#10;object-identity based loss clusters together the instances that carry&#10;the same object-identity label and separates what would otherwise be&#10;mutually confusing object instances with two different object-identity&#10;labels from the same category in the object embedding space.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Object Recognition, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2404.17170" label="2404.17170">
-        <attvalues>
-          <attvalue for="0" value="Image Quality Assessment With Compressed Sampling" />
-          <attvalue for="1" value="  No-Reference Image Quality Assessment (NR-IQA) aims at estimating image&#10;quality in accordance with subjective human perception. However, most methods&#10;focus on exploring increasingly complex networks to improve the final&#10;performance,accompanied by limitations on input images. Especially when applied&#10;to high-resolution (HR) images, these methods offen have to adjust the size of&#10;original image to meet model input.To further alleviate the aforementioned&#10;issue, we propose two networks for NR-IQA with Compressive Sampling (dubbed&#10;CL-IQA and CS-IQA). They consist of four components: (1) The Compressed&#10;Sampling Module (CSM) to sample the image (2)The Adaptive Embedding Module&#10;(AEM). The measurements are embedded by AEM to extract high-level features. (3)&#10;The Vision Transformer and Scale Swin TranBlocksformer Moudle(SSTM) to extract&#10;deep features. (4) The Dual Branch (DB) to get final quality score. Experiments&#10;show that our proposed methods outperform other methods on various datasets&#10;with less data usage.&#10;" />
-          <attvalue for="2" value="&#10;While utilizing images from the internet, we should be mindful of their potential transformation journey. These images might have been captured by a camera, compressed for sharing online and disseminated widely before finally reaching us. These low-quality images can affect the our visual feelings and even cause deadly problems, especially in autonomous driving. For the reason, it is crucial to predict the perceptual image quality in our daily lives.&#10;&#10;Objective Image Quality Assessment (IQA) is an approach that uses computational models to ascertain the perceived quality of an image from a human perspective. The objective quality metrics can be categorized into two based on whether a lossless reference image is available: full-reference (FR-IQA)\cite{6562812, 7934456, 6247789, 6894484} and no-reference (NR-IQA). Unlike FR-IQA, NR-IQA\cite{6272356, 6172573, 7094273} does not have access to reference images which is harder and more widely used. With the success of deep learning and convolution neural networks (CNNs) in computer vision tasks, CNN-based methods have significantly outperformed traditional approaches in handling real-world distortions. Early deep learning NR-IQA methods employed stacked CNNs for feature extraction. Ma\cite{8110690} proposes a multi-task network where two sub-networks are trained for distortion identification and quality assessment. Hyper-IQA\cite{9156687} leverages both low-level and high-level features and makes the latter redirect the former. Furthermore, Zhu\cite{9156932} proposes a model that employs meta-learning to capture shared prior knowledge among different distortions. But in the era of high-resolution Image Quality Assessment, both methods are faced with substantial computational challenges while processing entire images. There already exist some data-efficient methods for quality assessment, such as FAST-VQA\cite{Wu2022FASTVQAEE} with random-crop. But cropping cannot guarantee that the remained part matters. &#10;&#10;Compressive sensing is a novel method that breaks through the limitations of the Nyquist theorem on signal sampling\cite{1614066, 4472240}. This technique shows that one can faithfully reconstruct an entire image from a minimal amount of measurements compared to the original image, thus significantly simplifying the sampling process. &#10;&#10;The concept of compressive learning (CL) was first proposed by Calderbank et al.\cite{article} and Davenport et al.\cite{10.1117/12.714460} when they built the inference system directly using measurements without reconstruction. Tran\cite{9070152} makes some theoretical works and gradually finds some multidimensional properties of CL. A recent work \cite{9841016} by Chong Mou et al. employs transformer as the backbone to cope with information loss through element-wise correlations and achieves amazing performance. &#10;&#10;In many applications, such as NR-IQA, the precise reconstructed image isn't a primary concern. Still, we emphasize more on the final score, which means we can directly perform high-level tasks after initialization or even on the compressed domain.Inspired by this work, to make NR-IQA more data-efficient and extract the feature of the original image globally, we introduce compressive sensing in NR-IQA task.We propose CL-IQA that integrates compressive learning with the NR-IQA task and CS-IQA which skip the AEM block of CL-IQA. They share the same backbone. Firstly, CSM allow us obtain measurements at an arbitrary ratio. Secondly, the measurements are adaptive embedded by AEM. Thirdly, the measurements or embedded measurements were fed into the vision transformer and SSTM to extract deep features. Finally, a dual-branch structure is employed for the score prediction. Our main contributions are as follows: &#10;&#10;$\bullet$ We apply compressive learning to image quality assessment with, enable sampling of images at arbitrary ratios. To our knowledge, CL-IQA is the first work integrating compressive learning and image quality assessment.&#10;&#10;$\bullet$ We pretrain the CSM with lightweight compressive sensing network enable it captures the feature about quality assessment more effectively. The proposed CS-IQA significantly reduce the complexity by skip the AEM module.&#10;&#10;$\bullet$ Experiments show that CL-IQA and CS-IQA outperforms state-of-the-art IQA methods with benefiting from the sampling modules.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Image Quality Assessment, Computer Science, Deep Learning Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2004.05508" label="2004.05508">
-        <attvalues>
-          <attvalue for="0" value="MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment" />
-          <attvalue for="1" value="  Recently, increasing interest has been drawn in exploiting deep convolutional&#10;neural networks (DCNNs) for no-reference image quality assessment (NR-IQA).&#10;Despite of the notable success achieved, there is a broad consensus that&#10;training DCNNs heavily relies on massive annotated data. Unfortunately, IQA is&#10;a typical small sample problem. Therefore, most of the existing DCNN-based IQA&#10;metrics operate based on pre-trained networks. However, these pre-trained&#10;networks are not designed for IQA task, leading to generalization problem when&#10;evaluating different types of distortions. With this motivation, this paper&#10;presents a no-reference IQA metric based on deep meta-learning. The underlying&#10;idea is to learn the meta-knowledge shared by human when evaluating the quality&#10;of images with various distortions, which can then be adapted to unknown&#10;distortions easily. Specifically, we first collect a number of NR-IQA tasks for&#10;different distortions. Then meta-learning is adopted to learn the prior&#10;knowledge shared by diversified distortions. Finally, the quality prior model&#10;is fine-tuned on a target NR-IQA task for quickly obtaining the quality model.&#10;Extensive experiments demonstrate that the proposed metric outperforms the&#10;state-of-the-arts by a large margin. Furthermore, the meta-model learned from&#10;synthetic distortions can also be easily generalized to authentic distortions,&#10;which is highly desired in real-world applications of IQA metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.02595" label="2207.02595">
-        <attvalues>
-          <attvalue for="0" value="FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment&#10;  Sampling" />
-          <attvalue for="1" value="  Current deep video quality assessment (VQA) methods are usually with high&#10;computational costs when evaluating high-resolution videos. This cost hinders&#10;them from learning better video-quality-related representations via end-to-end&#10;training. Existing approaches typically consider naive sampling to reduce the&#10;computational cost, such as resizing and cropping. However, they obviously&#10;corrupt quality-related information in videos and are thus not optimal for&#10;learning good representations for VQA. Therefore, there is an eager need to&#10;design a new quality-retained sampling scheme for VQA. In this paper, we&#10;propose Grid Mini-patch Sampling (GMS), which allows consideration of local&#10;quality by sampling patches at their raw resolution and covers global quality&#10;with contextual relations via mini-patches sampled in uniform grids. These&#10;mini-patches are spliced and aligned temporally, named as fragments. We further&#10;build the Fragment Attention Network (FANet) specially designed to accommodate&#10;fragments as inputs. Consisting of fragments and FANet, the proposed FrAgment&#10;Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and&#10;learns effective video-quality-related representations. It improves&#10;state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P&#10;high-resolution videos. The newly learned video-quality-related representations&#10;can also be transferred into smaller VQA datasets, boosting performance in&#10;these scenarios. Extensive experiments show that FAST-VQA has good performance&#10;on inputs of various resolutions while retaining high efficiency. We publish&#10;our code at https://github.com/timothyhtimothy/FAST-VQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.07481" label="1905.07481">
-        <attvalues>
-          <attvalue for="0" value="Multilinear Compressive Learning" />
-          <attvalue for="1" value="  Compressive Learning is an emerging topic that combines signal acquisition&#10;via compressive sensing and machine learning to perform inference tasks&#10;directly on a small number of measurements. Many data modalities naturally have&#10;a multi-dimensional or tensorial format, with each dimension or tensor mode&#10;representing different features such as the spatial and temporal information in&#10;video sequences or the spatial and spectral information in hyperspectral&#10;images. However, in existing compressive learning frameworks, the compressive&#10;sensing component utilizes either random or learned linear projection on the&#10;vectorized signal to perform signal acquisition, thus discarding the&#10;multi-dimensional structure of the signals. In this paper, we propose&#10;Multilinear Compressive Learning, a framework that takes into account the&#10;tensorial nature of multi-dimensional signals in the acquisition step and&#10;builds the subsequent inference model on the structurally sensed measurements.&#10;Our theoretical complexity analysis shows that the proposed framework is more&#10;efficient compared to its vector-based counterpart in both memory and&#10;computation requirement. With extensive experiments, we also empirically show&#10;that our Multilinear Compressive Learning framework outperforms the&#10;vector-based framework in object classification and face recognition tasks, and&#10;scales favorably when the dimensionalities of the original signals increase,&#10;making it highly efficient for high-dimensional multi-dimensional signals.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.11972" label="2207.11972">
-        <attvalues>
-          <attvalue for="0" value="TransCL: Transformer Makes Strong and Flexible Compressive Learning" />
-          <attvalue for="1" value="  Compressive learning (CL) is an emerging framework that integrates signal&#10;acquisition via compressed sensing (CS) and machine learning for inference&#10;tasks directly on a small number of measurements. It can be a promising&#10;alternative to classical image-domain methods and enjoys great advantages in&#10;memory saving and computational efficiency. However, previous attempts on CL&#10;are not only limited to a fixed CS ratio, which lacks flexibility, but also&#10;limited to MNIST/CIFAR-like datasets and do not scale to complex real-world&#10;high-resolution (HR) data or vision tasks. In this paper, a novel&#10;transformer-based compressive learning framework on large-scale images with&#10;arbitrary CS ratios, dubbed TransCL, is proposed. Specifically, TransCL first&#10;utilizes the strategy of learnable block-based compressed sensing and proposes&#10;a flexible linear projection strategy to enable CL to be performed on&#10;large-scale images in an efficient block-by-block manner with arbitrary CS&#10;ratios. Then, regarding CS measurements from all blocks as a sequence, a pure&#10;transformer-based backbone is deployed to perform vision tasks with various&#10;task-oriented heads. Our sufficient analysis presents that TransCL exhibits&#10;strong resistance to interference and robust adaptability to arbitrary CS&#10;ratios. Extensive experiments for complex HR data demonstrate that the proposed&#10;TransCL can achieve state-of-the-art performance in image classification and&#10;semantic segmentation tasks. In particular, TransCL with a CS ratio of $10\%$&#10;can obtain almost the same performance as when operating directly on the&#10;original data and can still obtain satisfying performance even with an&#10;extremely low CS ratio of $1\%$. The source codes of our proposed TransCL is&#10;available at \url{https://github.com/MC-E/TransCL/}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.03326" label="1907.03326">
-        <attvalues>
-          <attvalue for="0" value="Spacetime Graph Optimization for Video Object Segmentation" />
-          <attvalue for="1" value="  We address the challenging task of foreground object discovery and&#10;segmentation in video. We introduce an efficient solution, suitable for both&#10;unsupervised and supervised scenarios, based on a spacetime graph&#10;representation of the video sequence. We ensure a fine grained representation&#10;with one-to-one correspondences between graph nodes and video pixels. We&#10;formulate the task as a spectral clustering problem by exploiting the&#10;spatio-temporal consistency between the scene elements in terms of motion and&#10;appearance. Graph nodes that belong to the main object of interest should form&#10;a strong cluster, as they are linked through long range optical flow chains and&#10;have similar motion and appearance features along those chains. On one hand,&#10;the optimization problem aims to maximize the segmentation clustering score&#10;based on the motion structure through space and time. On the other hand, the&#10;segmentation should be consistent with respect to node features. Our approach&#10;leads to a graph formulation in which the segmentation solution becomes the&#10;principal eigenvector of a novel Feature-Motion matrix. While the actual matrix&#10;is not computed explicitly, the proposed algorithm efficiently computes, in a&#10;few iteration steps, the principal eigenvector that captures the segmentation&#10;of the main object in the video. The proposed algorithm, GO-VOS, produces a&#10;global optimum solution and, consequently, it does not depend on&#10;initialization. In practice, GO-VOS achieves state of the art results on three&#10;challenging datasets used in current literature: DAVIS, SegTrack and&#10;YouTube-Objects.&#10;" />
-          <attvalue for="2" value="&#10;Discovering objects in videos without human supervision, as they move and change appearance over space and time, is one of the most challenging, and still unsolved problems in computer vision. This impacts the way we learn about objects and how we process large amounts of video data, which is widely available at low cost. One of our core goals is to understand how much we could learn automatically from a video about the main object of interest. We are interested in how object properties relate in both space and time and how we could exploit these consistencies in order to discover the objects in a fast and accurate manner. While human segmentation annotations of the same video are not always in agreement, people tend to agree on which is the main object captured in a given video shot, when the camera is indeed focusing on a single object. There are many questions to answer: which are the particular features that make a group of pixels stand out as a single, main object in a given sequence? Is it the pattern of motion different from the surrounding background? Is it the contrast in appearance, the symmetry or good form of an object that make it stand out? Or is it a combination of such factors and maybe others, still unknown? What we do know is the fact that people can easily detect and segment the foreground object when the camera is focusing on a single object, and this fact has been recognized and studied since the early days of the Gestalt school of psychology \cite{koffka2013principles}.&#10;&#10;We make two main assumptions, which become the basis of our approach: 1)~pixels that belong to the same object are highly likely to be connected through long range optical flow chains, as we define them in Sec. \ref{sec:approach_space_time_graph}; 2)~pixels of the same object are also likely to have similar motion and distinctive appearance patterns in space and time. In other words, what looks alike and moves together, is likely to belong together. While these ideas are not new, we propose a novel graph structure in space and time, with motion and appearance constraints, in which segmentation is defined as a clustering problem &#10;in the spacetime graph, in which the global optimum is found as the &#10;the leading eigenvector of a novel Feature-Motion matrix (Sec. \ref{sec:alg_analysis_convergence}).&#10;&#10;Main contribution: We introduce a novel graph structure in space and time, with nodes at the dense pixel level such that each pixel is connected to other pixels through optical flow chains. The graph combines long range motion patterns with local appearance information at the dense pixel level, in a single spacetime graph defined by the Feature-Motion matrix (Sec \ref{sec:alg_analysis_convergence}). We define segmentation (Problems 1 and 2 - Eq.~\ref{optimization_problem} and Eq.~\ref{optimization_problem_alg}) as a spectral clustering problem and&#10;propose a fast algorithm that computes the global optimum as the principal eigenvector of the Feature-Motion matrix. One of the main tricks is that the matrix is never computed explicitly, such that the algorithm is fast and also accurate, with state of the art performance on three challenging benchmarks in the literature.&#10;&#10;Related work: One of the most important aspects that differentiates between different approaches in foreground object segmentation in video is the amount of supervision used, which could vary from complete absence of any human annotations \cite{lao2018extending, papazoglou2013fast, keuper2015motion, faktor2014video, haller2017unsupervised} to using models pretrained for video object segmentation \cite{luiten2018premvos, maninis2017video, voigtlaender2017online, bao2018cnn, wug2018fast, cheng2018fast, caelles2017one, perazzi2017learning, chen2018blazingly, song2018pyramid, tokmakov2017learning, jain2017fusionseg} or having access to the human annotation for the first video frame \cite{Perazzi2016}. &#10;We propose a video object segmentation framework that has the ability to accommodate both supervised and unsupervised scenarios. In this paper we focus on the unsupervised case, for which no human supervision was used during training. &#10;&#10;Our approach uses optical flow in order to define the graph structure at the dense pixel level. There are several notable works that use optical flow or a graph representation for tasks related to video object segmentation (VOS). Brox and Malik introduce in \cite{brox2010object} a motion clustering method that simultaneous estimates optical flow and object segmentation masks. They operate on pairs of images and minimize an energy function incorporating classical optical flow constrains and a descriptor matching term, with no mechanism for selecting the main object. Another similar approach is proposed by Tsai \etal in \cite{tsai2016video}. Zhuo \etal \cite{zhuo2018unsupervised} build salient motion masks that are further combined with &quot;objectness&quot; masks in order to generate the final segmentation. Li \etal \cite{li2017primary} and Wang \etal \cite{wang2015saliency} introduce approaches that are based on averaging saliency masks over connections defined by optical flow. Keuper \etal introduce in \cite{keuper2015motion} a method for motion segmentation, where elements of the scene are clustered function of their motion patterns, with no focus on the main object. They formulate the task as a minimum cost multicut problem over a graph whose nodes are defined by motion trajectories. &#10;&#10;Our spacetime graph considers direct connections between all video pixels, in contrast to superpixel level nodes or trajectory nodes. In contrast to \cite{keuper2015motion, brox2010object} that perform motion segmentation, we discover the strongest cluster in space and time by taking in consideration both long range motion connections as well as local motion and appearance patterns. In contrast to other works in VOS, we provide a spectral clustering formulation with global solution that can be obtained very fast by power iteration as the leading eigenvector of a huge Feature-Motion matrix (of size $n \times n$, where $n$ is the total number of pixels in the video), which is never explicitly computed. Thus we ensure a dense consistency in space and time at the level of the whole video sequence, in contrast to local consistency that is ensured in most existing solutions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Graph-Based Clustering, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Video Object Segmentation, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2208.06888" label="2208.06888">
-        <attvalues>
-          <attvalue for="0" value="AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility" />
-          <attvalue for="1" value="  One of the key factors behind the recent success in visual tracking is the&#10;availability of dedicated benchmarks. While being greatly benefiting to the&#10;tracking research, existing benchmarks do not pose the same difficulty as&#10;before with recent trackers achieving higher performance mainly due to (i) the&#10;introduction of more sophisticated transformers-based methods and (ii) the lack&#10;of diverse scenarios with adverse visibility such as, severe weather&#10;conditions, camouflage and imaging effects.&#10;  We introduce AVisT, a dedicated benchmark for visual tracking in diverse&#10;scenarios with adverse visibility. AVisT comprises 120 challenging sequences&#10;with 80k annotated frames, spanning 18 diverse scenarios broadly grouped into&#10;five attributes with 42 object categories. The key contribution of AVisT is&#10;diverse and challenging scenarios covering severe weather conditions such as,&#10;dense fog, heavy rain and sandstorm; obstruction effects including, fire, sun&#10;glare and splashing water; adverse imaging effects such as, low-light; target&#10;effects including, small targets and distractor objects along with camouflage.&#10;We further benchmark 17 popular and recent trackers on AVisT with detailed&#10;analysis of their tracking performance across attributes, demonstrating a big&#10;room for improvement in performance. We believe that AVisT can greatly benefit&#10;the tracking community by complementing the existing benchmarks, in developing&#10;new creative tracking solutions in order to continue pushing the boundaries of&#10;the state-of-the-art. Our dataset along with the complete tracking performance&#10;evaluation is available at: https://github.com/visionml/pytracking&#10;" />
-          <attvalue for="2" value="&#10; Visual object tracking is one of the fundamental problems in computer vision, where the objective is to estimate the target state and trajectory in an image sequence, provided only its initial location. The target object is not known a priori and is not constrained to be from a specific object class. Therefore, the main challenge is to accurately learn the appearance of the target object in unconstrained real-world scenarios.&#10;&#10;Recent years have witnessed a significant progress in the field of visual tracking with a plethora of trackers introduced in the literature. One of the major contributing factors towards these recent advances in tracking is the introduction of several benchmarks~\cite{WU_2015_TPAMI_OTB, Galoogahi_2017_ICCV_NFS, 2018_Muller_Trackingnet, Huang_2021_TPAMI_GOT10k, Fan_2019_CVPR_Lasot}. OTB~\cite{WU_2015_TPAMI_OTB} was one of the first large-scale datasets, containing 100 videos. Afterwards, tracking benchmarks are introduced to evaluate different aspects of tracking such as, the impact of color information~\cite{TempleColor}, fast target motion~\cite{Galoogahi_2017_ICCV_NFS} as well as tracking in aerial imagery~\cite{Mueller_2016_ECCV_UAV123}. More recently, the tracking community has focused on constructing datasets~\cite{2018_Muller_Trackingnet, Fan_2019_CVPR_Lasot,Huang_2021_TPAMI_GOT10k} with large-scale training splits, to benefit from task-specific deep learning. Among these, GOT-10K~\cite{Huang_2021_TPAMI_GOT10k} comprises a large collection of shorter videos, whereas LaSOT~\cite{Fan_2019_CVPR_Lasot} focuses on longer sequences. Moreover, there exist dedicated benchmarks, such as the VOT series~\cite{Kristan_2016_TPAMI_VOT} associated with annual tracking challenge competitions.&#10;&#10;While all of these datasets have greatly benefited the tracking research, they no longer pose the same difficulty as before to the current state-of-the-art trackers, due to the rapid progress in the field.&#10;Most notably, the state-of-the-art trackers now achieve AUC scores of above 70\% also on LaSOT (see Tab.~\ref{tab:dataset-comparison}), which is one of the most difficult established datasets. On the other hand, since the introduction of OTB in 2013~\cite{OTB13Ming}, the existence of highly difficult tracking benchmarks has been vital, in order to challenge researchers to designing ever more robust and accurate trackers, applicable for increasingly diverse scenarios. In this work, we therefore set out to develop a new, highly challenging dataset, in order to promote further progress in the visual tracking field. &#10;&#10;We believe that one of the main reasons that the aforementioned datasets do not pose sufficient challenge to new trackers is that diverse scenarios such as, adverse visibility due to weather conditions, camouflage and illumination effects are underrepresented. In practice, robust handling of adverse visibility is essential in many applications. For instance, autonomous driving applications require the target to be tracked under all weather conditions, such as heavy rain, dense fog, and sandstorms. Similarly, rescue missions involving drones require robust and accurate object tracking in adverse scenarios, such as fire, smoke, and strong winds. Further, wildlife conservation often relies on monitoring different animal populations in their natural habitats, where many animal species are difficult to distinguish from the surrounding environments due to their camouflaged appearance.&#10;Contributions: We propose AVisT, a benchmark for visual object tracking in diverse scenarios with adverse visibility. AVisT better accommodates the difficult conditions encountered in the aforementioned real-world applications, while being severely challenging even for the most recent trackers (see Tab.~\ref{tab:dataset-comparison}). Our dataset comprises 120 challenging sequences, spanning 18 diverse scenarios and 42 object categories. The scenarios cover adverse weather conditions, including heavy rain, dense fog, and hurricane; obstruction effects such as, splashing water, fire, sun glare, and smoke; adverse imaging effects; target effects such as, fast motion and small target; along with camouflage. The proposed AVisT is densely annotated with accurate bounding boxes following a thorough quality control. Moreover, every frame is annotated with flags for occlusion, partial occlusion, out of view, and extreme visibility.&#10;&#10;We evaluate 17 popular trackers on AVisT, including the most recent state-of-the-art methods. The best method, MixFormer-22k \cite{Cui_2022_CVPR_Mixformer} which employs an ImageNet-22K pre-trained backbone achieves an AUC score of only 56.0\%, demonstrating the challenging nature of AVisT. We further analyze the performance of different trackers across attributes, which can provide valuable insights for specific applications. For instance, we note that ImageNet-22K pre-training is important for improved performance on the weather conditions attribute. Fig.~\ref{fig:teaser} shows a qualitative comparison of recent trackers belonging to different tracking paradigms: discriminative classifiers, Siamese networks and transformers.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Visual Tracking, Computer Science, Benchmark Development" />
-        </attvalues>
-      </node>
-      <node id="1703.05884" label="1703.05884">
-        <attvalues>
-          <attvalue for="0" value="Need for Speed: A Benchmark for Higher Frame Rate Object Tracking" />
-          <attvalue for="1" value="  In this paper, we propose the first higher frame rate video dataset (called&#10;Need for Speed - NfS) and benchmark for visual object tracking. The dataset&#10;consists of 100 videos (380K frames) captured with now commonly available&#10;higher frame rate (240 FPS) cameras from real world scenarios. All frames are&#10;annotated with axis aligned bounding boxes and all sequences are manually&#10;labelled with nine visual attributes - such as occlusion, fast motion,&#10;background clutter, etc. Our benchmark provides an extensive evaluation of many&#10;recent and state-of-the-art trackers on higher frame rate sequences. We ranked&#10;each of these trackers according to their tracking accuracy and real-time&#10;performance. One of our surprising conclusions is that at higher frame rates,&#10;simple trackers such as correlation filters outperform complex methods based on&#10;deep networks. This suggests that for practical applications (such as in&#10;robotics or embedded vision), one needs to carefully tradeoff bandwidth&#10;constraints associated with higher frame rate acquisition, computational costs&#10;of real-time analysis, and the required application accuracy. Our dataset and&#10;benchmark allows for the first time (to our knowledge) systematic exploration&#10;of such issues, and will be made available to allow for further research in&#10;this space.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.10794" label="1803.10794">
-        <attvalues>
-          <attvalue for="0" value="TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in&#10;  the Wild" />
-          <attvalue for="1" value="  Despite the numerous developments in object tracking, further development of&#10;current tracking algorithms is limited by small and mostly saturated datasets.&#10;As a matter of fact, data-hungry trackers based on deep-learning currently rely&#10;on object detection datasets due to the scarcity of dedicated large-scale&#10;tracking datasets. In this work, we present TrackingNet, the first large-scale&#10;dataset and benchmark for object tracking in the wild. We provide more than 30K&#10;videos with more than 14 million dense bounding box annotations. Our dataset&#10;covers a wide selection of object classes in broad and diverse context. By&#10;releasing such a large-scale dataset, we expect deep trackers to further&#10;improve and generalize. In addition, we introduce a new benchmark composed of&#10;500 novel videos, modeled with a distribution similar to our training dataset.&#10;By sequestering the annotation of the test set and providing an online&#10;evaluation server, we provide a fair benchmark for future development of object&#10;trackers. Deep trackers fine-tuned on a fraction of our dataset improve their&#10;performance by up to 1.6% on OTB100 and up to 1.7% on TrackingNet Test. We&#10;provide an extensive benchmark on TrackingNet by evaluating more than 20&#10;trackers. Our results suggest that object tracking in the wild is far from&#10;being solved.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.11981" label="1810.11981">
-        <attvalues>
-          <attvalue for="0" value="GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in&#10;  the Wild" />
-          <attvalue for="1" value="  We introduce here a large tracking database that offers an unprecedentedly&#10;wide coverage of common moving objects in the wild, called GOT-10k.&#10;Specifically, GOT-10k is built upon the backbone of WordNet structure and it&#10;populates the majority of over 560 classes of moving objects and 87 motion&#10;patterns, magnitudes wider than the most recent similar-scale counterparts. The&#10;contributions of this paper are summarized in the following: (1) GOT-10k offers&#10;over 10,000 video segments with more than 1.5 million manually labeled bounding&#10;boxes, enabling unified training and stable evaluation of deep trackers. (2)&#10;GOT-10k is by far the first video trajectory dataset that uses the semantic&#10;hierarchy of WordNet to guide class population. (3) For the first time, GOT-10k&#10;introduces the one-shot protocol for tracker evaluation, where the training and&#10;test classes are zero-overlapped. The protocol avoids biased evaluation results&#10;towards familiar objects and it promotes generalization in tracker development.&#10;(4) We conduct extensive tracking experiments with 39 typical tracking&#10;algorithms on GOT-10k and analyze their results in this paper. (5) Finally, we&#10;develop a comprehensive platform for the tracking community that offers&#10;full-featured evaluation toolkits, an online evaluation server, and a&#10;responsive leaderboard. The annotations of GOT-10k's test data are kept private&#10;to avoid tuning parameters on it. The database, toolkits, evaluation server and&#10;baseline results are available at http://got-10k.aitestunion.com.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.07845" label="1809.07845">
-        <attvalues>
-          <attvalue for="0" value="LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking" />
-          <attvalue for="1" value="  In this paper, we present LaSOT, a high-quality benchmark for Large-scale&#10;Single Object Tracking. LaSOT consists of 1,400 sequences with more than 3.5M&#10;frames in total. Each frame in these sequences is carefully and manually&#10;annotated with a bounding box, making LaSOT the largest, to the best of our&#10;knowledge, densely annotated tracking benchmark. The average video length of&#10;LaSOT is more than 2,500 frames, and each sequence comprises various challenges&#10;deriving from the wild where target objects may disappear and re-appear again&#10;in the view. By releasing LaSOT, we expect to provide the community with a&#10;large-scale dedicated benchmark with high quality for both the training of deep&#10;trackers and the veritable evaluation of tracking algorithms. Moreover,&#10;considering the close connections of visual appearance and natural language, we&#10;enrich LaSOT by providing additional language specification, aiming at&#10;encouraging the exploration of natural linguistic feature for tracking. A&#10;thorough experimental evaluation of 35 tracking algorithms on LaSOT is&#10;presented with detailed analysis, and the results demonstrate that there is&#10;still a big room for improvements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.01313" label="1503.01313">
-        <attvalues>
-          <attvalue for="0" value="A Novel Performance Evaluation Methodology for Single-Target Trackers" />
-          <attvalue for="1" value="  This paper addresses the problem of single-target tracker performance&#10;evaluation. We consider the performance measures, the dataset and the&#10;evaluation system to be the most important components of tracker evaluation and&#10;propose requirements for each of them. The requirements are the basis of a new&#10;evaluation methodology that aims at a simple and easily interpretable tracker&#10;comparison. The ranking-based methodology addresses tracker equivalence in&#10;terms of statistical significance and practical differences. A fully-annotated&#10;dataset with per-frame annotations with several visual attributes is&#10;introduced. The diversity of its visual properties is maximized in a novel way&#10;by clustering a large number of videos according to their visual attributes.&#10;This makes it the most sophistically constructed and annotated dataset to date.&#10;A multi-platform evaluation system allowing easy integration of third-party&#10;trackers is presented as well. The proposed evaluation methodology was tested&#10;on the VOT2014 challenge on the new dataset and 38 trackers, making it the&#10;largest benchmark to date. Most of the tested trackers are indeed&#10;state-of-the-art since they outperform the standard baselines, resulting in a&#10;highly-challenging benchmark. An exhaustive analysis of the dataset from the&#10;perspective of tracking difficulty is carried out. To facilitate tracker&#10;comparison a new performance visualization technique is proposed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.03404" label="2409.03404">
-        <attvalues>
-          <attvalue for="0" value="KAN See In the Dark" />
-          <attvalue for="1" value="  Existing low-light image enhancement methods are difficult to fit the complex&#10;nonlinear relationship between normal and low-light images due to uneven&#10;illumination and noise effects. The recently proposed Kolmogorov-Arnold&#10;networks (KANs) feature spline-based convolutional layers and learnable&#10;activation functions, which can effectively capture nonlinear dependencies. In&#10;this paper, we design a KAN-Block based on KANs and innovatively apply it to&#10;low-light image enhancement. This method effectively alleviates the limitations&#10;of current methods constrained by linear network structures and lack of&#10;interpretability, further demonstrating the potential of KANs in low-level&#10;vision tasks. Given the poor perception of current low-light image enhancement&#10;methods and the stochastic nature of the inverse diffusion process, we further&#10;introduce frequency-domain perception for visually oriented enhancement.&#10;Extensive experiments demonstrate the competitive performance of our method on&#10;benchmark datasets. The code will be available at:&#10;https://github.com/AXNing/KSID}{https://github.com/AXNing/KSID.&#10;" />
-          <attvalue for="2" value="&#10;Low-light image enhancement (LLIE) is a critical task in computer vision and is essential for various applications ranging from surveillance to autonomous driving\cite{li2021low}. Images captured in low-light environments often suffer from low contrast and loss of detail, making downstream tasks such as object or text detection, semantic segmentation, and others highly challenging\cite{xu2024seeing}. Therefore, to further enhance various visual applications in poor environments, low-light image enhancement tasks have received extensive attention from researchers\cite{he2024zero,xue2024low,singh2024illumination}.\par&#10;Traditional methods utilize retinex theory\cite{guo2016lime} and gamma correction\cite{rahman2016adaptive} to correct image illumination. With the development of deep learning, some methods\cite{zhang2023multi,zhang2023lrt,xu2023low,cai2023retinexformer} significantly improve the performance of low-light image enhancement by learning the mapping between low-light and normal images in a data-driven way. Recently, the diffusion model \cite{ho2020denoising,nichol2021improved} has received much attention for its remarkable performance in generative tasks. \cite{fei2023generative} introduced the diffusion model into the low-light image enhancement task to improve the recovery of image details and textures in low-light conditions. \cite{10663248} leverages the generative capabilities of the latent diffusion model to accelerate inference speed while achieving excellent perceptual fidelity. \cite{hou2024global} establishes a global structure-aware regularization that promotes the retention of complex details and enhances contrast during the diffusion process, further improving image quality. \cite{xu2024upt} proposed a multi-scale Transformer conditional normalized flow (UPT-Flow) based on non-equilibrium point guidance for low-light image enhancement. However, there are nonlinear degradation factors in low-light enhancement tasks, such as uneven illumination and varying degrees of noise in low-light images, and it is difficult for existing methods to model complex nonlinear relationships on limited data. On the other hand, although current state-of-the-art technologies have achieved remarkable breakthroughs in the field of LLIE, its inner workings are often viewed as a black box that is difficult to decipher, limiting the development of the model in specific domains. \par&#10;The recent introduction of Kolmogorov-Arnold Networks\cite{liu2024kan} has raised hopes of opening the black box of traditional networks\cite{yu2023white}. It enables networks to efficiently represent complex multivariate functions by employing the Kolmogorov-Arnold representation theorem\cite{kolmogorov1957representation}. Unlike MLPs, which have fixed activation functions at nodes, KANs use fixed activation functions at edges.&#10;This decomposition helps to reveal the decision-making process and output of the model, which enhances the interpretability of the model. \cite{li2024u} was the first to introduce KANs into visual tasks, reformulating U-Net as U-KAN to improve medical image segmentation and generation.&#10;Despite these initial explorations, the potential of KANs for low-level visual tasks such as low-light image enhancement has not yet been demonstrated.\par&#10;&#10;In this letter, we propose a novel LLIE method (KSID) that introduces KANs to low-level visual tasks for the first time to learn better the nonlinear dependencies between the normal and low-light domains. Specifically, we design the KAN-Block and embed it into the U-Net used for denoising by the diffusion model. KAN-Block consists of the KAN-Layer and DwConv. The KAN-Layer, featuring spline-based convolutional layers and learnable activation functions, effectively captures nonlinear dependencies, significantly enhancing the quality of images generated by the diffusion model. In addition, to improve the stability and visualization of the generation process, we reconstruct the image at each step in the denoising process and introduce frequency domain perception using the Fast Fourier Transform (FFT) to further refine the image details by learning the spectrum of the normal image. As shown in Fig. \ref{sample}, our method shows a significant improvement over the current state-of-the-art method \cite{hou2024global}. We performed extensive experiments on benchmark datasets to demonstrate the effectiveness of our method. Our contribution can be summarized as follows:&#10;\begin{itemize}&#10;\item To the best of our knowledge, we are the first to successfully introduce KANs into the LLIE task, significantly improving the quality of low-light image restoration.&#10;&#10;\item We introduce frequency-domain perception for visual orientation enhancement by learning the spectrum of a normal image through the Fast Fourier Transform.&#10;\item We performed extensive experiments on the low-light image enhancement benchmark datasets and achieved impressive performance.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Physics, Deep Learning, Computer Science, Image Enhancement, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2404.08965" label="2404.08965">
-        <attvalues>
-          <attvalue for="0" value="Seeing Text in the Dark: Algorithm and Benchmark" />
-          <attvalue for="1" value="  Localizing text in low-light environments is challenging due to visual&#10;degradations. Although a straightforward solution involves a two-stage pipeline&#10;with low-light image enhancement (LLE) as the initial step followed by&#10;detector, LLE is primarily designed for human vision instead of machine and can&#10;accumulate errors. In this work, we propose an efficient and effective&#10;single-stage approach for localizing text in dark that circumvents the need for&#10;LLE. We introduce a constrained learning module as an auxiliary mechanism&#10;during the training stage of the text detector. This module is designed to&#10;guide the text detector in preserving textual spatial features amidst feature&#10;map resizing, thus minimizing the loss of spatial information in texts under&#10;low-light visual degradations. Specifically, we incorporate spatial&#10;reconstruction and spatial semantic constraints within this module to ensure&#10;the text detector acquires essential positional and contextual range knowledge.&#10;Our approach enhances the original text detector's ability to identify text's&#10;local topological features using a dynamic snake feature pyramid network and&#10;adopts a bottom-up contour shaping strategy with a novel rectangular&#10;accumulation technique for accurate delineation of streamlined text features.&#10;In addition, we present a comprehensive low-light dataset for arbitrary-shaped&#10;text, encompassing diverse scenes and languages. Notably, our method achieves&#10;state-of-the-art results on this low-light dataset and exhibits comparable&#10;performance on standard normal light datasets. The code and dataset will be&#10;released.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.02879" label="2403.02879">
-        <attvalues>
-          <attvalue for="0" value="Zero-LED: Zero-Reference Lighting Estimation Diffusion Model for&#10;  Low-Light Image Enhancement" />
-          <attvalue for="1" value="  Diffusion model-based low-light image enhancement methods rely heavily on&#10;paired training data, leading to limited extensive application. Meanwhile,&#10;existing unsupervised methods lack effective bridging capabilities for unknown&#10;degradation. To address these limitations, we propose a novel zero-reference&#10;lighting estimation diffusion model for low-light image enhancement called&#10;Zero-LED. It utilizes the stable convergence ability of diffusion models to&#10;bridge the gap between low-light domains and real normal-light domains and&#10;successfully alleviates the dependence on pairwise training data via&#10;zero-reference learning. Specifically, we first design the initial optimization&#10;network to preprocess the input image and implement bidirectional constraints&#10;between the diffusion model and the initial optimization network through&#10;multiple objective functions. Subsequently, the degradation factors of the&#10;real-world scene are optimized iteratively to achieve effective light&#10;enhancement. In addition, we explore a frequency-domain based and semantically&#10;guided appearance reconstruction module that encourages feature alignment of&#10;the recovered image at a fine-grained level and satisfies subjective&#10;expectations. Finally, extensive experiments demonstrate the superiority of our&#10;approach to other state-of-the-art methods and more significant generalization&#10;capabilities. We will open the source code upon acceptance of the paper.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.03788" label="2401.03788">
-        <attvalues>
-          <attvalue for="0" value="Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion" />
-          <attvalue for="1" value="  Low-light image enhancement techniques have significantly progressed, but&#10;unstable image quality recovery and unsatisfactory visual perception are still&#10;significant challenges. To solve these problems, we propose a novel and robust&#10;low-light image enhancement method via CLIP-Fourier Guided Wavelet Diffusion,&#10;abbreviated as CFWD. Specifically, CFWD leverages multimodal visual-language&#10;information in the frequency domain space created by multiple wavelet&#10;transforms to guide the enhancement process. Multi-scale supervision across&#10;different modalities facilitates the alignment of image features with semantic&#10;features during the wavelet diffusion process, effectively bridging the gap&#10;between degraded and normal domains. Moreover, to further promote the effective&#10;recovery of the image details, we combine the Fourier transform based on the&#10;wavelet transform and construct a Hybrid High Frequency Perception Module&#10;(HFPM) with a significant perception of the detailed features. This module&#10;avoids the diversity confusion of the wavelet diffusion process by guiding the&#10;fine-grained structure recovery of the enhancement results to achieve&#10;favourable metric and perceptually oriented enhancement. Extensive quantitative&#10;and qualitative experiments on publicly available real-world benchmarks show&#10;that our approach outperforms existing state-of-the-art methods, achieving&#10;significant progress in image quality and noise suppression. The project code&#10;is available at https://github.com/hejh8/CFWD.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.02197" label="2209.02197">
-        <attvalues>
-          <attvalue for="0" value="LRT: An Efficient Low-Light Restoration Transformer for Dark Light Field&#10;  Images" />
-          <attvalue for="1" value="  Light field (LF) images containing information for multiple views have&#10;numerous applications, which can be severely affected by low-light imaging.&#10;Recent learning-based methods for low-light enhancement have some&#10;disadvantages, such as a lack of noise suppression, complex training process&#10;and poor performance in extremely low-light conditions. To tackle these&#10;deficiencies while fully utilizing the multi-view information, we propose an&#10;efficient Low-light Restoration Transformer (LRT) for LF images, with multiple&#10;heads to perform intermediate tasks within a single network, including&#10;denoising, luminance adjustment, refinement and detail enhancement, achieving&#10;progressive restoration from small scale to full scale. Moreover, we design an&#10;angular transformer block with an efficient view-token scheme to model the&#10;global angular dependencies, and a multi-scale spatial transformer block to&#10;encode the multi-scale local and global information within each view. To&#10;address the issue of insufficient training data, we formulate a synthesis&#10;pipeline by simulating the major noise sources with the estimated noise&#10;parameters of LF camera. Experimental results demonstrate that our method&#10;achieves the state-of-the-art performance on low-light LF restoration with high&#10;efficiency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.05839" label="2305.05839">
-        <attvalues>
-          <attvalue for="0" value="Low-Light Image Enhancement via Structure Modeling and Guidance" />
-          <attvalue for="1" value="  This paper proposes a new framework for low-light image enhancement by&#10;simultaneously conducting the appearance as well as structure modeling. It&#10;employs the structural feature to guide the appearance enhancement, leading to&#10;sharp and realistic results. The structure modeling in our framework is&#10;implemented as the edge detection in low-light images. It is achieved with a&#10;modified generative model via designing a structure-aware feature extractor and&#10;generator. The detected edge maps can accurately emphasize the essential&#10;structural information, and the edge prediction is robust towards the noises in&#10;dark areas. Moreover, to improve the appearance modeling, which is implemented&#10;with a simple U-Net, a novel structure-guided enhancement module is proposed&#10;with structure-guided feature synthesis layers. The appearance modeling, edge&#10;detector, and enhancement module can be trained end-to-end. The experiments are&#10;conducted on representative datasets (sRGB and RAW domains), showing that our&#10;model consistently achieves SOTA performance on all datasets with the same&#10;architecture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Low-light enhancement with learning.&#10;The current low-light image enhancement methods with deep learning mainly optimize the appearance reconstruction error between the output and the ground truth~\cite{yan2014learning, yan2016automatic, lore2017llnet, cai2018learning,Zero-DCE,zamir2020learning,xu2020learning,zeng2020learning,kim2021representative,zhao2021deep,zheng2021adaptive,wang2021real,liu2021retinex,yang2021sparse,jiang2021enlightengan,yang2021band}.&#10;However, the enhanced appearance tends to be blurry in the dark regions even with SOTA approaches~\cite{xu2022snr}.&#10;To this, some works utilize the structural information~\cite{zhu2020eemefn,kim2021deep} to enhance appearance. &#10;To obtain the structure maps, some of them use offline-computed edges/gradients~\cite{kim2021deep,ren2019low,hao2020low,tanaka2019gradient} or offline-trained networks~\cite{liang2022semantically} which are not adaptively optimized with the low-light data and result in artifacts inevitably.&#10;Although several methods~\cite{zhu2020eemefn,rana2021edge} have been proposed to train an edge detector and an image enhancement network simultaneously with both regression losses, their improvements are still limited.&#10;Moreover, existing strategies mainly concatenate the extracted structure maps with the images to facilitate the enhancement~\cite{kim2021deep,zhu2020eemefn,rana2021edge}, which, however, can not set the guidance in all layers of the enhancement network.&#10;Different from current works, we propose a new generative model to robustly extract the edge map from a dark input image with the GAN loss, providing the critical edge information for enhancement in every feature layer.&#10;&#10;Generative model for restoration.&#10;The recent development on the generative model enables a wide range of networks, e.g., StyleGAN~\cite{karras2020analyzing}, to achieve significant restoration effects in some tasks, e.g., face restoration~\cite{yang2021gan,he2022gcfsr,zhu2022blind}.&#10;However, their performances are highly dependent on the pre-trained model within several categories.&#10;In this paper, we demonstrate that generative models with the GAN loss can be utilized in a novel way to synthesize structural information for the restoration task of low-light image enhancement.&#10;Such an approach does not rely on the pre-trained model, and such an idea can actually be extended to various restoration tasks, opening up a new research direction.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2303.06705" label="2303.06705">
-        <attvalues>
-          <attvalue for="0" value="Retinexformer: One-stage Retinex-based Transformer for Low-light Image&#10;  Enhancement" />
-          <attvalue for="1" value="  When enhancing low-light images, many deep learning algorithms are based on&#10;the Retinex theory. However, the Retinex model does not consider the&#10;corruptions hidden in the dark or introduced by the light-up process. Besides,&#10;these methods usually require a tedious multi-stage training pipeline and rely&#10;on convolutional neural networks, showing limitations in capturing long-range&#10;dependencies. In this paper, we formulate a simple yet principled One-stage&#10;Retinex-based Framework (ORF). ORF first estimates the illumination information&#10;to light up the low-light image and then restores the corruption to produce the&#10;enhanced image. We design an Illumination-Guided Transformer (IGT) that&#10;utilizes illumination representations to direct the modeling of non-local&#10;interactions of regions with different lighting conditions. By plugging IGT&#10;into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative&#10;and qualitative experiments demonstrate that our Retinexformer significantly&#10;outperforms state-of-the-art methods on thirteen benchmarks. The user study and&#10;application on low-light object detection also reveal the latent practical&#10;values of our method. Code, models, and results are available at&#10;https://github.com/caiyuanhao1998/Retinexformer&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.09672" label="2102.09672">
-        <attvalues>
-          <attvalue for="0" value="Improved Denoising Diffusion Probabilistic Models" />
-          <attvalue for="1" value="  Denoising diffusion probabilistic models (DDPM) are a class of generative&#10;models which have recently been shown to produce excellent samples. We show&#10;that with a few simple modifications, DDPMs can also achieve competitive&#10;log-likelihoods while maintaining high sample quality. Additionally, we find&#10;that learning variances of the reverse diffusion process allows sampling with&#10;an order of magnitude fewer forward passes with a negligible difference in&#10;sample quality, which is important for the practical deployment of these&#10;models. We additionally use precision and recall to compare how well DDPMs and&#10;GANs cover the target distribution. Finally, we show that the sample quality&#10;and likelihood of these models scale smoothly with model capacity and training&#10;compute, making them easily scalable. We release our code at&#10;https://github.com/openai/improved-diffusion&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.01247" label="2304.01247">
-        <attvalues>
-          <attvalue for="0" value="Generative Diffusion Prior for Unified Image Restoration and Enhancement" />
-          <attvalue for="1" value="  Existing image restoration methods mostly leverage the posterior distribution&#10;of natural images. However, they often assume known degradation and also&#10;require supervised training, which restricts their adaptation to complex real&#10;applications. In this work, we propose the Generative Diffusion Prior (GDP) to&#10;effectively model the posterior distributions in an unsupervised sampling&#10;manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM)&#10;for solving linear inverse, non-linear, or blind problems. Specifically, GDP&#10;systematically explores a protocol of conditional guidance, which is verified&#10;more practical than the commonly used guidance way. Furthermore, GDP is&#10;strength at optimizing the parameters of degradation model during the denoising&#10;process, achieving blind image restoration. Besides, we devise hierarchical&#10;guidance and patch-based methods, enabling the GDP to generate images of&#10;arbitrary resolutions. Experimentally, we demonstrate GDP's versatility on&#10;several image datasets for linear problems, such as super-resolution,&#10;deblurring, inpainting, and colorization, as well as non-linear and blind&#10;issues, such as low-light enhancement and HDR image recovery. GDP outperforms&#10;the current leading unsupervised methods on the diverse benchmarks in&#10;reconstruction quality and perceptual quality. Moreover, GDP also generalizes&#10;well for natural images or synthesized images with arbitrary sizes from various&#10;tasks out of the distribution of the ImageNet training set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Linear Inverse Image Restoration. Most diffusion models toward linear inverse problems have employed unconditional models for the conditional tasks\cite{meng2021sdedit,song2020score}, where only one model needs to be trained. However, unconditional tasks tend to be more difficult than conditional tasks. Moreover, the multi-linear task is also a relatively under-explored subject in image restoration. For instance, \cite{qian2019trinity, yu2018crafting} train simultaneously on multiple tasks, but they mainly concentrate on the enhancement tasks like deblurring and so on. Some works have also handled the multi-scale super-resolution by simultaneous training over multiple degradations \cite{kim2016deeply}. Here, we propose \GDP as a single model for dealing with single linear inverse or multiple linear inverse tasks. &#10;&#10;Non-linear Image Restoration. The non-linear image formation model provides an accurate description of several imaging systems, including camera response functions in high-dynamic-range imaging\cite{Rameshan2011high}. The non-linear image restoration model is more accurate but is often more computationally intractable. Recently, great attention has been paid to non-linear image restoration problems. For example, HDR-GAN~\cite{niu2021hdr} is proposed for synthesizing HDR images from multi-exposed LDR images, while EnlightenGAN \cite{jiang2021enlightengan} is devised as an unsupervised GAN to generalize very well on various real-world test images. The diffusion models are rarely studied for non-linear image restoration.&#10;&#10;Blind Image Restoration. Early supervised attempts~\cite{begin2004blind,he2009soft} tend to estimate the unknown point spread function. As an example,~\cite{ke2021unsupervised} designs a class of structured denoisers, and ~\cite{shocher2018zero} employs a fixed down-sampling operation to generate synthetic pairs during testing. However, these methods often are incapable of obtaining the parameters or distribution of the observed data due to the complicated degradation types. Another way to solve the blind image restoration is to utilize unsupervised learning methods~\cite{zhu2017unpaired,du2020learning}. Following the CycleGAN~\cite{zhu2017unpaired}, CinCGAN \cite{yuan2018unsupervised} and MCinCGAN \cite{zhang2019multiple} employ a pre-trained SR model together with cycle consistency loss to learn a mapping from the input image to high-quality image space. However, it still remains a challenge to exploit a unified architecture for blind image restoration. By the merits of the powerful GDP, these blind problems could also be solved by simultaneously estimating the recovered image and a specific degradation model.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2310.17577" label="2310.17577">
-        <attvalues>
-          <attvalue for="0" value="Global Structure-Aware Diffusion Process for Low-Light Image Enhancement" />
-          <attvalue for="1" value="  This paper studies a diffusion-based framework to address the low-light image&#10;enhancement problem. To harness the capabilities of diffusion models, we delve&#10;into this intricate process and advocate for the regularization of its inherent&#10;ODE-trajectory. To be specific, inspired by the recent research that low&#10;curvature ODE-trajectory results in a stable and effective diffusion process,&#10;we formulate a curvature regularization term anchored in the intrinsic&#10;non-local structures of image data, i.e., global structure-aware&#10;regularization, which gradually facilitates the preservation of complicated&#10;details and the augmentation of contrast during the diffusion process. This&#10;incorporation mitigates the adverse effects of noise and artifacts resulting&#10;from the diffusion process, leading to a more precise and flexible enhancement.&#10;To additionally promote learning in challenging regions, we introduce an&#10;uncertainty-guided regularization technique, which wisely relaxes constraints&#10;on the most extreme regions of the image. Experimental evaluations reveal that&#10;the proposed diffusion-based framework, complemented by rank-informed&#10;regularization, attains distinguished performance in low-light enhancement. The&#10;outcomes indicate substantial advancements in image quality, noise suppression,&#10;and contrast amplification in comparison with state-of-the-art methods. We&#10;believe this innovative approach will stimulate further exploration and&#10;advancement in low-light image processing, with potential implications for&#10;other applications of diffusion models. The code is publicly available at&#10;https://github.com/jinnh/GSAD.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.19756" label="2404.19756">
-        <attvalues>
-          <attvalue for="0" value="KAN: Kolmogorov-Arnold Networks" />
-          <attvalue for="1" value="  Inspired by the Kolmogorov-Arnold representation theorem, we propose&#10;Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer&#10;Perceptrons (MLPs). While MLPs have fixed activation functions on nodes&#10;(&quot;neurons&quot;), KANs have learnable activation functions on edges (&quot;weights&quot;).&#10;KANs have no linear weights at all -- every weight parameter is replaced by a&#10;univariate function parametrized as a spline. We show that this seemingly&#10;simple change makes KANs outperform MLPs in terms of accuracy and&#10;interpretability. For accuracy, much smaller KANs can achieve comparable or&#10;better accuracy than much larger MLPs in data fitting and PDE solving.&#10;Theoretically and empirically, KANs possess faster neural scaling laws than&#10;MLPs. For interpretability, KANs can be intuitively visualized and can easily&#10;interact with human users. Through two examples in mathematics and physics,&#10;KANs are shown to be useful collaborators helping scientists (re)discover&#10;mathematical and physical laws. In summary, KANs are promising alternatives for&#10;MLPs, opening opportunities for further improving today's deep learning models&#10;which rely heavily on MLPs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.01129" label="2306.01129">
-        <attvalues>
-          <attvalue for="0" value="White-Box Transformers via Sparse Rate Reduction" />
-          <attvalue for="1" value="  In this paper, we contend that the objective of representation learning is to&#10;compress and transform the distribution of the data, say sets of tokens,&#10;towards a mixture of low-dimensional Gaussian distributions supported on&#10;incoherent subspaces. The quality of the final representation can be measured&#10;by a unified objective function called sparse rate reduction. From this&#10;perspective, popular deep networks such as transformers can be naturally viewed&#10;as realizing iterative schemes to optimize this objective incrementally.&#10;Particularly, we show that the standard transformer block can be derived from&#10;alternating optimization on complementary parts of this objective: the&#10;multi-head self-attention operator can be viewed as a gradient descent step to&#10;compress the token sets by minimizing their lossy coding rate, and the&#10;subsequent multi-layer perceptron can be viewed as attempting to sparsify the&#10;representation of the tokens. This leads to a family of white-box&#10;transformer-like deep network architectures which are mathematically fully&#10;interpretable. Despite their simplicity, experiments show that these networks&#10;indeed learn to optimize the designed objective: they compress and sparsify&#10;representations of large-scale real-world vision datasets such as ImageNet, and&#10;achieve performance very close to thoroughly engineered transformers such as&#10;ViT. Code is at \url{https://github.com/Ma-Lab-Berkeley/CRATE}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.02918" label="2406.02918">
-        <attvalues>
-          <attvalue for="0" value="U-KAN Makes Strong Backbone for Medical Image Segmentation and&#10;  Generation" />
-          <attvalue for="1" value="  U-Net has become a cornerstone in various visual applications such as image&#10;segmentation and diffusion probability models. While numerous innovative&#10;designs and improvements have been introduced by incorporating transformers or&#10;MLPs, the networks are still limited to linearly modeling patterns as well as&#10;the deficient interpretability. To address these challenges, our intuition is&#10;inspired by the impressive results of the Kolmogorov-Arnold Networks (KANs) in&#10;terms of accuracy and interpretability, which reshape the neural network&#10;learning via the stack of non-linear learnable activation functions derived&#10;from the Kolmogorov-Anold representation theorem. Specifically, in this paper,&#10;we explore the untapped potential of KANs in improving backbones for vision&#10;tasks. We investigate, modify and re-design the established U-Net pipeline by&#10;integrating the dedicated KAN layers on the tokenized intermediate&#10;representation, termed U-KAN. Rigorous medical image segmentation benchmarks&#10;verify the superiority of U-KAN by higher accuracy even with less computation&#10;cost. We further delved into the potential of U-KAN as an alternative U-Net&#10;noise predictor in diffusion models, demonstrating its applicability in&#10;generating task-oriented model architectures. These endeavours unveil valuable&#10;insights and sheds light on the prospect that with U-KAN, you can make strong&#10;backbone for medical image segmentation and generation. Project&#10;page:\url{https://yes-u-kan.github.io/}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00151" label="1908.00151">
-        <attvalues>
-          <attvalue for="0" value="Multi-path Learning for Object Pose Estimation Across Domains" />
-          <attvalue for="1" value="  We introduce a scalable approach for object pose estimation trained on&#10;simulated RGB views of multiple 3D models together. We learn an encoding of&#10;object views that does not only describe an implicit orientation of all objects&#10;seen during training, but can also relate views of untrained objects. Our&#10;single-encoder-multi-decoder network is trained using a technique we denote&#10;&quot;multi-path learning&quot;: While the encoder is shared by all objects, each decoder&#10;only reconstructs views of a single object. Consequently, views of different&#10;instances do not have to be separated in the latent space and can share common&#10;features. The resulting encoder generalizes well from synthetic to real data&#10;and across various instances, categories, model types and datasets. We&#10;systematically investigate the learned encodings, their generalization, and&#10;iterative refinement strategies on the ModelNet40 and T-LESS dataset. Despite&#10;training jointly on multiple objects, our 6D Object Detection pipeline achieves&#10;state-of-the-art results on T-LESS at much lower runtimes than competing&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value=" &#10;&#10;Object pose estimation has been widely studied in the literature,&#10;mainly from the below aspects.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Computer Science, Object Pose Estimation, Deep Learning Techniques, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1905.03304" label="1905.03304">
-        <attvalues>
-          <attvalue for="0" value="Deep Closest Point: Learning Representations for Point Cloud&#10;  Registration" />
-          <attvalue for="1" value="  Point cloud registration is a key problem for computer vision applied to&#10;robotics, medical imaging, and other applications. This problem involves&#10;finding a rigid transformation from one point cloud into another so that they&#10;align. Iterative Closest Point (ICP) and its variants provide simple and&#10;easily-implemented iterative methods for this task, but these algorithms can&#10;converge to spurious local optima. To address local optima and other&#10;difficulties in the ICP pipeline, we propose a learning-based method, titled&#10;Deep Closest Point (DCP), inspired by recent techniques in computer vision and&#10;natural language processing. Our model consists of three parts: a point cloud&#10;embedding network, an attention-based module combined with a pointer generation&#10;layer, to approximate combinatorial matching, and a differentiable singular&#10;value decomposition (SVD) layer to extract the final rigid transformation. We&#10;train our model end-to-end on the ModelNet40 dataset and show in several&#10;settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR),&#10;and the recently-proposed learning-based method PointNetLK. Beyond providing a&#10;state-of-the-art registration technique, we evaluate the suitability of our&#10;learned features transferred to unseen objects. We also provide preliminary&#10;analysis of our learned model to help understand whether domain-specific and/or&#10;global features facilitate rigid registration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Traditional point cloud registration methods: ICP \cite{Besl1992} is the best-known algorithm for solving rigid registration problems; it alternates between finding point cloud correspondences and solving a least-squares problem to update the alignment. ICP variants \cite{Rusinkiewicz2001, Segal2009, Bouaziz2013sgp} consider issues with the basic method, like noise, partiality, and sparsity; probabilistic models \cite{agamennoniIROS16, iser_2016_hinzmann, Hahnel02probabilisticmatching} also can improve resilience to uncertain data. ICP can be viewed as an optimization algorithm searching jointly for a matching and a rigid alignment; hence, \cite{Fitzgibbon01c} propose using the Levenberg--Marquardt algorithm to optimize the objective directly, which can yield a better solution. For more information, \cite{Pomerleau2015, Rusinkiewicz2001} summarize ICP and its variants developed over the last 20 years. &#10;&#10;ICP-style methods are prone to local minima due to non-convexity. To find a good optimum in polynomial time, Go-ICP \cite{Yang2016GoICP} uses a branch-and-bound (BnB) method to search the motion space $\mathrm{SE}(3)$. It outperforms local ICP methods when a global solution is desired but is several orders of magnitude slower than other ICP variants despite using local ICP to accelerate the search process. Other methods attempt to identify global optima using Riemannian optimization \cite{Rosen16wafr-sesync}, convex relaxation \cite{Maron2016}, and mixed-integer programming \cite{Izatt2017}.&#10;&#10;Learning on graphs and point sets: A broad class of deep architectures for geometric data termed geometric deep learning \cite{bronstein2017geometric} includes recent methods learning on graphs \cite{Zonghan2019, Zhou2018GraphNN, Fey2018splinecnn} and point clouds \cite{QiSMG17, Qi2017pn++, dgcnn, Zaheer2017}.&#10;&#10;The graph neural network (GNN) is introduced in \cite{Scarselli2009}; similarly, \cite{Duvenaud2015} defines convolution on graphs (GCN) for molecular data. \cite{kipf2016semi} uses renormalization to adapt to graph structure and applies GCN to semi-supervised learning on graphs. MoNet \cite{Monti17} learns a dynamic aggregation function based on graph structure, generalizing GNNs. Finally, graph attention networks (GATs) \cite{velickovic2018graph} incorporate multi-head attention into GCNs. DGCNN \cite{dgcnn} can be regarded as graph neural network applied to point clouds with dynamic edges.&#10;&#10;Another branch of geometric deep learning includes PointNet \cite{QiSMG17} and other algorithms designed to process point clouds. PointNet can be seen as applying GCN to graphs without edges, mapping points in $\mathbb{R}^3$ to high-dimensional space. PointNet only encodes global features gathered from the point cloud's embedding, impeding application to tasks involving local geometry. To address this issue, PointNet++ \cite{Qi2017pn++} applies a shared PointNet to $k$-nearest neighbor clusters to learn local features. As an alternative, DGCNN \cite{dgcnn} explicitly recovers the graph structure in both Euclidean space and feature space and applies graph neural networks to the result. PCNN \cite{Atzmon2018} uses an extension operator to define convolution on point clouds, while PointCNN \cite{Li2018pointcnn} applies Euclidean convolution after applying a learned transformation. Finally, SPLATNet \cite{su18splatnet} encodes point clouds on a lattice and performs bilateral convolution. All these works aim to apply convolution-like operations to point clouds and extract local geometric features. &#10;&#10;Sequence-to-sequence learning and pointer networks: Many tasks in natural language processing, including machine translation, language modeling, and question answering, can be formulated as sequence-to-sequence problems (seq2seq). \cite{Sutskever2014} first uses deep neural networks (DNN) to address seq2seq problems at large scale. Seq2seq, however, often involves predicting discrete tokens corresponding to positions in the input sequence. This problem is difficult because there is an exponential number of possible matchings between input and output positions. Similar problems can be found in optimal transportation \cite{Justin2018ot, Gabriel2019}, combinatorial optimization \cite{Ivanov2002}, and graph matching \cite{Yan2016}. To address this issue, In our registration pipeline, we use a related method to Pointer Networks \cite{Vinyals2015}, which use attention as a pointer to select from the input sequence. In each output step, a Pointer Network predicts a distribution over positions and uses it as a ``soft pointer.'' The pointer module is fully differentiable, and the whole network can be trained end-to-end. &#10;&#10;Non-local approaches: To denoise images, non-local means \cite{Buades2005} leverages the simple observation that Gaussian noise can be removed by non-locally weighted averaging all pixels in an image. Recently, non-local neural networks \cite{NonLocal2018} have been proposed to capture long-range dependencies in video understanding; \cite{Xie2018denoise} uses the non-local module to denoise feature maps to defend against adversarial attacks. Another instantiation of non-local neural networks, known as relational networks \cite{Santoro2017}, has shown effectiveness in visual reasoning \cite{Santoro2017}, meta-learning \cite{sung2018learning}, object detection \cite{hu2017relation}, and reinforcement learning \cite{Flores2018}. Its counterpart in natural language processing, attention, is arguably the most fruitful recent advance in this discipline. \cite {Vaswani2017} replaces recurrent neural networks \cite{Jordan1990, Hochreiter1997} with a model called Transformer, consisting of several stacked multi-head attention modules. Transformer-based models \cite{Devlin2018, radford2019language} outperform other recurrent models by a considerable amount in natural language processing. In our work, we also use Transformer to learn contextual information of point clouds. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Engineering, Machine Learning, Mathematics, Deep Learning Methods, Computer Science, Point Cloud Registration, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2212.04712" label="2212.04712">
-        <attvalues>
-          <attvalue for="0" value="Occluded Person Re-Identification via Relational Adaptive Feature&#10;  Correction Learning" />
-          <attvalue for="1" value="  Occluded person re-identification (Re-ID) in images captured by multiple&#10;cameras is challenging because the target person is occluded by pedestrians or&#10;objects, especially in crowded scenes. In addition to the processes performed&#10;during holistic person Re-ID, occluded person Re-ID involves the removal of&#10;obstacles and the detection of partially visible body parts. Most existing&#10;methods utilize the off-the-shelf pose or parsing networks as pseudo labels,&#10;which are prone to error. To address these issues, we propose a novel Occlusion&#10;Correction Network (OCNet) that corrects features through relational-weight&#10;learning and obtains diverse and representative features without using external&#10;networks. In addition, we present a simple concept of a center feature in order&#10;to provide an intuitive solution to pedestrian occlusion scenarios.&#10;Furthermore, we suggest the idea of Separation Loss (SL) for focusing on&#10;different parts between global features and part features. We conduct extensive&#10;experiments on five challenging benchmark datasets for occluded and holistic&#10;Re-ID tasks to demonstrate that our method achieves superior performance to&#10;state-of-the-art methods especially on occluded scene.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Person re-identification (Re-ID) involves identifying people appearing in the images captured by multiple cameras with non-overlapping domains. Most of methods utilize a representation by extracting global features from complete pedestrian images. However, these approaches have limitations in addressing the occluded Re-ID, which frequently occurs in crowded and complex scenes featuring multiple obstacles.&#10;&#10;Occlusion is classified into object occlusion and pedestrian-Interference (PI)~\cite{zhao2020not} based on the type of object that covers the target. In object occlusion cases, the global feature extracted by the network may be mixed with unnecessary information. In order to obtain only the target features, most methods~\cite{he2019foreground, miao2019pose} use attention maps or pose information generated by other additional networks such as pose estimation and human parsing networks. However, these methods are highly dependent on the performance of the external networks and are hard to train well with an error-prone pseudo label. Without such external help, it is impossible to flexibly deal with misalignment problems. The misalignment problem occurs when features are extracted in the order of important parts from the input image and simply concatenated. For example, in the Fig.~\ref{mis} (a), part features are extracted and concatenated in body-feet-upper body-head order and body-head-upper body-feet order for probe and gallery images; This leads to misalignment of the final representation features, lowering similarity and unmatching IDs.&#10;Another issue is PI, which occurs when two or more people appear in the bounding box because of the limitations of detector performance in crowded scenes. As the network recognizes multiple pedestrians as the foreground, it is difficult to remove the non-target pedestrians. This phenomenon leads to mismatching between people and IDs. In existing methods~\cite{zhao2020not, he2020guided}, external networks are used or an attention map is obtained by performing additional operations upon query and gallery image to find the target.&#10;&#10;To address the aforementioned issues, we propose Occlusion Correction Network (OCNet), a novel network that extracts global and various part features while performing feature correction in order to be robust to occlusion. Unlike the previous method~\cite{sun2018beyond} of simply concatenating the part features, which causes feature misalignment, our model predicts the weights through relational information between various features and corrects irrelevant information to create the final representation. Through relational-weight learning, the weights of the occluded part are adaptively determined and aligned with feature correction. Thus, our method extracts various features from the visible part of the given images even if occluded; it then characterizes the relationship between them to create a final representation feature that is robust to occlusion and contains a discriminative representation.&#10;&#10;In this paper, our main contributions are as follows: (1) We design an Occlusion Correction Network that corrects the final representation through relational-weight learning to determine whether a part feature is crucial or not, so that a more diverse and representative feature is obtained which avoids occluded parts. (2) To provide a robust solution to pedestrian occlusion, we propose the essential concept of a center feature for the first time to provide a robust solution to pedestrian occlusion. (3) We suggest Separation Loss for focusing on different information between global features and part features. (4) Our network achieves superior performance not only in occluded dataset but also in holistic datasets, without relying on additionally trained networks or external annotations.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Occlusion Correction, Computer Science, Pattern Recognition, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="2008.06963" label="2008.06963">
-        <attvalues>
-          <attvalue for="0" value="Do Not Disturb Me: Person Re-identification Under the Interference of&#10;  Other Pedestrians" />
-          <attvalue for="1" value="  In the conventional person Re-ID setting, it is widely assumed that cropped&#10;person images are for each individual. However, in a crowded scene,&#10;off-shelf-detectors may generate bounding boxes involving multiple people,&#10;where the large proportion of background pedestrians or human occlusion exists.&#10;The representation extracted from such cropped images, which contain both the&#10;target and the interference pedestrians, might include distractive information.&#10;This will lead to wrong retrieval results. To address this problem, this paper&#10;presents a novel deep network termed Pedestrian-Interference Suppression&#10;Network (PISNet). PISNet leverages a Query-Guided Attention Block (QGAB) to&#10;enhance the feature of the target in the gallery, under the guidance of the&#10;query. Furthermore, the involving Guidance Reversed Attention Module and the&#10;Multi-Person Separation Loss promote QGAB to suppress the interference of other&#10;pedestrians. Our method is evaluated on two new pedestrian-interference&#10;datasets and the results show that the proposed method performs favorably&#10;against existing Re-ID methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04975" label="1904.04975">
-        <attvalues>
-          <attvalue for="0" value="Foreground-aware Pyramid Reconstruction for Alignment-free Occluded&#10;  Person Re-identification" />
-          <attvalue for="1" value="  Re-identifying a person across multiple disjoint camera views is important&#10;for intelligent video surveillance, smart retailing and many other&#10;applications. However, existing person re-identification (ReID) methods are&#10;challenged by the ubiquitous occlusion over persons and suffer from performance&#10;degradation. This paper proposes a novel occlusion-robust and alignment-free&#10;model for occluded person ReID and extends its application to realistic and&#10;crowded scenarios. The proposed model first leverages the full convolution&#10;network (FCN) and pyramid pooling to extract spatial pyramid features. Then an&#10;alignment-free matching approach, namely Foreground-aware Pyramid&#10;Reconstruction (FPR), is developed to accurately compute matching scores&#10;between occluded persons, despite their different scales and sizes. FPR uses&#10;the error from robust reconstruction over spatial pyramid features to measure&#10;similarities between two persons. More importantly, we design an&#10;occlusion-sensitive foreground probability generator that focuses more on clean&#10;human body parts to refine the similarity computation with less contamination&#10;from occlusion. The FPR is easily embedded into any end-to-end person ReID&#10;models. The effectiveness of the proposed method is clearly demonstrated by the&#10;experimental results (Rank-1 accuracy) on three occluded person datasets:&#10;Partial REID (78.30\%), Partial iLIDS (68.08\%) and Occluded REID (81.00\%);&#10;and three benchmark person datasets: Market1501 (95.42\%), DukeMTMC (88.64\%)&#10;and CUHK03 (76.08\%)&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.09349" label="1711.09349">
-        <attvalues>
-          <attvalue for="0" value="Beyond Part Models: Person Retrieval with Refined Part Pooling (and a&#10;  Strong Convolutional Baseline)" />
-          <attvalue for="1" value="  Employing part-level features for pedestrian image description offers&#10;fine-grained information and has been verified as beneficial for person&#10;retrieval in very recent literature. A prerequisite of part discovery is that&#10;each part should be well located. Instead of using external cues, e.g., pose&#10;estimation, to directly locate parts, this paper lays emphasis on the content&#10;consistency within each part.&#10;  Specifically, we target at learning discriminative part-informed features for&#10;person retrieval and make two contributions. (i) A network named Part-based&#10;Convolutional Baseline (PCB). Given an image input, it outputs a convolutional&#10;descriptor consisting of several part-level features. With a uniform partition&#10;strategy, PCB achieves competitive results with the state-of-the-art methods,&#10;proving itself as a strong convolutional baseline for person retrieval.&#10;  (ii) A refined part pooling (RPP) method. Uniform partition inevitably incurs&#10;outliers in each part, which are in fact more similar to other parts. RPP&#10;re-assigns these outliers to the parts they are closest to, resulting in&#10;refined parts with enhanced within-part consistency. Experiment confirms that&#10;RPP allows PCB to gain another round of performance boost. For instance, on the&#10;Market-1501 dataset, we achieve (77.4+4.2)% mAP and (92.3+1.5)% rank-1&#10;accuracy, surpassing the state of the art by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.11969" label="2004.11969">
-        <attvalues>
-          <attvalue for="0" value="Leveraging Planar Regularities for Point Line Visual-Inertial Odometry" />
-          <attvalue for="1" value="  With monocular Visual-Inertial Odometry (VIO) system, 3D point cloud and&#10;camera motion can be estimated simultaneously. Because pure sparse 3D points&#10;provide a structureless representation of the environment, generating 3D mesh&#10;from sparse points can further model the environment topology and produce dense&#10;mapping. To improve the accuracy of 3D mesh generation and localization, we&#10;propose a tightly-coupled monocular VIO system, PLP-VIO, which exploits point&#10;features and line features as well as plane regularities. The co-planarity&#10;constraints are used to leverage additional structure information for the more&#10;accurate estimation of 3D points and spatial lines in state estimator. To&#10;detect plane and 3D mesh robustly, we combine both the line features with point&#10;features in the detection method. The effectiveness of the proposed method is&#10;verified on both synthetic data and public datasets and is compared with other&#10;state-of-the-art algorithms.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Simultaneous motion estimating and dense mapping are important to many robotic applications like autonomous driving, augmented reality, and building inspection. Camera and inertial measurement units (IMUs) are low-cost and effective sensors, based on which the Visual-Inertial Odometry (VIO) system can be implemented. Existing VIO methods \cite{mourikis2007multi,qin2018vins,leutenegger2015keyframe} can estimate pose accurately but only provide a sparse map of 3D points. In \cite{pizzoli2014remode,newcombe2011dtam} dense mapping can be realized based on monocular vision and pixel-wise reconstruction, which is time-consuming and highly relies on Graphics Processing Unit (GPU) to guarantee real-time performance.&#10;&#10;For light-weight dense mapping, Lucas et al.\cite{teixeira2016real} used sparse point features on image to generate 2D Delaunay triangulation and build a dense map for aerial inspection. 3D Delaunay triangulation, which is generated from sparse 3D points estimated by ORB-SLAM, is used to build a dense map in real-time for robot navigation \cite{ling2017building}. However, these algorithms decouple pose estimation and dense mapping, causing information loss. Antoni et al. \cite{rosinol2019incremental,rosinol2019master} proposed an incremental visual-inertial 3D mesh generation system that uses 3D mesh to detect plane and enforces planar structural regularities to improve localization accuracy. However, the spares points-based map is structureless so that the mesh does not fit the environment well.&#10;&#10;For heterogeneous features utilization, the line and plane feature have attracted researchers' attention. In our previous work \cite{he2018pl}, The environment reconstructed with 3D straight lines can reflect more geometric structure information than point features. Moreover, line features incorporated into the SLAM system can improve pose accuracy and the robustness under illumination-changing scenes \cite{he2018pl,pumarola2017pl,zuo2017robust}. However, they do not consider the geometric constraint between point and line features. Structural regularity from the Atlanta world model is introduced into the line based visual odometry system \cite{zou2019structvio, li2019leveraging}. But point features are not considered in their system. Moreover, all these works did not use structural landmarks to build a dense map.&#10;Planar constraints used in these SLAM system are more often relies on the depth sensor\cite{ataer2013tracking,kaess2015simultaneous,yang2019tightly,zuo2019lic}.&#10;Lu et al.\cite{lu2015visual} presented a multi-layer feature graph utilizing points, lines, planes, and vanishing points to improve the accuracy of bundle adjustment. However, they used RANSAC-based approaches to detect planes which require numerous iteration and are time-consuming. Yang et al.\cite{yang2018observability} investigated the observability of the VIO using heterogeneous features of points, lines, and planes.&#10;\cite{nardi2019unified} proposed a unified representation for all the point, line and plane features but they need depth sensors. &#10;&#10;In this paper, we propose a tightly-coupled monocular VIO system that involves heterogeneous visual features including points, lines, and planes as well as their co-planarity constraints (PLP-VIO), to realize accurate motion estimation and dense mapping, as shown in Fig.\ref{fig:show_contri}. A related work \cite{rosinol2019incremental} also utilized planar regularities, which only used sparse point features. In comparison, our proposed method not only leveraged the planar regularities but also involves both point features and line features to detect planes, so that richer structural information is used for 3D mesh generation. The main contributions of this work include:&#10;&#10;\begin{itemize}&#10;&#09;\item We propose a non-iterative plane detection method and a 3D mesh generation method based on sparse points and spatial lines for monocular VIO. Our proposed method utilizes structural line to improves the correctness and robustness of plane detection and mesh generation compared to the method involving only sparse points\cite{rosinol2019incremental}.&#10;&#10;\item We develop a tightly-coupled monocular VIO system that utilizes heterogeneous visual features, include points, lines, and planes, as well as their co-planarity constraints. The richer visual information and spatial constraints between landmarks improve the estimator accuracy and robust.&#10;&#10;\item We provide an ablation experiment to verify the effectiveness of heterogeneous features on both synthetic data and the EuRoc dataset \cite{burri2016euroc}. Experimental results demonstrated that our system simultaneously improves the accuracies of pose estimation and mapping.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Geometric Computing, Computer Science, Visual Odometry, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1708.03852" label="1708.03852">
-        <attvalues>
-          <attvalue for="0" value="VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State&#10;  Estimator" />
-          <attvalue for="1" value="  A monocular visual-inertial system (VINS), consisting of a camera and a&#10;low-cost inertial measurement unit (IMU), forms the minimum sensor suite for&#10;metric six degrees-of-freedom (DOF) state estimation. However, the lack of&#10;direct distance measurement poses significant challenges in terms of IMU&#10;processing, estimator initialization, extrinsic calibration, and nonlinear&#10;optimization. In this work, we present VINS-Mono: a robust and versatile&#10;monocular visual-inertial state estimator.Our approach starts with a robust&#10;procedure for estimator initialization and failure recovery. A tightly-coupled,&#10;nonlinear optimization-based method is used to obtain high accuracy&#10;visual-inertial odometry by fusing pre-integrated IMU measurements and feature&#10;observations. A loop detection module, in combination with our tightly-coupled&#10;formulation, enables relocalization with minimum computation overhead.We&#10;additionally perform four degrees-of-freedom pose graph optimization to enforce&#10;global consistency. We validate the performance of our system on public&#10;datasets and real-world experiments and compare against other state-of-the-art&#10;algorithms. We also perform onboard closed-loop autonomous flight on the MAV&#10;platform and port the algorithm to an iOS-based demonstration. We highlight&#10;that the proposed work is a reliable, complete, and versatile system that is&#10;applicable for different applications that require high accuracy localization.&#10;We open source our implementations for both PCs and iOS mobile devices.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.01067" label="1903.01067">
-        <attvalues>
-          <attvalue for="0" value="Incremental Visual-Inertial 3D Mesh Generation with Structural&#10;  Regularities" />
-          <attvalue for="1" value="  Visual-Inertial Odometry (VIO) algorithms typically rely on a point cloud&#10;representation of the scene that does not model the topology of the&#10;environment. A 3D mesh instead offers a richer, yet lightweight, model.&#10;Nevertheless, building a 3D mesh out of the sparse and noisy 3D landmarks&#10;triangulated by a VIO algorithm often results in a mesh that does not fit the&#10;real scene. In order to regularize the mesh, previous approaches decouple state&#10;estimation from the 3D mesh regularization step, and either limit the 3D mesh&#10;to the current frame or let the mesh grow indefinitely. We propose instead to&#10;tightly couple mesh regularization and state estimation by detecting and&#10;enforcing structural regularities in a novel factor-graph formulation. We also&#10;propose to incrementally build the mesh by restricting its extent to the&#10;time-horizon of the VIO optimization; the resulting 3D mesh covers a larger&#10;portion of the scene than a per-frame approach while its memory usage and&#10;computational complexity remain bounded. We show that our approach successfully&#10;regularizes the mesh, while improving localization accuracy, when structural&#10;regularities are present, and remains operational in scenes without&#10;regularities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08654" label="1711.08654">
-        <attvalues>
-          <attvalue for="0" value="Robust Visual SLAM with Point and Line Features" />
-          <attvalue for="1" value="  In this paper, we develop a robust efficient visual SLAM system that utilizes&#10;heterogeneous point and line features. By leveraging ORB-SLAM [1], the proposed&#10;system consists of stereo matching, frame tracking, local mapping, loop&#10;detection, and bundle adjustment of both point and line features. In&#10;particular, as the main theoretical contributions of this paper, we, for the&#10;first time, employ the orthonormal representation as the minimal&#10;parameterization to model line features along with point features in visual&#10;SLAM and analytically derive the Jacobians of the re-projection errors with&#10;respect to the line parameters, which significantly improves the SLAM solution.&#10;The proposed SLAM has been extensively tested in both synthetic and real-world&#10;experiments whose results demonstrate that the proposed system outperforms the&#10;state-of-the-art methods in various scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.04102" label="1909.04102">
-        <attvalues>
-          <attvalue for="0" value="LIC-Fusion: LiDAR-Inertial-Camera Odometry" />
-          <attvalue for="1" value="  This paper presents a tightly-coupled multi-sensor fusion algorithm termed&#10;LiDAR-inertial-camera fusion (LIC-Fusion), which efficiently fuses IMU&#10;measurements, sparse visual features, and extracted LiDAR points. In&#10;particular, the proposed LIC-Fusion performs online spatial and temporal sensor&#10;calibration between all three asynchronous sensors, in order to compensate for&#10;possible calibration variations. The key contribution is the optimal (up to&#10;linearization errors) multi-modal sensor fusion of detected and tracked sparse&#10;edge/surf feature points from LiDAR scans within an efficient MSCKF-based&#10;framework, alongside sparse visual feature observations and IMU readings. We&#10;perform extensive experiments in both indoor and outdoor environments, showing&#10;that the proposed LIC-Fusion outperforms the state-of-the-art visual-inertial&#10;odometry (VIO) and LiDAR odometry methods in terms of estimation accuracy and&#10;robustness to aggressive motions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.05876" label="1805.05876">
-        <attvalues>
-          <attvalue for="0" value="Observability Analysis of Aided INS with Heterogeneous Features of&#10;  Points, Lines and Planes" />
-          <attvalue for="1" value="  In this paper, we perform a thorough observability analysis for linearized&#10;inertial navigation systems (INS) aided by exteroceptive range and/or bearing&#10;sensors (such as cameras, LiDAR and sonars) with different geometric features&#10;(points, lines and planes). While the observability of vision-aided INS (VINS)&#10;with point features has been extensively studied in the literature, we&#10;analytically show that the general aided INS with point features preserves the&#10;same observability property: that is, 4 unobservable directions, corresponding&#10;to the global yaw and the global position of the sensor platform. We further&#10;prove that there are at least 5 (and 7) unobservable directions for the&#10;linearized aided INS with a single line (and plane) feature; and, for the first&#10;time, analytically derive the unobservable subspace for the case of multiple&#10;lines/planes. Building upon this, we examine the system observability of the&#10;linearized aided INS with different combinations of points, lines and planes,&#10;and show that, in general, the system preserves at least 4 unobservable&#10;directions, while if global measurements are available, as expected, some&#10;unobservable directions diminish. In particular, when using plane features, we&#10;propose to use a minimal, closest point (CP) representation; and we also study&#10;in-depth the effects of 5 degenerate motions identified on observability. To&#10;numerically validate our analysis, we develop and evaluate both EKF-based&#10;visual-inertial SLAM and visual-inertial odometry (VIO) using heterogeneous&#10;geometric features in Monte Carlo simulations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.14997" label="2309.14997">
-        <attvalues>
-          <attvalue for="0" value="IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network" />
-          <attvalue for="1" value="  Infrared and visible image fusion (IVIF) is used to generate fusion images&#10;with comprehensive features of both images, which is beneficial for downstream&#10;vision tasks. However, current methods rarely consider the illumination&#10;condition in low-light environments, and the targets in the fused images are&#10;often not prominent. To address the above issues, we propose an&#10;Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet.&#10;In our framework, an illumination enhancement network first estimates the&#10;incident illumination maps of input images. Afterwards, with the help of&#10;proposed adaptive differential fusion module (ADFM) and salient target aware&#10;module (STAM), an image fusion network effectively integrates the salient&#10;features of the illumination-enhanced infrared and visible images into a fusion&#10;image of high visual quality. Extensive experimental results verify that our&#10;method outperforms five state-of-the-art methods of fusing infrared and visible&#10;images.&#10;" />
-          <attvalue for="2" value="&#10;&#10;D{ue} to the limited information captured by a single modality of the imaging device, the images obtained by multiple sensors are usually fused to produce a more comprehensive image about the monitored scene. In recent years, researchers have developed various methods to fuse multi-source images of the same scene as a high-quality image, which can strongly support the downstream computer vision tasks. &#10;In the field of image fusion, Infrared and Visible Image Fusion (IVIF) has been widely studied~\cite{LI2017100}. &#10;Specifically, infrared images capture and reflect the thermal radiation emitted from objects, making them suitable for detecting hidden targets. However, infrared images often lack significant textural details of the scene.&#10;In contrast, visible images contain the majority of visual information within a monitored scene, making them highly suitable for perception by the human visual system~\cite{zhang2017infrared}.&#10;Hence, the integration of the complementary features from the infrared image and visible image into a single fusion image is of utmost importance in fully comprehending the monitored scene.&#10;Although IVIF has been widely studied and applied in the military domain~\cite{wong2017nitroaromatic, XIAO2023144}, semantic segmentation~\cite{tang2022image, TANG2023101870} and target detection~\cite{cao2019pedestrian,tang2023divfusion}, it continues to pose a significant challenge due to the low-quality images captured in low-light environments and the absence of ground-truth fusion images.&#10;&#10;In recent decades, deep learning techniques have undergone significant development for IVIF. These techniques can be broadly categorized into three groups: auto-encoder (AE)-based \cite{li2018densefuse, li2021rfn, Reviewer2-3, 9502544, 9127964, 9349250}, convolutional neural network (CNN)-based \cite{liu2021smoa, ZHANG202099, xu2020u2fusion, di2022unsupervised, li2021different, liu2022coconet}, and generative adversarial network (GAN)-based methods \cite{ma2019fusiongan, ma2020ganmcc, 9894670, liu2022target, Reviewer2-1, Reviewer2-2}. DIDFuse~\cite{Reviewer2-3} pioneered the application of a decomposition network with a dual-stream structure to extract high and low-frequency features. SuperFusion~\cite{SuperFusion} integrates image registration, fusion, and the semantic requirements of high-level vision tasks into a comprehensive framework. Subsequently, SwinFusion~\cite{Reviewer1-1} and CDDFuse~\cite{Reviewer2-4} incorporate the Transformer structure, which models long-range dependencies, into the fusion network.&#10;The above methods could well fuse images and facilitate downstream multi-object detection tasks in the daytime, but their performance would be significantly decreased in low-light environments.&#10;&#10;Recently, a few methods were proposed to restore details in dark regions of fused images. To obtain fused images with adaptive brightness, PIA~\cite{tang2022piafusion} balanced the brightness information from different source images by considering the illumination contribution. DIVFusion~\cite{tang2023divfusion} addressed the visual degradation defect of fused images in low-light environments by the Retinex theory~\cite{rahman2004retinex}. Although these works have achieved promising performance, there are still some drawbacks:&#10;1) In the low-light environment, existing methods cannot efficiently balance the illumination characteristics of infrared and visible images to obtain high-quality fused images.&#10;2) The difference between multiple modalities may be large in extreme environments, introducing lots of noise or color distortion in fused images.&#10;&#10;To address the above challenges, in this work, we propose a novel infrared and visible image fusion framework based on illumination enhancement, namely IAIFNet. The main contributions of this work are mainly threefold:&#10;&#10;\begin{itemize}&#10;\item We propose a novel infrared and visible image fusion framework, which exploits illumination-aware information to effectively generate a fusion image of appropriate brightness and contrast.&#10;&#10;\item We design two novel modules, i.e., an Adaptive Differential Fusion Module (ADFM) and a Salient Target Aware Module (STAM), in particular to solve the issues caused by the incident light, alleviating the influence of under-exposure or over-exposure and enhancing the salient targets in the fused images.&#10;&#10;\item The proposed network is designed in a lightweight manner, thus it can quickly fuse infrared and visible images.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Physics, Electrical Engineering, Computer Vision Enhancement, Computer Science, Image Fusion Techniques, Low-Light Environment Processing, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1804.08361" label="1804.08361">
-        <attvalues>
-          <attvalue for="0" value="DenseFuse: A Fusion Approach to Infrared and Visible Images" />
-          <attvalue for="1" value="  In this paper, we present a novel deep learning architecture for infrared and&#10;visible images fusion problem. In contrast to conventional convolutional&#10;networks, our encoding network is combined by convolutional layers, fusion&#10;layer and dense block in which the output of each layer is connected to every&#10;other layer. We attempt to use this architecture to get more useful features&#10;from source images in encoding process. And two fusion layers(fusion&#10;strategies) are designed to fuse these features. Finally, the fused image is&#10;reconstructed by decoder. Compared with existing fusion methods, the proposed&#10;fusion method achieves state-of-the-art performance in objective and subjective&#10;assessment. Code and pre-trained models are available at&#10;https://github.com/hli1221/imagefusion_densefuse&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.09210" label="2003.09210">
-        <attvalues>
-          <attvalue for="0" value="DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion" />
-          <attvalue for="1" value="  Infrared and visible image fusion, a hot topic in the field of image&#10;processing, aims at obtaining fused images keeping the advantages of source&#10;images. This paper proposes a novel auto-encoder (AE) based fusion network. The&#10;core idea is that the encoder decomposes an image into background and detail&#10;feature maps with low- and high-frequency information, respectively, and that&#10;the decoder recovers the original image. To this end, the loss function makes&#10;the background/detail feature maps of source images similar/dissimilar. In the&#10;test phase, background and detail feature maps are respectively merged via a&#10;fusion module, and the fused image is recovered by the decoder. Qualitative and&#10;quantitative results illustrate that our method can generate fusion images&#10;containing highlighted targets and abundant detail texture information with&#10;strong robustness and meanwhile surpass state-of-the-art (SOTA) approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.14461" label="2211.14461">
-        <attvalues>
-          <attvalue for="0" value="CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for&#10;  Multi-Modality Image Fusion" />
-          <attvalue for="1" value="  Multi-modality (MM) image fusion aims to render fused images that maintain&#10;the merits of different modalities, e.g., functional highlight and detailed&#10;textures. To tackle the challenge in modeling cross-modality features and&#10;decomposing desirable modality-specific and modality-shared features, we&#10;propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse)&#10;network. Firstly, CDDFuse uses Restormer blocks to extract cross-modality&#10;shallow features. We then introduce a dual-branch Transformer-CNN feature&#10;extractor with Lite Transformer (LT) blocks leveraging long-range attention to&#10;handle low-frequency global features and Invertible Neural Networks (INN)&#10;blocks focusing on extracting high-frequency local information. A&#10;correlation-driven loss is further proposed to make the low-frequency features&#10;correlated while the high-frequency features uncorrelated based on the embedded&#10;information. Then, the LT-based global fusion and INN-based local fusion layers&#10;output the fused image. Extensive experiments demonstrate that our CDDFuse&#10;achieves promising results in multiple fusion tasks, including infrared-visible&#10;image fusion and medical image fusion. We also show that CDDFuse can boost the&#10;performance in downstream infrared-visible semantic segmentation and object&#10;detection in a unified benchmark. The code is available at&#10;https://github.com/Zhaozixiang1228/MMIF-CDDFuse.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.11316" label="1909.11316">
-        <attvalues>
-          <attvalue for="0" value="Cross-View Kernel Similarity Metric Learning Using Pairwise Constraints&#10;  for Person Re-identification" />
-          <attvalue for="1" value="  Person re-identification is the task of matching pedestrian images across&#10;non-overlapping cameras. In this paper, we propose a non-linear cross-view&#10;similarity metric learning for handling small size training data in practical&#10;re-ID systems. The method employs non-linear mappings combined with cross-view&#10;discriminative subspace learning and cross-view distance metric learning based&#10;on pairwise similarity constraints. It is a natural extension of XQDA from&#10;linear to non-linear mappings using kernels, and learns non-linear&#10;transformations for efficiently handling complex non-linearity of person&#10;appearance across camera views. Importantly, the proposed method is very&#10;computationally efficient. Extensive experiments on four challenging datasets&#10;shows that our method attains competitive performance against state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Person re-identification (re-ID) is the problem of matching person images from one camera view against the images captured from other non-overlapping camera views. Re-ID is a very challenging task as images of same person have significant appearance changes across views, due to large variation in illumination, background and pose. Also the low resolution surveillance cameras and common pedestrian attributes cause high visual similarity among different persons. &#10;&#10;Most existing methods for person re-identification concentrate on (i) design of identity discriminative feature descriptors and (ii) distance metric learning. The hand crafted feature descriptors \cite{LOMO,GOG,LisantiPAMI14} have improved the re-ID performance, but they are alone insufficient in handling the large appearance changes across cameras. Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated. &#10;&#10;In recent years, though deep learning methods \cite{ImprDeep,MuDeep,PTGAN,Beyond:triplet_loss, DGD, TCP, SpindleNet, SLSTM, SCNN} have made good improvement in re-ID performance, they have a fundamental limitation in practical deployment as they need a large, annotated training data. &#10;Even with pre-trained networks, based on auxiliary/external supervision, such methods struggle to perform on small size training data.&#10;Hence we refrain from using deep learning methods in this paper and instead concentrate on the following problem: &quot;Given a small size training data with given feature descriptors, can we design a better re-ID system, without using any auxiliary/external supervision&quot;. &#10;&#10;Metric learning methods have shown a good performance in handling small size training data. &#10; However, most of them have two fundamental limitations: &#10; (I) Small Sample Size (SSS) problem: The SSS problem occurs when the number of training samples is less than the feature dimension. This creates singularity of inter/intra class scatter matrices. Hence most methods use unsupervised dimensionality reduction, which tend to make them sub-optimal. &#10; (II) Less Efficient Models: Person appearance undergoes complex non-linear transformation across views. However, most existing methods use an inherent linear transformation of the input features, which limits their capability in learning non-linear features. &#10;&#10;For addressing the above two limitations, we propose a new non-linear metric learning method, referred to as, Kernel Cross-view Quadratic Discriminant Analysis (k-XQDA).&#10;It is a kernalized (non-linear) counterpart of XQDA\cite{LOMO}, which is one of the most popularly applied metric learning method in re-ID literature. k-XQDA uses mapping of the data samples to a very high dimensional kernel space, where it learns a cross-view distance metric and a cross-view discriminative subspace simultaneously, using pairwise similarity constraints. It is capable of learning highly effective non-linear features in the input feature space. k-XQDA efficiently handles the non-linearity in cross-view appearance and perform competitively against state-of-the-art methods. Importantly, our kernelized approach is computationally more efficient compared to the baseline methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Pattern Recognition, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="1806.09695" label="1806.09695">
-        <attvalues>
-          <attvalue for="0" value="Person Re-Identification in Identity Regression Space" />
-          <attvalue for="1" value="  Most existing person re-identification (re-id) methods are unsuitable for&#10;real-world deployment due to two reasons: Unscalability to large population&#10;size, and Inadaptability over time. In this work, we present a unified solution&#10;to address both problems. Specifically, we propose to construct an Identity&#10;Regression Space (IRS) based on embedding different training person identities&#10;(classes) and formulate re-id as a regression problem solved by identity&#10;regression in the IRS. The IRS approach is characterised by a closed-form&#10;solution with high learning efficiency and an inherent incremental learning&#10;capability with human-in-the-loop. Extensive experiments on four benchmarking&#10;datasets(VIPeR, CUHK01, CUHK03 and Market-1501) show that the IRS model not&#10;only outperforms state-of-the-art re-id methods, but also is more scalable to&#10;large re-id population size by rapidly updating model and actively selecting&#10;informative samples with reduced human labelling effort.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.03905" label="1910.03905">
-        <attvalues>
-          <attvalue for="0" value="A Semi-Supervised Maximum Margin Metric Learning Approach for Small&#10;  Scale Person Re-identification" />
-          <attvalue for="1" value="  In video surveillance, person re-identification is the task of searching&#10;person images in non-overlapping cameras. Though supervised methods for person&#10;re-identification have attained impressive performance, obtaining large scale&#10;cross-view labeled training data is very expensive. However, unlabelled data is&#10;available in abundance. In this paper, we propose a semi-supervised metric&#10;learning approach that can utilize information in unlabelled data with the help&#10;of a few labelled training samples. We also address the small sample size&#10;problem that inherently occurs due to the few labeled training data. Our method&#10;learns a discriminative space where within class samples collapse to singular&#10;points, achieving the least within class variance, and then use a maximum&#10;margin criterion over a high dimensional kernel space to maximally separate the&#10;distinct class samples. A maximum margin criterion with two levels of high&#10;dimensional mappings to kernel space is used to obtain better cross-view&#10;discrimination of the identities. Cross-view affinity learning with reciprocal&#10;nearest neighbor constraints is used to mine new pseudo-classes from the&#10;unlabelled data and update the distance metric iteratively. We attain&#10;state-of-the-art performance on four challenging datasets with a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.05165" label="1709.05165">
-        <attvalues>
-          <attvalue for="0" value="Multi-scale Deep Learning Architectures for Person Re-identification" />
-          <attvalue for="1" value="  Person Re-identification (re-id) aims to match people across non-overlapping&#10;camera views in a public space. It is a challenging problem because many people&#10;captured in surveillance videos wear similar clothes. Consequently, the&#10;differences in their appearance are often subtle and only detectable at the&#10;right location and scales. Existing re-id models, particularly the recently&#10;proposed deep learning based ones match people at a single scale. In contrast,&#10;in this paper, a novel multi-scale deep learning model is proposed. Our model&#10;is able to learn deep discriminative feature representations at different&#10;scales and automatically determine the most suitable scales for matching. The&#10;importance of different spatial locations for extracting discriminative&#10;features is also learned explicitly. Experiments are carried out to demonstrate&#10;that the proposed model outperforms the state-of-the art on a number of&#10;benchmarks&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08565" label="1711.08565">
-        <attvalues>
-          <attvalue for="0" value="Person Transfer GAN to Bridge Domain Gap for Person Re-Identification" />
-          <attvalue for="1" value="  Although the performance of person Re-Identification (ReID) has been&#10;significantly boosted, many challenging issues in real scenarios have not been&#10;fully investigated, e.g., the complex scenes and lighting variations, viewpoint&#10;and pose changes, and the large number of identities in a camera network. To&#10;facilitate the research towards conquering those issues, this paper contributes&#10;a new dataset called MSMT17 with many important features, e.g., 1) the raw&#10;videos are taken by an 15-camera network deployed in both indoor and outdoor&#10;scenes, 2) the videos cover a long period of time and present complex lighting&#10;variations, and 3) it contains currently the largest number of annotated&#10;identities, i.e., 4,101 identities and 126,441 bounding boxes. We also observe&#10;that, domain gap commonly exists between datasets, which essentially causes&#10;severe performance drop when training and testing on different datasets. This&#10;results in that available training data cannot be effectively leveraged for new&#10;testing domains. To relieve the expensive costs of annotating new training&#10;samples, we propose a Person Transfer Generative Adversarial Network (PTGAN) to&#10;bridge the domain gap. Comprehensive experiments show that the domain gap could&#10;be substantially narrowed-down by the PTGAN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.04613" label="2212.04613">
-        <attvalues>
-          <attvalue for="0" value="Contrastive View Design Strategies to Enhance Robustness to Domain&#10;  Shifts in Downstream Object Detection" />
-          <attvalue for="1" value="  Contrastive learning has emerged as a competitive pretraining method for&#10;object detection. Despite this progress, there has been minimal investigation&#10;into the robustness of contrastively pretrained detectors when faced with&#10;domain shifts. To address this gap, we conduct an empirical study of&#10;contrastive learning and out-of-domain object detection, studying how&#10;contrastive view design affects robustness. In particular, we perform a case&#10;study of the detection-focused pretext task Instance Localization (InsLoc) and&#10;propose strategies to augment views and enhance robustness in&#10;appearance-shifted and context-shifted scenarios. Amongst these strategies, we&#10;propose changes to cropping such as altering the percentage used, adding IoU&#10;constraints, and integrating saliency based object priors. We also explore the&#10;addition of shortcut-reducing augmentations such as Poisson blending, texture&#10;flattening, and elastic deformation. We benchmark these strategies on abstract,&#10;weather, and context domain shifts and illustrate robust ways to combine them,&#10;in both pretraining on single-object and multi-object image datasets. Overall,&#10;our results and insights show how to ensure robustness through the choice of&#10;views in contrastive learning.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Self-supervised learning has been rising in popularity in computer vision, with many top methods using contrastive learning \cite{hadsell2006dimensionality}, a form of learning that optimizes feature representations for positive samples to be close together and for negative samples to be far apart. Self-supervised contrastive models such as SimCLR \cite{chen2020simple} and MoCo \cite{he2020momentum} have been shown to approach or surpass the performance of supervised models when representations are transferred to downstream image classification, object detection, and semantic segmentation tasks \cite{ericsson2021well}. This success is in part due to strategic data augmentation pipelines that these models use to create effective positive and negative views (samples) for learning. &#10;&#10;Despite this progress, the out-of-distribution robustness of contrastive representations has been minimally studied, especially with regards to object detection. We hypothesize that existing data augmentation pipelines in contrastive learning may result in representations that lack robustness in such domain-shifted detection scenarios. For example, as shown in Fig. \ref{intro_fig}, state-of-the-art pipelines (e.g. SimCLR, MoCo) create positive views with aggressive random cropping of a single image. Such use of this augmentation can lead to features for object regions being made similar to those for the background or co-occurring objects, potentially causing contextual bias and hurting in out-of-context scenarios. Random cropping may also result in texture-biased rather than shape-biased features, since the shapes of object parts may not be consistent across crops. A lack of shape in representations can lead to degraded performance when texture is shifted (e.g. appearance changes due to weather).&#10;&#10;In this work, we explore strategies to improve contrastive view design for enhanced robustness in domain-shifted detection. In particular, we conduct an empirical study of the state-of-the-art detection pretext task InsLoc \cite{yang2021instance} and strategically alter the views created from InsLoc's data augmentation pipeline by adjusting cropping, adding shortcut-reducing appearance augmentations, and integrating saliency-based object priors. These strategies are evaluated in-domain and out-of-domain in appearance-shifted and context-shifted detection scenarios. Experiments are also conducted following pretraining on both single-object and multi-object image datasets. From these experiments, we present these insights into contrastive view design:&#10;&#10;\begin{itemize}[itemsep=1mm, parsep=0pt]&#10; \item Increasing the minimum \% of an image used for crops or adding an IoU constraint between views causes the model to learn from spatially consistent views with larger object parts. These strategies encourage the learning of shape and improve robustness to appearance shifts.&#10; \item Shortcut-reducing augmentations enhance the effectiveness of non-aggressive cropping, exemplified by improvements over InsLoc both in-domain (up to +2.73 AP) and out-of-domain (up to +3.07 AP).&#10; \item The use of saliency priors in views is effective for out-of-context robustness. Their use is best in a mechanism that removes background and tightens crops to object regions.&#10; \item Applying shortcut-reducing augmentation to the non-salient regions in views, in combination with crop tightening and shape strategies, is effective for enhancing robustness to both appearance and context shifts.&#10; \end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Contrastive Learning, Computer Science, Domain Shift Robustness" />
-        </attvalues>
-      </node>
-      <node id="2011.13377" label="2011.13377">
-        <attvalues>
-          <attvalue for="0" value="How Well Do Self-Supervised Models Transfer?" />
-          <attvalue for="1" value="  Self-supervised visual representation learning has seen huge progress&#10;recently, but no large scale evaluation has compared the many models now&#10;available. We evaluate the transfer performance of 13 top self-supervised&#10;models on 40 downstream tasks, including many-shot and few-shot recognition,&#10;object detection, and dense prediction. We compare their performance to a&#10;supervised baseline and show that on most tasks the best self-supervised models&#10;outperform supervision, confirming the recently observed trend in the&#10;literature. We find ImageNet Top-1 accuracy to be highly correlated with&#10;transfer to many-shot recognition, but increasingly less so for few-shot,&#10;object detection and dense prediction. No single self-supervised method&#10;dominates overall, suggesting that universal pre-training is still unsolved.&#10;Our analysis of features suggests that top self-supervised learners fail to&#10;preserve colour information as well as supervised alternatives, but tend to&#10;induce better classifier calibration, and less attentive overfitting than&#10;supervised learners.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\keypoint{Self-supervised learning} Self-supervised representation learning is now a large topic that it is impossible to cover completely here, and we point the reader to excellent recent surveys \cite{Jing2020Self-supervisedSurvey,Liu2020Self-supervisedContrastive} for thorough reviews. In this paper, we focus on still-image self-supervised learning, where a common paradigm is to pre-train on ImageNet \cite{Deng2009ImageNet:Database} using a variety of pre-text tasks from jigsaw puzzles \cite{Noroozi2016UnsupervisedPuzzles} to colorization \cite{Zhang2016ColorfulColorization,Larsson2016LearningColorization} to instance discrimination \cite{Wu2018UnsupervisedDiscrimination,Dosovitskiy2014DiscriminativeNetworks,Chen2020ARepresentations,He2019MomentumLearning} and clustering \cite{Li2020PrototypicalRepresentations,Caron2020UnsupervisedAssignments}. Evaluation is then typically performed by using the learned representation to train a linear classifier on ImageNet \cite{He2019MomentumLearning}, or finetune the representation with a small amount of data \cite{Chen2020BigLearners}. However, evaluation of the impact on different downstream datasets (where there is domain shift \cite{Zhang2020ImpactAdaptation} with respect to ImageNet), and non-recognition tasks has been highly inconsistent -- a gap in the literature that we aim to remedy in this paper.&#10;&#10;To do this we wish to evaluate a large number of self-supervised methods, covering a wide range of training objectives. Many recent works adopt a form of instance discrimination \cite{Dosovitskiy2014DiscriminativeNetworks,Wu2018UnsupervisedDiscrimination,Misra2020Self-SupervisedRepresentations}, whereby each training image is treated as its own class. By applying strong data augmentation to these images, and comparing them using a contrastive \cite{Gutmann2010Noise-contrastiveModels,Tian2019ContrastiveCoding,Henaff2019Data-EfficientCoding} loss, a model can learn features which are resilient to various changes in view. The main difficulty in instance discrimination lies in approximating the loss over all instances, as it becomes intractable for large datasets. This leads to metric learning methods which require large numbers of pairwise comparisons. The scaling problem that still remains has been tackled by using memory banks of features \cite{Wu2018UnsupervisedDiscrimination}, momentum encoders \cite{He2019MomentumLearning} or very large batches \cite{Chen2020ARepresentations}. On the other side, clustering-based approaches \cite{Caron2018DeepFeatures,Asano2020Self-labellingLearning} compare groups of images with similar features, sidestepping the intractability of instance discrimination. The problem here instead is computing the cluster assignments over the entire training set. These approaches therefore tend to focus on ways of performing this assignment online \cite{Zhan2020OnlineLearning,Caron2020UnsupervisedAssignments}. Among recent methods, BYOL stands out as one which does not directly use either a contrastive or clustering approach, but as noted by \cite{Tian2020UnderstandingNetworks}, an implicit contrastive loss term is created by their use of batch normalisation. In this paper, we evaluate methods using all of the above approaches, investigating the effect of training objective on transfer performance and representation quality.&#10;&#10;\keypoint{Prior evaluations and benchmarks} The importance of empirical evaluation of general purpose representation learning is highlighted by the growing number of major evaluation papers in this area \cite{Kornblith2019DoBetter,Goyal2019ScalingLearning,Zhai2019ABenchmark,Kolesnikov2019RevisitingLearning}.&#10;In terms of transfer performance from supervised pre-training, \cite{Kornblith2019DoBetter} proposes a suite of downstream recognition task evaluations and evaluates transfer performance of several supervised models of varying architecture and pre-training details. They find very strong correlations between ImageNet performance and transfer performance on downstream tasks. In contrast, we compare pre-trained models of exactly the same (ResNet-50) architecture, and instead evaluate the impact of the different training objectives and augmentation strategies used by self-supervised learners; as well as considering a more diverse suite of downstream benchmarks including few-shot recognition, object detection and dense prediction. Our results are more nuanced, with high correlation visible in recognition tasks similar to ImageNet and lower correlation elsewhere. \cite{Goyal2019ScalingLearning} propose a richer range of downstream benchmarks to evaluate self-supervised pre-training, but focus on the impact of different pre-training datasets and CNN architectures. In contrast, we provide the first comprehensive comparison of different self-supervised algorithms, holding architecture and dataset constant. \cite{Kolesnikov2019RevisitingLearning} compares a few architectures and SSL algorithms on a small number of downstream tasks, and draw observations such as pre-text task performance being uncorrelated with representation performance on ImageNet recognition. In contrast, we evaluate whether performance on the commonly evaluated ImageNet recognition is indicative of in-the-wild performance on diverse downstream datasets and non-recognition tasks. The evaluation in \cite{Zhai2019ABenchmark} finds that self-supervised methods can not beat supervised models. We find that a more recent family of self-supervised learners consistently achieve the highest performances, on recognition, detection, surface normal estimation and semantic segmentation, with the one exception of few-shot recognition on ImageNet-like data.&#10; " />
-        </attvalues>
-      </node>
-      <node id="2306.02894" label="2306.02894">
-        <attvalues>
-          <attvalue for="0" value="Recyclable Semi-supervised Method Based on Multi-model Ensemble for&#10;  Video Scene Parsing" />
-          <attvalue for="1" value="  Pixel-level Scene Understanding is one of the fundamental problems in&#10;computer vision, which aims at recognizing object classes, masks and semantics&#10;of each pixel in the given image. Since the real-world is actually video-based&#10;rather than a static state, learning to perform video semantic segmentation is&#10;more reasonable and practical for realistic applications. In this paper, we&#10;adopt Mask2Former as architecture and ViT-Adapter as backbone. Then, we propose&#10;a recyclable semi-supervised training method based on multi-model ensemble. Our&#10;method achieves the mIoU scores of 62.97% and 65.83% on Development test and&#10;final test respectively. Finally, we obtain the 2nd place in the Video Scene&#10;Parsing in the Wild Challenge at CVPR 2023.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Semantic segmentation is a dense pixel prediction task that assigns a semantic label to each pixel in an image or video, such as person, car, building, etc. Thanks to the rapid development of deep learning and the availability of various semantic segmentation datasets, significant progress has been made in image semantic segmentation. However, the real world is dynamic and constantly changing, and it is more practical to learn how to perform video semantic segmentation for realistic applications rather than relying solely on static images. Video semantic segmentation is even more challenging. It requires considering the relationship between the temporal and spatial dimensions, as well as tackling issues such as object motion, occlusion, and background variations in videos.&#10;&#10;Many studies have focused on exploiting the inter-frame redundancy of videos, such as using optical flow information, spatio-temporal attention networks, etc., which can better consider the temporal and spatial relationships in videos and improve the performance of video semantic segmentation. In addition to model structure advancements, the quality and scale of datasets have also received increasing attention. The Video Scene Parsing in the Wild(VSPW) dataset \cite{miao2021vspw} is the first attempt to tackle the challenging video scene parsing task by considering diverse scenarios, which covers a wide range of real-world scenarios (e.g., art galleries, lecture rooms, beach, and street views) and categories from both things (e.g., person, car, desk) and stuff (e.g., road, wall, sky). Furthermore, joint learning and weakly supervised learning have become hot topics in recent years. Joint learning can combine image and video data to improve the performance of video semantic segmentation, while weakly supervised learning can train models with fewer labeled data, reducing annotation costs.&#10;&#10;In this paper, we propose the recyclable semi-supervised method\cite{chen2021semi}\cite{hung2018adversarial} based on multi-model ensemble. We finally obtain the scores of 62.97\% and 65.83\% mIoU on test\_part1 and testing set of VSPW, respectively. And our solution is ranked second place on PVUW2023 Video semantic segmentation challenge at CVPR2023.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2106.01226" label="2106.01226">
-        <attvalues>
-          <attvalue for="0" value="Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision" />
-          <attvalue for="1" value="  In this paper, we study the semi-supervised semantic segmentation problem via&#10;exploring both labeled data and extra unlabeled data. We propose a novel&#10;consistency regularization approach, called cross pseudo supervision (CPS). Our&#10;approach imposes the consistency on two segmentation networks perturbed with&#10;different initialization for the same input image. The pseudo one-hot label&#10;map, output from one perturbed segmentation network, is used to supervise the&#10;other segmentation network with the standard cross-entropy loss, and vice&#10;versa. The CPS consistency has two roles: encourage high similarity between the&#10;predictions of two perturbed networks for the same input image, and expand&#10;training data by using the unlabeled data with pseudo labels. Experiment&#10;results show that our approach achieves the state-of-the-art semi-supervised&#10;segmentation performance on Cityscapes and PASCAL VOC 2012. Code is available&#10;at https://git.io/CPS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.07934" label="1802.07934">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Learning for Semi-Supervised Semantic Segmentation" />
-          <attvalue for="1" value="  We propose a method for semi-supervised semantic segmentation using an&#10;adversarial network. While most existing discriminators are trained to classify&#10;input images as real or fake on the image level, we design a discriminator in a&#10;fully convolutional manner to differentiate the predicted probability maps from&#10;the ground truth segmentation distribution with the consideration of the&#10;spatial resolution. We show that the proposed discriminator can be used to&#10;improve semantic segmentation accuracy by coupling the adversarial loss with&#10;the standard cross entropy loss of the proposed model. In addition, the fully&#10;convolutional discriminator enables semi-supervised learning through&#10;discovering the trustworthy regions in predicted results of unlabeled images,&#10;thereby providing additional supervisory signals. In contrast to existing&#10;methods that utilize weakly-labeled images, our method leverages unlabeled&#10;images to enhance the segmentation model. Experimental results on the PASCAL&#10;VOC 2012 and Cityscapes datasets demonstrate the effectiveness of the proposed&#10;algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.10540" label="2002.10540">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Saliency Metrics: Farthest-Neighbor Area Under Curve" />
-          <attvalue for="1" value="  Saliency detection has been widely studied because it plays an important role&#10;in various vision applications, but it is difficult to evaluate saliency&#10;systems because each measure has its own bias. In this paper, we first revisit&#10;the problem of applying the widely used saliency metrics on modern&#10;Convolutional Neural Networks(CNNs). Our investigation shows the saliency&#10;datasets have been built based on different choices of parameters and CNNs are&#10;designed to fit a dataset-specific distribution. Secondly, we show that the&#10;Shuffled Area Under Curve(S-AUC) metric still suffers from spatial biases. We&#10;propose a new saliency metric based on the AUC property, which aims at sampling&#10;a more directional negative set for evaluation, denoted as Farthest-Neighbor&#10;AUC(FN-AUC). We also propose a strategy to measure the quality of the sampled&#10;negative set. Our experiment shows FN-AUC can measure spatial biases, central&#10;and peripheral, more effectively than S-AUC without penalizing the fixation&#10;locations. Thirdly, we propose a global smoothing function to overcome the&#10;problem of few value degrees (output quantization) in computing AUC metrics.&#10;Comparing with random noise, our smooth function can create unique values&#10;without losing the relative saliency relationship.&#10;" />
-          <attvalue for="2" value="&#10;Extensive studies have been proposed to predict the most salient region within an image. Saliency methods can be roughly grouped into two categories, bottom-up and top-down. The former considers the visual stimuli of an image to determine the Regions Of Interest(ROIs); while the latter one assumes the ROI is task-dependent, prior knowledge plays a significant part in saliency prediction. With the recent development of Convolutional Neural Networks (CNNs), saliency prediction heavily relies on model-based algorithms which can be trained in an end-to-end fashion. A new question has emerged regarding what type of saliency features are the best design for applications, top-down or bottom-up? Hand-crafted or CNN-learned?&#10;&#10;The measurement of saliency is still challenging because the definition of ``saliency'' varies depending on the vision task \cite{Boisvert16}, so saliency algorithms can also be grouped by different taxonomies for different purposes \cite{Bylinskii15,Frintrop10,Borji13b}. In this paper, we follow the common problem setting in \cite{MIT1003,SPROC}, computational models are trained to predict the most salient region to the Human Visual System(HVS). Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}. However, it is still difficult to comprehensively evaluate saliency models due to the bias of each metric. For instance, the challenge of Large-Scale Scene Understanding \cite{SALICON} uses seven saliency metrics, Shuffled Area Under Curve (S-AUC), Information Gain (IG) \cite{Matthias15}, Normalized Scanpath Saliency (NSS) \cite{NSS}, Pearson’s Correlation Coefficient (CC), AUC-Judd, SIMilarity (SIM) and Kullback–Leibler Divergence (KLD) \cite{KL}. Another public saliency benchmark, MIT300 \cite{MIT300}, applies eight saliency metrics, AUC-Judd, AUC-Borji, S-AUC, NSS, CC, KLD, SIM and Earth Mover’s Distance (EMD) \cite{EMD}. The use of multiple measures indicates that it is difficult to evaluate a model from only one angle. Previous studies intended to categorize and compare those metrics, e.g., \cite{Riche13,Bylinskii19} grouped the metrics into location-based and distribution-based. As shown in Figure~\ref{fig:sample}, the location-based measure consists of a set of fixation locations captured by an eye tracker or mouse click. While the distribution of saliency is normally considered as a post-process on the raw data by applying a Gaussian filter. &#10;&#10;To evaluate a saliency model, one solution is to overcome the ``disagreement'' among the metrics. Kummerer et al. \cite{Matthias15} proposed to optimize the saliency scale, center bias and spatial blurring jointly. But their post-process can hardly satisfy all the metrics simultaneously, the process requires all the compared models and optimization only uses the loss of IG. Later this idea was extended to a metric-tailored design \cite{Matthias18}, saliency models and saliency maps are decoupled so that one model can output different metric specific maps. However, their solutions are based on the assumption that all the metrics are able to evaluate saliency reasonably, the output should be optimized separately and specifically for each metric. In this work, we are more interested in investigating the differences among those metrics in theory. We believe some of the metrics may contain inherent drawbacks so that not necessarily all the metrics should be considered. The first contribution of this work is that we revisit the widely used saliency metrics based on \cite{Bylinskii19}, showing that the ``balanced'' metrics, NSS and CC, still have limitations in evaluating modern CNN-based systems. Saliency datasets have been created using their own choices of Gaussian standard deviation and a CNN model learns to fit this biased distribution, see Section~\ref{sec:metric}. &#10;&#10;Center bias is a long-standing problem in saliency evaluation, simply placing a Gaussian distribution at the center could outperform a well-designed system on most of the metrics \cite{SUN}. S-AUC is a common solution used by the SALICON and the MIT benchmarks, a negative set is sampled based on the positive from other images within the same dataset. The study \cite{Bylinskii19} shows a centered Gaussian distribution can only achieve an S-AUC score of $0.5$. However, S-AUC has a strong bias that it only considers negatives near the center, peripherally-favored systems can achieve higher scores \cite{Bruce15}. In this paper, we propose a new saliency metric which introduces one more constraint on the spatial relationship between the positive and negative. We show that the distribution of all fixations can be interpreted as a 2D probability density distribution. Our method builds the negative set for each image by searching its farthest neighbors according to the distribution similarity, denoted as FN-AUC. We also propose a fast version of our FN-AUC in case the size of the dataset is too large. To compare with S-AUC, we propose a strategy to measure the quality of the sampled negative set, which takes the spatial relationship of both the center bias and the positive into account, see Section~\ref{sec:fnauc}. Our experiment shows FN-AUC can draw a more reasonable negative set in order to penalize the center bias only without undermining the true positives, Section~\ref{sec:case}.&#10;&#10;Another contribution of this work is that we propose a global smoothing strategy in computing AUC metrics. Based on the MIT benchmark, it is problematic to compute the receiver operating characteristic (ROC) property when many locations share the same value magnitude, which could result in a lower performance as well. This is a scenario that is common for CNN based models which can produce near binary outputs. One solution could be jittering a map by adding small random noise, but this may break the relative saliency rank. We propose to apply a Gaussian filter using a relatively large standard deviation, the output is expected to cover all regions within a map. In this way, our method can generate a map with unique values for an AUC metric, meanwhile retaining the relative saliency relationship, see Section~\ref{sec:global}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Convolutional Neural Networks, Mathematics, Saliency Detection, Artificial Intelligence, Cognitive Science, Computer Science, Evaluation Metrics" />
-        </attvalues>
-      </node>
-      <node id="1611.09571" label="1611.09571">
-        <attvalues>
-          <attvalue for="0" value="Predicting Human Eye Fixations via an LSTM-based Saliency Attentive&#10;  Model" />
-          <attvalue for="1" value="  Data-driven saliency has recently gained a lot of attention thanks to the use&#10;of Convolutional Neural Networks for predicting gaze fixations. In this paper&#10;we go beyond standard approaches to saliency prediction, in which gaze maps are&#10;computed with a feed-forward network, and present a novel model which can&#10;predict accurate saliency maps by incorporating neural attentive mechanisms.&#10;The core of our solution is a Convolutional LSTM that focuses on the most&#10;salient regions of the input image to iteratively refine the predicted saliency&#10;map. Additionally, to tackle the center bias typical of human eye fixations,&#10;our model can learn a set of prior maps generated with Gaussian functions. We&#10;show, through an extensive evaluation, that the proposed architecture&#10;outperforms the current state of the art on public saliency prediction&#10;datasets. We further study the contribution of each key component to&#10;demonstrate their robustness on different scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.01064" label="1609.01064">
-        <attvalues>
-          <attvalue for="0" value="A Deep Multi-Level Network for Saliency Prediction" />
-          <attvalue for="1" value="  This paper presents a novel deep architecture for saliency prediction.&#10;Current state of the art models for saliency prediction employ Fully&#10;Convolutional networks that perform a non-linear combination of features&#10;extracted from the last convolutional layer to predict saliency maps. We&#10;propose an architecture which, instead, combines features extracted at&#10;different levels of a Convolutional Neural Network (CNN). Our model is composed&#10;of three main blocks: a feature extraction CNN, a feature encoding network,&#10;that weights low and high level feature maps, and a prior learning network. We&#10;compare our solution with state of the art saliency models on two public&#10;benchmarks datasets. Results show that our model outperforms under all&#10;evaluation metrics on the SALICON dataset, which is currently the largest&#10;public dataset for saliency prediction, and achieves competitive results on the&#10;MIT300 benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.1045" label="1411.1045">
-        <attvalues>
-          <attvalue for="0" value="Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on&#10;  ImageNet" />
-          <attvalue for="1" value="  Recent results suggest that state-of-the-art saliency models perform far from&#10;optimal in predicting fixations. This lack in performance has been attributed&#10;to an inability to model the influence of high-level image features such as&#10;objects. Recent seminal advances in applying deep neural networks to tasks like&#10;object recognition suggests that they are able to capture this kind of&#10;structure. However, the enormous amount of training data necessary to train&#10;these networks makes them difficult to apply directly to saliency prediction.&#10;We present a novel way of reusing existing neural networks that have been&#10;pretrained on the task of object recognition in models of fixation prediction.&#10;Using the well-known network of Krizhevsky et al. (2012), we come up with a new&#10;saliency model that significantly outperforms all state-of-the-art models on&#10;the MIT Saliency Benchmark. We show that the structure of this network allows&#10;new insights in the psychophysics of fixation selection and potentially their&#10;neural implementation. To train our network, we build on recent work on the&#10;modeling of saliency as point processes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.01793" label="1804.01793">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Saliency Mapping via Probability Distribution Prediction" />
-          <attvalue for="1" value="  Most saliency estimation methods aim to explicitly model low-level&#10;conspicuity cues such as edges or blobs and may additionally incorporate&#10;top-down cues using face or text detection. Data-driven methods for training&#10;saliency models using eye-fixation data are increasingly popular, particularly&#10;with the introduction of large-scale datasets and deep architectures. However,&#10;current methods in this latter paradigm use loss functions designed for&#10;classification or regression tasks whereas saliency estimation is evaluated on&#10;topographical maps. In this work, we introduce a new saliency map model which&#10;formulates a map as a generalized Bernoulli distribution. We then train a deep&#10;architecture to predict such maps using novel loss functions which pair the&#10;softmax activation function with measures designed to compute distances between&#10;probability distributions. We show in extensive experiments the effectiveness&#10;of such loss functions over standard ones on four public benchmark datasets,&#10;and demonstrate improved performance over state-of-the-art saliency methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.06904" label="1704.06904">
-        <attvalues>
-          <attvalue for="0" value="Residual Attention Network for Image Classification" />
-          <attvalue for="1" value="  In this work, we propose &quot;Residual Attention Network&quot;, a convolutional neural&#10;network using attention mechanism which can incorporate with state-of-art feed&#10;forward network architecture in an end-to-end training fashion. Our Residual&#10;Attention Network is built by stacking Attention Modules which generate&#10;attention-aware features. The attention-aware features from different modules&#10;change adaptively as layers going deeper. Inside each Attention Module,&#10;bottom-up top-down feedforward structure is used to unfold the feedforward and&#10;feedback attention process into a single feedforward process. Importantly, we&#10;propose attention residual learning to train very deep Residual Attention&#10;Networks which can be easily scaled up to hundreds of layers. Extensive&#10;analyses are conducted on CIFAR-10 and CIFAR-100 datasets to verify the&#10;effectiveness of every module mentioned above. Our Residual Attention Network&#10;achieves state-of-the-art object recognition performance on three benchmark&#10;datasets including CIFAR-10 (3.90% error), CIFAR-100 (20.45% error) and&#10;ImageNet (4.8% single model and single crop, top-5 error). Note that, our&#10;method achieves 0.6% top-1 accuracy improvement with 46% trunk depth and 69%&#10;forward FLOPs comparing to ResNet-200. The experiment also demonstrates that&#10;our network is robust against noisy labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.01125" label="1809.01125">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Video Object Segmentation using Motion Saliency-Guided&#10;  Spatio-Temporal Propagation" />
-          <attvalue for="1" value="  Unsupervised video segmentation plays an important role in a wide variety of&#10;applications from object identification to compression. However, to date, fast&#10;motion, motion blur and occlusions pose significant challenges. To address&#10;these challenges for unsupervised video segmentation, we develop a novel&#10;saliency estimation technique as well as a novel neighborhood graph, based on&#10;optical flow and edge cues. Our approach leads to significantly better initial&#10;foreground-background estimates and their robust as well as accurate diffusion&#10;across time. We evaluate our proposed algorithm on the challenging DAVIS,&#10;SegTrack v2 and FBMS-59 datasets. Despite the usage of only a standard edge&#10;detector trained on 200 images, our method achieves state-of-the-art results&#10;outperforming deep learning based methods in the unsupervised setting. We even&#10;demonstrate competitive results comparable to deep learning based methods in&#10;the semi-supervised setting on the DAVIS dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.03605" label="1604.03605">
-        <attvalues>
-          <attvalue for="0" value="What do different evaluation metrics tell us about saliency models?" />
-          <attvalue for="1" value="  How best to evaluate a saliency model's ability to predict where humans look&#10;in images is an open research question. The choice of evaluation metric depends&#10;on how saliency is defined and how the ground truth is represented. Metrics&#10;differ in how they rank saliency models, and this results from how false&#10;positives and false negatives are treated, whether viewing biases are accounted&#10;for, whether spatial deviations are factored in, and how the saliency maps are&#10;pre-processed. In this paper, we provide an analysis of 8 different evaluation&#10;metrics and their properties. With the help of systematic experiments and&#10;visualizations of metric computations, we add interpretability to saliency&#10;scores and more transparency to the evaluation of saliency models. Building off&#10;the differences in metric properties and behaviors, we make recommendations for&#10;metric selections under specific assumptions and for specific applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.08615" label="1704.08615">
-        <attvalues>
-          <attvalue for="0" value="Saliency Benchmarking Made Easy: Separating Models, Maps and Metrics" />
-          <attvalue for="1" value="  Dozens of new models on fixation prediction are published every year and&#10;compared on open benchmarks such as MIT300 and LSUN. However, progress in the&#10;field can be difficult to judge because models are compared using a variety of&#10;inconsistent metrics. Here we show that no single saliency map can perform well&#10;under all metrics. Instead, we propose a principled approach to solve the&#10;benchmarking problem by separating the notions of saliency models, maps and&#10;metrics. Inspired by Bayesian decision theory, we define a saliency model to be&#10;a probabilistic model of fixation density prediction and a saliency map to be a&#10;metric-specific prediction derived from the model density which maximizes the&#10;expected performance on that metric given the model density. We derive these&#10;optimal saliency maps for the most commonly used saliency metrics (AUC, sAUC,&#10;NSS, CC, SIM, KL-Div) and show that they can be computed analytically or&#10;approximated with high precision. We show that this leads to consistent&#10;rankings in all metrics and avoids the penalties of using one saliency map for&#10;all metrics. Our method allows researchers to have their model compete on many&#10;different metrics with state-of-the-art in those metrics: &quot;good&quot; models will&#10;perform well in all metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.13714" label="2208.13714">
-        <attvalues>
-          <attvalue for="0" value="SphereDepth: Panorama Depth Estimation from Spherical Domain" />
-          <attvalue for="1" value="  The panorama image can simultaneously demonstrate complete information of the&#10;surrounding environment and has many advantages in virtual tourism, games,&#10;robotics, etc. However, the progress of panorama depth estimation cannot&#10;completely solve the problems of distortion and discontinuity caused by the&#10;commonly used projection methods. This paper proposes SphereDepth, a novel&#10;panorama depth estimation method that predicts the depth directly on the&#10;spherical mesh without projection preprocessing. The core idea is to establish&#10;the relationship between the panorama image and the spherical mesh and then use&#10;a deep neural network to extract features on the spherical domain to predict&#10;depth. To address the efficiency challenges brought by the high-resolution&#10;panorama data, we introduce two hyper-parameters for the proposed spherical&#10;mesh processing framework to balance the inference speed and accuracy.&#10;Validated on three public panorama datasets, SphereDepth achieves comparable&#10;results with the state-of-the-art methods of panorama depth estimation.&#10;Benefiting from the spherical domain setting, SphereDepth can generate a&#10;high-quality point cloud and significantly alleviate the issues of distortion&#10;and discontinuity.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Depth estimation attempts to infer the distance between the objects and the camera in the captured scene, which is a classical but important vision problem for many upstream 3D applications, such as scene reconstruction, semantic understanding, robotics navigation, etc. Traditionally, researchers focus on the problem of monocular depth estimation \cite{monodepth2017}, which learns the traditional 2D content captured by typical pinhole projection model-based cameras. To obtain the novel information (360$^\circ$ sensing) of a 3D scene, they integrate those single views based on multi-view consistency \cite{gmvs2019,yao2018mvsnet} and structure-from-motion (SfM) \cite{sfm2018,schonberger2016structure}. However, as the consumer-level 360$^\circ$ cameras are becoming more popular and rapidly developed in recent years, inferring the depth information from a panorama image naturally attracts the community. The panorama camera has a field-of-view (FoV) of 360$^\circ$ and can acquire a comprehensive view in just one shooting spot. &#10;&#10;However, as the best way to display the panorama image is using the sphere (a non-Euclidean space), the existing studies of monocular depth estimation based on convolutional neural networks (CNNs) \cite{eigen2014depth,laina2016deeper,zhou2017unsupervised} cannot be directly applied to its depth task. Therefore, the most popular solution is using some projection methods to convert the panorama image into a standard 2D image, such as equirectangular projection and cube map projection. Equirectangular projection provides a wide FoV mimicking a peripheral vision but introduces distortion \cite{sun2019horizonnet,su2017learning,zioulis2018omnidepth}, while cube map projection provides a smaller and non-distorted FoV mimicking the foveal vision but introduces discontinuity \cite{cheng2018cube,jiang2021unifuse,jin2020geometric}. Overall speaking, those methods based on projection will inevitably suffer more or less from distortion and discontinuity and bring systematic errors to the upstream visual perception tasks. &#10;&#10;To this end, this paper proposes a novel panorama depth estimation method called SphereDepth, that directly predicts the depth map in the spherical domain without any projection. SphereDepth uses a spherical mesh to approximate the sphere and treats each triangle in a spherical mesh as a pixel in an image. After assigning each triangle with an RGB value, we can use a customized mesh convolution kernel to extract the features for depth estimation. We then construct a mesh-based convolution to directly perform depth estimation on a spherical mesh. Fig. \ref{fig:compare} compares the point cloud generated by SphereDepth with the existing state-of-the-art algorithm BiFuse \cite{wang2020bifuse} and SliceNet \cite{pintore2021slicenet}. From the enlarged parts of the point cloud, we observe that SphereDepth provides accurate and smooth results while BiFuse and SliceNet exhibit discontinuity and distortion.&#10;&#10;However, representing a high-resolution panorama image needs a spherical mesh with high resolution and more triangles. Unlike the perspective image organized in a 2D pixel matrix, the spherical mesh uses triangles to represent each pixel. The topological complexity of the spherical mesh exponentially grows with the number of triangles increasing, which degrades the computing efficiency and increases the memory footprint.&#10;&#10;To address this challenge, we split the spherical resolution (SR) into two types of resolutions: the mesh resolution (MR), which indicates the number of triangles on a spherical mesh, and the triangle resolution (TR), which represents how many pixels each triangle has. On the one hand, a higher MR can represent a higher resolution panorama image with more details and naturally lead to larger resource consumption. On the other hand, compared to increasing MR, a high TR can enhance the local features of a mesh triangle with much less computational demand and memory demand. In practice, we can tune MR and TR in the spherical domain to balance the accuracy and processing speed of the deep network. Our contributions are summarized as follows.&#10;&#10;\begin{itemize}&#10;&#09;\item SphereDepth is an end-to-end network to infer the panorama depth map in the spherical domain, which fundamentally solves the problem of distortion and discontinuity caused by projection methods;&#10;&#09;\item We sample the features from a panorama image in the spherical domain and propose customized operations to tackle spherical features for panorama depth estimation. We also propose two hyper-parameters for SphereDepth, MR and TR, to balance the efficiency and accuracy;&#10;&#09;\item Superior to those existing methods, SphereDepth achieves comparable accuracy in panorama depth estimation and generates a much higher quality point cloud. SphereDepth suggests a new direction to apply the sphere mesh convolution to the panorama image instead of traditional convolution networks.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Engineering, Mathematics, Artificial Intelligence, Depth Estimation Methods, Geometric Computing, Computer Science, Panorama Image Processing" />
-        </attvalues>
-      </node>
-      <node id="1904.08103" label="1904.08103">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale Geometric Consistency Guided Multi-View Stereo" />
-          <attvalue for="1" value="  In this paper, we propose an efficient multi-scale geometric consistency&#10;guided multi-view stereo method for accurate and complete depth map estimation.&#10;We first present our basic multi-view stereo method with Adaptive Checkerboard&#10;sampling and Multi-Hypothesis joint view selection (ACMH). It leverages&#10;structured region information to sample better candidate hypotheses for&#10;propagation and infer the aggregation view subset at each pixel. For the depth&#10;estimation of low-textured areas, we further propose to combine ACMH with&#10;multi-scale geometric consistency guidance (ACMM) to obtain the reliable depth&#10;estimates for low-textured areas at coarser scales and guarantee that they can&#10;be propagated to finer scales. To correct the erroneous estimates propagated&#10;from the coarser scales, we present a novel detail restorer. Experiments on&#10;extensive datasets show our method achieves state-of-the-art performance,&#10;recovering the depth estimation not only in low-textured areas but also in&#10;details.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.02505" label="1804.02505">
-        <attvalues>
-          <attvalue for="0" value="MVSNet: Depth Inference for Unstructured Multi-view Stereo" />
-          <attvalue for="1" value="  We present an end-to-end deep learning architecture for depth map inference&#10;from multi-view images. In the network, we first extract deep visual image&#10;features, and then build the 3D cost volume upon the reference camera frustum&#10;via the differentiable homography warping. Next, we apply 3D convolutions to&#10;regularize and regress the initial depth map, which is then refined with the&#10;reference image to generate the final output. Our framework flexibly adapts&#10;arbitrary N-view inputs using a variance-based cost metric that maps multiple&#10;features into one cost feature. The proposed MVSNet is demonstrated on the&#10;large-scale indoor DTU dataset. With simple post-processing, our method not&#10;only significantly outperforms previous state-of-the-arts, but also is several&#10;times faster in runtime. We also evaluate MVSNet on the complex outdoor Tanks&#10;and Temples dataset, where our method ranks first before April 18, 2018 without&#10;any fine-tuning, showing the strong generalization ability of MVSNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.09620" label="1807.09620">
-        <attvalues>
-          <attvalue for="0" value="OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas" />
-          <attvalue for="1" value="  Recent work on depth estimation up to now has only focused on projective&#10;images ignoring 360 content which is now increasingly and more easily produced.&#10;We show that monocular depth estimation models trained on traditional images&#10;produce sub-optimal results on omnidirectional images, showcasing the need for&#10;training directly on 360 datasets, which however, are hard to acquire. In this&#10;work, we circumvent the challenges associated with acquiring high quality 360&#10;datasets with ground truth depth annotations, by re-using recently released&#10;large scale 3D datasets and re-purposing them to 360 via rendering. This&#10;dataset, which is considerably larger than similar projective datasets, is&#10;publicly offered to the community to enable future research in this direction.&#10;We use this dataset to learn in an end-to-end fashion the task of depth&#10;estimation from 360 images. We show promising results in our synthesized data&#10;as well as in unseen realistic images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.13107" label="2112.13107">
-        <attvalues>
-          <attvalue for="0" value="Invertible Network for Unpaired Low-light Image Enhancement" />
-          <attvalue for="1" value="  Existing unpaired low-light image enhancement approaches prefer to employ the&#10;two-way GAN framework, in which two CNN generators are deployed for enhancement&#10;and degradation separately. However, such data-driven models ignore the&#10;inherent characteristics of transformation between the low and normal light&#10;images, leading to unstable training and artifacts. Here, we propose to&#10;leverage the invertible network to enhance low-light image in forward process&#10;and degrade the normal-light one inversely with unpaired learning. The&#10;generated and real images are then fed into discriminators for adversarial&#10;learning. In addition to the adversarial loss, we design various loss functions&#10;to ensure the stability of training and preserve more image details.&#10;Particularly, a reversibility loss is introduced to alleviate the over-exposure&#10;problem. Moreover, we present a progressive self-guided enhancement process for&#10;low-light images and achieve favorable performance against the SOTAs.&#10;" />
-          <attvalue for="2" value="&#10;&#09;High quality images are of great importance in various computer vision applications.&#10;&#09;Unfortunately, images captured in low-light environments usually suffer from unsatisfied visibility, such as low contrast.&#10;&#09;Therefore, low-light Image Enhancement (LIE) remains a challenge, especially due to the details in under-exposed regions are usually imperceptible.&#10;&#10;Traditional methods often enhance the low-light images by adjusting its contrast with global operator based on histogram equalization (HE)~\cite{he-10.1109/TCE.2007.4429280, he-article} or gamma correction~\cite{Huang2013EfficientCE}.&#10;&#09;Considering the insufficient illumination in low-light environments, Retinex methods~\cite{NPE, Fu_2016_CVPR, Guo2017LIMELI} propose to extract the illumination and reflection components. Various hand-crafted constraints and priors are designed in optimization models to recover or enhance the illumination of the image.&#10;&#09;However, these classical approaches are limited in model capacity and cannot describe the physical model of low-light image accurately, thus are difficult to deal with challenging inputs and cannot produce satisfactory results.&#10;&#10;In recent years, deep learning based methods have been explored extensively for low-light image enhancement. Prior works mainly rely on the paired data for supervised learning~\cite{Chen2018Retinex, Wang_2019_CVPR, kind++, liu2021ruas}.&#10;&#09;However, the simulated paired data cannot describe the characteristics of the real low-light images well and are prone to result in artifacts or color distortion.&#10;&#09;While the paired training set from real scenes is cost-expensive to acquire and may suffer from misalignment issue.&#10;&#09;Consequently, the generalization ability of the model with paired supervision inevitably degrades and makes it intractable to handle the variety and complexity of the input low-light images.&#10;&#10;As a remedy, numerous efforts have been made on unsupervised learning for low light enhancement.&#10;&#09;Based on unpaired training set, GAN-based methods provide an intuitive solution to avoid the dependence on paired data effectively and achieve promising results in the general image enhancement.&#10;&#09;For example, both DPE~\cite{Chen_2018_CVPR_DPE} and QAGAN~\cite{QAGAN} adopt the two-way GAN~\cite{CycleGAN2017} framework to implement the user-oriented image enhancement with unpaired training set.&#10;&#09;GEN\&amp;LEN~\cite{GleNet} even proposes a two-stage training scheme for unpaired image enhancement with two-way GAN.&#10;&#09;However, above methods are not specifically designed for low-light images, thereby showing limited performance on extremely dark pictures.&#10;&#09;EnlightenGAN~\cite{jiang2021enlightengan} is the first to successfully apply the unpaired learning for low-light image enhancement. It employs an one way GAN structure to stabilize training and achieves considerable results.&#10;&#09;In the meanwhile, Guo et. al.~\cite{Zero-DCE} recently propose to train a lightweight CNN to estimate the image-specific and pixel-wise curves for dynamic range adjustment without the requirement of any paired or unpaired data.&#10;&#09;However, both of the models suffer from severe artifacts, while EnlightenGAN also produce over-exposed results.&#10;&#10;Two-way GAN based unpaired learning method are typical data-driven models relied on the elaborately designed discriminators, loss functions and the selection of unpaired data, ignoring the inherent characteristics of the transformation between the low and normal light images.&#10;&#09;{In practice, illumination variation can produce normal and low light images in a certain scene.}&#10;&#09;Therefore, it is more reasonable to learn a bijective function for modeling the special translation between the low and normal light images, which inspires us to adopt the invertible models.&#10;&#09;Using the unpaired training set, here, we propose an invertible network based framework for low-light image enhancement.&#10;&#09;In the forward process, the low-light image is enhanced to a normal-light photograph and the unpaired normal-light image is degraded to the corresponding low-light sample inversely.&#10;&#09;Then the generated and real images are leveraged to train different discriminators for adversarial learning.&#10;&#09;In the training phase, in addition to the adversarial loss, we design the transformation-consistent loss for stable training, the detail-preserving loss to maintain details of low light image, and the reversibility loss to alleviate the over-exposure problem.&#10;&#09;To prevent checkerboard artifacts caused by squeeze layers, we also present a progressive self-guided enhancement for inference and achieve the state-of-the-art performance.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Image Processing, Deep Learning, Computer Science, Image Enhancement" />
-        </attvalues>
-      </node>
-      <node id="2012.15052" label="2012.15052">
-        <attvalues>
-          <attvalue for="0" value="Unpaired Image Enhancement with Quality-Attention Generative Adversarial&#10;  Network" />
-          <attvalue for="1" value="  In this work, we aim to learn an unpaired image enhancement model, which can&#10;enrich low-quality images with the characteristics of high-quality images&#10;provided by users. We propose a quality attention generative adversarial&#10;network (QAGAN) trained on unpaired data based on the bidirectional Generative&#10;Adversarial Network (GAN) embedded with a quality attention module (QAM). The&#10;key novelty of the proposed QAGAN lies in the injected QAM for the generator&#10;such that it learns domain-relevant quality attention directly from the two&#10;domains. More specifically, the proposed QAM allows the generator to&#10;effectively select semantic-related characteristics from the spatial-wise and&#10;adaptively incorporate style-related attributes from the channel-wise,&#10;respectively. Therefore, in our proposed QAGAN, not only discriminators but&#10;also the generator can directly access both domains which significantly&#10;facilitates the generator to learn the mapping function. Extensive experimental&#10;results show that, compared with the state-of-the-art methods based on unpaired&#10;learning, our proposed method achieves better performance in both objective and&#10;subjective evaluations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.06972" label="1906.06972">
-        <attvalues>
-          <attvalue for="0" value="EnlightenGAN: Deep Light Enhancement without Paired Supervision" />
-          <attvalue for="1" value="  Deep learning-based methods have achieved remarkable success in image&#10;restoration and enhancement, but are they still competitive when there is a&#10;lack of paired training data? As one such example, this paper explores the&#10;low-light image enhancement problem, where in practice it is extremely&#10;challenging to simultaneously take a low-light and a normal-light photo of the&#10;same visual scene. We propose a highly effective unsupervised generative&#10;adversarial network, dubbed EnlightenGAN, that can be trained without&#10;low/normal-light image pairs, yet proves to generalize very well on various&#10;real-world test images. Instead of supervising the learning using ground truth&#10;data, we propose to regularize the unpaired training using the information&#10;extracted from the input itself, and benchmark a series of innovations for the&#10;low-light image enhancement problem, including a global-local discriminator&#10;structure, a self-regularized perceptual loss fusion, and attention mechanism.&#10;Through extensive experiments, our proposed approach outperforms recent methods&#10;under a variety of metrics in terms of visual quality and subjective user&#10;study. Thanks to the great flexibility brought by unpaired training,&#10;EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world&#10;images from various domains. The code is available at&#10;\url{https://github.com/yueruchen/EnlightenGAN}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.06826" label="2001.06826">
-        <attvalues>
-          <attvalue for="0" value="Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement" />
-          <attvalue for="1" value="  The paper presents a novel method, Zero-Reference Deep Curve Estimation&#10;(Zero-DCE), which formulates light enhancement as a task of image-specific&#10;curve estimation with a deep network. Our method trains a lightweight deep&#10;network, DCE-Net, to estimate pixel-wise and high-order curves for dynamic&#10;range adjustment of a given image. The curve estimation is specially designed,&#10;considering pixel value range, monotonicity, and differentiability. Zero-DCE is&#10;appealing in its relaxed assumption on reference images, i.e., it does not&#10;require any paired or unpaired data during training. This is achieved through a&#10;set of carefully formulated non-reference loss functions, which implicitly&#10;measure the enhancement quality and drive the learning of the network. Our&#10;method is efficient as image enhancement can be achieved by an intuitive and&#10;simple nonlinear curve mapping. Despite its simplicity, we show that it&#10;generalizes well to diverse lighting conditions. Extensive experiments on&#10;various benchmarks demonstrate the advantages of our method over&#10;state-of-the-art methods qualitatively and quantitatively. Furthermore, the&#10;potential benefits of our Zero-DCE to face detection in the dark are discussed.&#10;Code and model will be available at https://github.com/Li-Chongyi/Zero-DCE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.04159" label="2010.04159">
-        <attvalues>
-          <attvalue for="0" value="Deformable DETR: Deformable Transformers for End-to-End Object Detection" />
-          <attvalue for="1" value="  DETR has been recently proposed to eliminate the need for many hand-designed&#10;components in object detection while demonstrating good performance. However,&#10;it suffers from slow convergence and limited feature spatial resolution, due to&#10;the limitation of Transformer attention modules in processing image feature&#10;maps. To mitigate these issues, we proposed Deformable DETR, whose attention&#10;modules only attend to a small set of key sampling points around a reference.&#10;Deformable DETR can achieve better performance than DETR (especially on small&#10;objects) with 10 times less training epochs. Extensive experiments on the COCO&#10;benchmark demonstrate the effectiveness of our approach. Code is released at&#10;https://github.com/fundamentalvision/Deformable-DETR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Efficient Attention Mechanism.~&#10;Transformers~\cite{vaswani2017attention} involve both self-attention and cross-attention mechanisms. One of the most well-known concern of Transformers is the high time and memory complexity at vast key element numbers, which hinders model scalability in many cases. Recently, many efforts have been made to address this problem~\cite{tay2020efficient}, which can be roughly divided into three categories in practice.&#10;&#10;The first category is to use pre-defined sparse attention patterns on keys.&#10;The most straightforward paradigm is restricting the attention pattern to be fixed local windows.&#10;Most works~\cite{liu2018generating,parmar2018image,child2019generating,huang2019ccnet, ho2019axial,wang2020axial,hu2019local,ramachandran2019stand,qiu2019blockwise,beltagy2020longformer,ainslie2020etc,zaheer2020big} follow this paradigm.&#10;Although restricting the attention pattern to a local neighborhood can decrease the complexity, it loses global information.&#10;To compensate, \cite{child2019generating, huang2019ccnet, ho2019axial,wang2020axial} attend key elements at fixed intervals to significantly increase the receptive field on keys.&#10;\cite{beltagy2020longformer,ainslie2020etc,zaheer2020big} allow a small number of special tokens having access to all key elements. \cite{zaheer2020big,qiu2019blockwise} also add some pre-fixed sparse attention patterns to attend distant key elements directly.&#10;&#10;The second category is to learn data-dependent sparse attention. \cite{kitaev2020reformer} proposes a locality sensitive hashing (LSH) based attention, which hashes both the query and key elements to different bins. A similar idea is proposed by \cite{roy2020efficient}, where k-means finds out the most related keys. \cite{tay2020sparse} learns block permutation for block-wise sparse attention. &#10;&#10;The third category is to explore the low-rank property in self-attention. \cite{wang2020linformer} reduces the number of key elements through a linear projection on the size dimension instead of the channel dimension. \cite{katharopoulos2020transformers,choromanski2020masked} rewrite the calculation of self-attention through kernelization approximation.&#10;&#10;In the image domain, the designs of efficient attention mechanism (e.g., \cite{parmar2018image,child2019generating,huang2019ccnet,ho2019axial,wang2020axial,hu2019local,ramachandran2019stand}) are still limited to the first category. Despite the theoretically reduced complexity, \cite{ramachandran2019stand,hu2019local} admit such approaches are much slower in implementation than traditional convolution with the same FLOPs (at least 3$\times$ slower), due to the intrinsic limitation in memory access patterns.&#10;&#10;On the other hand, as discussed in \cite{zhu2019empirical}, there are variants of&#10;convolution, such as deformable convolution~\cite{dai2017deformable,zhu2019deformable} and dynamic convolution~\cite{wu2019pay}, that also can be viewed as self-attention mechanisms. Especially, deformable convolution operates much more effectively&#10;and efficiently on image recognition than Transformer self-attention. Meanwhile, it lacks the element relation modeling mechanism.&#10;&#10;Our proposed deformable attention module is inspired by deformable convolution, and belongs to the second category. It only focuses on a small fixed set of sampling points predicted from the feature of query elements.&#10;Different from \cite{ramachandran2019stand,hu2019local}, deformable attention is just slightly slower than the traditional convolution under the same FLOPs.&#10;&#10;Multi-scale Feature Representation for Object Detection.~&#10;One of the main difficulties in object detection is to effectively represent objects at vastly different scales. Modern object detectors usually exploit multi-scale features to accommodate this.&#10;As one of the pioneering works, FPN~\cite{lin2017feature} proposes a top-down path to combine multi-scale features. PANet~\cite{liu2018path} further adds an bottom-up path on the top of FPN. \cite{kong2018deep} combines features from all scales by a global attention operation. \cite{zhao2019m2det} proposes a U-shape module to fuse multi-scale features. Recently, NAS-FPN~\cite{ghiasi2019fpn} and Auto-FPN~\cite{xu2019auto} are proposed to automatically design cross-scale connections via neural architecture search. \cite{tan2020efficientdet} proposes the BiFPN, which is a repeated simplified version of PANet.&#10;Our proposed multi-scale deformable attention module can naturally aggregate multi-scale feature maps via attention mechanism, without the help of these feature pyramid networks.&#10;&#10;" />
-          <attvalue for="4" value="Transformer Attention, Computer Vision, Engineering, Machine Learning, Mathematics, Image Feature Processing, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.12087" label="2006.12087">
-        <attvalues>
-          <attvalue for="0" value="Progressive Graph Learning for Open-Set Domain Adaptation" />
-          <attvalue for="1" value="  Domain shift is a fundamental problem in visual recognition which typically&#10;arises when the source and target data follow different distributions. The&#10;existing domain adaptation approaches which tackle this problem work in the&#10;closed-set setting with the assumption that the source and the target data&#10;share exactly the same classes of objects. In this paper, we tackle a more&#10;realistic problem of open-set domain shift where the target data contains&#10;additional classes that are not present in the source data. More specifically,&#10;we introduce an end-to-end Progressive Graph Learning (PGL) framework where a&#10;graph neural network with episodic training is integrated to suppress&#10;underlying conditional shift and adversarial learning is adopted to close the&#10;gap between the source and target distributions. Compared to the existing&#10;open-set adaptation approaches, our approach guarantees to achieve a tighter&#10;upper bound of the target error. Extensive experiments on three standard&#10;open-set benchmarks evidence that our approach significantly outperforms the&#10;state-of-the-arts in open-set domain adaptation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;While deep learning has made remarkable advances across a wide variety of machine-learning tasks and applications, it is commonly assumed that the training and test data are drawn from the same distribution. In practice, however, this assumption can be violated due to a number of factors, such as the change of lighting conditions, background, environment, or data modalities, which is referred to as the domain shift problem. &#10;&#10;Unsupervised Domain Adaptation (UDA) approaches tackle the domain shift problem by aligning the training (source) and test (target) distributions, and can be roughly divided into statistical matching ~\cite{dip, manifold, distribution, DDC, JDA, DAN, DANN,ADDA,jingjing} or adversarial learning~\cite{DANN,ADDA,JAN,DRCN} methods. Theoretical analysis of UDA approaches has been widely studied~\cite{david1, discrepancy, bridgingtheory}, which provides rigorous error bounds on the target data.&#10;&#10;Existing UDA algorithms are developed under the assumption that the source and target domains share an identical group of classes. Such a scenario typically refers to a closed-set setting, which could be hardly guaranteed in real-world applications. Therefore, a more realistic Open-set adaptation setting has been introduced recently \cite{OSBP} which allows the target data to contain an additional ``unknown'' category, covering all irrelevant classes not present in the source domain. &#10;&#10;The core idea of unsupervised open-set domain adaptation (OUDA) approaches~\cite{ATI, mahsa,OSBP, STA, Attract-UTS} is to learn a classifier from a larger hypothesis space for both shared and unknown classes in the source and target domains. According to~\cite{david1, discrepancy}, the target error is bounded by the source risk, discrepancy distance across the domains, the shared error coming from the conditional shift~\cite{conditionalshift}, and the open-set risk. Open-set risk contributes the most to the error bound when a large percentage of data is unknown.&#10;&#10;While promising, existing OUDA approaches~\cite{ATI,OSBP,mahsa,STA,Attract-UTS} lack an essential theoretical analysis of the aforementioned partial risks and the upper bound for the target risk, thus omitting potential solutions for improvement and leading to a biased solution. With the aim of minimising the aforementioned partial risks and achieving a tighter error bound for open-set adaptation, we combine the following four strategies in an end-to-end progressive learning framework:&#10;&#10;\begin{enumerate}&#10; \item To suppress the source risk, we decompose the original hypothesis space $\mathcal{H}$ into two subspaces $\mathcal{H}_1$ and $\mathcal{H}_2$, where $\mathcal{H}_1$ includes classifiers for the shared classes of the source and target domains and $\mathcal{H}_2$ is specific to classifying unknowns in the target domain. With a restricted size of the subspace $\mathcal{H}_1$, the possibility of misclassifying source data as unknowns will be reduced.&#10;&#10;\item To control the open-set risk, we adopt the progressive learning paradigm~\cite{curriculum}, where the target samples with low classification confidence are gradually rejected from the target domain and inserted as the pseudo-labeled unknown set in the source domain. This mechanism suppresses the potential negative transfer where the private representations across domains are falsely aligned. &#10;&#10;\item We address conditional shift~\cite{conditionalshift} at the both sample- and manifold-level in a transductive setting. Specifically, we design an episodic training scheme and align conditional distributions across domains by gradually replacing the source data with the pseudo-labeled known data in each episode. We learn class-specific representations by aggregating the source and target features and passing episodes through deep graph neural networks.&#10;&#10;\item Our algorithm is seamlessly equipped with an adversarial domain discriminator, which effectively closes the gap between the source and target marginal distributions for the known categories.&#10;&#10;\end{enumerate}&#10;&#10;We applied our method on three challenging open-set object recognition benchmarks, i.e., the Office-Home, VisDA-17, and Syn2Real-O, to confirm its superiority to the existing state-of-the-art open-set domain adaptation approaches.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Statistics, Visual Recognition, Graph Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="1505.07818" label="1505.07818">
-        <attvalues>
-          <attvalue for="0" value="Domain-Adversarial Training of Neural Networks" />
-          <attvalue for="1" value="  We introduce a new representation learning approach for domain adaptation, in&#10;which data at training and test time come from similar but different&#10;distributions. Our approach is directly inspired by the theory on domain&#10;adaptation suggesting that, for effective domain transfer to be achieved,&#10;predictions must be made based on features that cannot discriminate between the&#10;training (source) and test (target) domains. The approach implements this idea&#10;in the context of neural network architectures that are trained on labeled data&#10;from the source domain and unlabeled data from the target domain (no labeled&#10;target-domain data is necessary). As the training progresses, the approach&#10;promotes the emergence of features that are (i) discriminative for the main&#10;learning task on the source domain and (ii) indiscriminate with respect to the&#10;shift between the domains. We show that this adaptation behaviour can be&#10;achieved in almost any feed-forward model by augmenting it with few standard&#10;layers and a new gradient reversal layer. The resulting augmented architecture&#10;can be trained using standard backpropagation and stochastic gradient descent,&#10;and can thus be implemented with little effort using any of the deep learning&#10;packages. We demonstrate the success of our approach for two distinct&#10;classification problems (document sentiment analysis and image classification),&#10;where state-of-the-art domain adaptation performance on standard benchmarks is&#10;achieved. We also validate the approach for descriptor learning task in the&#10;context of person re-identification application.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.06636" label="1605.06636">
-        <attvalues>
-          <attvalue for="0" value="Deep Transfer Learning with Joint Adaptation Networks" />
-          <attvalue for="1" value="  Deep networks have been successfully applied to learn transferable features&#10;for adapting models from a source domain to a different target domain. In this&#10;paper, we present joint adaptation networks (JAN), which learn a transfer&#10;network by aligning the joint distributions of multiple domain-specific layers&#10;across domains based on a joint maximum mean discrepancy (JMMD) criterion.&#10;Adversarial training strategy is adopted to maximize JMMD such that the&#10;distributions of the source and target domains are made more distinguishable.&#10;Learning can be performed by stochastic gradient descent with the gradients&#10;computed by back-propagation in linear-time. Experiments testify that our model&#10;yields state of the art results on standard datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1607.03516" label="1607.03516">
-        <attvalues>
-          <attvalue for="0" value="Deep Reconstruction-Classification Networks for Unsupervised Domain&#10;  Adaptation" />
-          <attvalue for="1" value="  In this paper, we propose a novel unsupervised domain adaptation algorithm&#10;based on deep learning for visual object recognition. Specifically, we design a&#10;new model called Deep Reconstruction-Classification Network (DRCN), which&#10;jointly learns a shared encoding representation for two tasks: i) supervised&#10;classification of labeled source data, and ii) unsupervised reconstruction of&#10;unlabeled target data.In this way, the learnt representation not only preserves&#10;discriminability, but also encodes useful information from the target domain.&#10;Our new DRCN model can be optimized by using backpropagation similarly as the&#10;standard neural networks.&#10;  We evaluate the performance of DRCN on a series of cross-domain object&#10;recognition tasks, where DRCN provides a considerable improvement (up to ~8% in&#10;accuracy) over the prior state-of-the-art algorithms. Interestingly, we also&#10;observe that the reconstruction pipeline of DRCN transforms images from the&#10;source domain into images whose appearance resembles the target dataset. This&#10;suggests that DRCN's performance is due to constructing a single composite&#10;representation that encodes information about both the structure of target&#10;images and the classification of source images. Finally, we provide a formal&#10;analysis to justify the algorithm's objective in domain adaptation context.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.10427" label="1804.10427">
-        <attvalues>
-          <attvalue for="0" value="Open Set Domain Adaptation by Backpropagation" />
-          <attvalue for="1" value="  Numerous algorithms have been proposed for transferring knowledge from a&#10;label-rich domain (source) to a label-scarce domain (target). Almost all of&#10;them are proposed for a closed-set scenario, where the source and the target&#10;domain completely share the class of their samples. We call the shared class&#10;the \doublequote{known class.} However, in practice, when samples in target&#10;domain are not labeled, we cannot know whether the domains share the class. A&#10;target domain can contain samples of classes that are not shared by the source&#10;domain. We call such classes the \doublequote{unknown class} and algorithms&#10;that work well in the open set situation are very practical. However, most&#10;existing distribution matching methods for domain adaptation do not work well&#10;in this setting because unknown target samples should not be aligned with the&#10;source.&#10;  In this paper, we propose a method for an open set domain adaptation scenario&#10;which utilizes adversarial training. A classifier is trained to make a boundary&#10;between the source and the target samples whereas a generator is trained to&#10;make target samples far from the boundary. Thus, we assign two options to the&#10;feature generator: aligning them with source known samples or rejecting them as&#10;unknown target samples. This approach allows extracting features that separate&#10;unknown target samples from known target samples. Our method was extensively&#10;evaluated in domain adaptation setting and outperformed other methods with a&#10;large margin in most settings.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.12277" label="1805.12277">
-        <attvalues>
-          <attvalue for="0" value="Learning Factorized Representations for Open-set Domain Adaptation" />
-          <attvalue for="1" value="  Domain adaptation for visual recognition has undergone great progress in the&#10;past few years. Nevertheless, most existing methods work in the so-called&#10;closed-set scenario, assuming that the classes depicted by the target images&#10;are exactly the same as those of the source domain. In this paper, we tackle&#10;the more challenging, yet more realistic case of open-set domain adaptation,&#10;where new, unknown classes can be present in the target data. While, in the&#10;unsupervised scenario, one cannot expect to be able to identify each specific&#10;new class, we aim to automatically detect which samples belong to these new&#10;classes and discard them from the recognition process. To this end, we rely on&#10;the intuition that the source and target samples depicting the known classes&#10;can be generated by a shared subspace, whereas the target samples from unknown&#10;classes come from a different, private subspace. We therefore introduce a&#10;framework that factorizes the data into shared and private parts, while&#10;encouraging the shared representation to be discriminative. Our experiments on&#10;standard benchmarks evidence that our approach significantly outperforms the&#10;state-of-the-art in open-set domain adaptation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.01925" label="1908.01925">
-        <attvalues>
-          <attvalue for="0" value="Attract or Distract: Exploit the Margin of Open Set" />
-          <attvalue for="1" value="  Open set domain adaptation aims to diminish the domain shift across domains,&#10;with partially shared classes. There exist unknown target samples out of the&#10;knowledge of source domain. Compared to the close set setting, how to separate&#10;the unknown (unshared) class from the known (shared) ones plays a key role.&#10;Whereas, previous methods did not emphasize the semantic structure of the open&#10;set data, which may introduce bias into the domain alignment and confuse the&#10;classifier around the decision boundary. In this paper, we exploit the semantic&#10;structure of open set data from two aspects: 1) Semantic Categorical Alignment,&#10;which aims to achieve good separability of target known classes by&#10;categorically aligning the centroid of target with the source. 2)Semantic&#10;Contrastive Mapping, which aims to push the unknown class away from the&#10;decision boundary. Empirically, we demonstrate that our method performs&#10;favourably against the state-of-the-art methods on representative benchmarks,&#10;e.g. Digit datasets and Office-31 datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.07867" label="2207.07867">
-        <attvalues>
-          <attvalue for="0" value="Automatic dataset generation for specific object detection" />
-          <attvalue for="1" value="  In the past decade, object detection tasks are defined mostly by large public&#10;datasets. However, building object detection datasets is not scalable due to&#10;inefficient image collecting and labeling. Furthermore, most labels are still&#10;in the form of bounding boxes, which provide much less information than the&#10;real human visual system. In this paper, we present a method to synthesize&#10;object-in-scene images, which can preserve the objects' detailed features&#10;without bringing irrelevant information. In brief, given a set of images&#10;containing a target object, our algorithm first trains a model to find an&#10;approximate center of the object as an anchor, then makes an outline regression&#10;to estimate its boundary, and finally blends the object into a new scene. Our&#10;result shows that in the synthesized image, the boundaries of objects blend&#10;very well with the background. Experiments also show that SOTA segmentation&#10;models work well with our synthesized data.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object detection and recognition is a popular research field and have been widely used in industry and in daily life.&#10;Because of the breakthrough and rapid adoption of deep learning, many highly accurate object detection algorithms and methods have been proposed. However, one of the biggest drawbacks of using the SOTA object detection system is a large amount of labeled data because the cost of labeling the dataset manually is high. Public datasets, such as PASCAL VOC\cite{Everingham}, MS COCO\cite{lin}, ImageNet\cite{deng} can be used to train a general object detector. However, for a specific object, public datasets may bring the following problems: (1) Public datasets may not contain specific objects. Meanwhile, collecting and labeling images with various circumstances is expensive. (2) Public datasets always assume certain combination between the object and the background. For example, refrigerator and kitchen. This correlation causes the network to extract some features that are not related to the object itself. Such features may increase performance in the competitions on public dataset but will affect the accuracy of object detection when the object migrates to an unknown environment.&#10;&#10;One popular way for overcoming these problems is rendering scenes and objects using 3D models \cite{tremblay}\cite{prakash}. Another method is generating a synthetic dataset by placing real segmented object images onto background images. Sagues et al. propose a specialized method to generate a synthetic labeled dataset for kitchen object \cite{sagues}. Georgakis et al. superimpose 2D images of textured object models into images of real indoor environments \cite{georgakis}.&#10;Dwibedi et al. present a method to make detectors ignore the artifacts during training \cite{dwibedi}. \cite{yun} introduces an approach to diversify the foreground with GAN-based seed images in order to balance domain gaps. &#10;Compared with the existing public dataset, the advantages of synthetic dataset are as follows: (1) The number of synthetic data can be much larger than human labeled data. (2) The objects and backgrounds are not limited to some certain combination. Thus, the model can learn more intrinsic object features. (3) The synthetic dataset can be labeled in an arbitrary way. Contours, masks, bounding boxes and any other forms of annotations related to a practical project are possible.&#10;&#10;In this paper, we propose a method to generate synthetic dataset. We first get images of different views of the object and apply a convolutional neural network to find the center point of the specific object in the image. Then we train another network to obtain the distance of the object contour relative to the center point in different directions.&#10;After that, a rough contour of the object is drawn based on these distances. The method in \cite{levin} and \cite{perez} can help us segment the precise contour of the object and blend it into the background. Fig. \ref{fig:overview} briefly shows the whole pipeline of our method.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Cognitive Science, Computer Science, Image Synthesis" />
-        </attvalues>
-      </node>
-      <node id="1804.06516" label="1804.06516">
-        <attvalues>
-          <attvalue for="0" value="Training Deep Networks with Synthetic Data: Bridging the Reality Gap by&#10;  Domain Randomization" />
-          <attvalue for="1" value="  We present a system for training deep neural networks for object detection&#10;using synthetic images. To handle the variability in real-world data, the&#10;system relies upon the technique of domain randomization, in which the&#10;parameters of the simulator$-$such as lighting, pose, object textures,&#10;etc.$-$are randomized in non-realistic ways to force the neural network to&#10;learn the essential features of the object of interest. We explore the&#10;importance of these parameters, showing that it is possible to produce a&#10;network with compelling performance using only non-artistically-generated&#10;synthetic data. With additional fine-tuning on real data, the network yields&#10;better performance than using real data alone. This result opens up the&#10;possibility of using inexpensive synthetic data for training neural networks&#10;while avoiding the need to collect large amounts of hand-annotated real-world&#10;data or to generate high-fidelity synthetic worlds$-$both of which remain&#10;bottlenecks for many applications. The approach is evaluated on bounding box&#10;detection of cars on the KITTI dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.10093" label="1810.10093">
-        <attvalues>
-          <attvalue for="0" value="Structured Domain Randomization: Bridging the Reality Gap by&#10;  Context-Aware Synthetic Data" />
-          <attvalue for="1" value="  We present structured domain randomization (SDR), a variant of domain&#10;randomization (DR) that takes into account the structure and context of the&#10;scene. In contrast to DR, which places objects and distractors randomly&#10;according to a uniform probability distribution, SDR places objects and&#10;distractors randomly according to probability distributions that arise from the&#10;specific problem at hand. In this manner, SDR-generated imagery enables the&#10;neural network to take the context around an object into consideration during&#10;detection. We demonstrate the power of SDR for the problem of 2D bounding box&#10;car detection, achieving competitive results on real data after training only&#10;on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that&#10;SDR outperforms other approaches to generating synthetic data (VKITTI, Sim&#10;200k, or DR), as well as real data collected in a different domain (BDD100K).&#10;Moreover, synthetic SDR data combined with real KITTI data outperforms real&#10;KITTI data alone.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1702.07836" label="1702.07836">
-        <attvalues>
-          <attvalue for="0" value="Synthesizing Training Data for Object Detection in Indoor Scenes" />
-          <attvalue for="1" value="  Detection of objects in cluttered indoor environments is one of the key&#10;enabling functionalities for service robots. The best performing object&#10;detection approaches in computer vision exploit deep Convolutional Neural&#10;Networks (CNN) to simultaneously detect and categorize the objects of interest&#10;in cluttered scenes. Training of such models typically requires large amounts&#10;of annotated training data which is time consuming and costly to obtain. In&#10;this work we explore the ability of using synthetically generated composite&#10;images for training state-of-the-art object detectors, especially for object&#10;instance detection. We superimpose 2D images of textured object models into&#10;images of real environments at variety of locations and scales. Our experiments&#10;evaluate different superimposition strategies ranging from purely image-based&#10;blending all the way to depth and semantics informed positioning of the object&#10;models into real scenes. We demonstrate the effectiveness of these object&#10;detector training strategies on two publicly available datasets, the&#10;GMU-Kitchens and the Washington RGB-D Scenes v2. As one observation, augmenting&#10;some hand-labeled training data with synthetic examples carefully composed onto&#10;scenes yields object detectors with comparable performance to using much more&#10;hand-labeled data. Broadly, this work charts new opportunities for training&#10;detectors for new objects by exploiting existing object model repositories in&#10;either a purely automatic fashion or with only a very small number of&#10;human-annotated examples.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.01642" label="1708.01642">
-        <attvalues>
-          <attvalue for="0" value="Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection" />
-          <attvalue for="1" value="  A major impediment in rapidly deploying object detection models for instance&#10;detection is the lack of large annotated datasets. For example, finding a large&#10;labeled dataset containing instances in a particular kitchen is unlikely. Each&#10;new environment with new instances requires expensive data collection and&#10;annotation. In this paper, we propose a simple approach to generate large&#10;annotated instance datasets with minimal effort. Our key insight is that&#10;ensuring only patch-level realism provides enough training signal for current&#10;object detector models. We automatically `cut' object instances and `paste'&#10;them on random backgrounds. A naive way to do this results in pixel artifacts&#10;which result in poor performance for trained models. We show how to make&#10;detectors ignore these artifacts during training and generate data that gives&#10;competitive performance on real data. Our method outperforms existing synthesis&#10;approaches and when combined with real images improves relative performance by&#10;more than 21% on benchmark datasets. In a cross-domain setting, our synthetic&#10;data combined with just 10% real data outperforms models trained on all real&#10;data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.11972" label="1909.11972">
-        <attvalues>
-          <attvalue for="0" value="Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object&#10;  Instance Detection" />
-          <attvalue for="1" value="  Training an object instance detector where only a few training object images&#10;are available is a challenging task. One solution is a cut-and-paste method&#10;that generates a training dataset by cutting object areas out of training&#10;images and pasting them onto other background images. A detector trained on a&#10;dataset generated with a cut-and-paste method suffers from the conventional&#10;domain shift problem, which stems from a discrepancy between the source domain&#10;(generated training dataset) and the target domain (real test dataset). Though&#10;state-of-the-art domain adaptation methods are able to reduce this gap, it is&#10;limited because they do not consider the difference of domain gaps of&#10;foreground and background. In this study, we present that the conventional&#10;domain gap can be divided into two sub-domain gaps for foreground and&#10;background. Then, we show that the original cut-and-paste approach suffers from&#10;a new domain gap problem, an unbalanced domain gaps, because it has two&#10;separate source domains for foreground and background, unlike the conventional&#10;domain shift problem. Then, we introduce an advanced cut-and-paste method to&#10;balance the unbalanced domain gaps by diversifying the foreground with GAN&#10;(generative adversarial network)-generated seed images and simplifying the&#10;background using image processing techniques. Experimental results show that&#10;our method is effective for balancing domain gaps and improving the accuracy of&#10;object instance detection in a cluttered indoor environment using only a few&#10;seed images. Furthermore, we show that balancing domain gaps can improve the&#10;detection accuracy of state-of-the-art domain adaptation methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.02291" label="1811.02291">
-        <attvalues>
-          <attvalue for="0" value="MDLatLRR: A novel decomposition method for infrared and visible image&#10;  fusion" />
-          <attvalue for="1" value="  Image decomposition is crucial for many image processing tasks, as it allows&#10;to extract salient features from source images. A good image decomposition&#10;method could lead to a better performance, especially in image fusion tasks. We&#10;propose a multi-level image decomposition method based on latent low-rank&#10;representation(LatLRR), which is called MDLatLRR. This decomposition method is&#10;applicable to many image processing fields. In this paper, we focus on the&#10;image fusion task. We develop a novel image fusion framework based on MDLatLRR,&#10;which is used to decompose source images into detail parts(salient features)&#10;and base parts. A nuclear-norm based fusion strategy is used to fuse the detail&#10;parts, and the base parts are fused by an averaging strategy. Compared with&#10;other state-of-the-art fusion methods, the proposed algorithm exhibits better&#10;fusion performance in both subjective and objective evaluation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;I{n} the field of multi-sensor image fusion, a common task is to combine infrared and visible images. It arises in many applications, such as surveillance \cite{shrinidhi2018ir}, object detection and target recognition \cite{li2019rgb} \cite{lan2018modality}\cite{jiang2019hierarchical}. The key assumption is that different modalities convey complementary information. The main purpose of image fusion is to generate a single image, which combines the complementary sources of information from multiple images of the same scene \cite{li2017pixel}. The key problem is to extract the salient image content from both modalities and then fuse it to produce enhanced output. Many fusion methods have been proposed for this purpose.&#10;The methods most commonly used for image fusion are multi-scale transforms \cite{chen2020infrared}\cite{vishwakarma2018image} and representation learning based methods \cite{li2018densefuse}\cite{borsoi2019super}. &#10;&#10;In the multi-scale transform domain, there are many fusion tools, such as the discrete wavelet transform(DWT) \cite{ben2005multiscale}, contourlet transform \cite{yang2010image}, shift-invariant shearlet transform \cite{wang2014eggdd}, quaternion wavelet transform \cite{pang2012multifocus} and nonsubsampled shearlet transform \cite{luo2017image}. These decomposition methods project source images to the frequency domain, which increases the computational complexity. Hence, researchers try to find other methods to process source images without a transform, such as representation learning based methods.&#10;&#10;In the representation learning domain, the most common methods are based on sparse representation(SR)\cite{wright2008robust} and dictionary learning\cite{chen2019sparse}\cite{chen2019noise}. For instance, Zong et al. \cite{zong2017medical} proposed a novel medical image fusion method based on SR and Histogram of Oriented Gradients(HOG). There are also many algorithms based on combining SR and other tools including pulse coupled neural network(PCNN) \cite{lu2014infrared}, low-rank representation(LRR) \cite{li2017multi} and shearlet transform \cite{yin2017novel}. Other options include the joint sparse representation \cite{liu2017infrared} and cosparse representation \cite{gao2017image}.&#10;&#10;Although SR-based fusion methods achieve good fusion performance, these methods are complex, and dictionary learning is a very time-consuming operation, especially online learning. These problems have motivated a growing interest in deep learning to replace the dictionary learning phase in SR.&#10;&#10;In deep learning-based fusion methods, deep features of the source images are used to generate the fused image. In \cite{liu2016image}, Liu et al. proposed a fusion method based on convolutional sparse representation(CSR), in which multi-scale and multi-layer features are used to construct the fused output. A follow up work was presented in Liu et al. \cite{liu2017multi}. Image patches which contain different blur versions are used to train a network to produce a decision map. The fused image is obtained by using the decision map and source images. In ICCV 2017, Prabhakar et.al. \cite{prabhakar2017deepfuse} proposed a novel CNN-based fusion framework for a multi-exposure image fusion task.&#10;&#10;However, these deep learning-based methods still have drawbacks. The network is difficult to train when the training data is insufficient, especially in infrared and visible image fusion tasks. CNN-based methods tend to work for specific image fusion tasks. To overcome this drawback, Li et.al. \cite{li2018infrared}\cite{li2019infrared} proposed two novel fusion frameworks based on a pretrained network(VGG-19 \cite{simonyan2014very} and ResNet-50 \cite{he2016deep}). The pretrained network is used as a feature extraction tool. They compared this approach with a strategy involving training a novel network architecture (DenseFuse) \cite{li2018densefuse}. Ma et.al. \cite{ma2019fusiongan} \cite{ma2020infrared}, on the other hand, applied Generative Adversarial Networks to the image fusion task, which also achieves impressive fusion performance. A simple, yet effictive end-to-end fusion framework was proposed by Zhang et. al \cite{zhang2020ifcnn}. Interestingly, in these deep learning based methods, a very little attention is paid to image decomposition.&#10;&#10;The latent low rank representation (LatLRR)\cite{liu2011latent} is commonly used in clustering analysis tasks. The authors \cite{liu2011latent} also mentioned that LatLRR can be used to extract salient features from the input data. In this paper we follow this suggestion and propose a novel multi-level decomposition method(MDLatLRR) as a basis of a fusion framework for infrared and visible image fusion.&#10;&#10;In our fusion framework, MDLatLRR is used to decompose source images, so as to extract salient detail parts and the base parts. The detail parts are reconstructed by an adaptive fusion strategy and by a reshape operator. The base parts are fused by averaging strategy. Finally, the fused image is constructed by combining the detail parts and the base part. Compared with the state-of-the-art fusion methods, our fusion framework achieves better fusion performance in both subjective and objective evaluations.&#10;&#10;The main contributions are summarized as follows,&#10;&#10;(1) A LatLRR based multi-level decomposition method (MDLatLRR) is proposed for the image fusion task. In contrast to another LRR-based fusion method~\cite{li2017multi}, which calculates the low-rank coefficients for each image, in our method, the LatLRR problem is only solved once in the training phase to learn a projection matrix $L$. As the size of $L$ is only related to the image patch size, the fusion method can process the source images of an arbitrary size.&#10;&#10;(2) With the multi-level decomposition operation, the proposed fusion method captures more detail information from the source images. As the base part contains preponderance of contour features, it makes the weight-average strategy more effective.&#10;&#10;(3) Contrasting it with the $l_1$-norm and $l_2$-norm, the nuclear-norm(the fusion strategy for detail parts) calculates the sum of singular values, which preserves 2D information from the source images. It will generate a larger weight when an input patch contains more texture and structural information. This behaviour is desirable in image processing tasks.&#10;&#10;This paper is structured as follows. In Section \ref{pre}, we introduce preliminaries. In Section \ref{mdlatlrr}, we present our multi-level decomposition method(MDLatLRR) in detail. In Section \ref{fusion-method}, the proposed fusion framework based on MDLatLRR is introducted. The experimental results are presented in Section \ref{experiment}. Finally, the conclusions are drawn in Section \ref{conclusions}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Image Decomposition, Mathematics, Computer Science, Image Fusion, Latent Representation, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1808.10072" label="1808.10072">
-        <attvalues>
-          <attvalue for="0" value="Super-Resolution for Hyperspectral and Multispectral Image Fusion&#10;  Accounting for Seasonal Spectral Variability" />
-          <attvalue for="1" value="  Image fusion combines data from different heterogeneous sources to obtain&#10;more precise information about an underlying scene. Hyperspectral-multispectral&#10;(HS-MS) image fusion is currently attracting great interest in remote sensing&#10;since it allows the generation of high spatial resolution HS images,&#10;circumventing the main limitation of this imaging modality. Existing HS-MS&#10;fusion algorithms, however, neglect the spectral variability often existing&#10;between images acquired at different time instants. This time difference causes&#10;variations in spectral signatures of the underlying constituent materials due&#10;to different acquisition and seasonal conditions. This paper introduces a novel&#10;HS-MS image fusion strategy that combines an unmixing-based formulation with an&#10;explicit parametric model for typical spectral variability between the two&#10;images. Simulations with synthetic and real data show that the proposed&#10;strategy leads to a significant performance improvement under spectral&#10;variability and state-of-the-art performance otherwise.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.08355" label="1804.08355">
-        <attvalues>
-          <attvalue for="0" value="Multi-focus Image Fusion using dictionary learning and Low-Rank&#10;  Representation" />
-          <attvalue for="1" value="  Among the representation learning, the low-rank representation (LRR) is one&#10;of the hot research topics in many fields, especially in image processing and&#10;pattern recognition. Although LRR can capture the global structure, the ability&#10;of local structure preservation is limited because LRR lacks dictionary&#10;learning. In this paper, we propose a novel multi-focus image fusion method&#10;based on dictionary learning and LRR to get a better performance in both global&#10;and local structure. Firstly, the source images are divided into several&#10;patches by sliding window technique. Then, the patches are classified according&#10;to the Histogram of Oriented Gradient (HOG) features. And the sub-dictionaries&#10;of each class are learned by K-singular value decomposition (K-SVD) algorithm.&#10;Secondly, a global dictionary is constructed by combining these&#10;sub-dictionaries. Then, we use the global dictionary in LRR to obtain the LRR&#10;coefficients vector for each patch. Finally, the l_1-norm and choose-max fuse&#10;strategy for each coefficients vector is adopted to reconstruct fused image&#10;from the fused LRR coefficients and the global dictionary. Experimental results&#10;demonstrate that the proposed method can obtain state-of-the-art performance in&#10;both qualitative and quantitative evaluations compared with serval classical&#10;methods and novel methods.The Code of our fusion method is available at&#10;https://github.com/hli1221/imagefusion_dllrr&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.05240" label="1704.05240">
-        <attvalues>
-          <attvalue for="0" value="Image Fusion With Cosparse Analysis Operator" />
-          <attvalue for="1" value="  The paper addresses the image fusion problem, where multiple images captured&#10;with different focus distances are to be combined into a higher quality&#10;all-in-focus image. Most current approaches for image fusion strongly rely on&#10;the unrealistic noise-free assumption used during the image acquisition, and&#10;then yield limited robustness in fusion processing. In our approach, we&#10;formulate the multi-focus image fusion problem in terms of an analysis sparse&#10;model, and simultaneously perform the restoration and fusion of multi-focus&#10;images. Based on this model, we propose an analysis operator learning, and&#10;define a novel fusion function to generate an all-in-focus image. Experimental&#10;evaluations confirm the effectiveness of the proposed fusion approach both&#10;visually and quantitatively, and show that our approach outperforms&#10;state-of-the-art fusion methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.07384" label="1712.07384">
-        <attvalues>
-          <attvalue for="0" value="DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme&#10;  Exposure Image Pairs" />
-          <attvalue for="1" value="  We present a novel deep learning architecture for fusing static&#10;multi-exposure images. Current multi-exposure fusion (MEF) approaches use&#10;hand-crafted features to fuse input sequence. However, the weak hand-crafted&#10;representations are not robust to varying input conditions. Moreover, they&#10;perform poorly for extreme exposure image pairs. Thus, it is highly desirable&#10;to have a method that is robust to varying input conditions and capable of&#10;handling extreme exposure without artifacts. Deep representations have known to&#10;be robust to input conditions and have shown phenomenal performance in a&#10;supervised setting. However, the stumbling block in using deep learning for MEF&#10;was the lack of sufficient training data and an oracle to provide the&#10;ground-truth for supervision. To address the above issues, we have gathered a&#10;large dataset of multi-exposure image stacks for training and to circumvent the&#10;need for ground truth images, we propose an unsupervised deep learning&#10;framework for MEF utilizing a no-reference quality metric as loss function. The&#10;proposed approach uses a novel CNN architecture trained to learn the fusion&#10;operation without reference ground truth image. The model fuses a set of common&#10;low level features extracted from each image to generate artifact-free&#10;perceptually pleasing results. We perform extensive quantitative and&#10;qualitative evaluation and show that the proposed technique outperforms&#10;existing state-of-the-art approaches for a variety of natural images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06992" label="1804.06992">
-        <attvalues>
-          <attvalue for="0" value="Infrared and Visible Image Fusion using a Deep Learning Framework" />
-          <attvalue for="1" value="  In recent years, deep learning has become a very active research tool which&#10;is used in many image processing fields. In this paper, we propose an effective&#10;image fusion method using a deep learning framework to generate a single image&#10;which contains all the features from infrared and visible images. First, the&#10;source images are decomposed into base parts and detail content. Then the base&#10;parts are fused by weighted-averaging. For the detail content, we use a deep&#10;learning network to extract multi-layer features. Using these features, we use&#10;l_1-norm and weighted-average strategy to generate several candidates of the&#10;fused detail content. Once we get these candidates, the max selection strategy&#10;is used to get final fused detail content. Finally, the fused image will be&#10;reconstructed by combining the fused base part and detail content. The&#10;experimental results demonstrate that our proposed method achieves&#10;state-of-the-art performance in both objective assessment and visual quality.&#10;The Code of our fusion method is available at&#10;https://github.com/hli1221/imagefusion_deeplearning&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.07119" label="1806.07119">
-        <attvalues>
-          <attvalue for="0" value="Infrared and Visible Image Fusion with ResNet and zero-phase component&#10;  analysis" />
-          <attvalue for="1" value="  Feature extraction and processing tasks play a key role in Image Fusion, and&#10;the fusion performance is directly affected by the different features and&#10;processing methods undertaken. By contrast, most of deep learning-based methods&#10;use deep features directly without feature extraction or processing. This leads&#10;to the fusion performance degradation in some cases. To solve these drawbacks,&#10;we propose a deep features and zero-phase component analysis (ZCA) based novel&#10;fusion framework is this paper. Firstly, the residual network (ResNet) is used&#10;to extract deep features from source images. Then ZCA is utilized to normalize&#10;the deep features and obtain initial weight maps. The final weight maps are&#10;obtained by employing a soft-max operation in association with the initial&#10;weight maps. Finally, the fused image is reconstructed using a&#10;weighted-averaging strategy. Compared with the existing fusion methods,&#10;experimental results demonstrate that the proposed framework achieves better&#10;performance in both objective assessment and visual quality. The code of our&#10;fusion algorithm is available at&#10;https://github.com/hli1221/imagefusion_resnet50&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.12939" label="1909.12939">
-        <attvalues>
-          <attvalue for="0" value="A weakly supervised adaptive triplet loss for deep metric learning" />
-          <attvalue for="1" value="  We address the problem of distance metric learning in visual similarity&#10;search, defined as learning an image embedding model which projects images into&#10;Euclidean space where semantically and visually similar images are closer and&#10;dissimilar images are further from one another. We present a weakly supervised&#10;adaptive triplet loss (ATL) capable of capturing fine-grained semantic&#10;similarity that encourages the learned image embedding models to generalize&#10;well on cross-domain data. The method uses weakly labeled product description&#10;data to implicitly determine fine grained semantic classes, avoiding the need&#10;to annotate large amounts of training data. We evaluate on the Amazon fashion&#10;retrieval benchmark and DeepFashion in-shop retrieval data. The method boosts&#10;the performance of triplet loss baseline by 10.6% on cross-domain data and&#10;out-performs the state-of-art model on all evaluation metrics.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Distance metric learning is used for many computer vision tasks such as image retrieval, duplicate detection and visual search. In fashion, one important application of metric learning is visual similarity recommendation which serves customers in many ways. For example, given a query image, the model finds similar products (see Figure \ref{fig-sim-search}). Visually similar recommendations provide a customer more choices, where the core fashion features are maintained but other non-visual factors are diversified (such as price range, brand or shipping cost). &#10;Another important application is visual similarity based clustering, which creates a new shopping experience in addition to keyword based search. Customers can explore the whole fashion catalog through clusters which contain visually similar products. In each collection, customers are more likely to find a product which satisfies their requirements.&#10;&#10;A wide variety of approaches have been proposed for distance metric learning such as contrastive loss \cite{hadsell2006dimensionality}, triplet loss \cite{wu2017sampling, ge2018deep} and NormSoftmax loss \cite{zhai2018making}. &#10;In triplet loss training, a triplet contains two images belonging to the same class, referred to as the anchor and positive samples, and a third image, from a different class, which is referred to as the negative sample. The triplet loss function is given as, $[d(a, p) - d(a, n) + m]+$, where $a$, $p$ and $n$ are anchor, positive, and negative samples, respectively. $d(\cdot, \cdot)$ is the learned metric function and $m$ is a margin term which encourages the negative sample to be further from the anchor than the positive sample. DNN based triplet loss training commonly uses stochastic gradient decent (SGD) on mini batches. &#10;&#10;Most deep metric learning algorithms, which only use coarse-grained product ID or classes, fail to learn distances that capture fine-grained sub-categories. Such fine-grained visual similarity distances are important to learn generalized visual features and to have robust performance on cross-domain data. Here, we construct an embedding of the product from its free form text product description and use this to drive an adaptive triplet loss. This allow us to take advantage of fine-grained semantic similarity information without explicitly determining what fine-grained classes exist in our dataset or incurring the cost of labeling a large training dataset.&#10;&#10;Our major contributions are as follows: (1) We propose a weakly supervised adaptive triplet loss (ATL) based on word embeddings of free form product descriptions. This also allows the metric learning process to generalize well on cross-domain datasets (table \ref{comp-dp}). (3) Inspired by \cite{ge2018deep}, We propose an online anchor nearest neighbor sampling (OANNS) method which addresses the batch sampling issue for large scale training data. Our method out-performs state-of-art on both our collected fashion retrieval benchmark and significantly boosts the performance on a cross-domain benchmark even without further tuning.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Science, Distance Metric Learning, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Visual Similarity Search, Computer Science, Image Embedding Models" />
-        </attvalues>
-      </node>
-      <node id="1706.07567" label="1706.07567">
-        <attvalues>
-          <attvalue for="0" value="Sampling Matters in Deep Embedding Learning" />
-          <attvalue for="1" value="  Deep embeddings answer one simple question: How similar are two images?&#10;Learning these embeddings is the bedrock of verification, zero-shot learning,&#10;and visual search. The most prominent approaches optimize a deep convolutional&#10;network with a suitable loss function, such as contrastive loss or triplet&#10;loss. While a rich line of work focuses solely on the loss functions, we show&#10;in this paper that selecting training examples plays an equally important role.&#10;We propose distance weighted sampling, which selects more informative and&#10;stable examples than traditional approaches. In addition, we show that a simple&#10;margin based loss is sufficient to outperform all other loss functions. We&#10;evaluate our approach on the Stanford Online Products, CAR196, and the&#10;CUB200-2011 datasets for image retrieval and clustering, and on the LFW dataset&#10;for face verification. Our method achieves state-of-the-art performance on all&#10;of them.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.06951" label="1810.06951">
-        <attvalues>
-          <attvalue for="0" value="Deep Metric Learning with Hierarchical Triplet Loss" />
-          <attvalue for="1" value="  We present a novel hierarchical triplet loss (HTL) capable of automatically&#10;collecting informative training samples (triplets) via a defined hierarchical&#10;tree that encodes global context information. This allows us to cope with the&#10;main limitation of random sampling in training a conventional triplet loss,&#10;which is a central issue for deep metric learning. Our main contributions are&#10;two-fold. (i) we construct a hierarchical class-level tree where neighboring&#10;classes are merged recursively. The hierarchical structure naturally captures&#10;the intrinsic data distribution over the whole database. (ii) we formulate the&#10;problem of triplet collection by introducing a new violate margin, which is&#10;computed dynamically based on the designed hierarchical tree. This allows it to&#10;automatically select meaningful hard samples with the guide of global context.&#10;It encourages the model to learn more discriminative features from visual&#10;similar classes, leading to faster convergence and better performance. Our&#10;method is evaluated on the tasks of image retrieval and face recognition, where&#10;it outperforms the standard triplet loss substantially by 1%-18%. It achieves&#10;new state-of-the-art performance on a number of benchmarks, with much fewer&#10;learning iterations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.17319" label="2402.17319">
-        <attvalues>
-          <attvalue for="0" value="A Vanilla Multi-Task Framework for Dense Visual Prediction Solution to&#10;  1st VCL Challenge -- Multi-Task Robustness Track" />
-          <attvalue for="1" value="  In this report, we present our solution to the multi-task robustness track of&#10;the 1st Visual Continual Learning (VCL) Challenge at ICCV 2023 Workshop. We&#10;propose a vanilla framework named UniNet that seamlessly combines various&#10;visual perception algorithms into a multi-task model. Specifically, we choose&#10;DETR3D, Mask2Former, and BinsFormer for 3D object detection, instance&#10;segmentation, and depth estimation tasks, respectively. The final submission is&#10;a single model with InternImage-L backbone, and achieves a 49.6 overall score&#10;(29.5 Det mAP, 80.3 mTPS, 46.4 Seg mAP, and 7.93 silog) on SHIFT validation&#10;set. Besides, we provide some interesting observations in our experiments which&#10;may facilitate the development of multi-task learning in dense visual&#10;prediction.&#10;" />
-          <attvalue for="2" value="&#10;&#10;We introduce a vanilla multi-task dense visual prediction framework for 1$^{st}$ visual continual learning (VCL) Challenge -- multi-task robustness track at ICCV 2023 Workshop. It introduces SHIFT dataset \cite{sun2022shift}, a multi-task driving dataset featuring the most important perception tasks under a variety of conditions and with a comprehensive sensor setup. It consists of 4,850 video sequences (2.5M image frames), as well as rich annotations, covering 2D/3D detection and tracking, depth estimation, instance/semantic segmentation tasks. The multi-task robustness track evaluates the overall model performance on the 3D detection, instance segmentation, and depth estimation tasks. &#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Continual Learning, Visual Perception Algorithms, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Multi-Task Learning" />
-        </attvalues>
-      </node>
-      <node id="2206.08367" label="2206.08367">
-        <attvalues>
-          <attvalue for="0" value="SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain&#10;  Adaptation" />
-          <attvalue for="1" value="  Adapting to a continuously evolving environment is a safety-critical&#10;challenge inevitably faced by all autonomous driving systems. Existing image&#10;and video driving datasets, however, fall short of capturing the mutable nature&#10;of the real world. In this paper, we introduce the largest multi-task synthetic&#10;dataset for autonomous driving, SHIFT. It presents discrete and continuous&#10;shifts in cloudiness, rain and fog intensity, time of day, and vehicle and&#10;pedestrian density. Featuring a comprehensive sensor suite and annotations for&#10;several mainstream perception tasks, SHIFT allows investigating the degradation&#10;of a perception system performance at increasing levels of domain shift,&#10;fostering the development of continuous adaptation strategies to mitigate this&#10;problem and assess model robustness and generality. Our dataset and benchmark&#10;toolkit are publicly available at www.vis.xyz/shift.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.00212" label="2307.00212">
-        <attvalues>
-          <attvalue for="0" value="Internal-External Boundary Attention Fusion for Glass Surface&#10;  Segmentation" />
-          <attvalue for="1" value="  Glass surfaces of transparent objects and mirrors are not able to be uniquely&#10;and explicitly characterized by their visual appearances because they contain&#10;the visual appearance of other reflected or transmitted surfaces as well.&#10;Detecting glass regions from a single-color image is a challenging task. Recent&#10;deep-learning approaches have paid attention to the description of glass&#10;surface boundary where the transition of visual appearances between glass and&#10;non-glass surfaces are observed. In this work, we analytically investigate how&#10;glass surface boundary helps to characterize glass objects. Inspired by prior&#10;semantic segmentation approaches with challenging image types such as X-ray or&#10;CT scans, we propose separated internal-external boundary attention modules&#10;that individually learn and selectively integrate visual characteristics of the&#10;inside and outside region of glass surface from a single color image. Our&#10;proposed method is evaluated on six public benchmarks comparing with&#10;state-of-the-art methods showing promising results.&#10;" />
-          <attvalue for="2" value="&#10;Transparent objects and mirror surfaces are everywhere around such as windows, bottles, eye glasses and omnipresent mirrors. They are expected to be detected, localized and reconstructed from color images in computer vision tasks as other opaque objects are done. However, glass surface is not able to be uniquely characterized by their visual appearances. Most transparent surface region show visual appearance of both transmitted background scene and reflected objects. Visual appearance observed from mirror surface is entirely from reflected objects. Therefore, visual characteristics of glass object are not able to be uniquely defined and observed. This makes performance limitations of computer vision methods in practical many applications. For example, autonomous mobile robot may collide with transparent front door or mirror wall. Robot arm struggles to grip a transparent bottle. &#10;&#10;Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}. Long \etal~\cite{long2015fully} transpose feature extraction layers to a segmentation task and update parameters by fine tuning. At the same time, it designs a novel structure that combines low-level and high-level semantics, which is extensively utilized by following semantic segmentation networks such as \cite{zhao2017pyramid}.&#10;Lin \etal~\cite{lin2019refinenet} integrate the features of encoder and decoder phases and utilizes chained residual pooling in the module to capture contextual information over a wide range of contexts. Non-local operators \cite{wang2018non} based on a self-attention mechanism \cite{chorowski2015attention} are applied in semantic segmentation \cite{fu2019dual}\cite{babiloni2020tesa}\cite{zhang2023faster}. Song \etal~\cite{song2022fully} propose fully attentional blocks to encode spatial and channel attentions in similarity graphs with high computational efficiency. Most of these methods, however, fail to characterize common visual appearance of glass surface within the boundary of transparent or mirror object region.&#10;&#10;Recently, glass surface object segmentation methods are proposed including transparent objects \cite{xie2020segmenting} \cite{zheng2022glassnet} \cite{he2021enhanced} \cite{he2021enhanced} and strongly reflective objects such as mirror \cite{guan2022learning} \cite{mei2021depth} \cite{yang2019my} \cite{han2023segment}. Mei \etal~\cite{mei2020don} propose to use different dilation convolutions and thus use different sensory fields to capture contrast features and discontinuities in semantic information. However, overlapped non-glass objects lead to semantic discontinuities.&#10;Xie \etal~\cite{xie2020segmenting} propose a new benchmark for glass object segmentation and considers the characteristics of transparent object boundary features. EBLNet \cite{he2021enhanced} extracts boundary features employing a non-local approach. It enhances whole boundary features of glass surface and consider the relationship between the inside and the boundary of glass object region. By assigning additional attention on the boundary of glass object, they alleviate the problem of the deficiency of common visual appearance inside the boundary of transparent or mirror objects. Guan \etal~\cite{guan2022learning} propose a model to exploit the semantic associations between the mirror and its surrounding objects for a reliable mirror localization. However, the method tends to be biased to the surrounding objects appeared in the training dataset that are not core features of glass objects. Xie \etal~\cite{xie2021segmenting} employ attentional mechanisms to explore relationships among glass objects of different categories. Reflections and ghosts are employed as clues for the identification of glass surface. RCAM\cite{lin2021rich} guides the network to identify reflections based on the feature information obtained from the reflection removal network\cite{zhang2018single}. &#10;&#10;On the other hand, external boundaries have played a crucial role in salient object segmentation\cite{wei2020f3net}\cite{zhu2023sharp}\cite{qiao2023robustness}, especially in medical imaging, such as CT image segmentation\cite{ghadimi2015skull} \cite{shojaii2005automatic} \cite{bai2016automatic}. In CT scan or X-ray images, visual textures of human organ are not clearly separable to each other. Instead, thick skeleton or outer boundary features have been utilized for the segmentation of organ regions in CT images\cite{shojaii2005automatic}. &#10;In CT scans, boundary of organ region is ambiguous containing lots of noises. In X-ray images, bones and organs are overlapped showing confused boundary connections. In such cases, organ region segmentation focuses on the boundary and texture features of external boundary, which is the order of human observation\cite{ghadimi2015skull}.&#10;Our careful observations on the prior methods and diverse transparent and mirror objects bring the conclusion that neither reflections or ghosts inside glass region nor surroundings of glass object can reliably describe glass surface. As the clues (either at inside or outside of glass region) are located farther from the boundary of a glass object, observed features are less relevant to the characteristics of glass surface. Instead, we have to focus more on the inner and outer vicinity of the boundary. Outer vicinity of the boundary contains coherent appearance that decide the physical frame of a glass object. Inner vicinity strongly reveals the characteristics of glass surface based on the light refraction or distortion, while is less affected by the appearance of transmitted background objects than the middle area of glass surface. These two aspects compositely describe the existence of glass surface. Actually, this scheme is how human perceive glass surface under challenging environment.&#10;&#10;According to the insight, we define external boundary (exboundary) and internal boundary (inboundary) as respective region of outer vicinity and inner vicinity of the boundary of glass object. Our aim is to address glass object surface segmentation problem. We design deep neural networks based on the following four observations: (1) Human recognize an object from salient features first. This is also obvious with glass objects. Human tend to identify the potential area of a glass surface object from its easily distinguishable internal and external boundary regions, rather than entire glass surface region. (2) Glass surface objects show diverse internal and external boundary appearances and their relative importance. Window frame is strong clue observed from external boundary region of transparent surface. A glass bottle does not have external boundary but have strong internal boundary characteristics from light refraction. (3) Internal and external boundaries have different role is the recognition of glass surface region. Window frame observed at the external boundary of window decide potential glass surface region. Further investigation of internal boundary searching for reflections or ghosts determines the presence of window glass. Ignoring such semantic differences between internal and external boundary regions may result in incorrect segmentation of window frames without window glass or opened doors. (4) Closer region to the boundary shows stronger characteristics of glass surface. Figure \ref{firstpic} shows two challenging glass objects. Glass cup shows strong internal boundary characteristics but weak external boundary. On the other hand window has strong external boundary clue (window frame).&#10;&#10;Based on our study and observations we claim that glass object segmentation networks have to collect external and internal boundary features individually. And then they have to be selectively integrated for potential glass surface region segmentation and presence determination of glass surface, respectively. To this end, we construct Internal-External Boundary Attention Module (IEBAM) with step-by-step supervision improving the feature fusion of residual networks\cite{he2016deep}.&#10;IEBAM utilizes an attention mechanism to implement the human scheme of glass surface detection. It extracts salient external and internal boundary features of glass surface object such as glass frame and light distortion respectively. &#10;After that it obtains more detailed internal boundary features for glass detection. In order to handle glass surface objects with weak external boundary such as glass cups where external boundary fails to provide significant features, we propose Fused Boundary Attention Module (FBAM). FBAM combines internal and external boundaries and weighs more on internal boundary features by learning the semantic clue of transparent objects. Glass surface region is determined by finding the relationship between the reinforced boundary and glass features such as reflections occurring inside the transparent object. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Physics, Image Processing, Deep Learning Approaches, Computer Science, Glass Surface Detection, Image Segmentation Techniques" />
-        </attvalues>
-      </node>
-      <node id="2311.11465" label="2311.11465">
-        <attvalues>
-          <attvalue for="0" value="Understanding Segment Anything Model: SAM is Biased Towards Texture&#10;  Rather than Shape" />
-          <attvalue for="1" value="  In contrast to the human vision that mainly depends on the shape for&#10;recognizing the objects, deep image recognition models are widely known to be&#10;biased toward texture. Recently, Meta research team has released the first&#10;foundation model for image segmentation, termed segment anything model (SAM),&#10;which has attracted significant attention. In this work, we understand SAM from&#10;the perspective of texture \textit{v.s.} shape. Different from label-oriented&#10;recognition tasks, the SAM is trained to predict a mask for covering the object&#10;shape based on a promt. With this said, it seems self-evident that the SAM is&#10;biased towards shape. In this work, however, we reveal an interesting finding:&#10;the SAM is strongly biased towards texture-like dense features rather than&#10;shape. This intriguing finding is supported by a novel setup where we&#10;disentangle texture and shape cues and design texture-shape cue conflict for&#10;mask prediction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.11717" label="2303.11717">
-        <attvalues>
-          <attvalue for="0" value="A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to&#10;  GPT-5 All You Need?" />
-          <attvalue for="1" value="  As ChatGPT goes viral, generative AI (AIGC, a.k.a AI-generated content) has&#10;made headlines everywhere because of its ability to analyze and create text,&#10;images, and beyond. With such overwhelming media coverage, it is almost&#10;impossible for us to miss the opportunity to glimpse AIGC from a certain angle.&#10;In the era of AI transitioning from pure analysis to creation, it is worth&#10;noting that ChatGPT, with its most recent language model GPT-4, is just a tool&#10;out of numerous AIGC tasks. Impressed by the capability of the ChatGPT, many&#10;people are wondering about its limits: can GPT-5 (or other future GPT variants)&#10;help ChatGPT unify all AIGC tasks for diversified content creation? Toward&#10;answering this question, a comprehensive review of existing AIGC tasks is&#10;needed. As such, our work comes to fill this gap promptly by offering a first&#10;look at AIGC, ranging from its techniques to applications. Modern generative AI&#10;relies on various technical foundations, ranging from model architecture and&#10;self-supervised pretraining to generative modeling methods (like GAN and&#10;diffusion models). After introducing the fundamental techniques, this work&#10;focuses on the technological development of various AIGC tasks based on their&#10;output type, including text, images, videos, 3D content, etc., which depicts&#10;the full potential of ChatGPT's future. Moreover, we summarize their&#10;significant applications in some mainstream industries, such as education and&#10;creativity content. Finally, we discuss the challenges currently faced and&#10;present an outlook on how generative AI might evolve in the near future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.06488" label="2304.06488">
-        <attvalues>
-          <attvalue for="0" value="One Small Step for Generative AI, One Giant Leap for AGI: A Complete&#10;  Survey on ChatGPT in AIGC Era" />
-          <attvalue for="1" value="  OpenAI has recently released GPT-4 (a.k.a. ChatGPT plus), which is&#10;demonstrated to be one small step for generative AI (GAI), but one giant leap&#10;for artificial general intelligence (AGI). Since its official release in&#10;November 2022, ChatGPT has quickly attracted numerous users with extensive&#10;media coverage. Such unprecedented attention has also motivated numerous&#10;researchers to investigate ChatGPT from various aspects. According to Google&#10;scholar, there are more than 500 articles with ChatGPT in their titles or&#10;mentioning it in their abstracts. Considering this, a review is urgently&#10;needed, and our work fills this gap. Overall, this work is the first to survey&#10;ChatGPT with a comprehensive review of its underlying technology, applications,&#10;and challenges. Moreover, we present an outlook on how ChatGPT might evolve to&#10;realize general-purpose AIGC (a.k.a. AI-generated content), which will be a&#10;significant milestone for the development of AGI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.01105" label="1612.01105">
-        <attvalues>
-          <attvalue for="0" value="Pyramid Scene Parsing Network" />
-          <attvalue for="1" value="  Scene parsing is challenging for unrestricted open vocabulary and diverse&#10;scenes. In this paper, we exploit the capability of global context information&#10;by different-region-based context aggregation through our pyramid pooling&#10;module together with the proposed pyramid scene parsing network (PSPNet). Our&#10;global prior representation is effective to produce good quality results on the&#10;scene parsing task, while PSPNet provides a superior framework for pixel-level&#10;prediction tasks. The proposed approach achieves state-of-the-art performance&#10;on various datasets. It came first in ImageNet scene parsing challenge 2016,&#10;PASCAL VOC 2012 benchmark and Cityscapes benchmark. A single PSPNet yields new&#10;record of mIoU accuracy 85.4% on PASCAL VOC 2012 and accuracy 80.2% on&#10;Cityscapes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.07503" label="1506.07503">
-        <attvalues>
-          <attvalue for="0" value="Attention-Based Models for Speech Recognition" />
-          <attvalue for="1" value="  Recurrent sequence generators conditioned on input data through an attention&#10;mechanism have recently shown very good performance on a range of tasks in-&#10;cluding machine translation, handwriting synthesis and image caption gen-&#10;eration. We extend the attention-mechanism with features needed for speech&#10;recognition. We show that while an adaptation of the model used for machine&#10;translation in reaches a competitive 18.7% phoneme error rate (PER) on the&#10;TIMIT phoneme recognition task, it can only be applied to utterances which are&#10;roughly as long as the ones it was trained on. We offer a qualitative&#10;explanation of this failure and propose a novel and generic method of adding&#10;location-awareness to the attention mechanism to alleviate this issue. The new&#10;method yields a model that is robust to long inputs and achieves 18% PER in&#10;single utterances and 20% in 10-times longer (repeated) utterances. Finally, we&#10;propose a change to the at- tention mechanism that prevents it from&#10;concentrating too much on single frames, which further reduces PER to 17.6%&#10;level.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.02983" label="1809.02983">
-        <attvalues>
-          <attvalue for="0" value="Dual Attention Network for Scene Segmentation" />
-          <attvalue for="1" value="  In this paper, we address the scene segmentation task by capturing rich&#10;contextual dependencies based on the selfattention mechanism. Unlike previous&#10;works that capture contexts by multi-scale features fusion, we propose a Dual&#10;Attention Networks (DANet) to adaptively integrate local features with their&#10;global dependencies. Specifically, we append two types of attention modules on&#10;top of traditional dilated FCN, which model the semantic interdependencies in&#10;spatial and channel dimensions respectively. The position attention module&#10;selectively aggregates the features at each position by a weighted sum of the&#10;features at all positions. Similar features would be related to each other&#10;regardless of their distances. Meanwhile, the channel attention module&#10;selectively emphasizes interdependent channel maps by integrating associated&#10;features among all channel maps. We sum the outputs of the two attention&#10;modules to further improve feature representation which contributes to more&#10;precise segmentation results. We achieve new state-of-the-art segmentation&#10;performance on three challenging scene segmentation datasets, i.e., Cityscapes,&#10;PASCAL Context and COCO Stuff dataset. In particular, a Mean IoU score of 81.5%&#10;on Cityscapes test set is achieved without using coarse data. We make the code&#10;and trained model publicly available at https://github.com/junfu1115/DANet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.14289" label="2306.14289">
-        <attvalues>
-          <attvalue for="0" value="Faster Segment Anything: Towards Lightweight SAM for Mobile Applications" />
-          <attvalue for="1" value="  Segment Anything Model (SAM) has attracted significant attention due to its&#10;impressive zero-shot transfer performance and high versatility for numerous&#10;vision applications (like image editing with fine-grained control). Many of&#10;such applications need to be run on resource-constraint edge devices, like&#10;mobile phones. In this work, we aim to make SAM mobile-friendly by replacing&#10;the heavyweight image encoder with a lightweight one. A naive way to train such&#10;a new SAM as in the original SAM paper leads to unsatisfactory performance,&#10;especially when limited training sources are available. We find that this is&#10;mainly caused by the coupled optimization of the image encoder and mask&#10;decoder, motivated by which we propose decoupled distillation. Concretely, we&#10;distill the knowledge from the heavy image encoder (ViT-H in the original SAM)&#10;to a lightweight image encoder, which can be automatically compatible with the&#10;mask decoder in the original SAM. The training can be completed on a single GPU&#10;within less than one day, and the resulting lightweight SAM is termed MobileSAM&#10;which is more than 60 times smaller yet performs on par with the original SAM.&#10;For inference speed, With a single GPU, MobileSAM runs around 10ms per image:&#10;8ms on the image encoder and 4ms on the mask decoder. With superior&#10;performance, our MobileSAM is around 5 times faster than the concurrent FastSAM&#10;and 7 times smaller, making it more suitable for mobile applications. Moreover,&#10;we show that MobileSAM can run relatively smoothly on CPU. The code for our&#10;project is provided at&#10;\href{https://github.com/ChaoningZhang/MobileSAM}{\textcolor{red}{MobileSAM}}),&#10;with a demo showing that MobileSAM can run relatively smoothly on CPU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.04108" label="2112.04108">
-        <attvalues>
-          <attvalue for="0" value="Fully Attentional Network for Semantic Segmentation" />
-          <attvalue for="1" value="  Recent non-local self-attention methods have proven to be effective in&#10;capturing long-range dependencies for semantic segmentation. These methods&#10;usually form a similarity map of RC*C (by compressing spatial dimensions) or&#10;RHW*HW (by compressing channels) to describe the feature relations along either&#10;channel or spatial dimensions, where C is the number of channels, H and W are&#10;the spatial dimensions of the input feature map. However, such practices tend&#10;to condense feature dependencies along the other dimensions,hence causing&#10;attention missing, which might lead to inferior results for small/thin&#10;categories or inconsistent segmentation inside large objects. To address this&#10;problem, we propose anew approach, namely Fully Attentional Network (FLANet),to&#10;encode both spatial and channel attentions in a single similarity map while&#10;maintaining high computational efficiency. Specifically, for each channel map,&#10;our FLANet can harvest feature responses from all other channel maps, and the&#10;associated spatial positions as well, through a novel fully attentional module.&#10;Our new method has achieved state-of-the-art performance on three challenging&#10;semantic segmentation datasets,i.e., 83.6%, 46.99%, and 88.5% on the Cityscapes&#10;test set,the ADE20K validation set, and the PASCAL VOC test set,respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.13948" label="2003.13948">
-        <attvalues>
-          <attvalue for="0" value="Segmenting Transparent Objects in the Wild" />
-          <attvalue for="1" value="  Transparent objects such as windows and bottles made by glass widely exist in&#10;the real world. Segmenting transparent objects is challenging because these&#10;objects have diverse appearance inherited from the image background, making&#10;them had similar appearance with their surroundings. Besides the technical&#10;difficulty of this task, only a few previous datasets were specially designed&#10;and collected to explore this task and most of the existing datasets have major&#10;drawbacks. They either possess limited sample size such as merely a thousand of&#10;images without manual annotations, or they generate all images by using&#10;computer graphics method (i.e. not real image). To address this important&#10;problem, this work proposes a large-scale dataset for transparent object&#10;segmentation, named Trans10K, consisting of 10,428 images of real scenarios&#10;with carefully manual annotations, which are 10 times larger than the existing&#10;datasets. The transparent objects in Trans10K are extremely challenging due to&#10;high diversity in scale, viewpoint and occlusion as shown in Fig. 1. To&#10;evaluate the effectiveness of Trans10K, we propose a novel boundary-aware&#10;segmentation method, termed TransLab, which exploits boundary as the clue to&#10;improve segmentation of transparent objects. Extensive experiments and ablation&#10;studies demonstrate the effectiveness of Trans10K and validate the practicality&#10;of learning object boundary in TransLab. For example, TransLab significantly&#10;outperforms 20 recent object segmentation methods based on deep learning,&#10;showing that this task is largely unsolved. We believe that both Trans10K and&#10;TransLab have important contributions to both the academia and industry,&#10;facilitating future researches and applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.11117" label="2108.11117">
-        <attvalues>
-          <attvalue for="0" value="GlassNet: Label Decoupling-based Three-stream Neural Network for Robust&#10;  Image Glass Detection" />
-          <attvalue for="1" value="  Most of the existing object detection methods generate poor glass detection&#10;results, due to the fact that the transparent glass shares the same appearance&#10;with arbitrary objects behind it in an image. Different from traditional deep&#10;learning-based wisdoms that simply use the object boundary as auxiliary&#10;supervision, we exploit label decoupling to decompose the original labeled&#10;ground-truth (GT) map into an interior-diffusion map and a boundary-diffusion&#10;map. The GT map in collaboration with the two newly generated maps breaks the&#10;imbalanced distribution of the object boundary, leading to improved glass&#10;detection quality. We have three key contributions to solve the transparent&#10;glass detection problem: (1) We propose a three-stream neural network (call&#10;GlassNet for short) to fully absorb beneficial features in the three maps. (2)&#10;We design a multi-scale interactive dilation module to explore a wider range of&#10;contextual information. (3) We develop an attention-based boundary-aware&#10;feature Mosaic module to integrate multi-modal information. Extensive&#10;experiments on the benchmark dataset exhibit clear improvements of our method&#10;over SOTAs, in terms of both the overall glass detection accuracy and boundary&#10;clearness.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.15734" label="2103.15734">
-        <attvalues>
-          <attvalue for="0" value="Enhanced Boundary Learning for Glass-like Object Segmentation" />
-          <attvalue for="1" value="  Glass-like objects such as windows, bottles, and mirrors exist widely in the&#10;real world. Sensing these objects has many applications, including robot&#10;navigation and grasping. However, this task is very challenging due to the&#10;arbitrary scenes behind glass-like objects. This paper aims to solve the&#10;glass-like object segmentation problem via enhanced boundary learning. In&#10;particular, we first propose a novel refined differential module that outputs&#10;finer boundary cues. We then introduce an edge-aware point-based graph&#10;convolution network module to model the global shape along the boundary. We use&#10;these two modules to design a decoder that generates accurate and clean&#10;segmentation results, especially on the object contours. Both modules are&#10;lightweight and effective: they can be embedded into various segmentation&#10;models. In extensive experiments on three recent glass-like object segmentation&#10;datasets, including Trans10k, MSD, and GDD, our approach establishes new&#10;state-of-the-art results. We also illustrate the strong generalization&#10;properties of our method on three generic segmentation datasets, including&#10;Cityscapes, BDD, and COCO Stuff. Code and models is available at&#10;\url{https://github.com/hehao13/EBLNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.00278" label="2305.00278">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects&#10;  Cannot Be Easily Detected" />
-          <attvalue for="1" value="  Meta AI Research has recently released SAM (Segment Anything Model) which is&#10;trained on a large segmentation dataset of over 1 billion masks. As a&#10;foundation model in the field of computer vision, SAM (Segment Anything Model)&#10;has gained attention for its impressive performance in generic object&#10;segmentation. Despite its strong capability in a wide range of zero-shot&#10;transfer tasks, it remains unknown whether SAM can detect things in challenging&#10;setups like transparent objects. In this work, we perform an empirical&#10;evaluation of two glass-related challenging scenarios: mirror and transparent&#10;objects. We found that SAM often fails to detect the glass in both scenarios,&#10;which raises concern for deploying the SAM in safety-critical situations that&#10;have various forms of glass.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.08461" label="2101.08461">
-        <attvalues>
-          <attvalue for="0" value="Segmenting Transparent Object in the Wild with Transformer" />
-          <attvalue for="1" value="  This work presents a new fine-grained transparent object segmentation&#10;dataset, termed Trans10K-v2, extending Trans10K-v1, the first large-scale&#10;transparent object segmentation dataset. Unlike Trans10K-v1 that only has two&#10;limited categories, our new dataset has several appealing benefits. (1) It has&#10;11 fine-grained categories of transparent objects, commonly occurring in the&#10;human domestic environment, making it more practical for real-world&#10;application. (2) Trans10K-v2 brings more challenges for the current advanced&#10;segmentation methods than its former version. Furthermore, a novel&#10;transformer-based segmentation pipeline termed Trans2Seg is proposed. Firstly,&#10;the transformer encoder of Trans2Seg provides the global receptive field in&#10;contrast to CNN's local receptive field, which shows excellent advantages over&#10;pure CNN architectures. Secondly, by formulating semantic segmentation as a&#10;problem of dictionary look-up, we design a set of learnable prototypes as the&#10;query of Trans2Seg's transformer decoder, where each prototype learns the&#10;statistics of one category in the whole dataset. We benchmark more than 20&#10;recent semantic segmentation methods, demonstrating that Trans2Seg&#10;significantly outperforms all the CNN-based methods, showing the proposed&#10;algorithm's potential ability to solve transparent object segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.05376" label="1806.05376">
-        <attvalues>
-          <attvalue for="0" value="Single Image Reflection Separation with Perceptual Losses" />
-          <attvalue for="1" value="  We present an approach to separating reflection from a single image. The&#10;approach uses a fully convolutional network trained end-to-end with losses that&#10;exploit low-level and high-level image information. Our loss function includes&#10;two perceptual losses: a feature loss from a visual perception network, and an&#10;adversarial loss that encodes characteristics of images in the transmission&#10;layers. We also propose a novel exclusion loss that enforces pixel-level layer&#10;separation. We create a dataset of real-world images with reflection and&#10;corresponding ground-truth transmission layers for quantitative evaluation and&#10;model training. We validate our method through comprehensive quantitative&#10;experiments and show that our approach outperforms state-of-the-art reflection&#10;removal methods in PSNR, SSIM, and perceptual user study. We also extend our&#10;method to two other image enhancement tasks to demonstrate the generality of&#10;our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.07431" label="2301.07431">
-        <attvalues>
-          <attvalue for="0" value="Sharp Eyes: A Salient Object Detector Working The Same Way as Human&#10;  Visual Characteristics" />
-          <attvalue for="1" value="  Current methods aggregate multi-level features or introduce edge and skeleton&#10;to get more refined saliency maps. However, little attention is paid to how to&#10;obtain the complete salient object in cluttered background, where the targets&#10;are usually similar in color and texture to the background. To handle this&#10;complex scene, we propose a sharp eyes network (SENet) that first seperates the&#10;object from scene, and then finely segments it, which is in line with human&#10;visual characteristics, i.e., to look first and then focus. Different from&#10;previous methods which directly integrate edge or skeleton to supplement the&#10;defects of objects, the proposed method aims to utilize the expanded objects to&#10;guide the network obtain complete prediction. Specifically, SENet mainly&#10;consists of target separation (TS) brach and object segmentation (OS) branch&#10;trained by minimizing a new hierarchical difference aware (HDA) loss. In the TS&#10;branch, we construct a fractal structure to produce saliency features with&#10;expanded boundary via the supervision of expanded ground truth, which can&#10;enlarge the detail difference between foreground and background. In the OS&#10;branch, we first aggregate multi-level features to adaptively select&#10;complementary components, and then feed the saliency features with expanded&#10;boundary into aggregated features to guide the network obtain complete&#10;prediction. Moreover, we propose the HDA loss to further improve the structural&#10;integrity and local details of the salient objects, which assigns weight to&#10;each pixel according to its distance from the boundary hierarchically. Hard&#10;pixels with similar appearance in border region will be given more attention&#10;hierarchically to emphasize their importance in completeness prediction.&#10;Comprehensive experimental results on five datasets demonstrate that the&#10;proposed approach outperforms the state-of-the-art methods both quantitatively&#10;and qualitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.07713" label="2306.07713">
-        <attvalues>
-          <attvalue for="0" value="Robustness of SAM: Segment Anything Under Corruptions and Beyond" />
-          <attvalue for="1" value="  Segment anything model (SAM), as the name suggests, is claimed to be capable&#10;of cutting out any object and demonstrates impressive zero-shot transfer&#10;performance with the guidance of prompts. However, there is currently a lack of&#10;comprehensive evaluation regarding its robustness under various corruptions.&#10;Understanding the robustness of SAM across different corruption scenarios is&#10;crucial for its real-world deployment. Prior works show that SAM is biased&#10;towards texture (style) rather than shape, motivated by which we start by&#10;investigating its robustness against style transfer, which is synthetic&#10;corruption. Following by interpreting the effects of synthetic corruption as&#10;style changes, we proceed to conduct a comprehensive evaluation for its&#10;robustness against 15 types of common corruption. These corruptions mainly fall&#10;into categories such as digital, noise, weather, and blur, and within each&#10;corruption category, we explore 5 severity levels to simulate real-world&#10;corruption scenarios. Beyond the corruptions, we further assess the robustness&#10;of SAM against local occlusion and local adversarial patch attacks. To the best&#10;of our knowledge, our work is the first of its kind to evaluate the robustness&#10;of SAM under style change, local occlusion, and local adversarial patch&#10;attacks. Given that patch attacks visible to human eyes are easily detectable,&#10;we further assess its robustness against global adversarial attacks that are&#10;imperceptible to human eyes. Overall, this work provides a comprehensive&#10;empirical study of the robustness of SAM, evaluating its performance under&#10;various corruptions and extending the assessment to critical aspects such as&#10;local occlusion, local adversarial patch attacks, and global adversarial&#10;attacks. These evaluations yield valuable insights into the practical&#10;applicability and effectiveness of SAM in addressing real-world challenges.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.03243" label="1810.03243">
-        <attvalues>
-          <attvalue for="0" value="Arc-support Line Segments Revisited: An Efficient and High-quality&#10;  Ellipse Detection" />
-          <attvalue for="1" value="  Over the years many ellipse detection algorithms spring up and are studied&#10;broadly, while the critical issue of detecting ellipses accurately and&#10;efficiently in real-world images remains a challenge. In this paper, we propose&#10;a valuable industry-oriented ellipse detector by arc-support line segments,&#10;which simultaneously reaches high detection accuracy and efficiency. To&#10;simplify the complicated curves in an image while retaining the general&#10;properties including convexity and polarity, the arc-support line segments are&#10;extracted, which grounds the successful detection of ellipses. The arc-support&#10;groups are formed by iteratively and robustly linking the arc-support line&#10;segments that latently belong to a common ellipse. Afterward, two complementary&#10;approaches, namely, locally selecting the arc-support group with higher&#10;saliency and globally searching all the valid paired groups, are adopted to fit&#10;the initial ellipses in a fast way. Then, the ellipse candidate set can be&#10;formulated by hierarchical clustering of 5D parameter space of initial&#10;ellipses. Finally, the salient ellipse candidates are selected and refined as&#10;detections subject to the stringent and effective verification. Extensive&#10;experiments on three public datasets are implemented and our method achieves&#10;the best F-measure scores compared to the state-of-the-art methods. The source&#10;code is available at&#10;https://github.com/AlanLuSun/High-quality-ellipse-detection.&#10;" />
-          <attvalue for="2" value="&#10;E{llipse} detection is a fundamental technique in image processing field and plays an indispensable role in shape detection and geometric measurement.&#10;Actually, ellipse detector can be utilized to handle various real-world problems.&#10;In PCB inspection field, one basic function of the defect detection machine is to precisely as well as fast locate the circular pads or holes. Moreover, accurate measurement of circular control points and elliptic fiducial markers is helpful to homography estimation and camera calibration~\cite{huang2016homography,calvet2016detection,heikkila2000geometric}, and some irregular objects could be fitted as ellipses to simplify the shape structure for efficient mathematical modeling \cite{crocco2016structure,da2010fitting,bai2009splitting,kothari2009automated}.&#10;However, to our best knowledge, there exist few robust, stable, efficient and accurate ellipse detector algorithms to universally handle the ellipse detection problem in real-world images, which may have the presence of cluttered edges, motion blur, illumination, occlusion, noise and so on. The major reason is that an ellipse involves five parameters rather than that a circle just needs three, which results in detecting ellipse both efficiently and accurately to be a tough problem. Recently, convolutional neural network (CNN) \cite{lecun2015deep} is revolutionizing objection detection field, like Mask R-CNN \cite{he2017mask} and YOLO \cite{redmon2018yolov3}. Deep learning based methods can provide image proposals which contain oval objects for image pre-processing while they are still inappropriate to directly handle ellipse detection due to the issues of limited segmentation accuracy and expensive manual annotation. In most real-world applications, the practical requirements with regard to higher location accuracy and faster speed make ellipse detection problem even challenging.\par&#10;&#10;Existing commonly used methods for ellipse detection can&#10;be briefly grouped into two categories: 1) Hough Transform; 2) Edge Following.\par&#10;&#10;Hough Transform (HT) has been widely used for detecting geometric primitives such as line segment (or LS for short), circle and ellipse~\cite{Duda1972Use}. The basic idea of HT for ellipse detection is voting arbitrary edge pixels into 5D parameter space. The local peak will occur when the corresponding bin of accumulator exceeds a threshold of votes, which implies for detecting an ellipse. But it is almost impractical to directly apply HT in practice due to the heavy computation burden and excessive consumption of memory. To alleviate these issues, considerable improved methods are put forward. Probabilistic Hough Transform (PHT) randomly selects a small subset of the edge points which is used as input for HT~\cite{kiryati1991probabilistic}, but large-scale attempts are taken to find the points all sharing a common ellipse and it leads to inferior performance when substantial noise exists. Yuen et al. decomposed the 5D parameter space by finding the ellipse center using some geometric properties like colinearity and symmetry on the first stage and then finding the remaining three parameters on the second stage~\cite{yuen1989detecting,tsuji1978detection}. Instead of transforming each edge point into a 5D parameter space, Xu et al.~\cite{xu1990RHT} proposed a Randomized Hough Transform (RHT) to detect curves, which randomly chooses five edge pixels each time and maps them into a point of the ellipse parameter space. McLaughlin et al. \cite{mclaughlin1998randomized} combined the aforementioned two-stage decomposition method and RHT at the aim of reducing the computation time and improving the detection performance compared with the standard HT, which becomes a baseline of ellipse detection method in the literature (Fig.~\ref{fig1}(c)). However, it is still not efficient enough in practice and always generates false detections due to the lack of novel validation strategies. Despite the simplicity of HT, HT based ellipse detection methods suffer from the following legacy problems: First, it is vulnerable in front of substantial image noise and complicated real-world background due to false peaks; Second, it requires much effort to tune the model parameters, e.g. bin size and peak threshold.\par&#10;&#10;The second well-known family of ellipse detection methods is edge following, in which the connectivity between edge pixels, convexity of arc segments and geometric constraints are used. The general idea of edge following always starts from computing the binary edge map and corresponding gradients acquired by Canny or Sobel detector \cite{canny1986computational, plataniotis2013color} and then refining the arc segments from the binary edge for the ellipse fitting.\par&#10;&#10;Many of edge following methods use line segments (LSs), which are extracted from the binary edges, as an intermediary to find the arc segments. The approach proposed by Kim et al. \cite{kim2002fast} merges the very short LSs to represent arc segments, where the arc fitting algorithms are frequently called. \cite{mai2008hierarchical} shares similar ideas with \cite{kim2002fast} to extract short LSs from the edge map while the difference lies in linking the LSs as well as the LS's edge points to form arc segments by using simple preset adjacency threshold and proper curvature condition. This method further iteratively groups two arc segments and applies the Random Sample Consensus (RANSAC) to the arc segment groups to recover the ellipse models. Although \cite{mai2008hierarchical} tries to promote the ellipse detector's robustness by iterative grouping and RANSAC, the massive missing detections (FNs) and false positives (FPs) appear.&#10;The method proposed by Chia et al.~\cite{chia2011split} improves the framework illustrated in~\cite{mai2008hierarchical}, but a more complicated fragments merging and grouping procedures were employed. The merging of arc fragments is formalized as an alignment problem, where an alignment function is defined to score the rationality of merging, and a total cost function is built to incrementally search the optimal paired arc segments for grouping. Though the complex and iterative mathematical optimization boosts the detection performance to some extent, \cite{chia2011split} shows slow speed in the real-world images as reported in \cite{prasad2012edge,patraucean2017joint}. The ellipse detector proposed by Prasad et al. \cite{prasad2012edge} incorporates the edge curvature and convexity to extract smooth edge contours and performs a 2D HT to rank the edge contours in a group by the relationship scores for the better generation of ellipse hypotheses. But it also suffers from a long computation time, as shown in Fig.~\ref{fig1}(d).\par&#10;&#10;Another stream of edge following methods tries to extract arc segments from binary edge directly and prunes straight edges for the purpose of fast detection speed. The ellipse detector proposed by Fornaciari et al. \cite{fornaciari2014fast} assigns a bounding box for each arc, removes the straight edges and determines the convexity of the arc by comparing the areas of region under and over the arc. In addition, this method accelerates the detection process by utilizing the property of that ellipse center should be colinear to the midpoints of parallel chords. However, it raises the detection speed at the cost of localization accuracy and robustness (Fig.~\ref{fig1}(e)). Recently, the method presented by Qi et al. \cite{jia2017fast} inherits \cite{fornaciari2014fast} and uses the similar convexity classification approach, but the difference lies in that \cite{jia2017fast} filters straight edges efficiently by calculating the edge connected component's characteristic number, which is a kind of projective invariant being able to distinguish the lines and conic curves within images. \cite{jia2017fast} is fast and yet prone to generate duplicates due to the absence of novel clustering (Fig.~\ref{fig1}(f)). In addition, both \cite{fornaciari2014fast} and \cite{jia2017fast} require at least three arc segments to recover the ellipse model, which might disable the algorithms when handling the incomplete ellipses.\par&#10;&#10;Some researchers generalize the LS detection method to be a multi-functional detector which can jointly detect the LS and elliptic arcs. ELSDc proposed by P{\u{a}}tr{\u{a}}ucean et al. \cite{patraucean2017joint} uses an improved LSD \cite{grompone2010lsd} version for detecting LS, and then iteratively searches the remaining LSs from the start and end points of the detected LS. Eventually, both LS detection and grouping tasks are established simultaneously. Notably, ELSDc stands out other methods by detecting LSs from the greyscale image instead of binary edge such that abundant gradient and geometric cues can be fully exploited.&#10;ELSDc and our proposed method are both based on LSD \cite{grompone2010lsd} for LS detection from the greyscale image, but they are fundamentally different from the generated LS type, ellipse candidates generation and validation strategies. Our method merely generates the arc-support LSs and do not chain them in the LS generation step. Moreover, ELSDc fits and validates the locally grouped LSs, omitting the global situation, which may be prone to produce the false positives (Fig.~\ref{fig1}(g)).\par&#10;&#10;Arc-support LS is our previous work as introduced in \cite{Lu2017Circle}, each pair of which is successfully used for circle detection. However, it cannot handle the ellipse detection scheme since an arbitrary ellipse cannot be determined by two paired LSs. Admittedly, ellipse detection owns much higher complexity and requires more geometric cues. For example, the continuity feature, which is neglected in \cite{Lu2017Circle}, can be fully embodied in the arc-support group and is important in ellipse detection. Therefore, the careful arc-support groups forming, complicated geometric constraints, accurate ellipse generation and clustering, and novel validation strategy accustomed to ellipse detection are required, which will be addressed in this paper.\par&#10;&#10;The main research purpose of this paper is to propose a high-quality ellipse detection method to handle the long-standing issue that cannot detect ellipses both accurately and efficiently in ellipse detection field. To that end, for the first time, we take the advantage of arc-support LSs for ellipse detection. The arc-support groups are formed by robustly linking the consecutive arc-support LSs which share similar geometric properties in point statistics level. Each arc-support group will be measured and assigned a saliency score. Secondly, we generate the initial ellipse set by two complementary approaches both locally and globally. The superposition principle of ellipse fitting and the novel geometric constraints, which are polarity constraint, region restriction and adaptive inliers criterion, are employed to consolidate the proposed method's accuracy and efficiency. Thirdly, we decompose the 5D ellipse parameter space into three subspaces according to ellipse center, orientation and semi-axes and perform three-stage efficient clustering. Finally, the candidates which pass the rigorous and effective verification will be refined by fitting again.\par&#10;&#10;The rest of this paper is organized as follows. Section \ref{sec:Preliminary} introduces the preliminaries about arc-support LS and superposition principle of ellipse fitting. Section \ref{sec:ellipse detection} presents the high-quality ellipse detection framework, as a four-stage detection procedure: arc-support groups forming, initial ellipse set generation, clustering, and candidate verification. Section \ref{sec:complexity} analyzes the computation complexity of the proposed ellipse detection algorithm. Experimental results, as well as the accuracy and efficiency detection performance of the proposed method, are detailed in Section \ref{sec:experimental results}. Section \ref{sec:conclusion} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Computer Vision Applications, Ellipse Detection Algorithms, Mathematics, Image Processing, Computer Science, Pattern Recognition, Image Processing Techniques" />
-        </attvalues>
-      </node>
-      <node id="1608.07470" label="1608.07470">
-        <attvalues>
-          <attvalue for="0" value="A Fast Ellipse Detector Using Projective Invariant Pruning" />
-          <attvalue for="1" value="  Detecting elliptical objects from an image is a central task in robot&#10;navigation and industrial diagnosis where the detection time is always a&#10;critical issue. Existing methods are hardly applicable to these real-time&#10;scenarios of limited hardware resource due to the huge number of fragment&#10;candidates (edges or arcs) for fitting ellipse equations. In this paper, we&#10;present a fast algorithm detecting ellipses with high accuracy. The algorithm&#10;leverage a newly developed projective invariant to significantly prune the&#10;undesired candidates and to pick out elliptical ones. The invariant is able to&#10;reflect the intrinsic geometry of a planar curve, giving the value of -1 on any&#10;three collinear points and +1 for any six points on an ellipse. Thus, we apply&#10;the pruning and picking by simply comparing these binary values. Moreover, the&#10;calculation of the invariant only involves the determinant of a 3*3 matrix.&#10;Extensive experiments on three challenging data sets with 650 images&#10;demonstrate that our detector runs 20%-50% faster than the state-of-the-art&#10;algorithms with the comparable or higher precision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.11572" label="2403.11572">
-        <attvalues>
-          <attvalue for="0" value="Augment Before Copy-Paste: Data and Memory Efficiency-Oriented Instance&#10;  Segmentation Framework for Sport-scenes" />
-          <attvalue for="1" value="  Instance segmentation is a fundamental task in computer vision with broad&#10;applications across various industries. In recent years, with the proliferation&#10;of deep learning and artificial intelligence applications, how to train&#10;effective models with limited data has become a pressing issue for both&#10;academia and industry. In the Visual Inductive Priors challenge (VIPriors2023),&#10;participants must train a model capable of precisely locating individuals on a&#10;basketball court, all while working with limited data and without the use of&#10;transfer learning or pre-trained models. We propose Memory effIciency inStance&#10;Segmentation framework based on visual inductive prior flow propagation that&#10;effectively incorporates inherent prior information from the dataset into both&#10;the data preprocessing and data augmentation stages, as well as the inference&#10;phase. Our team (ACVLAB) experiments demonstrate that our model achieves&#10;promising performance (0.509 AP@0.50:0.95) even under limited data and memory&#10;constraints.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Instance segmentation, a cornerstone of computer vision within deep learning, boasts diverse applications like pedestrian detection and multi-object tracking. With the rise of deep learning, industries are integrating it into various aspects of their businesses to gain a competitive edge. However, implementing deep learning at these granular levels faces challenges such as insufficient annotated data and limited computational resources. These constraints can lead to models performing below expectations. Therefore, effectively leveraging limited data to train models has become one of today's research hotspots. The 'VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning' workshop \cite{VIP2th}\cite{VIP3th} introduces a challenge, urging participants to build generalizable models by fusing dataset-specific prior knowledge in resource-scarce contexts. Notably, the use of pre-trained models is disallowed.&#10;&#10;In the instance segmentation track, the goal is to predict players, basketballs, referees and coaches in a basketball court. Task-specific data augmentation based on instance traits yields substantial performance gains, validated in the challenge. Nevertheless, conventional state-of-the-art\cite{2021sota}\cite{2022sota} prioritize performance without due regard for computational resources. We propose a &quot;augmentation before copy-paste&quot; pipeline with RGB distortion and geometry transformation before copy-paste augmentation, exploiting object-derived semantic representation effectively.&#10;&#10;Furthermore, we remove redundant image area to find out a region of interests. Utilizing prior knowledge from basketball court backgrounds—often defined by rectangular boundary lines, we extract court areas for training and inference on smaller images to minimize the computational consumption without feature distortion.&#10;&#10;In recent years, deep learning-based instance segmentation methods have garnered widespread recognition, exemplified by approaches such as Cascade-MaskRCNN\cite{CascadeRCNN}, Maskformer\cite{Maskformer}, queryinst\cite{QueryInst}, and CBNet\cite{CBNet}, among others. Over the past couple of years, \cite{2021sota}\cite{2022sota}. have demonstrated remarkable achievements within the VIPriors Workshop through the CBNet-based model architectures.&#10;&#10;CBNet is distinguished by its capability to synergistically couple multiple backbone networks and detectors, enabling the effective fusion of both low-level and high-level semantic representations. This architecture, characterized by its scalability and ease of training, imparts a diverse range of model inductive attributes. It accomplishes this while maintaining noteworthy detection precision and generalization ability without compromising inference speed.&#10;&#10;Data augmentation constitutes a critical facet in training deep learning models, especially when confronted with limited data. By subjecting original images to operations such as hue variations, geometric transformations, and erasure transformations, the diversity of features is augmented, enhancing the model's capacity for generalization across unseen domains.&#10;&#10;Copy-paste augmentation\cite{CopyPaste} emerges as an effective augmentation strategy for instance segmentation task. It leverages prior knowledge of objects to enhance a model's generalization and robustness to out-of-domain objects. In VIPriors instance segmentation challenge, \cite{2021sota}\cite{2022sota} introduced task-specific copy-paste augmentation. This procedure leverages object-derived visual prior knowledge to optimize the data pipeline, ensuring that the features of pasted objects align more closely with plausible scenarios.&#10;For instance, constraints can be applied to the coordinates of pasted images based on the probable player positions, effectively filtering out implausible results.&#10;&#10;The experimental results confirm that our model can be effectively trained on a single GPU with 24 GB of memory while maintaining promising performance.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Data Efficiency, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2201.08625" label="2201.08625">
-        <attvalues>
-          <attvalue for="0" value="VIPriors 2: Visual Inductive Priors for Data-Efficient Deep Learning&#10;  Challenges" />
-          <attvalue for="1" value="  The second edition of the &quot;VIPriors: Visual Inductive Priors for&#10;Data-Efficient Deep Learning&quot; challenges featured five data-impaired&#10;challenges, where models are trained from scratch on a reduced number of&#10;training samples for various key computer vision tasks. To encourage new and&#10;creative ideas on incorporating relevant inductive biases to improve the data&#10;efficiency of deep learning models, we prohibited the use of pre-trained&#10;checkpoints and other transfer learning techniques. The provided baselines are&#10;outperformed by a large margin in all five challenges, mainly thanks to&#10;extensive data augmentation policies, model ensembling, and data efficient&#10;network architectures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.19688" label="2305.19688">
-        <attvalues>
-          <attvalue for="0" value="VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning&#10;  Challenges" />
-          <attvalue for="1" value="  The third edition of the &quot;VIPriors: Visual Inductive Priors for&#10;Data-Efficient Deep Learning&quot; workshop featured four data-impaired challenges,&#10;focusing on addressing the limitations of data availability in training deep&#10;learning models for computer vision tasks. The challenges comprised of four&#10;distinct data-impaired tasks, where participants were required to train models&#10;from scratch using a reduced number of training samples. The primary objective&#10;was to encourage novel approaches that incorporate relevant inductive biases to&#10;enhance the data efficiency of deep learning models. To foster creativity and&#10;exploration, participants were strictly prohibited from utilizing pre-trained&#10;checkpoints and other transfer learning techniques. Significant advancements&#10;were made compared to the provided baselines, where winning solutions surpassed&#10;the baselines by a considerable margin in all four tasks. These achievements&#10;were primarily attributed to the effective utilization of extensive data&#10;augmentation policies, model ensembling techniques, and the implementation of&#10;data-efficient training methods, including self-supervised representation&#10;learning. This report highlights the key aspects of the challenges and their&#10;outcomes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.00470" label="2110.00470">
-        <attvalues>
-          <attvalue for="0" value="Instance Segmentation Challenge Track Technical Report, VIPriors&#10;  Workshop at ICCV 2021: Task-Specific Copy-Paste Data Augmentation Method for&#10;  Instance Segmentation" />
-          <attvalue for="1" value="  Copy-Paste has proven to be a very effective data augmentation for instance&#10;segmentation which can improve the generalization of the model. We used a&#10;task-specific Copy-Paste data augmentation method to achieve good performance&#10;on the instance segmentation track of the 2nd VIPriors workshop challenge. We&#10;also applied additional data augmentation techniques including RandAugment and&#10;GridMask. Our segmentation model is the HTC detector on the CBSwin-B with CBFPN&#10;with some tweaks. This model was trained at the multi-scale mode by a random&#10;sampler on the 6x schedule and tested at the single-scale mode. By combining&#10;these techniques, we achieved 0.398 AP@0.50:0.95 with the validation set and&#10;0.433 AP@0.50:0.95 with the test set. Finally, we reached 0.477 AP@0.50:0.95&#10;with the test set by adding the validation set to the training data. Source&#10;code is available at https://github.com/jahongir7174/VIP2021.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.11282" label="2211.11282">
-        <attvalues>
-          <attvalue for="0" value="Task-Specific Data Augmentation and Inference Processing for VIPriors&#10;  Instance Segmentation Challenge" />
-          <attvalue for="1" value="  Instance segmentation is applied widely in image editing, image analysis and&#10;autonomous driving, etc. However, insufficient data is a common problem in&#10;practical applications. The Visual Inductive Priors(VIPriors) Instance&#10;Segmentation Challenge has focused on this problem. VIPriors for Data-Efficient&#10;Computer Vision Challenges ask competitors to train models from scratch in a&#10;data-deficient setting, but there are some visual inductive priors that can be&#10;used. In order to address the VIPriors instance segmentation problem, we&#10;designed a Task-Specific Data Augmentation(TS-DA) strategy and Inference&#10;Processing(TS-IP) strategy. The main purpose of task-specific data augmentation&#10;strategy is to tackle the data-deficient problem. And in order to make the most&#10;of visual inductive priors, we designed a task-specific inference processing&#10;strategy. We demonstrate the applicability of proposed method on VIPriors&#10;Instance Segmentation Challenge. The segmentation model applied is Hybrid Task&#10;Cascade based detector on the Swin-Base based CBNetV2 backbone. Experimental&#10;results demonstrate that proposed method can achieve a competitive result on&#10;the test set of 2022 VIPriors Instance Segmentation Challenge, with 0.531&#10;AP@0.50:0.95.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.06278" label="2107.06278">
-        <attvalues>
-          <attvalue for="0" value="Per-Pixel Classification is Not All You Need for Semantic Segmentation" />
-          <attvalue for="1" value="  Modern approaches typically formulate semantic segmentation as a per-pixel&#10;classification task, while instance-level segmentation is handled with an&#10;alternative mask classification. Our key insight: mask classification is&#10;sufficiently general to solve both semantic- and instance-level segmentation&#10;tasks in a unified manner using the exact same model, loss, and training&#10;procedure. Following this observation, we propose MaskFormer, a simple mask&#10;classification model which predicts a set of binary masks, each associated with&#10;a single global class label prediction. Overall, the proposed mask&#10;classification-based method simplifies the landscape of effective approaches to&#10;semantic and panoptic segmentation tasks and shows excellent empirical results.&#10;In particular, we observe that MaskFormer outperforms per-pixel classification&#10;baselines when the number of classes is large. Our mask classification-based&#10;method outperforms both current state-of-the-art semantic (55.6 mIoU on ADE20K)&#10;and panoptic segmentation (52.7 PQ on COCO) models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.01928" label="2105.01928">
-        <attvalues>
-          <attvalue for="0" value="Instances as Queries" />
-          <attvalue for="1" value="  Recently, query based object detection frameworks achieve comparable&#10;performance with previous state-of-the-art object detectors. However, how to&#10;fully leverage such frameworks to perform instance segmentation remains an open&#10;problem. In this paper, we present QueryInst (Instances as Queries), a query&#10;based instance segmentation method driven by parallel supervision on dynamic&#10;mask heads. The key insight of QueryInst is to leverage the intrinsic&#10;one-to-one correspondence in object queries across different stages, as well as&#10;one-to-one correspondence between mask RoI features and object queries in the&#10;same stage. This approach eliminates the explicit multi-stage mask head&#10;connection and the proposal distribution inconsistency issues inherent in&#10;non-query based multi-stage instance segmentation methods. We conduct extensive&#10;experiments on three challenging benchmarks, i.e., COCO, CityScapes, and&#10;YouTube-VIS to evaluate the effectiveness of QueryInst in instance segmentation&#10;and video instance segmentation (VIS) task. Specifically, using ResNet-101-FPN&#10;backbone, QueryInst obtains 48.1 box AP and 42.8 mask AP on COCO test-dev,&#10;which is 2 points higher than HTC in terms of both box AP and mask AP, while&#10;runs 2.4 times faster. For video instance segmentation, QueryInst achieves the&#10;best performance among all online VIS approaches and strikes a decent&#10;speed-accuracy trade-off. Code is available at&#10;\url{https://github.com/hustvl/QueryInst}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Query Based Methods.&#10;Recently, query based methods emerged to tackle the set-prediction problems. &#10;Concretely, DETR~\cite{DETR} first introduces the query based methods with transformer architecture to object detection.&#10;Deformable DETR~\cite{DefDETR}, UP-DETR~\cite{UP-DETR}, ACT~\cite{ACT} and TSP~\cite{TSP} improve the performance on the top of DETR.&#10;The recently proposed Sparse R-CNN~\cite{SparseRCNN} builds a query based set-prediction framework upon R-CNN~\cite{RCNN,FastRCNN,FasterRCNN} based detector. &#10;For segmentation, VisTR~\cite{VisTR} introduces a query based sequence matching and segmentation method to video instance segmentation, building a fully end-to-end framework for instance segmentation in video. Max-DeepLab\cite{MaxDeepLab} presents the first box-free end-to-end panoptic segmentation model with a global memory as external query.&#10;Trackformer~\cite{Trackformer} and Transtrack~\cite{Transtrack} build a query based multiple object tracktor upon DETR and Deformable DETR, respectively, and attain comparable results to the non-query based methods. &#10;AS-Net~\cite{AS-Net} introduces a query based set-prediction pipeline to human object interaction and obtains promising results. &#10;Despite query based set-prediction method is being widely used to many computer vision tasks, few efforts are conducted to build a successful query based instance segmentation framework. &#10;We aim to achieve this goal in this paper.&#10;&#10;Object Detection.&#10;Object detection is a fundamental computer vision task which aims to detect visual objects with bounding boxes.&#10;With the propose of R-CNN~\cite{RCNN}, Fast R-CNN~\cite{FastRCNN} and Faster R-CNN~\cite{FasterRCNN}, anchor based methods \cite{CascadeRCNN, LibraRCNN, GridRCNN, RetinaNet, SSD} dominate object detection for a long period. &#10;CenterNet~\cite{CenterNet} and FCOS~\cite{FCOSpro} establish anchor-free detectors with competitive detection performance.&#10;Recently, with the proposed DETR~\cite{DETR}, query based set-prediction methods catch lots of attentions.&#10;Deformable DETR~\cite{DefDETR} introduces deformable convolution~\cite{DCNv2} to the DETR framework, achieving better performance with faster training convergence. &#10;UP-DETR~\cite{UP-DETR} extends DETR to unsupervised scenarios.&#10;ACT~\cite{ACT} and TSP~\cite{TSP} introduce the adaptive clustering module and a new bipartite matching method to DETR.&#10;Sparse R-CNN~\cite{SparseRCNN} build a query based detector on top of R-CNN architecture, while OneNet~\cite{OneNet} and DeFCN~\cite{DeFCN} are end-to-end detector built upon the one-stage FCOS~\cite{FCOSpro}. &#10;In this work, we present a query based instance segmentation method on the top of the query based Sparse R-CNN detector.&#10;&#10;Instance Segmentation.&#10;&#10;Instance segmentation is a fundamental yet challenging computer vision task that requires an algorithm to assign a pixel-level mask with a category label for each instance of interest in image.&#10;Mask R-CNN~\cite{MaskRCNN} introduces a fully convolutional mask head to Faster R-CNN~\cite{FastRCNN} detector. &#10;Casacde Mask R-CNN~\cite{CascadeRCNN_TPAMI} simply combine the Casacde R-CNN~\cite{CascadeRCNN} with Mask R-CNN. HTC~\cite{HTC} presents interleaved execution and mask information flow and achieves state-of-the-art performance. &#10;In addition to R-CNN based methods, YOLACT~\cite{YOLACT, YOLACTpp}, SipMask~\cite{SipMask}, CondInst~\cite{CondInst_TPAMI} and SOLO~\cite{SOLO, SOLOv2} build one-stage instance segmentation framework on the top of one-stage framework, achieving comparable results with favorable inference speed.&#10;Following the R-CNN based methods, we present a query based instance segmentation framework.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2107.00420" label="2107.00420">
-        <attvalues>
-          <attvalue for="0" value="CBNet: A Composite Backbone Network Architecture for Object Detection" />
-          <attvalue for="1" value="  Modern top-performing object detectors depend heavily on backbone networks,&#10;whose advances bring consistent performance gains through exploring more&#10;effective network structures. In this paper, we propose a novel and flexible&#10;backbone framework, namely CBNetV2, to construct high-performance detectors&#10;using existing open-sourced pre-trained backbones under the pre-training&#10;fine-tuning paradigm. In particular, CBNetV2 architecture groups multiple&#10;identical backbones, which are connected through composite connections.&#10;Specifically, it integrates the high- and low-level features of multiple&#10;backbone networks and gradually expands the receptive field to more efficiently&#10;perform object detection. We also propose a better training strategy with&#10;assistant supervision for CBNet-based detectors. Without additional&#10;pre-training of the composite backbone, CBNetV2 can be adapted to various&#10;backbones (CNN-based vs. Transformer-based) and head designs of most mainstream&#10;detectors (one-stage vs. two-stage, anchor-based vs. anchor-free-based).&#10;Experiments provide strong evidence that, compared with simply increasing the&#10;depth and width of the network, CBNetV2 introduces a more efficient, effective,&#10;and resource-friendly way to build high-performance backbone networks.&#10;Particularly, our Dual-Swin-L achieves 59.4% box AP and 51.6% mask AP on COCO&#10;test-dev under the single-model and single-scale testing protocol, which is&#10;significantly better than the state-of-the-art result (57.7% box AP and 50.2%&#10;mask AP) achieved by Swin-L, while the training schedule is reduced by&#10;6$\times$. With multi-scale testing, we push the current best single model&#10;result to a new record of 60.1% box AP and 52.3% mask AP without using extra&#10;training data. Code is available at https://github.com/VDIGPKU/CBNetV2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.07177" label="2012.07177">
-        <attvalues>
-          <attvalue for="0" value="Simple Copy-Paste is a Strong Data Augmentation Method for Instance&#10;  Segmentation" />
-          <attvalue for="1" value="  Building instance segmentation models that are data-efficient and can handle&#10;rare object categories is an important challenge in computer vision. Leveraging&#10;data augmentations is a promising direction towards addressing this challenge.&#10;Here, we perform a systematic study of the Copy-Paste augmentation ([13, 12])&#10;for instance segmentation where we randomly paste objects onto an image. Prior&#10;studies on Copy-Paste relied on modeling the surrounding visual context for&#10;pasting the objects. However, we find that the simple mechanism of pasting&#10;objects randomly is good enough and can provide solid gains on top of strong&#10;baselines. Furthermore, we show Copy-Paste is additive with semi-supervised&#10;methods that leverage extra data through pseudo labeling (e.g. self-training).&#10;On COCO instance segmentation, we achieve 49.1 mask AP and 57.3 box AP, an&#10;improvement of +0.6 mask AP and +1.5 box AP over the previous state-of-the-art.&#10;We further demonstrate that Copy-Paste can lead to significant improvements on&#10;the LVIS benchmark. Our baseline model outperforms the LVIS 2020 Challenge&#10;winning entry by +3.6 mask AP on rare categories.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.03535" label="1806.03535">
-        <attvalues>
-          <attvalue for="0" value="Cell Detection with Star-convex Polygons" />
-          <attvalue for="1" value="  Automatic detection and segmentation of cells and nuclei in microscopy images&#10;is important for many biological applications. Recent successful learning-based&#10;approaches include per-pixel cell segmentation with subsequent pixel grouping,&#10;or localization of bounding boxes with subsequent shape refinement. In&#10;situations of crowded cells, these can be prone to segmentation errors, such as&#10;falsely merging bordering cells or suppressing valid cell instances due to the&#10;poor approximation with bounding boxes. To overcome these issues, we propose to&#10;localize cell nuclei via star-convex polygons, which are a much better shape&#10;representation as compared to bounding boxes and thus do not need shape&#10;refinement. To that end, we train a convolutional neural network that predicts&#10;for every pixel a polygon for the cell instance at that position. We&#10;demonstrate the merits of our approach on two synthetic datasets and one&#10;challenging dataset of diverse fluorescence microscopy images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.02677" label="1604.02677">
-        <attvalues>
-          <attvalue for="0" value="DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation" />
-          <attvalue for="1" value="  The morphology of glands has been used routinely by pathologists to assess&#10;the malignancy degree of adenocarcinomas. Accurate segmentation of glands from&#10;histology images is a crucial step to obtain reliable morphological statistics&#10;for quantitative diagnosis. In this paper, we proposed an efficient deep&#10;contour-aware network (DCAN) to solve this challenging problem under a unified&#10;multi-task learning framework. In the proposed network, multi-level contextual&#10;features from the hierarchical architecture are explored with auxiliary&#10;supervision for accurate gland segmentation. When incorporated with multi-task&#10;regularization during the training, the discriminative capability of&#10;intermediate features can be further improved. Moreover, our network can not&#10;only output accurate probability maps of glands, but also depict clear contours&#10;simultaneously for separating clustered objects, which further boosts the gland&#10;segmentation performance. This unified framework can be efficient when applied&#10;to large-scale histopathological data without resorting to additional steps to&#10;generate contours based on low-level cues for post-separating. Our method won&#10;the 2015 MICCAI Gland Segmentation Challenge out of 13 competitive teams,&#10;surpassing all the other methods by a significant margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.01616" label="1909.01616">
-        <attvalues>
-          <attvalue for="0" value="SSAP: Single-Shot Instance Segmentation With Affinity Pyramid" />
-          <attvalue for="1" value="  Recently, proposal-free instance segmentation has received increasing&#10;attention due to its concise and efficient pipeline. Generally, proposal-free&#10;methods generate instance-agnostic semantic segmentation labels and&#10;instance-aware features to group pixels into different object instances.&#10;However, previous methods mostly employ separate modules for these two&#10;sub-tasks and require multiple passes for inference. We argue that treating&#10;these two sub-tasks separately is suboptimal. In fact, employing multiple&#10;separate modules significantly reduces the potential for application. The&#10;mutual benefits between the two complementary sub-tasks are also unexplored. To&#10;this end, this work proposes a single-shot proposal-free instance segmentation&#10;method that requires only one single pass for prediction. Our method is based&#10;on a pixel-pair affinity pyramid, which computes the probability that two&#10;pixels belong to the same instance in a hierarchical manner. The affinity&#10;pyramid can also be jointly learned with the semantic class labeling and&#10;achieve mutual benefits. Moreover, incorporating with the learned affinity&#10;pyramid, a novel cascaded graph partition module is presented to sequentially&#10;generate instances from coarse to fine. Unlike previous time-consuming graph&#10;partition methods, this module achieves $5\times$ speedup and 9% relative&#10;improvement on Average-Precision (AP). Our approach achieves state-of-the-art&#10;results on the challenging Cityscapes dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.08303" label="1611.08303">
-        <attvalues>
-          <attvalue for="0" value="Deep Watershed Transform for Instance Segmentation" />
-          <attvalue for="1" value="  Most contemporary approaches to instance segmentation use complex pipelines&#10;involving conditional random fields, recurrent neural networks, object&#10;proposals, or template matching schemes. In our paper, we present a simple yet&#10;powerful end-to-end convolutional neural network to tackle this task. Our&#10;approach combines intuitions from the classical watershed transform and modern&#10;deep learning to produce an energy map of the image where object instances are&#10;unambiguously represented as basins in the energy map. We then perform a cut at&#10;a single energy level to directly yield connected components corresponding to&#10;object instances. Our model more than doubles the performance of the&#10;state-of-the-art on the challenging Cityscapes Instance Level Segmentation&#10;task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.09821" label="2004.09821">
-        <attvalues>
-          <attvalue for="0" value="Instance Segmentation of Biomedical Images with an Object-aware&#10;  Embedding Learned with Local Constraints" />
-          <attvalue for="1" value="  Automatic instance segmentation is a problem that occurs in many biomedical&#10;applications. State-of-the-art approaches either perform semantic segmentation&#10;or refine object bounding boxes obtained from detection methods. Both suffer&#10;from crowded objects to varying degrees, merging adjacent objects or&#10;suppressing a valid object. In this work, we assign an embedding vector to each&#10;pixel through a deep neural network. The network is trained to output embedding&#10;vectors of similar directions for pixels from the same object, while adjacent&#10;objects are orthogonal in the embedding space, which effectively avoids the&#10;fusion of objects in a crowd. Our method yields state-of-the-art results even&#10;with a light-weighted backbone network on a cell segmentation (BBBC006 +&#10;DSB2018) and a leaf segmentation data set (CVPPP2017). The code and model&#10;weights are public available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.08273" label="1712.08273">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Pixel Embedding for Instance Grouping" />
-          <attvalue for="1" value="  We introduce a differentiable, end-to-end trainable framework for solving&#10;pixel-level grouping problems such as instance segmentation consisting of two&#10;novel components. First, we regress pixels into a hyper-spherical embedding&#10;space so that pixels from the same group have high cosine similarity while&#10;those from different groups have similarity below a specified margin. We&#10;analyze the choice of embedding dimension and margin, relating them to&#10;theoretical results on the problem of distributing points uniformly on the&#10;sphere. Second, to group instances, we utilize a variant of mean-shift&#10;clustering, implemented as a recurrent neural network parameterized by kernel&#10;bandwidth. This recurrent grouping module is differentiable, enjoys convergent&#10;dynamics and probabilistic interpretability. Backpropagating the group-weighted&#10;loss through this module allows learning to focus on only correcting embedding&#10;errors that won't be resolved during subsequent clustering. Our framework,&#10;while conceptually simple and theoretically abundant, is also practically&#10;effective and computationally efficient. We demonstrate substantial&#10;improvements over state-of-the-art instance segmentation for object proposal&#10;generation, as well as demonstrating the benefits of grouping loss for&#10;classification tasks such as boundary detection and semantic segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.06660" label="2007.06660">
-        <attvalues>
-          <attvalue for="0" value="Improving Pixel Embedding Learning through Intermediate Distance&#10;  Regression Supervision for Instance Segmentation" />
-          <attvalue for="1" value="  As a proposal-free approach, instance segmentation through pixel embedding&#10;learning and clustering is gaining more emphasis. Compared with bounding box&#10;refinement approaches, such as Mask R-CNN, it has potential advantages in&#10;handling complex shapes and dense objects. In this work, we propose a simple,&#10;yet highly effective, architecture for object-aware embedding learning. A&#10;distance regression module is incorporated into our architecture to generate&#10;seeds for fast clustering. At the same time, we show that the features learned&#10;by the distance regression module are able to promote the accuracy of learned&#10;object-aware embeddings significantly. By simply concatenating features of the&#10;distance regression module to the images as inputs of the embedding module, the&#10;mSBD scores on the CVPPP Leaf Segmentation Challenge can be further improved by&#10;more than 8% compared to the identical set-up without concatenation, yielding&#10;the best overall result amongst the leaderboard at CodaLab.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.02690" label="2103.02690">
-        <attvalues>
-          <attvalue for="0" value="A comprehensive survey on point cloud registration" />
-          <attvalue for="1" value="  Registration is a transformation estimation problem between two point clouds,&#10;which has a unique and critical role in numerous computer vision applications.&#10;The developments of optimization-based methods and deep learning methods have&#10;improved registration robustness and efficiency. Recently, the combinations of&#10;optimization-based and deep learning methods have further improved performance.&#10;However, the connections between optimization-based and deep learning methods&#10;are still unclear. Moreover, with the recent development of 3D sensors and 3D&#10;reconstruction techniques, a new research direction emerges to align&#10;cross-source point clouds. This survey conducts a comprehensive survey,&#10;including both same-source and cross-source registration methods, and summarize&#10;the connections between optimization-based and deep learning methods, to&#10;provide further research insight. This survey also builds a new benchmark to&#10;evaluate the state-of-the-art registration algorithms in solving cross-source&#10;challenges. Besides, this survey summarizes the benchmark data sets and&#10;discusses point cloud registration applications across various domains.&#10;Finally, this survey proposes potential research directions in this rapidly&#10;growing field.&#10;" />
-          <attvalue for="2" value="&#10;Point cloud has become the primary data format to represent the 3D world as the fast development of high precision sensors such as LiDAR and Kinect. Because the sensors can only capture scans within their limited view range, the registration algorithm is required to generate a large 3D scene. Point cloud registration is a problem to estimate the transformation matrix between two-point cloud scans. Applying the transformation matrices, we can merge the partial scans about the same 3D scene or object into a complete 3D point cloud. &#10;&#10;The value of point cloud registration is its unique and critical role in numerous computer vision applications. Firstly, 3D reconstruction. Generating a complete 3D scene is a basic and significant technique for various computer vision applications, including high-precision 3D map reconstruction in autonomous driving, 3D environment reconstruction in robotics and 3D reconstruction for real-time monitoring underground mining. For example, registration could construct the 3D environment for route plan and decision-making in robotics applications. Another example could be a large 3D scene reconstruction in the underground mining space to monitor mining safety accurately. Secondly, 3D localization. Locating the position of the agent in the 3D environment is particularly important for robotics. For example, a driverless car estimates its position on the map (e.g. $&lt;10cm$) and its distance to the road's boundary line. Point cloud registration could accurately match a current real-time 3D view to its belonging 3D environment to provide a high-precision localization service. This application shows that the registration provides a solution to interact with the 3D environment for an autonomous agent (e.g. robots or drive-less car). Thirdly, pose estimation. Aligning a point cloud $A$ (3D real-time view) to another point cloud $B$ (the 3D environment) could generate the pose information of point cloud $A$ related to point cloud $B$. The pose information could be used for decision-making in robotics. For example, the registration could get the robotics arm's pose information to decide where to move to grab an object accurately. The pose estimation application shows that the registration also provides a solution to know the environment's agent information. Since point cloud registration plays a critical role in numerous valuable computer vision applications, there is a significant urgent need to conduct a comprehensive survey of the point cloud registration to benefit these applications.&#10;&#10;The registration problem has endured thorough investigation from optimization aspects \cite{Bernard_2018_CVPR,bes1992method,dym2017ds++,forstner2017efficient,iglesias2020global,kezurer2015tight,le2019sdrsac,segal2009generalized,yang2019polynomial}. Most of the existing registration methods are formulated by minimizing a geometric projection error through two processes: correspondence searching and transformation estimation. These two processes alternatively conduct until the geometric projection error is minimum. Upon the accurate correspondences known, the transformation estimation has a close-form solution \cite{bes1992method}. &#10;&#10;Recently, there are many development in 3D deep learning techniques \cite{3dmatch,deng20193d,choy2019fully,yang2020learning,valsesia2020learning}. These techniques aim to extract distinctive features for 3D points and find accurate correspondences. Then, these correspondences are used to estimate a transformation with a separate transformation estimation stage. &#10;There is also some combination of conventional registration optimization strategies and deep learning techniques in an end-to-end framework \cite{huang2020feature,choy2020deep,aoki2019pointnetlk,wang2019deep}. Their experiments show a significant performance gain. However, the connections between optimization-based and deep learning methods are still unclear.&#10;&#10;Moreover, there is an emerging topic about cross-source point cloud registration with the development of 3D sensors, such as Kinect and Lidar. Each 3D sensor has its distinct advantages and limitations. For example, Kinect can generate dense point clouds, while the view range is usually limited to 5 meters. Lidar has a long view range while generating sparse point clouds. Data fusion of these different kinds of 3D sensors combines their advantages and is a cross-source point cloud registration problem \cite{huang2017coarse,huang2017systematic,huang2016coarse}. The cross-source point cloud registration has wide applications such as building construction, augmented reality, and driverless vehicles. For example, the builders compare the 3D CAD model with real-time LiDAR scans to evaluate the contract's current construction quality. The development in both same-source and cross-source point cloud registration also requires a comprehensive survey to summarize the recent advances.&#10;&#10;Although there are a few existing reviews on point cloud registration \cite{cheng2018registration,pomerleau2015review,saiti2020application}, they mainly focus on the view of conventional point cloud registration. \cite{zhang2020deep} surveys deep learning techniques. However, the recent development of cross-source point cloud registration has not been surveyed, and the connections between conventional optimization and recent deep learning methods are unclear. To stimulate point cloud registration development in industrial and academic, we conduct a comprehensive survey by summarizing the recent fast development of point cloud registration (1992-2021), including both same-source and cross-source, conventional optimization and current deep learning methods. Moreover, we summarize the connections between optimization strategies and deep learning techniques. &#10;&#10;Besides, while the recent deep learning-based registration techniques achieve high accuracy on same-source point cloud databases, cross-source point clouds' performance is less reported. This survey will build a benchmark to evaluate the recent state-of-the-art registration algorithms on a cross-source dataset.&#10;&#10;Our contributions. Our paper makes notable contributions summarized as follows:&#10;&#10;\begin{itemize}&#10;&#09;\item Comprehensive review. We provide the most comprehensive overview for same-source point cloud registration, including conventional optimization and modern deep learning methods (1992-2021). We summarize the challenges, analyze the advantages and limitations of each category of registration methods. Moreover, the connections between conventional optimization and modern deep learning methods are summarized in this paper. These connections could provide insights for future research.&#10;&#09;\item Review of cross-source registration. For the first time, we provide a literature review about cross-source point cloud registration. This survey provides insights for data fusion research from different 3D sensors (e.g., Kinect and Lidar). Figure \ref{f_category} shows a taxonomy of point cloud registration.&#10;&#09;\item New comparison. We build a novel cross-source point cloud benchmark. Then, the existing state-of-the-art registration algorithms' performance is evaluated and compared on the new cross-source point cloud benchmark. This survey can provide a guide for choosing and developing new registration approaches for cross-source point cloud applications.&#10;&#09;\item Applications and future directions. We summarize the potential applications of point cloud registration and explore the research directions in real applications. Besides, we suggest possible future research directions and open questions in the point cloud registration field.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Deep Learning Methods, Computer Science, Point Cloud Registration" />
-        </attvalues>
-      </node>
-      <node id="1711.10733" label="1711.10733">
-        <attvalues>
-          <attvalue for="0" value="DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching&#10;  Problems" />
-          <attvalue for="1" value="  In this work we study convex relaxations of quadratic optimisation problems&#10;over permutation matrices. While existing semidefinite programming approaches&#10;can achieve remarkably tight relaxations, they have the strong disadvantage&#10;that they lift the original $n {\times} n$-dimensional variable to an $n^2&#10;{\times} n^2$-dimensional variable, which limits their practical applicability.&#10;In contrast, here we present a lifting-free convex relaxation that is provably&#10;at least as tight as existing (lifting-free) convex relaxations. We demonstrate&#10;experimentally that our approach is superior to existing convex and non-convex&#10;methods for various problems, including image arrangement and multi-graph&#10;matching.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.06148" label="1705.06148">
-        <attvalues>
-          <attvalue for="0" value="DS++: A flexible, scalable and provably tight relaxation for matching&#10;  problems" />
-          <attvalue for="1" value="  Correspondence problems are often modelled as quadratic optimization problems&#10;over permutations. Common scalable methods for approximating solutions of these&#10;NP-hard problems are the spectral relaxation for non-convex energies and the&#10;doubly stochastic (DS) relaxation for convex energies. Lately, it has been&#10;demonstrated that semidefinite programming relaxations can have considerably&#10;improved accuracy at the price of a much higher computational cost. We present&#10;a convex quadratic programming relaxation which is provably stronger than both&#10;DS and spectral relaxations, with the same scalability as the DS relaxation.&#10;The derivation of the relaxation also naturally suggests a projection method&#10;for achieving meaningful integer solutions which improves upon the standard&#10;closest-permutation projection. Our method can be easily extended to&#10;optimization over doubly stochastic matrices, partial or injective matching,&#10;and problems with additional linear constraints. We employ recent advances in&#10;optimization of linear-assignment type problems to achieve an efficient&#10;algorithm for solving the convex relaxation.&#10;  We present experiments indicating that our method is more accurate than local&#10;minimization or competing relaxations for non-convex problems. We successfully&#10;apply our algorithm to shape matching and to the problem of ordering images in&#10;a grid, obtaining results which compare favorably with state of the art&#10;methods. We believe our results indicate that our method should be considered&#10;the method of choice for quadratic optimization over permutations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03483" label="1904.03483">
-        <attvalues>
-          <attvalue for="0" value="SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud&#10;  Registration without Correspondences" />
-          <attvalue for="1" value="  This paper presents a novel randomized algorithm for robust point cloud&#10;registration without correspondences. Most existing registration approaches&#10;require a set of putative correspondences obtained by extracting invariant&#10;descriptors. However, such descriptors could become unreliable in noisy and&#10;contaminated settings. In these settings, methods that directly handle input&#10;point sets are preferable. Without correspondences, however, conventional&#10;randomized techniques require a very large number of samples in order to reach&#10;satisfactory solutions. In this paper, we propose a novel approach to address&#10;this problem. In particular, our work enables the use of randomized methods for&#10;point cloud registration without the need of putative correspondences. By&#10;considering point cloud alignment as a special instance of graph matching and&#10;employing an efficient semi-definite relaxation, we propose a novel sampling&#10;mechanism, in which the size of the sampled subsets can be larger-than-minimal.&#10;Our tight relaxation scheme enables fast rejection of the outliers in the&#10;sampled sets, resulting in high-quality hypotheses. We conduct extensive&#10;experiments to demonstrate that our approach outperforms other state-of-the-art&#10;methods. Importantly, our proposed method serves as a generic framework which&#10;can be extended to problems with known correspondences.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.08588" label="1903.08588">
-        <attvalues>
-          <attvalue for="0" value="A Polynomial-time Solution for Robust Registration with Extreme Outlier&#10;  Rates" />
-          <attvalue for="1" value="  We propose a robust approach for the registration of two sets of 3D points in&#10;the presence of a large amount of outliers. Our first contribution is to&#10;reformulate the registration problem using a Truncated Least Squares (TLS) cost&#10;that makes the estimation insensitive to a large fraction of spurious&#10;point-to-point correspondences. The second contribution is a general framework&#10;to decouple rotation, translation, and scale estimation, which allows solving&#10;in cascade for the three transformations. Since each subproblem (scale,&#10;rotation, and translation estimation) is still non-convex and combinatorial in&#10;nature, out third contribution is to show that (i) TLS scale and&#10;(component-wise) translation estimation can be solved exactly and in polynomial&#10;time via an adaptive voting scheme, (ii) TLS rotation estimation can be relaxed&#10;to a semidefinite program and the relaxation is tight in practice, even in the&#10;presence of an extreme amount of outliers. We validate the proposed algorithm,&#10;named TEASER (Truncated least squares Estimation And SEmidefinite Relaxation),&#10;in standard registration benchmarks showing that the algorithm outperforms&#10;RANSAC and robust local optimization techniques, and favorably compares with&#10;Branch-and-Bound methods, while being a polynomial-time algorithm. TEASER can&#10;tolerate up to 99% outliers and returns highly-accurate solutions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.08182" label="1603.08182">
-        <attvalues>
-          <attvalue for="0" value="3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions" />
-          <attvalue for="1" value="  Matching local geometric features on real-world depth images is a challenging&#10;task due to the noisy, low-resolution, and incomplete nature of 3D scan data.&#10;These difficulties limit the performance of current state-of-art methods, which&#10;are typically based on histograms over geometric properties. In this paper, we&#10;present 3DMatch, a data-driven model that learns a local volumetric patch&#10;descriptor for establishing correspondences between partial 3D data. To amass&#10;training data for our model, we propose a self-supervised feature learning&#10;method that leverages the millions of correspondence labels found in existing&#10;RGB-D reconstructions. Experiments show that our descriptor is not only able to&#10;match local geometry in new scenes for reconstruction, but also generalize to&#10;different tasks and spatial scales (e.g. instance-level object model alignment&#10;for the Amazon Picking Challenge, and mesh surface correspondence). Results&#10;show that 3DMatch consistently outperforms other state-of-the-art approaches by&#10;a significant margin. Code, data, benchmarks, and pre-trained models are&#10;available online at http://3dmatch.cs.princeton.edu&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04281" label="1904.04281">
-        <attvalues>
-          <attvalue for="0" value="3D Local Features for Direct Pairwise Registration" />
-          <attvalue for="1" value="  We present a novel, data driven approach for solving the problem of&#10;registration of two point cloud scans. Our approach is direct in the sense that&#10;a single pair of corresponding local patches already provides the necessary&#10;transformation cue for the global registration. To achieve that, we first endow&#10;the state of the art PPF-FoldNet auto-encoder (AE) with a pose-variant sibling,&#10;where the discrepancy between the two leads to pose-specific descriptors. Based&#10;upon this, we introduce RelativeNet, a relative pose estimation network to&#10;assign correspondence-specific orientations to the keypoints, eliminating any&#10;local reference frame computations. Finally, we devise a simple yet effective&#10;hypothesize-and-verify algorithm to quickly use the predictions and align two&#10;point sets. Our extensive quantitative and qualitative experiments suggests&#10;that our approach outperforms the state of the art in challenging real datasets&#10;of pairwise registration and that augmenting the keypoints with local pose&#10;information leads to better generalization and a dramatic speed-up.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.12099" label="1904.12099">
-        <attvalues>
-          <attvalue for="0" value="Learning to Fuse Local Geometric Features for 3D Rigid Data Matching" />
-          <attvalue for="1" value="  This paper presents a simple yet very effective data-driven approach to fuse&#10;both low-level and high-level local geometric features for 3D rigid data&#10;matching. It is a common practice to generate distinctive geometric descriptors&#10;by fusing low-level features from various viewpoints or subspaces, or enhance&#10;geometric feature matching by leveraging multiple high-level features. In prior&#10;works, they are typically performed via linear operations such as concatenation&#10;and min pooling. We show that more compact and distinctive representations can&#10;be achieved by optimizing a neural network (NN) model under the triplet&#10;framework that non-linearly fuses local geometric features in Euclidean spaces.&#10;The NN model is trained by an improved triplet loss function that fully&#10;leverages all pairwise relationships within the triplet. Moreover, the fused&#10;descriptor by our approach is also competitive to deep learned descriptors from&#10;raw data while being more lightweight and rotational invariant. Experimental&#10;results on four standard datasets with various data modalities and application&#10;contexts confirm the advantages of our approach in terms of both feature&#10;matching and geometric registration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.01014" label="2005.01014">
-        <attvalues>
-          <attvalue for="0" value="Feature-metric Registration: A Fast Semi-supervised Approach for Robust&#10;  Point Cloud Registration without Correspondences" />
-          <attvalue for="1" value="  We present a fast feature-metric point cloud registration framework, which&#10;enforces the optimisation of registration by minimising a feature-metric&#10;projection error without correspondences. The advantage of the feature-metric&#10;projection error is robust to noise, outliers and density difference in&#10;contrast to the geometric projection error. Besides, minimising the&#10;feature-metric projection error does not need to search the correspondences so&#10;that the optimisation speed is fast. The principle behind the proposed method&#10;is that the feature difference is smallest if point clouds are aligned very&#10;well. We train the proposed method in a semi-supervised or unsupervised&#10;approach, which requires limited or no registration label data. Experiments&#10;demonstrate our method obtains higher accuracy and robustness than the&#10;state-of-the-art methods. Besides, experimental results show that the proposed&#10;method can handle significant noise and density difference, and solve both&#10;same-source and cross-source point cloud registration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.11540" label="2004.11540">
-        <attvalues>
-          <attvalue for="0" value="Deep Global Registration" />
-          <attvalue for="1" value="  We present Deep Global Registration, a differentiable framework for pairwise&#10;registration of real-world 3D scans. Deep global registration is based on three&#10;modules: a 6-dimensional convolutional network for correspondence confidence&#10;prediction, a differentiable Weighted Procrustes algorithm for closed-form pose&#10;estimation, and a robust gradient-based SE(3) optimizer for pose refinement.&#10;Experiments demonstrate that our approach outperforms state-of-the-art methods,&#10;both learning-based and classical, on real-world data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.05143" label="1608.05143">
-        <attvalues>
-          <attvalue for="0" value="A Systematic Approach for Cross-source Point Cloud Registration by&#10;  Preserving Macro and Micro Structures" />
-          <attvalue for="1" value="  We propose a systematic approach for registering cross-source point clouds.&#10;The compelling need for cross-source point cloud registration is motivated by&#10;the rapid development of a variety of 3D sensing techniques, but many existing&#10;registration methods face critical challenges as a result of the large&#10;variations in cross-source point clouds. This paper therefore illustrates a&#10;novel registration method which successfully aligns two cross-source point&#10;clouds in the presence of significant missing data, large variations in point&#10;density, scale difference and so on. The robustness of the method is attributed&#10;to the extraction of macro and micro structures. Our work has three main&#10;contributions: (1) a systematic pipeline to deal with cross-source point cloud&#10;registration; (2) a graph construction method to maintain macro and micro&#10;structures; (3) a new graph matching method is proposed which considers the&#10;global geometric constraint to robustly register these variable graphs.&#10;Compared to most of the related methods, the experiments show that the proposed&#10;method successfully registers in cross-source datasets, while other methods&#10;have difficulty achieving satisfactory results. The proposed method also shows&#10;great ability in same-source datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.07324" label="1610.07324">
-        <attvalues>
-          <attvalue for="0" value="A coarse-to-fine algorithm for registration in 3D street-view&#10;  cross-source point clouds" />
-          <attvalue for="1" value="  With the development of numerous 3D sensing technologies, object registration&#10;on cross-source point cloud has aroused researchers' interests. When the point&#10;clouds are captured from different kinds of sensors, there are large and&#10;different kinds of variations. In this study, we address an even more&#10;challenging case in which the differently-source point clouds are acquired from&#10;a real street view. One is produced directly by the LiDAR system and the other&#10;is generated by using VSFM software on image sequence captured from RGB&#10;cameras. When it confronts to large scale point clouds, previous methods mostly&#10;focus on point-to-point level registration, and the methods have many&#10;limitations.The reason is that the least mean error strategy shows poor ability&#10;in registering large variable cross-source point clouds. In this paper,&#10;different from previous ICP-based methods, and from a statistic view, we&#10;propose a effective coarse-to-fine algorithm to detect and register a small&#10;scale SFM point cloud in a large scale Lidar point cloud. Seen from the&#10;experimental results, the model can successfully run on LiDAR and SFM point&#10;clouds, hence it can make a contribution to many applications, such as robotics&#10;and smart city development.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.04979" label="2406.04979">
-        <attvalues>
-          <attvalue for="0" value="Semantic Segmentation on VSPW Dataset through Masked Video Consistency" />
-          <attvalue for="1" value="  Pixel-level Video Understanding requires effectively integrating&#10;three-dimensional data in both spatial and temporal dimensions to learn&#10;accurate and stable semantic information from continuous frames. However,&#10;existing advanced models on the VSPW dataset have not fully modeled&#10;spatiotemporal relationships. In this paper, we present our solution for the&#10;PVUW competition, where we introduce masked video consistency (MVC) based on&#10;existing models. MVC enforces the consistency between predictions of masked&#10;frames where random patches are withheld. The model needs to learn the&#10;segmentation results of the masked parts through the context of images and the&#10;relationship between preceding and succeeding frames of the video.&#10;Additionally, we employed test-time augmentation, model aggeregation and a&#10;multimodal model-based post-processing method. Our approach achieves 67.27%&#10;mIoU performance on the VSPW dataset, ranking 2nd place in the PVUW2024&#10;challenge VSS track.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The PVUW2024 VSS Track focuses on the Pixel-level Video Understanding in the Wild \cite{miao2021vspw, Miao_2022_CVPR}, which is a critical task in advancing computer vision. Unlike traditional image segmentation, this challenge addresses the need for video segmentation, recognizing that real-world applications are video-based rather than static. The goal is to assign predefined semantic labels to every pixel in all frames of a given video, leveraging temporal information to improve predictive accuracy. This task is more challenging than image-based segmentation since it requires the integration of spatial and temporal data to produce accurate and stable semantic information across continuous frames. The final evaluation is conducted on the VSPW \cite{miao2021vspw} dataset, with using mean Intersectionover Union (mIoU) and Video Consistency (VC) as evalutation metrics.&#10;&#10;However, existing methods \cite{zhang2023dvis, zhang2023dvis++} face several challenges: they struggle with finely segmenting object edges, often misclassifying regions of the same object with nearly unchanged visual features in the original image as different categories. Additionally, their capability for temporal modeling remains underdeveloped, resulting in the model's inability to maintain stable segmentation of the same semantic category across adjacent frames. To address these issues, we introduce Masked Video Consistency (MVC). MVC optimizes existing models by enforcing consistency constraints in both temporal and spatial dimensions. It encourages the model to maintain consistent predictions for masked frames, where random patches are hidden, by leveraging the contextual information from surrounding frames and the spatial structure of the video. This dual-focus on spatial and temporal consistency aims to overcome the limitations of current models and achieve more accurate and stable video semantic segmentation. &#10;&#10;To alleviate the burden during the model training process, we employed a Test-Time Augmentation (TTA) strategy that integrates segmentation results from various input modifications using a single model, rather than assembling multiple models. Specifically, we applied different input scales and image flips. This approach allows us to leverage the strengths of multiple augmented views of the input data, enhancing the robustness and accuracy of the segmentation results without the additional computational overhead of training and maintaining multiple models. By using TTA, we ensure that our model can generalize better to diverse input conditions, leading to more reliable performance in practical applications.&#10;&#10;In the post-processing stage, we utilized the large multimodal model \cite{liu2023improved, liu2024visual, achiam2023gpt} to correct some of the video segmentation results. During our observation of the final segmentation outcomes, we found that the model struggled to maintain consistency in segmenting challenging stuff classes across the entire video sequence. Additionally, it sometimes misclassified a single class into multiple similar class regions within a single frame. We attributed this issue to the model's lack of sufficient training data. To address this, we employed a large multimodal model to refine these results, ultimately ensuring consistent segmentation throughout the entire video.&#10;&#10;By combining the above strategies, we achieved a 67.27\% mIoU on the final test set. Additionally, VC$_8$ and VC$_{16}$ scores were 94.99\% and 93.12\%, respectively, demonstrating the consistency and stability of our segmentation results.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning Models, Video Data Analysis, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2306.03413" label="2306.03413">
-        <attvalues>
-          <attvalue for="0" value="DVIS: Decoupled Video Instance Segmentation Framework" />
-          <attvalue for="1" value="  Video instance segmentation (VIS) is a critical task with diverse&#10;applications, including autonomous driving and video editing. Existing methods&#10;often underperform on complex and long videos in real world, primarily due to&#10;two factors. Firstly, offline methods are limited by the tightly-coupled&#10;modeling paradigm, which treats all frames equally and disregards the&#10;interdependencies between adjacent frames. Consequently, this leads to the&#10;introduction of excessive noise during long-term temporal alignment. Secondly,&#10;online methods suffer from inadequate utilization of temporal information. To&#10;tackle these challenges, we propose a decoupling strategy for VIS by dividing&#10;it into three independent sub-tasks: segmentation, tracking, and refinement.&#10;The efficacy of the decoupling strategy relies on two crucial elements: 1)&#10;attaining precise long-term alignment outcomes via frame-by-frame association&#10;during tracking, and 2) the effective utilization of temporal information&#10;predicated on the aforementioned accurate alignment outcomes during refinement.&#10;We introduce a novel referring tracker and temporal refiner to construct the&#10;\textbf{D}ecoupled \textbf{VIS} framework (\textbf{DVIS}). DVIS achieves new&#10;SOTA performance in both VIS and VPS, surpassing the current SOTA methods by&#10;7.3 AP and 9.6 VPQ on the OVIS and VIPSeg datasets, which are the most&#10;challenging and realistic benchmarks. Moreover, thanks to the decoupling&#10;strategy, the referring tracker and temporal refiner are super light-weight&#10;(only 1.69\% of the segmenter FLOPs), allowing for efficient training and&#10;inference on a single GPU with 11G memory. The code is available at&#10;\href{https://github.com/zhang-tao-whu/DVIS}{https://github.com/zhang-tao-whu/DVIS}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Online Video Instance Segmentation. Most mainstream online VIS methods follow a pipeline of segmenting and associating instances. MaskTrack R-CNN \cite{masktrackrcnn} incorporates a tracking head based on \cite{maskrcnn} and associates instances in adjacent frames using multiple cues such as similarity score, semantic consistency, spatial correlation, and detection confidence. \cite{sipmask} replaces the segmenter in the above pipeline with a one-stage instance segmentation network. \cite{crossvis} proposes a crossover learning scheme that segments the same instances in another frame using the instance features of the current frame. With stronger segmenters and the widespread application of transformers in vision tasks \cite{mask2former,detr}, recent works such as \cite{minvis,idol} have achieved outstanding performance. \cite{minvis} proposes a minimal VIS framework based on \cite{mask2former} that achieves instance association by measuring the similarity between the same instances in adjacent frames. \cite{idol} introduces contrastive learning in VIS to obtain a more discriminative instance representation. \cite{rovis,genvis} completely remove heuristic matching algorithms by delivering instance representations and modeling inter-frame association. Inspired by \cite{efficientvis, genvis, minvis, TrackFormer}, DVIS also performs tracking based on instance representations, which significantly reduces memory requirements. Our proposed DVIS introduces a novel component called the referring tracker, which models inter-frame association by denoising current instance representations with the help of previous frame instance representations.&#10;&#10;Offline Video Instance Segmentation. Previous offline video instance segmentation (VIS) methods have used various approaches to model the spatio-temporal representations of instances in the video. In \cite{stemseg}, instance spatio-temporal embeddings are modeled using a 3D CNN. The first transformer-based VIS architecture proposed in \cite{vistr} uses learnable initial embeddings for each instance of each frame, making it challenging to model instances with complex motion trajectories. \cite{ifc} introduces inter-frame communication, which reduces computation and memory overhead while improving performance. By directly modeling a video-level representation for each instance, \cite{mask2formervis} achieves impressive results. \cite{seqformer} constructs a VIS framework based on deformable attention \cite{deformabledetr} to separate temporal and spatial interactions between instance representations and videos. To significantly reduce memory consumption and enable offline methods to handle long videos, \cite{vita} constructs the video-level instance representation from the instance representations of each frame. While \cite{genvis} implements a semi-online VIS framework by replacing frames with clips, no significant gains were observed compared to the online version. The current SOTA methods for VIS have been demonstrated to overlook the importance of the refinement sub-task. Specifically, the refinement process has been neglected by \cite{idol,minvis,rovis,genvis}, while \cite{ifc,mask2formervis,seqformer,vita} exhibit a lack of clear separation between refinement and other aspects of the segmentation and tracking sub-tasks. Our proposed DVIS achieves SOTA performance by decoupling the VIS task and designing an efficient temporal refiner to fully utilize the information of the overall video.&#10;" />
-        </attvalues>
-      </node>
-      <node id="2310.03744" label="2310.03744">
-        <attvalues>
-          <attvalue for="0" value="Improved Baselines with Visual Instruction Tuning" />
-          <attvalue for="1" value="  Large multimodal models (LMM) have recently shown encouraging progress with&#10;visual instruction tuning. In this note, we show that the fully-connected&#10;vision-language cross-modal connector in LLaVA is surprisingly powerful and&#10;data-efficient. With simple modifications to LLaVA, namely, using&#10;CLIP-ViT-L-336px with an MLP projection and adding academic-task-oriented VQA&#10;data with simple response formatting prompts, we establish stronger baselines&#10;that achieve state-of-the-art across 11 benchmarks. Our final 13B checkpoint&#10;uses merely 1.2M publicly available data, and finishes full training in ~1 day&#10;on a single 8-A100 node. We hope this can make state-of-the-art LMM research&#10;more accessible. Code and model will be publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.04042" label="2405.04042">
-        <attvalues>
-          <attvalue for="0" value="Space-time Reinforcement Network for Video Object Segmentation" />
-          <attvalue for="1" value="  Recently, video object segmentation (VOS) networks typically use memory-based&#10;methods: for each query frame, the mask is predicted by space-time matching to&#10;memory frames. Despite these methods having superior performance, they suffer&#10;from two issues: 1) Challenging data can destroy the space-time coherence&#10;between adjacent video frames. 2) Pixel-level matching will lead to undesired&#10;mismatching caused by the noises or distractors. To address the aforementioned&#10;issues, we first propose to generate an auxiliary frame between adjacent&#10;frames, serving as an implicit short-temporal reference for the query one.&#10;Next, we learn a prototype for each video object and prototype-level matching&#10;can be implemented between the query and memory. The experiment demonstrated&#10;that our network outperforms the state-of-the-art method on the DAVIS 2017,&#10;achieving a J&amp;F score of 86.4%, and attains a competitive result 85.0% on&#10;YouTube VOS 2018. In addition, our network exhibits a high inference speed of&#10;32+ FPS.&#10;" />
-          <attvalue for="2" value="&#10;Semi-supervised video object segmentation (VOS) stands as a challenging task in computer vision, drawing widespread attention in autonomous driving, robotics and video editing. The key to this task is to fully utilize the given limited signals between frames, where the first-frame annotation is provided by the user, and it segments objects in the remaining frames as accurately as possible.&#10;&#10;Recently, matching-based methods \cite{match1,chen3,match2} have achieved great success in semi-supervised VOS, achieving object segmentation through pixel-level matching between query frames and memory frames. Among these methods, memory-based networks \cite{1,2,3,qihe,4,5,6,7,chen2,9,chen1,10,mem1,mem2,mem3} attracted a lot of attention. For instance, the space-time storage network (STM) \cite{1} firstly is proposed to construct a feature memory for each object, and applies space-time matching between query and memory frames, which can better solve the problems of object occlusion and drift. After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on. Particularly, the latest related state-of-the-art methods, known as STCN \cite{9} and XMem \cite{10}, share multiple memory stores for all objects to capture different spatial-temporal contexts, have achieved prominent performance. &#10;&#10;Although these methods have achieved good results, there are still two issues that need to be carefully considered. Firstly, the existing methods \cite{3,7,10} investigate the space-time coherence between adjacent video frames to exclude distractor objects, however the coherence is often completely destroyed due to occlusion, fast motion and irregular deformation. This can be referred to Fig.\ref{fig:res1}(a), where the features between the query and its adjacent frame are inconsistent. Secondly, the dense pixel-level matching between the memory features and the query features will leads to undesired mismatching caused by the noises or distractors as shown in Fig.\ref{fig:res1}(b). Thus, a high-level matching is needful as an auxiliary manner for solving the problem.&#10;&#10;Motivated by the above discussions, we propose a Space-time Reinforcement Network (SRNet) for video object segmentation to address the two mentioned weaknesses. In more details, the core ideas of this paper come from two aspects: i) Instead of directly exploring the space-time coherence between adjacent video frames, we propose to generate an auxiliary frame from adjacent frames, serving as an implicit short-temporal reference for the query one. The proposed auxiliary frame is more consistent with the query frame as shown in Fig.\ref{fig:res1}(a). ii) Beyond pixel-level matching, we propose to additionally learn a prototype for each video object. Thus, a high-level semantic matching, i.e., prototype-level matching, can be implemented between the query and memory to effectively improve the robustness of the method, also shown in Fig.\ref{fig:res1}(b).&#10;&#10;Our contributions can be summarized as follows:&#10;&#10;\begin{itemize}&#10; \item We propose to generate an implicit auxiliary frame between adjacent frames to improve the space-time coherence.&#10; \item We introduce a high-level semantic matching, i.e., prototype-level matching, to reduce undesired mismatching caused by the noises or distractors.&#10; \item Our SRNet achieves state-of-the-art performance on DAVIS 2017 and competitive results on YouTube 2018, while maintaining a high inference speed of 32+ FPS.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning Methods, Computer Science, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1902.09513" label="1902.09513">
-        <attvalues>
-          <attvalue for="0" value="FEELVOS: Fast End-to-End Embedding Learning for Video Object&#10;  Segmentation" />
-          <attvalue for="1" value="  Many of the recent successful methods for video object segmentation (VOS) are&#10;overly complicated, heavily rely on fine-tuning on the first frame, and/or are&#10;slow, and are hence of limited practical use. In this work, we propose FEELVOS&#10;as a simple and fast method which does not rely on fine-tuning. In order to&#10;segment a video, for each frame FEELVOS uses a semantic pixel-wise embedding&#10;together with a global and a local matching mechanism to transfer information&#10;from the first frame and from the previous frame of the video to the current&#10;frame. In contrast to previous work, our embedding is only used as an internal&#10;guidance of a convolutional network. Our novel dynamic segmentation head allows&#10;us to train the network, including the embedding, end-to-end for the multiple&#10;object segmentation task with a cross entropy loss. We achieve a new state of&#10;the art in video object segmentation without fine-tuning with a J&amp;F measure of&#10;71.5% on the DAVIS 2017 validation set. We make our code and models available&#10;at https://github.com/tensorflow/models/tree/master/research/feelvos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.12934" label="2103.12934">
-        <attvalues>
-          <attvalue for="0" value="Efficient Regional Memory Network for Video Object Segmentation" />
-          <attvalue for="1" value="  Recently, several Space-Time Memory based networks have shown that the object&#10;cues (e.g. video frames as well as the segmented object masks) from the past&#10;frames are useful for segmenting objects in the current frame. However, these&#10;methods exploit the information from the memory by global-to-global matching&#10;between the current and past frames, which lead to mismatching to similar&#10;objects and high computational complexity. To address these problems, we&#10;propose a novel local-to-local matching solution for semi-supervised VOS,&#10;namely Regional Memory Network (RMNet). In RMNet, the precise regional memory&#10;is constructed by memorizing local regions where the target objects appear in&#10;the past frames. For the current query frame, the query regions are tracked and&#10;predicted based on the optical flow estimated from the previous frame. The&#10;proposed local-to-local matching effectively alleviates the ambiguity of&#10;similar objects in both memory and query frames, which allows the information&#10;to be passed from the regional memory to the query region efficiently and&#10;effectively. Experimental results indicate that the proposed RMNet performs&#10;favorably against state-of-the-art methods on the DAVIS and YouTube-VOS&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Propagation-based Methods.&#10;Early methods treat video object segmentation as a temporal label propagation problem.&#10;ObjectFlow~\cite{DBLP:conf/cvpr/Tsai0B16}, SegFlow~\cite{DBLP:conf/iccv/ChengTW017} and DVSNet~\cite{DBLP:conf/cvpr/XuFYL18} consider video segmentation and optical flow estimation simultaneously.&#10;To adapt to specific instances, MaskTrack~\cite{DBLP:conf/cvpr/PerazziKBSS17} fine-tunes the network on the first frame during testing.&#10;MaskRNN~\cite{DBLP:conf/nips/HuHS17} predicts the instance-level segmentation of multiple objects from the estimated optical flow and bounding boxes of objects.&#10;CINN~\cite{DBLP:conf/cvpr/BaoW018} introduces a Markov Random Field to establish spatio-temporal dependencies for pixels.&#10;DyeNet~\cite{DBLP:conf/eccv/LiL18} and PReMVOS~\cite{DBLP:conf/accv/LuitenVL18} combine temporal propagation and re-identification functionalities into a single framework.&#10;Apart from propagating masks with optical flows, object tracking is also widely used in semi-supervised VOS.&#10;FAVOS~\cite{DBLP:conf/cvpr/ChengTHW018} predicts the mask of an object from several tracking boxes of the object parts.&#10;LucidTracker~\cite{DBLP:journals/ijcv/KhorevaBIBS19} synthesizes in-domain data to train a specialized pixel-level video object segmenter.&#10;SAT~\cite{DBLP:conf/cvpr/ChenLYYSQ20} takes advantage of the inter-frame consistency and deals with each target object as a tracklet.&#10;Despite promising results, these methods are not robust to occlusion and drifting, which causes error accumulation during the propagation~\cite{DBLP:conf/cvpr/OhLXK19}.&#10;&#10;Matching-based Methods.&#10;To handle object occlusion and drifting, recent methods perform feature matching to find objects that are similar to the target objects in the rest of the video.&#10;OSVOS~\cite{DBLP:conf/cvpr/CaellesMPLCG17} transfers the generic semantic information to the task of foreground segmentation by fine-tuning the network on the first frame.&#10;RGMP~\cite{DBLP:conf/cvpr/OhLSK18} takes the mask of the previous frame as input, which provides a spatial prior for the current frame.&#10;PML~\cite{DBLP:conf/cvpr/ChenPMG18} learns a pixel-wise embedding using a triplet loss and assigns a label to each pixel by nearest neighbor matching in pixel space to the first frame.&#10;VideoMatch~\cite{DBLP:conf/eccv/HuHS18a} uses a soft matching layer that maps the pixels of the current frame to the first frame in the learned embedding space.&#10;Following PML and VideoMatch, FEELVOS~\cite{DBLP:conf/cvpr/VoigtlaenderCSA19} extends the pixel-level matching mechanism by additionally matching between the current frame and the first frame.&#10;Based on FEELVOS, CFBI~\cite{DBLP:conf/eccv/YangWY20} promotes the results by explicitly considering the feature matching of the target foreground object and the corresponding background. &#10;STM~\cite{DBLP:conf/iccv/OhLXK19} leverages a memory network to perform pixel-level matching from past frames, which outperforms all previous methods.&#10;Based on STM, KMN~\cite{DBLP:conf/eccv/SeongHK20} applies a Gaussian kernel to reduce the mismatched pixels.&#10;EGMN~\cite{DBLP:conf/eccv/LuWDZSG20} employs an episodic memory network to store frames as nodes and capture cross-frame correlations by edges.&#10;Other STM-based methods~\cite{DBLP:conf/cvpr/XieHXLS20,DBLP:conf/cvpr/ZhangHZP20} use depth estimation and spatial constraint modules to improve the accuracy of STM.&#10;However, matching-based methods perform unnecessary matching in regions without target objects and are less effective to distinguish objects with similar appearance.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2205.03761" label="2205.03761">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Dynamic Embedding for Video Object Segmentation" />
-          <attvalue for="1" value="  Space-time memory (STM) based video object segmentation (VOS) networks&#10;usually keep increasing memory bank every several frames, which shows excellent&#10;performance. However, 1) the hardware cannot withstand the ever-increasing&#10;memory requirements as the video length increases. 2) Storing lots of&#10;information inevitably introduces lots of noise, which is not conducive to&#10;reading the most important information from the memory bank. In this paper, we&#10;propose a Recurrent Dynamic Embedding (RDE) to build a memory bank of constant&#10;size. Specifically, we explicitly generate and update RDE by the proposed&#10;Spatio-temporal Aggregation Module (SAM), which exploits the cue of historical&#10;information. To avoid error accumulation owing to the recurrent usage of SAM,&#10;we propose an unbiased guidance loss during the training stage, which makes SAM&#10;more robust in long videos. Moreover, the predicted masks in the memory bank&#10;are inaccurate due to the inaccurate network inference, which affects the&#10;segmentation of the query frame. To address this problem, we design a novel&#10;self-correction strategy so that the network can repair the embeddings of masks&#10;with different qualities in the memory bank. Extensive experiments show our&#10;method achieves the best tradeoff between performance and speed. Code is&#10;available at https://github.com/Limingxing00/RDE-VOS-CVPR2022.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.11840" label="2304.11840">
-        <attvalues>
-          <attvalue for="0" value="Robust and Efficient Memory Network for Video Object Segmentation" />
-          <attvalue for="1" value="  This paper proposes a Robust and Efficient Memory Network, referred to as&#10;REMN, for studying semi-supervised video object segmentation (VOS).&#10;Memory-based methods have recently achieved outstanding VOS performance by&#10;performing non-local pixel-wise matching between the query and memory. However,&#10;these methods have two limitations. 1) Non-local matching could cause&#10;distractor objects in the background to be incorrectly segmented. 2) Memory&#10;features with high temporal redundancy consume significant computing resources.&#10;For limitation 1, we introduce a local attention mechanism that tackles the&#10;background distraction by enhancing the features of foreground objects with the&#10;previous mask. For limitation 2, we first adaptively decide whether to update&#10;the memory features depending on the variation of foreground objects to reduce&#10;temporal redundancy. Second, we employ a dynamic memory bank, which uses a&#10;lightweight and differentiable soft modulation gate to decide how many memory&#10;features need to be removed in the temporal dimension. Experiments demonstrate&#10;that our REMN achieves state-of-the-art results on DAVIS 2017, with a&#10;$\mathcal{J\&amp;F}$ score of 86.3% and on YouTube-VOS 2018, with a $\mathcal{G}$&#10;over mean of 85.5%. Furthermore, our network shows a high inference speed of&#10;25+ FPS and uses relatively few computing resources.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.07115" label="2207.07115">
-        <attvalues>
-          <attvalue for="0" value="XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin&#10;  Memory Model" />
-          <attvalue for="1" value="  We present XMem, a video object segmentation architecture for long videos&#10;with unified feature memory stores inspired by the Atkinson-Shiffrin memory&#10;model. Prior work on video object segmentation typically only uses one type of&#10;feature memory. For videos longer than a minute, a single feature memory model&#10;tightly links memory consumption and accuracy. In contrast, following the&#10;Atkinson-Shiffrin model, we develop an architecture that incorporates multiple&#10;independent yet deeply-connected feature memory stores: a rapidly updated&#10;sensory memory, a high-resolution working memory, and a compact thus sustained&#10;long-term memory. Crucially, we develop a memory potentiation algorithm that&#10;routinely consolidates actively used working memory elements into the long-term&#10;memory, which avoids memory explosion and minimizes performance decay for&#10;long-term prediction. Combined with a new memory reading mechanism, XMem&#10;greatly exceeds state-of-the-art performance on long-video datasets while being&#10;on par with state-of-the-art methods (that do not work on long videos) on&#10;short-video datasets. Code is available at https://hkchengrex.github.io/XMem&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.09666" label="2101.09666">
-        <attvalues>
-          <attvalue for="0" value="Grad-CAM guided channel-spatial attention module for fine-grained visual&#10;  classification" />
-          <attvalue for="1" value="  Fine-grained visual classification (FGVC) is becoming an important research&#10;field, due to its wide applications and the rapid development of computer&#10;vision technologies. The current state-of-the-art (SOTA) methods in the FGVC&#10;usually employ attention mechanisms to first capture the semantic parts and&#10;then discover their subtle differences between distinct classes. The&#10;channel-spatial attention mechanisms, which focus on the discriminative&#10;channels and regions simultaneously, have significantly improved the&#10;classification performance. However, the existing attention modules are poorly&#10;guided since part-based detectors in the FGVC depend on the network learning&#10;ability without the supervision of part annotations. As obtaining such part&#10;annotations is labor-intensive, some visual localization and explanation&#10;methods, such as gradient-weighted class activation mapping (Grad-CAM), can be&#10;utilized for supervising the attention mechanism. We propose a Grad-CAM guided&#10;channel-spatial attention module for the FGVC, which employs the Grad-CAM to&#10;supervise and constrain the attention weights by generating the coarse&#10;localization maps. To demonstrate the effectiveness of the proposed method, we&#10;conduct comprehensive experiments on three popular FGVC datasets, including&#10;CUB-$200$-$2011$, Stanford Cars, and FGVC-Aircraft datasets. The proposed&#10;method outperforms the SOTA attention modules in the FGVC task. In addition,&#10;visualizations of feature maps also demonstrate the superiority of the proposed&#10;method against the SOTA approaches.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Fine-grained visual classification (FGVC) aims to distinguish fine-grained classes under the same coarse class labels,~e.g., birds~\cite{birds}, airplanes~\cite{air}, cars~\cite{cars}， and flowers~\cite{flowers},~etc. The main challenge of the FGVC task is the tiny inter-class difference along with significant intra-class diversity. For example, it is difficult to distinguish a redhead woodpecker from a pileated woodpecker and a downy woodpecker caused by highly similar sub-categories, but with the adjustment of poses, scales, and rotations, the redhead woodpecker can be photographed in a very different visual view. In order to generate discriminative features more precisely, we better have the ability to capture the key characteristics of the red head and ignore the background and other irrelevant regions, which is an obvious way for overcoming the challenge. &#10;&#10;The existing approaches can be roughly divided into two classes: ($1$) searching the informative regions that contribute the most to the classification task~\cite{anno,TASN,NTS} and ($2$) paying more attention to extract high-order features from the images~\cite{BP,BT,mamcloss,mc}. For the former one, previous approaches~\cite{anno} usually employed the prior location information such as part-level bounding boxes and segmented masks to generate the discriminative parts. Meanwhile, for the latter one, the powerful deep networks~\cite{BP,BT} were employed for feature extraction, and different loss functions~\cite{mamcloss,mc} were designed for constraining these networks to improve the discrimination of the extracted features. Recently, the attention mechanisms~\cite{SENet,MA-CNN,RA-CNN}, which only require image labels for training, have gradually replaced the manual annotation methods, since part annotations are time-consuming and laborious that limits the flexibility and versatility of the real-world FGVC applications. Compared with the approaches that introduce complex structures, the attention-based methods add fewer learned parameters for model training, which efficiently reduces the computation cost.&#10;&#10;The attention mechanisms fully simulate the observation habits of human eyes, which always concentrate on the most distinctive regions for observing. For example, we can easily pay attention to the head and the wings of a bird and ignore the other common regions to identify its species. Based on this motivation, many methods have been proposed by utilizing the attention mechanisms to detect the discriminative information from the images, including channel attention~\cite{TASN,SENet}, spatial attention~\cite{RA-CNN,PA-CNN}, and channel-spatial attention~\cite{CBAM}. Specifically, SENet~\cite{SENet} introduced ``squeeze-and-excitation'' (SE) blocks to adaptively recalibrate the feature maps in channel-wise by modeling the interactions between channels. The trilinear attention sampling network~\cite{TASN} generated attention maps by integrating feature channels with their relationship matrix and highlighted the attended parts with high resolution. The recurrent attention convolutional neural network (RA-CNN)~\cite{RA-CNN} introduced attention proposal network (APN) to capture the region relevance information based on the extracted features, and then amplified the attention region crops to make the network gradually focus on the key areas. The convolutional block attention mechanism (CBAM)~\cite{CBAM} is a channel-spatial attention method that utilizes both the channel-level and region-level information. It can effectively improve the characteristic expression ability of the networks. The existing methods~\cite{TASN,SENet,RA-CNN,CBAM} usually utilize different attention mechanisms to generally adjust the distributions of the attention weights for balancing the contributions of feature maps extracted from each &#10;part. Although these methods for obtaining the weights are different, they are all constructed based on the original feature maps only, without part information supervision. Obviously, if the feature maps focus on the non-significant parts such as backgrounds and distractions, the attention mechanism is meaningless under the unsupervised conditions. &#10;&#10;To address this issue, we propose a weakly-supervised guideline for discriminative part mining and informative feature learning. It drives the networks to focus on the parts which have specific characteristic information, such as the head and the beak of a bird. In addition, as each channel of the feature maps can be also considered as a semantic part~\cite{MA-CNN}, supervision on discriminative parts can be transferred to that on channels. Gradient-weighted class activation mapping (Grad-CAM)~\cite{cam} is usually introduced to illustrate attentions of the networks with heat maps and visualize the attentions in each part by weighted averaging channels, which can be used for guiding the networks to focus on more efficient parts and discard the redundant information for the classification. In this paper, we introduce a channel-spatial attention mechanism with Grad-CAM guided, in which the channel weighted feature maps are pooled along with the channel dimensions and multiplied by the original feature maps to obtain the channel-spatial attention maps. Meanwhile, a novel Grad-CAM guided channel-spatial attention mechanism loss (GGAM-Loss) is applied for guiding the learning process of the channel weights and forcing the attention module to focus on the parts that contribute most to the classification. As shown in Figure \ref{fig:attention}, we employ the channel weights obtained from the gradient backpropagation in the Grad-CAM to constrain the channel weights of the forward propagation. &#10;&#10;Our contributions can be summarized as follows:&#10;\begin{itemize}&#10;&#09;\item We address the challenge of the FGVC by proposing a Grad-CAM guided channel-spatial attention module, which constrains the channel-spatial attention mechanism to focus on the parts that contribute most to the classification.&#10;&#09;\item We propose a Grad-CAM guided channel-spatial attention mechanism loss (GGAM-Loss) which employs the Grad-CAM to supervise and constrain the attention weights. Moreover, it is not limited to a specific network architecture.&#10;&#09;\item We conduct comprehensive experiments on the three commonly used FGVC datasets, i.e., CUB-$200$-$2011$~\cite{birds}, FGVC-Aircraft~\cite{air}， and Stanford Cars~\cite{cars} datasets. The results show the effectiveness of our method.&#10;\end{itemize}&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Attention Mechanisms, Fine-Grained Classification, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1306.5151" label="1306.5151">
-        <attvalues>
-          <attvalue for="0" value="Fine-Grained Visual Classification of Aircraft" />
-          <attvalue for="1" value="  This paper introduces FGVC-Aircraft, a new dataset containing 10,000 images&#10;of aircraft spanning 100 aircraft models, organised in a three-level hierarchy.&#10;At the finer level, differences between models are often subtle but always&#10;visually measurable, making visual recognition challenging but possible. A&#10;benchmark is obtained by defining corresponding classification tasks and&#10;evaluation protocols, and baseline results are presented. The construction of&#10;this dataset was made possible by the work of aircraft enthusiasts, a strategy&#10;that can extend to the study of number of other object classes. Compared to the&#10;domains usually considered in fine-grained visual classification (FGVC), for&#10;example animals, aircraft are rigid and hence less deformable. They, however,&#10;present other interesting modes of variation, including purpose, size,&#10;designation, structure, historical style, and branding.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06150" label="1903.06150">
-        <attvalues>
-          <attvalue for="0" value="Looking for the Devil in the Details: Learning Trilinear Attention&#10;  Sampling Network for Fine-grained Image Recognition" />
-          <attvalue for="1" value="  Learning subtle yet discriminative features (e.g., beak and eyes for a bird)&#10;plays a significant role in fine-grained image recognition. Existing&#10;attention-based approaches localize and amplify significant parts to learn&#10;fine-grained details, which often suffer from a limited number of parts and&#10;heavy computational cost. In this paper, we propose to learn such fine-grained&#10;features from hundreds of part proposals by Trilinear Attention Sampling&#10;Network (TASN) in an efficient teacher-student manner. Specifically, TASN&#10;consists of 1) a trilinear attention module, which generates attention maps by&#10;modeling the inter-channel relationships, 2) an attention-based sampler which&#10;highlights attended parts with high resolution, and 3) a feature distiller,&#10;which distills part features into a global one by weight sharing and feature&#10;preserving strategies. Extensive experiments verify that TASN yields the best&#10;performance under the same settings with the most competitive approaches, in&#10;iNaturalist-2017, CUB-Bird, and Stanford-Cars datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.00287" label="1809.00287">
-        <attvalues>
-          <attvalue for="0" value="Learning to Navigate for Fine-grained Classification" />
-          <attvalue for="1" value="  Fine-grained classification is challenging due to the difficulty of finding&#10;discriminative features. Finding those subtle traits that fully characterize&#10;the object is not straightforward. To handle this circumstance, we propose a&#10;novel self-supervision mechanism to effectively localize informative regions&#10;without the need of bounding-box/part annotations. Our model, termed NTS-Net&#10;for Navigator-Teacher-Scrutinizer Network, consists of a Navigator agent, a&#10;Teacher agent and a Scrutinizer agent. In consideration of intrinsic&#10;consistency between informativeness of the regions and their probability being&#10;ground-truth class, we design a novel training paradigm, which enables&#10;Navigator to detect most informative regions under the guidance from Teacher.&#10;After that, the Scrutinizer scrutinizes the proposed regions from Navigator and&#10;makes predictions. Our model can be viewed as a multi-agent cooperation,&#10;wherein agents benefit from each other, and make progress together. NTS-Net can&#10;be trained end-to-end, while provides accurate fine-grained classification&#10;predictions as well as highly informative regions during inference. We achieve&#10;state-of-the-art performance in extensive benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.09915" label="1807.09915">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition" />
-          <attvalue for="1" value="  Fine-grained visual recognition is challenging because it highly relies on&#10;the modeling of various semantic parts and fine-grained feature learning.&#10;Bilinear pooling based models have been shown to be effective at fine-grained&#10;recognition, while most previous approaches neglect the fact that inter-layer&#10;part feature interaction and fine-grained feature learning are mutually&#10;correlated and can reinforce each other. In this paper, we present a novel&#10;model to address these issues. First, a cross-layer bilinear pooling approach&#10;is proposed to capture the inter-layer part feature relations, which results in&#10;superior performance compared with other bilinear pooling based approaches.&#10;Second, we propose a novel hierarchical bilinear pooling framework to integrate&#10;multiple cross-layer bilinear features to enhance their representation&#10;capability. Our formulation is intuitive, efficient and achieves&#10;state-of-the-art results on the widely used fine-grained recognition datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.03621" label="1911.03621">
-        <attvalues>
-          <attvalue for="0" value="Learning Deep Bilinear Transformation for Fine-grained Image&#10;  Representation" />
-          <attvalue for="1" value="  Bilinear feature transformation has shown the state-of-the-art performance in&#10;learning fine-grained image representations. However, the computational cost to&#10;learn pairwise interactions between deep feature channels is prohibitively&#10;expensive, which restricts this powerful transformation to be used in deep&#10;neural networks. In this paper, we propose a deep bilinear transformation (DBT)&#10;block, which can be deeply stacked in convolutional neural networks to learn&#10;fine-grained image representations. The DBT block can uniformly divide input&#10;channels into several semantic groups. As bilinear transformation can be&#10;represented by calculating pairwise interactions within each group, the&#10;computational cost can be heavily relieved. The output of each block is further&#10;obtained by aggregating intra-group bilinear features, with residuals from the&#10;entire input features. We found that the proposed network achieves new&#10;state-of-the-art in several fine-grained image recognition benchmarks,&#10;including CUB-Bird, Stanford-Car, and FGVC-Aircraft.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.05372" label="1806.05372">
-        <attvalues>
-          <attvalue for="0" value="Multi-Attention Multi-Class Constraint for Fine-grained Image&#10;  Recognition" />
-          <attvalue for="1" value="  Attention-based learning for fine-grained image recognition remains a&#10;challenging task, where most of the existing methods treat each object part in&#10;isolation, while neglecting the correlations among them. In addition, the&#10;multi-stage or multi-scale mechanisms involved make the existing methods less&#10;efficient and hard to be trained end-to-end. In this paper, we propose a novel&#10;attention-based convolutional neural network (CNN) which regulates multiple&#10;object parts among different input images. Our method first learns multiple&#10;attention region features of each input image through the one-squeeze&#10;multi-excitation (OSME) module, and then apply the multi-attention multi-class&#10;constraint (MAMC) in a metric learning framework. For each anchor feature, the&#10;MAMC functions by pulling same-attention same-class features closer, while&#10;pushing different-attention or different-class features away. Our method can be&#10;easily trained end-to-end, and is highly efficient which requires only one&#10;training stage. Moreover, we introduce Dogs-in-the-Wild, a comprehensive dog&#10;species dataset that surpasses similar existing datasets by category coverage,&#10;data volume and annotation quality. This dataset will be released upon&#10;acceptance to facilitate the research of fine-grained image recognition.&#10;Extensive experiments are conducted to show the substantial improvements of our&#10;method on four benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.04264" label="2002.04264">
-        <attvalues>
-          <attvalue for="0" value="The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image&#10;  Classification" />
-          <attvalue for="1" value="  Key for solving fine-grained image categorization is finding discriminate and&#10;local regions that correspond to subtle visual traits. Great strides have been&#10;made, with complex networks designed specifically to learn part-level&#10;discriminate feature representations. In this paper, we show it is possible to&#10;cultivate subtle details without the need for overly complicated network&#10;designs or training mechanisms -- a single loss is all it takes. The main trick&#10;lies with how we delve into individual feature channels early on, as opposed to&#10;the convention of starting from a consolidated feature map. The proposed loss&#10;function, termed as mutual-channel loss (MC-Loss), consists of two&#10;channel-specific components: a discriminality component and a diversity&#10;component. The discriminality component forces all feature channels belonging&#10;to the same class to be discriminative, through a novel channel-wise attention&#10;mechanism. The diversity component additionally constraints channels so that&#10;they become mutually exclusive on spatial-wise. The end result is therefore a&#10;set of feature channels that each reflects different locally discriminative&#10;regions for a specific class. The MC-Loss can be trained end-to-end, without&#10;the need for any bounding-box/part annotations, and yields highly&#10;discriminative regions during inference. Experimental results show our MC-Loss&#10;when implemented on top of common base networks can achieve state-of-the-art&#10;performance on all four fine-grained categorization datasets (CUB-Birds,&#10;FGVC-Aircraft, Flowers-102, and Stanford-Cars). Ablative studies further&#10;demonstrate the superiority of MC-Loss when compared with other recently&#10;proposed general-purpose losses for visual classification, on two different&#10;base networks. Code available at&#10;https://github.com/dongliangchang/Mutual-Channel-Loss&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.06521" label="1807.06521">
-        <attvalues>
-          <attvalue for="0" value="CBAM: Convolutional Block Attention Module" />
-          <attvalue for="1" value="  We propose Convolutional Block Attention Module (CBAM), a simple yet&#10;effective attention module for feed-forward convolutional neural networks.&#10;Given an intermediate feature map, our module sequentially infers attention&#10;maps along two separate dimensions, channel and spatial, then the attention&#10;maps are multiplied to the input feature map for adaptive feature refinement.&#10;Because CBAM is a lightweight and general module, it can be integrated into any&#10;CNN architectures seamlessly with negligible overheads and is end-to-end&#10;trainable along with base CNNs. We validate our CBAM through extensive&#10;experiments on ImageNet-1K, MS~COCO detection, and VOC~2007 detection datasets.&#10;Our experiments show consistent improvements in classification and detection&#10;performances with various models, demonstrating the wide applicability of CBAM.&#10;The code and models will be publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.02391" label="1610.02391">
-        <attvalues>
-          <attvalue for="0" value="Grad-CAM: Visual Explanations from Deep Networks via Gradient-based&#10;  Localization" />
-          <attvalue for="1" value="  We propose a technique for producing &quot;visual explanations&quot; for decisions from&#10;a large class of CNN-based models, making them more transparent. Our approach -&#10;Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of&#10;any target concept, flowing into the final convolutional layer to produce a&#10;coarse localization map highlighting important regions in the image for&#10;predicting the concept. Grad-CAM is applicable to a wide variety of CNN&#10;model-families: (1) CNNs with fully-connected layers, (2) CNNs used for&#10;structured outputs, (3) CNNs used in tasks with multimodal inputs or&#10;reinforcement learning, without any architectural changes or re-training. We&#10;combine Grad-CAM with fine-grained visualizations to create a high-resolution&#10;class-discriminative visualization and apply it to off-the-shelf image&#10;classification, captioning, and visual question answering (VQA) models,&#10;including ResNet-based architectures. In the context of image classification&#10;models, our visualizations (a) lend insights into their failure modes, (b) are&#10;robust to adversarial images, (c) outperform previous methods on localization,&#10;(d) are more faithful to the underlying model and (e) help achieve&#10;generalization by identifying dataset bias. For captioning and VQA, we show&#10;that even non-attention based models can localize inputs. We devise a way to&#10;identify important neurons through Grad-CAM and combine it with neuron names to&#10;provide textual explanations for model decisions. Finally, we design and&#10;conduct human studies to measure if Grad-CAM helps users establish appropriate&#10;trust in predictions from models and show that Grad-CAM helps untrained users&#10;successfully discern a 'stronger' nodel from a 'weaker' one even when both make&#10;identical predictions. Our code is available at&#10;https://github.com/ramprs/grad-cam/, along with a demo at&#10;http://gradcam.cloudcv.org, and a video at youtu.be/COjUB9Izk6E.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.13693" label="2212.13693">
-        <attvalues>
-          <attvalue for="0" value="Parsing Objects at a Finer Granularity: A Survey" />
-          <attvalue for="1" value="  Fine-grained visual parsing, including fine-grained part segmentation and&#10;fine-grained object recognition, has attracted considerable critical attention&#10;due to its importance in many real-world applications, e.g., agriculture,&#10;remote sensing, and space technologies. Predominant research efforts tackle&#10;these fine-grained sub-tasks following different paradigms, while the inherent&#10;relations between these tasks are neglected. Moreover, given most of the&#10;research remains fragmented, we conduct an in-depth study of the advanced work&#10;from a new perspective of learning the part relationship. In this perspective,&#10;we first consolidate recent research and benchmark syntheses with new&#10;taxonomies. Based on this consolidation, we revisit the universal challenges in&#10;fine-grained part segmentation and recognition tasks and propose new solutions&#10;by part relationship learning for these important challenges. Furthermore, we&#10;conclude several promising lines of research in fine-grained visual parsing for&#10;future research.&#10;" />
-          <attvalue for="2" value="&#10;&#10;F{ine-grained} visual parsing of image objects is a basic and crucial task in the computer vision community, which is fundamentally difficult, owing that there are usually subtle visual cues for distinguishing different objects or part regions. Recent advances in deep learning have significantly boosted the image understanding abilities of machine systems,~\eg, its performance on the large-scale ImageNet dataset~\cite{deng2009imagenet} surpasses the human-level recognition, but it is still a great challenge facing the fine-grained visual tasks. In particular, we consider two representative fine-grained visual parsing tasks in this paper,~\ie, semantic part segmentation and fine-grained object recognition.&#10;&#10;In contrast with coarse-grained object segmentation and base-level classification, fine-grained parsing is meant to segment or distinguish visually similar objects that belong to different fine-grained concepts, for example, decomposing objects into parts and dividing the base category into subcategories.&#10;A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint,&#10;zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}. Conventional machine learning techniques build explicit structures for parsing and understanding these fine-grained objects,~\eg, graph and tree structures for part segmentation~\cite{zhu2008unsupervised,hsieh2010segmentation,wang2011learning,car2014parsing}, and part learning in fine-grained recognition~\cite{zhang2012pose,branson2011strong}. In the era of deep learning, fine-grained segmentation and recognition approaches follow different paradigms, which achieve huge success compared to conventional models. Although there are more than 100 research papers each year to investigate this important problem, these papers seem to be disorganized, owing to various sundry research focuses including new task settings, benchmarking, and learning strategies. In particular, there are few survey papers that summarize the recent advances in fine-grained part segmentation. Thus the relationships among different fine-grained sub-tasks are still under-explored, and these sub-tasks are developed independently by regarding them as less-relevant tasks.&#10;&#10;In this paper, we make a comprehensive study of advances in fine-grained visual parsing tasks in the last decade. Besides analyzing recent deep learning works, we seek to explain the differences between non-deep learning and deep models, since these works often share similar intuitions and observations, and some of the previous studies could inspire further research. For consolidating these recent advances, we propose a new taxonomy for fine-grained part segmentation and recognition tasks, and also provide a collection of predominant benchmark datasets following our taxonomy. Besides these improvements compared to other survey papers~\cite{zhao2017survey,wei2021fine}, in this paper, we start from the novel view of~part relationship learning and regard it as the correspondences of different fine-grained sub-tasks. In this view, we revisit both the individual and universal challenges of part segmentation and fine-grained recognition and make an attempted solution using the guidance of~part relationship learning. In addition to these insights, we finish by discussing the future directions of fine-grained visual parsing tasks.&#10;&#10;To summarize, the main contributions of this survey are as follows:&#10;\begin{enumerate}&#10;\item We present a comprehensive survey of fine-grained visual parsing tasks by collecting recent advances of two representative tasks,~\ie, semantic part segmentation and fine-grained recognition.&#10;&#10;\item We revisit these fine-grained visual tasks from a novel perspective of part relationship learning, by revealing the connections of these fine-grained tasks and providing a promising solution to tackle challenges in fine-grained tasks.&#10;&#10;\item We consolidate recent fine-grained research by reorganizing these works with new taxonomy, providing a collection of prevailing benchmark datasets, and make comprehensive discussions to inspire future works.&#10;&#10;\item We provide promising future directions of fine-grained visual parsing tasks to inform further studies.&#10;\end{enumerate}&#10;&#10;The remainder of this paper is organized as follows:~Section \ref{sec:partseg} provides new taxonomies, benchmark settings, and recent research on the problem of fine-grained part segmentation.~Section \ref{sec:fgvc} consolidates benchmarks, challenges, and advanced research on fine-grained object recognition. In~Section \ref{sec:partrelation}, we delve into the connections of different fine-grained visual tasks from the perspective of part relationship learning and provide new solutions to improve existing challenges. We then highlight the future directions in~Section \ref{sec:direction} and then conclude this paper in~Section \ref{sec:conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Part Relationship Learning, Image Processing, Fine-grained Visual Parsing, Computer Science, Object Recognition Techniques" />
-        </attvalues>
-      </node>
-      <node id="1511.04510" label="1511.04510">
-        <attvalues>
-          <attvalue for="0" value="Semantic Object Parsing with Local-Global Long Short-Term Memory" />
-          <attvalue for="1" value="  Semantic object parsing is a fundamental task for understanding objects in&#10;detail in computer vision community, where incorporating multi-level contextual&#10;information is critical for achieving such fine-grained pixel-level&#10;recognition. Prior methods often leverage the contextual information through&#10;post-processing predicted confidence maps. In this work, we propose a novel&#10;deep Local-Global Long Short-Term Memory (LG-LSTM) architecture to seamlessly&#10;incorporate short-distance and long-distance spatial dependencies into the&#10;feature learning over all pixel positions. In each LG-LSTM layer, local&#10;guidance from neighboring positions and global guidance from the whole image&#10;are imposed on each position to better exploit complex local and global&#10;contextual information. Individual LSTMs for distinct spatial dimensions are&#10;also utilized to intrinsically capture various spatial layouts of semantic&#10;parts in the images, yielding distinct hidden and memory cells of each position&#10;for each dimension. In our parsing approach, several LG-LSTM layers are stacked&#10;and appended to the intermediate convolutional layers to directly enhance&#10;visual features, allowing network parameters to be learned in an end-to-end&#10;way. The long chains of sequential computation by stacked LG-LSTM layers also&#10;enable each pixel to sense a much larger region for inference benefiting from&#10;the memorization of previous dependencies in all positions along all&#10;dimensions. Comprehensive evaluations on three public datasets well demonstrate&#10;the significant superiority of our LG-LSTM over other state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.03383" label="1708.03383">
-        <attvalues>
-          <attvalue for="0" value="Joint Multi-Person Pose Estimation and Semantic Part Segmentation" />
-          <attvalue for="1" value="  Human pose estimation and semantic part segmentation are two complementary&#10;tasks in computer vision. In this paper, we propose to solve the two tasks&#10;jointly for natural multi-person images, in which the estimated pose provides&#10;object-level shape prior to regularize part segments while the part-level&#10;segments constrain the variation of pose locations. Specifically, we first&#10;train two fully convolutional neural networks (FCNs), namely Pose FCN and Part&#10;FCN, to provide initial estimation of pose joint potential and semantic part&#10;potential. Then, to refine pose joint location, the two types of potentials are&#10;fused with a fully-connected conditional random field (FCRF), where a novel&#10;segment-joint smoothness term is used to encourage semantic and spatial&#10;consistency between parts and joints. To refine part segments, the refined pose&#10;and the original part potential are integrated through a Part FCN, where the&#10;skeleton feature from pose serves as additional regularization cues for part&#10;segments. Finally, to reduce the complexity of the FCRF, we induce human&#10;detection boxes and infer the graph inside each box, making the inference forty&#10;times faster.&#10;  Since there's no dataset that contains both part segments and pose labels, we&#10;extend the PASCAL VOC part dataset with human pose joints and perform extensive&#10;experiments to compare our method against several most recent strategies. We&#10;show that on this dataset our algorithm surpasses competing methods by a large&#10;margin in both tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.08086" label="1512.08086">
-        <attvalues>
-          <attvalue for="0" value="Part-Stacked CNN for Fine-Grained Visual Categorization" />
-          <attvalue for="1" value="  In the context of fine-grained visual categorization, the ability to&#10;interpret models as human-understandable visual manuals is sometimes as&#10;important as achieving high classification accuracy. In this paper, we propose&#10;a novel Part-Stacked CNN architecture that explicitly explains the fine-grained&#10;recognition process by modeling subtle differences from object parts. Based on&#10;manually-labeled strong part annotations, the proposed architecture consists of&#10;a fully convolutional network to locate multiple object parts and a two-stream&#10;classification network that en- codes object-level and part-level cues&#10;simultaneously. By adopting a set of sharing strategies between the computation&#10;of multiple object parts, the proposed architecture is very efficient running&#10;at 20 frames/sec during inference. Experimental results on the CUB-200-2011&#10;dataset reveal the effectiveness of the proposed architecture, from both the&#10;perspective of classification accuracy and model interpretability.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.10411" label="2005.10411">
-        <attvalues>
-          <attvalue for="0" value="Interpretable and Accurate Fine-grained Recognition via Region Grouping" />
-          <attvalue for="1" value="  We present an interpretable deep model for fine-grained visual recognition.&#10;At the core of our method lies the integration of region-based part discovery&#10;and attribution within a deep neural network. Our model is trained using&#10;image-level object labels, and provides an interpretation of its results via&#10;the segmentation of object parts and the identification of their contributions&#10;towards classification. To facilitate the learning of object parts without&#10;direct supervision, we explore a simple prior of the occurrence of object&#10;parts. We demonstrate that this prior, when combined with our region-based part&#10;discovery and attribution, leads to an interpretable model that remains highly&#10;accurate. Our model is evaluated on major fine-grained recognition datasets,&#10;including CUB-200, CelebA and iNaturalist. Our results compare favorably to&#10;state-of-the-art methods on classification tasks, and our method outperforms&#10;previous approaches on the localization of object parts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.05569" label="2103.05569">
-        <attvalues>
-          <attvalue for="0" value="FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in&#10;  High-Resolution Remote Sensing Imagery" />
-          <attvalue for="1" value="  With the rapid development of deep learning, many deep learning-based&#10;approaches have made great achievements in object detection task. It is&#10;generally known that deep learning is a data-driven method. Data directly&#10;impact the performance of object detectors to some extent. Although existing&#10;datasets have included common objects in remote sensing images, they still have&#10;some limitations in terms of scale, categories, and images. Therefore, there is&#10;a strong requirement for establishing a large-scale benchmark on object&#10;detection in high-resolution remote sensing images. In this paper, we propose a&#10;novel benchmark dataset with more than 1 million instances and more than 15,000&#10;images for Fine-grAined object recognItion in high-Resolution remote sensing&#10;imagery which is named as FAIR1M. All objects in the FAIR1M dataset are&#10;annotated with respect to 5 categories and 37 sub-categories by oriented&#10;bounding boxes. Compared with existing detection datasets dedicated to object&#10;detection, the FAIR1M dataset has 4 particular characteristics: (1) it is much&#10;larger than other existing object detection datasets both in terms of the&#10;quantity of instances and the quantity of images, (2) it provides more rich&#10;fine-grained category information for objects in remote sensing images, (3) it&#10;contains geographic information such as latitude, longitude and resolution, (4)&#10;it provides better image quality owing to a careful data cleaning procedure. To&#10;establish a baseline for fine-grained object recognition, we propose a novel&#10;evaluation method and benchmark fine-grained object detection tasks and a&#10;visual classification task using several State-Of-The-Art (SOTA) deep&#10;learning-based models on our FAIR1M dataset. Experimental results strongly&#10;indicate that the FAIR1M dataset is closer to practical application and it is&#10;considerably more challenging than existing datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.08580" label="1703.08580">
-        <attvalues>
-          <attvalue for="0" value="Deep Residual Learning for Instrument Segmentation in Robotic Surgery" />
-          <attvalue for="1" value="  Detection, tracking, and pose estimation of surgical instruments are crucial&#10;tasks for computer assistance during minimally invasive robotic surgery. In the&#10;majority of cases, the first step is the automatic segmentation of surgical&#10;tools. Prior work has focused on binary segmentation, where the objective is to&#10;label every pixel in an image as tool or background. We improve upon previous&#10;work in two major ways. First, we leverage recent techniques such as deep&#10;residual learning and dilated convolutions to advance binary-segmentation&#10;performance. Second, we extend the approach to multi-class segmentation, which&#10;lets us segment different parts of the tool, in addition to background. We&#10;demonstrate the performance of this method on the MICCAI Endoscopic Vision&#10;Challenge Robotic Instruments dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.2375" label="1406.2375">
-        <attvalues>
-          <attvalue for="0" value="Parsing Semantic Parts of Cars Using Graphical Models and Segment&#10;  Appearance Consistency" />
-          <attvalue for="1" value="  This paper addresses the problem of semantic part parsing (segmentation) of&#10;cars, i.e.assigning every pixel within the car to one of the parts (e.g.body,&#10;window, lights, license plates and wheels). We formulate this as a landmark&#10;identification problem, where a set of landmarks specifies the boundaries of&#10;the parts. A novel mixture of graphical models is proposed, which dynamically&#10;couples the landmarks to a hierarchy of segments. When modeling pairwise&#10;relation between landmarks, this coupling enables our model to exploit the&#10;local image contents in addition to spatial deformation, an aspect that most&#10;existing graphical models ignore. In particular, our model enforces appearance&#10;consistency between segments within the same part. Parsing the car, including&#10;finding the optimal coupling between landmarks and segments in the hierarchy,&#10;is performed by dynamic programming. We evaluate our method on a subset of&#10;PASCAL VOC 2010 car images and on the car subset of 3D Object Category dataset&#10;(CAR3D). We show good results and, in particular, quantify the effectiveness of&#10;using the segment appearance consistency in terms of accuracy of part&#10;localization and segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.06119" label="2111.06119">
-        <attvalues>
-          <attvalue for="0" value="Fine-Grained Image Analysis with Deep Learning: A Survey" />
-          <attvalue for="1" value="  Fine-grained image analysis (FGIA) is a longstanding and fundamental problem&#10;in computer vision and pattern recognition, and underpins a diverse set of&#10;real-world applications. The task of FGIA targets analyzing visual objects from&#10;subordinate categories, e.g., species of birds or models of cars. The small&#10;inter-class and large intra-class variation inherent to fine-grained image&#10;analysis makes it a challenging problem. Capitalizing on advances in deep&#10;learning, in recent years we have witnessed remarkable progress in deep&#10;learning powered FGIA. In this paper we present a systematic survey of these&#10;advances, where we attempt to re-define and broaden the field of FGIA by&#10;consolidating two fundamental fine-grained research areas -- fine-grained image&#10;recognition and fine-grained image retrieval. In addition, we also review other&#10;key issues of FGIA, such as publicly available benchmark datasets and related&#10;domain-specific applications. We conclude by highlighting several research&#10;directions and open problems which need further exploration from the community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.11831" label="2011.11831">
-        <attvalues>
-          <attvalue for="0" value="Dissecting Image Crops" />
-          <attvalue for="1" value="  The elementary operation of cropping underpins nearly every computer vision&#10;system, ranging from data augmentation and translation invariance to&#10;computational photography and representation learning. This paper investigates&#10;the subtle traces introduced by this operation. For example, despite&#10;refinements to camera optics, lenses will leave behind certain clues, notably&#10;chromatic aberration and vignetting. Photographers also leave behind other&#10;clues relating to image aesthetics and scene composition. We study how to&#10;detect these traces, and investigate the impact that cropping has on the image&#10;distribution. While our aim is to dissect the fundamental impact of spatial&#10;crops, there are also a number of practical implications to our work, such as&#10;revealing faulty photojournalism and equipping neural network researchers with&#10;a better understanding of shortcut learning. Code is available at&#10;https://github.com/basilevh/dissecting-image-crops.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The basic operation of cropping an image underpins nearly every computer vision paper that you will be reading this week.&#10;Within the first few lectures of most introductory computer vision courses, convolutions are motivated as enabling feature invariance to spatial shifts and cropping \cite{brown_course_cv, stanford_course_cv, mit_course_cv}. Neural networks rely on image crops as a form of data augmentation \cite{krizhevsky2017imagenet, szegedy2015going, he2016deep}. Computational photography applications will automatically crop photos in order to improve their aesthetics \cite{samii2015data, chen2016automatic, zeng2019reliable}.&#10;Predictive models extrapolate pixels out from crops \cite{teterwak2019boundless, wang2019wide, van2019image}.&#10;Even the latest self-supervised efforts depend on crops for contrastive learning to induce rich visual representations \cite{chen2020simple,he2020momentum,pathak2016context,selvaraju2020casting}.&#10;&#10;This core visual operation can have a significant impact on photographs. As Oliva and Torralba told us twenty years ago, scene context drives perception \cite{oliva2001modeling}. Recently, image cropping has been at the heart of media disinformation.&#10;Figure \ref{fig:teaser_answer} shows two popular photographs where the photographer or media organization spatially cropped out part of the context, altering the message of the image.&#10;Twitter's auto-crop feature relied on a saliency prediction network that was racially biased \cite{canales_2020}.&#10;&#10;The guiding question of this paper is to understand the traces left behind from this fundamental operation. What impact does image cropping have on the visual distribution? Can we determine when and how a photo has been cropped?&#10;&#10;Despite extensive refinements to the manufacturing process of camera optics and sensors, nearly every modern camera pipeline will leave behind subtle lens artefacts onto the photos that it captures. For example, vignetting is caused by a lens focusing more light at the center of the sensor, creating images that are slightly brighter in the middle than near its borders \cite{lopez2015revisiting}. Chromatic aberration, also known as purple fringing, is caused by the lens focusing each wave length differently \cite{beeson2007patterns}. Since these artefacts are correlated with their spatial position in the image plane, they cause image crops to have trace signatures. &#10;&#10;Physical aberrations are not the only traces left behind during the operation. Photographers will prefer to take photos of interesting objects and in canonical poses \cite{torralba2011unbiased, barbu2019objectnet, hendrycks2019natural}.&#10;Aesthetically pleasing shots will have sensible compositions that respect symmetry and certain ratios in the scene. Violating these principles leaves behind another trace of the cropping operation.&#10;&#10;These traces are very subtle,&#10;and the human eye often cannot detect them, which makes studying and characterizing them challenging. However, neural networks are excellent at identifying these patterns. Indeed, extensive effort goes into preventing neural networks from learning such shortcuts enabled by image crops \cite{doersch2015unsupervised, noroozi2017representation}.&#10;&#10;In this paper, we flip this around and declare that these shortcuts are not bugs, but instead an opportunity to dissect and understand the subtle clues left behind from image cropping. Capitalizing on a large, high-quality collection of natural images, we train a convolutional neural network to predict the absolute spatial location of a patch within an image. This is only possible if there exist visual features that are not spatially invariant. Our experiments analyze the types of features that this model learns, and we show that it is possible to detect traces of the cropping operation. We can also use the discovered artefacts, along with semantic information, to recover where the crop was positioned in the original sensor plane.&#10;&#10;While the aim of this paper is to analyze the fundamental traces of image cropping in order to question conventional assumptions about translational invariance and the crucial role of data augmentation pervading the field, we believe our investigation could have a large practical impact as well. Historically, asking fundamental questions has spurred significant insight into core computer vision problems, such as invariances to scale \cite{burt1983laplacian}, asymmetries in time \cite{pickup2014seeing}, the speediness of videos \cite{benaim2020speednet}, and visual chirality \cite{lin2020visual}. For example, insight into image crops could enable detection of soft tampering operations, or spur developments to mitigate shortcut learning.&#10;&#10;In Section 2, we briefly review related work in lens imperfections and self-supervised learning.&#10;In Section 3, we describe the principles that govern our curation of an appropriate dataset.&#10;In Section 4, we learn a representation for detecting absolute patch locations, and subsequently motivate the complete network architecture that incorporates both aggregated patch-wise and global context.&#10;In Section 5, we analyze what our model has learned by measuring its performance under various controlled circumstances.&#10;Finally, in Section 6, we visualize and interpret how the model addresses several interesting examples of image crops.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Photography Forensics, Mathematics, Image Processing, Physics, Image Analysis, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1908.07007" label="1908.07007">
-        <attvalues>
-          <attvalue for="0" value="Boundless: Generative Adversarial Networks for Image Extension" />
-          <attvalue for="1" value="  Image extension models have broad applications in image editing,&#10;computational photography and computer graphics. While image inpainting has&#10;been extensively studied in the literature, it is challenging to directly apply&#10;the state-of-the-art inpainting methods to image extension as they tend to&#10;generate blurry or repetitive pixels with inconsistent semantics. We introduce&#10;semantic conditioning to the discriminator of a generative adversarial network&#10;(GAN), and achieve strong results on image extension with coherent semantics&#10;and visually pleasing colors and textures. We also show promising results in&#10;extreme extensions, such as panorama generation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.04630" label="2012.04630">
-        <attvalues>
-          <attvalue for="0" value="CASTing Your Model: Learning to Localize Improves Self-Supervised&#10;  Representations" />
-          <attvalue for="1" value="  Recent advances in self-supervised learning (SSL) have largely closed the gap&#10;with supervised ImageNet pretraining. Despite their success these methods have&#10;been primarily applied to unlabeled ImageNet images, and show marginal gains&#10;when trained on larger sets of uncurated images. We hypothesize that current&#10;SSL methods perform best on iconic images, and struggle on complex scene images&#10;with many objects. Analyzing contrastive SSL methods shows that they have poor&#10;visual grounding and receive poor supervisory signal when trained on scene&#10;images. We propose Contrastive Attention-Supervised Tuning(CAST) to overcome&#10;these limitations. CAST uses unsupervised saliency maps to intelligently sample&#10;crops, and to provide grounding supervision via a Grad-CAM attention loss.&#10;Experiments on COCO show that CAST significantly improves the features learned&#10;by SSL methods on scene images, and further experiments show that CAST-trained&#10;models are more robust to changes in backgrounds.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.05192" label="1505.05192">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Visual Representation Learning by Context Prediction" />
-          <attvalue for="1" value="  This work explores the use of spatial context as a source of free and&#10;plentiful supervisory signal for training a rich visual representation. Given&#10;only a large, unlabeled image collection, we extract random pairs of patches&#10;from each image and train a convolutional neural net to predict the position of&#10;the second patch relative to the first. We argue that doing well on this task&#10;requires the model to learn to recognize objects and their parts. We&#10;demonstrate that the feature representation learned using this within-image&#10;context indeed captures visual similarity across images. For example, this&#10;representation allows us to perform unsupervised visual discovery of objects&#10;like cats, people, and even birds from the Pascal VOC 2011 detection dataset.&#10;Furthermore, we show that the learned ConvNet can be used in the R-CNN&#10;framework and provides a significant boost over a randomly-initialized ConvNet,&#10;resulting in state-of-the-art performance among algorithms which use only&#10;Pascal-provided training set annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.06130" label="2004.06130">
-        <attvalues>
-          <attvalue for="0" value="SpeedNet: Learning the Speediness in Videos" />
-          <attvalue for="1" value="  We wish to automatically predict the &quot;speediness&quot; of moving objects in&#10;videos---whether they move faster, at, or slower than their &quot;natural&quot; speed.&#10;The core component in our approach is SpeedNet---a novel deep network trained&#10;to detect if a video is playing at normal rate, or if it is sped up. SpeedNet&#10;is trained on a large corpus of natural videos in a self-supervised manner,&#10;without requiring any manual annotations. We show how this single, binary&#10;classification network can be used to detect arbitrary rates of speediness of&#10;objects. We demonstrate prediction results by SpeedNet on a wide range of&#10;videos containing complex natural motions, and examine the visual cues it&#10;utilizes for making those predictions. Importantly, we show that through&#10;predicting the speed of videos, the model learns a powerful and meaningful&#10;space-time representation that goes beyond simple motion cues. We demonstrate&#10;how those learned features can boost the performance of self-supervised action&#10;recognition, and can be used for video retrieval. Furthermore, we also apply&#10;SpeedNet for generating time-varying, adaptive video speedups, which can allow&#10;viewers to watch videos faster, but with less of the jittery, unnatural motions&#10;typical to videos that are sped up uniformly.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.09512" label="2006.09512">
-        <attvalues>
-          <attvalue for="0" value="Visual Chirality" />
-          <attvalue for="1" value="  How can we tell whether an image has been mirrored? While we understand the&#10;geometry of mirror reflections very well, less has been said about how it&#10;affects distributions of imagery at scale, despite widespread use for data&#10;augmentation in computer vision. In this paper, we investigate how the&#10;statistics of visual data are changed by reflection. We refer to these changes&#10;as &quot;visual chirality&quot;, after the concept of geometric chirality - the notion of&#10;objects that are distinct from their mirror image. Our analysis of visual&#10;chirality reveals surprising results, including low-level chiral signals&#10;pervading imagery stemming from image processing in cameras, to the ability to&#10;discover visual chirality in images of people and faces. Our work has&#10;implications for data augmentation, self-supervised learning, and image&#10;forensics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.07510" label="2206.07510">
-        <attvalues>
-          <attvalue for="0" value="Deep Multi-Task Networks For Occluded Pedestrian Pose Estimation" />
-          <attvalue for="1" value="  Most of the existing works on pedestrian pose estimation do not consider&#10;estimating the pose of an occluded pedestrian, as the annotations of the&#10;occluded parts are not available in relevant automotive datasets. For example,&#10;CityPersons, a well-known dataset for pedestrian detection in automotive scenes&#10;does not provide pose annotations, whereas MS-COCO, a non-automotive dataset,&#10;contains human pose estimation. In this work, we propose a multi-task framework&#10;to extract pedestrian features through detection and instance segmentation&#10;tasks performed separately on these two distributions. Thereafter, an encoder&#10;learns pose specific features using an unsupervised instance-level domain&#10;adaptation method for the pedestrian instances from both distributions. The&#10;proposed framework has improved state-of-the-art performances of pose&#10;estimation, pedestrian detection, and instance segmentation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Human Pose Estimation (HPE)&#10;is a widely deployed computer vision application \cite{chen2020monocular}, along with semantic segmentation \cite{segdasvisapp19}, and object tracking and trajectory prediction \cite{sridevi2021object}. Addressing HPE for occluded pedestrians was first reported in ClueNet \cite{kishore2019cluenet} and later person-to-person occlusion was investigated in \cite{das2020end}. In this paper, we propose a novel end-to-end two stage fully convolutional network for the purpose of estimating pose of occluded pedestrians, in the context of a Multi-Task Learning (MTL) architecture where object detection and segmentation are performed. The proposed framework can 1) learn pedestrian pose estimation (PPE) from a dataset that does not contain pose annotations and 2) accurately estimate the pose of the occluded parts of the human body without the annotations of the same occluded parts. To address the issue of the non-availability of pose annotations during training in the CityPersons \cite{zhang2017citypersons} dataset, we consider a related dataset (MS-COCO \cite{lin2014microsoft}) to get adequate supervision specific to PPE during training. To achieve this, we apply unsupervised instance level domain adaptation on each pedestrian into the target domain. Here, the dataset with human pose annotations acts as the source domain, whereas the samples with only pedestrian detection and segmentation annotations become the target domain in domain adaptation. The main contributions of this study are: 1) the proposal of a two-stage end-to-end fully convolutional network to perform occluded PPE, 2) preserving the information of detection and segmentation in an MTL architecture, and 3) in achieving state-of-the-art performance on pedestrian detection, instance segmentation and PPE tasks respectively. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Pedestrian Pose Estimation, Instance Segmentation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.01423" label="2006.01423">
-        <attvalues>
-          <attvalue for="0" value="Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods" />
-          <attvalue for="1" value="  Vision-based monocular human pose estimation, as one of the most fundamental&#10;and challenging problems in computer vision, aims to obtain posture of the&#10;human body from input images or video sequences. The recent developments of&#10;deep learning techniques have been brought significant progress and remarkable&#10;breakthroughs in the field of human pose estimation. This survey extensively&#10;reviews the recent deep learning-based 2D and 3D human pose estimation methods&#10;published since 2014. This paper summarizes the challenges, main frameworks,&#10;benchmark datasets, evaluation metrics, performance comparison, and discusses&#10;some promising future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.06580" label="1901.06580">
-        <attvalues>
-          <attvalue for="0" value="Design of Real-time Semantic Segmentation Decoder for Automated Driving" />
-          <attvalue for="1" value="  Semantic segmentation remains a computationally intensive algorithm for&#10;embedded deployment even with the rapid growth of computation power. Thus&#10;efficient network design is a critical aspect especially for applications like&#10;automated driving which requires real-time performance. Recently, there has&#10;been a lot of research on designing efficient encoders that are mostly task&#10;agnostic. Unlike image classification and bounding box object detection tasks,&#10;decoders are computationally expensive as well for semantic segmentation task.&#10;In this work, we focus on efficient design of the segmentation decoder and&#10;assume that an efficient encoder is already designed to provide shared features&#10;for a multi-task learning system. We design a novel efficient non-bottleneck&#10;layer and a family of decoders which fit into a small run-time budget using&#10;VGG10 as efficient encoder. We demonstrate in our dataset that experimentation&#10;with various design choices led to an improvement of 10\% from a baseline&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.06697" label="2110.06697">
-        <attvalues>
-          <attvalue for="0" value="Semantic Image Fusion" />
-          <attvalue for="1" value="  Image fusion methods and metrics for their evaluation have conventionally&#10;used pixel-based or low-level features. However, for many applications, the aim&#10;of image fusion is to effectively combine the semantic content of the input&#10;images. This paper proposes a novel system for the semantic combination of&#10;visual content using pre-trained CNN network architectures. Our proposed&#10;semantic fusion is initiated through the fusion of the top layer feature map&#10;outputs (for each input image)through gradient updating of the fused image&#10;input (so-called image optimisation). Simple &quot;choose maximum&quot; and &quot;local&#10;majority&quot; filter based fusion rules are utilised for feature map fusion. This&#10;provides a simple method to combine layer outputs and thus a unique framework&#10;to fuse single-channel and colour images within a decomposition pre-trained for&#10;classification and therefore aligned with semantic fusion. Furthermore, class&#10;activation mappings of each input image are used to combine semantic&#10;information at a higher level. The developed methods are able to give&#10;equivalent low-level fusion performance to state of the art methods while&#10;providing a unique architecture to combine semantic information from multiple&#10;images.&#10;" />
-          <attvalue for="2" value="&#10;Image fusion is the combination of multiple images into a single image that aims to combine the most important visual information from all sources~\cite{hill2002image}. Image fusion has been motivated by the need to improve visual representations, visualisation, scene understanding and situational awareness in multi-sensor and multi-camera applications such as remote sensing~\cite{GHASSEMIAN201675}, medicine~\cite{james2014medical} and surveillance~\cite{paramanandham2018multi}.&#10;&#10;Image fusion has been driven by device and sensor limitations. For example, not all important visual information can be captured by one type of sensor (e.g.\ IR, visible etc.) or within one single shooting setting (i.e.\ focus, angle etc.). Furthermore, complementary imaging modalities co-exist within domains such as remote sensing and medicine that contain very different and important visual information. The effective combination of all the visual information from all image sources is therefore the aim of image fusion. Such a combined image is often effective for subsequent tasks such as scene understanding and target recognition.&#10;&#10;Image fusion has been a highly researched area over the last half century. During this time image fusion is performed at or decision-level, feature-level and pixel-level~\cite{li2017pixel}. Simple signal processing based pixel-level image fusion has given excellent results in preceding decades and continues to be used due to its high-efficiency and lack of a need for training data~\cite{li2017pixel}. Pixel-level fusion can be further classified into decomposition based methods or Sparse Representation (SR) based methods~\cite{li2017pixel}. In decomposition based techniques, the input images are decomposed into transform domains using methods such as complex wavelets (DT-CWT)~\cite{hill2002image}, the Discrete Wavelet Transform (DWT)~\cite{ben2005multiscale} and the contourlet transform~\cite{yang2010image}. Within the transform domain, coefficients are combined using suitably defined fusion rules (such as weighted-averaging \cite{jang2012contrast} and &quot;choose maximum&quot; \cite{hu2012multiscale}). &#10;&#10;More recently, state-of-the-art image fusion techniques have focused on the use of network based methods~\cite{javan2021review, hermessi2021multimodal,li2020nestfuse,li2018infrared,shao2018remote,tang2018pixel, amin2019ensemble,xu2020u2fusion}. Due to the requirements of needing training data these methods are often domain focused with state-of-the-art results reported for IR/Visible fusion~\cite{li2020nestfuse, li2018infrared}, remote sensing (multispectral)~\cite{shao2018remote} and multi-focus areas~\cite{tang2018pixel, amin2019ensemble}. Recent work has also been focused on universal image fusion methods that can effectively combine multiple sources within all of these domains~\cite{xu2020u2fusion}. &#10;&#10;There have only been a very small number of previously developed fusion methods that utilise semantic information for image fusion~\cite{fernandez2018remote, fan2019semantic}. However, these methods do not use the joint classification and class activation maps to semantically fuse the input sources as proposed in our work. These previous works also only use semantic information in a very limited way.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Information Processing, Machine Learning, Engineering, Computer Vision, Image Fusion Methods, Mathematics, Image Processing, Deep Learning Architectures, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1401.0166" label="1401.0166">
-        <attvalues>
-          <attvalue for="0" value="Medical Image Fusion: A survey of the state of the art" />
-          <attvalue for="1" value="  Medical image fusion is the process of registering and combining multiple&#10;images from single or multiple imaging modalities to improve the imaging&#10;quality and reduce randomness and redundancy in order to increase the clinical&#10;applicability of medical images for diagnosis and assessment of medical&#10;problems. Multi-modal medical image fusion algorithms and devices have shown&#10;notable achievements in improving clinical accuracy of decisions based on&#10;medical images. This review article provides a factual listing of methods and&#10;summarizes the broad scientific challenges faced in the field of medical image&#10;fusion. We characterize the medical image fusion research based on (1) the&#10;widely used image fusion methods, (2) imaging modalities, and (3) imaging of&#10;organs that are under study. This review concludes that even though there&#10;exists several open ended technological and scientific challenges, the fusion&#10;of medical images has proved to be useful for advancing the clinical&#10;reliability of using medical imaging for medical diagnostics and analysis, and&#10;is a scientific discipline that has the potential to significantly grow in the&#10;coming years.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.00328" label="2007.00328">
-        <attvalues>
-          <attvalue for="0" value="NestFuse: An Infrared and Visible Image Fusion Architecture based on&#10;  Nest Connection and Spatial/Channel Attention Models" />
-          <attvalue for="1" value="  In this paper we propose a novel method for infrared and visible image fusion&#10;where we develop nest connection-based network and spatial/channel attention&#10;models. The nest connection-based network can preserve significant amounts of&#10;information from input data in a multi-scale perspective. The approach&#10;comprises three key elements: encoder, fusion strategy and decoder&#10;respectively. In our proposed fusion strategy, spatial attention models and&#10;channel attention models are developed that describe the importance of each&#10;spatial position and of each channel with deep features. Firstly, the source&#10;images are fed into the encoder to extract multi-scale deep features. The novel&#10;fusion strategy is then developed to fuse these features for each scale.&#10;Finally, the fused image is reconstructed by the nest connection-based decoder.&#10;Experiments are performed on publicly available datasets. These exhibit that&#10;our proposed approach has better fusion performance than other state-of-the-art&#10;methods. This claim is justified through both subjective and objective&#10;evaluation. The code of our fusion method is available at&#10;https://github.com/hli1221/imagefusion-nestfuse&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04541" label="2003.04541">
-        <attvalues>
-          <attvalue for="0" value="PBRnet: Pyramidal Bounding Box Refinement to Improve Object Localization&#10;  Accuracy" />
-          <attvalue for="1" value="  Many recently developed object detectors focused on coarse-to-fine framework&#10;which contains several stages that classify and regress proposals from&#10;coarse-grain to fine-grain, and obtains more accurate detection gradually.&#10;Multi-resolution models such as Feature Pyramid Network(FPN) integrate&#10;information of different levels of resolution and effectively improve the&#10;performance. Previous researches also have revealed that localization can be&#10;further improved by: 1) using fine-grained information which is more&#10;translational variant; 2) refining local areas which is more focused on local&#10;boundary information. Based on these principles, we designed a novel boundary&#10;refinement architecture to improve localization accuracy by combining&#10;coarse-to-fine framework with feature pyramid structure, named as Pyramidal&#10;Bounding Box Refinement network(PBRnet), which parameterizes gradually focused&#10;boundary areas of objects and leverages lower-level feature maps to extract&#10;finer local information when refining the predicted bounding boxes. Extensive&#10;experiments are performed on the MS-COCO dataset. The PBRnet brings a&#10;significant performance gains by roughly 3 point of $mAP$ when added to FPN or&#10;Libra R-CNN. Moreover, by treating Cascade R-CNN as a coarse-to-fine detector&#10;and replacing its localization branch by the regressor of PBRnet, it leads an&#10;extra performance improvement by 1.5 $mAP$, yielding a total performance&#10;boosting by as high as 5 point of $mAP$.&#10;" />
-          <attvalue for="2" value="&#10;Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}. Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}. It formulates the detection as two procedures:1)region proposal network produces high quality proposals by distinguishing anchors between foreground and background as well as regressing them to the assigned ground truth locations;2)anchor generated Region-of-Interests(RoIs) are fine classified and their coordinates are further refined by a constraint such as cross-entropy or Smooth L1 loss.&#10;&#10;Several researches have developed the coarse-to-fine framework to improve the performance of object detection models. Iterative bounding box regression\cite{gidaris2016attend} claims that a single workflow is not sufficient for detection and iteratively applies additional post-processing steps. Cascade R-CNN\cite{cai2018cascade}, which is served as one of the best single model detector, designs a multi-stage architecture by iteratively increasing the IoU threshold to train the detector(named as quality of the detector). Recently, Side-Aware Boundary Localization (SABL)\cite{wang2019side} demonstrates that local areas are more suitable for boundary refinement, and proposes a lightweight two step bucketing scheme for localization which searches the bucket where the boundary resides and then predicts the offsets of the bucket.&#10;&#10;Feature Pyramid Network(FPN)\cite{lin2017feature,pang2019libra,qin2019thundernet,liu2018path} is an effective backbone which uses a top-down architecture with lateral connections to build an in-network feature pyramid structure. It surpasses other detectors a large margin by using a multi-solution detector and assigning each proposal to an appropriate map according to its scale. The Faster R-CNN achieves excellent gains in both accuracy and speed by using FPN backbone for feature extraction.&#10;&#10;It is a fact that feature maps of different levels in hierarchical feature pyramid integrate information from different grains, the diverse information concealed in the pyramid structure provides a way to further improve the object detection performance.On the one hand, lower-level feature map with high resolution consists more information of an object, and thus are capable for extracting finer-grained description compare with higher-level feature maps. On the other hand, as pointed in \cite{cheng2018revisiting}, localization requires translational variant features, the information in the lower-level feature map is more sensitive to displacement and translation as the smaller stride compare with that in the higher-level feature maps. Therefore, the performance of localization could be improved by refining the predicted bounding boxes on more fine-grained feature maps after detecting the objects.&#10;&#10;In this paper, we combined coarse-to-fine framework with multi-resolution detector, and proposed a new localization architecture, Pyramidal Boundary Refinement Network(PBRnet), based on the Faster-R-CNN-FPN framework. It decomposes localization branch into a coarse-to-fine framework that each procedure extracts gradually focused local boundary areas and leverages feature maps of lower level(same level if it is already on the lowest level) to refine the proposal to its assigned ground truth. Similar to the Cascade R-CNN\cite{cai2018cascade}, PBRnet also iteratively refines the bounding boxes but there are three advantages of the PBRnet over the Cascade R-CNN as illustrated in Figure \ref{fig1}. Firstly, assuming a proposal is firstly assigned to the $ConvK$th feature map, different from the Cascade R-CNN in which all the stages adopt the $ConvK$th feature map to assign the proposals and the corresponding predicted boxes, PBRnet is able to leverage the fine-grained information by gradually mapping bounding boxes to low-level feature maps. Secondly, PBRnet extracts local boundary areas of objects for regression which is more precise and more focused on information related to the boundaries. Besides, PBRnet does not involve any process of re-sampling proposals and re-assigning ground truth, and only the top score proposals are selected and transferred for finer refinement. As a result, PBRnet brings less extra computational and memory costs. Furthermore, PBRnet is also independent of the detector architecture and backbones, and can be combined with almost all the state-of-the-art detector to further improve the localization accuracy.&#10;&#10;In this work, by using the Faster R-CNN-FPN\cite{lin2017feature,ren2015faster} and ResNet\cite{he2016deep} as backbone, PBRnet brings about 3 point of the $mAP$ gains on COCO2014 benchmark\cite{lin2014microsoft}. It achieves a comparable performance to the Cascade R-CNN\cite{cai2018cascade} but introduces much less extra memory and parameters. Furthermore, by combining the PBRnet with Cascade R-CNN, the $mAP$ is further increased by 1.5, yielding a performance boosting by as high as 5 point.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Image Localization, Mathematics, Artificial Intelligence, Object Detection, Deep Learning Architectures, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1711.07240" label="1711.07240">
-        <attvalues>
-          <attvalue for="0" value="MegDet: A Large Mini-Batch Object Detector" />
-          <attvalue for="1" value="  The improvements in recent CNN-based object detection works, from R-CNN [11],&#10;Fast/Faster R-CNN [10, 31] to recent Mask R-CNN [14] and RetinaNet [24], mainly&#10;come from new network, new framework, or novel loss design. But mini-batch&#10;size, a key factor in the training, has not been well studied. In this paper,&#10;we propose a Large MiniBatch Object Detector (MegDet) to enable the training&#10;with much larger mini-batch size than before (e.g. from 16 to 256), so that we&#10;can effectively utilize multiple GPUs (up to 128 in our experiments) to&#10;significantly shorten the training time. Technically, we suggest a learning&#10;rate policy and Cross-GPU Batch Normalization, which together allow us to&#10;successfully train a large mini-batch detector in much less time (e.g., from 33&#10;hours to 4 hours), and achieve even better accuracy. The MegDet is the backbone&#10;of our submission (mmAP 52.5%) to COCO 2017 Challenge, where we won the 1st&#10;place of Detection task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1602.07261" label="1602.07261">
-        <attvalues>
-          <attvalue for="0" value="Inception-v4, Inception-ResNet and the Impact of Residual Connections on&#10;  Learning" />
-          <attvalue for="1" value="  Very deep convolutional networks have been central to the largest advances in&#10;image recognition performance in recent years. One example is the Inception&#10;architecture that has been shown to achieve very good performance at relatively&#10;low computational cost. Recently, the introduction of residual connections in&#10;conjunction with a more traditional architecture has yielded state-of-the-art&#10;performance in the 2015 ILSVRC challenge; its performance was similar to the&#10;latest generation Inception-v3 network. This raises the question of whether&#10;there are any benefit in combining the Inception architecture with residual&#10;connections. Here we give clear empirical evidence that training with residual&#10;connections accelerates the training of Inception networks significantly. There&#10;is also some evidence of residual Inception networks outperforming similarly&#10;expensive Inception networks without residual connections by a thin margin. We&#10;also present several new streamlined architectures for both residual and&#10;non-residual Inception networks. These variations improve the single-frame&#10;recognition performance on the ILSVRC 2012 classification task significantly.&#10;We further demonstrate how proper activation scaling stabilizes the training of&#10;very wide residual Inception networks. With an ensemble of three residual and&#10;one Inception-v4, we achieve 3.08 percent top-5 error on the test set of the&#10;ImageNet classification (CLS) challenge&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.05431" label="1611.05431">
-        <attvalues>
-          <attvalue for="0" value="Aggregated Residual Transformations for Deep Neural Networks" />
-          <attvalue for="1" value="  We present a simple, highly modularized network architecture for image&#10;classification. Our network is constructed by repeating a building block that&#10;aggregates a set of transformations with the same topology. Our simple design&#10;results in a homogeneous, multi-branch architecture that has only a few&#10;hyper-parameters to set. This strategy exposes a new dimension, which we call&#10;&quot;cardinality&quot; (the size of the set of transformations), as an essential factor&#10;in addition to the dimensions of depth and width. On the ImageNet-1K dataset,&#10;we empirically show that even under the restricted condition of maintaining&#10;complexity, increasing cardinality is able to improve classification accuracy.&#10;Moreover, increasing cardinality is more effective than going deeper or wider&#10;when we increase the capacity. Our models, named ResNeXt, are the foundations&#10;of our entry to the ILSVRC 2016 classification task in which we secured 2nd&#10;place. We further investigate ResNeXt on an ImageNet-5K set and the COCO&#10;detection set, also showing better results than its ResNet counterpart. The&#10;code and models are publicly available online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.03540" label="1604.03540">
-        <attvalues>
-          <attvalue for="0" value="Training Region-based Object Detectors with Online Hard Example Mining" />
-          <attvalue for="1" value="  The field of object detection has made significant advances riding on the&#10;wave of region-based ConvNets, but their training procedure still includes many&#10;heuristics and hyperparameters that are costly to tune. We present a simple yet&#10;surprisingly effective online hard example mining (OHEM) algorithm for training&#10;region-based ConvNet detectors. Our motivation is the same as it has always&#10;been -- detection datasets contain an overwhelming number of easy examples and&#10;a small number of hard examples. Automatic selection of these hard examples can&#10;make training more effective and efficient. OHEM is a simple and intuitive&#10;algorithm that eliminates several heuristics and hyperparameters in common use.&#10;But more importantly, it yields consistent and significant boosts in detection&#10;performance on benchmarks like PASCAL VOC 2007 and 2012. Its effectiveness&#10;increases as datasets become larger and more difficult, as demonstrated by the&#10;results on the MS COCO dataset. Moreover, combined with complementary advances&#10;in the field, OHEM leads to state-of-the-art results of 78.9% and 76.3% mAP on&#10;PASCAL VOC 2007 and 2012 respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.06211" label="1703.06211">
-        <attvalues>
-          <attvalue for="0" value="Deformable Convolutional Networks" />
-          <attvalue for="1" value="  Convolutional neural networks (CNNs) are inherently limited to model&#10;geometric transformations due to the fixed geometric structures in its building&#10;modules. In this work, we introduce two new modules to enhance the&#10;transformation modeling capacity of CNNs, namely, deformable convolution and&#10;deformable RoI pooling. Both are based on the idea of augmenting the spatial&#10;sampling locations in the modules with additional offsets and learning the&#10;offsets from target tasks, without additional supervision. The new modules can&#10;readily replace their plain counterparts in existing CNNs and can be easily&#10;trained end-to-end by standard back-propagation, giving rise to deformable&#10;convolutional networks. Extensive experiments validate the effectiveness of our&#10;approach on sophisticated vision tasks of object detection and semantic&#10;segmentation. The code would be released.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.04446" label="1606.04446">
-        <attvalues>
-          <attvalue for="0" value="Attend Refine Repeat: Active Box Proposal Generation via In-Out&#10;  Localization" />
-          <attvalue for="1" value="  The problem of computing category agnostic bounding box proposals is utilized&#10;as a core component in many computer vision tasks and thus has lately attracted&#10;a lot of attention. In this work we propose a new approach to tackle this&#10;problem that is based on an active strategy for generating box proposals that&#10;starts from a set of seed boxes, which are uniformly distributed on the image,&#10;and then progressively moves its attention on the promising image areas where&#10;it is more likely to discover well localized bounding box proposals. We call&#10;our approach AttractioNet and a core component of it is a CNN-based category&#10;agnostic object location refinement module that is capable of yielding accurate&#10;and robust bounding box predictions regardless of the object category.&#10;  We extensively evaluate our AttractioNet approach on several image datasets&#10;(i.e. COCO, PASCAL, ImageNet detection and NYU-Depth V2 datasets) reporting on&#10;all of them state-of-the-art results that surpass the previous work in the&#10;field by a significant margin and also providing strong empirical evidence that&#10;our approach is capable to generalize to unseen categories. Furthermore, we&#10;evaluate our AttractioNet proposals in the context of the object detection task&#10;using a VGG16-Net based detector and the achieved detection performance on COCO&#10;manages to significantly surpass all other VGG16-Net based detectors while even&#10;being competitive with a heavily tuned ResNet-101 based detector. Code as well&#10;as box proposals computed for several datasets are available at::&#10;https://github.com/gidariss/AttractioNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.04260" label="1912.04260">
-        <attvalues>
-          <attvalue for="0" value="Side-Aware Boundary Localization for More Precise Object Detection" />
-          <attvalue for="1" value="  Current object detection frameworks mainly rely on bounding box regression to&#10;localize objects. Despite the remarkable progress in recent years, the&#10;precision of bounding box regression remains unsatisfactory, hence limiting&#10;performance in object detection. We observe that precise localization requires&#10;careful placement of each side of the bounding box. However, the mainstream&#10;approach, which focuses on predicting centers and sizes, is not the most&#10;effective way to accomplish this task, especially when there exists&#10;displacements with large variance between the anchors and the targets. In this&#10;paper, we propose an alternative approach, named as Side-Aware Boundary&#10;Localization (SABL), where each side of the bounding box is respectively&#10;localized with a dedicated network branch. To tackle the difficulty of precise&#10;localization in the presence of displacements with large variance, we further&#10;propose a two-step localization scheme, which first predicts a range of&#10;movement through bucket prediction and then pinpoints the precise position&#10;within the predicted bucket. We test the proposed method on both two-stage and&#10;single-stage detection frameworks. Replacing the standard bounding box&#10;regression branch with the proposed design leads to significant improvements on&#10;Faster R-CNN, RetinaNet, and Cascade R-CNN, by 3.0%, 1.7%, and 0.9%,&#10;respectively. Code is available at https://github.com/open-mmlab/mmdetection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.02701" label="1904.02701">
-        <attvalues>
-          <attvalue for="0" value="Libra R-CNN: Towards Balanced Learning for Object Detection" />
-          <attvalue for="1" value="  Compared with model architectures, the training process, which is also&#10;crucial to the success of detectors, has received relatively less attention in&#10;object detection. In this work, we carefully revisit the standard training&#10;practice of detectors, and find that the detection performance is often limited&#10;by the imbalance during the training process, which generally consists in three&#10;levels - sample level, feature level, and objective level. To mitigate the&#10;adverse effects caused thereby, we propose Libra R-CNN, a simple but effective&#10;framework towards balanced learning for object detection. It integrates three&#10;novel components: IoU-balanced sampling, balanced feature pyramid, and balanced&#10;L1 loss, respectively for reducing the imbalance at sample, feature, and&#10;objective level. Benefitted from the overall balanced design, Libra R-CNN&#10;significantly improves the detection performance. Without bells and whistles,&#10;it achieves 2.5 points and 2.0 points higher Average Precision (AP) than FPN&#10;Faster R-CNN and RetinaNet respectively on MSCOCO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.10621" label="2103.10621">
-        <attvalues>
-          <attvalue for="0" value="Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement" />
-          <attvalue for="1" value="  Low-light image enhancement aims to improve an image's visibility while&#10;keeping its visual naturalness. Different from existing methods tending to&#10;accomplish the relighting task directly by ignoring the fidelity and&#10;naturalness recovery, we investigate the intrinsic degradation and relight the&#10;low-light image while refining the details and color in two steps. Inspired by&#10;the color image formulation (diffuse illumination color plus environment&#10;illumination color), we first estimate the degradation from low-light inputs to&#10;simulate the distortion of environment illumination color, and then refine the&#10;content to recover the loss of diffuse illumination color. To this end, we&#10;propose a novel Degradation-to-Refinement Generation Network (DRGN). Its&#10;distinctive features can be summarized as 1) A novel two-step generation&#10;network for degradation learning and content refinement. It is not only&#10;superior to one-step methods, but also capable of synthesizing sufficient&#10;paired samples to benefit the model training; 2) A multi-resolution fusion&#10;network to represent the target information (degradation or contents) in a&#10;multi-scale cooperative manner, which is more effective to address the complex&#10;unmixing problems. Extensive experiments on both the enhancement task and joint&#10;detection task have verified the effectiveness and efficiency of our proposed&#10;method, surpassing the SOTA by \textit{0.70dB on average and 3.18\% in mAP},&#10;respectively. The code is available at&#10;\url{https://github.com/kuijiang0802/DRGN}.&#10;" />
-          <attvalue for="2" value="&#10;Low-light conditions cause a series of visibility degradation and even sometimes destroy the color or content of the image. Such signal distortion and detail loss often lead to the failure of many computer vision systems~\cite{Bae2019aaai,HuangLYWRH2018AAAI,yang2020mutualnet,yu2021towards,&#10;xu2021rethinking,zhong2021grayscale,huang2021occluded,xu2021exploring}. Therefore, there is a pressing need to develop algorithms to produce normally exposed outputs.&#10;&#10;Early low-light enhancement works mainly focus on contrast enhancement to recover the visibility of dark regions~\cite{land1977retinex}. Since the hand-crafted priors and assumptions are introduced for specific scenes or degradation conditions, these techniques~\cite{kaur2011survey} are less effective on scenarios when the predefined models do not hold. Recently, deep learning frameworks have emerged as a promising solution to low-light image enhancement~\cite{zhu2020eemefn, chen2018learning, zhang2021beyond, jiang2021enlightengan, zhu2020eemefn}. The existing methods can be roughly divided into two categories regarding the imaging models under low-light conditions.&#10;&#10;\tabcolsep=0.5pt&#10;&#10;Scheme One: An observed low-light image $I_{L}$ is featured as a superposition of a normal-light image $I$ with some perturbations $I_N$ (noise, exposure, etc.) through a special combination manner $\theta(\cdot)$, defined as&#10;\begin{equation}&#10;I_{L} = \theta(I, I_N).&#10;\end{equation}&#10;Given $I_{L}$, the goal of low-light image enhancement is to predict $I$. When $\theta(\cdot)$ is simplified as the pixel-wise summation~\cite{gharbi2017deep}, the effort is to learn the perturbation $I_N$ and then subtract it from the observed image $I_{L}$. Some methods~\cite{Lv2018MBLLEN,yang2020fidelity} also attempt to directly estimate the optimal approximation of $I$. However, these methods rely heavily on abundant data, innovative architectures, and training strategies, ignoring the intrinsic degradation itself, resulting in unsatisfactory and unnatural contents in texture, color, contrast, etc.&#10;&#10;Scheme Two: The low-light image $I_{L}$ is mathematically modeled as a combination of the reflectance $I_R$ and illumination $I_{T}$ component through Eq.~\ref{eq:D2}:&#10;\begin{equation}&#10;I_{L} = I_{R}\circ I_{T},&#10;\end{equation}&#10;where $\circ$ denotes the element-wise product. With this kind of scheme, an image can be enhanced either by estimating and adjusting the illumination map~\cite{li2018lightennet,wang2019underexposed}, or by learning joint features of these two components~\cite{Zhang2019Kindling}. It is known as the Retinex based method~\cite{land1977retinex, wei2018deep}. These methods follow the layer decomposition paradigm and show impressive results in stretching contrast and removing noise in some cases, but still have three problems. 1) Traditional models rely on properly enforced priors and regularization. 2) Deep RetinexNets require a large number of paired training samples. 3) The enhanced reflectance $I_R$ is often treated as the normal-light image~\cite{yue2017contrast}, which often fails to produce high-fidelity and naturalness results due to the gap between the ideal situation and reality with simplified imaging models.&#10;&#10;In sum, there are two main drawbacks of these two schemes. First, their performances heavily rely on the diversity and quality of synthetic training samples. Second, they simplify the imaging model when coping with the enhancement task, thus sacrificing representation precision, especially when being trained on a small dataset. More specifically, since the degradation additionally damages the texture, color, and contrast of normal-light images, we argue that subtracting the perturbation $I_N$ from its low-light input $I_{L}$ directly (scheme one) cannot fully recover multiple kinds of fine details and color distribution (refer to MIRNet~\cite{zamir2020learning} in Figure~\ref{fig:sample}). Meanwhile, adjusting the illumination map to enhance reflectance $I_R$ (scheme two) cannot fully keep the visual naturalness of the predicted normal-light image&#10;(refer to RetinexNet~\cite{wei2018deep} and DeepUPE~\cite{wang2019underexposed} in Figure~\ref{fig:sample}).&#10;&#10;In light of the color image formulation, an image is composed of the diffuse illumination color and the environment illumination color, determined by the light source, albedo (material), environment light, noise, etc~\cite{wu2017interactive}. We thus promote scheme one by considering the blending relations between the normal-light image $I$ and the degradation $I_{D}$, denoted as $f(I_{D}, I)$. The newly designed low-light image generation process can be modeled as&#10;\begin{equation}&#10;I_{L} = I_{D} + I + f(I, I_{D}) = I_{D} + \psi(I_{D}, I). \end{equation}&#10;We argue ($I$, $f(I_{D}, I)$) represents the intrinsic manifold projection from $I$ to $I_L$ via degradation $I_{D}$. We formulate this complex mapping as a high-dimensional non-analytic transfer map $\psi(I, I_{D})$ since there is no explicit analytic function to present the blending relations caused by the albedo (material) ($I$), environment light, noise, etc ($I_{D}$).&#10;&#10;By doing so, we believe the degradation $I_{D}$ can be learned from the low-light input $I_{L}$ via a degradation generator (DeG), formulated as $\phi(I_{L})$. Thus, given $I_L$, we use Eq.~\ref{eq:DeAndRe} to predict the normal-light image $I$ as&#10;\begin{equation}&#10;I = \psi^{-1}(I_{L} - \phi(I_{L})).&#10;\end{equation} To alleviate the learning difficulty, our enhancement task is then decomposed into two stages: (1) simulating the degradation $I_{D}$ via DeG, and (2) refining the color and contrast information by a refinement generator (ReG), which is parameterized by $\psi^{-1}(\cdot)$ (learning the inverse representation of $\psi(\cdot)$). More specifically, inspired by the impressive results of GAN in image synthesis and translation~\cite{GatysEB15,bulat2018to}, we train a parametric generator to learn the degradation factors from the low-light input in the first stage, denoted as $\phi(I_{L})$. Note that simulated degradation factors are also applied to another referenced high-quality dataset to generate synthetic low-light samples to mitigate the limitation of sample monotonicity and repetitiveness (More details on the data generation are described in Sec.~\ref{sec:DA}). Meanwhile, we produce the base enhancement result $I_B$ by removing the predicted degradation from the low-light input. Thus, the second stage takes $I_B$ as input, and applies ReG to estimate the inverse transformation matrix to further refine the color and textural details.&#10;&#10;DeG is implemented with a multi-resolution fusion strategy because the degradation always acts on various texture richness levels. We thus propose a multi-resolution fusion network (MFN) to learn the joint representation of multi-scale features. In the second stage, ReG shares the same architecture as DeG for simplicity, since the cooperative representation among the multi-scale features also contributes more to the recovery of image textures.&#10;Our main contributions are summarized as follows:&#10;\begin{itemize}&#10; \item We propose to decompose and characterize the low-light degradation. Thanks to the degradation generator, it can augment an arbitrary number of paired samples using a small synthetic dataset, and achieves impressive performance and robustness. Our proposed degradation formulation can be easily extended to other enhancement tasks, to help get rid of paired dataset collection. Based on this, we construct a novel two-stage Degradation-to-Refinement Generation Network (DRGN) by cascading DeG and ReG for low-light image enhancement tasks.&#10; \item Besides achieving impressive enhancement performance (surpassing KinD++~\cite{zhang2021beyond} by 0.70dB in PSNR on average), our method is also competitive on the joint low-light image enhancement and detection tasks on ExDark~\cite{Exdark} and NightSurveillance~\cite{NightSurveillance} datasets.&#10; \item We also introduce two novel low/normal-light datasets (COCO24700 for training and COCO1000 for evaluation) based on COCO~\cite{caesar2018coco} dataset, which can be used for the joint low-light image enhancement and detection tasks under extreme night-time degradation scenarios. \end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Physics, Image Processing, Deep Learning, Computer Science, Image Enhancement" />
-        </attvalues>
-      </node>
-      <node id="1901.08225" label="1901.08225">
-        <attvalues>
-          <attvalue for="0" value="Object Detection based on Region Decomposition and Assembly" />
-          <attvalue for="1" value="  Region-based object detection infers object regions for one or more&#10;categories in an image. Due to the recent advances in deep learning and region&#10;proposal methods, object detectors based on convolutional neural networks&#10;(CNNs) have been flourishing and provided the promising detection results.&#10;However, the detection accuracy is degraded often because of the low&#10;discriminability of object CNN features caused by occlusions and inaccurate&#10;region proposals. In this paper, we therefore propose a region decomposition&#10;and assembly detector (R-DAD) for more accurate object detection.&#10;  In the proposed R-DAD, we first decompose an object region into multiple&#10;small regions. To capture an entire appearance and part details of the object&#10;jointly, we extract CNN features within the whole object region and decomposed&#10;regions. We then learn the semantic relations between the object and its parts&#10;by combining the multi-region features stage by stage with region assembly&#10;blocks, and use the combined and high-level semantic features for the object&#10;classification and localization. In addition, for more accurate region&#10;proposals, we propose a multi-scale proposal layer that can generate object&#10;proposals of various scales. We integrate the R-DAD into several feature&#10;extractors, and prove the distinct performance improvement on PASCAL07/12 and&#10;MSCOCO18 compared to the recent convolutional detectors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.12978" label="1909.12978">
-        <attvalues>
-          <attvalue for="0" value="MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and&#10;  Resolution" />
-          <attvalue for="1" value="  We propose the width-resolution mutual learning method (MutualNet) to train a&#10;network that is executable at dynamic resource constraints to achieve adaptive&#10;accuracy-efficiency trade-offs at runtime. Our method trains a cohort of&#10;sub-networks with different widths using different input resolutions to&#10;mutually learn multi-scale representations for each sub-network. It achieves&#10;consistently better ImageNet top-1 accuracy over the state-of-the-art adaptive&#10;network US-Net under different computation constraints, and outperforms the&#10;best compound scaled MobileNet in EfficientNet by 1.5%. The superiority of our&#10;method is also validated on COCO object detection and instance segmentation as&#10;well as transfer learning. Surprisingly, the training strategy of MutualNet can&#10;also boost the performance of a single network, which substantially outperforms&#10;the powerful AutoAugmentation in both efficiency (GPU search hours: 15000 vs.&#10;0) and accuracy (ImageNet: 77.6% vs. 78.6%). Code is available at&#10;\url{https://github.com/taoyang1122/MutualNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.03822" label="2011.03822">
-        <attvalues>
-          <attvalue for="0" value="Towards Resolving the Challenge of Long-tail Distribution in UAV Images&#10;  for Object Detection" />
-          <attvalue for="1" value="  Existing methods for object detection in UAV images ignored an important&#10;challenge - imbalanced class distribution in UAV images - which leads to poor&#10;performance on tail classes. We systematically investigate existing solutions&#10;to long-tail problems and unveil that re-balancing methods that are effective&#10;on natural image datasets cannot be trivially applied to UAV datasets. To this&#10;end, we rethink long-tailed object detection in UAV images and propose the Dual&#10;Sampler and Head detection Network (DSHNet), which is the first work that aims&#10;to resolve long-tail distribution in UAV images. The key components in DSHNet&#10;include Class-Biased Samplers (CBS) and Bilateral Box Heads (BBH), which are&#10;developed to cope with tail classes and head classes in a dual-path manner.&#10;Without bells and whistles, DSHNet significantly boosts the performance of tail&#10;classes on different detection frameworks. Moreover, DSHNet significantly&#10;outperforms base detectors and generic approaches for long-tail problems on&#10;VisDrone and UAVDT datasets. It achieves new state-of-the-art performance when&#10;combining with image cropping methods. Code is available at&#10;https://github.com/we1pingyu/DSHNet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.16124" label="2007.16124">
-        <attvalues>
-          <attvalue for="0" value="Exploring Image Enhancement for Salient Object Detection in Low Light&#10;  Images" />
-          <attvalue for="1" value="  Low light images captured in a non-uniform illumination environment usually&#10;are degraded with the scene depth and the corresponding environment lights.&#10;This degradation results in severe object information loss in the degraded&#10;image modality, which makes the salient object detection more challenging due&#10;to low contrast property and artificial light influence. However, existing&#10;salient object detection models are developed based on the assumption that the&#10;images are captured under a sufficient brightness environment, which is&#10;impractical in real-world scenarios. In this work, we propose an image&#10;enhancement approach to facilitate the salient object detection in low light&#10;images. The proposed model directly embeds the physical lighting model into the&#10;deep neural network to describe the degradation of low light images, in which&#10;the environment light is treated as a point-wise variate and changes with local&#10;content. Moreover, a Non-Local-Block Layer is utilized to capture the&#10;difference of local content of an object against its local neighborhood&#10;favoring regions. To quantitative evaluation, we construct a low light Images&#10;dataset with pixel-level human-labeled ground-truth annotations and report&#10;promising results on four public datasets and our benchmark dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.02880" label="1707.02880">
-        <attvalues>
-          <attvalue for="0" value="Deep Bilateral Learning for Real-Time Image Enhancement" />
-          <attvalue for="1" value="  Performance is a critical challenge in mobile image processing. Given a&#10;reference imaging pipeline, or even human-adjusted pairs of images, we seek to&#10;reproduce the enhancements and enable real-time evaluation. For this, we&#10;introduce a new neural network architecture inspired by bilateral grid&#10;processing and local affine color transforms. Using pairs of input/output&#10;images, we train a convolutional neural network to predict the coefficients of&#10;a locally-affine model in bilateral space. Our architecture learns to make&#10;local, global, and content-dependent decisions to approximate the desired image&#10;transformation. At runtime, the neural network consumes a low-resolution&#10;version of the input image, produces a set of affine transformations in&#10;bilateral space, upsamples those transformations in an edge-preserving fashion&#10;using a new slicing node, and then applies those upsampled transformations to&#10;the full-resolution image. Our algorithm processes high-resolution images on a&#10;smartphone in milliseconds, provides a real-time viewfinder at 1080p&#10;resolution, and matches the quality of state-of-the-art approximation&#10;techniques on a large class of image operators. Unlike previous work, our model&#10;is trained off-line from data and therefore does not require access to the&#10;original operator at runtime. This allows our model to learn complex,&#10;scene-dependent transformations for which no reference implementation is&#10;available, such as the photographic edits of a human retoucher.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.04161" label="1905.04161">
-        <attvalues>
-          <attvalue for="0" value="Kindling the Darkness: A Practical Low-light Image Enhancer" />
-          <attvalue for="1" value="  Images captured under low-light conditions often suffer from (partially) poor&#10;visibility. Besides unsatisfactory lightings, multiple types of degradations,&#10;such as noise and color distortion due to the limited quality of cameras, hide&#10;in the dark. In other words, solely turning up the brightness of dark regions&#10;will inevitably amplify hidden artifacts. This work builds a simple yet&#10;effective network for \textbf{Kin}dling the \textbf{D}arkness (denoted as&#10;KinD), which, inspired by Retinex theory, decomposes images into two&#10;components. One component (illumination) is responsible for light adjustment,&#10;while the other (reflectance) for degradation removal. In such a way, the&#10;original space is decoupled into two smaller subspaces, expecting to be better&#10;regularized/learned. It is worth to note that our network is trained with&#10;paired images shot under different exposure conditions, instead of using any&#10;ground-truth reflectance and illumination information. Extensive experiments&#10;are conducted to demonstrate the efficacy of our design and its superiority&#10;over state-of-the-art alternatives. Our KinD is robust against severe visual&#10;defects, and user-friendly to arbitrarily adjust light levels. In addition, our&#10;model spends less than 50ms to process an image in VGA resolution on a 2080Ti&#10;GPU. All the above merits make our KinD attractive for practical use.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.06792" label="2003.06792">
-        <attvalues>
-          <attvalue for="0" value="Learning Enriched Features for Real Image Restoration and Enhancement" />
-          <attvalue for="1" value="  With the goal of recovering high-quality image content from its degraded&#10;version, image restoration enjoys numerous applications, such as in&#10;surveillance, computational photography, medical imaging, and remote sensing.&#10;Recently, convolutional neural networks (CNNs) have achieved dramatic&#10;improvements over conventional approaches for image restoration task. Existing&#10;CNN-based methods typically operate either on full-resolution or on&#10;progressively low-resolution representations. In the former case, spatially&#10;precise but contextually less robust results are achieved, while in the latter&#10;case, semantically reliable but spatially less accurate outputs are generated.&#10;In this paper, we present a novel architecture with the collective goals of&#10;maintaining spatially-precise high-resolution representations through the&#10;entire network and receiving strong contextual information from the&#10;low-resolution representations. The core of our approach is a multi-scale&#10;residual block containing several key elements: (a) parallel multi-resolution&#10;convolution streams for extracting multi-scale features, (b) information&#10;exchange across the multi-resolution streams, (c) spatial and channel attention&#10;mechanisms for capturing contextual information, and (d) attention based&#10;multi-scale feature aggregation. In a nutshell, our approach learns an enriched&#10;set of features that combines contextual information from multiple scales,&#10;while simultaneously preserving the high-resolution spatial details. Extensive&#10;experiments on five real image benchmark datasets demonstrate that our method,&#10;named as MIRNet, achieves state-of-the-art results for a variety of image&#10;processing tasks, including image denoising, super-resolution, and image&#10;enhancement. The source code and pre-trained models are available at&#10;https://github.com/swz30/MIRNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.11458" label="1807.11458">
-        <attvalues>
-          <attvalue for="0" value="To learn image super-resolution, use a GAN to learn how to do image&#10;  degradation first" />
-          <attvalue for="1" value="  This paper is on image and face super-resolution. The vast majority of prior&#10;work for this problem focus on how to increase the resolution of low-resolution&#10;images which are artificially generated by simple bilinear down-sampling (or in&#10;a few cases by blurring followed by down-sampling).We show that such methods&#10;fail to produce good results when applied to real-world low-resolution, low&#10;quality images. To circumvent this problem, we propose a two-stage process&#10;which firstly trains a High-to-Low Generative Adversarial Network (GAN) to&#10;learn how to degrade and downsample high-resolution images requiring, during&#10;training, only unpaired high and low-resolution images. Once this is achieved,&#10;the output of this network is used to train a Low-to-High GAN for image&#10;super-resolution using this time paired low- and high-resolution images. Our&#10;main result is that this network can be now used to efectively increase the&#10;quality of real-world low-resolution images. We have applied the proposed&#10;pipeline for the problem of face super-resolution where we report large&#10;improvement over baselines and prior work although the proposed method is&#10;potentially applicable to other object categories.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.03716" label="1612.03716">
-        <attvalues>
-          <attvalue for="0" value="COCO-Stuff: Thing and Stuff Classes in Context" />
-          <attvalue for="1" value="  Semantic classes can be either things (objects with a well-defined shape,&#10;e.g. car, person) or stuff (amorphous background regions, e.g. grass, sky).&#10;While lots of classification and detection works focus on thing classes, less&#10;attention has been given to stuff classes. Nonetheless, stuff classes are&#10;important as they allow to explain important aspects of an image, including (1)&#10;scene type; (2) which thing classes are likely to be present and their location&#10;(through contextual reasoning); (3) physical attributes, material types and&#10;geometric properties of the scene. To understand stuff and things in context we&#10;introduce COCO-Stuff, which augments all 164K images of the COCO 2017 dataset&#10;with pixel-wise annotations for 91 stuff classes. We introduce an efficient&#10;stuff annotation protocol based on superpixels, which leverages the original&#10;thing annotations. We quantify the speed versus quality trade-off of our&#10;protocol and explore the relation between annotation time and boundary&#10;complexity. Furthermore, we use COCO-Stuff to analyze: (a) the importance of&#10;stuff and thing classes in terms of their surface cover and how frequently they&#10;are mentioned in image captions; (b) the spatial relations between stuff and&#10;things, highlighting the rich contextual relations that make our dataset&#10;unique; (c) the performance of a modern semantic segmentation method on stuff&#10;and thing classes, and whether stuff is easier to segment than things.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.02881" label="2307.02881">
-        <attvalues>
-          <attvalue for="0" value="Probabilistic and Semantic Descriptions of Image Manifolds and Their&#10;  Applications" />
-          <attvalue for="1" value="  This paper begins with a description of methods for estimating image&#10;probability density functions that reflects the observation that such data is&#10;usually constrained to lie in restricted regions of the high-dimensional image&#10;space-not every pattern of pixels is an image. It is common to say that images&#10;lie on a lower-dimensional manifold in the high-dimensional space. However, it&#10;is not the case that all points on the manifold have an equal probability of&#10;being images. Images are unevenly distributed on the manifold, and our task is&#10;to devise ways to model this distribution as a probability distribution. We&#10;therefore consider popular generative models. For our purposes,&#10;generative/probabilistic models should have the properties of 1) sample&#10;generation: the possibility to sample from this distribution with the modelled&#10;density function, and 2) probability computation: given a previously unseen&#10;sample from the dataset of interest, one should be able to compute its&#10;probability, at least up to a normalising constant. To this end, we investigate&#10;the use of methods such as normalising flow and diffusion models. We then show&#10;how semantic interpretations are used to describe points on the manifold. To&#10;achieve this, we consider an emergent language framework that uses variational&#10;encoders for a disentangled representation of points that reside on a given&#10;manifold. Trajectories between points on a manifold can then be described as&#10;evolving semantic descriptions. We also show that such probabilistic&#10;descriptions (bounded) can be used to improve semantic consistency by&#10;constructing defences against adversarial attacks. We evaluate our methods with&#10;improved semantic robustness and OoD detection capability, explainable and&#10;editable semantic interpolation, and improved classification accuracy under&#10;patch attacks. We also discuss the limitation in diffusion models.&#10;" />
-          <attvalue for="2" value="&#10;Understanding the complex probability distribution of the data is essential&#10;for&#10;image authenticity and quality analysis, but is challenging due to its high&#10;dimensionality and intricate domain variations~&#10;\cite{manifold_dimension,manifold_geo}. Seen images usually have high&#10;probabilities on a low-dimensional manifold embedded in the higher-dimensional&#10;space of the image encoder.&#10;\revision{&#10;Nevertheless, the phenomenon that image embeddings encoded using methods such as&#10;a pretrained CLIP encoder~\cite{dall_e} lie within a narrow cone of the unit&#10;sphere instead of the entire sphere~\cite{cone_1,cone_2}, which degrades the&#10;aforementioned pattern of probability distribution.&#10;Hence, on such a manifold, it is unlikely that} every point can&#10;be&#10;decoded into a realistic image because of the unevenly distributed&#10;probabilities. Therefore, it is important to compute the probability in the&#10;latent space to indicate whether the corresponding image is in a high-density&#10;region of the space~&#10;\cite{manifold_mle,manifold_flow_flow,manifold_flow_gan,nf_prob,manifold_gan,manifold_hyperbolic,manifold_flow_beyond}. &#10;This helps to&#10;distinguish seen images from unseen images, or synthetic images from real&#10;images.&#10;Some works train a discriminator with positive (real) and negative&#10;(synthetic) examples in the manner of contrastive&#10;learning~\cite{detect_gen_by_real} or analyse their frequency&#10;differences~\cite{frequency}. However, they do not address this problem using&#10;the probabilistic framework afforded by modern generative models.&#10;&#10;In this work, we calculate the exact log-probability of an image by&#10;utilising generative models that assign high probabilities to seen images&#10;and low probabilities to unseen images.&#10;\revision{The confidence of such probabilities is usually related to&#10;image fidelity,&#10;we hence also introduce efficient and effective (with improved semantic robustness)&#10;generation strategies using hierarchical&#10;structure and large sampling steps with the Runge-Kutta method&#10;(RK4)~\cite{RK41,RK42} for stabilisation.}&#10;Specifically, we use normalising flow&#10;(NF)~\cite{vae_nf,nf_prob} and diffusion&#10;models (DMs)~\cite{ddpm,ddpm_understand,ddim} as image generators. NF models&#10;learn an image embedding space that conforms to a predefined distribution,&#10;usually a Gaussian.&#10;In contrast, DMs diffuse images with Gaussian noise in each&#10;forward step and learn denoising gradients for the backward steps.&#10;A random sample from the Gaussian distribution can then be&#10;analytically represented on an image manifold&#10;and visualised through an image decoder (for NF models) or denoiser (for&#10;diffusion models).&#10;\revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2}&#10;and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have&#10;been explored in model training.&#10;To the best of our knowledge, however, it has not been studied by investigating such&#10;likelihood distribution of seen and unseen images with a hierarchical&#10;structure (without losing the image quality) from the manifold perspective.&#10;This is also applied to the diffusion models noting the&#10;difficulty of combining such exact likelihood with the mean squared error&#10;loss in diffusion training.}&#10;&#10;\revision{Samples from these image generators} can be thought of having several meaningful semantic attributes.&#10;It is often desirable that these attributes be orthogonal to each other&#10;in the sample latent space so as to achieve a controllable and&#10;interpretable representation.&#10;In this work, \revision{we disentangle semantics&#10;in the latent space by using a variational autoencoder&#10;(VAE)~\cite{kingma2013auto} in the framework of emergent languages&#10;(EL)~&#10;\cite{havrylov2017emergence,el_control,el_communication,el_dynamic,el_miccai}. &#10;This allows the latent representation on the manifold to be more&#10;robust, interpretable, compositional, controllable, and transferable.&#10;\revision{Although some VAE variant models such as $\beta$-TCVAE~\cite{tcvae},&#10;GuidedVAE~\cite{guidedvae}, and DCVAE~\cite{dcvae} achieve qualified semantic&#10;disentanglement results,&#10;we mainly focus on understanding the effectiveness of the emergent language&#10;framework for VAE based disentanglement inspired by \cite{el_model} and&#10;emphasizing the feasibility of our GridVAE (with mixture of Gaussian priors)&#10;under such an EL framework to study semantic distributions on the image manifold.}&#10;We also evaluate their semantic robustness on such a manifold against&#10;adversarial and patch attacks~&#10;\cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using&#10;semantic consistency with a purification loss.}&#10;&#10;We organise this paper into three sections, each with their own&#10;experiments: log-likelihood estimation for a given image under normalising&#10;flows and diffusion models (see Section~\ref{sec:mle}),&#10;\revision{semantic disentanglement in emergent languages&#10;for a latent representation of object attributes, using a proposed&#10;GridVAE model (see Section~\ref{sec:el}), and adversarial&#10;attacks and defences in image space to preserve semantics (see&#10;Section~\ref{sec:attacks}).}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Probability Modeling, Machine Learning, Computer Vision, Semantic Interpretation, Mathematics, Artificial Intelligence, Generative Models, Computer Science, Statistics" />
-        </attvalues>
-      </node>
-      <node id="2104.08894" label="2104.08894">
-        <attvalues>
-          <attvalue for="0" value="The Intrinsic Dimension of Images and Its Impact on Learning" />
-          <attvalue for="1" value="  It is widely believed that natural image data exhibits low-dimensional&#10;structure despite the high dimensionality of conventional pixel&#10;representations. This idea underlies a common intuition for the remarkable&#10;success of deep learning in computer vision. In this work, we apply dimension&#10;estimation tools to popular datasets and investigate the role of&#10;low-dimensional structure in deep learning. We find that common natural image&#10;datasets indeed have very low intrinsic dimension relative to the high number&#10;of pixels in the images. Additionally, we find that low dimensional datasets&#10;are easier for neural networks to learn, and models solving these tasks&#10;generalize better from training to test data. Along the way, we develop a&#10;technique for validating our dimension estimation tools on synthetic data&#10;generated by GANs allowing us to actively manipulate the intrinsic dimension by&#10;controlling the image generation process. Code for our experiments may be found&#10;here https://github.com/ppope/dimensions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06151" label="1904.06151">
-        <attvalues>
-          <attvalue for="0" value="Geometry-Aware Maximum Likelihood Estimation of Intrinsic Dimension" />
-          <attvalue for="1" value="  The existing approaches to intrinsic dimension estimation usually are not&#10;reliable when the data are nonlinearly embedded in the high dimensional space.&#10;In this work, we show that the explicit accounting to geometric properties of&#10;unknown support leads to the polynomial correction to the standard maximum&#10;likelihood estimate of intrinsic dimension for flat manifolds. The proposed&#10;algorithm (GeoMLE) realizes the correction by regression of standard MLEs based&#10;on distances to nearest neighbors for different sizes of neighborhoods.&#10;Moreover, the proposed approach also efficiently handles the case of nonuniform&#10;sampling of the manifold. We perform numerous experiments on different&#10;synthetic and real-world datasets. The results show that our algorithm achieves&#10;state-of-the-art performance, while also being computationally efficient and&#10;robust to noise in the data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.12092" label="2102.12092">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Text-to-Image Generation" />
-          <attvalue for="1" value="  Text-to-image generation has traditionally focused on finding better modeling&#10;assumptions for training on a fixed dataset. These assumptions might involve&#10;complex architectures, auxiliary losses, or side information such as object&#10;part labels or segmentation masks supplied during training. We describe a&#10;simple approach for this task based on a transformer that autoregressively&#10;models the text and image tokens as a single stream of data. With sufficient&#10;data and scale, our approach is competitive with previous domain-specific&#10;models when evaluated in a zero-shot fashion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.12009" label="1907.12009">
-        <attvalues>
-          <attvalue for="0" value="Representation Degeneration Problem in Training Natural Language&#10;  Generation Models" />
-          <attvalue for="1" value="  We study an interesting problem in training neural network-based models for&#10;natural language generation tasks, which we call the \emph{representation&#10;degeneration problem}. We observe that when training a model for natural&#10;language generation tasks through likelihood maximization with the weight tying&#10;trick, especially with big training datasets, most of the learnt word&#10;embeddings tend to degenerate and be distributed into a narrow cone, which&#10;largely limits the representation power of word embeddings. We analyze the&#10;conditions and causes of this problem and propose a novel regularization method&#10;to address it. Experiments on language modeling and machine translation show&#10;that our method can largely mitigate the representation degeneration problem&#10;and achieve better performance than baseline algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.02487" label="2211.02487">
-        <attvalues>
-          <attvalue for="0" value="Flows for Flows: Training Normalizing Flows Between Arbitrary&#10;  Distributions with Maximum Likelihood Estimation" />
-          <attvalue for="1" value="  Normalizing flows are constructed from a base distribution with a known&#10;density and a diffeomorphism with a tractable Jacobian. The base density of a&#10;normalizing flow can be parameterised by a different normalizing flow, thus&#10;allowing maps to be found between arbitrary distributions. We demonstrate and&#10;explore the utility of this approach and show it is particularly interesting in&#10;the case of conditional normalizing flows and for introducing optimal transport&#10;constraints on maps that are constructed using normalizing flows.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.08868" label="1705.08868">
-        <attvalues>
-          <attvalue for="0" value="Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in&#10;  Generative Models" />
-          <attvalue for="1" value="  Adversarial learning of probabilistic models has recently emerged as a&#10;promising alternative to maximum likelihood. Implicit models such as generative&#10;adversarial networks (GAN) often generate better samples compared to explicit&#10;models trained by maximum likelihood. Yet, GANs sidestep the characterization&#10;of an explicit density which makes quantitative evaluations challenging. To&#10;bridge this gap, we propose Flow-GANs, a generative adversarial network for&#10;which we can perform exact likelihood evaluation, thus supporting both&#10;adversarial and maximum likelihood training. When trained adversarially,&#10;Flow-GANs generate high-quality samples but attain extremely poor&#10;log-likelihood scores, inferior even to a mixture model memorizing the training&#10;data; the opposite is true when trained by maximum likelihood. Results on MNIST&#10;and CIFAR-10 demonstrate that hybrid training can attain high held-out&#10;likelihoods while retaining visual fidelity in the generated samples.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.02762" label="1912.02762">
-        <attvalues>
-          <attvalue for="0" value="Normalizing Flows for Probabilistic Modeling and Inference" />
-          <attvalue for="1" value="  Normalizing flows provide a general mechanism for defining expressive&#10;probability distributions, only requiring the specification of a (usually&#10;simple) base distribution and a series of bijective transformations. There has&#10;been much recent work on normalizing flows, ranging from improving their&#10;expressive power to expanding their application. We believe the field has now&#10;matured and is in need of a unified perspective. In this review, we attempt to&#10;provide such a perspective by describing flows through the lens of&#10;probabilistic modeling and inference. We place special emphasis on the&#10;fundamental principles of flow design, and discuss foundational topics such as&#10;expressive power and computational trade-offs. We also broaden the conceptual&#10;framing of flows by relating them to more general probability transformations.&#10;Lastly, we summarize the use of flows for tasks such as generative modeling,&#10;approximate inference, and supervised learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.05468" label="2207.05468">
-        <attvalues>
-          <attvalue for="0" value="Sliced-Wasserstein normalizing flows: beyond maximum likelihood training" />
-          <attvalue for="1" value="  Despite their advantages, normalizing flows generally suffer from several&#10;shortcomings including their tendency to generate unrealistic data (e.g.,&#10;images) and their failing to detect out-of-distribution data. One reason for&#10;these deficiencies lies in the training strategy which traditionally exploits a&#10;maximum likelihood principle only. This paper proposes a new training paradigm&#10;based on a hybrid objective function combining the maximum likelihood principle&#10;(MLE) and a sliced-Wasserstein distance. Results obtained on synthetic toy&#10;examples and real image data sets show better generative abilities in terms of&#10;both likelihood and visual aspects of the generated samples. Reciprocally, the&#10;proposed approach leads to a lower likelihood of out-of-distribution data,&#10;demonstrating a greater data fidelity of the resulting flows.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.11035" label="1912.11035">
-        <attvalues>
-          <attvalue for="0" value="CNN-generated images are surprisingly easy to spot... for now" />
-          <attvalue for="1" value="  In this work we ask whether it is possible to create a &quot;universal&quot; detector&#10;for telling apart real images from these generated by a CNN, regardless of&#10;architecture or dataset used. To test this, we collect a dataset consisting of&#10;fake images generated by 11 different CNN-based image generator models, chosen&#10;to span the space of commonly used architectures today (ProGAN, StyleGAN,&#10;BigGAN, CycleGAN, StarGAN, GauGAN, DeepFakes, cascaded refinement networks,&#10;implicit maximum likelihood estimation, second-order attention&#10;super-resolution, seeing-in-the-dark). We demonstrate that, with careful pre-&#10;and post-processing and data augmentation, a standard image classifier trained&#10;on only one specific CNN generator (ProGAN) is able to generalize surprisingly&#10;well to unseen architectures, datasets, and training methods (including the&#10;just released StyleGAN2). Our findings suggest the intriguing possibility that&#10;today's CNN-generated images share some common systematic flaws, preventing&#10;them from achieving realistic image synthesis. Code and pre-trained networks&#10;are available at https://peterwang512.github.io/CNNDetection/ .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.11970" label="2208.11970">
-        <attvalues>
-          <attvalue for="0" value="Understanding Diffusion Models: A Unified Perspective" />
-          <attvalue for="1" value="  Diffusion models have shown incredible capabilities as generative models;&#10;indeed, they power the current state-of-the-art models on text-conditioned&#10;image generation such as Imagen and DALL-E 2. In this work we review,&#10;demystify, and unify the understanding of diffusion models across both&#10;variational and score-based perspectives. We first derive Variational Diffusion&#10;Models (VDM) as a special case of a Markovian Hierarchical Variational&#10;Autoencoder, where three key assumptions enable tractable computation and&#10;scalable optimization of the ELBO. We then prove that optimizing a VDM boils&#10;down to learning a neural network to predict one of three potential objectives:&#10;the original source input from any arbitrary noisification of it, the original&#10;source noise from any arbitrarily noisified input, or the score function of a&#10;noisified input at any arbitrary noise level. We then dive deeper into what it&#10;means to learn the score function, and connect the variational perspective of a&#10;diffusion model explicitly with the Score-based Generative Modeling perspective&#10;through Tweedie's Formula. Lastly, we cover how to learn a conditional&#10;distribution using diffusion models via guidance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07579" label="2110.07579">
-        <attvalues>
-          <attvalue for="0" value="Diffusion Normalizing Flow" />
-          <attvalue for="1" value="  We present a novel generative modeling method called diffusion normalizing&#10;flow based on stochastic differential equations (SDEs). The algorithm consists&#10;of two neural SDEs: a forward SDE that gradually adds noise to the data to&#10;transform the data into Gaussian random noise, and a backward SDE that&#10;gradually removes the noise to sample from the data distribution. By jointly&#10;training the two neural SDEs to minimize a common cost function that quantifies&#10;the difference between the two, the backward SDE converges to a diffusion&#10;process the starts with a Gaussian distribution and ends with the desired data&#10;distribution. Our method is closely related to normalizing flow and diffusion&#10;probabilistic models and can be viewed as a combination of the two. Compared&#10;with normalizing flow, diffusion normalizing flow is able to learn&#10;distributions with sharp boundaries. Compared with diffusion probabilistic&#10;models, diffusion normalizing flow requires fewer discretization steps and thus&#10;has better sampling efficiency. Our algorithm demonstrates competitive&#10;performance in both high-dimension data density estimation and image generation&#10;tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.00029" label="2010.00029">
-        <attvalues>
-          <attvalue for="0" value="RG-Flow: A hierarchical and explainable flow model based on&#10;  renormalization group and sparse prior" />
-          <attvalue for="1" value="  Flow-based generative models have become an important class of unsupervised&#10;learning approaches. In this work, we incorporate the key ideas of&#10;renormalization group (RG) and sparse prior distribution to design a&#10;hierarchical flow-based generative model, RG-Flow, which can separate&#10;information at different scales of images and extract disentangled&#10;representations at each scale. We demonstrate our method on synthetic&#10;multi-scale image datasets and the CelebA dataset, showing that the&#10;disentangled representations enable semantic manipulation and style mixing of&#10;the images at different scales. To visualize the latent representations, we&#10;introduce receptive fields for flow-based models and show that the receptive&#10;fields of RG-Flow are similar to those of convolutional neural networks. In&#10;addition, we replace the widely adopted isotropic Gaussian prior distribution&#10;by the sparse Laplacian distribution to further enhance the disentanglement of&#10;representations. From a theoretical perspective, our proposed method has&#10;$O(\log L)$ complexity for inpainting of an image with edge length $L$,&#10;compared to previous generative models with $O(L^2)$ complexity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.05301" label="2108.05301">
-        <attvalues>
-          <attvalue for="0" value="Hierarchical Conditional Flow: A Unified Framework for Image&#10;  Super-Resolution and Image Rescaling" />
-          <attvalue for="1" value="  Normalizing flows have recently demonstrated promising results for low-level&#10;vision tasks. For image super-resolution (SR), it learns to predict diverse&#10;photo-realistic high-resolution (HR) images from the low-resolution (LR) image&#10;rather than learning a deterministic mapping. For image rescaling, it achieves&#10;high accuracy by jointly modelling the downscaling and upscaling processes.&#10;While existing approaches employ specialized techniques for these two tasks, we&#10;set out to unify them in a single formulation. In this paper, we propose the&#10;hierarchical conditional flow (HCFlow) as a unified framework for image SR and&#10;image rescaling. More specifically, HCFlow learns a bijective mapping between&#10;HR and LR image pairs by modelling the distribution of the LR image and the&#10;rest high-frequency component simultaneously. In particular, the high-frequency&#10;component is conditional on the LR image in a hierarchical manner. To further&#10;enhance the performance, other losses such as perceptual loss and GAN loss are&#10;combined with the commonly used negative log-likelihood loss in training.&#10;Extensive experiments on general image SR, face image SR and image rescaling&#10;have demonstrated that the proposed HCFlow achieves state-of-the-art&#10;performance in terms of both quantitative metrics and visual quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;In this section, we will briefly review image SR and image rescaling with a particular focus on two highly related flow-based methods, \ie, SRFlow~\cite{lugmayr2020srflow} and IRN~\cite{xiao2020IRN}.&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="2106.08462" label="2106.08462">
-        <attvalues>
-          <attvalue for="0" value="Multi-Resolution Continuous Normalizing Flows" />
-          <attvalue for="1" value="  Recent work has shown that Neural Ordinary Differential Equations (ODEs) can&#10;serve as generative models of images using the perspective of Continuous&#10;Normalizing Flows (CNFs). Such models offer exact likelihood calculation, and&#10;invertible generation/density estimation. In this work we introduce a&#10;Multi-Resolution variant of such models (MRCNF), by characterizing the&#10;conditional distribution over the additional information required to generate a&#10;fine image that is consistent with the coarse image. We introduce a&#10;transformation between resolutions that allows for no change in the log&#10;likelihood. We show that this approach yields comparable likelihood values for&#10;various image datasets, with improved performance at higher resolutions, with&#10;fewer parameters, using only 1 GPU. Further, we examine the out-of-distribution&#10;properties of (Multi-Resolution) Continuous Normalizing Flows, and find that&#10;they are similar to those of other likelihood-based generative models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.11192" label="1705.11192">
-        <attvalues>
-          <attvalue for="0" value="Emergence of Language with Multi-agent Games: Learning to Communicate&#10;  with Sequences of Symbols" />
-          <attvalue for="1" value="  Learning to communicate through interaction, rather than relying on explicit&#10;supervision, is often considered a prerequisite for developing a general AI. We&#10;study a setting where two agents engage in playing a referential game and, from&#10;scratch, develop a communication protocol necessary to succeed in this game.&#10;Unlike previous work, we require that messages they exchange, both at train and&#10;test time, are in the form of a language (i.e. sequences of discrete symbols).&#10;We compare a reinforcement learning approach and one using a differentiable&#10;relaxation (straight-through Gumbel-softmax estimator) and observe that the&#10;latter is much faster to converge and it results in more effective protocols.&#10;Interestingly, we also observe that the protocol we induce by optimizing the&#10;communication success exhibits a degree of compositionality and variability&#10;(i.e. the same information can be phrased in different ways), both properties&#10;characteristic of natural languages. As the ultimate goal is to ensure that&#10;communication is accomplished in natural language, we also perform experiments&#10;where we inject prior information about natural language into our model and&#10;study properties of the resulting protocol.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.01828" label="2108.01828">
-        <attvalues>
-          <attvalue for="0" value="Emergent Discrete Communication in Semantic Spaces" />
-          <attvalue for="1" value="  Neural agents trained in reinforcement learning settings can learn to&#10;communicate among themselves via discrete tokens, accomplishing as a team what&#10;agents would be unable to do alone. However, the current standard of using&#10;one-hot vectors as discrete communication tokens prevents agents from acquiring&#10;more desirable aspects of communication such as zero-shot understanding.&#10;Inspired by word embedding techniques from natural language processing, we&#10;propose neural agent architectures that enables them to communicate via&#10;discrete tokens derived from a learned, continuous space. We show in a decision&#10;theoretic framework that our technique optimizes communication over a wide&#10;range of scenarios, whereas one-hot tokens are only optimal under restrictive&#10;assumptions. In self-play experiments, we validate that our trained agents&#10;learn to cluster tokens in semantically-meaningful ways, allowing them&#10;communicate in noisy environments where other techniques fail. Lastly, we&#10;demonstrate both that agents using our method can effectively respond to novel&#10;human communication and that humans can understand unlabeled emergent agent&#10;communication, outperforming the use of one-hot communication.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.09448" label="2007.09448">
-        <attvalues>
-          <attvalue for="0" value="Towards Emergent Language Symbolic Semantic Segmentation and Model&#10;  Interpretability" />
-          <attvalue for="1" value="  Recent advances in methods focused on the grounding problem have resulted in&#10;techniques that can be used to construct a symbolic language associated with a&#10;specific domain. Inspired by how humans communicate complex ideas through&#10;language, we developed a generalized Symbolic Semantic ($\text{S}^2$) framework&#10;for interpretable segmentation. Unlike adversarial models (e.g., GANs), we&#10;explicitly model cooperation between two agents, a Sender and a Receiver, that&#10;must cooperate to achieve a common goal. The Sender receives information from a&#10;high layer of a segmentation network and generates a symbolic sentence derived&#10;from a categorical distribution. The Receiver obtains the symbolic sentences&#10;and co-generates the segmentation mask. In order for the model to converge, the&#10;Sender and Receiver must learn to communicate using a private language. We&#10;apply our architecture to segment tumors in the TCGA dataset. A UNet-like&#10;architecture is used to generate input to the Sender network which produces a&#10;symbolic sentence, and a Receiver network co-generates the segmentation mask&#10;based on the sentence. Our Segmentation framework achieved similar or better&#10;performance compared with state-of-the-art segmentation methods. In addition,&#10;our results suggest direct interpretation of the symbolic sentences to&#10;discriminate between normal and tumor tissue, tumor morphology, and other image&#10;characteristics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.01255" label="2004.01255">
-        <attvalues>
-          <attvalue for="0" value="Guided Variational Autoencoder for Disentanglement Learning" />
-          <attvalue for="1" value="  We propose an algorithm, guided variational autoencoder (Guided-VAE), that is&#10;able to learn a controllable generative model by performing latent&#10;representation disentanglement learning. The learning objective is achieved by&#10;providing signals to the latent encoding/embedding in VAE without changing its&#10;main backbone architecture, hence retaining the desirable properties of the&#10;VAE. We design an unsupervised strategy and a supervised strategy in Guided-VAE&#10;and observe enhanced modeling and controlling capability over the vanilla VAE.&#10;In the unsupervised strategy, we guide the VAE learning by introducing a&#10;lightweight decoder that learns latent geometric transformation and principal&#10;components; in the supervised strategy, we use an adversarial excitation and&#10;inhibition mechanism to encourage the disentanglement of the latent variables.&#10;Guided-VAE enjoys its transparency and simplicity for the general&#10;representation learning task, as well as disentanglement learning. On a number&#10;of experiments for representation learning, improved synthesis/sampling, better&#10;disentanglement for classification, and reduced classification errors in&#10;meta-learning have been observed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.10063" label="2011.10063">
-        <attvalues>
-          <attvalue for="0" value="Dual Contradistinctive Generative Autoencoder" />
-          <attvalue for="1" value="  We present a new generative autoencoder model with dual contradistinctive&#10;losses to improve generative autoencoder that performs simultaneous inference&#10;(reconstruction) and synthesis (sampling). Our model, named dual&#10;contradistinctive generative autoencoder (DC-VAE), integrates an instance-level&#10;discriminative loss (maintaining the instance-level fidelity for the&#10;reconstruction/synthesis) with a set-level adversarial loss (encouraging the&#10;set-level fidelity for there construction/synthesis), both being&#10;contradistinctive. Extensive experimental results by DC-VAE across different&#10;resolutions including 32x32, 64x64, 128x128, and 512x512 are reported. The two&#10;contradistinctive losses in VAE work harmoniously in DC-VAE leading to a&#10;significant qualitative and quantitative performance enhancement over the&#10;baseline VAEs without architectural changes. State-of-the-art or competitive&#10;results among generative autoencoders for image reconstruction, image&#10;synthesis, image interpolation, and representation learning are observed.&#10;DC-VAE is a general-purpose VAE model, applicable to a wide variety of&#10;downstream tasks in computer vision and machine learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.00482" label="2210.00482">
-        <attvalues>
-          <attvalue for="0" value="Compositional Generalization in Unsupervised Compositional&#10;  Representation Learning: A Study on Disentanglement and Emergent Language" />
-          <attvalue for="1" value="  Deep learning models struggle with compositional generalization, i.e. the&#10;ability to recognize or generate novel combinations of observed elementary&#10;concepts. In hopes of enabling compositional generalization, various&#10;unsupervised learning algorithms have been proposed with inductive biases that&#10;aim to induce compositional structure in learned representations (e.g.&#10;disentangled representation and emergent language learning). In this work, we&#10;evaluate these unsupervised learning algorithms in terms of how well they&#10;enable compositional generalization. Specifically, our evaluation protocol&#10;focuses on whether or not it is easy to train a simple model on top of the&#10;learned representation that generalizes to new combinations of compositional&#10;factors. We systematically study three unsupervised representation learning&#10;algorithms - $\beta$-VAE, $\beta$-TCVAE, and emergent language (EL)&#10;autoencoders - on two datasets that allow directly testing compositional&#10;generalization. We find that directly using the bottleneck representation with&#10;simple models and few labels may lead to worse generalization than using&#10;representations from layers before or after the learned representation itself.&#10;In addition, we find that the previously proposed metrics for evaluating the&#10;levels of compositionality are not correlated with actual compositional&#10;generalization in our framework. Surprisingly, we find that increasing pressure&#10;to produce a disentangled representation produces representations with worse&#10;generalization, while representations from EL models show strong compositional&#10;generalization. Taken together, our results shed new light on the compositional&#10;generalization behavior of different unsupervised learning algorithms with a&#10;new setting to rigorously test this behavior, and suggest the potential&#10;benefits of delevoping EL learning algorithms for more generalizable&#10;representations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.09665" label="1712.09665">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Patch" />
-          <attvalue for="1" value="  We present a method to create universal, robust, targeted adversarial image&#10;patches in the real world. The patches are universal because they can be used&#10;to attack any scene, robust because they work under a wide variety of&#10;transformations, and targeted because they can cause a classifier to output any&#10;target class. These adversarial patches can be printed, added to any scene,&#10;photographed, and presented to image classifiers; even when the patches are&#10;small, they cause the classifiers to ignore the other items in the scene and&#10;report a chosen target class.&#10;  To reproduce the results from the paper, our code is available at&#10;https://github.com/tensorflow/cleverhans/tree/master/examples/adversarial_patch&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.09257" label="2005.09257">
-        <attvalues>
-          <attvalue for="0" value="Bias-based Universal Adversarial Patch Attack for Automatic Check-out" />
-          <attvalue for="1" value="  Adversarial examples are inputs with imperceptible perturbations that easily&#10;misleading deep neural networks(DNNs). Recently, adversarial patch, with noise&#10;confined to a small and localized patch, has emerged for its easy feasibility&#10;in real-world scenarios. However, existing strategies failed to generate&#10;adversarial patches with strong generalization ability. In other words, the&#10;adversarial patches were input-specific and failed to attack images from all&#10;classes, especially unseen ones during training. To address the problem, this&#10;paper proposes a bias-based framework to generate class-agnostic universal&#10;adversarial patches with strong generalization ability, which exploits both the&#10;perceptual and semantic bias of models. Regarding the perceptual bias, since&#10;DNNs are strongly biased towards textures, we exploit the hard examples which&#10;convey strong model uncertainties and extract a textural patch prior from them&#10;by adopting the style similarities. The patch prior is more close to decision&#10;boundaries and would promote attacks. To further alleviate the heavy dependency&#10;on large amounts of data in training universal attacks, we further exploit the&#10;semantic bias. As the class-wise preference, prototypes are introduced and&#10;pursued by maximizing the multi-class margin to help universal training. Taking&#10;AutomaticCheck-out (ACO) as the typical scenario, extensive experiments&#10;including white-box and black-box settings in both digital-world(RPC, the&#10;largest ACO related dataset) and physical-world scenario(Taobao and JD, the&#10;world' s largest online shopping platforms) are conducted. Experimental results&#10;demonstrate that our proposed framework outperforms state-of-the-art&#10;adversarial patch attack methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.10884" label="2005.10884">
-        <attvalues>
-          <attvalue for="0" value="PatchGuard: A Provably Robust Defense against Adversarial Patches via&#10;  Small Receptive Fields and Masking" />
-          <attvalue for="1" value="  Localized adversarial patches aim to induce misclassification in machine&#10;learning models by arbitrarily modifying pixels within a restricted region of&#10;an image. Such attacks can be realized in the physical world by attaching the&#10;adversarial patch to the object to be misclassified, and defending against such&#10;attacks is an unsolved/open problem. In this paper, we propose a general&#10;defense framework called PatchGuard that can achieve high provable robustness&#10;while maintaining high clean accuracy against localized adversarial patches.&#10;The cornerstone of PatchGuard involves the use of CNNs with small receptive&#10;fields to impose a bound on the number of features corrupted by an adversarial&#10;patch. Given a bounded number of corrupted features, the problem of designing&#10;an adversarial patch defense reduces to that of designing a secure feature&#10;aggregation mechanism. Towards this end, we present our robust masking defense&#10;that robustly detects and masks corrupted features to recover the correct&#10;prediction. Notably, we can prove the robustness of our defense against any&#10;adversary within our threat model. Our extensive evaluation on ImageNet,&#10;ImageNette (a 10-class subset of ImageNet), and CIFAR-10 datasets demonstrates&#10;that our defense achieves state-of-the-art performance in terms of both&#10;provable robust accuracy and clean accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.00292" label="1812.00292">
-        <attvalues>
-          <attvalue for="0" value="SentiNet: Detecting Localized Universal Attacks Against Deep Learning&#10;  Systems" />
-          <attvalue for="1" value="  SentiNet is a novel detection framework for localized universal attacks on&#10;neural networks. These attacks restrict adversarial noise to contiguous&#10;portions of an image and are reusable with different images -- constraints that&#10;prove useful for generating physically-realizable attacks. Unlike most other&#10;works on adversarial detection, SentiNet does not require training a model or&#10;preknowledge of an attack prior to detection. Our approach is appealing due to&#10;the large number of possible mechanisms and attack-vectors that an&#10;attack-specific defense would have to consider. By leveraging the neural&#10;network's susceptibility to attacks and by using techniques from model&#10;interpretability and object detection as detection mechanisms, SentiNet turns a&#10;weakness of a model into a strength. We demonstrate the effectiveness of&#10;SentiNet on three different attacks -- i.e., data poisoning attacks, trojaned&#10;networks, and adversarial patches (including physically realizable attacks) --&#10;and show that our defense is able to achieve very competitive performance&#10;metrics for all three threats. Finally, we show that SentiNet is robust against&#10;strong adaptive adversaries, who build adversarial patches that specifically&#10;target the components of SentiNet's architecture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.06083" label="1706.06083">
-        <attvalues>
-          <attvalue for="0" value="Towards Deep Learning Models Resistant to Adversarial Attacks" />
-          <attvalue for="1" value="  Recent work has demonstrated that deep neural networks are vulnerable to&#10;adversarial examples---inputs that are almost indistinguishable from natural&#10;data and yet classified incorrectly by the network. In fact, some of the latest&#10;findings suggest that the existence of adversarial attacks may be an inherent&#10;weakness of deep learning models. To address this problem, we study the&#10;adversarial robustness of neural networks through the lens of robust&#10;optimization. This approach provides us with a broad and unifying view on much&#10;of the prior work on this topic. Its principled nature also enables us to&#10;identify methods for both training and attacking neural networks that are&#10;reliable and, in a certain sense, universal. In particular, they specify a&#10;concrete security guarantee that would protect against any adversary. These&#10;methods let us train networks with significantly improved resistance to a wide&#10;range of adversarial attacks. They also suggest the notion of security against&#10;a first-order adversary as a natural and broad security guarantee. We believe&#10;that robustness against such well-defined classes of adversaries is an&#10;important stepping stone towards fully resistant deep learning models. Code and&#10;pre-trained models are available at https://github.com/MadryLab/mnist_challenge&#10;and https://github.com/MadryLab/cifar10_challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.04644" label="1608.04644">
-        <attvalues>
-          <attvalue for="0" value="Towards Evaluating the Robustness of Neural Networks" />
-          <attvalue for="1" value="  Neural networks provide state-of-the-art results for most machine learning&#10;tasks. Unfortunately, neural networks are vulnerable to adversarial examples:&#10;given an input $x$ and any target classification $t$, it is possible to find a&#10;new input $x'$ that is similar to $x$ but classified as $t$. This makes it&#10;difficult to apply neural networks in security-critical areas. Defensive&#10;distillation is a recently proposed approach that can take an arbitrary neural&#10;network, and increase its robustness, reducing the success rate of current&#10;attacks' ability to find adversarial examples from $95\%$ to $0.5\%$.&#10;  In this paper, we demonstrate that defensive distillation does not&#10;significantly increase the robustness of neural networks by introducing three&#10;new attack algorithms that are successful on both distilled and undistilled&#10;neural networks with $100\%$ probability. Our attacks are tailored to three&#10;distance metrics used previously in the literature, and when compared to&#10;previous adversarial example generation algorithms, our attacks are often much&#10;more effective (and never worse). Furthermore, we propose using high-confidence&#10;adversarial examples in a simple transferability test we show can also be used&#10;to break defensive distillation. We hope our attacks will be used as a&#10;benchmark in future defense attempts to create neural networks that resist&#10;adversarial examples.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.13065" label="2111.13065">
-        <attvalues>
-          <attvalue for="0" value="Robust Object Detection with Multi-input Multi-output Faster R-CNN" />
-          <attvalue for="1" value="  Recent years have seen impressive progress in visual recognition on many&#10;benchmarks, however, generalization to the real-world in out-of-distribution&#10;setting remains a significant challenge. A state-of-the-art method for robust&#10;visual recognition is model ensembling. however, recently it was shown that&#10;similarly competitive results could be achieved with a much smaller cost, by&#10;using multi-input multi-output architecture (MIMO). In this work, a&#10;generalization of the MIMO approach is applied to the task of object detection&#10;using the general-purpose Faster R-CNN model. It was shown that using the MIMO&#10;framework allows building strong feature representation and obtains very&#10;competitive accuracy when using just two input/output pairs. Furthermore, it&#10;adds just 0.5\% additional model parameters and increases the inference time by&#10;15.9\% when compared to the standard Faster R-CNN. It also works comparably to,&#10;or outperforms the Deep Ensemble approach in terms of model accuracy,&#10;robustness to out-of-distribution setting, and uncertainty calibration when the&#10;same number of predictions is used. This work opens up avenues for applying the&#10;MIMO approach in other high-level tasks such as semantic segmentation and depth&#10;estimation.&#10;" />
-          <attvalue for="2" value="&#10;Convolutional Neural Networks (CNNs) have recently become a standard method for image processing as they achieve excellent results on many benchmarks. Despite their impressive performance, the current machine learning techniques lack robustness when presented with an image that does not follow the training dataset distribution (out-of-domain data). It was shown that the current models are vulnerable to noisy input \cite{corruptions,fourier}, novel weather conditions \cite{wintercoming}, and background changes \cite{background}, which creates safety considerations for models deployed to the real world, e.g., autonomous driving or medical applications. To improve visual recognition models’ robustness it was proposed to use data augmentations which change objects' appearance, for example by using style-transfer \cite{cnnbiased,towards} data augmentation or color distortions \cite{simclr,gta}.&#10;&#10;Additionally, the current models are often overconfident in their predictions \cite{calibration}, and the problem becomes more evident with out-of-domain data \cite{canyoutrust,gta}. &#10;Sampling based-methods were shown to obtain very good results in terms of accuracy, out-of-domain robustness and improving model predictive uncertainty \cite{medconf,canyoutrust,gta}. The gold standard is the ensemble approach \cite{ensembles_1990}, which involves combining the output of several, diverse models. Several methods were developed to reduce the high computational cost, such as test-time dropout \cite{Gal} or batch ensemble \cite{batchensemble}. Another competitive approach is the m-heads model \cite{mheads}, which can be viewed as an ensemble with parameter sharing in the first layers of the network. Those methods, however, do not always match ensembling accuracy, as the success of sampling-based methods lies in the diversity of the predictions, which is a challenging problem \cite{deepensemble,pitfalls}. &#10;&#10;Recently, the literature on obtaining many predictions from one model using a single inference step has increased. These methods were inspired by the compression methods, which show that it is possible to remove even 90\% of the weights, without affecting the final model accuracy \cite{nipsprune,ticket}. Therefore, instead of compressing the model, it should be possible to fit more than one subnetwork within the main network. For example, \cite{superposition,supermasks} use a single model in the multi-task setting, and the latter approach retrieves a subnetwork (from the main model) to efficiently solve the target task. Another method uses the multi-input multi-output (MIMO) approach, where a single model makes multiple predictions simultaneously. MIMO was shown to only slightly increase the computational cost, while matching the accuracy of model ensembling and was showcased on the image classification task. &#10;Yet, whether the MIMO approach would work in multi-task setting such as object detection, particularly when regressing objects’ localizations is unknown. In this work, the MIMO method is adapted for object detection tasks and further evaluated. To summarize, the contributions of this work are as follows: &#10;\begin{itemize}&#10; \item The multi-input multi-output model was adapted to the object detection task and the architectural changes and implementation details are presented, &#10; \item It was shown that such a model acting as a strong regularizer brings significant improvements in terms of in-domain and out-of-domain accuracy, and in classification calibration, by adding only a small computational cost at inference time,&#10; \item The robustness of the MIMO approach robustness is presented by comparing its results to different Deep Ensemble approaches, which it outperforms (unless a larger number of models are used for the Deep Ensemble) or matches in accuracy.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science, Visual Recognition" />
-        </attvalues>
-      </node>
-      <node id="1906.08988" label="1906.08988">
-        <attvalues>
-          <attvalue for="0" value="A Fourier Perspective on Model Robustness in Computer Vision" />
-          <attvalue for="1" value="  Achieving robustness to distributional shift is a longstanding and&#10;challenging goal of computer vision. Data augmentation is a commonly used&#10;approach for improving robustness, however robustness gains are typically not&#10;uniform across corruption types. Indeed increasing performance in the presence&#10;of random noise is often met with reduced performance on other corruptions such&#10;as contrast change. Understanding when and why these sorts of trade-offs occur&#10;is a crucial step towards mitigating them. Towards this end, we investigate&#10;recently observed trade-offs caused by Gaussian data augmentation and&#10;adversarial training. We find that both methods improve robustness to&#10;corruptions that are concentrated in the high frequency domain while reducing&#10;robustness to corruptions that are concentrated in the low frequency domain.&#10;This suggests that one way to mitigate these trade-offs via data augmentation&#10;is to use a more diverse set of augmentations. Towards this end we observe that&#10;AutoAugment, a recently proposed data augmentation policy optimized for clean&#10;accuracy, achieves state-of-the-art robustness on the CIFAR-10-C benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.07484" label="1907.07484">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking Robustness in Object Detection: Autonomous Driving when&#10;  Winter is Coming" />
-          <attvalue for="1" value="  The ability to detect objects regardless of image distortions or weather&#10;conditions is crucial for real-world applications of deep learning like&#10;autonomous driving. We here provide an easy-to-use benchmark to assess how&#10;object detection models perform when image quality degrades. The three&#10;resulting benchmark datasets, termed Pascal-C, Coco-C and Cityscapes-C, contain&#10;a large variety of image corruptions. We show that a range of standard object&#10;detection models suffer a severe performance loss on corrupted images (down to&#10;30--60\% of the original performance). However, a simple data augmentation&#10;trick---stylizing the training images---leads to a substantial increase in&#10;robustness across corruption type, severity and dataset. We envision our&#10;comprehensive benchmark to track future progress towards building robust object&#10;detection models. Benchmark, code and data are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.09994" label="2006.09994">
-        <attvalues>
-          <attvalue for="0" value="Noise or Signal: The Role of Image Backgrounds in Object Recognition" />
-          <attvalue for="1" value="  We assess the tendency of state-of-the-art object recognition models to&#10;depend on signals from image backgrounds. We create a toolkit for disentangling&#10;foreground and background signal on ImageNet images, and find that (a) models&#10;can achieve non-trivial accuracy by relying on the background alone, (b) models&#10;often misclassify images even in the presence of correctly classified&#10;foregrounds--up to 87.5% of the time with adversarially chosen backgrounds, and&#10;(c) more accurate models tend to depend on backgrounds less. Our analysis of&#10;backgrounds brings us closer to understanding which correlations machine&#10;learning models use, and how they determine models' out of distribution&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.12231" label="1811.12231">
-        <attvalues>
-          <attvalue for="0" value="ImageNet-trained CNNs are biased towards texture; increasing shape bias&#10;  improves accuracy and robustness" />
-          <attvalue for="1" value="  Convolutional Neural Networks (CNNs) are commonly thought to recognise&#10;objects by learning increasingly complex representations of object shapes. Some&#10;recent studies suggest a more important role of image textures. We here put&#10;these conflicting hypotheses to a quantitative test by evaluating CNNs and&#10;human observers on images with a texture-shape cue conflict. We show that&#10;ImageNet-trained CNNs are strongly biased towards recognising textures rather&#10;than shapes, which is in stark contrast to human behavioural evidence and&#10;reveals fundamentally different classification strategies. We then demonstrate&#10;that the same standard architecture (ResNet-50) that learns a texture-based&#10;representation on ImageNet is able to learn a shape-based representation&#10;instead when trained on &quot;Stylized-ImageNet&quot;, a stylized version of ImageNet.&#10;This provides a much better fit for human behavioural performance in our&#10;well-controlled psychophysical lab setting (nine experiments totalling 48,560&#10;psychophysical trials across 97 observers) and comes with a number of&#10;unexpected emergent benefits such as improved object detection performance and&#10;previously unseen robustness towards a wide range of image distortions,&#10;highlighting advantages of a shape-based representation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.00076" label="2106.00076">
-        <attvalues>
-          <attvalue for="0" value="Closer Look at the Uncertainty Estimation in Semantic Segmentation under&#10;  Distributional Shift" />
-          <attvalue for="1" value="  While recent computer vision algorithms achieve impressive performance on&#10;many benchmarks, they lack robustness - presented with an image from a&#10;different distribution, (e.g. weather or lighting conditions not considered&#10;during training), they may produce an erroneous prediction. Therefore, it is&#10;desired that such a model will be able to reliably predict its confidence&#10;measure. In this work, uncertainty estimation for the task of semantic&#10;segmentation is evaluated under a varying level of domain shift: in a&#10;cross-dataset setting and when adapting a model trained on data from the&#10;simulation. It was shown that simple color transformations already provide a&#10;strong baseline, comparable to using more sophisticated style-transfer data&#10;augmentation. Further, by constructing an ensemble consisting of models using&#10;different backbones and/or augmentation methods, it was possible to improve&#10;significantly model performance in terms of overall accuracy and uncertainty&#10;estimation under the domain shift setting. The Expected Calibration Error (ECE)&#10;on challenging GTA to Cityscapes adaptation was reduced from 4.05 to the&#10;competitive value of 1.1. Further, an ensemble of models was utilized in the&#10;self-training setting to improve the pseudo-labels generation, which resulted&#10;in a significant gain in the final model accuracy, compared to the standard&#10;fine-tuning (without ensemble).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.04599" label="1706.04599">
-        <attvalues>
-          <attvalue for="0" value="On Calibration of Modern Neural Networks" />
-          <attvalue for="1" value="  Confidence calibration -- the problem of predicting probability estimates&#10;representative of the true correctness likelihood -- is important for&#10;classification models in many applications. We discover that modern neural&#10;networks, unlike those from a decade ago, are poorly calibrated. Through&#10;extensive experiments, we observe that depth, width, weight decay, and Batch&#10;Normalization are important factors influencing calibration. We evaluate the&#10;performance of various post-processing calibration methods on state-of-the-art&#10;architectures with image and document classification datasets. Our analysis and&#10;experiments not only offer insights into neural network learning, but also&#10;provide a simple and straightforward recipe for practical settings: on most&#10;datasets, temperature scaling -- a single-parameter variant of Platt Scaling --&#10;is surprisingly effective at calibrating predictions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.02530" label="1906.02530">
-        <attvalues>
-          <attvalue for="0" value="Can You Trust Your Model's Uncertainty? Evaluating Predictive&#10;  Uncertainty Under Dataset Shift" />
-          <attvalue for="1" value="  Modern machine learning methods including deep learning have achieved great&#10;success in predictive accuracy for supervised learning tasks, but may still&#10;fall short in giving useful estimates of their predictive {\em uncertainty}.&#10;Quantifying uncertainty is especially critical in real-world settings, which&#10;often involve input distributions that are shifted from the training&#10;distribution due to a variety of factors including sample bias and&#10;non-stationarity. In such settings, well calibrated uncertainty estimates&#10;convey information about when a model's output should (or should not) be&#10;trusted. Many probabilistic deep learning methods, including Bayesian-and&#10;non-Bayesian methods, have been proposed in the literature for quantifying&#10;predictive uncertainty, but to our knowledge there has not previously been a&#10;rigorous large-scale empirical comparison of these methods under dataset shift.&#10;We present a large-scale benchmark of existing state-of-the-art methods on&#10;classification problems and investigate the effect of dataset shift on accuracy&#10;and calibration. We find that traditional post-hoc calibration does indeed fall&#10;short, as do several other previous methods. However, some methods that&#10;marginalize over models give surprisingly strong results across a broad&#10;spectrum of tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.13273" label="1911.13273">
-        <attvalues>
-          <attvalue for="0" value="Confidence Calibration and Predictive Uncertainty Estimation for Deep&#10;  Medical Image Segmentation" />
-          <attvalue for="1" value="  Fully convolutional neural networks (FCNs), and in particular U-Nets, have&#10;achieved state-of-the-art results in semantic segmentation for numerous medical&#10;imaging applications. Moreover, batch normalization and Dice loss have been&#10;used successfully to stabilize and accelerate training. However, these networks&#10;are poorly calibrated i.e. they tend to produce overconfident predictions both&#10;in correct and erroneous classifications, making them unreliable and hard to&#10;interpret. In this paper, we study predictive uncertainty estimation in FCNs&#10;for medical image segmentation. We make the following contributions: 1) We&#10;systematically compare cross entropy loss with Dice loss in terms of&#10;segmentation quality and uncertainty estimation of FCNs; 2) We propose model&#10;ensembling for confidence calibration of the FCNs trained with batch&#10;normalization and Dice loss; 3) We assess the ability of calibrated FCNs to&#10;predict segmentation quality of structures and detect out-of-distribution test&#10;examples. We conduct extensive experiments across three medical image&#10;segmentation applications of the brain, the heart, and the prostate to evaluate&#10;our contributions. The results of this study offer considerable insight into&#10;the predictive uncertainty estimation and out-of-distribution detection in&#10;medical image segmentation and provide practical recipes for confidence&#10;calibration. Moreover, we consistently demonstrate that model ensembling&#10;improves confidence calibration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1506.02142" label="1506.02142">
-        <attvalues>
-          <attvalue for="0" value="Dropout as a Bayesian Approximation: Representing Model Uncertainty in&#10;  Deep Learning" />
-          <attvalue for="1" value="  Deep learning tools have gained tremendous attention in applied machine&#10;learning. However such tools for regression and classification do not capture&#10;model uncertainty. In comparison, Bayesian models offer a mathematically&#10;grounded framework to reason about model uncertainty, but usually come with a&#10;prohibitive computational cost. In this paper we develop a new theoretical&#10;framework casting dropout training in deep neural networks (NNs) as approximate&#10;Bayesian inference in deep Gaussian processes. A direct result of this theory&#10;gives us tools to model uncertainty with dropout NNs -- extracting information&#10;from existing models that has been thrown away so far. This mitigates the&#10;problem of representing uncertainty in deep learning without sacrificing either&#10;computational complexity or test accuracy. We perform an extensive study of the&#10;properties of dropout's uncertainty. Various network architectures and&#10;non-linearities are assessed on tasks of regression and classification, using&#10;MNIST as an example. We show a considerable improvement in predictive&#10;log-likelihood and RMSE compared to existing state-of-the-art methods, and&#10;finish by using dropout's uncertainty in deep reinforcement learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.06715" label="2002.06715">
-        <attvalues>
-          <attvalue for="0" value="BatchEnsemble: An Alternative Approach to Efficient Ensemble and&#10;  Lifelong Learning" />
-          <attvalue for="1" value="  Ensembles, where multiple neural networks are trained individually and their&#10;predictions are averaged, have been shown to be widely successful for improving&#10;both the accuracy and predictive uncertainty of single neural networks.&#10;However, an ensemble's cost for both training and testing increases linearly&#10;with the number of networks, which quickly becomes untenable.&#10;  In this paper, we propose BatchEnsemble, an ensemble method whose&#10;computational and memory costs are significantly lower than typical ensembles.&#10;BatchEnsemble achieves this by defining each weight matrix to be the Hadamard&#10;product of a shared weight among all ensemble members and a rank-one matrix per&#10;member. Unlike ensembles, BatchEnsemble is not only parallelizable across&#10;devices, where one device trains one member, but also parallelizable within a&#10;device, where multiple ensemble members are updated simultaneously for a given&#10;mini-batch. Across CIFAR-10, CIFAR-100, WMT14 EN-DE/EN-FR translation, and&#10;out-of-distribution tasks, BatchEnsemble yields competitive accuracy and&#10;uncertainties as typical ensembles; the speedup at test time is 3X and memory&#10;reduction is 3X at an ensemble of size 4. We also apply BatchEnsemble to&#10;lifelong learning, where on Split-CIFAR-100, BatchEnsemble yields comparable&#10;performance to progressive neural networks while having a much lower&#10;computational and memory costs. We further show that BatchEnsemble can easily&#10;scale up to lifelong learning on Split-ImageNet which involves 100 sequential&#10;learning tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.06314" label="1511.06314">
-        <attvalues>
-          <attvalue for="0" value="Why M Heads are Better than One: Training a Diverse Ensemble of Deep&#10;  Networks" />
-          <attvalue for="1" value="  Convolutional Neural Networks have achieved state-of-the-art performance on a&#10;wide range of tasks. Most benchmarks are led by ensembles of these powerful&#10;learners, but ensembling is typically treated as a post-hoc procedure&#10;implemented by averaging independently trained models with model variation&#10;induced by bagging or random initialization. In this paper, we rigorously treat&#10;ensembling as a first-class problem to explicitly address the question: what&#10;are the best strategies to create an ensemble? We first compare a large number&#10;of ensembling strategies, and then propose and evaluate novel strategies, such&#10;as parameter sharing (through a new family of models we call TreeNets) as well&#10;as training under ensemble-aware and diversity-encouraging losses. We&#10;demonstrate that TreeNets can improve ensemble performance and that diverse&#10;ensembles can be trained end-to-end under a unified loss, achieving&#10;significantly higher &quot;oracle&quot; accuracies than classical ensembles.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.06470" label="2002.06470">
-        <attvalues>
-          <attvalue for="0" value="Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep&#10;  Learning" />
-          <attvalue for="1" value="  Uncertainty estimation and ensembling methods go hand-in-hand. Uncertainty&#10;estimation is one of the main benchmarks for assessment of ensembling&#10;performance. At the same time, deep learning ensembles have provided&#10;state-of-the-art results in uncertainty estimation. In this work, we focus on&#10;in-domain uncertainty for image classification. We explore the standards for&#10;its quantification and point out pitfalls of existing metrics. Avoiding these&#10;pitfalls, we perform a broad study of different ensembling techniques. To&#10;provide more insight in this study, we introduce the deep ensemble equivalent&#10;score (DEE) and show that many sophisticated ensembling techniques are&#10;equivalent to an ensemble of only few independently trained networks in terms&#10;of test performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.03635" label="1803.03635">
-        <attvalues>
-          <attvalue for="0" value="The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" />
-          <attvalue for="1" value="  Neural network pruning techniques can reduce the parameter counts of trained&#10;networks by over 90%, decreasing storage requirements and improving&#10;computational performance of inference without compromising accuracy. However,&#10;contemporary experience is that the sparse architectures produced by pruning&#10;are difficult to train from the start, which would similarly improve training&#10;performance.&#10;  We find that a standard pruning technique naturally uncovers subnetworks&#10;whose initializations made them capable of training effectively. Based on these&#10;results, we articulate the &quot;lottery ticket hypothesis:&quot; dense,&#10;randomly-initialized, feed-forward networks contain subnetworks (&quot;winning&#10;tickets&quot;) that - when trained in isolation - reach test accuracy comparable to&#10;the original network in a similar number of iterations. The winning tickets we&#10;find have won the initialization lottery: their connections have initial&#10;weights that make training particularly effective.&#10;  We present an algorithm to identify winning tickets and a series of&#10;experiments that support the lottery ticket hypothesis and the importance of&#10;these fortuitous initializations. We consistently find winning tickets that are&#10;less than 10-20% of the size of several fully-connected and convolutional&#10;feed-forward architectures for MNIST and CIFAR10. Above this size, the winning&#10;tickets that we find learn faster than the original network and reach higher&#10;test accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.05522" label="1902.05522">
-        <attvalues>
-          <attvalue for="0" value="Superposition of many models into one" />
-          <attvalue for="1" value="  We present a method for storing multiple models within a single set of&#10;parameters. Models can coexist in superposition and still be retrieved&#10;individually. In experiments with neural networks, we show that a surprisingly&#10;large number of models can be effectively stored within a single parameter&#10;instance. Furthermore, each of these models can undergo thousands of training&#10;steps without significantly interfering with other models within the&#10;superposition. This approach may be viewed as the online complement of&#10;compression: rather than reducing the size of a network after training, we make&#10;use of the unrealized capacity of a network during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.14769" label="2006.14769">
-        <attvalues>
-          <attvalue for="0" value="Supermasks in Superposition" />
-          <attvalue for="1" value="  We present the Supermasks in Superposition (SupSup) model, capable of&#10;sequentially learning thousands of tasks without catastrophic forgetting. Our&#10;approach uses a randomly initialized, fixed base network and for each task&#10;finds a subnetwork (supermask) that achieves good performance. If task identity&#10;is given at test time, the correct subnetwork can be retrieved with minimal&#10;memory usage. If not provided, SupSup can infer the task using gradient-based&#10;optimization to find a linear superposition of learned supermasks which&#10;minimizes the output entropy. In practice we find that a single gradient step&#10;is often sufficient to identify the correct mask, even among 2500 tasks. We&#10;also showcase two promising extensions. First, SupSup models can be trained&#10;entirely without task identity information, as they may detect when they are&#10;uncertain about new data and allocate an additional supermask for the new&#10;training distribution. Finally the entire, growing set of supermasks can be&#10;stored in a constant-sized reservoir by implicitly storing them as attractors&#10;in a fixed-sized Hopfield network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.13075" label="2302.13075">
-        <attvalues>
-          <attvalue for="0" value="BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of&#10;  Specific Rigid Objects" />
-          <attvalue for="1" value="  We present the evaluation methodology, datasets and results of the BOP&#10;Challenge 2022, the fourth in a series of public competitions organized with&#10;the goal to capture the status quo in the field of 6D object pose estimation&#10;from an RGB/RGB-D image. In 2022, we witnessed another significant improvement&#10;in the pose estimation accuracy -- the state of the art, which was 56.9 AR$_C$&#10;in 2019 (Vidal et al.) and 69.8 AR$_C$ in 2020 (CosyPose), moved to new heights&#10;of 83.7 AR$_C$ (GDRNPP). Out of 49 pose estimation methods evaluated since&#10;2019, the top 18 are from 2022. Methods based on point pair features, which&#10;were introduced in 2010 and achieved competitive results even in 2020, are now&#10;clearly outperformed by deep learning methods. The synthetic-to-real domain gap&#10;was again significantly reduced, with 82.7 AR$_C$ achieved by GDRNPP trained&#10;only on synthetic images from BlenderProc. The fastest variant of GDRNPP&#10;reached 80.5 AR$_C$ with an average time per image of 0.23s. Since most of the&#10;recent methods for 6D object pose estimation begin by detecting/segmenting&#10;objects, we also started evaluating 2D object detection and segmentation&#10;performance based on the COCO metrics. Compared to the Mask R-CNN results from&#10;CosyPose in 2020, detection improved from 60.3 to 77.3 AP$_C$ and segmentation&#10;from 40.5 to 58.7 AP$_C$. The online evaluation system stays open and is&#10;available at: \href{http://bop.felk.cvut.cz/}{bop.felk.cvut.cz}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Estimating the 6D pose, \ie, the 3D translation and 3D rotation, of specific rigid objects from a single image is an important task for application fields such as robotic manipulation, augmented reality, or autonomous driving.&#10;The BOP Challenge 2022 is the fourth in a series of public challenges that are part of the BOP project aiming to continuously report the state of the art in 6D object pose estimation.&#10;The first challenge was organized in 2017~\cite{hodan2017sixd} and the results were published in~\cite{hodan2018bop}.&#10;Results of the second challenge from 2019~\cite{hodan2019bop}, the third from 2020 \cite{hodan2020bop}, and the fourth from 2022 are included and discussed in this paper.&#10;&#10;Participants of the 2022 challenge were competing on three tasks: 6D object localization, 2D object detection, and 2D object segmentation.&#10;The 6D object localization task has the same evaluation methodology and leaderboard since 2019, while the latter two tasks were introduced in 2022.&#10;&#10;In the 6D object localization task, methods report their predictions on the basis of two sources of information. Firstly, at training time, a method is given 3D object models and training images showing the objects in known 6D poses. Secondly, at test time, the method is provided with a test image and a list of object instances visible in the image, and the goal is to estimate 6D poses of the listed instances. The images consist of RGB-D (aligned color and depth) channels and intrinsic camera parameters are known.&#10;&#10;The 2D object detection and segmentation tasks were introduced to address the design of the majority of recent object pose estimation methods, which start by detecting/segmenting objects and then estimate their poses from the predicted image regions. Evaluating the detection/segmentation and pose estimation stages separately enables a better understanding of advances in the two stages.&#10;To create an opportunity for detector-agnostic comparison of pose estimation methods and to allow participants to focus only on the pose estimation stage, we also provided default detections and segmentations from Mask R-CNN~\cite{he2017mask} trained for CosyPose~\cite{labbe2020cosypose}, the winning method in 2020.&#10;&#10;The challenge primarily focuses on the practical scenario where no real images are available at training time, only the 3D object models and images synthesized using the models. While capturing real images of objects under various conditions and annotating the images with 6D object poses requires a significant human effort~\cite{hodan2017tless}, the 3D models are either available before the physical objects, which is often the case for manufactured objects, or can be reconstructed at an admissible cost.&#10;Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are established~\cite{newcombe2011kinectfusion,reizenstein2021common} and promising approaches for transparent and highly specular objects are emerging~\cite{wu2018full,Munkberg_2022_CVPR,hasselgren2022shape,verbin2022ref}.&#10;&#10;In the 2019 challenge, methods using the depth image channel were mostly based on point pair features (PPF's)~\cite{drost2010model} and clearly outperformed methods relying only on the RGB channels, all of which were based on deep neural networks (DNN's). DNN-based methods need large amounts of annotated training images, which had been typically obtained by OpenGL rendering of the 3D object models on random backgrounds~\cite{kehl2017ssd,hinterstoisser2017pre}. However, as suggested in~\cite{hodan2019photorealistic}, the evident domain gap between these ``render\;\&amp;\;paste'' training images and real test images&#10;limits the potential of the DNN-based methods. To reduce the gap between the synthetic and real domains and thus to bring fresh air to the DNN world, we joined the development of BlenderProc~\cite{denninger2019blenderproc,denninger2020blenderproc}, an open-source, physically-based renderer (PBR).&#10;For the 2020 challenge, we then provided participants with 350K&#10;PBR training images (see~\cite{hodan2020bop} for examples), which helped the DNN-based methods to achieve noticeably higher accuracy&#10;and to finally catch up with the PPF-based methods.&#10;&#10;In the 2022 challenge, DNN-based methods for 6D object localization clearly outperformed PPF-based methods in both accuracy and speed, with the performance gains coming mostly from advances in network architectures and training schemes. The largest improvements were achieved on challenging industry-relevant datasets ITODD~\cite{drost2017introducing} and T-LESS~\cite{hodan2017tless}, and on the HB dataset~\cite{kaskman2019homebreweddb} which includes diverse objects captured under various levels of occlusion. Remarkably, RGB methods from 2022 surpassed RGB-D methods from 2020, the performance gap between methods trained only on PBR images and methods trained also on real images noticeably shrinked, and some methods started training on the depth image channel in addition to the RGB channels. On the new 2D object detection and segmentation tasks, large gains were achieved \wrt a baseline from 2020.&#10;&#10;Sec.~\ref{sec:methodology} of this paper defines the evaluation methodology, Sec.~\ref{sec:datasets} introduces datasets,&#10;Sec.~\ref{sec:evaluation} describes the experimental setup and analyzes the results, Sec.~\ref{sec:awards} presents the awards of the BOP Challenge 2022, and Sec.~\ref{sec:conclusion} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Evaluation, Mathematics, Deep Learning Methods, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1808.08319" label="1808.08319">
-        <attvalues>
-          <attvalue for="0" value="BOP: Benchmark for 6D Object Pose Estimation" />
-          <attvalue for="1" value="  We propose a benchmark for 6D pose estimation of a rigid object from a single&#10;RGB-D input image. The training data consists of a texture-mapped 3D object&#10;model or images of the object in known 6D poses. The benchmark comprises of: i)&#10;eight datasets in a unified format that cover different practical scenarios,&#10;including two new datasets focusing on varying lighting conditions, ii) an&#10;evaluation methodology with a pose-error function that deals with pose&#10;ambiguities, iii) a comprehensive evaluation of 15 diverse recent methods that&#10;captures the status quo of the field, and iv) an online evaluation system that&#10;is open for continuous submission of new results. The evaluation shows that&#10;methods based on point-pair features currently perform best, outperforming&#10;template matching methods, learning-based methods and methods based on 3D local&#10;features. The project website is available at bop.felk.cvut.cz.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.07378" label="2009.07378">
-        <attvalues>
-          <attvalue for="0" value="BOP Challenge 2020 on 6D Object Localization" />
-          <attvalue for="1" value="  This paper presents the evaluation methodology, datasets, and results of the&#10;BOP Challenge 2020, the third in a series of public competitions organized with&#10;the goal to capture the status quo in the field of 6D object pose estimation&#10;from an RGB-D image. In 2020, to reduce the domain gap between synthetic&#10;training and real test RGB images, the participants were provided 350K&#10;photorealistic training images generated by BlenderProc4BOP, a new open-source&#10;and light-weight physically-based renderer (PBR) and procedural data generator.&#10;Methods based on deep neural networks have finally caught up with methods based&#10;on point pair features, which were dominating previous editions of the&#10;challenge. Although the top-performing methods rely on RGB-D image channels,&#10;strong results were achieved when only RGB channels were used at both training&#10;and test time - out of the 26 evaluated methods, the third method was trained&#10;on RGB channels of PBR and real images, while the fifth on RGB channels of PBR&#10;images only. Strong data augmentation was identified as a key component of the&#10;top-performing CosyPose method, and the photorealism of PBR images was&#10;demonstrated effective despite the augmentation. The online evaluation system&#10;stays open and is available on the project website: bop.felk.cvut.cz.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Estimating the 6D pose, \ie the 3D translation and 3D rotation, of rigid objects from a single input image is a crucial task for numerous application fields such as robotic manipulation, augmented reality, or autonomous driving.&#10;The BOP&#10;Challenge 2020 is the third in a series of public challenges that are part of the BOP&#10;project aiming to continuously report the state of the art in 6D object pose estimation.&#10;The first challenge was organized in 2017~\cite{hodan2017sixd} and the results were published in~\cite{hodan2018bop}.&#10;The second challenge from 2019~\cite{hodan2019bop} and the third from 2020 share the same evaluation methodology and leaderboard and the results from both are included in this paper.&#10;&#10;Participating methods are evaluated on the 6D object localization task~\cite{hodan2016evaluation}, where the methods report their predictions on the basis of two sources of information. Firstly, at training time, a method is given 3D object models and training images showing the objects in known 6D poses. Secondly, at test time, the&#10;method is provided with a test image and a list of object instances visible in the image, and the goal of the method is to estimate 6D poses of the listed object instances. The training and test images consist of RGB-D (aligned color and depth) channels and the intrinsic camera parameters are known.&#10;&#10;The challenge primarily focuses on the practical scenario where no real images are available at training time, only the 3D object models and images synthesized using the models. While capturing real images of objects under various conditions and annotating the images with 6D object poses requires a significant human effort~\cite{hodan2017tless}, the 3D models are either available before the physical objects, which is often the case for manufactured objects, or can be reconstructed at an admissible cost.&#10;Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are well established~\cite{newcombe2011kinectfusion} and promising approaches for transparent and highly specular objects are emerging~\cite{qian20163d,wu2018full,godard2015multi}.&#10;&#10;In the BOP Challenge 2019, methods using the depth image channel, which were mostly based on the point pair features (PPF's)~\cite{drost2010model}, clearly outperformed methods relying only on the RGB channels, all of which were based on deep neural networks (DNN's).&#10;The PPF-based methods match pairs of oriented 3D points between the point cloud of the test scene and the 3D object model, and aggregate the matches via a voting scheme. As each pair is described by only the distance and relative orientation of the two points, PPF-based methods can be effectively trained directly on the 3D object models, without the need to synthesize any training images. In contrast, DNN-based methods require large amounts of annotated training images, which have been typically obtained by OpenGL rendering of the 3D object models on top of&#10;random backgrounds~\cite{kehl2017ssd,rad2017bb8,hinterstoisser2017pre,dwibedi2017cut}. However, as suggested in~\cite{hodan2019photorealistic}, the evident domain gap between these ``render\;\&amp;\;paste'' training images and real test images presumably limits the potential of the DNN-based methods.&#10;&#10;To reduce the gap between the synthetic and real domains and thus to bring fresh air to the DNN-based methods, we have created BlenderProc4BOP~\cite{denninger2019blenderproc,denninger2020blenderproc}, an open-source and light-weight physically-based renderer (PBR).&#10;Furthermore, to reduce the entry barrier of the challenge and to standardize the training set, the participants were provided with 350K pre-rendered PBR images (Fig.~\ref{fig:pbr_examples}).&#10;&#10;In 2020, the DNN-based methods have finally caught up with the PPF-based methods -- five methods outperformed Vidal-Sensors18~\cite{vidal2018method}, the PPF-based winner from 2017 and 2019. Three of the top five methods, including the top-performing one, are single-view variants of CosyPose, a DNN-based method by Labb{\'e}~\etal~\cite{labbe2020cosypose}.&#10;Strong data augmentation, similar to~\cite{sundermeyer2019augmented}, was identified as one of the key ingredients of this method. The second is a hybrid DNN+PPF method by K\&quot;onig and Drost~\cite{koenig2020hybrid}, and the fourth is Pix2Pose, a DNN-based method by Park \etal~\cite{park2019pix2pose}. The first two methods used RGB-D image channels, while the third method achieved strong results with RGB channels only.&#10;&#10;Methods achieved noticeably higher accuracy scores when trained on PBR training images than when trained on ``render\;\&amp;\;paste'' images. Although adding real training images yielded even higher scores, competitive results were achieved with PBR images only -- out of the 26 evaluated methods, the fifth was trained only on PBR images. Interestingly, the increased photorealism from PBR images led to clear improvements of also the CosyPose method, despite the strong data augmentation which this method applies to the training images.&#10;&#10;The rest of the paper is organized as follows. Section~\ref{sec:methodology} defines the evaluation methodology, Section~\ref{sec:datasets} introduces datasets and the implemented approach to synthesize photorealistic training images, Section~\ref{sec:evaluation} describes the experimental setup and analyzes the results, Section~\ref{sec:awards} presents the awards of the BOP Challenge 2020, and Section~\ref{sec:conclusion} concludes the paper. A discussion on the choices made when defining the evaluation methodology is provided in the supplement.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.05498" label="1701.05498">
-        <attvalues>
-          <attvalue for="0" value="T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects" />
-          <attvalue for="1" value="  We introduce T-LESS, a new public dataset for estimating the 6D pose, i.e.&#10;translation and rotation, of texture-less rigid objects. The dataset features&#10;thirty industry-relevant objects with no significant texture and no&#10;discriminative color or reflectance properties. The objects exhibit symmetries&#10;and mutual similarities in shape and/or size. Compared to other datasets, a&#10;unique property is that some of the objects are parts of others. The dataset&#10;includes training and test images that were captured with three synchronized&#10;sensors, specifically a structured-light and a time-of-flight RGB-D sensor and&#10;a high-resolution RGB camera. There are approximately 39K training and 10K test&#10;images from each sensor. Additionally, two types of 3D models are provided for&#10;each object, i.e. a manually created CAD model and a semi-automatically&#10;reconstructed one. Training images depict individual objects against a black&#10;background. Test images originate from twenty test scenes having varying&#10;complexity, which increases from simple scenes with several isolated objects to&#10;very challenging ones with multiple instances of several objects and with a&#10;high amount of clutter and occlusion. The images were captured from a&#10;systematically sampled view sphere around the object/scene, and are annotated&#10;with accurate ground truth 6D poses of all modeled objects. Initial evaluation&#10;results indicate that the state of the art in 6D object pose estimation has&#10;ample room for improvement, especially in difficult cases with significant&#10;occlusion. The T-LESS dataset is available online at cmp.felk.cvut.cz/t-less.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.00512" label="2109.00512">
-        <attvalues>
-          <attvalue for="0" value="Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life&#10;  3D Category Reconstruction" />
-          <attvalue for="1" value="  Traditional approaches for learning 3D object categories have been&#10;predominantly trained and evaluated on synthetic datasets due to the&#10;unavailability of real 3D-annotated category-centric data. Our main goal is to&#10;facilitate advances in this field by collecting real-world data in a magnitude&#10;similar to the existing synthetic counterparts. The principal contribution of&#10;this work is thus a large-scale dataset, called Common Objects in 3D, with real&#10;multi-view images of object categories annotated with camera poses and ground&#10;truth 3D point clouds. The dataset contains a total of 1.5 million frames from&#10;nearly 19,000 videos capturing objects from 50 MS-COCO categories and, as such,&#10;it is significantly larger than alternatives both in terms of the number of&#10;categories and objects. We exploit this new dataset to conduct one of the first&#10;large-scale &quot;in-the-wild&quot; evaluations of several new-view-synthesis and&#10;category-centric 3D reconstruction methods. Finally, we contribute NerFormer -&#10;a novel neural rendering method that leverages the powerful Transformer to&#10;reconstruct an object given a small number of its views. The CO3D dataset is&#10;available at https://github.com/facebookresearch/co3d .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.03482" label="1805.03482">
-        <attvalues>
-          <attvalue for="0" value="Full 3D Reconstruction of Transparent Objects" />
-          <attvalue for="1" value="  Numerous techniques have been proposed for reconstructing 3D models for&#10;opaque objects in past decades. However, none of them can be directly applied&#10;to transparent objects. This paper presents a fully automatic approach for&#10;reconstructing complete 3D shapes of transparent objects. Through positioning&#10;an object on a turntable, its silhouettes and light refraction paths under&#10;different viewing directions are captured. Then, starting from an initial rough&#10;model generated from space carving, our algorithm progressively optimizes the&#10;model under three constraints: surface and refraction normal consistency,&#10;surface projection and silhouette consistency, and surface smoothness.&#10;Experimental results on both synthetic and real objects demonstrate that our&#10;method can successfully recover the complex shapes of transparent objects and&#10;faithfully reproduce their light refraction properties.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.12503" label="2111.12503">
-        <attvalues>
-          <attvalue for="0" value="Extracting Triangular 3D Models, Materials, and Lighting From Images" />
-          <attvalue for="1" value="  We present an efficient method for joint optimization of topology, materials&#10;and lighting from multi-view image observations. Unlike recent multi-view&#10;reconstruction approaches, which typically produce entangled 3D representations&#10;encoded in neural networks, we output triangle meshes with spatially-varying&#10;materials and environment lighting that can be deployed in any traditional&#10;graphics engine unmodified. We leverage recent work in differentiable&#10;rendering, coordinate-based networks to compactly represent volumetric&#10;texturing, alongside differentiable marching tetrahedrons to enable&#10;gradient-based optimization directly on the surface mesh. Finally, we introduce&#10;a differentiable formulation of the split sum approximation of environment&#10;lighting to efficiently recover all-frequency lighting. Experiments show our&#10;extracted models used in advanced scene editing, material decomposition, and&#10;high quality view interpolation, all running at interactive rates in&#10;triangle-based renderers (rasterizers and path tracers). Project website:&#10;https://nvlabs.github.io/nvdiffrec/ .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.03907" label="2112.03907">
-        <attvalues>
-          <attvalue for="0" value="Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance&#10;  Fields" />
-          <attvalue for="1" value="  Neural Radiance Fields (NeRF) is a popular view synthesis technique that&#10;represents a scene as a continuous volumetric function, parameterized by&#10;multilayer perceptrons that provide the volume density and view-dependent&#10;emitted radiance at each location. While NeRF-based techniques excel at&#10;representing fine geometric structures with smoothly varying view-dependent&#10;appearance, they often fail to accurately capture and reproduce the appearance&#10;of glossy surfaces. We address this limitation by introducing Ref-NeRF, which&#10;replaces NeRF's parameterization of view-dependent outgoing radiance with a&#10;representation of reflected radiance and structures this function using a&#10;collection of spatially-varying scene properties. We show that together with a&#10;regularizer on normal vectors, our model significantly improves the realism and&#10;accuracy of specular reflections. Furthermore, we show that our model's&#10;internal representation of outgoing radiance is interpretable and useful for&#10;scene editing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.10710" label="1710.10710">
-        <attvalues>
-          <attvalue for="0" value="On Pre-Trained Image Features and Synthetic Images for Deep Learning" />
-          <attvalue for="1" value="  Deep Learning methods usually require huge amounts of training data to&#10;perform at their full potential, and often require expensive manual labeling.&#10;Using synthetic images is therefore very attractive to train object detectors,&#10;as the labeling comes for free, and several approaches have been proposed to&#10;combine synthetic and real images for training.&#10;  In this paper, we show that a simple trick is sufficient to train very&#10;effectively modern object detectors with synthetic images only: We freeze the&#10;layers responsible for feature extraction to generic layers pre-trained on real&#10;images, and train only the remaining layers with plain OpenGL rendering. Our&#10;experiments with very recent deep architectures for object recognition&#10;(Faster-RCNN, R-FCN, Mask-RCNN) and image feature extractors (InceptionResnet&#10;and Resnet) show this simple approach performs surprisingly well.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.03334" label="1902.03334">
-        <attvalues>
-          <attvalue for="0" value="Photorealistic Image Synthesis for Object Instance Detection" />
-          <attvalue for="1" value="  We present an approach to synthesize highly photorealistic images of 3D&#10;object models, which we use to train a convolutional neural network for&#10;detecting the objects in real images. The proposed approach has three key&#10;ingredients: (1) 3D object models are rendered in 3D models of complete scenes&#10;with realistic materials and lighting, (2) plausible geometric configuration of&#10;objects and cameras in a scene is generated using physics simulations, and (3)&#10;high photorealism of the synthesized images achieved by physically based&#10;rendering. When trained on images synthesized by the proposed approach, the&#10;Faster R-CNN object detector achieves a 24% absolute improvement of mAP@.75IoU&#10;on Rutgers APC and 11% on LineMod-Occluded datasets, compared to a baseline&#10;where the training images are synthesized by rendering object models on top of&#10;random photographs. This work is a step towards being able to effectively train&#10;object detectors without capturing or annotating any real images. A dataset of&#10;600K synthetic images with ground truth annotations for various computer vision&#10;tasks will be released on the project website: thodan.github.io/objectsynth.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.01911" label="1911.01911">
-        <attvalues>
-          <attvalue for="0" value="BlenderProc" />
-          <attvalue for="1" value="  BlenderProc is a modular procedural pipeline, which helps in generating real&#10;looking images for the training of convolutional neural networks. These can be&#10;used in a variety of use cases including segmentation, depth, normal and pose&#10;estimation and many others. A key feature of our extension of blender is the&#10;simple to use modular pipeline, which was designed to be easily extendable. By&#10;offering standard modules, which cover a variety of scenarios, we provide a&#10;starting point on which new modules can be created.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03167" label="1904.03167">
-        <attvalues>
-          <attvalue for="0" value="HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects" />
-          <attvalue for="1" value="  Among the most important prerequisites for creating and evaluating 6D object&#10;pose detectors are datasets with labeled 6D poses. With the advent of deep&#10;learning, demand for such datasets is growing continuously. Despite the fact&#10;that some of exist, they are scarce and typically have restricted setups, such&#10;as a single object per sequence, or they focus on specific object types, such&#10;as textureless industrial parts. Besides, two significant components are often&#10;ignored: training using only available 3D models instead of real data and&#10;scalability, i.e. training one method to detect all objects rather than&#10;training one detector per object. Other challenges, such as occlusions,&#10;changing light conditions and changes in object appearance, as well precisely&#10;defined benchmarks are either not present or are scattered among different&#10;datasets. In this paper we present a dataset for 6D pose estimation that covers&#10;the above-mentioned challenges, mainly targeting training from 3D models (both&#10;textured and textureless), scalability, occlusions, and changes in light&#10;conditions and object appearance. The dataset features 33 objects (17 toy, 8&#10;household and 8 industry-relevant objects) over 13 scenes of various&#10;difficulty. We also present a set of benchmarks to test various desired&#10;detector properties, particularly focusing on scalability with respect to the&#10;number of objects and resistance to changing light conditions, occlusions and&#10;clutter. We also set a baseline for the presented benchmarks using a&#10;state-of-the-art DPOD detector. Considering the difficulty of making such&#10;datasets, we plan to release the code allowing other researchers to extend this&#10;dataset or make their own datasets in the future.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Detection of 3D objects in images and recovery of their 6D poses is crucial for a wide range of computer vision tasks. In robotics it is essential to determine 6DoF of the object for the tasks of object grasping, manipulation and automatic assembly. Also, precise 6D poses in RGB images are of utmost importance in augmented reality (AR) applications, where overlaying 3D models on top of the real objects is critical for AR-driven assembly or repair tasks. With the rise of deep learning methods, demonstrating better performance than traditional template matching approaches or methods based on handcrafted features, it is essential to have appropriate datasets, which would encourage development and thorough evaluation of the new approaches.&#10;&#10;A number of 6D pose object datasets exist, each focusing on one of the aspects of this challenging task. For example, datasets like T-LESS~\cite{hodan2017tless} and LineMOD~\cite{hinterstoisserModelBasedTraining2013} cover textureless and target-specific object types in particular scenarios. LineMOD contains mainly toy and household objects of distinct geometry, concentrating on object detection in cluttered environments with nonexistent or minor occlusions. It includes a total of 15 separate image sequences, however each of them features 6D pose annotations only for a single object. T-LESS is a much bigger dataset, focusing exclusively on textureless industrial objects, which exhibit strong inter-object similarities and symmetries. In T-LESS, separate training and test images are explicitly provided, while it remains unclear how to sample or produce training data from LineMOD. This ambiguity has led to wide inconsistencies in presented results, making it hard to conduct a comprehensive assessment of the developed methods. Striving for the best possible results, researchers tend to train their detectors on subsets of provided real images with ground truth poses very similar to those in a test split. This strategy inevitably leads to overfitting to a particular dataset, restricts applicability of the detectors and undermines fair comparison of various approaches.&#10;&#10;One crucial aspect that has often been neglected by 6D pose detectors is scalability. The majority of the datasets contain a rather small number of objects. This is natural since producing large number of 3D models and scenes with annotated poses is a tedious task, especially when poses of all the objects in all frames must be available. When it comes to deep learning methods, training detectors on real data yields the best results. However, the fact that 3D models of the objects are available and training data can be synthesized by rendering them has been used in only a few studies, most notably using such detectors as SSD6D~\cite{kehlSSD6DMakingRGBbased2017b}, AAE~\cite{sundermeyerImplicit3DOrientation2018} and DPOD~\cite{zakharov2019dpod}. It is remarkable that all deep learning 6DoF object detectors trained either on real or synthetic data use a single neural network per object, in contrast to 2D object detectors, such as YOLO~\cite{redmonYouOnlyLook2015}, SSD~\cite{liuSSDSingleShot2016} or R-CNNs~\cite{girshickRichFeatureHierarchies2014a, girshickFastRcnn2015, renFasterRCNNRealTime2015, heMaskRCNN2017}, which use one network for all object classes. A major reason for this issue is the unavailability of a proper dataset with a variety of sequences and well-defined benchmarks, causing a fallback to a well-studied simplistic dataset such as LineMOD. Nevertheless, training one network per object defeats the scalability aspect that is naturally a characteristic of deep neural networks. Therefore, central aspects of our proposed dataset is its ability to test methods' scalability by introducing corresponding benchmarks and to encourage training of detectors on renderings of 3D models instead of using real data.&#10;&#10;Our dataset may be considered to be aligned with the OCCLUSION dataset~\cite{brachmannLearning6DObject2014a}, which contains poses of all the objects present in each frame. However, instead of just a single sequence with 1214 frames, our dataset contains&#10;13 full-circle scenes filmed with both PrimeSense Carmine 1.09 (structured light) and Microsoft Kinect 2 (time-of-light) RGB-D cameras resulting in a total of 34,830 fully annotated frames with poses for all objects in all frames. The complexity of the scenes increases from simple (several separated objects per scene) to heavily-cluttered and occluded (objects close together or on top of each other and also mixed with other objects not present as 3D models). Another aspect that is not addressed in the other datasets is strong variation of the&#10;illumination, including not only changes in light intensity, but also in light color. Finally, driven by the fact that in industry objects can undergo severe appearance changes, we created a benchmark where the object appearance is altered compared to the one in available 3D models. &#10;&#10;In the following sections, besides describing the steps of the dataset creation pipeline, we also present detection and 6D pose estimation results for all the newly-defined benchmarks of one of a recently introduced methods - Dense Pose Object Detector (DPOD)~\cite{zakharov2019dpod}. This method is trained on all the objects at once or on all the objects present in the test scene on strictly synthetic renderings of provided 3D models. The aim of our experiment was not only to test scalability of the current methods but also to demonstrate that even the best-performing methods trained on real images with one network per object obtain mediocre results when trained on multiple objects. This establishes a baseline and opens a new challenge to the community related mainly to scalability and synthetic training data. Light and appearance changes are another new aspect of this challenging dataset.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.16220" label="2305.16220">
-        <attvalues>
-          <attvalue for="0" value="On the Robustness of Segment Anything" />
-          <attvalue for="1" value="  Segment anything model (SAM) has presented impressive objectness&#10;identification capability with the idea of prompt learning and a new collected&#10;large-scale dataset. Given a prompt (e.g., points, bounding boxes, or masks)&#10;and an input image, SAM is able to generate valid segment masks for all objects&#10;indicated by the prompts, presenting high generalization across diverse&#10;scenarios and being a general method for zero-shot transfer to downstream&#10;vision tasks. Nevertheless, it remains unclear whether SAM may introduce errors&#10;in certain threatening scenarios. Clarifying this is of significant importance&#10;for applications that require robustness, such as autonomous vehicles. In this&#10;paper, we aim to study the testing-time robustness of SAM under adversarial&#10;scenarios and common corruptions. To this end, we first build a testing-time&#10;robustness evaluation benchmark for SAM by integrating existing public&#10;datasets. Second, we extend representative adversarial attacks against SAM and&#10;study the influence of different prompts on robustness. Third, we study the&#10;robustness of SAM under diverse corruption types by evaluating SAM on corrupted&#10;datasets with different prompts. With experiments conducted on SA-1B and KITTI&#10;datasets, we find that SAM exhibits remarkable robustness against various&#10;corruptions, except for blur-related corruption. Furthermore, SAM remains&#10;susceptible to adversarial attacks, particularly when subjected to PGD and BIM&#10;attacks. We think such a comprehensive study could highlight the importance of&#10;the robustness issues of SAM and trigger a series of new tasks for SAM as well&#10;as downstream vision tasks.&#10;" />
-          <attvalue for="2" value="&#10;With the emergence of large language models (\eg, ChatGPT \cite{openai2023gpt4}), it has been demonstrated that training these models on vast amounts of data can effectively enable them to learn common tasks. In the field of computer vision, the Segment Anything Model (SAM) \cite{kirillov2023segany} is the pioneering foundation model that adopts this paradigm. As a result, SAM exhibits remarkable generalization capabilities when applied to image segmentation tasks.&#10;&#10;SAM is a highly versatile model that has been trained on a comprehensive dataset (SA-1B), comprising over 1 billion masks derived from 11 million images. By utilizing prompt inputs such as points or bounding boxes, SAM demonstrates exceptional performance in zero-shot transfer learning for downstream segmentation tasks, even with new image distributions. Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}. Furthermore, SAM is also used as a foundation model for promoting other computer vision tasks such as semantic segmentation \cite{lin2018multi,lin2019zigzagnet,lin2022tagnet,shen2020ranet}, image editing \cite{yu2023inpaint,xie2023edit}, style transfer \cite{liu2023any}, object counting \cite{ma2023can}. Nevertheless, considering SAM's broad application scope, ensuring robustness becomes a paramount concern. &#10;A few notable works \cite{wang2023empirical, zhang2023attack} have explored the evaluation of SAM's robustness against corruption and adversarial attacks respectively. Compared with them, we not only employ the evaluation on SA-1B and KITTI datasets, \ie, examining the effectiveness of SAM under both identically distributed (IID) and out-of-distribution (OOD) conditions, but also discuss SAM's robustness in a more fine-grained way by evaluating SAM on relatively small and big objects. Furthermore, we additionally explore attacks specifically designed for segmentation tasks rather than classification tasks, which is more reasonable than that in \cite{zhang2023attack}. In experiments, we provide more detailed quantitative results than previous work, which can provide comprehensive information for a better understanding of the results.&#10;&#10;Our primary objective is to investigate the test-timing robustness of SAM in the face of adversarial scenarios intentionally created by humans, as well as common corruptions that occur naturally. To accomplish this, we establish an evaluation benchmark using publicly available datasets, specifically SA-1B and KITTI \cite{Alhaija2018IJCV}. The experiments conducted on the SA-1B dataset aim to assess SAM's robustness in an independently and identically distributed (IID) setting. Conversely, the experiments carried out on the KITTI focus on evaluating SAM's robustness in safety-critical applications under out-of-distribution (OOD) conditions. Additionally, we perform experiments to evaluate SAM's robustness concerning relatively small or large objects within the KITTI, which allows for a more detailed and comprehensive analysis. Surprisingly, our findings indicate that SAM exhibits remarkable robustness against various corruptions, except for blur-related corruption. Furthermore, our research reveals that SAM remains susceptible to adversarial attacks, particularly when subjected to PGD and BIM attacks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Adversarial Robustness Testing, Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Artificial Intelligence, Computer Science, Object Detection Models" />
-        </attvalues>
-      </node>
-      <node id="2303.08774" label="2303.08774">
-        <attvalues>
-          <attvalue for="0" value="GPT-4 Technical Report" />
-          <attvalue for="1" value="  We report the development of GPT-4, a large-scale, multimodal model which can&#10;accept image and text inputs and produce text outputs. While less capable than&#10;humans in many real-world scenarios, GPT-4 exhibits human-level performance on&#10;various professional and academic benchmarks, including passing a simulated bar&#10;exam with a score around the top 10% of test takers. GPT-4 is a&#10;Transformer-based model pre-trained to predict the next token in a document.&#10;The post-training alignment process results in improved performance on measures&#10;of factuality and adherence to desired behavior. A core component of this&#10;project was developing infrastructure and optimization methods that behave&#10;predictably across a wide range of scales. This allowed us to accurately&#10;predict some aspects of GPT-4's performance based on models trained with no&#10;more than 1/1,000th the compute of GPT-4.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.12620" label="2304.12620">
-        <attvalues>
-          <attvalue for="0" value="Medical SAM Adapter: Adapting Segment Anything Model for Medical Image&#10;  Segmentation" />
-          <attvalue for="1" value="  The Segment Anything Model (SAM) has recently gained popularity in the field&#10;of image segmentation due to its impressive capabilities in various&#10;segmentation tasks and its prompt-based interface. However, recent studies and&#10;individual experiments have shown that SAM underperforms in medical image&#10;segmentation, since the lack of the medical specific knowledge. This raises the&#10;question of how to enhance SAM's segmentation capability for medical images. In&#10;this paper, instead of fine-tuning the SAM model, we propose the Medical SAM&#10;Adapter (Med-SA), which incorporates domain-specific medical knowledge into the&#10;segmentation model using a light yet effective adaptation technique. In Med-SA,&#10;we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical&#10;images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned&#10;adaptation. We conduct comprehensive evaluation experiments on 17 medical image&#10;segmentation tasks across various image modalities. Med-SA outperforms several&#10;state-of-the-art (SOTA) medical image segmentation methods, while updating only&#10;2\% of the parameters. Our code is released at&#10;https://github.com/KidsWithTokens/Medical-SAM-Adapter.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.13785" label="2304.13785">
-        <attvalues>
-          <attvalue for="0" value="Customized Segment Anything Model for Medical Image Segmentation" />
-          <attvalue for="1" value="  We propose SAMed, a general solution for medical image segmentation.&#10;Different from the previous methods, SAMed is built upon the large-scale image&#10;segmentation model, Segment Anything Model (SAM), to explore the new research&#10;paradigm of customizing large-scale models for medical image segmentation.&#10;SAMed applies the low-rank-based (LoRA) finetuning strategy to the SAM image&#10;encoder and finetunes it together with the prompt encoder and the mask decoder&#10;on labeled medical image segmentation datasets. We also observe the warmup&#10;finetuning strategy and the AdamW optimizer lead SAMed to successful&#10;convergence and lower loss. Different from SAM, SAMed could perform semantic&#10;segmentation on medical images. Our trained SAMed model achieves 81.88 DSC and&#10;20.64 HD on the Synapse multi-organ segmentation dataset, which is on par with&#10;the state-of-the-art methods. We conduct extensive experiments to validate the&#10;effectiveness of our design. Since SAMed only updates a small fraction of the&#10;SAM parameters, its deployment cost and storage cost are quite marginal in&#10;practical usage. The code of SAMed is available at&#10;https://github.com/hitachinsk/SAMed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.13973" label="2304.13973">
-        <attvalues>
-          <attvalue for="0" value="SkinSAM: Empowering Skin Cancer Segmentation with Segment Anything Model" />
-          <attvalue for="1" value="  Skin cancer is a prevalent and potentially fatal disease that requires&#10;accurate and efficient diagnosis and treatment. Although manual tracing is the&#10;current standard in clinics, automated tools are desired to reduce human labor&#10;and improve accuracy. However, developing such tools is challenging due to the&#10;highly variable appearance of skin cancers and complex objects in the&#10;background. In this paper, we present SkinSAM, a fine-tuned model based on the&#10;Segment Anything Model that showed outstanding segmentation performance. The&#10;models are validated on HAM10000 dataset which includes 10015 dermatoscopic&#10;images. While larger models (ViT_L, ViT_H) performed better than the smaller&#10;one (ViT_b), the finetuned model (ViT_b_finetuned) exhibited the greatest&#10;improvement, with a Mean pixel accuracy of 0.945, Mean dice score of 0.8879,&#10;and Mean IoU score of 0.7843. Among the lesion types, vascular lesions showed&#10;the best segmentation results. Our research demonstrates the great potential of&#10;adapting SAM to medical image segmentation tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.00293" label="2305.00293">
-        <attvalues>
-          <attvalue for="0" value="Polyp-SAM: Transfer SAM for Polyp Segmentation" />
-          <attvalue for="1" value="  Colon polyps are considered important precursors for colorectal cancer.&#10;Automatic segmentation of colon polyps can significantly reduce the&#10;misdiagnosis of colon cancer and improve physician annotation efficiency. While&#10;many methods have been proposed for polyp segmentation, training large-scale&#10;segmentation networks with limited colonoscopy data remains a challenge.&#10;Recently, the Segment Anything Model (SAM) has recently gained much attention&#10;in both natural and medical image segmentation. SAM demonstrates superior&#10;performance in several image benchmarks and therefore shows great potential for&#10;medical image segmentation. In this study, we propose Poly-SAM, a finetuned SAM&#10;model for polyp segmentation, and compare its performance to several&#10;state-of-the-art polyp segmentation models. We also compare two transfer&#10;learning strategies of SAM with and without finetuning its encoders. Evaluated&#10;on five public datasets, our Polyp-SAM achieves state-of-the-art performance on&#10;two datasets and impressive performance on three datasets, with dice scores all&#10;above 88%. This study demonstrates the great potential of adapting SAM to&#10;medical image segmentation tasks. We plan to release the code and model weights&#10;for this paper at: https://github.com/ricklisz/Polyp-SAM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.04709" label="2304.04709">
-        <attvalues>
-          <attvalue for="0" value="Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection" />
-          <attvalue for="1" value="  SAM is a segmentation model recently released by Meta AI Research and has&#10;been gaining attention quickly due to its impressive performance in generic&#10;object segmentation. However, its ability to generalize to specific scenes such&#10;as camouflaged scenes is still unknown. Camouflaged object detection (COD)&#10;involves identifying objects that are seamlessly integrated into their&#10;surroundings and has numerous practical applications in fields such as&#10;medicine, art, and agriculture. In this study, we try to ask if SAM can address&#10;the COD task and evaluate the performance of SAM on the COD benchmark by&#10;employing maximum segmentation evaluation and camouflage location evaluation.&#10;We also compare SAM's performance with 22 state-of-the-art COD methods. Our&#10;results indicate that while SAM shows promise in generic object segmentation,&#10;its performance on the COD task is limited. This presents an opportunity for&#10;further research to explore how to build a stronger SAM that may address the&#10;COD task. The results of this paper are provided in&#10;\url{https://github.com/luckybird1994/SAMCOD}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.05750" label="2304.05750">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything Is Not Always Perfect: An Investigation of SAM on&#10;  Different Real-world Applications" />
-          <attvalue for="1" value="  Recently, Meta AI Research approaches a general, promptable Segment Anything&#10;Model (SAM) pre-trained on an unprecedentedly large segmentation dataset&#10;(SA-1B). Without a doubt, the emergence of SAM will yield significant benefits&#10;for a wide array of practical image segmentation applications. In this study,&#10;we conduct a series of intriguing investigations into the performance of SAM&#10;across various applications, particularly in the fields of natural images,&#10;agriculture, manufacturing, remote sensing, and healthcare. We analyze and&#10;discuss the benefits and limitations of SAM, while also presenting an outlook&#10;on its future development in segmentation tasks. By doing so, we aim to give a&#10;comprehensive understanding of SAM's practical applications. This work is&#10;expected to provide insights that facilitate future research activities toward&#10;generic segmentation. Source code is publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.14006" label="2304.14006">
-        <attvalues>
-          <attvalue for="0" value="Edit Everything: A Text-Guided Generative System for Images Editing" />
-          <attvalue for="1" value="  We introduce a new generative system called Edit Everything, which can take&#10;image and text inputs and produce image outputs. Edit Everything allows users&#10;to edit images using simple text instructions. Our system designs prompts to&#10;guide the visual module in generating requested images. Experiments demonstrate&#10;that Edit Everything facilitates the implementation of the visual aspects of&#10;Stable Diffusion with the use of Segment Anything model and CLIP. Our system is&#10;publicly available at https://github.com/DefengXie/Edit_Everything.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.09728" label="2304.09728">
-        <attvalues>
-          <attvalue for="0" value="Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate" />
-          <attvalue for="1" value="  Style transfer aims to render the style of a given image for style reference&#10;to another given image for content reference, and has been widely adopted in&#10;artistic generation and image editing. Existing approaches either apply the&#10;holistic style of the style image in a global manner, or migrate local colors&#10;and textures of the style image to the content counterparts in a pre-defined&#10;way. In either case, only one result can be generated for a specific pair of&#10;content and style images, which therefore lacks flexibility and is hard to&#10;satisfy different users with different preferences. We propose here a novel&#10;strategy termed Any-to-Any Style Transfer to address this drawback, which&#10;enables users to interactively select styles of regions in the style image and&#10;apply them to the prescribed content regions. In this way, personalizable style&#10;transfer is achieved through human-computer interaction. At the heart of our&#10;approach lies in (1) a region segmentation module based on Segment Anything,&#10;which supports region selection with only some clicks or drawing on images and&#10;thus takes user inputs conveniently and flexibly; (2) and an attention fusion&#10;module, which converts inputs from users to controlling signals for the style&#10;transfer model. Experiments demonstrate the effectiveness for personalizable&#10;style transfer. Notably, our approach performs in a plug-and-play manner&#10;portable to any style transfer method and enhance the controllablity. Our code&#10;is available \href{https://github.com/Huage001/Transfer-Any-Style}{here}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.10817" label="2304.10817">
-        <attvalues>
-          <attvalue for="0" value="Can SAM Count Anything? An Empirical Study on SAM Counting" />
-          <attvalue for="1" value="  Meta AI recently released the Segment Anything model (SAM), which has&#10;garnered attention due to its impressive performance in class-agnostic&#10;segmenting. In this study, we explore the use of SAM for the challenging task&#10;of few-shot object counting, which involves counting objects of an unseen&#10;category by providing a few bounding boxes of examples. We compare SAM's&#10;performance with other few-shot counting methods and find that it is currently&#10;unsatisfactory without further fine-tuning, particularly for small and crowded&#10;objects. Code can be found at&#10;\url{https://github.com/Vision-Intelligence-and-Robots-Group/count-anything}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.06422" label="2305.06422">
-        <attvalues>
-          <attvalue for="0" value="An Empirical Study on the Robustness of the Segment Anything Model (SAM)" />
-          <attvalue for="1" value="  The Segment Anything Model (SAM) is a foundation model for general image&#10;segmentation. Although it exhibits impressive performance predominantly on&#10;natural images, understanding its robustness against various image&#10;perturbations and domains is critical for real-world applications where such&#10;challenges frequently arise. In this study we conduct a comprehensive&#10;robustness investigation of SAM under diverse real-world conditions. Our&#10;experiments encompass a wide range of image perturbations. Our experimental&#10;results demonstrate that SAM's performance generally declines under perturbed&#10;images, with varying degrees of vulnerability across different perturbations.&#10;By customizing prompting techniques and leveraging domain knowledge based on&#10;the unique characteristics of each dataset, the model's resilience to these&#10;perturbations can be enhanced, addressing dataset-specific challenges. This&#10;work sheds light on the limitations and strengths of SAM in real-world&#10;applications, promoting the development of more robust and versatile image&#10;segmentation solutions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.08641" label="2212.08641">
-        <attvalues>
-          <attvalue for="0" value="GFPose: Learning 3D Human Pose Prior with Gradient Fields" />
-          <attvalue for="1" value="  Learning 3D human pose prior is essential to human-centered AI. Here, we&#10;present GFPose, a versatile framework to model plausible 3D human poses for&#10;various applications. At the core of GFPose is a time-dependent score network,&#10;which estimates the gradient on each body joint and progressively denoises the&#10;perturbed 3D human pose to match a given task specification. During the&#10;denoising process, GFPose implicitly incorporates pose priors in gradients and&#10;unifies various discriminative and generative tasks in an elegant framework.&#10;Despite the simplicity, GFPose demonstrates great potential in several&#10;downstream tasks. Our experiments empirically show that 1) as a&#10;multi-hypothesis pose estimator, GFPose outperforms existing SOTAs by 20% on&#10;Human3.6M dataset. 2) as a single-hypothesis pose estimator, GFPose achieves&#10;comparable results to deterministic SOTAs, even with a vanilla backbone. 3)&#10;GFPose is able to produce diverse and realistic samples in pose denoising,&#10;completion and generation tasks. Project page&#10;https://sites.google.com/view/gfpose/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="2011.05669" label="2011.05669">
-        <attvalues>
-          <attvalue for="0" value="A Hybrid Approach for 6DoF Pose Estimation" />
-          <attvalue for="1" value="  We propose a method for 6DoF pose estimation of rigid objects that uses a&#10;state-of-the-art deep learning based instance detector to segment object&#10;instances in an RGB image, followed by a point-pair based voting method to&#10;recover the object's pose. We additionally use an automatic method selection&#10;that chooses the instance detector and the training set as that with the&#10;highest performance on the validation set. This hybrid approach leverages the&#10;best of learning and classic approaches, using CNNs to filter highly&#10;unstructured data and cut through the clutter, and a local geometric approach&#10;with proven convergence for robust pose estimation. The method is evaluated on&#10;the BOP core datasets where it significantly exceeds the baseline method and is&#10;the best fast method in the BOP 2020 Challenge.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Many of the recently published approaches for 6DoF object detection, especially on the BOP benchmark~\cite{hodan2018bop}, follow a two-stage pipeline. The first stage is a state of the art deep learning object detector that outputs the potential locations of object instances, often as bounding boxes or instance masks. The second stage iterates over those instances and, for each, estimates the instance's pose.&#10;&#10;The main technical differences in the approaches are in the second, pose estimation stage. The deep-learning based approaches can roughly be categorized by the type of data representation they use (mostly image-based convolutions~\cite{park2019pix2pose,Sundermeyer_2020_CVPR,sundermeyer2020augmented,li2019cdpn} vs. Graph-based networks~\cite{gao2018occlusion}), and the kind of pose estimation they employ (direct regression of pose parameters~\cite{gao2018occlusion}, regression of 3D model coordinates of the scene points usually followed by PnP~\cite{park2019pix2pose,li2019cdpn,DBLP:journals/corr/abs-1902-11020,hodan2020epos}, or employing a codebook for estimating the rotation of an object~\cite{Sundermeyer_2020_CVPR,sundermeyer2020augmented}).&#10;&#10;A key observation from the BOP 2019 challenge was that while point-pair voting based methods~\cite{drost2010model,vidal2018method}, which are not learning based, overall had the highest recognition rates, they were also among the slowest methods. One reason is the large search space, as the voting is performed on the complete scene without pre-segmentation or pre-detection of instances. Well-trained deep-learning based instance segmentation methods, on the other hand, estimate the locations of the objects in a scene rather fast.&#10;&#10;We therefore combine the two approaches and perform the point-pair voting of~\cite{drost2010model} only on locations returned by an instance segmentation network. This combines the advantages of both methods: the deep network's ability to quickly filter through complex real-world data and to narrow down the search space, and the provable robustness of the point-pair voting for recovering the pose.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning Methods, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1808.05498" label="1808.05498">
-        <attvalues>
-          <attvalue for="0" value="Occlusion Resistant Object Rotation Regression from Point Cloud Segments" />
-          <attvalue for="1" value="  Rotation estimation of known rigid objects is important for robotic&#10;applications such as dexterous manipulation. Most existing methods for rotation&#10;estimation use intermediate representations such as templates, global or local&#10;feature descriptors, or object coordinates, which require multiple steps in&#10;order to infer the object pose. We propose to directly regress a pose vector&#10;from raw point cloud segments using a convolutional neural network.&#10;Experimental results show that our method can potentially achieve competitive&#10;performance compared to a state-of-the-art method, while also showing more&#10;robustness against occlusion. Our method does not require any post processing&#10;such as refinement with the iterative closest point algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.04231" label="1911.04231">
-        <attvalues>
-          <attvalue for="0" value="PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose&#10;  Estimation" />
-          <attvalue for="1" value="  In this work, we present a novel data-driven method for robust 6DoF object&#10;pose estimation from a single RGBD image. Unlike previous methods that directly&#10;regressing pose parameters, we tackle this challenging task with a&#10;keypoint-based approach. Specifically, we propose a deep Hough voting network&#10;to detect 3D keypoints of objects and then estimate the 6D pose parameters&#10;within a least-squares fitting manner. Our method is a natural extension of&#10;2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It&#10;allows us to fully utilize the geometric constraint of rigid objects with the&#10;extra depth information and is easy for a network to learn and optimize.&#10;Extensive experiments were conducted to demonstrate the effectiveness of&#10;3D-keypoint detection in the 6D pose estimation task. Experimental results also&#10;show our method outperforms the state-of-the-art methods by large margins on&#10;several benchmarks. Code and video are available at&#10;https://github.com/ethnhe/PVN3D.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning Methods, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1705.01156" label="1705.01156">
-        <attvalues>
-          <attvalue for="0" value="Shading Annotations in the Wild" />
-          <attvalue for="1" value="  Understanding shading effects in images is critical for a variety of vision&#10;and graphics problems, including intrinsic image decomposition, shadow removal,&#10;image relighting, and inverse rendering. As is the case with other vision&#10;tasks, machine learning is a promising approach to understanding shading - but&#10;there is little ground truth shading data available for real-world images. We&#10;introduce Shading Annotations in the Wild (SAW), a new large-scale, public&#10;dataset of shading annotations in indoor scenes, comprised of multiple forms of&#10;shading judgments obtained via crowdsourcing, along with shading annotations&#10;automatically generated from RGB-D imagery. We use this data to train a&#10;convolutional neural network to predict per-pixel shading information in an&#10;image. We demonstrate the value of our data and network in an application to&#10;intrinsic images, where we can reduce decomposition artifacts produced by&#10;existing algorithms. Our database is available at&#10;http://opensurfaces.cs.cornell.edu/saw/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.00582" label="1804.00582">
-        <attvalues>
-          <attvalue for="0" value="Learning Intrinsic Image Decomposition from Watching the World" />
-          <attvalue for="1" value="  Single-view intrinsic image decomposition is a highly ill-posed problem, and&#10;so a promising approach is to learn from large amounts of data. However, it is&#10;difficult to collect ground truth training data at scale for intrinsic images.&#10;In this paper, we explore a different approach to learning intrinsic images:&#10;observing image sequences over time depicting the same scene under changing&#10;illumination, and learning single-view decompositions that are consistent with&#10;these changes. This approach allows us to learn without ground truth&#10;decompositions, and to instead exploit information available from multiple&#10;images when training. Our trained model can then be applied at test time to&#10;single views. We describe a new learning framework based on this idea,&#10;including new loss functions that can be efficiently evaluated over entire&#10;sequences. While prior learning-based methods achieve good performance on&#10;specific benchmarks, we show that our approach generalizes well to several&#10;diverse datasets, including MIT intrinsic images, Intrinsic Images in the Wild&#10;and Shading Annotations in the Wild.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.00824" label="1905.00824">
-        <attvalues>
-          <attvalue for="0" value="Single Image Portrait Relighting" />
-          <attvalue for="1" value="  Lighting plays a central role in conveying the essence and depth of the&#10;subject in a portrait photograph. Professional photographers will carefully&#10;control the lighting in their studio to manipulate the appearance of their&#10;subject, while consumer photographers are usually constrained to the&#10;illumination of their environment. Though prior works have explored techniques&#10;for relighting an image, their utility is usually limited due to requirements&#10;of specialized hardware, multiple images of the subject under controlled or&#10;known illuminations, or accurate models of geometry and reflectance. To this&#10;end, we present a system for portrait relighting: a neural network that takes&#10;as input a single RGB image of a portrait taken with a standard cellphone&#10;camera in an unconstrained environment, and from that image produces a relit&#10;image of that subject as though it were illuminated according to any provided&#10;environment map. Our method is trained on a small database of 18 individuals&#10;captured under different directional light sources in a controlled light stage&#10;setup consisting of a densely sampled sphere of lights. Our proposed technique&#10;produces quantitatively superior results on our dataset's validation set&#10;compared to prior works, and produces convincing qualitative relighting results&#10;on a dataset of hundreds of real-world cellphone portraits. Because our&#10;technique can produce a 640 $\times$ 640 image in only 160 milliseconds, it may&#10;enable interactive user-facing photographic applications in the future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.02333" label="2006.02333">
-        <attvalues>
-          <attvalue for="0" value="Scene relighting with illumination estimation in the latent space on an&#10;  encoder-decoder scheme" />
-          <attvalue for="1" value="  The image relighting task of transferring illumination conditions between two&#10;images offers an interesting and difficult challenge with potential&#10;applications in photography, cinematography and computer graphics. In this&#10;report we present methods that we tried to achieve that goal. Our models are&#10;trained on a rendered dataset of artificial locations with varied scene&#10;content, light source location and color temperature. With this dataset, we&#10;used a network with illumination estimation component aiming to infer and&#10;replace light conditions in the latent space representation of the concerned&#10;scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.10773" label="2001.10773">
-        <attvalues>
-          <attvalue for="0" value="Virtual KITTI 2" />
-          <attvalue for="1" value="  This paper introduces an updated version of the well-known Virtual KITTI&#10;dataset which consists of 5 sequence clones from the KITTI tracking benchmark.&#10;In addition, the dataset provides different variants of these sequences such as&#10;modified weather conditions (e.g. fog, rain) or modified camera configurations&#10;(e.g. rotated by 15 degrees). For each sequence, we provide multiple sets of&#10;images containing RGB, depth, class segmentation, instance segmentation, flow,&#10;and scene flow data. Camera parameters and poses as well as vehicle locations&#10;are available as well. In order to showcase some of the dataset's capabilities,&#10;we ran multiple relevant experiments using state-of-the-art algorithms from the&#10;field of autonomous driving. The dataset is available for download at&#10;https://europe.naverlabs.com/Research/Computer-Vision/Proxy-Virtual-Worlds.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.14686" label="2401.14686">
-        <attvalues>
-          <attvalue for="0" value="SSR: SAM is a Strong Regularizer for domain adaptive semantic&#10;  segmentation" />
-          <attvalue for="1" value="  We introduced SSR, which utilizes SAM (segment-anything) as a strong&#10;regularizer during training, to greatly enhance the robustness of the image&#10;encoder for handling various domains. Specifically, given the fact that SAM is&#10;pre-trained with a large number of images over the internet, which cover a&#10;diverse variety of domains, the feature encoding extracted by the SAM is&#10;obviously less dependent on specific domains when compared to the traditional&#10;ImageNet pre-trained image encoder. Meanwhile, the ImageNet pre-trained image&#10;encoder is still a mature choice of backbone for the semantic segmentation&#10;task, especially when the SAM is category-irrelevant. As a result, our SSR&#10;provides a simple yet highly effective design. It uses the ImageNet pre-trained&#10;image encoder as the backbone, and the intermediate feature of each stage (ie&#10;there are 4 stages in MiT-B5) is regularized by SAM during training. After&#10;extensive experimentation on GTA5$\rightarrow$Cityscapes, our SSR significantly&#10;improved performance over the baseline without introducing any extra inference&#10;overhead.&#10;" />
-          <attvalue for="2" value="&#10;Image semantic segmentation is a challenging task in computer vision. &#10;This technique classifies each pixel of an image and serves as a foundation for visual representation learning.&#10;Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e.g\onedot VOC2012, Cityscapes and COCOStuff).&#10;&#10;In supervised learning, each pixel in an image needs to be labeled, which is also known as dense labeling. This makes semantic segmentation a costly process. For instance, labeling a Cityscapes image can take up to 2 hours for a professional. This high labeling cost leads to limited training data and a lack of diversity, which in turn causes the model to be only generalized on a small domain and has a large domain gap with many others. Ultimately, this affects the robustness of the semantic segmentation model.&#10;&#10;Unsupervised Domain Adaptation (UDA) is a research field that aims to tackle the problem of domain gap. Current UDA methods typically focus on improving the model or using more advanced data augmentation strategies. Despite significant progress in recent years, their performance is still noticeably inferior to that of purely supervised models.&#10;&#10;In this work, we were inspired by some of the latest ideas in the field, such as the foundation model and big data. We didn't want to limit ourselves to existing UDA patterns, so we decided to use big data to explore new ways of improving UDA performance.&#10;Due to budget constraints, we utilize the foundation model to benefit from internet data indirectly.&#10;&#10;SAM (Segment-anything~\cite{cSAM}) is a vision foundation model trained on a massive-scale dataset (SA-1B). The feature map that SAM produced, though it cannot directly make pixel classification, is obviously much more domain-independent than a traditional pixel-segmentation model trained on a small dataset.&#10;Thus, We proposed SSR (SAM is a Strong regularize), using the relatively robust SAM feature map to regularize our model during training to learn much more domain-independent weights for UDA, and during inference, SAM is no longer required for the efficient purpose. Thus, the proposed SSR does not cause extra inference overhead. Experimentally, after the SAM intervention during training, our model's performance significantly improved compared to the baseline.&#10;&#10;To summarize, our contributions are listed below:&#10;\begin{itemize}&#10; \item We use SAM to indirectly take advantage of big data during training and significantly improve the model performance on UDA.&#10; \item We have designed a `shadow branch` parallel to the SAM regularization so that SAM is not required for inference and avoids extra inference overhead. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Image Encoding, Computer Science, Domain Robustness" />
-        </attvalues>
-      </node>
-      <node id="1412.7062" label="1412.7062">
-        <attvalues>
-          <attvalue for="0" value="Semantic Image Segmentation with Deep Convolutional Nets and Fully&#10;  Connected CRFs" />
-          <attvalue for="1" value="  Deep Convolutional Neural Networks (DCNNs) have recently shown state of the&#10;art performance in high level vision tasks, such as image classification and&#10;object detection. This work brings together methods from DCNNs and&#10;probabilistic graphical models for addressing the task of pixel-level&#10;classification (also called &quot;semantic image segmentation&quot;). We show that&#10;responses at the final layer of DCNNs are not sufficiently localized for&#10;accurate object segmentation. This is due to the very invariance properties&#10;that make DCNNs good for high level tasks. We overcome this poor localization&#10;property of deep networks by combining the responses at the final DCNN layer&#10;with a fully connected Conditional Random Field (CRF). Qualitatively, our&#10;&quot;DeepLab&quot; system is able to localize segment boundaries at a level of accuracy&#10;which is beyond previous methods. Quantitatively, our method sets the new&#10;state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching&#10;71.6% IOU accuracy in the test set. We show how these results can be obtained&#10;efficiently: Careful network re-purposing and a novel application of the 'hole'&#10;algorithm from the wavelet community allow dense computation of neural net&#10;responses at 8 frames per second on a modern GPU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.03821" label="1804.03821">
-        <attvalues>
-          <attvalue for="0" value="ExFuse: Enhancing Feature Fusion for Semantic Segmentation" />
-          <attvalue for="1" value="  Modern semantic segmentation frameworks usually combine low-level and&#10;high-level features from pre-trained backbone convolutional models to boost&#10;performance. In this paper, we first point out that a simple fusion of&#10;low-level and high-level features could be less effective because of the gap in&#10;semantic levels and spatial resolution. We find that introducing semantic&#10;information into low-level features and high-resolution details into high-level&#10;features is more effective for the later fusion. Based on this observation, we&#10;propose a new framework, named ExFuse, to bridge the gap between low-level and&#10;high-level features thus significantly improve the segmentation quality by&#10;4.0\% in total. Furthermore, we evaluate our approach on the challenging PASCAL&#10;VOC 2012 segmentation benchmark and achieve 87.9\% mean IoU, which outperforms&#10;the previous state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.11721" label="1811.11721">
-        <attvalues>
-          <attvalue for="0" value="CCNet: Criss-Cross Attention for Semantic Segmentation" />
-          <attvalue for="1" value="  Contextual information is vital in visual understanding problems, such as&#10;semantic segmentation and object detection. We propose a Criss-Cross Network&#10;(CCNet) for obtaining full-image contextual information in a very effective and&#10;efficient way. Concretely, for each pixel, a novel criss-cross attention module&#10;harvests the contextual information of all the pixels on its criss-cross path.&#10;By taking a further recurrent operation, each pixel can finally capture the&#10;full-image dependencies. Besides, a category consistent loss is proposed to&#10;enforce the criss-cross attention module to produce more discriminative&#10;features. Overall, CCNet is with the following merits: 1) GPU memory friendly.&#10;Compared with the non-local block, the proposed recurrent criss-cross attention&#10;module requires 11x less GPU memory usage. 2) High computational efficiency.&#10;The recurrent criss-cross attention significantly reduces FLOPs by about 85% of&#10;the non-local block. 3) The state-of-the-art performance. We conduct extensive&#10;experiments on semantic segmentation benchmarks including Cityscapes, ADE20K,&#10;human parsing benchmark LIP, instance segmentation benchmark COCO, video&#10;segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU&#10;scores of 81.9%, 45.76% and 55.47% on the Cityscapes test set, the ADE20K&#10;validation set and the LIP validation set respectively, which are the new&#10;state-of-the-art results. The source codes are available at&#10;\url{https://github.com/speedinghzl/CCNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.07678" label="1908.07678">
-        <attvalues>
-          <attvalue for="0" value="Asymmetric Non-local Neural Networks for Semantic Segmentation" />
-          <attvalue for="1" value="  The non-local module works as a particularly useful technique for semantic&#10;segmentation while criticized for its prohibitive computation and GPU memory&#10;occupation. In this paper, we present Asymmetric Non-local Neural Network to&#10;semantic segmentation, which has two prominent components: Asymmetric Pyramid&#10;Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB&#10;leverages a pyramid sampling module into the non-local block to largely reduce&#10;the computation and memory consumption without sacrificing the performance.&#10;AFNB is adapted from APNB to fuse the features of different levels under a&#10;sufficient consideration of long range dependencies and thus considerably&#10;improves the performance. Extensive experiments on semantic segmentation&#10;benchmarks demonstrate the effectiveness and efficiency of our work. In&#10;particular, we report the state-of-the-art performance of 81.3 mIoU on the&#10;Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a&#10;non-local block on GPU while 28 times smaller in GPU running memory occupation.&#10;Code is available at: https://github.com/MendelXu/ANN.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.00916" label="1809.00916">
-        <attvalues>
-          <attvalue for="0" value="OCNet: Object Context Network for Scene Parsing" />
-          <attvalue for="1" value="  In this paper, we address the semantic segmentation task with a new context&#10;aggregation scheme named \emph{object context}, which focuses on enhancing the&#10;role of object information. Motivated by the fact that the category of each&#10;pixel is inherited from the object it belongs to, we define the object context&#10;for each pixel as the set of pixels that belong to the same category as the&#10;given pixel in the image. We use a binary relation matrix to represent the&#10;relationship between all pixels, where the value one indicates the two selected&#10;pixels belong to the same category and zero otherwise.&#10;  We propose to use a dense relation matrix to serve as a surrogate for the&#10;binary relation matrix. The dense relation matrix is capable to emphasize the&#10;contribution of object information as the relation scores tend to be larger on&#10;the object pixels than the other pixels. Considering that the dense relation&#10;matrix estimation requires quadratic computation overhead and memory&#10;consumption w.r.t. the input size, we propose an efficient interlaced sparse&#10;self-attention scheme to model the dense relations between any two of all&#10;pixels via the combination of two sparse relation matrices.&#10;  To capture richer context information, we further combine our interlaced&#10;sparse self-attention scheme with the conventional multi-scale context schemes&#10;including pyramid pooling~\citep{zhao2017pyramid} and atrous spatial pyramid&#10;pooling~\citep{chen2018deeplab}. We empirically show the advantages of our&#10;approach with competitive performances on five challenging benchmarks&#10;including: Cityscapes, ADE20K, LIP, PASCAL-Context and COCO-Stuff&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.02651" label="1909.02651">
-        <attvalues>
-          <attvalue for="0" value="Semantic Correlation Promoted Shape-Variant Context for Segmentation" />
-          <attvalue for="1" value="  Context is essential for semantic segmentation. Due to the diverse shapes of&#10;objects and their complex layout in various scene images, the spatial scales&#10;and shapes of contexts for different objects have very large variation. It is&#10;thus ineffective or inefficient to aggregate various context information from a&#10;predefined fixed region. In this work, we propose to generate a scale- and&#10;shape-variant semantic mask for each pixel to confine its contextual region. To&#10;this end, we first propose a novel paired convolution to infer the semantic&#10;correlation of the pair and based on that to generate a shape mask. Using the&#10;inferred spatial scope of the contextual region, we propose a shape-variant&#10;convolution, of which the receptive field is controlled by the shape mask that&#10;varies with the appearance of input. In this way, the proposed network&#10;aggregates the context information of a pixel from its semantic-correlated&#10;region instead of a predefined fixed region. Furthermore, this work also&#10;proposes a labeling denoising model to reduce wrong predictions caused by the&#10;noisy low-level features. Without bells and whistles, the proposed segmentation&#10;network achieves new state-of-the-arts consistently on the six public&#10;segmentation datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.01547" label="2004.01547">
-        <attvalues>
-          <attvalue for="0" value="Context Prior for Scene Segmentation" />
-          <attvalue for="1" value="  Recent works have widely explored the contextual dependencies to achieve more&#10;accurate segmentation results. However, most approaches rarely distinguish&#10;different types of contextual dependencies, which may pollute the scene&#10;understanding. In this work, we directly supervise the feature aggregation to&#10;distinguish the intra-class and inter-class context clearly. Specifically, we&#10;develop a Context Prior with the supervision of the Affinity Loss. Given an&#10;input image and corresponding ground truth, Affinity Loss constructs an ideal&#10;affinity map to supervise the learning of Context Prior. The learned Context&#10;Prior extracts the pixels belonging to the same category, while the reversed&#10;prior focuses on the pixels of different classes. Embedded into a conventional&#10;deep CNN, the proposed Context Prior Layer can selectively capture the&#10;intra-class and inter-class contextual dependencies, leading to robust feature&#10;representation. To validate the effectiveness, we design an effective Context&#10;Prior Network (CPNet). Extensive quantitative and qualitative evaluations&#10;demonstrate that the proposed model performs favorably against state-of-the-art&#10;semantic segmentation approaches. More specifically, our algorithm achieves&#10;46.3% mIoU on ADE20K, 53.9% mIoU on PASCAL-Context, and 81.3% mIoU on&#10;Cityscapes. Code is available at https://git.io/ContextPrior.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.09408" label="1909.09408">
-        <attvalues>
-          <attvalue for="0" value="ACFNet: Attentional Class Feature Network for Semantic Segmentation" />
-          <attvalue for="1" value="  Recent works have made great progress in semantic segmentation by exploiting&#10;richer context, most of which are designed from a spatial perspective. In&#10;contrast to previous works, we present the concept of class center which&#10;extracts the global context from a categorical perspective. This class-level&#10;context describes the overall representation of each class in an image. We&#10;further propose a novel module, named Attentional Class Feature (ACF) module,&#10;to calculate and adaptively combine different class centers according to each&#10;pixel. Based on the ACF module, we introduce a coarse-to-fine segmentation&#10;network, called Attentional Class Feature Network (ACFNet), which can be&#10;composed of an ACF module and any off-the-shell segmentation network (base&#10;network). In this paper, we use two types of base networks to evaluate the&#10;effectiveness of ACFNet. We achieve new state-of-the-art performance of 81.85%&#10;mIoU on Cityscapes dataset with only finely annotated data used for training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.13426" label="1907.13426">
-        <attvalues>
-          <attvalue for="0" value="Expectation-Maximization Attention Networks for Semantic Segmentation" />
-          <attvalue for="1" value="  Self-attention mechanism has been widely used for various tasks. It is&#10;designed to compute the representation of each position by a weighted sum of&#10;the features at all positions. Thus, it can capture long-range relations for&#10;computer vision tasks. However, it is computationally consuming. Since the&#10;attention maps are computed w.r.t all other positions. In this paper, we&#10;formulate the attention mechanism into an expectation-maximization manner and&#10;iteratively estimate a much more compact set of bases upon which the attention&#10;maps are computed. By a weighted summation upon these bases, the resulting&#10;representation is low-rank and deprecates noisy information from the input. The&#10;proposed Expectation-Maximization Attention (EMA) module is robust to the&#10;variance of input and is also friendly in memory and computation. Moreover, we&#10;set up the bases maintenance and normalization methods to stabilize its&#10;training procedure. We conduct extensive experiments on popular semantic&#10;segmentation benchmarks including PASCAL VOC, PASCAL Context and COCO Stuff, on&#10;which we set new records.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.05740" label="1907.05740">
-        <attvalues>
-          <attvalue for="0" value="Gated-SCNN: Gated Shape CNNs for Semantic Segmentation" />
-          <attvalue for="1" value="  Current state-of-the-art methods for image segmentation form a dense image&#10;representation where the color, shape and texture information are all processed&#10;together inside a deep CNN. This however may not be ideal as they contain very&#10;different type of information relevant for recognition. Here, we propose a new&#10;two-stream CNN architecture for semantic segmentation that explicitly wires&#10;shape information as a separate processing branch, i.e. shape stream, that&#10;processes information in parallel to the classical stream. Key to this&#10;architecture is a new type of gates that connect the intermediate layers of the&#10;two streams. Specifically, we use the higher-level activations in the classical&#10;stream to gate the lower-level activations in the shape stream, effectively&#10;removing noise and helping the shape stream to only focus on processing the&#10;relevant boundary-related information. This enables us to use a very shallow&#10;architecture for the shape stream that operates on the image-level resolution.&#10;Our experiments show that this leads to a highly effective architecture that&#10;produces sharper predictions around object boundaries and significantly boosts&#10;performance on thinner and smaller objects. Our method achieves&#10;state-of-the-art performance on the Cityscapes benchmark, in terms of both mask&#10;(mIoU) and boundary (F-score) quality, improving by 2% and 4% over strong&#10;baselines.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.07160" label="2203.07160">
-        <attvalues>
-          <attvalue for="0" value="CAR: Class-aware Regularizations for Semantic Segmentation" />
-          <attvalue for="1" value="  Recent segmentation methods, such as OCR and CPNet, utilizing &quot;class level&quot;&#10;information in addition to pixel features, have achieved notable success for&#10;boosting the accuracy of existing network modules. However, the extracted&#10;class-level information was simply concatenated to pixel features, without&#10;explicitly being exploited for better pixel representation learning. Moreover,&#10;these approaches learn soft class centers based on coarse mask prediction,&#10;which is prone to error accumulation. In this paper, aiming to use class level&#10;information more effectively, we propose a universal Class-Aware Regularization&#10;(CAR) approach to optimize the intra-class variance and inter-class distance&#10;during feature learning, motivated by the fact that humans can recognize an&#10;object by itself no matter which other objects it appears with. Three novel&#10;loss functions are proposed. The first loss function encourages more compact&#10;class representations within each class, the second directly maximizes the&#10;distance between different class centers, and the third further pushes the&#10;distance between inter-class centers and pixels. Furthermore, the class center&#10;in our approach is directly generated from ground truth instead of from the&#10;error-prone coarse prediction. Our method can be easily applied to most&#10;existing segmentation models during training, including OCR and CPNet, and can&#10;largely improve their accuracy at no additional inference overhead. Extensive&#10;experiments and ablation studies conducted on multiple benchmark datasets&#10;demonstrate that the proposed CAR can boost the accuracy of all baseline models&#10;by up to 2.23% mIOU with superior generalization ability. The complete code is&#10;available at https://github.com/edwardyehuang/CAR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.11872" label="2312.11872">
-        <attvalues>
-          <attvalue for="0" value="Beyond Prototypes: Semantic Anchor Regularization for Better&#10;  Representation Learning" />
-          <attvalue for="1" value="  One of the ultimate goals of representation learning is to achieve&#10;compactness within a class and well-separability between classes. Many&#10;outstanding metric-based and prototype-based methods following the&#10;Expectation-Maximization paradigm, have been proposed for this objective.&#10;However, they inevitably introduce biases into the learning process,&#10;particularly with long-tail distributed training data. In this paper, we reveal&#10;that the class prototype is not necessarily to be derived from training&#10;features and propose a novel perspective to use pre-defined class anchors&#10;serving as feature centroid to unidirectionally guide feature learning.&#10;However, the pre-defined anchors may have a large semantic distance from the&#10;pixel features, which prevents them from being directly applied. To address&#10;this issue and generate feature centroid independent from feature learning, a&#10;simple yet effective Semantic Anchor Regularization (SAR) is proposed. SAR&#10;ensures the interclass separability of semantic anchors in the semantic space&#10;by employing a classifier-aware auxiliary cross-entropy loss during training&#10;via disentanglement learning. By pulling the learned features to these semantic&#10;anchors, several advantages can be attained: 1) the intra-class compactness and&#10;naturally inter-class separability, 2) induced bias or errors from feature&#10;learning can be avoided, and 3) robustness to the long-tailed problem. The&#10;proposed SAR can be used in a plug-and-play manner in the existing models.&#10;Extensive experiments demonstrate that the SAR performs better than previous&#10;sophisticated prototype-based methods. The implementation is available at&#10;https://github.com/geyanqi/SAR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.12828" label="2201.12828">
-        <attvalues>
-          <attvalue for="0" value="Comprehensive Saliency Fusion for Object Co-segmentation" />
-          <attvalue for="1" value="  Object co-segmentation has drawn significant attention in recent years,&#10;thanks to its clarity on the expected foreground, the shared object in a group&#10;of images. Saliency fusion has been one of the promising ways to carry it out.&#10;However, prior works either fuse saliency maps of the same image or saliency&#10;maps of different images to extract the expected foregrounds. Also, they rely&#10;on hand-crafted saliency extraction and correspondence processes in most cases.&#10;This paper revisits the problem and proposes fusing saliency maps of both the&#10;same image and different images. It also leverages advances in deep learning&#10;for the saliency extraction and correspondence processes. Hence, we call it&#10;comprehensive saliency fusion. Our experiments reveal that our approach&#10;achieves much-improved object co-segmentation results compared to prior works&#10;on important benchmark datasets such as iCoseg, MSRC, and Internet Images.&#10;" />
-          <attvalue for="2" value="&#10;Object co-segmentation is the task of extracting common objects as foregrounds in a group of images. Compared to the single image segmentation, where it's difficult to ascertain what's foreground, we have a clear-cut definition of what foreground we wish to extract in the object co-segmentation task. Rother $et. al.$ \cite{rother2006cosegmentation} first introduced the concept of co-segmentation by developing a histogram matching method to extract common parts from a pair of images. However, Vicente $et. al.$ \cite{vicente2011object} were the first to propose that co-segmentation should be about things (or objects) and there is a need to incorporate a measure of objectness in the models. They proposed a Random Forest classifier to find the similarity between a pair of proposed segmentation of objects in each image of a group followed by the A* search algorithm \cite{andres2010empirical} to find the segmented objects with maximum similarity score. The major applications of object co-segmentation include image grouping\cite{kim2012hierarchical}, object recognition\cite{gallagher2008clothing}, and object tracking\cite{tsai2016semantic} which are among the fundamental tasks of Computer Vision.\par&#10;&#10;The problem of object co-segmentation is highly correlated to finding the common regions of interest in a group of images. \cite{jerripothula2014automatic, jerripothula2015group} approach the problem of co-segmentation by trying to highlight these common regions of interest. \cite{meng2012object} proposed a shortest path algorithm between the salient regions of images, \cite{rubinstein2013unsupervised} found saliency maps and the computed dense correspondence for all the similar images for obtaining matching scores for pixels across the images. Geometric Mean Saliency (GMS) \cite{jerripothula2014automatic} uses dense SIFT Correspondence \cite{liu2010sift} for each pair of images to align saliency maps and fuses them. \cite{jerripothula2015group} extended \cite{jerripothula2014automatic} by making it more efficient through introducing the key image concept. \cite{tao2017image} use saliency prior and superpixels to partition images into foreground and background. They calculate several feature level descriptors on superpixel level and organized in Bag of Words (BoW) to perform clustering of common foregrounds. All of these use single saliency source. Rather than going through the same trend, we propose the usage of multiple saliency sources to benefit from all of them. \\&#10;&#10;In the last decade, there has been a immensely growing interest towards deep learning in the computer vision coomunity, thanks to better availability of computational resources and platforms for large-scale annotations. A large variety of deep neural networks have been proposed for the task of object co-segmentation. \cite{li2018deep, chen2018semantic} use deep siamese networks to perform object co-segmentation. \cite{li2018deep} use a pre-trained VGG16 network for feature extraction and pass them to a correlation layer to find the similarities among a pair of images as inspired by Flownet \cite{dosovitskiy2015flownet}. The features from the encoder are combined with the correspondence obtained from the correlation layer and passed to the siamese decoder to obtain the output co-segmentation masks. \cite{chen2018semantic} developed a similar architecture to perform deep object co-segmentation by passing the global image features obtained from pre-trained VGG16 model to a channel-wise attention model which trains a fully connected network. The output features are then upsampled and sent to a convolution network which acts as a siamese decoder and generates the cosegmentation mask for the input images. Recently, \cite{zhang2020deep} developed a unique deep learning architecture which uses HRNet\cite{sun2019deep} pre-trained on ImageNet as a backbone to extract image features and then capture the correlation between features using a spatial modulator to generate a mask which can localize the common foreground object. They also developed a semantic modulator which acts as a supervised image classification model. The combined outputs of both modulators is able to co-segment common objects by using the multi-resolution image features. \\&#10;&#10;However, it is well-known that deep learning based co-segmentation methods require training which can make these approaches class specific. The performance of these models on unseen data will largely depend on the similarity between the seen and unseen classes. In such a scenario, saliency based co-segmentation methods might give superior performance because they are based on generic saliency methods. In the proposed approach, we exploit the inherent genericness of deep learning based saliency methods to identify common salient objects. An important aspect in a co-segmentation method is finding the correspondence between the images in a group. Usually, it's done through hand-crafted feature descriptors like SIFT \cite{liu2010sift}, GSS \cite{deselaers2010global}, etc. However these traditional methods do not work well when there are large viewpoint or illumination changes. In the proposed approach we use deep ResNet features and pre-trained DGC Net\cite{melekhov2019dgc} to establish global and local correspondences, respectively.\\&#10;&#10;We make two contributions: (1) using multiple saliency sources and (2) using learning based saliency extraction and correspondence processes for effective object co-segmentation. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Saliency Fusion, Object Co-segmentation" />
-        </attvalues>
-      </node>
-      <node id="1804.06423" label="1804.06423">
-        <attvalues>
-          <attvalue for="0" value="Deep Object Co-Segmentation" />
-          <attvalue for="1" value="  This work presents a deep object co-segmentation (DOCS) approach for&#10;segmenting common objects of the same class within a pair of images. This means&#10;that the method learns to ignore common, or uncommon, background stuff and&#10;focuses on objects. If multiple object classes are presented in the image pair,&#10;they are jointly extracted as foreground. To address this task, we propose a&#10;CNN-based Siamese encoder-decoder architecture. The encoder extracts high-level&#10;semantic features of the foreground objects, a mutual correlation layer detects&#10;the common objects, and finally, the decoder generates the output foreground&#10;masks for each image. To train our model, we compile a large object&#10;co-segmentation dataset consisting of image pairs from the PASCAL VOC dataset&#10;with common objects masks. We evaluate our approach on commonly used datasets&#10;for co-segmentation tasks and observe that our approach consistently&#10;outperforms competing methods, for both seen and unseen object classes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1504.06852" label="1504.06852">
-        <attvalues>
-          <attvalue for="0" value="FlowNet: Learning Optical Flow with Convolutional Networks" />
-          <attvalue for="1" value="  Convolutional neural networks (CNNs) have recently been very successful in a&#10;variety of computer vision tasks, especially on those linked to recognition.&#10;Optical flow estimation has not been among the tasks where CNNs were&#10;successful. In this paper we construct appropriate CNNs which are capable of&#10;solving the optical flow estimation problem as a supervised learning task. We&#10;propose and compare two architectures: a generic architecture and another one&#10;including a layer that correlates feature vectors at different image locations.&#10;  Since existing ground truth data sets are not sufficiently large to train a&#10;CNN, we generate a synthetic Flying Chairs dataset. We show that networks&#10;trained on this unrealistic data still generalize very well to existing&#10;datasets such as Sintel and KITTI, achieving competitive accuracy at frame&#10;rates of 5 to 10 fps.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.06859" label="1810.06859">
-        <attvalues>
-          <attvalue for="0" value="Semantic Aware Attention Based Deep Object Co-segmentation" />
-          <attvalue for="1" value="  Object co-segmentation is the task of segmenting the same objects from&#10;multiple images. In this paper, we propose the Attention Based Object&#10;Co-Segmentation for object co-segmentation that utilize a novel attention&#10;mechanism in the bottleneck layer of deep neural network for the selection of&#10;semantically related features. Furthermore, we take the benefit of attention&#10;learner and propose an algorithm to segment multi-input images in linear time&#10;complexity. Experiment results demonstrate that our model achieves state of the&#10;art performance on multiple datasets, with a significant reduction of&#10;computational time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.12950" label="1911.12950">
-        <attvalues>
-          <attvalue for="0" value="Deep Object Co-segmentation via Spatial-Semantic Network Modulation" />
-          <attvalue for="1" value="  Object co-segmentation is to segment the shared objects in multiple relevant&#10;images, which has numerous applications in computer vision. This paper presents&#10;a spatial and semantic modulated deep network framework for object&#10;co-segmentation. A backbone network is adopted to extract multi-resolution&#10;image features. With the multi-resolution features of the relevant images as&#10;input, we design a spatial modulator to learn a mask for each image. The&#10;spatial modulator captures the correlations of image feature descriptors via&#10;unsupervised learning. The learned mask can roughly localize the shared&#10;foreground object while suppressing the background. For the semantic modulator,&#10;we model it as a supervised image classification task. We propose a&#10;hierarchical second-order pooling module to transform the image features for&#10;classification use. The outputs of the two modulators manipulate the&#10;multi-resolution features by a shift-and-scale operation so that the features&#10;focus on segmenting co-object regions. The proposed model is trained end-to-end&#10;without any intricate post-processing. Extensive experiments on four image&#10;co-segmentation benchmark datasets demonstrate the superior accuracy of the&#10;proposed method compared to state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.08393" label="1810.08393">
-        <attvalues>
-          <attvalue for="0" value="DGC-Net: Dense Geometric Correspondence Network" />
-          <attvalue for="1" value="  This paper addresses the challenge of dense pixel correspondence estimation&#10;between two images. This problem is closely related to optical flow estimation&#10;task where ConvNets (CNNs) have recently achieved significant progress. While&#10;optical flow methods produce very accurate results for the small pixel&#10;translation and limited appearance variation scenarios, they hardly deal with&#10;the strong geometric transformations that we consider in this work. In this&#10;paper, we propose a coarse-to-fine CNN-based framework that can leverage the&#10;advantages of optical flow approaches and extend them to the case of large&#10;transformations providing dense and subpixel accurate estimates. It is trained&#10;on synthetic transformations and demonstrates very good performance to unseen,&#10;realistic, data. Further, we apply our method to the problem of relative camera&#10;pose estimation and demonstrate that the model outperforms existing dense&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.07816" label="2407.07816">
-        <attvalues>
-          <attvalue for="0" value="A Survey on Deep Stereo Matching in the Twenties" />
-          <attvalue for="1" value="  Stereo matching is close to hitting a half-century of history, yet witnessed&#10;a rapid evolution in the last decade thanks to deep learning. While previous&#10;surveys in the late 2010s covered the first stage of this revolution, the last&#10;five years of research brought further ground-breaking advancements to the&#10;field. This paper aims to fill this gap in a two-fold manner: first, we offer&#10;an in-depth examination of the latest developments in deep stereo matching,&#10;focusing on the pioneering architectural designs and groundbreaking paradigms&#10;that have redefined the field in the 2020s; second, we present a thorough&#10;analysis of the critical challenges that have emerged alongside these advances,&#10;providing a comprehensive taxonomy of these issues and exploring the&#10;state-of-the-art techniques proposed to address them. By reviewing both the&#10;architectural innovations and the key challenges, we offer a holistic view of&#10;deep stereo matching and highlight the specific areas that require further&#10;investigation. To accompany this survey, we maintain a regularly updated&#10;project page that catalogs papers on deep stereo matching in our&#10;Awesome-Deep-Stereo-Matching&#10;(https://github.com/fabiotosi92/Awesome-Deep-Stereo-Matching) repository.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Stereo matching -- the task of estimating dense disparity maps from a pair of rectified images -- has been a fundamental problem in computer vision for nearly half a century, playing a crucial role in a wide range of applications, such as autonomous driving, robotics, and augmented reality.&#10;After about twenty-five years of hand-designed stereo algorithms \cite{scharstein2002taxonomy}, the use of end-to-end deep neural networks has become the dominant paradigm in the late 2010s. &#10;&#10;Existing surveys \cite{poggi2021synergies,laga2020survey} have offered valuable insights on this rapid revolution, categorizing end-to-end architectures into 2D and 3D classes according to their cost-volume computation and optimization strategies, while also emphasizing the challenges remaining open.&#10;&#10;Since then, however, the domain has progressed rapidly, with the emergence of novel methods and paradigms inspired by breakthroughs in other areas of deep learning. Transformer-based \cite{Li_2021_ICCV_STTR} and iterative refinement \cite{lipson2021raft} architectures are prime examples of how the field has evolved, demonstrating the potential for further improvements in accuracy and efficiency.&#10;&#10;Despite the remarkable achievements, multiple challenges have emerged as deep stereo matching has advanced. One of the most critical issues, already highlighted in previous surveys, is the lack of generalization, particularly when facing domain shifts between synthetic and real data. Although synthetic datasets have been crucial for pre-training deep stereo networks, these models often perform poorly when applied to real scenes without fine-tuning. Recognizing the importance of this issue, several techniques have been proposed in recent years to improve zero-shot generalization and to develop domain adaptation methods for seamless adaptation to unknown target domains.&#10;&#10;Alongside the domain-shift problem, deep stereo matching has overcome several other limitations. The tendency of deep networks to over-smooth depth at object boundaries, leading to inaccurate 3D reconstructions, has been a persistent challenge. The increasing diversity of camera setups and the need to process images with different resolutions has underscored the importance of developing flexible algorithms. The demand for high-resolution and highly detailed disparity estimation, combined with the necessity for real-time performance on resource-constrained devices, has further complicated the task. These challenges have led to the development of more robust, versatile, and efficient deep stereo models that effectively address these limitations and improve their practical application.&#10;&#10;Furthermore, the integration of complementary imaging modalities, including depth sensors, non-visible spectrum cameras, event cameras, and gated cameras, has opened new avenues for enhancing the robustness and accuracy of stereo matching in challenging environments. &#10;By leveraging the strengths and complementary nature of these modalities, multimodal stereo matching techniques aim to extend the applicability and reliability of stereo vision to a broader range of real-world scenarios, overcoming the limitations of relying solely on visible-spectrum cameras.&#10;&#10;Driven by these significant developments, this survey provides a thorough and up-to-date review of the latest advancements in deep stereo matching that have emerged during the 2020s. We cover a wide range of topics explored since the last surveys \cite{poggi2021synergies,laga2020survey}, including novel architectures, multi-modal approaches, and cutting-edge techniques designed to address the aforementioned critical challenges. Our overview comprehensively analyzes over 100 distinct contributions to deep stereo matching, all of which have been presented at top conferences and published in prestigious journals. Our goal is to offer a thorough overview of the state-of-the-art, discussing the progress made since previous surveys and highlighting the current trends and future directions in the field.&#10;&#10;The remaining sections are organized as follows:&#10;&#10;\begin{itemize}&#10; \item Section \ref{sec:architectures} covers the latest developments in deep stereo matching architectures, highlighting novel paradigms, computational efficiency techniques, and specialized multi-modality designs.&#10; \item Section \ref{sec:challenges} explores key challenges in deep stereo matching and investigates recent approaches to address them.&#10; \item Section \ref{sec:experiments} reports quantitative results on the main online benchmarks, both established and recently introduced, demonstrating the progress made in the field over the last few years. &#10; \item Sections \ref{sec:discussion} and \ref{sec:conclusion} focus on potential future research directions and provide a summary of the survey. &#10;\end{itemize}&#10;&#10;\ifx\arxiv\undefined&#10;&#10;For a more detailed overview of the background information, we recommend referring to the supplementary material. This additional resource provides detailed descriptions of the datasets commonly used in the context of deep stereo matching and the primary evaluation metrics employed in this domain. &#10;&#10;\ifx\supplementary\undefined&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Stereo Matching, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2004.08566" label="2004.08566">
-        <attvalues>
-          <attvalue for="0" value="On the Synergies between Machine Learning and Binocular Stereo for Depth&#10;  Estimation from Images: a Survey" />
-          <attvalue for="1" value="  Stereo matching is one of the longest-standing problems in computer vision&#10;with close to 40 years of studies and research. Throughout the years the&#10;paradigm has shifted from local, pixel-level decision to various forms of&#10;discrete and continuous optimization to data-driven, learning-based methods.&#10;Recently, the rise of machine learning and the rapid proliferation of deep&#10;learning enhanced stereo matching with new exciting trends and applications&#10;unthinkable until a few years ago. Interestingly, the relationship between&#10;these two worlds is two-way. While machine, and especially deep, learning&#10;advanced the state-of-the-art in stereo matching, stereo itself enabled new&#10;ground-breaking methodologies such as self-supervised monocular depth&#10;estimation based on deep networks. In this paper, we review recent research in&#10;the field of learning-based depth estimation from single and binocular images&#10;highlighting the synergies, the successes achieved so far and the open&#10;challenges the community is going to face in the immediate future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.02535" label="2006.02535">
-        <attvalues>
-          <attvalue for="0" value="A Survey on Deep Learning Techniques for Stereo-based Depth Estimation" />
-          <attvalue for="1" value="  Estimating depth from RGB images is a long-standing ill-posed problem, which&#10;has been explored for decades by the computer vision, graphics, and machine&#10;learning communities. Among the existing techniques, stereo matching remains&#10;one of the most widely used in the literature due to its strong connection to&#10;the human binocular system. Traditionally, stereo-based depth estimation has&#10;been addressed through matching hand-crafted features across multiple images.&#10;Despite the extensive amount of research, these traditional techniques still&#10;suffer in the presence of highly textured areas, large uniform regions, and&#10;occlusions. Motivated by their growing success in solving various 2D and 3D&#10;vision problems, deep learning for stereo-based depth estimation has attracted&#10;growing interest from the community, with more than 150 papers published in&#10;this area between 2014 and 2019. This new generation of methods has&#10;demonstrated a significant leap in performance, enabling applications such as&#10;autonomous driving and augmented reality. In this article, we provide a&#10;comprehensive survey of this new and continuously growing field of research,&#10;summarize the most commonly used pipelines, and discuss their benefits and&#10;limitations. In retrospect of what has been achieved so far, we also conjecture&#10;what the future may hold for deep learning-based stereo for depth estimation&#10;research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.02910" label="2011.02910">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Stereo Depth Estimation From a Sequence-to-Sequence&#10;  Perspective with Transformers" />
-          <attvalue for="1" value="  Stereo depth estimation relies on optimal correspondence matching between&#10;pixels on epipolar lines in the left and right images to infer depth. In this&#10;work, we revisit the problem from a sequence-to-sequence correspondence&#10;perspective to replace cost volume construction with dense pixel matching using&#10;position information and attention. This approach, named STereo TRansformer&#10;(STTR), has several advantages: It 1) relaxes the limitation of a fixed&#10;disparity range, 2) identifies occluded regions and provides confidence&#10;estimates, and 3) imposes uniqueness constraints during the matching process.&#10;We report promising results on both synthetic and real-world datasets and&#10;demonstrate that STTR generalizes across different domains, even without&#10;fine-tuning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.07202" label="2110.07202">
-        <attvalues>
-          <attvalue for="0" value="Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution" />
-          <attvalue for="1" value="  In this paper, we introduce a variational Bayesian algorithm (VBA) for image&#10;blind deconvolution. Our generic framework incorporates smoothness priors on&#10;the unknown blur/image and possible affine constraints (e.g., sum to one) on&#10;the blur kernel. One of our main contributions is the integration of VBA within&#10;a neural network paradigm, following an unrolling methodology. The proposed&#10;architecture is trained in a supervised fashion, which allows us to optimally&#10;set two key hyperparameters of the VBA model and lead to further improvements&#10;in terms of resulting visual quality. Various experiments involving&#10;grayscale/color images and diverse kernel shapes, are performed. The numerical&#10;examples illustrate the high performance of our approach when compared to&#10;state-of-the-art techniques based on optimization, Bayesian estimation, or deep&#10;learning.&#10;" />
-          <attvalue for="2" value="&#10;Image blind deconvolution problem appears in many fields of image processing such as astronomy \cite{Murtagh2007}, biology \cite{Holmes2006} and medical imaging \cite{Michailovich2007}.&#10;Given a degraded, blurred and noisy image, the aim is to restore a clean image along with an estimate of the blur kernel. Blind deconvolution is a severely ill-posed problem as there exists an infinite number of pairs (image/blur) that lead to the same observed image. Blind deconvolution methods adopt either a sequential identification process \cite{Carasso2001}, or a joint estimation approach \cite{Levin2009}. In the former, the blur kernel is identified first, possibly through a calibration step \cite{Bell16,Yunshi2021,Lefort2019}. Then the unknown image is inferred using a non-blind image restoration method. In the latter, the blur kernel and unknown image are simultaneously estimated. Since the problem is highly ill-posed, it is mandatory to incorporate prior knowledge on the sought unknowns. The retained prior strongly influences the choice for the solver. Let us distinguish three main classes of joint blind deconvolution approaches. A first option consists of formulating the problem as the minimization of a cost function gathering a data fidelity term (e.g., least-squares discrepancy) and penalties/constraints acting on the image and kernel variables. In such a way, it is quite standard to impose normalization and sparsity enhancing constraints on the kernel coefficients to avoid scale ambiguity inherent to the blind deconvolution model \cite{Komodakis2012,Kotera2013,Krishnan2011}. One can also easily impose the smoothness of the image, by adopting total-variation based regularization \cite{Chan1998}. Several other efficient choices have been proposed in the literature, along with suitable iterative optimization methods to solve the resulting problems \cite{Levin2006,Shan2008,Joshi2008,Abboud2019,Bolte2010}. The main advantage of this family of methods is probably its flexibility. But this comes at the price of heavy parameter tuning. The second option is to resort to a Bayesian formulation to express the model and a priori knowledge on the variables. The estimates are then defined from the estimation of the moments (typically, the mean) of a posterior distribution given the observed data and prior. As this typically involves the evaluation of intractable integrals, sampling \cite{Robert04,luengo2020survey} or approximation \cite{Evans95} strategies are used. Markov chain Monte Carlo (MCMC) methods have been widely used for blind deconvolution involving 1D sparse signals \cite{Rosec2003,Ge2011,Kail}, but it is up to our knowledge scarcely employed in large-scale problems \cite{Bishop2008}, probably for computational time reasons. Another family of approach consists in adopting the so-called variational Bayesian approximation paradigm \cite{Foxtutorial,David2016}. Then, a simpler (usually separable) approximation to the posterior is built through the minimization of a suitable divergence. This approach leads to fast Bayesian-based algorithms, whose great performance has been assessed in the context of non-blind \cite{Yorsa2017,Zheng6990542} and blind \cite{Babacan2009} image restoration. Bayesian-based techniques usually require less parameters than optimization-based ones. Moreover, they can provide higher-order moments estimates, such as covariance matrices, which are of high interest for assessing probabilistically the uncertainty of the results. However, dealing with complex noise models and priors in such methods may be tricky, and the algorithms may be quite computationally heavy. A recent trend is to insert optimization-based steps in Bayesian sampling/approximation methods for a more versatile modality and faster computations. See, for example, \cite{Yorsa2017,Pereyra2016, Marnissi2020} for applications of such ideas in the context of large-scale image processing. The third category of methods is more recent, as the references only go back to the last decade~\cite{Kupyn2018,Dong2016,Albluwi2018}. These methods rely on the deep learning methodology. More precisely, a supervised learning strategy is adopted to learn (implicitly) some prior information on the image/kernel from a so-called training set. A highly non-linear and multi-layers architecture is built, and its parameters (i.e., neuron weights) are estimated by back-propagation to minimize a given loss function associated with the task at hand (e.g., image visual quality). Several recent works propose neural network architectures dedicated to the problem of image blind deconvolution. Let us mention DeblurGAN \cite{Kupyn2018}, based on conditional generative adversarial networks and a multi-component loss function, SRCNN \cite{Dong2016} and its extended version, DBSRCNN \cite{Albluwi2018}, relying on a CNN architecture, SelfDeblur \cite{Ren2020} combining an optimization-based method with two GAN networks. These methods can reach very good performance, as long as the training set is large and representative enough. Moreover, they are well suited to GPU-based implementation. However, they have traditionally suffered from lack of interpretability and robustness \cite{Bietti2018}. An emerging set of methods, in the field of inverse problems in signal/image processing, performs algorithm unrolling \cite{Vishal2021}, where an iterative method (e.g., an optimization algorithm) is unrolled as layers of a neural network. The reduced set of parameters of this network are learnt by supervised training. Promising results have been obtained in the context of image deconvolution in \cite{Li2020, Bertocchi2019,Pesquet2020}. Theoretical results assessing the stability and robustness of unrolling techniques can be found in \cite{Pesquet2020,Patrick2020,Emilie2021}. These methods are also closely related to plug-and-play techniques where a trained neural network is employed as the denoiser \cite{Pesquet2020, Kamilov2021}.&#10;&#10;In this paper, we propose a novel approach for blind image deconvolution, that aims at gathering the best of the three aforementioned methods. We first introduce a variational Bayesian algorithm (VBA) enhanced by optimization-based ideas from \cite{Yorsa2017}, with the advantages to cope with a large set of priors on the kernel and the image, and to present a reduced computational cost. Then, we apply the unrolling paradigm to create a deep neural network architecture, where VBA iterations are integrated as layers. This allows us to (i) learn the hyperparameters (in particular, the noise level) of VBA in an automatic supervised fashion, (ii) improve further the quality of the results by choosing a dedicated loss in the training phase, (iii) implement the method by taking full advantage of possible GPU resources, thus considerably reducing the processing time during the test phase. In contrast to standard deep learning methods for blind deconvolution, all these benefits come along with a preservation of the interpretability of the method, thanks to the unrolling technique. Let us emphasize that variational Bayesian methods often appear in deep learning context. Indeed, they are backbones of variational autoencoders \cite{Kingma2014} and also constitute methods of choice for training Bayesian neural networks \cite{Jospin2020}. However, up to our knowledge, our work is the first to investigate the unrolling of a variational Bayesian technique. &#10;&#10;The rest of our paper is organized as follows. In Section \ref{sec:VBA}, we introduce the image degradation model and introduce our Bayesian modeling, and provide the background for deriving our algorithm. Section \ref{sec:proposed} explicitly describes the iterative updates of the proposed VBA. The unrolling of VBA is presented in Section \ref{sec:unfoldedVBA}. Numerical results, including comparisons with various methods, are presented in Section \ref{sec:experiments}. Section \ref{sec:conclusion} concludes this paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Deconvolution, Machine Learning, Engineering, Computer Vision, Mathematics, Bayesian Algorithms, Computer Science, Neural Networks, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2107.11820" label="2107.11820">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Monte Carlo Methods for Parameter Estimation" />
-          <attvalue for="1" value="  Statistical signal processing applications usually require the estimation of&#10;some parameters of interest given a set of observed data. These estimates are&#10;typically obtained either by solving a multi-variate optimization problem, as&#10;in the maximum likelihood (ML) or maximum a posteriori (MAP) estimators, or by&#10;performing a multi-dimensional integration, as in the minimum mean squared&#10;error (MMSE) estimators. Unfortunately, analytical expressions for these&#10;estimators cannot be found in most real-world applications, and the Monte Carlo&#10;(MC) methodology is one feasible approach. MC methods proceed by drawing random&#10;samples, either from the desired distribution or from a simpler one, and using&#10;them to compute consistent estimators. The most important families of MC&#10;algorithms are Markov chain MC (MCMC) and importance sampling (IS). On the one&#10;hand, MCMC methods draw samples from a proposal density, building then an&#10;ergodic Markov chain whose stationary distribution is the desired distribution&#10;by accepting or rejecting those candidate samples as the new state of the&#10;chain. On the other hand, IS techniques draw samples from a simple proposal&#10;density, and then assign them suitable weights that measure their quality in&#10;some appropriate way. In this paper, we perform a thorough review of MC methods&#10;for the estimation of static parameters in signal processing applications. A&#10;historical note on the development of MC schemes is also provided, followed by&#10;the basic MC method and a brief description of the rejection sampling (RS)&#10;algorithm, as well as three sections describing many of the most relevant MCMC&#10;and IS algorithms, and their combined use.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1601.00670" label="1601.00670">
-        <attvalues>
-          <attvalue for="0" value="Variational Inference: A Review for Statisticians" />
-          <attvalue for="1" value="  One of the core problems of modern statistics is to approximate&#10;difficult-to-compute probability densities. This problem is especially&#10;important in Bayesian statistics, which frames all inference about unknown&#10;quantities as a calculation involving the posterior density. In this paper, we&#10;review variational inference (VI), a method from machine learning that&#10;approximates probability densities through optimization. VI has been used in&#10;many applications and tends to be faster than classical methods, such as Markov&#10;chain Monte Carlo sampling. The idea behind VI is to first posit a family of&#10;densities and then to find the member of that family which is close to the&#10;target. Closeness is measured by Kullback-Leibler divergence. We review the&#10;ideas behind mean-field variational inference, discuss the special case of VI&#10;applied to exponential family models, present a full example with a Bayesian&#10;mixture of Gaussians, and derive a variant that uses stochastic optimization to&#10;scale up to massive data. We discuss modern research in VI and highlight&#10;important open problems. VI is powerful, but it is not yet well understood. Our&#10;hope in writing this paper is to catalyze statistical research on this class of&#10;algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.00273" label="1505.00273">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Stochastic Simulation and Optimization Methods in Signal&#10;  Processing" />
-          <attvalue for="1" value="  Modern signal processing (SP) methods rely very heavily on probability and&#10;statistics to solve challenging SP problems. SP methods are now expected to&#10;deal with ever more complex models, requiring ever more sophisticated&#10;computational inference techniques. This has driven the development of&#10;statistical SP methods based on stochastic simulation and optimization.&#10;Stochastic simulation and optimization algorithms are computationally intensive&#10;tools for performing statistical inference in models that are analytically&#10;intractable and beyond the scope of deterministic inference methods. They have&#10;been recently successfully applied to many difficult problems involving complex&#10;statistical models and sophisticated (often Bayesian) statistical inference&#10;techniques. This survey paper offers an introduction to stochastic simulation&#10;and optimization methods in signal and image processing. The paper addresses a&#10;variety of high-dimensional Markov chain Monte Carlo (MCMC) methods as well as&#10;deterministic surrogate methods, such as variational Bayes, the Bethe approach,&#10;belief and expectation propagation and approximate message passing algorithms.&#10;It also discusses a range of optimization methods that have been adopted to&#10;solve stochastic problems, as well as stochastic methods for deterministic&#10;optimization. Subsequently, areas of overlap between simulation and&#10;optimization, in particular optimization-within-MCMC and MCMC-driven&#10;optimization are discussed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.02197" label="1908.02197">
-        <attvalues>
-          <attvalue for="0" value="Neural Blind Deconvolution Using Deep Priors" />
-          <attvalue for="1" value="  Blind deconvolution is a classical yet challenging low-level vision problem&#10;with many real-world applications. Traditional maximum a posterior (MAP) based&#10;methods rely heavily on fixed and handcrafted priors that certainly are&#10;insufficient in characterizing clean images and blur kernels, and usually adopt&#10;specially designed alternating minimization to avoid trivial solution. In&#10;contrast, existing deep motion deblurring networks learn from massive training&#10;images the mapping to clean image or blur kernel, but are limited in handling&#10;various complex and large size blur kernels. To connect MAP and deep models, we&#10;in this paper present two generative networks for respectively modeling the&#10;deep priors of clean image and blur kernel, and propose an unconstrained neural&#10;optimization solution to blind deconvolution. In particular, we adopt an&#10;asymmetric Autoencoder with skip connections for generating latent clean image,&#10;and a fully-connected network (FCN) for generating blur kernel. Moreover, the&#10;SoftMax nonlinearity is applied to the output layer of FCN to meet the&#10;non-negative and equality constraints. The process of neural optimization can&#10;be explained as a kind of &quot;zero-shot&quot; self-supervised learning of the&#10;generative networks, and thus our proposed method is dubbed SelfDeblur.&#10;Experimental results show that our SelfDeblur can achieve notable quantitative&#10;gains as well as more visually plausible deblurring results in comparison to&#10;state-of-the-art blind deconvolution methods on benchmark datasets and&#10;real-world blurry images. The source code is available at&#10;https://github.com/csdwren/SelfDeblur&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.10557" label="1912.10557">
-        <attvalues>
-          <attvalue for="0" value="Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal&#10;  and Image Processing" />
-          <attvalue for="1" value="  Deep neural networks provide unprecedented performance gains in many real&#10;world problems in signal and image processing. Despite these gains, future&#10;development and practical deployment of deep networks is hindered by their&#10;blackbox nature, i.e., lack of interpretability, and by the need for very large&#10;training sets. An emerging technique called algorithm unrolling or unfolding&#10;offers promise in eliminating these issues by providing a concrete and&#10;systematic connection between iterative algorithms that are used widely in&#10;signal processing and deep neural networks. Unrolling methods were first&#10;proposed to develop fast neural network approximations for sparse coding. More&#10;recently, this direction has attracted enormous attention and is rapidly&#10;growing both in theoretic investigations and practical applications. The&#10;growing popularity of unrolled deep networks is due in part to their potential&#10;in developing efficient, high-performance and yet interpretable network&#10;architectures from reasonable size training sets. In this article, we review&#10;algorithm unrolling for signal and image processing. We extensively cover&#10;popular techniques for algorithm unrolling in various domains of signal and&#10;image processing including imaging, vision and recognition, and speech&#10;processing. By reviewing previous works, we reveal the connections between&#10;iterative algorithms and neural networks and present recent theoretical&#10;results. Finally, we provide a discussion on current limitations of unrolling&#10;and suggest possible future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.13247" label="2012.13247">
-        <attvalues>
-          <attvalue for="0" value="Learning Maximally Monotone Operators for Image Recovery" />
-          <attvalue for="1" value="  We introduce a new paradigm for solving regularized variational problems.&#10;These are typically formulated to address ill-posed inverse problems&#10;encountered in signal and image processing. The objective function is&#10;traditionally defined by adding a regularization function to a data fit term,&#10;which is subsequently minimized by using iterative optimization algorithms.&#10;Recently, several works have proposed to replace the operator related to the&#10;regularization by a more sophisticated denoiser. These approaches, known as&#10;plug-and-play (PnP) methods, have shown excellent performance. Although it has&#10;been noticed that, under some Lipschitz properties on the denoisers, the&#10;convergence of the resulting algorithm is guaranteed, little is known about&#10;characterizing the asymptotically delivered solution. In the current article,&#10;we propose to address this limitation. More specifically, instead of employing&#10;a functional regularization, we perform an operator regularization, where a&#10;maximally monotone operator (MMO) is learned in a supervised manner. This&#10;formulation is flexible as it allows the solution to be characterized through a&#10;broad range of variational inequalities, and it includes convex regularizations&#10;as special cases. From an algorithmic standpoint, the proposed approach&#10;consists in replacing the resolvent of the MMO by a neural network (NN). We&#10;present a universal approximation theorem proving that nonexpansive NNs are&#10;suitable models for the resolvent of a wide class of MMOs. The proposed&#10;approach thus provides a sound theoretical framework for analyzing the&#10;asymptotic behavior of first-order PnP algorithms. In addition, we propose a&#10;numerical strategy to train NNs corresponding to resolvents of MMOs. We apply&#10;our approach to image restoration problems and demonstrate its validity in&#10;terms of both convergence and quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.07526" label="1808.07526">
-        <attvalues>
-          <attvalue for="0" value="Deep Neural Network Structures Solving Variational Inequalities" />
-          <attvalue for="1" value="  Motivated by structures that appear in deep neural networks, we investigate&#10;nonlinear composite models alternating proximity and affine operators defined&#10;on different spaces. We first show that a wide range of activation operators&#10;used in neural networks are actually proximity operators. We then establish&#10;conditions for the averagedness of the proposed composite constructs and&#10;investigate their asymptotic properties. It is shown that the limit of the&#10;resulting process solves a variational inequality which, in general, does not&#10;derive from a minimization problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.15044" label="2105.15044">
-        <attvalues>
-          <attvalue for="0" value="Inversion of Integral Models: a Neural Network Approach" />
-          <attvalue for="1" value="  We introduce a neural network architecture to solve inverse problems linked&#10;to a one-dimensional integral operator. This architecture is built by unfolding&#10;a forward-backward algorithm derived from the minimization of an objective&#10;function which consists of the sum of a data-fidelity function and a&#10;Tikhonov-type regularization function. The robustness of this inversion method&#10;with respect to a perturbation of the input is theoretically analyzed. Ensuring&#10;robustness is consistent with inverse problem theory since it guarantees both&#10;the continuity of the inversion method and its insensitivity to small noise.&#10;The latter is a critical property as deep neural networks have been shown to be&#10;vulnerable to adversarial perturbations. One of the main novelties of our work&#10;is to show that the proposed network is also robust to perturbations of its&#10;bias. In our architecture, the bias accounts for the observed data in the&#10;inverse problem. We apply our method to the inversion of Abel integral&#10;operators, which define a fractional integration involved in wide range of&#10;physical processes. The neural network is numerically implemented and tested to&#10;illustrate the efficiency of the method. Lipschitz constants after training are&#10;computed to measure the robustness of the neural networks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.06519" label="2205.06519">
-        <attvalues>
-          <attvalue for="0" value="Evaluation of Video Coding for Machines without Ground Truth" />
-          <attvalue for="1" value="  In the emerging field of video coding for machines, video datasets with&#10;pristine video quality and high-quality annotations are required for a&#10;comprehensive evaluation. However, existing video datasets with detailed&#10;annotations are severely limited in size and video quality. Thus, current&#10;methods have to either evaluate their codecs on still images or on already&#10;compressed data. To mitigate this problem, we propose an evaluation method&#10;based on pseudo ground-truth data from the field of semantic segmentation to&#10;the evaluation of video coding for machines. Through extensive evaluation, this&#10;paper shows that the proposed ground-truth-agnostic evaluation method results&#10;in an acceptable absolute measurement error below 0.7 percentage points on the&#10;Bjontegaard Delta Rate compared to using the true ground truth for mid-range&#10;bitrates. We evaluate on the three tasks of semantic segmentation, instance&#10;segmentation, and object detection. Lastly, we utilize the&#10;ground-truth-agnostic method to measure the coding performances of the VVC&#10;compared against HEVC on the Cityscapes sequences. This reveals that the coding&#10;position has a significant influence on the task performance.&#10;" />
-          <attvalue for="2" value="&#10;&#09;Thanks to a drastic increase in accuracy and applicability, a significant amount of computer vision tasks is solved by neural networks today, such as in autonomous driving or surveillance of public spaces. &#10;&#09;In such applications, machines are directly interacting with each other without a human being involved.&#10;&#09;Hence, this type of communication is typically referred to as machine-to-machine~(M2M) communication.&#10;&#09;Cisco underlines the relevance of this emerging topic by stating that half of all global devices and connections will be accounted to M2M communication in 2023~\cite{cisco2020}.&#10;&#10;In practical applications, the input multimedia data are usually compressed and transmitted before being analyzed by the machine.&#10;&#09;This saves eventual transmission rate or storage space on hard disks.&#10;&#09;Together with the growing importance of M2M communication, this requires specialized compression schemes in M2M applications.&#10;&#09;Early work in this area optimized the compression to preserve extracted features~\cite{chao2011preserving,chao2012sift,chao2013design}.&#10;&#09;More recent work focuses on coding for M2M communication, where a neural network analyzes the decoded image instead of a human observer~\cite{choi2018,galteri2018,fischer2020_ICIP,fischer2020_FRDO, le2021_ICASSP, fischer2021_ICASSP}.&#10;&#09;This topic is commonly referred to as video coding for machines~(VCM).&#10;&#09;Additionally, the Moving Picture Experts Group (MPEG) founded an ad hoc group on VCM in 2019, with the objective of standardizing an optimal bitstream for M2M communication~\cite{zhang2019}.&#10;&#10;To evaluate codecs for a VCM scenario, an arbitrary algorithm or neural network is applied to the compressed input image and its performance is measured depending on the deteriorations present.&#10;&#09;Thus, there are three main conditions for a proper dataset:&#10;&#09;1.)~The dataset needs to contain uncompressed, pristine input data, such that the coding efficiency measurement is not falsified by already existing artifacts.&#10;&#09;2.)~The dataset requires properly labeled ground-truth~(GT) data to measure the performance of the applied algorithm.&#10;&#09;3.)~The dataset needs to contain video data on top/instead of images, since practical scenarios are usually based on video streams.&#10;&#09;To the best of our knowledge, there is currently no dataset available that fulfills all three conditions.&#10;&#09;Thus, previous VCM research either sacrifices the condition of uncompressed input data~\cite{choi2018, galteri2018} or evaluates the coding frameworks on single images~\cite{fischer2020_ICIP, fischer2020_FRDO, le2021_ICASSP, fischer2021_ICASSP}.&#10;&#10;To alleviate the problem of requiring hand-labeled annotations, which most datasets do not have, several papers from the research field of future semantic segmentation either turn to simulated data~\cite{kuhn2021pixel} or generate pseudo GT data. Since we focus on real world applications, we do not consider simulation-based approaches. Luc~et~al.~\cite{luc2017} and Nabavi~et~al.~\cite{nabavi2018FutureSS} take the predictions of a state-of-the-art semantic segmentation network as pseudo GT to evaluate their future semantic segmentation models.&#10;&#09;Couprie~et~al.~\cite{couprie2018} applied pseudo GT annotations for the task of future instance segmentation for the Cityscapes~\cite{cordts2016} dataset.&#10;&#09;Aqqa~et~al.~\cite{aqqa2019} already measured the VCM performance of four different state-of-the-art object detection networks on different H.264/AVC~\cite{wiegand_h264} compression levels taking the predictions on uncompressed data as GT.&#10;&#09;However, they do not provide any statement on how accurate their measurement is compared to GT-based evaluations.&#10;&#10;Inspired by those approaches, this paper proposes to apply a GT-agnostic evaluation framework to evaluate VCM scenarios.&#10;&#09;We use the network predictions on uncompressed, pristine input data as pseudo GT.&#10;&#09;Thereby, this paper contributes an extensive analysis of the GT-agnostic evaluation framework and quantifies the measurement error compared with the traditional GT-based method.&#10;&#09;Second, with the help of the GT-agnostic method, we compare the two latest video codecs High Efficiency Video Coding~(H.265/HEVC)~\cite{sullivan2012_HEVC} and its successor Versatile Video Coding~(H.266/VVC)~\cite{chen2020vtm10} for inter coding in randomaccess configuration on the non-labeled Cityscapes sequences.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Machine Vision Tasks, Image Compression Methods, Mathematics, Computer Science, Video Coding Evaluation, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1710.11151" label="1710.11151">
-        <attvalues>
-          <attvalue for="0" value="High efficiency compression for object detection" />
-          <attvalue for="1" value="  Image and video compression has traditionally been tailored to human vision.&#10;However, modern applications such as visual analytics and surveillance rely on&#10;computers seeing and analyzing the images before (or instead of) humans. For&#10;these applications, it is important to adjust compression to computer vision.&#10;In this paper we present a bit allocation and rate control strategy that is&#10;tailored to object detection. Using the initial convolutional layers of a&#10;state-of-the-art object detector, we create an importance map that can guide&#10;bit allocation to areas that are important for object detection. The proposed&#10;method enables bit rate savings of 7% or more compared to default HEVC, at the&#10;equivalent object detection rate.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.05927" label="2203.05927">
-        <attvalues>
-          <attvalue for="0" value="On Intra Video Coding and In-loop Filtering for Neural Object Detection&#10;  Networks" />
-          <attvalue for="1" value="  Classical video coding for satisfying humans as the final user is a widely&#10;investigated field of studies for visual content, and common video codecs are&#10;all optimized for the human visual system (HVS). But are the assumptions and&#10;optimizations also valid when the compressed video stream is analyzed by a&#10;machine? To answer this question, we compared the performance of two&#10;state-of-the-art neural detection networks when being fed with deteriorated&#10;input images coded with HEVC and VVC in an autonomous driving scenario using&#10;intra coding. Additionally, the impact of the three VVC in-loop filters when&#10;coding images for a neural network is examined. The results are compared using&#10;the mean average precision metric to evaluate the object detection performance&#10;for the compressed inputs. Throughout these tests, we found that the&#10;Bj{\o}ntegaard Delta Rate savings with respect to PSNR of 22.2 % using VVC&#10;instead of HEVC cannot be reached when coding for object detection networks&#10;with only 13.6% in the best case. Besides, it is shown that disabling the VVC&#10;in-loop filters SAO and ALF results in bitrate savings of 6.4 % compared to the&#10;standard VTM at the same mean average precision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.05890" label="2203.05890">
-        <attvalues>
-          <attvalue for="0" value="Video Coding for Machines with Feature-Based Rate-Distortion&#10;  Optimization" />
-          <attvalue for="1" value="  Common state-of-the-art video codecs are optimized to deliver a low bitrate&#10;by providing a certain quality for the final human observer, which is achieved&#10;by rate-distortion optimization (RDO). But, with the steady improvement of&#10;neural networks solving computer vision tasks, more and more multimedia data is&#10;not observed by humans anymore, but directly analyzed by neural networks. In&#10;this paper, we propose a standard-compliant feature-based RDO (FRDO) that is&#10;designed to increase the coding performance, when the decoded frame is analyzed&#10;by a neural network in a video coding for machine scenario. To that extent, we&#10;replace the pixel-based distortion metrics in conventional RDO of VTM-8.0 with&#10;distortion metrics calculated in the feature space created by the first layers&#10;of a neural network. Throughout several tests with the segmentation network&#10;Mask R-CNN and single images from the Cityscapes dataset, we compare the&#10;proposed FRDO and its hybrid version HFRDO with different distortion measures&#10;in the feature space against the conventional RDO. With HFRDO, up to 5.49 %&#10;bitrate can be saved compared to the VTM-8.0 implementation in terms of&#10;Bj{\o}ntegaard Delta Rate and using the weighted average precision as quality&#10;metric. Additionally, allowing the encoder to vary the quantization parameter&#10;results in coding gains for the proposed HFRDO of up 9.95 % compared to&#10;conventional VTM.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.07684" label="1703.07684">
-        <attvalues>
-          <attvalue for="0" value="Predicting Deeper into the Future of Semantic Segmentation" />
-          <attvalue for="1" value="  The ability to predict and therefore to anticipate the future is an important&#10;attribute of intelligence. It is also of utmost importance in real-time&#10;systems, e.g. in robotics or autonomous driving, which depend on visual scene&#10;understanding for decision making. While prediction of the raw RGB pixel values&#10;in future video frames has been studied in previous work, here we introduce the&#10;novel task of predicting semantic segmentations of future frames. Given a&#10;sequence of video frames, our goal is to predict segmentation maps of not yet&#10;observed video frames that lie up to a second or further in the future. We&#10;develop an autoregressive convolutional neural network that learns to&#10;iteratively generate multiple frames. Our results on the Cityscapes dataset&#10;show that directly predicting future segmentations is substantially better than&#10;predicting and then segmenting future RGB frames. Prediction results up to half&#10;a second in the future are visually convincing and are much more accurate than&#10;those of a baseline based on warping semantic segmentations using optical flow.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.07946" label="1807.07946">
-        <attvalues>
-          <attvalue for="0" value="Future Semantic Segmentation with Convolutional LSTM" />
-          <attvalue for="1" value="  We consider the problem of predicting semantic segmentation of future frames&#10;in a video. Given several observed frames in a video, our goal is to predict&#10;the semantic segmentation map of future frames that are not yet observed. A&#10;reliable solution to this problem is useful in many applications that require&#10;real-time decision making, such as autonomous driving. We propose a novel model&#10;that uses convolutional LSTM (ConvLSTM) to encode the spatiotemporal&#10;information of observed frames for future prediction. We also extend our model&#10;to use bidirectional ConvLSTM to capture temporal information in both&#10;directions. Our proposed approach outperforms other state-of-the-art methods on&#10;the benchmark dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.06134" label="2006.06134">
-        <attvalues>
-          <attvalue for="0" value="Kalman Filter Based Multiple Person Head Tracking" />
-          <attvalue for="1" value="  For multi-target tracking, target representation plays a crucial rule in&#10;performance. State-of-the-art approaches rely on the deep learning-based visual&#10;representation that gives an optimal performance at the cost of high&#10;computational complexity. In this paper, we come up with a simple yet effective&#10;target representation for human tracking. Our inspiration comes from the fact&#10;that the human body goes through severe deformation and inter/intra occlusion&#10;over the passage of time. So, instead of tracking the whole body part, a&#10;relative rigid organ tracking is selected for tracking the human over an&#10;extended period of time. Hence, we followed the tracking-by-detection paradigm&#10;and generated the target hypothesis of only the spatial locations of heads in&#10;every frame. After the localization of head location, a Kalman filter with a&#10;constant velocity motion model is instantiated for each target that follows the&#10;temporal evolution of the targets in the scene. For associating the targets in&#10;the consecutive frames, combinatorial optimization is used that associates the&#10;corresponding targets in a greedy fashion. Qualitative results are evaluated on&#10;four challenging video surveillance dataset and promising results has been&#10;achieved.&#10;" />
-          <attvalue for="2" value="&#10;&#10;One of the primary tasks of machine learning is to enable computers to learn from the data and automatically do thoughtful predictions. Such capabilities has applications in airline scheduling \cite{bazargan2016airline}, crowd modeling \cite{ullah2020multi}, and face recognition based fraud detection \cite{alreshidi2020facial}. For the visual data, it helps to analyze and classify a visual scene \cite{ullah2019two, owens2018audio}. In the realm of visual scene analysis, multi-target tracking is one of the most important low-level computer vision problems that provides a backbone to many high level tasks like autonomous driving \cite{khan2019survey, li2019generic}, action recognition \cite{ullah2017human, wang2018generative, ullah2019stacked}, behavior analysis \cite{marsden2017resnetcrowd, ullah2016crowd}, anomaly detection\cite{wang2018generative, ullah2019hybrid, ullah2018anomalous}, crowd management \cite{ullah2017density, ullah2017density}, and sports players analysis \cite{thomas2017computer, khan2019person}, to name a few. Even though tracking on its own is a low-level computer vision problem, intrinsically, it consists of other low-level tasks like object segmentation\cite{ullah2018pednet}, object detection\cite{khan2019disam}, and motion modeling\cite{basalamah2019scale}. With the advancement in object detection algorithms \cite{redmon2016you, khan2019dimension}, the tracking-by-detection paradigm becomes the most suitable for tracking multiple objects in a visual scene. However, the biggest question that arises is which object part to track or track the whole object mass. Until now, almost all the tracking algorithms use the whole body detection \cite{voigtlaender2019mots, ullah2019single, schulter2017deep, ullah2017hierarchical, milan2014continuous, sun2019deep, ullah2018directed, bae2017confidence, ullah2016hog, son2017multi, chu2017online, ullah2018deep}. For example, Milan et al. \cite{milan2014continuous} proposed a highly non-convex cost function for multi-target tracking where different components like appearance, detection, motion, target mutual osculation, etc. are combined in a weighted average function. A gradient descent based optimization is used to optimize the cost and transdimentional jumps are used to avoid the local minima. Ullah et al. \cite{ullah2017hierarchical} proposed a bag of Bayesian filters to track multiple targets in the scene. Additionally, sparse coded deep features are incorporated to model the appearance of the targets. Schulter et al. \cite{schulter2017deep} modeled multi-target tracking as a network flow graph. Instead of calculating the edge cost of the graph manually through hand-crafted features, they learned the edge of the graph through back propagation. Similarly, Ullah et al. \cite{ullah2018deep} also generated a directed acyclic graph for the multi-target tracking but used deep features for calculating the edge cost of the graph. Dehghan et al. \cite{dehghan2015gmmcp} formulated the tracking problem as a multi clique problem. Initially, a graph is generated from a batch of frames and later each target trajectory is found as the maximum clique of the graph. Chu et al. \cite{chu2017online} proposed a spatial-temporal attention mechanism for occulation handling and the interaction among different targets. They extract features from the different layers of CNN for modeling the appearance of the target. Compare to that, \cite{ullah2019single} proposed a Siamese neural network for modeling the appearance of targets and establishing the association between two targets. &#10;&#10;One of the common attributes among all the tracking techniques is that they represent the whole human body as a rectangular rigid object, even though the human body goes through severe articulations. Compared to the standard approaches, in this paper, we focused on the relatively rigid organ of the human body i.e. head of a person and first generated the head hypothesis in all the frames and then used Combinatorial optimization to associate the target head in the consecutive frames. The organization of the paper is the following: In section\ref{PA}, the proposed approach is briefly explained. The tracker details are given in section\ref{tracker}. The qualitative and quantitative results are given in section\ref{EX} and section\ref{CON} concludes the paper with future directions. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Multi-Target Tracking, Machine Learning, Engineering, Computer Vision, Mathematics, Human Body Representation, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1905.13258" label="1905.13258">
-        <attvalues>
-          <attvalue for="0" value="A survey of advances in vision-based vehicle re-identification" />
-          <attvalue for="1" value="  Vehicle re-identification (V-reID) has become significantly popular in the&#10;community due to its applications and research significance. In particular, the&#10;V-reID is an important problem that still faces numerous open challenges. This&#10;paper reviews different V-reID methods including sensor based methods, hybrid&#10;methods, and vision based methods which are further categorized into&#10;hand-crafted feature based methods and deep feature based methods. The vision&#10;based methods make the V-reID problem particularly interesting, and our review&#10;systematically addresses and evaluates these methods for the first time. We&#10;conduct experiments on four comprehensive benchmark datasets and compare the&#10;performances of recent hand-crafted feature based methods and deep feature&#10;based methods. We present the detail analysis of these methods in terms of mean&#10;average precision (mAP) and cumulative matching curve (CMC). These analyses&#10;provide objective insight into the strengths and weaknesses of these methods.&#10;We also provide the details of different V-reID datasets and critically discuss&#10;the challenges and future trends of V-reID methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.10698" label="1705.10698">
-        <attvalues>
-          <attvalue for="0" value="ResnetCrowd: A Residual Deep Learning Architecture for Crowd Counting,&#10;  Violent Behaviour Detection and Crowd Density Level Classification" />
-          <attvalue for="1" value="  In this paper we propose ResnetCrowd, a deep residual architecture for&#10;simultaneous crowd counting, violent behaviour detection and crowd density&#10;level classification. To train and evaluate the proposed multi-objective&#10;technique, a new 100 image dataset referred to as Multi Task Crowd is&#10;constructed. This new dataset is the first computer vision dataset fully&#10;annotated for crowd counting, violent behaviour detection and density level&#10;classification. Our experiments show that a multi-task approach boosts&#10;individual task performance for all tasks and most notably for violent&#10;behaviour detection which receives a 9\% boost in ROC curve AUC (Area under the&#10;curve). The trained ResnetCrowd model is also evaluated on several additional&#10;benchmarks highlighting the superior generalisation of crowd analysis models&#10;trained for multiple objectives.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.08482" label="1706.08482">
-        <attvalues>
-          <attvalue for="0" value="Deep Network Flow for Multi-Object Tracking" />
-          <attvalue for="1" value="  Data association problems are an important component of many computer vision&#10;applications, with multi-object tracking being one of the most prominent&#10;examples. A typical approach to data association involves finding a graph&#10;matching or network flow that minimizes a sum of pairwise association costs,&#10;which are often either hand-crafted or learned as linear functions of fixed&#10;features. In this work, we demonstrate that it is possible to learn features&#10;for network-flow-based data association via backpropagation, by expressing the&#10;optimum of a smoothed network flow problem as a differentiable function of the&#10;pairwise association costs. We apply this approach to multi-object tracking&#10;with a network flow formulation. Our experiments demonstrate that we are able&#10;to successfully learn all cost functions for the association problem in an&#10;end-to-end fashion, which outperform hand-crafted costs in all settings. The&#10;integration and combination of various sources of inputs becomes easy and the&#10;cost functions can be learned entirely from data, alleviating tedious&#10;hand-designing of costs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.02843" label="1708.02843">
-        <attvalues>
-          <attvalue for="0" value="Online Multi-Object Tracking Using CNN-based Single Object Tracker with&#10;  Spatial-Temporal Attention Mechanism" />
-          <attvalue for="1" value="  In this paper, we propose a CNN-based framework for online MOT. This&#10;framework utilizes the merits of single object trackers in adapting appearance&#10;models and searching for target in the next frame. Simply applying single&#10;object tracker for MOT will encounter the problem in computational efficiency&#10;and drifted results caused by occlusion. Our framework achieves computational&#10;efficiency by sharing features and using ROI-Pooling to obtain individual&#10;features for each target. Some online learned target-specific CNN layers are&#10;used for adapting the appearance model for each target. In the framework, we&#10;introduce spatial-temporal attention mechanism (STAM) to handle the drift&#10;caused by occlusion and interaction among targets. The visibility map of the&#10;target is learned and used for inferring the spatial attention map. The spatial&#10;attention map is then applied to weight the features. Besides, the occlusion&#10;status can be estimated from the visibility map, which controls the online&#10;updating process via weighted loss on training samples with different occlusion&#10;statuses in different frames. It can be considered as temporal attention&#10;mechanism. The proposed algorithm achieves 34.3% and 46.0% in MOTA on&#10;challenging MOT15 and MOT16 benchmark dataset respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.05289" label="2104.05289">
-        <attvalues>
-          <attvalue for="0" value="StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision" />
-          <attvalue for="1" value="  In this paper, we propose StereoPIFu, which integrates the geometric&#10;constraints of stereo vision with implicit function representation of PIFu, to&#10;recover the 3D shape of the clothed human from a pair of low-cost rectified&#10;images. First, we introduce the effective voxel-aligned features from a stereo&#10;vision-based network to enable depth-aware reconstruction. Moreover, the novel&#10;relative z-offset is employed to associate predicted high-fidelity human depth&#10;and occupancy inference, which helps restore fine-level surface details.&#10;Second, a network structure that fully utilizes the geometry information from&#10;the stereo images is designed to improve the human body reconstruction quality.&#10;Consequently, our StereoPIFu can naturally infer the human body's spatial&#10;location in camera space and maintain the correct relative position of&#10;different parts of the human body, which enables our method to capture human&#10;performance. Compared with previous works, our StereoPIFu significantly&#10;improves the robustness, completeness, and accuracy of the clothed human&#10;reconstruction, which is demonstrated by extensive experimental results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10; {\bf{Reconstruction from Single Image.}} Different strategies have been proposed to recover 3D human body shape from a single image. A common way is to fit the input image by a parametric model (e.g., SMPL~\cite{SMPL:2015}) via 2D landmarks and silhouette~\cite{bogo2016keep, kanazawa2018end,pavlakos2019expressive,kocabas2020vibe, jiang2020disentangled}. However, these methods can only recover the naked body's shape due to the limited representation ability of the parametric model. Several methods~\cite{alldieck2018video, alldieck2018detailed, alldieck19cvpr, alldieck2019tex2shape} add a displacement on each vertex to represent fine-scale details. Bhatnagar~\etal~\cite{bhatnagar2019multi} and Jiang~\etal~\cite{jiang2020bcnet} additionally regress the clothes via parametric models. These methods have improved the quality but are still unable to recover high-fidelity geometry shape.&#10;&#10;Several methods~\cite{varol2018bodynet, zheng2019deephuman} take 3D CNN to regress volumetric representation of human bodies. However, they can not recover geometric details due to the large memory consumption of 3D CNN. Instead of regressing a volume with a fixed resolution, Saito~\etal~\cite{saito2019pifu} propose Pixel-Aligned Implicit Function~(PIFu) representation, which is memory-efficient and can predict the occupancy of any 3D point. However, it may generate incorrect body structures and struggle to recover fine-scale geometric details. To this end, PIFuHD~\cite{saito2020pifuhd} improves PIFu with additionally extracted high-resolution features and predicted normal maps. Huang~\etal~\cite{huang2020arch} and Zheng~\etal~\cite{zheng2020pamir} take a feature related to the parametric human model as the input of the implicit function to improve the stability of prediction. He~\etal~\cite{he2020geo} expands the pixel-aligned feature with geometry-aligned shape features, which serves as a shape prior for the reconstruction. Gabeur~\etal~\cite{gabeur2019moulding} convert this problem as depth prediction of the front- and back-side, and the complete shape is obtained by merging the recovered point clouds together. However, these methods suffer from depth ambiguity due to the inherent nature of single image input.&#10;&#10;{\bf{Reconstruction from Multi-view Images.}} Previous studies extract shape cues from the silhouette, stereo, and shading, to recover geometric shape from multi-view inputs~\cite{song2010volumetric, collet2015high, xu2017shading, alldieck2018video, guo2019relightables}. The high-quality 3D shape can be reconstructed with hundreds of cameras~\cite{collet2015high, guo2019relightables,furukawa2009accurate, vzbontar2016stereo}, but this hardware configuration is inaccessible to general consumers due to its special equipment and complexity. Gilbert~\etal~\cite{gilbert2018volumetric} predicts the high-fidelity volume of the human shape utilizing a coarse visual hull generated by sparse view silhouettes with a 3D convolutional autoencoder network. Alldieck~\etal~\cite{alldieck2018video} take a video of humans slowly rotating as input and exploit the silhouette of frames to optimize the SMPL+D representation. Alldieck~\etal~\cite{alldieck19cvpr} uses a network to substitute the optimization, which leads to faster inference and simpler input (1-8 frames). However, these methods are hard to recover fine-level surface details due to their limited geometric representation abilities. Several works~\cite{huang2018deep, saito2019pifu} aggregate multi-view pixel-aligned features with a pooling layer to help reconstruction. Although improved results can be obtained, the simple aggregation with the pooling strategy does not make full use of the information endowed by multi-view images.&#10;&#10;{\bf{Depth Estimation from Stereo Images.}} Depth estimation from stereo images has been studied extensively for decades. &#10;Although traditional methods have made significant progress~\cite{bleyer2011patchmatch, hosni2012fast}, they still suffer from the edge-fattening~\cite{scharstein2002taxonomy, min2011revisit} and poor performance in challenging situations like textureless regions. Recently, deep learning-based methods have been proposed to alleviate these problems~\cite{mayer2016large, kendall2017end,chang2018pyramid,khamis2018stereonet,chabra2019stereodrnet,zhang2019ga,xu2020aanet}. DispNet~\cite{mayer2016large} constructs a large-scale synthetic dataset, Scene Flow, and builds the first end-to-end trainable framework for disparity prediction. GC-Net~\cite{kendall2017end} concatenates the left and right images' feature and applies 3D convolutions to aggregate the resulting 4D feature cost volume. PSMNet~\cite{chang2018pyramid} further improves the accuracy by using a stacked hourglass block~\cite{newell2016stacked} with more 3D convolutional layers. Several methods~\cite{zhang2019ga, xu2020aanet} design novel modules to reduce the memory consumption and can obtain competitive performance with other state-of-the-art methods. However, these methods can only recover the shape of the human body's visible part as they do not utilize any human body shape prior.&#10;&#10;" />
-          <attvalue for="4" value="Human Modeling, Computer Vision, Machine Learning, Mathematics, Physics, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1911.00756" label="1911.00756">
-        <attvalues>
-          <attvalue for="0" value="Beta DVBF: Learning State-Space Models for Control from High Dimensional&#10;  Observations" />
-          <attvalue for="1" value="  Learning a model of dynamics from high-dimensional images can be a core&#10;ingredient for success in many applications across different domains,&#10;especially in sequential decision making. However, currently prevailing methods&#10;based on latent-variable models are limited to working with low resolution&#10;images only. In this work, we show that some of the issues with using&#10;high-dimensional observations arise from the discrepancy between the&#10;dimensionality of the latent and observable space, and propose solutions to&#10;overcome them.&#10;" />
-          <attvalue for="2" value="&#10;&#09;Learning a probabilistic model for sequential data is a key step towards solving a lot of interesting problems, including analysis and deconstruction of auditory sequences \cite{turner2010statistical}, predicting the next piece of information given previously recorded data such as video frames \cite{jayaraman2018time} and text \cite{xie2017neural}, and controlling an agent to perform specific tasks (model-based reinforcement learning \cite{deisenroth2011pilco}). While in this paper, we consider probabilistic models especially tuned for the needs of the last, i.e with control inputs, the approaches we discuss can be applied to control-less environments as well. &#10;&#10;A key feature required in control-based models is that they should be able to generate a feasible trajectory distribution given a control policy.&#09;To this end, one of the more successful solutions proposed in the past for modeling dynamical systems is DVBF \cite{karl2016deep} - a framework for learning a State-Space Model given sequential observations from the environment in an unsupervised manner. Although models learned via DVBF have shown exceptional predictive performance on low dimensional data, they lose this capability when scaled to high-dimensional observations. In this work, we conclude that the reason for this discrepancy lies in the relative difference between the dimensions of observation and latent-space. We propose an extension to the DVBF model that takes this relative difference into account, thus improving its scalability. In the experimental section, we further discuss how the different constructs introduced in DVBF and in this work assist in learning a model with good prediction capabilities and to which degree. We validate these discussions by showcasing results on a number of different data sets.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Latent Variable Methods, Mathematics, Artificial Intelligence, Physics, High-Dimensional Observations, Computer Science, Image Dynamics Modeling" />
-        </attvalues>
-      </node>
-      <node id="1808.07784" label="1808.07784">
-        <attvalues>
-          <attvalue for="0" value="Time-Agnostic Prediction: Predicting Predictable Video Frames" />
-          <attvalue for="1" value="  Prediction is arguably one of the most basic functions of an intelligent&#10;system. In general, the problem of predicting events in the future or between&#10;two waypoints is exceedingly difficult. However, most phenomena naturally pass&#10;through relatively predictable bottlenecks---while we cannot predict the&#10;precise trajectory of a robot arm between being at rest and holding an object&#10;up, we can be certain that it must have picked the object up. To exploit this,&#10;we decouple visual prediction from a rigid notion of time. While conventional&#10;approaches predict frames at regularly spaced temporal intervals, our&#10;time-agnostic predictors (TAP) are not tied to specific times so that they may&#10;instead discover predictable &quot;bottleneck&quot; frames no matter when they occur. We&#10;evaluate our approach for future and intermediate frame prediction across three&#10;robotic manipulation tasks. Our predictions are not only of higher visual&#10;quality, but also correspond to coherent semantic subgoals in temporally&#10;extended tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.09534" label="1711.09534">
-        <attvalues>
-          <attvalue for="0" value="Neural Text Generation: A Practical Guide" />
-          <attvalue for="1" value="  Deep learning methods have recently achieved great empirical success on&#10;machine translation, dialogue response generation, summarization, and other&#10;text generation tasks. At a high level, the technique has been to train&#10;end-to-end neural network models consisting of an encoder model to produce a&#10;hidden representation of the source text, followed by a decoder model to&#10;generate the target. While such models have significantly fewer pieces than&#10;earlier systems, significant tuning is still required to achieve good&#10;performance. For text generation models in particular, the decoder can behave&#10;in undesired ways, such as by generating truncated or repetitive outputs,&#10;outputting bland and generic responses, or in some cases producing&#10;ungrammatical gibberish. This paper is intended as a practical guide for&#10;resolving such undesired behavior in text generation models, with the aim of&#10;helping enable real-world applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.06432" label="1605.06432">
-        <attvalues>
-          <attvalue for="0" value="Deep Variational Bayes Filters: Unsupervised Learning of State Space&#10;  Models from Raw Data" />
-          <attvalue for="1" value="  We introduce Deep Variational Bayes Filters (DVBF), a new method for&#10;unsupervised learning and identification of latent Markovian state space&#10;models. Leveraging recent advances in Stochastic Gradient Variational Bayes,&#10;DVBF can overcome intractable inference distributions via variational&#10;inference. Thus, it can handle highly nonlinear input data with temporal and&#10;spatial dependencies such as image sequences without domain knowledge. Our&#10;experiments show that enabling backpropagation through transitions enforces&#10;state space assumptions and significantly improves information content of the&#10;latent embedding. This also enables realistic long-term prediction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.01123" label="1906.01123">
-        <attvalues>
-          <attvalue for="0" value="Depth-Aware Arbitrary Style Transfer Using Instance Normalization" />
-          <attvalue for="1" value="  Style transfer is the process of rendering one image with some content in the&#10;style of another image, representing the style. Recent studies of Liu et al.&#10;(2017) show that traditional style transfer methods of Gatys et al. (2016) and&#10;Johnson et al. (2016) fail to reproduce the depth of the content image, which&#10;is critical for human perception. They suggest to preserve the depth map by&#10;additional regularizer in the optimized loss function, forcing preservation of&#10;the depth map. However these traditional methods are either computationally&#10;inefficient or require training a separate neural network for each style. AdaIN&#10;method of Huang et al. (2017) allows efficient transferring of arbitrary style&#10;without training a separate model but is not able to reproduce the depth map of&#10;the content image. We propose an extension to this method, allowing depth map&#10;preservation by applying variable stylization strength. Qualitative analysis&#10;and results of user evaluation study indicate that the proposed method provides&#10;better stylizations, compared to the original AdaIN style transfer method.&#10;" />
-          <attvalue for="2" value="&#10;The problem of rendering an image (called the content image) in a particular style is known as style transfer and is a long studied problem in computer vision. Early approaches~\cite{gooch2001non,strothotte2002non,rosin2012image} used algorithms with human engineered features targeting to impose particular styles. &#10;&#10;In 2016 Gatys~et al.~\cite{gatys2016image} proposed an algorithm of imposing arbitrary style taken from user defined style image on arbitrary content image by using representations of images that could be obtained with deep convolutional networks. However their method needed a computationally expensive optimization in the space of images requiring several minutes of processing a single image of moderate resolution on powerful GPUs. &#10;Ulyanov et al.~\cite{ulyanov2016instance} and Jonson et al.~\cite{johnson2016perceptual} proposed a real-time style transfer algorithm by passing a content image through a pretrained fully convolutional transformer network. Their methods required training a separate transformation network for each new style. &#10;Work of Liu et. al (2017)~\cite{liu2017depth} highlighted an issue with traditional style transfer methods that they failed to reproduce the depth map of the content image, that was critical for human perception of the result. To address this issue they extended traditional methods~\cite{gatys2016image} and \cite{johnson2016perceptual} with a regularizer, forcing preservation of the depth map of the content image. This yielded significant improvement of style transfer rendering quality but required computationally complex algorithm, requiring either solving high dimensional optimization problem for each content-style pair or fitting a separate transformer network for each style.&#10;Later architectures, such as AdaIN~\cite{huang2017arbitrary} and other (\cite{ghiasi2017exploring}, \cite{li2017universal}), allowed transferring arbitrary style without training a separate network but lacked rendering quality due to failure to preserve the depth map of the content image.&#10;&#10;In the work a depth aware AdaIN method extension (DA-AdaIN for short) is proposed that allows to preserve the depth map of the content during stylization, as shown on fig.~\ref{fig:uniform_vs_depth_aware}d by applying style with spatially variable strength: more close regions, standing for foreground, are stylized less, and more distant regions, standing for background, are stylized more. &#10;&#10;Qualitative analysis suggests that proposed modification leads to rendering quality improvement. User study confirms that proposed algorithm gives on average better results. &#10;&#10;The remainder of the paper is organized as follows. In section~\ref{sec:depth-estimation} two recent depth estimation methods are compared and the best one is used in later analysis. Section~\ref{sec:methods} describes standard AdaIN method and our proposed modification. Section~\ref{sec:evaluation} provides qualitative analysis of the proposed method, its dependency on major parameters and results of a user study, where AdaIN and proposed methods are compared. Finally section~\ref{sec:conclusion} concludes.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Image Style Transfer, Human-Computer Interaction, Cognitive Science, Computer Science, Depth Map Preservation, Neural Network Optimization" />
-        </attvalues>
-      </node>
-      <node id="1607.08022" label="1607.08022">
-        <attvalues>
-          <attvalue for="0" value="Instance Normalization: The Missing Ingredient for Fast Stylization" />
-          <attvalue for="1" value="  It this paper we revisit the fast stylization method introduced in Ulyanov&#10;et. al. (2016). We show how a small change in the stylization architecture&#10;results in a significant qualitative improvement in the generated images. The&#10;change is limited to swapping batch normalization with instance normalization,&#10;and to apply the latter both at training and testing times. The resulting&#10;method can be used to train high-performance architectures for real-time image&#10;generation. The code will is made available on github at&#10;https://github.com/DmitryUlyanov/texture_nets. Full paper can be found at&#10;arXiv:1701.02096.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.06830" label="1705.06830">
-        <attvalues>
-          <attvalue for="0" value="Exploring the structure of a real-time, arbitrary neural artistic&#10;  stylization network" />
-          <attvalue for="1" value="  In this paper, we present a method which combines the flexibility of the&#10;neural algorithm of artistic style with the speed of fast style transfer&#10;networks to allow real-time stylization using any content/style image pair. We&#10;build upon recent work leveraging conditional instance normalization for&#10;multi-style transfer networks by learning to predict the conditional instance&#10;normalization parameters directly from a style image. The model is successfully&#10;trained on a corpus of roughly 80,000 paintings and is able to generalize to&#10;paintings previously unobserved. We demonstrate that the learned embedding&#10;space is smooth and contains a rich structure and organizes semantic&#10;information associated with paintings in an entirely unsupervised manner.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.08086" label="1705.08086">
-        <attvalues>
-          <attvalue for="0" value="Universal Style Transfer via Feature Transforms" />
-          <attvalue for="1" value="  Universal style transfer aims to transfer arbitrary visual styles to content&#10;images. Existing feed-forward based methods, while enjoying the inference&#10;efficiency, are mainly limited by inability of generalizing to unseen styles or&#10;compromised visual quality. In this paper, we present a simple yet effective&#10;method that tackles these limitations without training on any pre-defined&#10;styles. The key ingredient of our method is a pair of feature transforms,&#10;whitening and coloring, that are embedded to an image reconstruction network.&#10;The whitening and coloring transforms reflect a direct matching of feature&#10;covariance of the content image to a given style image, which shares similar&#10;spirits with the optimization of Gram matrix based cost in neural style&#10;transfer. We demonstrate the effectiveness of our algorithm by generating&#10;high-quality stylized images with comparisons to a number of recent methods. We&#10;also analyze our method by visualizing the whitened features and synthesizing&#10;textures via simple feature coloring.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.02508" label="2307.02508">
-        <attvalues>
-          <attvalue for="0" value="ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single&#10;  Object Tracking" />
-          <attvalue for="1" value="  The Associating Objects with Transformers (AOT) framework has exhibited&#10;exceptional performance in a wide range of complex scenarios for video object&#10;tracking and segmentation. In this study, we convert the bounding boxes to&#10;masks in reference frames with the help of the Segment Anything Model (SAM) and&#10;Alpha-Refine, and then propagate the masks to the current frame, transforming&#10;the task from Video Object Tracking (VOT) to video object segmentation (VOS).&#10;Furthermore, we introduce MSDeAOT, a variant of the AOT series that&#10;incorporates transformers at multiple feature scales. MSDeAOT efficiently&#10;propagates object masks from previous frames to the current frame using two&#10;feature scales of 16 and 8. As a testament to the effectiveness of our design,&#10;we achieved the 1st place in the EPIC-KITCHENS TREK-150 Object Tracking&#10;Challenge.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Video object tracking is a fundamental task in computer vision that involves automatically localizing and tracking a specific object of interest across consecutive frames in a video sequence. &#10;It plays a crucial role in various applications, such as surveillance systems, autonomous vehicles, and video analysis. &#10;&#10;The tracking process typically starts with an initial bounding box or mask annotation around the target object in the first frame. Subsequently, tracking algorithms analyze subsequent frames to predict the object's location, usually by exploiting motion and appearance cues. These algorithms employ a variety of techniques, including correlation filters, deep learning-based models, particle filters, and optical flow-based methods. &#10;&#10;In recent years, the field of video object tracking has witnessed remarkable advancements through deep learning-based approaches. SwinTrack~\cite{lin2022swintrack} stands out by leveraging transformers for feature extraction and fusion, enabling seamless interaction between the target object and the search area during tracking. Meanwhile, MixFormer~\cite{cui2022mixformer} adopts a novel approach by integrating template and test samples in the backbone network and directly regressing bounding boxes using a simple head, eliminating the need for post-processing. Another noteworthy method, SeqTrack~\cite{chen2023seqtrack}, tackles the visual tracking problem by reformulating it as a sequence generation task, enabling autoregressive prediction of the target bounding box. &#10;&#10;Although the methods mentioned above have yielded impressive outcomes, they are constrained in their handling of bounding boxes as both references and outputs. Real-world scenarios present challenges where bounding boxes tend to be prone to inaccuracies, potentially encompassing multiple objects. Additionally, the fixed shape of the bounding box hinders effective adaptation to target shape variations. &#10;&#10;Hence, tracking methods that utilize masks tend to achieve higher accuracy owing to their pixel-based alignment mechanism~\cite{li2023unified, cheng2023segment, zhou2022survey, li2022locality, yang2021collaborative}. Over the years, the Associating Objects with Transformers (AOT)~\cite{yang2021associating} series has demonstrated commendable performance in video object segmentation (VOS) tasks, prompting the natural extension of applying it to video object tracking (VOT) tasks. However, the annotation of masks poses greater difficulty compared to bounding box annotation. Fortunately, the introduction of the Segment Anything Model (SAM)~\cite{kirillov2023segment} has significantly alleviated this challenge by enabling the conversion of bounding boxes into masks with remarkable efficacy.&#10;&#10;Specifically, we propose a novel tracking framework that leverages the AOT series to track objects in videos. We first convert the bounding boxes to masks in reference frames with the help of SAM and Alpha-Refine~\cite{yan2021alpha}, and then feed the mask and frames into the VOS model. The model then propagates the masks to the current frame and the bounding box is obtained by the mask. We also introduce MSDeAOT, a variant of the AOT series that incorporates transformers at multiple feature scales. &#10;Leveraging above techniques, we achieve the 1st place in the EPIC-KITCHENS TREK-150 Object Tracking Challenge, attaining an impressive success score of 73.4\% under the multi-start evaluation protocol.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Transformer Models, Mathematics, Artificial Intelligence, Object Tracking, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2112.00995" label="2112.00995">
-        <attvalues>
-          <attvalue for="0" value="SwinTrack: A Simple and Strong Baseline for Transformer Tracking" />
-          <attvalue for="1" value="  Recently Transformer has been largely explored in tracking and shown&#10;state-of-the-art (SOTA) performance. However, existing efforts mainly focus on&#10;fusing and enhancing features generated by convolutional neural networks&#10;(CNNs). The potential of Transformer in representation learning remains&#10;under-explored. In this paper, we aim to further unleash the power of&#10;Transformer by proposing a simple yet efficient fully-attentional tracker,&#10;dubbed SwinTrack, within classic Siamese framework. In particular, both&#10;representation learning and feature fusion in SwinTrack leverage the&#10;Transformer architecture, enabling better feature interactions for tracking&#10;than pure CNN or hybrid CNN-Transformer frameworks. Besides, to further enhance&#10;robustness, we present a novel motion token that embeds historical target&#10;trajectory to improve tracking by providing temporal context. Our motion token&#10;is lightweight with negligible computation but brings clear gains. In our&#10;thorough experiments, SwinTrack exceeds existing approaches on multiple&#10;benchmarks. Particularly, on the challenging LaSOT, SwinTrack sets a new record&#10;with 0.713 SUC score. It also achieves SOTA results on other benchmarks. We&#10;expect SwinTrack to serve as a solid baseline for Transformer tracking and&#10;facilitate future research. Our codes and results are released at&#10;https://github.com/LitingLin/SwinTrack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.10100" label="2303.10100">
-        <attvalues>
-          <attvalue for="0" value="Unified Mask Embedding and Correspondence Learning for Self-Supervised&#10;  Video Segmentation" />
-          <attvalue for="1" value="  The objective of this paper is self-supervised learning of video object&#10;segmentation. We develop a unified framework which simultaneously models&#10;cross-frame dense correspondence for locally discriminative feature learning&#10;and embeds object-level context for target-mask decoding. As a result, it is&#10;able to directly learn to perform mask-guided sequential segmentation from&#10;unlabeled videos, in contrast to previous efforts usually relying on an oblique&#10;solution - cheaply &quot;copying&quot; labels according to pixel-wise correlations.&#10;Concretely, our algorithm alternates between i) clustering video pixels for&#10;creating pseudo segmentation labels ex nihilo; and ii) utilizing the pseudo&#10;labels to learn mask encoding and decoding for VOS. Unsupervised correspondence&#10;learning is further incorporated into this self-taught, mask embedding scheme,&#10;so as to ensure the generic nature of the learnt representation and avoid&#10;cluster degeneracy. Our algorithm sets state-of-the-arts on two standard&#10;benchmarks (i.e., DAVIS17 and YouTube-VOS), narrowing the gap between self- and&#10;fully-supervised VOS, in terms of both performance and network architecture&#10;design.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.06815" label="2012.06815">
-        <attvalues>
-          <attvalue for="0" value="Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box&#10;  Estimation" />
-          <attvalue for="1" value="  Visual object tracking aims to precisely estimate the bounding box for the&#10;given target, which is a challenging problem due to factors such as deformation&#10;and occlusion. Many recent trackers adopt the multiple-stage tracking strategy&#10;to improve the quality of bounding box estimation. These methods first coarsely&#10;locate the target and then refine the initial prediction in the following&#10;stages. However, existing approaches still suffer from limited precision, and&#10;the coupling of different stages severely restricts the method's&#10;transferability. This work proposes a novel, flexible, and accurate refinement&#10;module called Alpha-Refine (AR), which can significantly improve the base&#10;trackers' box estimation quality. By exploring a series of design options, we&#10;conclude that the key to successful refinement is extracting and maintaining&#10;detailed spatial information as much as possible. Following this principle,&#10;Alpha-Refine adopts a pixel-wise correlation, a corner prediction head, and an&#10;auxiliary mask head as the core components. Comprehensive experiments on&#10;TrackingNet, LaSOT, GOT-10K, and VOT2020 benchmarks with multiple base trackers&#10;show that our approach significantly improves the base trackers' performance&#10;with little extra latency. The proposed Alpha-Refine method leads to a series&#10;of strengthened trackers, among which the ARSiamRPN (AR strengthened SiamRPNpp)&#10;and the ARDiMP50 (ARstrengthened DiMP50) achieve good efficiency-precision&#10;trade-off, while the ARDiMPsuper (AR strengthened DiMP-super) achieves very&#10;competitive performance at a real-time speed. Code and pretrained models are&#10;available at https://github.com/MasterBin-IIAU/AlphaRefine.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07596" label="1611.07596">
-        <attvalues>
-          <attvalue for="0" value="Fast Fourier Color Constancy" />
-          <attvalue for="1" value="  We present Fast Fourier Color Constancy (FFCC), a color constancy algorithm&#10;which solves illuminant estimation by reducing it to a spatial localization&#10;task on a torus. By operating in the frequency domain, FFCC produces lower&#10;error rates than the previous state-of-the-art by 13-20% while being 250-3000&#10;times faster. This unconventional approach introduces challenges regarding&#10;aliasing, directional statistics, and preconditioning, which we address. By&#10;producing a complete posterior distribution over illuminants instead of a&#10;single illuminant estimate, FFCC enables better training techniques, an&#10;effective temporal smoothing technique, and richer methods for error analysis.&#10;Our implementation of FFCC runs at ~700 frames per second on a mobile device,&#10;allowing it to be used as an accurate, real-time, temporally-coherent automatic&#10;white balance algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Color Constancy, Mathematics, Physics, Image Processing, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2108.05997" label="2108.05997">
-        <attvalues>
-          <attvalue for="0" value="MUSIQ: Multi-scale Image Quality Transformer" />
-          <attvalue for="1" value="  Image quality assessment (IQA) is an important research topic for&#10;understanding and improving visual experience. The current state-of-the-art IQA&#10;methods are based on convolutional neural networks (CNNs). The performance of&#10;CNN-based models is often compromised by the fixed shape constraint in batch&#10;training. To accommodate this, the input images are usually resized and cropped&#10;to a fixed shape, causing image quality degradation. To address this, we design&#10;a multi-scale image quality Transformer (MUSIQ) to process native resolution&#10;images with varying sizes and aspect ratios. With a multi-scale image&#10;representation, our proposed method can capture image quality at different&#10;granularities. Furthermore, a novel hash-based 2D spatial embedding and a scale&#10;embedding is proposed to support the positional embedding in the multi-scale&#10;representation. Experimental results verify that our method can achieve&#10;state-of-the-art performance on multiple large scale IQA datasets such as&#10;PaQ-2-PiQ, SPAQ and KonIQ-10k.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Image Quality Assessment. Image quality assessment aims to quantitatively predict perceptual image quality. There are two important aspects for assessing image quality: technical quality \cite{hosu2020koniq} and aesthetic quality \cite{murray2012ava}. The former focuses on perceptual distortions while the latter also relates to image composition, artistic value and so on. In the past years, researchers proposed many IQA methods: early natural scene statistics based~\cite{ghadiyaram2017perceptual, mittal2012no, moorthy2011blind, zhang2015feature}, codebook-based~\cite{xue2013learning, ye2012unsupervised} and CNN-based~\cite{fang2020perceptual, su2020blindly, talebi2018nima, ying2020patches, zhang2018blind}. CNN-based methods achieve the state-of-the-art performance. However they usually need to crop or resize images to a fixed size in batch training, which affects the image quality. Several methods have been proposed to mitigate the distortion from resizing and cropping in CNN-based IQA. An ensemble of multi-crops from the original image is proven to be effective for IQA \cite{chen2020adaptive, hosu2019effective, ma2017lamp, sheng2018attention, su2020blindly}, but it introduces non-negligible inference cost. In addition, MNA-CNN \cite{mai2016composition} handles full-size input by adaptively pooling the feature map to a fixed shape. However, it only accepts a single input image for each training batch to preserve the original resolution which is not efficient for large scale training. \cite{hosu2019effective} extracted and stored the fixed-sized features from the full-size image for model training which costs extra storage for every augmented image and is inefficient for large scale training. \cite{chen2020adaptive} proposed an adaptive fractional dilated convolution to adapt the receptive field according to the image aspect ratio. The method preserves aspect ratio but cannot handle full-size input without resizing. It also needs smart grouping strategy in mini-batch training.&#10;&#10;Transformers in Vision. Transformers \cite{NIPS2017_3f5ee243} were first applied to NLP tasks and achieved great performance \cite{yang2019xlNet, devlin2018bert, liu2019roberta}. Recent works applied transformers on various vision tasks \cite{carion2020end, chen2020pre, pmlr-v119-chen20s, dosovitskiy2020}. Among these, the Vision Transformer (ViT) \cite{dosovitskiy2020} employs a pure Transformer architecture to classify images by treating an image as a sequence of patches. &#10;For batch training, ViT resizes the input images to a fixed squared size,~\eg, 224 $\times$ 224, where fixed number of patches are extracted and combined with fixed-length positional embedding. This constrains its usage for IQA since resizing will affect the image quality. To solve this, we propose a novel Transformer-based architecture that accepts the full-size image for IQA.&#10;&#10;Positional Embeddings. Positional embeddings are introduced in Transformers to encode the order of the input sequence \cite{NIPS2017_3f5ee243}. Without it, the self-attention operation is permutation-invariant \cite{bello2019attention}. \cite{NIPS2017_3f5ee243} used deterministic positional embeddings generated from sinusoidal functions. ViT \cite{dosovitskiy2020} showed that the deterministic and learnable positional embeddings \cite{gehring2017convolutional} works equally well. However, those positional embeddings are generated for fixed-length sequences. When the input resolution changes, the pre-trained positional embeddings is no longer meaningful. Relative positional embeddings~\cite{shaw2018self, bello2019attention} is proposed to encode relative distance instead of absolute position. Although the relative positional embeddings can work for variable length inputs, it requires substantial modifications in Transformer attention and cannot capture multi-scale positions in our use case. &#10;&#10;" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Image Quality Assessment, Deep Learning Methods, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2005.11546" label="2005.11546">
-        <attvalues>
-          <attvalue for="0" value="ProAlignNet : Unsupervised Learning for Progressively Aligning Noisy&#10;  Contours" />
-          <attvalue for="1" value="  Contour shape alignment is a fundamental but challenging problem in computer&#10;vision, especially when the observations are partial, noisy, and largely&#10;misaligned. Recent ConvNet-based architectures that were proposed to align&#10;image structures tend to fail with contour representation of shapes, mostly due&#10;to the use of proximity-insensitive pixel-wise similarity measures as loss&#10;functions in their training processes. This work presents a novel ConvNet,&#10;&quot;ProAlignNet&quot; that accounts for large scale misalignments and complex&#10;transformations between the contour shapes. It infers the warp parameters in a&#10;multi-scale fashion with progressively increasing complex transformations over&#10;increasing scales. It learns --without supervision-- to align contours,&#10;agnostic to noise and missing parts, by training with a novel loss function&#10;which is derived an upperbound of a proximity-sensitive and local&#10;shape-dependent similarity metric that uses classical Morphological Chamfer&#10;Distance Transform. We evaluate the reliability of these proposals on a&#10;simulated MNIST noisy contours dataset via some basic sanity check experiments.&#10;Next, we demonstrate the effectiveness of the proposed models in two real-world&#10;applications of (i) aligning geo-parcel data to aerial image maps and (ii)&#10;refining coarsely annotated segmentation labels. In both applications, the&#10;proposed models consistently perform superior to state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Contour shape alignment with noisy image observations is a fundamental, but challenging, problem in computer vision and graphics fields, with diverse applications including skeleton/silhouette alignment \cite{alexiadis2011evaluating} (for animation re-targeting), semantic boundary alignment \cite{yu2018simultaneous} and shape-to-scan alignment \cite{hirshberg2012coregistration} etc. &#10;For instance, consider the first row of Figure \ref{fig_intro}. It represents a process of geo-parcel alignment that requires aligning geo-parcel data (legal land boundaries maintained by local counties) to aerial image maps. &#10;These two modalities of geo-spatial data, if well aligned, are useful to assist the processes of property assessment and tax/insurance underwritings. &#10;Classically, contour alignment problems have been approached by finding key points or features of the shapes and aligning them by optimizing for the parameters of a predefined class of transformations such as affine or rigid transforms \cite{saxena2014survey}. These methods may not work for the shapes whose alignment requires a transform different from the hired ones. Nonrigid registration methods have also been proposed in the literature, mainly using intensity-based similarity metrics \cite{klein2009elastix}. However, these methods are often computationally expensive and sensitive towards corrupted parts of the shapes. &#10;&#10;Motivated by the strong invasion and great success of deep convolutional neural networks (ConvNets) in various vision tasks, some recent works \cite{kanazawa2016warpnet,miao2018dilated,guan2019deformable,hanocka2018alignet,de2017end} have designed ConvNet based architectures for shape alignment/registration and shown impressive results on the datasets with limited misalignments. However, we observe that these approaches tend to fail in noisy, partially observed and largely misaligned contour shape contexts. We believe that this is due to (a) direct prediction of alignment warp in one-go at input resolution and (b) training with proximity-insensitive pixel-level similarity metrics (such as normalized cross correlation \cite{de2017end}) that might result in non-informative gradients as the spatial overlap between contours may not be significant. &#10;&#10;To overcome the problems mentioned above, we propose a novel ConvNet architecture, ``ProAlignNet&quot; that learns to align a pair of contour images while being robust to noise and partial occlusions in the images. It finds an optimal transformation that aligns source to target contours in a multi-scale fashion with progressively increasing complex transformations over increasing scales. &#10;&#10;We also propose a novel loss function that accounts for not only local shape similarity but also proximity of similar shapes. The proposed loss is based on classical Chamfer distance that measures the proximity between the contours, thus, provides informative gradients even though there is no spatial overlap between the contours. Chamfer distance was a popular metric \cite{borgefors1984distance,gavrila1998multi} for binary shape alignment and was traditionally implemented efficiently using morphological distance transforms \cite{butt1998optimum}.&#10;However, this morphological chamfer distance transform (MCDT) is nondifferentiable wrt the warp parameters, which makes it nontrivial to use with BackProp. Hence, we device a reparameterization trick (inspired from homeomorphism properties of the transformation families we use) to make it usable with BackProp. &#10;However, this trick requires a backward/inverse transform (that aligns target to source) to be measured. Here, we propose to swap the roles of source and target features in the alignment modules to get the backward transform with the same network components without any additional parameters/layers. As a side benefit, this forward-backward transform consistency constraint acts as a powerful regularizer that improves generalization capabilities. &#10;&#10;Chamfer distance is amenable to noise as it uses Euclidean distance to find nearest neighbours, and it neglects local shape statistics around pixels. Hence, we introduce a data-dependent term into distance function that MCDT uses. This modification increases the proposed loss's robustness to noise pixels. We then derive an upperbound of this loss function, which is differentiable and computationally efficient. Since we use one instance of the loss at each scale, this multiscale objective passes gradients to all the layers simultaneously. It helps the training process to be more stable and less sensitive to hyperparameters such as learning rates as reported in \cite{karnewar2019msg}. Our training process is completely unsupervised as it does not require any ground-truth (GT) warps that the network is expected to reproduce. By training with a loss function that accounts for local shape contexts while being invariant to corrupted pixels, our network learns to be robust to noise and partial occlusions.&#10;&#10;In summary, our contributions are the following:&#10;\begin{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Image Alignment, Geographic Information Science" />
-        </attvalues>
-      </node>
-      <node id="1808.01992" label="1808.01992">
-        <attvalues>
-          <attvalue for="0" value="Simultaneous Edge Alignment and Learning" />
-          <attvalue for="1" value="  Edge detection is among the most fundamental vision problems for its role in&#10;perceptual grouping and its wide applications. Recent advances in&#10;representation learning have led to considerable improvements in this area.&#10;Many state of the art edge detection models are learned with fully&#10;convolutional networks (FCNs). However, FCN-based edge learning tends to be&#10;vulnerable to misaligned labels due to the delicate structure of edges. While&#10;such problem was considered in evaluation benchmarks, similar issue has not&#10;been explicitly addressed in general edge learning. In this paper, we show that&#10;label misalignment can cause considerably degraded edge learning quality, and&#10;address this issue by proposing a simultaneous edge alignment and learning&#10;framework. To this end, we formulate a probabilistic model where edge alignment&#10;is treated as latent variable optimization, and is learned end-to-end during&#10;network training. Experiments show several applications of this work, including&#10;improved edge detection with state of the art performance, and automatic&#10;refinement of noisy annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.05592" label="1604.05592">
-        <attvalues>
-          <attvalue for="0" value="WarpNet: Weakly Supervised Matching for Single-view Reconstruction" />
-          <attvalue for="1" value="  We present an approach to matching images of objects in fine-grained datasets&#10;without using part annotations, with an application to the challenging problem&#10;of weakly supervised single-view reconstruction. This is in contrast to prior&#10;works that require part annotations, since matching objects across class and&#10;pose variations is challenging with appearance features alone. We overcome this&#10;challenge through a novel deep learning architecture, WarpNet, that aligns an&#10;object in one image with a different object in another. We exploit the&#10;structure of the fine-grained dataset to create artificial data for training&#10;this network in an unsupervised-discriminative learning approach. The output of&#10;the network acts as a spatial prior that allows generalization at test time to&#10;match real images across variations in appearance, viewpoint and articulation.&#10;On the CUB-200-2011 dataset of bird categories, we improve the AP over an&#10;appearance-only network by 13.6%. We further demonstrate that our WarpNet&#10;matches, together with the structure of fine-grained datasets, allow&#10;single-view reconstructions with quality comparable to using annotated point&#10;correspondences.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.01651" label="1712.01651">
-        <attvalues>
-          <attvalue for="0" value="Dilated FCN for Multi-Agent 2D/3D Medical Image Registration" />
-          <attvalue for="1" value="  2D/3D image registration to align a 3D volume and 2D X-ray images is a&#10;challenging problem due to its ill-posed nature and various artifacts presented&#10;in 2D X-ray images. In this paper, we propose a multi-agent system with an auto&#10;attention mechanism for robust and efficient 2D/3D image registration.&#10;Specifically, an individual agent is trained with dilated Fully Convolutional&#10;Network (FCN) to perform registration in a Markov Decision Process (MDP) by&#10;observing a local region, and the final action is then taken based on the&#10;proposals from multiple agents and weighted by their corresponding confidence&#10;levels. The contributions of this paper are threefold. First, we formulate&#10;2D/3D registration as a MDP with observations, actions, and rewards properly&#10;defined with respect to X-ray imaging systems. Second, to handle various&#10;artifacts in 2D X-ray images, multiple local agents are employed efficiently&#10;via FCN-based structures, and an auto attention mechanism is proposed to favor&#10;the proposals from regions with more reliable visual cues. Third, a dilated&#10;FCN-based training mechanism is proposed to significantly reduce the Degree of&#10;Freedom in the simulation of registration environment, and drastically improve&#10;training efficiency by an order of magnitude compared to standard CNN-based&#10;training method. We demonstrate that the proposed method achieves high&#10;robustness on both spine cone beam Computed Tomography data with a low&#10;signal-to-noise ratio and data from minimally invasive spine surgery where&#10;severe image artifacts and occlusions are presented due to metal screws and&#10;guide wires, outperforming other state-of-the-art methods (single agent-based&#10;and optimization-based) by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.08497" label="1804.08497">
-        <attvalues>
-          <attvalue for="0" value="ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning" />
-          <attvalue for="1" value="  The process of aligning a pair of shapes is a fundamental operation in&#10;computer graphics. Traditional approaches rely heavily on matching&#10;corresponding points or features to guide the alignment, a paradigm that&#10;falters when significant shape portions are missing. These techniques generally&#10;do not incorporate prior knowledge about expected shape characteristics, which&#10;can help compensate for any misleading cues left by inaccuracies exhibited in&#10;the input shapes. We present an approach based on a deep neural network,&#10;leveraging shape datasets to learn a shape-aware prior for source-to-target&#10;alignment that is robust to shape incompleteness. In the absence of ground&#10;truth alignments for supervision, we train a network on the task of shape&#10;alignment using incomplete shapes generated from full shapes for&#10;self-supervision. Our network, called ALIGNet, is trained to warp complete&#10;source shapes to incomplete targets, as if the target shapes were complete,&#10;thus essentially rendering the alignment partial-shape agnostic. We aim for the&#10;network to develop specialized expertise over the common characteristics of the&#10;shapes in each dataset, thereby achieving a higher-level understanding of the&#10;expected shape space to which a local approach would be oblivious. We constrain&#10;ALIGNet through an anisotropic total variation identity regularization to&#10;promote piecewise smooth deformation fields, facilitating both partial-shape&#10;agnosticism and post-deformation applications. We demonstrate that ALIGNet&#10;learns to align geometrically distinct shapes, and is able to infer plausible&#10;mappings even when the target shape is significantly incomplete. We show that&#10;our network learns the common expected characteristics of shape collections,&#10;without over-fitting or memorization, enabling it to produce plausible&#10;deformations on unseen data during test time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.06065" label="1704.06065">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Unsupervised Deformable Image Registration with a&#10;  Convolutional Neural Network" />
-          <attvalue for="1" value="  In this work we propose a deep learning network for deformable image&#10;registration (DIRNet). The DIRNet consists of a convolutional neural network&#10;(ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet&#10;analyzes a pair of fixed and moving images and outputs parameters for the&#10;spatial transformer, which generates the displacement vector field that enables&#10;the resampler to warp the moving image to the fixed image. The DIRNet is&#10;trained end-to-end by unsupervised optimization of a similarity metric between&#10;input image pairs. A trained DIRNet can be applied to perform registration on&#10;unseen image pairs in one pass, thus non-iteratively. Evaluation was performed&#10;with registration of images of handwritten digits (MNIST) and cardiac cine MR&#10;scans (Sunnybrook Cardiac Data). The results demonstrate that registration with&#10;DIRNet is as accurate as a conventional deformable image registration method&#10;with substantially shorter execution times.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.15848" label="2303.15848">
-        <attvalues>
-          <attvalue for="0" value="4K-HAZE: A Dehazing Benchmark with 4K Resolution Hazy and Haze-Free&#10;  Images" />
-          <attvalue for="1" value="  Currently, mobile and IoT devices are in dire need of a series of methods to&#10;enhance 4K images with limited resource expenditure. The absence of large-scale&#10;4K benchmark datasets hampers progress in this area, especially for dehazing.&#10;The challenges in building ultra-high-definition (UHD) dehazing datasets are&#10;the absence of estimation methods for UHD depth maps, high-quality 4K depth&#10;estimation datasets, and migration strategies for UHD haze images from&#10;synthetic to real domains. To address these problems, we develop a novel&#10;synthetic method to simulate 4K hazy images (including nighttime and daytime&#10;scenes) from clear images, which first estimates the scene depth, simulates the&#10;light rays and object reflectance, then migrates the synthetic images to real&#10;domains by using a GAN, and finally yields the hazy effects on 4K resolution&#10;images. We wrap these synthesized images into a benchmark called the 4K-HAZE&#10;dataset. Specifically, we design the CS-Mixer (an MLP-based model that&#10;integrates \textbf{C}hannel domain and \textbf{S}patial domain) to estimate the&#10;depth map of 4K clear images, the GU-Net to migrate a 4K synthetic image to the&#10;real hazy domain. The most appealing aspect of our approach (depth estimation&#10;and domain migration) is the capability to run a 4K image on a single GPU with&#10;24G RAM in real-time (33fps). Additionally, this work presents an objective&#10;assessment of several state-of-the-art single-image dehazing methods that are&#10;evaluated using the 4K-HAZE dataset. At the end of the paper, we discuss the&#10;limitations of the 4K-HAZE dataset and its social implications.&#10;" />
-          <attvalue for="2" value="&#10;&#10;With the large-scale application of Ultra-High-Definition (UHD) images on handheld devices, haze degrades the visual quality of UHD images the outdoor.&#10;Most single-image dehazing methods address the visual degradation of low-resolution images, but these methods encounter the obstacle on running a 4K image.&#10;&#10;The obstacle in the UHD image dehazing field is that the dehazing models are trained on low-resolution images, and these models lack the knowledge (finer details of the structure, denser pixels in local areas) to make inferences about UHD image processing.&#10;To address the problem, we built a UHD dehazing dataset containing both daytime and nighttime scenes, named 4K-HAZE.&#10;Specifically, we propose the CS-Mixer model to estimate the depth of the 4K resolution image to assist the atmospheric scattering model to generate a non-homogeneous 4K haze image (haze distribution has a non-homogeneous character in many real scenes), then we use GU-Net to render a scene closer to the real haze based on a GAN scheme.&#10;Note that CS-Mixer is trained on a customized 4K depth estimation dataset ( including 500 pairs of images).&#10;The 4K depth map uses guide filtering and Adobe's Depth Scanner ().&#10;This study introduces 4K-HAZE which represents the first 4K image dehazing dataset with non-homogeneous hazy and haze-free (ground-truth) paired images.&#10;This work has two very noteworthy aspects: a) the blurring effect of the image comes from the estimation of the blurring kernel of the real-world image; b) both CS-Mixer and GU-Net can accurately conduct depth estimation and image degradation on a UHD resolution.&#10;&#10;4K-HAZE contains over 30,000 pairs of 4K images recorded outdoors.&#10;Additionally, this work presents a comprehensive evaluation of several state-of-the-art single-image dehazing methods~\cite{mgbl, psd, dcp, nl, cap, aod, fdmhn, pff, tang2019single, li2021dehazeflow, liu2022image}, that were objectively evaluated on our new dataset.&#10;As shown in Figure~\ref{ms-psnr}, the analyzed single-image dehazing techniques have been assessed quantitatively using two traditional metrics: PSNR and run time.&#10;DCP~\cite{dcp}, CAP~\cite{cap}, and NL~\cite{nl} have achieved grand success in image dehazing, but their performances are currently far inferior to deep learning methods.&#10;AODNet~\cite{aod}, FDMHN~\cite{fdmhn}, PFFNet~\cite{pff}, GCANet~\cite{tang2019single}, MGBL~\cite{mgbl}, DehazeFlow~\cite{li2021dehazeflow}, UDDN~\cite{liu2022image} and PSD~\cite{psd} can be dehazed in real time on a single 24G RAM RTX3090 GPU.&#10;Among them, MGBL~\cite{mgbl} and PSD~\cite{psd} achieved outstanding performance in terms of speed and accuracy.&#10;Note that all of the above methods act on the full-resolution images ($3 \times 3840 \times 2160$) during the training stage (these methods do not conduct resize, random cropping, mixture so on).&#10;To summarize, the main contributions of this paper are summarized as follows:&#10;\begin{itemize}&#10;\item We design a synthetic 4K image with a haze pipeline, which can handle a 4K image by using CS-Mixer and GU-Net in an end-to-end manner.&#10;\item We propose CS-Mixer and GU-Net, both of which have the core component of capturing global perception capability by rolling the spatial and channel domains; to the best of our knowledge this is the first strategy to remove the CNN stem and feed it directly into the MLPs.&#10;\item Our built 4K-HAZE provides well-rounded knowledge (including the nighttime and the daytime scenes) of existing dehazing methods, in addition to significant improvements in downstream tasks.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Dehazing Methods, Machine Learning, Engineering, Computer Vision, Physics, Computer Science, Benchmark Dataset Creation, Depth Estimation Techniques, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2005.05999" label="2005.05999">
-        <attvalues>
-          <attvalue for="0" value="Fast Deep Multi-patch Hierarchical Network for Nonhomogeneous Image&#10;  Dehazing" />
-          <attvalue for="1" value="  Recently, CNN based end-to-end deep learning methods achieve superiority in&#10;Image Dehazing but they tend to fail drastically in Non-homogeneous dehazing.&#10;Apart from that, existing popular Multi-scale approaches are runtime intensive&#10;and memory inefficient. In this context, we proposed a fast Deep Multi-patch&#10;Hierarchical Network to restore Non-homogeneous hazed images by aggregating&#10;features from multiple image patches from different spatial sections of the&#10;hazed image with fewer number of network parameters. Our proposed method is&#10;quite robust for different environments with various density of the haze or fog&#10;in the scene and very lightweight as the total size of the model is around 21.7&#10;MB. It also provides faster runtime compared to current multi-scale methods&#10;with an average runtime of 0.0145s to process 1200x1600 HD quality image.&#10;Finally, we show the superiority of this network on Dense Haze Removal to other&#10;state-of-the-art models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.02283" label="1810.02283">
-        <attvalues>
-          <attvalue for="0" value="Progressive Feature Fusion Network for Realistic Image Dehazing" />
-          <attvalue for="1" value="  Single image dehazing is a challenging ill-posed restoration problem. Various&#10;prior-based and learning-based methods have been proposed. Most of them follow&#10;a classic atmospheric scattering model which is an elegant simplified physical&#10;model based on the assumption of single-scattering and homogeneous atmospheric&#10;medium. The formulation of haze in realistic environment is more complicated.&#10;In this paper, we propose to take its essential mechanism as &quot;black box&quot;, and&#10;focus on learning an input-adaptive trainable end-to-end dehazing model. An&#10;U-Net like encoder-decoder deep network via progressive feature fusions has&#10;been proposed to directly learn highly nonlinear transformation function from&#10;observed hazy image to haze-free ground-truth. The proposed network is&#10;evaluated on two public image dehazing benchmarks. The experiments demonstrate&#10;that it can achieve superior performance when compared with popular&#10;state-of-the-art methods. With efficient GPU memory usage, it can&#10;satisfactorily recover ultra high definition hazed image up to 4K resolution,&#10;which is unaffordable by many deep learning based dehazing algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.08088" label="2112.08088">
-        <attvalues>
-          <attvalue for="0" value="Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions" />
-          <attvalue for="1" value="  Though deep learning-based object detection methods have achieved promising&#10;results on the conventional datasets, it is still challenging to locate objects&#10;from the low-quality images captured in adverse weather conditions. The&#10;existing methods either have difficulties in balancing the tasks of image&#10;enhancement and object detection, or often ignore the latent information&#10;beneficial for detection. To alleviate this problem, we propose a novel&#10;Image-Adaptive YOLO (IA-YOLO) framework, where each image can be adaptively&#10;enhanced for better detection performance. Specifically, a differentiable image&#10;processing (DIP) module is presented to take into account the adverse weather&#10;conditions for YOLO detector, whose parameters are predicted by a small&#10;convolutional neural net-work (CNN-PP). We learn CNN-PP and YOLOv3 jointly in&#10;an end-to-end fashion, which ensures that CNN-PP can learn an appropriate DIP&#10;to enhance the image for detection in a weakly supervised manner. Our proposed&#10;IA-YOLO approach can adaptively process images in both normal and adverse&#10;weather conditions. The experimental results are very encouraging,&#10;demonstrating the effectiveness of our proposed IA-YOLO method in both foggy&#10;and low-light scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.07424" label="1911.07424">
-        <attvalues>
-          <attvalue for="0" value="Fast and Accurate 3D Hand Pose Estimation via Recurrent Neural Network&#10;  for Capturing Hand Articulations" />
-          <attvalue for="1" value="  3D hand pose estimation from a single depth image plays an important role in&#10;computer vision and human-computer interaction. Although recent hand pose&#10;estimation methods using convolution neural network (CNN) have shown notable&#10;improvements in accuracy, most of them have a limitation that they rely on a&#10;complex network structure without fully exploiting the articulated structure of&#10;the hand. A hand, which is an articulated object, is composed of six local&#10;parts: the palm and five independent fingers. Each finger consists of&#10;sequential-joints that provide constrained motion, referred to as a kinematic&#10;chain. In this paper, we propose a hierarchically-structured convolutional&#10;recurrent neural network (HCRNN) with six branches that estimate the 3D&#10;position of the palm and five fingers independently. The palm position is&#10;predicted via fully-connected layers. Each sequential-joint, i.e. finger&#10;position, is obtained using a recurrent neural network (RNN) to capture the&#10;spatial dependencies between adjacent joints. Then the output features of the&#10;palm and finger branches are concatenated to estimate the global hand position.&#10;HCRNN directly takes the depth map as an input without a time-consuming data&#10;conversion, such as 3D voxels and point clouds. Experimental results on public&#10;datasets demonstrate that the proposed HCRNN not only outperforms most 2D&#10;CNN-based methods using the depth image as their inputs but also achieves&#10;competitive results with state-of-the-art 3D CNN-based methods with a highly&#10;efficient running speed of 285 fps on a single GPU.&#10;" />
-          <attvalue for="2" value="&#10;&#10;\Figure[t!](topskip=0pt, botskip=0pt, midskip=0pt)[width=0.99\linewidth]{yoo1.pdf}&#10;{Performance/inference speed trade-off on NYU dataset. The proposed HCRNN has advantages both in estimation accuracy and fast inference speed.}&#10;&#10;\Figure[t!](topskip=0pt, botskip=0pt, midskip=0pt)[width=1.0\linewidth]{yoo2.pdf}&#10;{The overall architecture of the proposed method for 3D hand pose estimation. It is designed with a hierarchical structure, where each separate branch estimates the 3D positions of the palm and five fingers. In the finger branches, the RNN-based regression block is added to capture the spatial dependencies between the sequential-joints of the finger.}&#10;&#10;\PARstart{A}{ccurate} 3D hand pose estimation has received considerable attention regarding a wide range of applications, such as virtual/augmented reality and human-computer interaction~\cite{erol2007vision}. As commercial depth cameras have been released and become more common, depth-based hand pose estimation methods have attracted significant research interest in recent decades~\cite{yuan2018depth}. Nevertheless, it is still a challenging problem to accurately estimate 3D hand pose, because of the low quality of depth images, large variations in hand orientations, high joint flexibility, and severe self-occlusion.&#10;&#10;Recently, most 3D hand pose estimation methods have been based on convolutional neural networks~(CNNs) with a single depth image. In these conventional CNN-based methods, there are two major approaches to improve estimation accuracy. &#10;The first involves 3D data representation of 2D depth images. To utilize 3D spatial information, Ge~et~al. and Moon~et~al. converted a depth image into a volumetric representation, such as 3D voxels, and then applied a 3D CNN for 3D hand pose estimation~\cite{ge20173d, ge2018real, moon2018v2v}. In addition, 3D representation of inputs based on a 3D point cloud has been proposed~\cite{chen2018shpr, ge2018hand, ge2018point}. Although these methods are effective for capturing the geometric properties of depth images~\cite{ge20173d, moon2018v2v}, they suffer from heavy parameters and complex data-conversion processes, resulting in high time complexity. For efficient training and testing, 2D CNN-based methods that attempt to extract more information from 2D inputs are still being widely researched. In this study, we adopt a 2D depth image itself as input without a further data representation process to utilize the efficiency of 2D CNN.&#10;&#10;The second approach involves effective network architecture designs that utilize the structural properties of hands~\cite{du2019crossinfonet, madadi2017end, sun2015cascaded, zhou2018hbe}. For a network model, hierarchical networks that divide the global hand pose estimation problem into sub-tasks have been proposed, where each one focuses on a specific finger or hand region. Sun~et~al.~\cite{sun2015cascaded} divided global hand pose estimation into local estimations of palm and finger pose and then updated the finger location according to the palm pose in a cascade manner. Madadi~et~al.~\cite{madadi2017end} designed a hierarchically tree-like, structured CNN using five branches to model the five fingers and an additional branch to model the palm orientation. Zhou~et~al.~\cite{zhou2018hbe} designed a three-branch network, where the three branches correspond to the thumb, index finger, and the three other fingers, according to the differences in the functional importance of different fingers. More recently, Du~et~al.~\cite{du2019crossinfonet} proposed a two-branch cross-connection network that hierarchically regresses palm and finger poses through information-sharing in a multi-task setup. These studies have demonstrated that handling different parts of the hand via a multi-branch CNN can improve the accuracy of 3D hand pose estimation. However, these methods estimate all joints of the finger directly without explicitly considering finger kinematics. For a finger, the movements of different joints in the finger are dependent on each other and can be represented as a kinematic chain. To better capture spatial dependencies between adjacent joints, in our work, we adopt a recurrent neural network~(RNN), which is mainly used to handle the sequential features of the joints in a finger. &#10;&#10;Fig.~\ref{fig:fig_error_vs_time} depicts the inference time~(ms) versus mean distance error~(mm) graph of some state-of-the-art 3D hand pose estimation methods on NYU dataset.&#10;Note that, in terms of estimation accuracy, the proposed method is ranked in the 4th place behind V2V-PoseNet~\cite{moon2018v2v}, A2J~\cite{xiong2019a2j}, and Point-to-Point~\cite{ge2018point} which have much higher computational complexity than the proposed method.&#10;The proposed HCRNN operates approximately 80, seven, and three times faster than V2V-PoseNet~\cite{moon2018v2v}, Point-to-Point~\cite{ge2018point}, and A2J~\cite{xiong2019a2j}, respectively. &#10;On the other hand, in terms of inference speed, the proposed HCRNN is in the 2nd place behind Feedback~\cite{oberweger2015training}, while HCRNN improves the estimation accuracy of Feedback about 40\%.&#10;In summary, the HCRNN achieves both goals, effectiveness and efficiency, in 3D hand pose estimation. &#10;&#10;Fig.~\ref{fig:framework} illustrates the proposed hierarchical convolutional RNN~(HCRNN), which takes the 3D geometry of a hand into account for 3D hand pose estimation. The six separate branches are based on the observation that the hand is composed of six local parts (i.e. the palm and five fingers) with different amounts of variations due to the articulated structure of the hand.&#10;Inspired by the recent study in which sequential features of a sequence-like object are obtained in a single image~\cite{shi2016end}, we first extract the sequential features of joints by applying the combination of convolutional and fully-connected~(FC) layers to the feature from the encoder network.&#10;Then, we propose to make use of an RNN taking these joint features as inputs to capture the sequential information of a finger and to extract the interdependent information of the finger joints along the kinematic chain.&#10;&#10;Our contributions can be summarized as follows:&#10;\begin{enumerate}&#10;\item We propose the HCRNN architecture that decomposes global hand estimation into sub-tasks of estimating the local parts of the hand. Based on the understanding that the palm and finger exhibit different flexibilities and degrees of freedom, the separate branches estimate the 3D positions of the palm and five fingers. We apply the RNN to utilize spatial information between the sequential-joints of the finger.&#10;\item We design a relatively efficient network that not only achieves promising performance with the mean errors of 6.5, 9.4, and 7.7 on the ICVL~\cite{tang2014latent}, NYU~\cite{tompson2014real}, and MSRA~\cite{sun2015cascaded} datasets, respectively, but also runs fast, at over 285 fps, on a single GPU. The speed versus accuracy trade-off graph can be seen in Fig.~\ref{fig:fig_error_vs_time}.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Human-Computer Interaction, Computer Science, Neural Network Architecture, Hand Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1712.03917" label="1712.03917">
-        <attvalues>
-          <attvalue for="0" value="Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future&#10;  Goals" />
-          <attvalue for="1" value="  In this paper, we strive to answer two questions: What is the current state&#10;of 3D hand pose estimation from depth images? And, what are the next challenges&#10;that need to be tackled? Following the successful Hands In the Million&#10;Challenge (HIM2017), we investigate the top 10 state-of-the-art methods on&#10;three tasks: single frame 3D pose estimation, 3D hand tracking, and hand pose&#10;estimation during object interaction. We analyze the performance of different&#10;CNN structures with regard to hand shape, joint visibility, view point and&#10;articulation distributions. Our findings include: (1) isolated 3D hand pose&#10;estimation achieves low mean errors (10 mm) in the view point range of [70,&#10;120] degrees, but it is far from being solved for extreme view points; (2) 3D&#10;volumetric representations outperform 2D CNNs, better capturing the spatial&#10;structure of the depth data; (3) Discriminative methods still generalize poorly&#10;to unseen hand shapes; (4) While joint occlusions pose a challenge for most&#10;methods, explicit modeling of structure constraints can significantly narrow&#10;the gap between errors on visible and occluded joints.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.09606" label="1705.09606">
-        <attvalues>
-          <attvalue for="0" value="End-to-end Global to Local CNN Learning for Hand Pose Recovery in Depth&#10;  Data" />
-          <attvalue for="1" value="  Despite recent advances in 3D pose estimation of human hands, especially&#10;thanks to the advent of CNNs and depth cameras, this task is still far from&#10;being solved. This is mainly due to the highly non-linear dynamics of fingers,&#10;which make hand model training a challenging task. In this paper, we exploit a&#10;novel hierarchical tree-like structured CNN, in which branches are trained to&#10;become specialized in predefined subsets of hand joints, called local poses. We&#10;further fuse local pose features, extracted from hierarchical CNN branches, to&#10;learn higher order dependencies among joints in the final pose by end-to-end&#10;training. Lastly, the loss function used is also defined to incorporate&#10;appearance and physical constraints about doable hand motion and deformation.&#10;Finally, we introduce a non-rigid data augmentation approach to increase the&#10;amount of training depth data. Experimental results suggest that feeding a&#10;tree-shaped CNN, specialized in local poses, into a fusion network for modeling&#10;joints correlations and dependencies, helps to increase the precision of final&#10;estimations, outperforming state-of-the-art results on NYU and SyntheticHand&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07399" label="1711.07399">
-        <attvalues>
-          <attvalue for="0" value="V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and&#10;  Human Pose Estimation from a Single Depth Map" />
-          <attvalue for="1" value="  Most of the existing deep learning-based methods for 3D hand and human pose&#10;estimation from a single depth map are based on a common framework that takes a&#10;2D depth map and directly regresses the 3D coordinates of keypoints, such as&#10;hand or human body joints, via 2D convolutional neural networks (CNNs). The&#10;first weakness of this approach is the presence of perspective distortion in&#10;the 2D depth map. While the depth map is intrinsically 3D data, many previous&#10;methods treat depth maps as 2D images that can distort the shape of the actual&#10;object through projection from 3D to 2D space. This compels the network to&#10;perform perspective distortion-invariant estimation. The second weakness of the&#10;conventional approach is that directly regressing 3D coordinates from a 2D&#10;image is a highly non-linear mapping, which causes difficulty in the learning&#10;procedure. To overcome these weaknesses, we firstly cast the 3D hand and human&#10;pose estimation problem from a single depth map into a voxel-to-voxel&#10;prediction that uses a 3D voxelized grid and estimates the per-voxel likelihood&#10;for each keypoint. We design our model as a 3D CNN that provides accurate&#10;estimates while running in real-time. Our system outperforms previous methods&#10;in almost all publicly available 3D hand and human pose estimation datasets and&#10;placed first in the HANDS 2017 frame-based 3D hand pose estimation challenge.&#10;The code is available in https://github.com/mks0601/V2V-PoseNet_RELEASE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1609.09698" label="1609.09698">
-        <attvalues>
-          <attvalue for="0" value="Training a Feedback Loop for Hand Pose Estimation" />
-          <attvalue for="1" value="  We propose an entirely data-driven approach to estimating the 3D pose of a&#10;hand given a depth image. We show that we can correct the mistakes made by a&#10;Convolutional Neural Network trained to predict an estimate of the 3D pose by&#10;using a feedback loop. The components of this feedback loop are also Deep&#10;Networks, optimized using training data. They remove the need for fitting a 3D&#10;model to the input data, which requires both a carefully designed fitting&#10;function and algorithm. We show that our approach outperforms state-of-the-art&#10;methods, and is efficient as our implementation runs at over 400 fps on a&#10;single GPU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1507.05717" label="1507.05717">
-        <attvalues>
-          <attvalue for="0" value="An End-to-End Trainable Neural Network for Image-based Sequence&#10;  Recognition and Its Application to Scene Text Recognition" />
-          <attvalue for="1" value="  Image-based sequence recognition has been a long-standing research topic in&#10;computer vision. In this paper, we investigate the problem of scene text&#10;recognition, which is among the most important and challenging tasks in&#10;image-based sequence recognition. A novel neural network architecture, which&#10;integrates feature extraction, sequence modeling and transcription into a&#10;unified framework, is proposed. Compared with previous systems for scene text&#10;recognition, the proposed architecture possesses four distinctive properties:&#10;(1) It is end-to-end trainable, in contrast to most of the existing algorithms&#10;whose components are separately trained and tuned. (2) It naturally handles&#10;sequences in arbitrary lengths, involving no character segmentation or&#10;horizontal scale normalization. (3) It is not confined to any predefined&#10;lexicon and achieves remarkable performances in both lexicon-free and&#10;lexicon-based scene text recognition tasks. (4) It generates an effective yet&#10;much smaller model, which is more practical for real-world application&#10;scenarios. The experiments on standard benchmarks, including the IIIT-5K,&#10;Street View Text and ICDAR datasets, demonstrate the superiority of the&#10;proposed algorithm over the prior arts. Moreover, the proposed algorithm&#10;performs well in the task of image-based music score recognition, which&#10;evidently verifies the generality of it.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.11141" label="2210.11141">
-        <attvalues>
-          <attvalue for="0" value="General Image Descriptors for Open World Image Retrieval using ViT CLIP" />
-          <attvalue for="1" value="  The Google Universal Image Embedding (GUIE) Challenge is one of the first&#10;competitions in multi-domain image representations in the wild, covering a wide&#10;distribution of objects: landmarks, artwork, food, etc. This is a fundamental&#10;computer vision problem with notable applications in image retrieval, search&#10;engines and e-commerce. In this work, we explain our 4th place solution to the&#10;GUIE Challenge, and our &quot;bag of tricks&quot; to fine-tune zero-shot Vision&#10;Transformers (ViT) pre-trained using CLIP.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image representations are a critical building block of computer vision applications~\cite{douze20212021}. Traditionally, research on image embedding learning has been conducted with a focus on per-domain models~\cite{araujo_retrieval, radenovic2018fine, google_landmark}. Generally, solutions are based on generic embedding learning techniques which are applied to different domains separately, rather than developing generic embedding models which could be applied to all domains combined.&#10;&#10;At the Google Universal Image Embedding (GUIE) Challenge, the proposed models are expected to retrieve relevant index database images to a given query image (i.e. images containing the same object as the query) considering a great variety of domains. Our proposed solution has real-world visual search applications, such as organizing photos, improving search engines, and visual e-commerce.&#10;&#10;Problem definition&#10;We seek for a function $\phi$ such that:&#10;&#10;\begin{equation}&#10; \phi: \mathbb{R}^{H\times W\times 3} \mapsto \mathbb{R}^{64} \quad \phi (x) = \mathbf{q} \in \mathbb{R}^{64} &#10;\end{equation}&#10;&#10;given an input 3-channel RGB image $x$ of dimension $H\times W$, our model $\phi$ extract a compact 64-dimensional ($64\mathcal{D}$) image descriptor or embedding $\phi (x)$.&#10;&#10;Then the image retrieval task\cite{cao2020unifying, araujo2020deep, radenovic2018fine} considers an index-reference database of images $Z=\{z_1, z_2, \dots, z_n\}$, and a given a query image $x$, we calculate&#10;\begin{equation}&#10; \underset{Z}{\mathrm{argmin}} \lVert \phi(x) - \phi(z_i) \rVert^{2}_{2}&#10;\end{equation}&#10;finally retrieve the top-$k$ most similar images (i.e. those that minimize the previous equation).&#10;&#10;Evaluation&#10;&#10;Methods are evaluated according to the mean Precision at $k=5$ (abbreviated as $mP@5$):&#10;\begin{equation}&#10; mP@5 = \frac{1}{Q} \sum_{q=1}^{Q} \frac{1}{min(n_q, 5)} \sum_{j=1}^{min(n_q, 5)} rel_q(j)&#10;\end{equation}&#10;&#10;where $Q$ is the number of query images, $n_q$ is the number of index images containing an object in common with the query image $q$. Note that $n_q&gt;0$ for any query image $q$. The term $rel_q(j)$ denotes the relevance of prediction $j$ for the $q$-th query: $rel_q(j)=1$ if the $j$-th prediction is correct, and 0 otherwise.&#10;Participants must submit a model file (e.g..pt). The model must take an image as an input, and return a float vector (i.e. the image embedding) as the output. The challenge platform Kaggle use the submitted model to:&#10;\begin{enumerate}&#10; \item Extract embeddings for the private test dataset (query and index images).&#10; \item Create a kNN ($k = 5$) lookup for each test sample, using the Euclidean distance between test and index embeddings. See Equation \ref{eq:search}.&#10; \item Score the quality of the lookups using Equation \ref{eq:metric}.&#10;\end{enumerate}&#10;&#10;In Figure~\ref{fig:app1} we provide an illustrative example of a real-world image retrieval system, similar to the one employed in this challenge for evaluating the quality of the produced image descriptors.&#10;&#10;Note that the evaluation dataset is kept private. There are $\approx5000$ test query images and $\approx200.000$ index images. &#10;&#10;In Figure \ref{fig:data-dist} we show the data distribution. The dataset images of the following types of object: apparel and accessories, packaged goods, landmarks, furniture and home decoration, storefronts, dishes, artwork, toys, memes, illustrations and cars. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Retrieval" />
-        </attvalues>
-      </node>
-      <node id="2001.05027" label="2001.05027">
-        <attvalues>
-          <attvalue for="0" value="Unifying Deep Local and Global Features for Image Search" />
-          <attvalue for="1" value="  Image retrieval is the problem of searching an image database for items that&#10;are similar to a query image. To address this task, two main types of image&#10;representations have been studied: global and local image features. In this&#10;work, our key contribution is to unify global and local features into a single&#10;deep model, enabling accurate retrieval with efficient feature extraction. We&#10;refer to the new model as DELG, standing for DEep Local and Global features. We&#10;leverage lessons from recent feature learning work and propose a model that&#10;combines generalized mean pooling for global features and attentive selection&#10;for local features. The entire network can be learned end-to-end by carefully&#10;balancing the gradient flow between two heads -- requiring only image-level&#10;labels. We also introduce an autoencoder-based dimensionality reduction&#10;technique for local features, which is integrated into the model, improving&#10;training efficiency and matching performance. Comprehensive experiments show&#10;that our model achieves state-of-the-art image retrieval on the Revisited&#10;Oxford and Paris datasets, and state-of-the-art single-model instance-level&#10;recognition on the Google Landmarks dataset v2. Code and models are available&#10;at https://github.com/tensorflow/models/tree/master/research/delf .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.02172" label="2011.02172">
-        <attvalues>
-          <attvalue for="0" value="Leveraging Temporal Joint Depths for Improving 3D Human Pose Estimation&#10;  in Video" />
-          <attvalue for="1" value="  The effectiveness of the approaches to predict 3D poses from 2D poses&#10;estimated in each frame of a video has been demonstrated for 3D human pose&#10;estimation. However, 2D poses without appearance information of persons have&#10;much ambiguity with respect to the joint depths. In this paper, we propose to&#10;estimate a 3D pose in each frame of a video and refine it considering temporal&#10;information. The proposed approach reduces the ambiguity of the joint depths&#10;and improves the 3D pose estimation accuracy.&#10;" />
-          <attvalue for="2" value="&#10;\ifjapanese&#10;3次元人物姿勢推定は単一または複数視点の画像や動画から人物関節点の3次元座標を特定するタスクであり、AR/VR、人物行動解析、車載カメラ映像解析などの応用先が存在する。本タスクの中でも単眼カメラ画像を入力とした3次元姿勢推定が特に盛んに研究されている。この場合、カメラから人物への奥行きおよび人物の大きさが不定となるため、基本的にはカメラ座標系において人物の腰を原点としたときの各関節点の相対座標を推定する問題設定となる。&#10;&#10;3D human pose estimation aims to localize human joints in a 3D coordinate system. It is important for many applications including AR/VR, human action analysis, and in-vehicle camera video analysis.&#10;&#10;\ifjapanese&#10;このように画像からの関節点の奥行きの推定には不定性が存在することが、タスクの難点の一つである。&#10;また、関節点の3次元座標のアノテーションには一般的に実験室環境で構築されたモーションキャプチャシステムが必要となるため、大規模なデータセットの構築が困難である。背景や人物の服装や姿勢などの外観の多様性が低いデータセットを用いると、学習されたモデルの汎化性能を損なう恐れがあるという課題がある。&#10;&#10;One of the difficulties of this task is that there is ambiguity in the estimation of the depth of the joints from an image.&#10;&#10;\ifjapanese&#10;近年の深層学習の研究の発展に伴い、3次元姿勢推定の精度も大きく向上している。&#10;その中でも、タスクを画像からの2次元姿勢推定と、推定された2次元姿勢からの3次元姿勢推定に分割するアプローチの有効性が確認されている\cite{martinez2017simple}。2次元姿勢推定には多様性の高いデータセットを用いて学習された既製の2次元姿勢推定器を利用できる点、また3次元姿勢推定の際に画像の外観情報を敢えて使用しないことで過学習を軽減している点が本アプローチの有効性に寄与していると考えられる。&#10;また、近年では動画における時系列情報の活用を図る研究が増えつつある。&#10;それらの研究では、フレーム毎に推定された2次元姿勢をモデルの入力とするアプローチが主流となっている\cite{rayat2018exploiting,pavllo20193d,cai2019exploiting}。&#10;&#10;In recent years, there has been an increasing number of studies on the use of temporal information in video. The most common approach in those studies is to utilize 2D poses estimated in each frame to predict final 3D pose \cite{rayat2018exploiting,pavllo20193d,cai2019exploiting}.&#10;&#10;\ifjapanese&#10;しかし、2次元姿勢は画像の外観情報を廃しているため、関節点の奥行きに関して大きな不定性を持っている。Pavlloらの実験において、正解の2次元姿勢を用いて推論を行っても完璧とは程遠い認識精度となったこと\cite{pavllo20193d}からも、2次元姿勢から3次元姿勢を推定するアプローチには精度的限界があると考えられる。&#10;&#10;However, since 2D pose does not include the appearance information, there is large ambiguity with respect to the depths of the joints. The experiments by Pavllo et al. \cite{pavllo20193d} suggest that estimating the 3D pose from the 2D pose is likely to have limited accuracy, even if the ground-truth 2D poses are used.&#10;&#10;\ifjapanese&#10;本稿では動画を入力した3次元姿勢推定において、動画の各フレームに対して3次元姿勢を推定し、それら複数フレームの推定結果を用いて最終的な3次元姿勢を推定するアプローチを提案する。&#10;本手法は2次元姿勢よりも情報量の多い画像からの方が精度良く関節点の奥行きを推定できるという仮定に基づいている。2次元姿勢を中間的に使用する場合とは異なり、関節点の奥行きに関する情報を途中で損なうことのない推論が可能である。&#10;本アプローチにおいて、前段と後段それぞれのモデルには任意のものを使用することができる。我々は、前段モデルには2次元、3次元データセットを併用した学習が可能なIntegral Regression \cite{sun2018integral}を採用し、後段モデルには2次元姿勢からの3次元姿勢推定で有効性が確認されている1次元畳み込みニューラルネットワーク\cite{pavllo20193d}を使用する。&#10;公開データセットを用いた評価実験により提案手法の有効性を示す。&#10;&#10;In this paper, we propose a novel 3D pose estimation approach that first estimates a 3D pose for each frame (first stage) and then aggregates the multi-frame predictions for estimating the final 3D poses (second stage). This method is based on the assumption that the joint depths can be estimated more correctly from an image, which has more information than 2D pose. Unlike using 2D poses intermediately, it is possible to perform inferences that do not lose the joint depth information.&#10;We demonstrate the effectiveness of the proposed method through experiments on a public dataset.&#10;&#10;\ifshowfig&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Human-Computer Interaction, Computer Science, Temporal Refinement, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1711.08229" label="1711.08229">
-        <attvalues>
-          <attvalue for="0" value="Integral Human Pose Regression" />
-          <attvalue for="1" value="  State-of-the-art human pose estimation methods are based on heat map&#10;representation. In spite of the good performance, the representation has a few&#10;issues in nature, such as not differentiable and quantization error. This work&#10;shows that a simple integral operation relates and unifies the heat map&#10;representation and joint regression, thus avoiding the above issues. It is&#10;differentiable, efficient, and compatible with any heat map based methods. Its&#10;effectiveness is convincingly validated via comprehensive ablation experiments&#10;under various settings, specifically on 3D pose estimation, for the first time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1902.01275" label="1902.01275">
-        <attvalues>
-          <attvalue for="0" value="Implicit 3D Orientation Learning for 6D Object Detection from RGB Images" />
-          <attvalue for="1" value="  We propose a real-time RGB-based pipeline for object detection and 6D pose&#10;estimation. Our novel 3D orientation estimation is based on a variant of the&#10;Denoising Autoencoder that is trained on simulated views of a 3D model using&#10;Domain Randomization. This so-called Augmented Autoencoder has several&#10;advantages over existing methods: It does not require real, pose-annotated&#10;training data, generalizes to various test sensors and inherently handles&#10;object and view symmetries. Instead of learning an explicit mapping from input&#10;images to object poses, it provides an implicit representation of object&#10;orientations defined by samples in a latent space. Our pipeline achieves&#10;state-of-the-art performance on the T-LESS dataset both in the RGB and RGB-D&#10;domain. We also evaluate on the LineMOD dataset where we can compete with other&#10;synthetically trained approaches. We further increase performance by correcting&#10;3D orientation estimates to account for perspective errors when the object&#10;deviates from the image center and show extended results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14093" label="2111.14093">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Reordering Sampler with Neurally Guided MAGSAC" />
-          <attvalue for="1" value="  We propose a new sampler for robust estimators that always selects the sample&#10;with the highest probability of consisting only of inliers. After every&#10;unsuccessful iteration, the inlier probabilities are updated in a principled&#10;way via a Bayesian approach. The probabilities obtained by the deep network are&#10;used as prior (so-called neural guidance) inside the sampler. Moreover, we&#10;introduce a new loss that exploits, in a geometrically justifiable manner, the&#10;orientation and scale that can be estimated for any type of feature, e.g., SIFT&#10;or SuperPoint, to estimate two-view geometry. The new loss helps to learn&#10;higher-order information about the underlying scene geometry. Benefiting from&#10;the new sampler and the proposed loss, we combine the neural guidance with the&#10;state-of-the-art MAGSAC++. Adaptive Reordering Sampler with Neurally Guided&#10;MAGSAC (ARS-MAGSAC) is superior to the state-of-the-art in terms of accuracy&#10;and run-time on the PhotoTourism and KITTI datasets for essential and&#10;fundamental matrix estimation. The code and trained models are available at&#10;https://github.com/weitong8591/ars_magsac.&#10;" />
-          <attvalue for="2" value="&#10;Robust estimation of two-view geometry is a fundamental problem in numerous computer vision applications, \eg, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, multi-model fitting~\cite{isack2012energy,pham2014interacting,barath2018multi}, initial pose recovery of Structure-from-Motion~\cite{schoenberger2016sfm,schoenberger2016mvs} and Simultaneous Localization and Mapping (SLAM)~\cite{mur2015orb} pipelines. &#10;RANSAC (RANdom SAmple Consensus)~\cite{RANSAC}, and its recent variants~\cite{raguram2013usac,barath2019magsac,barath2020magsac++,ivashechkin2021vsac,barath2022learning} have been widely applied in practice due to efficiency, simplicity, and accuracy that makes them appealing in real-world scenarios. &#10;In brief, RANSAC works in iterations by selecting a subset of data points, estimating the model (\eg, relative pose of a camera pair), and measuring its quality as the number of points consistent with it (\ie, its inliers). &#10;&#10;Since the publication of RANSAC, several modifications have been proposed to improve its accuracy and speed, focusing on specific components of the original algorithm. &#10;One common way to increase the accuracy is considering realistic noise distributions in model scoring, rather than relying on inlier counting that essentially assumes uniform noise. &#10;MLESAC~\cite{torr2000mlesac} uses a maximum likelihood procedure to reason about the quality of each minimal sample model. &#10;While MLESAC can achieve better accuracy than standard inlier counting, it can be computationally expensive compared to the original RANSAC.&#10;MSAC~\cite{torr2002bayesian} proposes the use of truncated $L_2$ loss, which has been shown to be equivalent to the maximum likelihood-based approaches, but it still heavily relies on a manually set threshold. &#10;More recently, MAGSAC~\cite{barath2019magsac} and MAGSAC++~\cite{barath2020magsac++} have been proposed to alleviate the dependence on manual threshold selection by marginalizing over an acceptable range of noise scales.&#10;As shown in the recent survey~\cite{ma2021image}, MAGSAC++ is currently the most accurate RANSAC variant as in \cite{ivashechkin2021vsac}. &#10;&#10;Improving the sampling procedure is another way to enhance the performance of RANSAC by selecting a good sample early and triggering the termination criterion.&#10;Several samplers have been proposed, each with its own assumptions and limitations. &#10;The NAPSAC~\cite{torr2002napsac} sampler assumes that inliers are spatially coherent and, thus, it draws samples from a hyper-sphere centered at the first, randomly selected, location-defining point. &#10;The GroupSAC algorithm~\cite{ni2009groupsac} assumes that inliers are often ``similar'' and, thus, can be separated into groups. &#10;PROSAC~\cite{chum2005matching} exploits an a priori predicted inlier probability rank of each point and starts the sampling with the most promising ones.&#10;Progressively, samples that are less likely to lead to the sought model are drawn.&#10;More recently, \cite{brachmann2019neural} proposes a sampler to exploit prior knowledge of inlier probabilities, \eg, from a deep network. &#10;The sampler selects minimal samples according to the probability, assuming that it follows a categorical distribution over the discrete set of observations.&#10;&#10;Recently, several algorithms have been proposed for robust relative pose estimation using neural networks. &#10;The first paper on the topic, Context Normalization Networks (PointCN)~\cite{cne2018} proposes the use of PointNet (MLP) with batch normalization as a context mechanism. &#10;Attentive Context Normalization Networks~\cite{acne2020} introduces a special architectural block for the task. &#10;The Deep Fundamental Matrix Estimation (DFE)~\cite{dfe2018} uses differentiable iteratively re-weighted least-squares with predicted weights.&#10;The OANet algorithm~\cite{oanet2019} introduces differentiable pooling and unpooling blocks for correspondence filtering.&#10;Neural Guided RANSAC (NG-RANSAC)~\cite{brachmann2019neural} uses a PointCN-like architecture with a different training objective, and the predicted correspondence scores are exploited inside RANSAC by using a guided sampling method that helps to find accurate models early.&#10;More recently, CLNet~\cite{clnet2021} introduces several algorithmic and architectural improvements to remove gross outliers with iterative pruning.&#10;These techniques provide alternatives for tentative correspondence pre-filtering and weighting. While these methods have shown promising results, they do not substitute standard robust estimation, as demonstrated in the RANSAC tutorial~\cite{cvpr2020ransactutorial}.&#10;&#10;This paper makes three main contributions. &#10;First, we propose a new sampler tailored for neurally guided robust estimators always selecting the sample with the highest probability of consisting only of inliers.&#10;It exploits the inlier probabilities obtained by deep networks as prior knowledge and updates them in a principled way via a Bayesian approach.&#10;Second, we propose a new loss that incorporates the orientation and scale from off-the-shelf feature detectors, \eg, SIFT~\cite{lowe2004distinctive}, in a geometrically meaningful manner directly into the training. &#10;Additionally, we show that this extra information can be obtained for any features, \eg SuperPoint~\cite{detone2018superpoint}, allowing to seamlessly integrate the proposed loss with recent detectors.&#10;The loss allows learning higher-order information about the underlying scene geometry and improves the robustness of the estimation in challenging environments.&#10;Third, as a technical contribution, we combine two state-of-the-art algorithms, \ie, MAGSAC++~\cite{barath2020magsac++} and NG-RANSAC~\cite{brachmann2019neural}, to improve the relative pose estimation accuracy on a wide range of scenes. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Robust Estimation, Mathematics, Deep Learning, Computer Science, Statistics" />
-        </attvalues>
-      </node>
-      <node id="1706.00827" label="1706.00827">
-        <attvalues>
-          <attvalue for="0" value="Multi-Class Model Fitting by Energy Minimization and Mode-Seeking" />
-          <attvalue for="1" value="  We propose a general formulation, called Multi-X, for multi-class&#10;multi-instance model fitting - the problem of interpreting the input data as a&#10;mixture of noisy observations originating from multiple instances of multiple&#10;classes. We extend the commonly used alpha-expansion-based technique with a new&#10;move in the label space. The move replaces a set of labels with the&#10;corresponding density mode in the model parameter domain, thus achieving fast&#10;and robust optimization. Key optimization parameters like the bandwidth of the&#10;mode seeking are set automatically within the algorithm. Considering that a&#10;group of outliers may form spatially coherent structures in the data, we&#10;propose a cross-validation-based technique removing statistically insignificant&#10;instances. Multi-X outperforms significantly the state-of-the-art on publicly&#10;available datasets for diverse problems: multiple plane and rigid motion&#10;detection; motion segmentation; simultaneous plane and cylinder fitting; circle&#10;and line fitting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.04132" label="1905.04132">
-        <attvalues>
-          <attvalue for="0" value="Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses" />
-          <attvalue for="1" value="  We present Neural-Guided RANSAC (NG-RANSAC), an extension to the classic&#10;RANSAC algorithm from robust optimization. NG-RANSAC uses prior information to&#10;improve model hypothesis search, increasing the chance of finding outlier-free&#10;minimal sets. Previous works use heuristic side-information like hand-crafted&#10;descriptor distance to guide hypothesis search. In contrast, we learn&#10;hypothesis search in a principled fashion that lets us optimize an arbitrary&#10;task loss during training, leading to large improvements on classic computer&#10;vision tasks. We present two further extensions to NG-RANSAC. Firstly, using&#10;the inlier count itself as training signal allows us to train neural guidance&#10;in a self-supervised fashion. Secondly, we combine neural guidance with&#10;differentiable RANSAC to build neural networks which focus on certain parts of&#10;the input data and make the output predictions as good as possible. We evaluate&#10;NG-RANSAC on a wide array of computer vision tasks, namely estimation of&#10;epipolar geometry, horizon line estimation and camera re-localization. We&#10;achieve superior or competitive results compared to state-of-the-art robust&#10;estimators, including very recent, learned ones.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.05971" label="1711.05971">
-        <attvalues>
-          <attvalue for="0" value="Learning to Find Good Correspondences" />
-          <attvalue for="1" value="  We develop a deep architecture to learn to find good correspondences for&#10;wide-baseline stereo. Given a set of putative sparse matches and the camera&#10;intrinsics, we train our network in an end-to-end fashion to label the&#10;correspondences as inliers or outliers, while simultaneously using them to&#10;recover the relative pose, as encoded by the essential matrix. Our architecture&#10;is based on a multi-layer perceptron operating on pixel coordinates rather than&#10;directly on the image, and is thus simple and small. We introduce a novel&#10;normalization technique, called Context Normalization, which allows us to&#10;process each data point separately while imbuing it with global information,&#10;and also makes the network invariant to the order of the correspondences. Our&#10;experiments on multiple challenging datasets demonstrate that our method is&#10;able to drastically improve the state of the art with little training data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.04964" label="1908.04964">
-        <attvalues>
-          <attvalue for="0" value="Learning Two-View Correspondences and Geometry Using Order-Aware Network" />
-          <attvalue for="1" value="  Establishing correspondences between two images requires both local and&#10;global spatial context. Given putative correspondences of feature points in two&#10;views, in this paper, we propose Order-Aware Network, which infers the&#10;probabilities of correspondences being inliers and regresses the relative pose&#10;encoded by the essential matrix. Specifically, this proposed network is built&#10;hierarchically and comprises three novel operations. First, to capture the&#10;local context of sparse correspondences, the network clusters unordered input&#10;correspondences by learning a soft assignment matrix. These clusters are in a&#10;canonical order and invariant to input permutations. Next, the clusters are&#10;spatially correlated to form the global context of correspondences. After that,&#10;the context-encoded clusters are recovered back to the original size through a&#10;proposed upsampling operator. We intensively experiment on both outdoor and&#10;indoor datasets. The accuracy of the two-view geometry and correspondences are&#10;significantly improved over the state-of-the-arts. Code will be available at&#10;https://github.com/zjhthu/OANet.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.00591" label="2101.00591">
-        <attvalues>
-          <attvalue for="0" value="Progressive Correspondence Pruning by Consensus Learning" />
-          <attvalue for="1" value="  Correspondence selection aims to correctly select the consistent matches&#10;(inliers) from an initial set of putative correspondences. The selection is&#10;challenging since putative matches are typically extremely unbalanced, largely&#10;dominated by outliers, and the random distribution of such outliers further&#10;complicates the learning process for learning-based methods. To address this&#10;issue, we propose to progressively prune the correspondences via a&#10;local-to-global consensus learning procedure. We introduce a ``pruning'' block&#10;that lets us identify reliable candidates among the initial matches according&#10;to consensus scores estimated using local-to-global dynamic graphs. We then&#10;achieve progressive pruning by stacking multiple pruning blocks sequentially.&#10;Our method outperforms state-of-the-arts on robust line fitting, camera pose&#10;estimation and retrieval-based image localization benchmarks by significant&#10;margins and shows promising generalization ability to different datasets and&#10;detector/descriptor combinations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.03041" label="2203.03041">
-        <attvalues>
-          <attvalue for="0" value="Highly Accurate Dichotomous Image Segmentation" />
-          <attvalue for="1" value="  We present a systematic study on a new task called dichotomous image&#10;segmentation (DIS) , which aims to segment highly accurate objects from natural&#10;images. To this end, we collected the first large-scale DIS dataset, called&#10;DIS5K, which contains 5,470 high-resolution (e.g., 2K, 4K or larger) images&#10;covering camouflaged, salient, or meticulous objects in various backgrounds.&#10;DIS is annotated with extremely fine-grained labels. Besides, we introduce a&#10;simple intermediate supervision baseline (IS-Net) using both feature-level and&#10;mask-level guidance for DIS model training. IS-Net outperforms various&#10;cutting-edge baselines on the proposed DIS5K, making it a general self-learned&#10;supervision network that can facilitate future research in DIS. Further, we&#10;design a new metric called human correction efforts (HCE) which approximates&#10;the number of mouse clicking operations required to correct the false positives&#10;and false negatives. HCE is utilized to measure the gap between models and&#10;real-world applications and thus can complement existing metrics. Finally, we&#10;conduct the largest-scale benchmark, evaluating 16 representative segmentation&#10;models, providing a more insightful discussion regarding object complexities,&#10;and showing several potential applications (e.g., background removal, art&#10;design, 3D reconstruction). Hoping these efforts can open up promising&#10;directions for both academic and industries. Project page:&#10;https://xuebinqin.github.io/dis/index.html.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;Tasks and Datasets of image segmentation are closely related in deep learning era. Some of the segmentation tasks like~\cite{DBLP:journals/pami/ChengMHTH15,wang2017learning,fan2021concealed,liew2021deep,yang2020meticulous,shen2016automatic,DBLP:journals/corr/abs-2108-11515,MnihThesis}, are even directly built upon the datasets. Their problem formulations are exactly the same: $P = F(\theta,I)$, where $I$ and $P$ are the input image and the binary map output, respectively. &#10;However, the relevance between most of these tasks are rarely studied, which somehow restricts the models trained for certain tasks from being generalized to wider applications. Besides, the datasets used in different tasks are not exclusive, &#10;which shows a unified task for dichotomous image segmentation (DIS) is possible. &#10;However, most of the existing datasets are built on low-resolution images with objects of simple structures. There lacks an dataset built on the accurately labeled high-resolution images which contain objects with diversified shape complexities from different categories. &#10;&#10;Models are often struggling with the conflicts between stronger representative capabilities and higher risks of over-fitting. To obtain more representative features, FCN-based models \cite{long2015fully}, Encoder-Decoder \cite{badrinarayanan2017segnet,ronneberger2015u}, Coarse-to-Fine \cite{wang2018detect}, Predict-Refine \cite{qin2019basnet,HRRN_ICCV2021}, Vision Transformer \cite{zheng2020rethinking} and so on are developed. Besides, many real-time models are designed~\cite{zhao2018icnet,yu2018bisenet,li2019dfanet,orsic2019defense,hu2020temporally,Fan_2021_CVPR,nirkin2020hyperseg} to balance the performance and the time costs. &#10;Other methods, such as weights regularization \cite{Goodfellow-et-al-2016}, dropout \cite{DBLP:journals/jmlr/SrivastavaHKSS14}, dense supervision \cite{lee2015deeply,qin2020u2,xie2015holistically}, and hybrid loss \cite{luc2016semantic,qin2019basnet,zhao2019egnet}, focus on alleviating the over-fitting. Dense supervision is one of the most effective ways for reducing the over-fitting. However, supervising the side outputs from the intermediate deep features may not be the best option because the supervision is weakened by the conversion from deep features (multi-channel) to side outputs (one-channel). &#10;&#10;Evaluation Metrics can be categorized as region-based (\eg, IoU or Jaccard index \cite{IoUJaccard}, F-measure \cite{DBLP:conf/muc/Chinchor92,Rijsbergen1979} or Dice's coefficient \cite{Thorvald1948}, weighted F-measure \cite{Margolin2014HowTE}), boundary-based (\eg, CM \cite{movahedi2010design}, boundary F-measure~\cite{DBLP:journals/pami/MartinFM04,perazzi2016benchmark,ehrig2005relaxed,mnih2010learning,saito2016multiple,zhang2018road,qin2019basnet}, boundary IoU \cite{cheng2021boundary}, boundary displacement error (BDE) \cite{DBLP:conf/eccv/FreixenetMRMC02}, Hausdorff distances \cite{Hausdorff1914,Henry1920,DBLP:conf/ifip7/BirsanT05}), structure-based (\eg, S-measure \cite{fan2017structure}, E-measure \cite{fan2018enhanced,fan2021cognitive}), confidence-based (\eg, MAE \cite{perazzi2012saliency}), \etc. They mainly measure the consistencies between the predictions and the ground truth from mathematical or cognitive perspectives. But the costs of synchronizing the predictions against the requirements in real-world applications are not well studied. &#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2307.15252" label="2307.15252">
-        <attvalues>
-          <attvalue for="0" value="A Solution to Co-occurrence Bias: Attributes Disentanglement via Mutual&#10;  Information Minimization for Pedestrian Attribute Recognition" />
-          <attvalue for="1" value="  Recent studies on pedestrian attribute recognition progress with either&#10;explicit or implicit modeling of the co-occurrence among attributes.&#10;Considering that this known a prior is highly variable and unforeseeable&#10;regarding the specific scenarios, we show that current methods can actually&#10;suffer in generalizing such fitted attributes interdependencies onto scenes or&#10;identities off the dataset distribution, resulting in the underlined bias of&#10;attributes co-occurrence. To render models robust in realistic scenes, we&#10;propose the attributes-disentangled feature learning to ensure the recognition&#10;of an attribute not inferring on the existence of others, and which is&#10;sequentially formulated as a problem of mutual information minimization.&#10;Rooting from it, practical strategies are devised to efficiently decouple&#10;attributes, which substantially improve the baseline and establish&#10;state-of-the-art performance on realistic datasets like PETAzs and RAPzs. Code&#10;is released on&#10;https://github.com/SDret/A-Solution-to-Co-occurence-Bias-in-Pedestrian-Attribute-Recognition.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pedestrian attribute recognition (PAR), as a key component of the pedestrian analysis stemming from development of the ubiquitous video surveillance, targets to determine the soft-biometrics of local or semantic attributes of a person given its captured main-body image. To date, researches investigate PAR basically along the analogous routine of discriminative deep multi-label classification. They can be basically abstracted as emphasizing on better attribute localization to mitigate the accuracy drop from inferring on irrelevant area~\cite{liu2018localization,DBLP:conf/aaai/JiaGHCH22}, or involving additional supervision or information, like pose keypoints~\cite{liu2018localization} and pedestrian video clip~\cite{chen2019temporal,2020Pedestrian}, to guide PAR under explicit assumptions regarding body topological structure or temporal context, etc..&#10;&#10;For work striving to enhance the attribute localization, ~\cite{fabbri2017generative,2017Learning} split the body image vertically, by a manually defined fixed strategy, into three parts and feeds each part into an individual network for feature extracting. As a further step,~\cite{liu2017hydraplus} proposed the HydraPlus-Net, learning to locate attributes of different scale with a multi-directional attention modules. Instead of learning to generate the attention map,~\cite{liu2018localization} produces attribute-specific features by means of the class activation map (CAM)~\cite{zhou2016learning}, which is initially designed to build a generic localizable deep representation. However, same technique is applied in~\cite{jia2021rethinking} to demonstrate that, even without explicitly modeling the attribute-specific area, network is still able to locate attributes precisely, implying that the fundamental task for PAR should be instead to explore better feature learning.&#10;&#10;Seeking for better feature learning of information exchanges, a thread of literature~\cite{2016Video,wang2017discovering,2017Multi,2020Correlation,2022Label2Label} aims to enhance the modeling of interdependencies among attributes, under the hypothesis that such correlations provide a contextual constraint complementary to visual attributes recognition. To this aim, Graph-based methods are often used to explicitly model the attributes co-occurrence and estimate the joint label probability. However, the strong variability and unpredictability exhibited by attributes co-occurrence actually cast doubts on the robustness of these methods for scenarios out of the datasets, which are typically in practical applications. Worse, given the observation in Figure.\ref{heatmap} that, such unreliable interdependency can be memorized by network, in a form of bias, even without explicit modeling of it, current methods would suffer in generalizing well onto realistic PAR, as evidenced in~\cite{jia2021rethinking}. &#10;&#10;Equipped with such perspective, this paper evolves in a novel spirit that we resort to infer attributes while entirely discarding their relations. Actually, it is a mechanism of PAR that accords with us human, say, we do not infer one's hat color by referring to even a single clue from its gender, instead, we look just at the hat for robustness, so should intelligent models. To embody this philosophy in stark contrast to the fragile mechanism of deep models for PAR, our attribute-disentangled learning for PAR is formalized as that the attributes specific feature learned for predicting one attribute should not use information pertinent to other attributes, i.e., the mutual information between one attribute and other attributes' specific feature should be minimized. &#10;&#10;To cope with the non-triviality of a direct mutual information minimization, an equivalent optimization-friendly training objective of it is deduced, which guarantees that the variation of other attributes' information results in no shift on the estimated posterior of a given attribute. Both mathematical insights and experimental evidence are provided, indicating that under this setting of the posterior-invariant learning, the proposed disentangled attribute learning can be attained, and thus tackle the issue of attribute co-occurrence bias. &#10;&#10;Sequentially, we also propose an efficient training strategy for the posterior-invariant learning, which enables it implemented in a manner of fast convergence. Practically, unlike most work in this field building on large networks that deteriorates the applicability, our method's plug-and-play nature makes it cater to various models, with almost no extra computational burden thereon.&#10;Albeit lightweight, our method establishes the-state-of-art performance on various realistic datasets like the PETAzs and RAPzs~\cite{jia2021rethinking}, with considerable margins over previous approaches. &#10;&#10;Our contribution is summarized as three folds: &#10;&#10;\begin{itemize} &#10;\item With solid experimental evidences, we establish a novel perspective for understanding the learned attributes interdependency bias as the current bottleneck of PAR from achieving robustness. It is not covered by existing work and can serve as a mindset of future researches for rethinking PAR.&#10;\item We propose one direction of improvement as to infer the attributes by disregarding their correlations. From it, a lightweight prescription of information-theoretic attributes-disentangled feature learning is developed.&#10;\item Along with ablation studies, we present analytical experiments on various realistic PAR datasets to demonstrate our generic proposal's efficacy and a spectrum of superiorities, validating that the proposed method might serve as a foothold of robust PAR. &#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Co-occurrence Modeling, Mathematics, Artificial Intelligence, Pedestrian Attribute Recognition, Robust Feature Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1808.09102" label="1808.09102">
-        <attvalues>
-          <attvalue for="0" value="Localization Guided Learning for Pedestrian Attribute Recognition" />
-          <attvalue for="1" value="  Pedestrian attribute recognition has attracted many attentions due to its&#10;wide applications in scene understanding and person analysis from surveillance&#10;videos. Existing methods try to use additional pose, part or viewpoint&#10;information to complement the global feature representation for attribute&#10;classification. However, these methods face difficulties in localizing the&#10;areas corresponding to different attributes. To address this problem, we&#10;propose a novel Localization Guided Network which assigns attribute-specific&#10;weights to local features based on the affinity between proposals pre-extracted&#10;proposals and attribute locations. The advantage of our model is that our local&#10;features are learned automatically for each attribute and emphasized by the&#10;interaction with global features. We demonstrate the effectiveness of our&#10;Localization Guided Network on two pedestrian attribute benchmarks (PA-100K and&#10;RAP). Our result surpasses the previous state-of-the-art in all five metrics on&#10;both datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.05742" label="1901.05742">
-        <attvalues>
-          <attvalue for="0" value="A Temporal Attentive Approach for Video-Based Pedestrian Attribute&#10;  Recognition" />
-          <attvalue for="1" value="  In this paper, we first tackle the problem of pedestrian attribute&#10;recognition by video-based approach. The challenge mainly lies in spatial and&#10;temporal modeling and how to integrating them for effective and dynamic&#10;pedestrian representation. To solve this problem, a novel multi-task model&#10;based on the conventional neural network and temporal attention strategy is&#10;proposed. Since publicly available dataset is rare, two new large-scale video&#10;datasets with expanded attribute definition are presented, on which the&#10;effectiveness of both video-based pedestrian attribute recognition methods and&#10;the proposed new network architecture is well demonstrated. The two datasets&#10;are published on http://irip.buaa.edu.cn/mars_duke_attributes/index.html.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.02240" label="1707.02240">
-        <attvalues>
-          <attvalue for="0" value="Generative Adversarial Models for People Attribute Recognition in&#10;  Surveillance" />
-          <attvalue for="1" value="  In this paper we propose a deep architecture for detecting people attributes&#10;(e.g. gender, race, clothing ...) in surveillance contexts. Our proposal&#10;explicitly deal with poor resolution and occlusion issues that often occur in&#10;surveillance footages by enhancing the images by means of Deep Convolutional&#10;Generative Adversarial Networks (DCGAN). Experiments show that by combining&#10;both our Generative Reconstruction and Deep Attribute Classification Network we&#10;can effectively extract attributes even when resolution is poor and in presence&#10;of strong occlusions up to 80\% of the whole person figure.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.06555" label="1710.06555">
-        <attvalues>
-          <attvalue for="0" value="Learning Deep Context-aware Features over Body and Latent Parts for&#10;  Person Re-identification" />
-          <attvalue for="1" value="  Person Re-identification (ReID) is to identify the same person across&#10;different cameras. It is a challenging task due to the large variations in&#10;person pose, occlusion, background clutter, etc How to extract powerful&#10;features is a fundamental problem in ReID and is still an open problem today.&#10;In this paper, we design a Multi-Scale Context-Aware Network (MSCAN) to learn&#10;powerful features over full body and body parts, which can well capture the&#10;local context knowledge by stacking multi-scale convolutions in each layer.&#10;Moreover, instead of using predefined rigid parts, we propose to learn and&#10;localize deformable pedestrian parts using Spatial Transformer Networks (STN)&#10;with novel spatial constraints. The learned body parts can release some&#10;difficulties, eg pose variations and background clutters, in part-based&#10;representation. Finally, we integrate the representation learning processes of&#10;full body and body parts into a unified framework for person ReID through&#10;multi-class person identification tasks. Extensive evaluations on current&#10;challenging large-scale person ReID datasets, including the image-based&#10;Market1501, CUHK03 and sequence-based MARS datasets, show that the proposed&#10;method achieves the state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.09930" label="1709.09930">
-        <attvalues>
-          <attvalue for="0" value="HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis" />
-          <attvalue for="1" value="  Pedestrian analysis plays a vital role in intelligent video surveillance and&#10;is a key component for security-centric computer vision systems. Despite that&#10;the convolutional neural networks are remarkable in learning discriminative&#10;features from images, the learning of comprehensive features of pedestrians for&#10;fine-grained tasks remains an open problem. In this study, we propose a new&#10;attention-based deep neural network, named as HydraPlus-Net (HP-net), that&#10;multi-directionally feeds the multi-level attention maps to different feature&#10;layers. The attentive deep features learned from the proposed HP-net bring&#10;unique advantages: (1) the model is capable of capturing multiple attentions&#10;from low-level to semantic-level, and (2) it explores the multi-scale&#10;selectiveness of attentive features to enrich the final feature representations&#10;for a pedestrian image. We demonstrate the effectiveness and generality of the&#10;proposed HP-net for pedestrian analysis on two tasks, i.e. pedestrian attribute&#10;recognition and person re-identification. Intensive experimental results have&#10;been provided to prove that the HP-net outperforms the state-of-the-art methods&#10;on various datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.03576" label="2107.03576">
-        <attvalues>
-          <attvalue for="0" value="Rethinking of Pedestrian Attribute Recognition: A Reliable Evaluation&#10;  under Zero-Shot Pedestrian Identity Setting" />
-          <attvalue for="1" value="  Pedestrian attribute recognition aims to assign multiple attributes to one&#10;pedestrian image captured by a video surveillance camera. Although numerous&#10;methods are proposed and make tremendous progress, we argue that it is time to&#10;step back and analyze the status quo of the area. We review and rethink the&#10;recent progress from three perspectives. First, given that there is no explicit&#10;and complete definition of pedestrian attribute recognition, we formally define&#10;and distinguish pedestrian attribute recognition from other similar tasks.&#10;Second, based on the proposed definition, we expose the limitations of the&#10;existing datasets, which violate the academic norm and are inconsistent with&#10;the essential requirement of practical industry application. Thus, we propose&#10;two datasets, PETA\textsubscript{$ZS$} and RAP\textsubscript{$ZS$}, constructed&#10;following the zero-shot settings on pedestrian identity. In addition, we also&#10;introduce several realistic criteria for future pedestrian attribute dataset&#10;construction. Finally, we reimplement existing state-of-the-art methods and&#10;introduce a strong baseline method to give reliable evaluations and fair&#10;comparisons. Experiments are conducted on four existing datasets and two&#10;proposed datasets to measure progress on pedestrian attribute recognition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.10659" label="1705.10659">
-        <attvalues>
-          <attvalue for="0" value="Discovering Visual Concept Structure with Sparse and Incomplete Tags" />
-          <attvalue for="1" value="  Discovering automatically the semantic structure of tagged visual data (e.g.&#10;web videos and images) is important for visual data analysis and&#10;interpretation, enabling the machine intelligence for effectively processing&#10;the fast-growing amount of multi-media data. However, this is non-trivial due&#10;to the need for jointly learning underlying correlations between heterogeneous&#10;visual and tag data. The task is made more challenging by inherently sparse and&#10;incomplete tags. In this work, we develop a method for modelling the inherent&#10;visual data concept structures based on a novel Hierarchical-Multi-Label Random&#10;Forest model capable of correlating structured visual and tag information so as&#10;to more accurately interpret the visual semantics, e.g. disclosing meaningful&#10;visual groups with similar high-level concepts, and recovering missing tags for&#10;individual visual data samples. Specifically, our model exploits hierarchically&#10;structured tags of different semantic abstractness and multiple tag statistical&#10;correlations in addition to modelling visual and tag interactions. As a result,&#10;our model is able to discover more accurate semantic correlation between&#10;textual tags and visual features, and finally providing favourable visual&#10;semantics interpretation even with highly sparse and incomplete tags. We&#10;demonstrate the advantages of our proposed approach in two fundamental&#10;applications, visual data clustering and missing tag completion, on&#10;benchmarking video (i.e. TRECVID MED 2011) and image (i.e. NUS-WIDE) datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.12668" label="2212.12668">
-        <attvalues>
-          <attvalue for="0" value="Differentiable Rendering for Pose Estimation in Proximity Operations" />
-          <attvalue for="1" value="  Differentiable rendering aims to compute the derivative of the image&#10;rendering function with respect to the rendering parameters. This paper&#10;presents a novel algorithm for 6-DoF pose estimation through gradient-based&#10;optimization using a differentiable rendering pipeline. We emphasize two key&#10;contributions: (1) instead of solving the conventional 2D to 3D correspondence&#10;problem and computing reprojection errors, images (rendered using the 3D model)&#10;are compared only in the 2D feature space via sparse 2D feature&#10;correspondences. (2) Instead of an analytical image formation model, we compute&#10;an approximate local gradient of the rendering process through online learning.&#10;The learning data consists of image features extracted from multi-viewpoint&#10;renders at small perturbations in the pose neighborhood. The gradients are&#10;propagated through the rendering pipeline for the 6-DoF pose estimation using&#10;nonlinear least squares. This gradient-based optimization regresses directly&#10;upon the pose parameters by aligning the 3D model to reproduce a reference&#10;image shape. Using representative experiments, we demonstrate the application&#10;of our approach to pose estimation in proximity operations.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Space-borne navigation relies on accurate estimation of relative position and attitude, known as relative pose, between a target and a chaser spacecraft. Advances in computer vision have propelled both the utility and ubiquity of vision-based relative navigation, using visual sensors to estimate relative pose \cite{du2009pose,cropp2000estimating,zhang2013vision}. Vision-based pose estimation is widely implemented in positioning applications such as autonomous rendezvous and docking \cite{6978896}, proximity operations \cite{junkins1999vision}, and terrain relative navigation \cite{verras2021vision, bhaskara2022fpga}. In cooperative proximity operations, the computation of pose using image feature coordinates is classically achieved by solving Perspective-n-Point (PnP) problem \cite{fischler1981random}. The geometric configuration of the features of the target object is usually known, either from deployed fiducial markers or from transmission of state information \cite{opromolla2017review, sung2020interferometric}.&#10;&#10;However, the estimation of the relative pose between an uncooperative target and a chaser is a more involved process. Without assistance from the target, the chaser must lead the efforts in feature acquisition and tracking for correspondence. Also, because 2D images lack depth information, pose cannot be determined using a single image. Monocular or stereo vision pipelines use at least two frames to first estimate the 3D locations of the feature points and then determine the relative pose \cite{eapen2022narpa}. However, the estimation of 3D feature coordinates is associated with its own errors and therefore could potentially corrupt the pose estimation accuracy. If the geometric model of a target is known in advance or built online \cite{wong2018photometric}, the pose can be determined by mapping the feature points to the 3D locations on the target model \cite{sharma2018robust}. Otherwise, in a model-free approach, sparse 3D point clouds of a target can be constructed using Structure from Motion (SfM) \cite{capuano2020monocular}. &#10;&#10;Traditionally, in the model-based relative navigation approach, pose is determined through the PnP algorithm after establishing correspondences between the 2D feature points and the 3D model points. This approach has two major drawbacks:&#10;\begin{enumerate}&#10; \item Robustness. Geometric methods, PnP or POSIT \cite{dementhon1995model}, solve the estimation problem as a function of pose only. However, rapid changes in illumination conditions could adversely affect the solution of feature correspondence and pose estimation problems. &#10; \item Accuracy limitations. Establishing 2D-3D correspondence as an initial step adds to the errors in pose estimation. Numerical approximations and pixel interpolations limit the precision of pose estimates. &#10;\end{enumerate}&#10;&#10;Mitigating these drawbacks, we build on our previous work \cite{eapen2022narpa} and propose a differentiable rendering pipeline for pose estimation. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Pose Estimation, Differentiable Rendering, Gradient-Based Optimization, Mathematics, Physics, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2211.01566" label="2211.01566">
-        <attvalues>
-          <attvalue for="0" value="NaRPA: Navigation and Rendering Pipeline for Astronautics" />
-          <attvalue for="1" value="  This paper presents Navigation and Rendering Pipeline for Astronautics&#10;(NaRPA) - a novel ray-tracing-based computer graphics engine to model and&#10;simulate light transport for space-borne imaging. NaRPA incorporates lighting&#10;models with attention to atmospheric and shading effects for the synthesis of&#10;space-to-space and ground-to-space virtual observations. In addition to image&#10;rendering, the engine also possesses point cloud, depth, and contour map&#10;generation capabilities to simulate passive and active vision-based sensors and&#10;to facilitate the designing, testing, or verification of visual navigation&#10;algorithms. Physically based rendering capabilities of NaRPA and the efficacy&#10;of the proposed rendering algorithm are demonstrated using applications in&#10;representative space-based environments. A key demonstration includes NaRPA as&#10;a tool for generating stereo imagery and application in 3D coordinate&#10;estimation using triangulation. Another prominent application of NaRPA includes&#10;a novel differentiable rendering approach for image-based attitude estimation&#10;is proposed to highlight the efficacy of the NaRPA engine for simulating&#10;vision-based navigation and guidance operations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.10674" label="2012.10674">
-        <attvalues>
-          <attvalue for="0" value="Camera-aware Proxies for Unsupervised Person Re-Identification" />
-          <attvalue for="1" value="  This paper tackles the purely unsupervised person re-identification (Re-ID)&#10;problem that requires no annotations. Some previous methods adopt clustering&#10;techniques to generate pseudo labels and use the produced labels to train Re-ID&#10;models progressively. These methods are relatively simple but effective.&#10;However, most clustering-based methods take each cluster as a pseudo identity&#10;class, neglecting the large intra-ID variance caused mainly by the change of&#10;camera views. To address this issue, we propose to split each single cluster&#10;into multiple proxies and each proxy represents the instances coming from the&#10;same camera. These camera-aware proxies enable us to deal with large intra-ID&#10;variance and generate more reliable pseudo labels for learning. Based on the&#10;camera-aware proxies, we design both intra- and inter-camera contrastive&#10;learning components for our Re-ID model to effectively learn the ID&#10;discrimination ability within and across cameras. Meanwhile, a proxy-balanced&#10;sampling strategy is also designed, which facilitates our learning further.&#10;Extensive experiments on three large-scale Re-ID datasets show that our&#10;proposed approach outperforms most unsupervised methods by a significant&#10;margin. Especially, on the challenging MSMT17 dataset, we gain $14.3\%$ Rank-1&#10;and $10.2\%$ mAP improvements when compared to the second place. Code is&#10;available at: \texttt{https://github.com/Terminator8758/CAP-master}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="2304.14321" label="2304.14321">
-        <attvalues>
-          <attvalue for="0" value="Person Re-ID through Unsupervised Hypergraph Rank Selection and Fusion" />
-          <attvalue for="1" value="  Person Re-ID has been gaining a lot of attention and nowadays is of&#10;fundamental importance in many camera surveillance applications. The task&#10;consists of identifying individuals across multiple cameras that have no&#10;overlapping views. Most of the approaches require labeled data, which is not&#10;always available, given the huge amount of demanded data and the difficulty of&#10;manually assigning a class for each individual. Recently, studies have shown&#10;that re-ranking methods are capable of achieving significant gains, especially&#10;in the absence of labeled data. Besides that, the fusion of feature extractors&#10;and multiple-source training is another promising research direction not&#10;extensively exploited. We aim to fill this gap through a manifold rank&#10;aggregation approach capable of exploiting the complementarity of different&#10;person Re-ID rankers. In this work, we perform a completely unsupervised&#10;selection and fusion of diverse ranked lists obtained from multiple and diverse&#10;feature extractors. Among the contributions, this work proposes a query&#10;performance prediction measure that models the relationship among images&#10;considering a hypergraph structure and does not require the use of any labeled&#10;data. Expressive gains were obtained in four datasets commonly used for person&#10;Re-ID. We achieved results competitive to the state-of-the-art in most of the&#10;scenarios.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Person Re-ID is of critical importance in the majority of modern security and surveillance applications~\cite{paperReIDApp1, paperReIDApp2, paperReIDApp3, paperReIDSurvey2016}.&#10;The task consists in, given a query image of one person, to identify the same individual across different cameras which have no overlapping views.&#10;There are many difficulties for Re-ID retrieval~\cite{deepSurvey2020}, among them:&#10;(i) different viewpoints,&#10;(ii) possible low-image resolutions,&#10;(iii) illumination changes,&#10;(iv) occlusions,&#10;(v) difficulty of manually labeling data for training,&#10;(vi) large amount of data to be processed.&#10;The challenge of improving the effectiveness of these systems, especially in open-world scenarios, has attracted a lot of research efforts from the scientific community~\cite{airport, paperReIDSurvey2016, deepSurvey2020}.&#10;&#10;Initially, person Re-ID retrieval systems were mainly based on the use of hand-crafted feature representations~\cite{paperSalientColorFeature, paperELF, paperGBICOV, paperGOG, paperLOMO, paperWHOS, market1501, paperHLBP, cuhk03}. Besides that, other strategies commonly used to generic image retrieval, like bag of visual words~\cite{paperSDC, market1501}, have also been employed.&#10;Aiming at further improving the quality of the results, rather than using traditional distance measures, researchers have proposed metric learning approaches for Re-ID~\cite{paperLOMO, paperLargeScaleML, paperModelFeatDistClass, paperKernelizedReidML, zhang2016learning}, most of them based on supervised models.&#10;In~\cite{airport}, an extensive evaluation of multiple combinations of feature extractors and metric learning approaches is discussed.&#10;&#10;Due to the significant impact of deep learning on common image retrieval and machine learning~\cite{deepSurvey2020}, the Convolutional Neural Networks (CNN) also have gained a lot of attention and have been widely employed to solve Re-ID tasks in the recent years~\cite{deepSurvey2020, paperOSNET, paperOSNET-IBN-AIN, paperMLFN, paperHACNN, paperPCB, paperLargestMS, paperEANET, paperATNET, paperAANET}.&#10;Among the works, existing networks have been trained for Re-ID~\cite{paperRESNET, paperMobileNetv2}, and some new architectures have been proposed solely focusing on Re-ID~\cite{paperOSNET, paperOSNET-IBN-AIN, paperMLFN, paperHACNN, paperPCB}.&#10;&#10;Despite the success of deep learning, one of the main difficulties resides in the lack of large publicly available Re-ID datasets for training~\cite{deepSurvey2020}, mainly because manually labeling images is a very difficulty task, especially in open-world scenarios~\cite{airport}, where the datasets may increase dynamically.&#10;In order to mitigate this problem, many authors have proposed multi-source training, which consists in joining multiple available datasets for training~\cite{paperEMTL, paperCAMEL, paperLargestMS}, usually increasing the network capacity for generalization.&#10;&#10;However, despite several advances in multi-source approaches, it is inevitable the need for intensive manual annotation for obtaining training data.&#10;In practice, the demand for extensive training data restricts the generalization and scalability of supervised approaches, especially on person Re-ID tasks, which are not only resource-intensive to acquire identity annotation but also impractical for large-scale data~\cite{PaperCrossSimTIP20,PaperUnsReIDSimLenCVPR20}.&#10;&#10;Usually, there are two main steps associated to Re-ID labeling process~\cite{paperIntraCam2021, deepSurvey2020, paperIntraCam2019}: (i) intra-camera annotation that requires to compare a person with all the other unlabeled persons in a single camera with multiple views; and (ii) inter-camera that requires to match a person across different cameras with multiple views.&#10;Let $P$ be the number of persons and $S$ the number of camera views.&#10;The intra-camera annotation complexity is $O(S \times P^2)$ and inter-camera annotation complexity ranges from $O(S \times P^2)$ and $O(S^2 \times P^2)$.&#10;The worst case of inter-camera occurs because not all persons appear in every camera view in the majority of cases, which makes the association requiring to repeat for all $V$ camera views~\cite{paperIntraCam2021, paperIntraCam2019}.&#10;Commonly, inter-camera association significantly increases standard annotation costs.&#10;There are approaches that mitigate these issues; among them there is the Intra-Camera Supervised (ICS)~\cite{paperIntraCam2021, paperIntraCam2019}, Multi-Task Multi-Label (MATE)~\cite{paperIntraCam2021, paperIntraCam2019}, and Cross-camera Feature Prediction~\cite{paperCrossCamSup2021}, which were recently proposed.&#10;&#10;Due to the challenge of obtaining large amounts of strongly labeled data, semi-supervised methods have been employed, which is a typical strategy for supervision minimization.&#10;Based on information learned from a small set of labeled data, the idea is to generate labels from unlabeled training data.&#10;Some research has been made in this direction~\cite{paperSemiSup2013, paperSemiSup2014, paperOpenSetReid2016, paperSemiSup2019}.&#10;However, these methods often suffer from performance degradation and often require a large proportion of expensive cross-view pairwise labeling~\cite{paperIntraCam2021}.&#10;&#10;There are also weakly supervised strategies that replace accurate labels with inaccurate annotations.&#10;In~\cite{weaklyreid2019}, the authors proposed the idea of obtaining multiple bounding boxes of the same person from untrimmed videos. This is done by training a deep learning model capable of extracting multiple bounding boxes of the same person in a video.&#10;Recently,~\cite{wang2020weakly} proposed to replace image-level annotations with bag-level annotations.&#10;Weakly supervised Re-ID is very challenging, since it is rather difficult to model the considerable variances across camera views (e.g. occlusion and illumination) without using strong label data~\cite{weaklyreid2019, paperIntraCam2021}.&#10;&#10;As an alternative solution, unsupervised approaches~\cite{paperARN, paperECN, paperMAR, paperTAUDL, paperUTAL, paperUnsCrossDom2019, paperDECAMEL,PaperCrossSimTIP20,PaperUnsReIDSimLenCVPR20} have been attracting a lot of attention from the research community, especially because, once labeled data are not required, the methods become more suitable for real-world scenarios.&#10;In a promising research direction, to address the lack of labels issue, there are works proposed to post-process person Re-ID results by analyzing similarity relationships encoded in the datasets.&#10;Several authors have proposed unsupervised re-ranking approaches for Re-ID~\cite{cuhk03_new_protocol, paperCNearNeighborReRank, paperRRReID, paperLeng2014, paperQueryBasedAdaptativeRR, paperReIDRerankAnalysis, paperMang2015, paperDiscInfAnRR, paperGuo2018}.&#10;In~\cite{cuhk03_new_protocol}, the original distances among images are improved by calculating Jaccard correlation scores of each ranked list.&#10;Various approaches exploit the reciprocal neighborhood information and other co-occurrence indexes aiming at improving the ranking results.&#10;&#10;Another strategy commonly employed on generic unsupervised image retrieval and few exploited on Re-ID tasks consists in fusion approaches~\cite{paperPirasFusion17}. &#10; In general, the both broad categories of fusion have been successfully used in generic image retrieval: (i) early fusion, which combines the feature vectors; and (ii) late fusion, which usually combines ranked lists.&#10;Significant results have been achieved based on fusion of different ranked lists and features~\cite{paperUSRAF, fusionRetrieval}, with the purpose of obtaining more effective results by exploiting the complementarity of each input.&#10;&#10;On person Re-ID tasks, some authors have proposed late fusion strategies based on rank aggregation~\cite{paperRankAggReID2016, paperQueryAdaptiveFusion}.&#10;In~\cite{paperQueryAdaptiveFusion}, the aggregation is performed by attributing weights for each query of each ranker, but no pre-selection of features is performed, and all the features are used as input for the fusion step.&#10;There are also early fusion approaches for Re-ID, as~\cite{paperHypergraphFeatureFusion17}, that propose a supervised multi-hypergraph fusion model for early fusion of feature extractors.&#10;It learns a hypergraph for each feature through a star expansion strategy and they are fused according to weights that the method has learned from training.&#10;In~\cite{paperHypergraphReID19}, a hypergraph structure is used with a deep learning model to improve the performance of the acquired features for Re-ID.&#10;&#10;Although fusing different features can represent a significant advantage due to extra information available, how to choose what features to fuse can be a challenging task.&#10;Even for supervised approaches, selecting high-effective combinations of visual features remains a complex task, since it is necessary to consider&#10;various aspects, such as diversity and complementarity of results.&#10;Therefore, selecting features in an unsupervised way, without&#10;any labeled data is even more challenging, since no information about the effectiveness of individual visual features is available.&#10;&#10;This paper addresses the challenging task of unsupervised selection and fusion of different features for more effective person re-identification.&#10;We propose a novel Hypergraph Rank Selection and Fusion (HRSF) framework, which combines an unsupervised rank-based formulation for feature selection~\cite{paperUSRAF} with a robust hypergraph model~\cite{paperLHRR} for query performance prediction and rank aggregation based on manifold learning.&#10;Among our main contributions, we can highlight:&#10;&#10;\begin{itemize}&#10;&#10;\item The proposed HRSF framework uses a rank-based late fusion model, suitable for selection and fusion of a broad diversity of features. &#10; The selection is performed by exploiting an unsupervised measure for query performance prediction; &#10;&#10;\item A hypergraph rank-based formulation is used to encode the high-order relationship among images. This strategy is exploited for both selection and fusion tasks. Hypergraph models were little exploited in Re-ID literature~\cite{paperHypergraphFeatureFusion17, paperHypergraphReID19};&#10;&#10;\item The proposed technique is able to learn representations through the hypergraph structure that encodes multiple features from different rankers. The manifold learning based on a hypergraph model~\cite{paperLHRR} allows effective fusion and final ranking. In addition, our approach innovates by fusing different feature extractors trained on different datasets;&#10;&#10;\item Different from most of fusion approaches for Re-ID which often consider ad hoc selections or combine all the features of the input set~\cite{paperRankAggReID2016, paperQueryAdaptiveFusion, paperHypergraphReID19}, our approach is capable of dealing with various features in a completely unsupervised scenario, selecting combinations in a very large search space.&#10; To the best of our knowledge, this is the first work which performs an explicit selection and subsequently fusion of features on person Re-ID tasks in a completely unsupervised way.&#10;\end{itemize}&#10;&#10;In fact, the proposed Hypergraph Rank Selection and Fusion (HRSF) approach is based on both~\cite{paperUSRAF} and~\cite{paperLHRR}.&#10;However, while there are some aspects in common, also there are crucial differences. Among them, we can mention:&#10;\begin{itemize}&#10; \item The problem of selecting and fusing rankers in unsupervised scenarios is very challenging. Hence, the use of an effective algorithm for selection is fundamental. While USRF~\cite{paperUSRAF} employs traditional query performance prediction approaches (Authority and Reciprocal scores), we propose a new measure named Hypergraph Query Performance Prediction (HQPP);&#10; \item Originally, USRF~\cite{paperUSRAF} uses the CPRR~\cite{paperCPRR} as a method for fusion tasks. Differently, the proposed HRSF uses the LHRR~\cite{paperLHRR} method, which in combination with the HQPP, makes both the selection and fusion based on hypergraph structures. The LHRR~\cite{paperLHRR} method is also more robust than the CPRR~\cite{paperCPRR}, achieving superior retrieval results in most datasets;&#10; \item Both~\cite{paperUSRAF} and~\cite{paperLHRR} were originally proposed and evaluated only on general purpose image retrieval scenarios. The proposed HRSF is employed and validated for person Re-ID tasks;&#10; \item In unsupervised scenarios, the optimal neighborhood size ($k$ parameters) can be challenging to define. The experiments revealed that HRSF is more robust than HQPP to different neighborhood sizes, leading to the most effective results in the majority of scenarios.&#10;\end{itemize}&#10;&#10;A wide experimental evaluation was conducted on 4 different datasets with sizes ranging from 14,097 to 39,902 images. Up to 28 different rankers where considered in each case, resulting in millions of possible combinations. &#10;Experiments indicated that our approach was capable of selecting and fusing the rankers, achieving high-effective results superior to all rankers in isolation and competitive to state-of-the-art, when more than 20 recent Re-ID approaches are considered.&#10;&#10;The remainder of this paper is organized as follows. Section~\ref{secProbDef} discusses a formal definition of problem setting.&#10;Section~\ref{secProposedAproach} presents the proposed rank selection and fusion for Re-ID. &#10;Section~\ref{secExpEval} discusses the conducted experimental evaluation and, finally, Section~\ref{secConc} presents our conclusions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Person Re-Identification, Unsupervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1605.09653" label="1605.09653">
-        <attvalues>
-          <attvalue for="0" value="A Systematic Evaluation and Benchmark for Person Re-Identification:&#10;  Features, Metrics, and Datasets" />
-          <attvalue for="1" value="  Person re-identification (re-id) is a critical problem in video analytics&#10;applications such as security and surveillance. The public release of several&#10;datasets and code for vision algorithms has facilitated rapid progress in this&#10;area over the last few years. However, directly comparing re-id algorithms&#10;reported in the literature has become difficult since a wide variety of&#10;features, experimental protocols, and evaluation metrics are employed. In order&#10;to address this need, we present an extensive review and performance evaluation&#10;of single- and multi-shot re-id algorithms. The experimental protocol&#10;incorporates the most recent advances in both feature extraction and metric&#10;learning. To ensure a fair comparison, all of the approaches were implemented&#10;using a unified code library that includes 11 feature extraction algorithms and&#10;22 metric learning and ranking techniques. All approaches were evaluated using&#10;a new large-scale dataset that closely mimics a real-world problem setting, in&#10;addition to 16 other publicly available datasets: VIPeR, GRID, CAVIAR,&#10;DukeMTMC4ReID, 3DPeS, PRID, V47, WARD, SAIVT-SoftBio, CUHK01, CHUK02, CUHK03,&#10;RAiD, iLIDSVID, HDA+ and Market1501. The evaluation codebase and results will&#10;be made publicly available for community use.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.00953" label="1905.00953">
-        <attvalues>
-          <attvalue for="0" value="Omni-Scale Feature Learning for Person Re-Identification" />
-          <attvalue for="1" value="  As an instance-level recognition problem, person re-identification (ReID)&#10;relies on discriminative features, which not only capture different spatial&#10;scales but also encapsulate an arbitrary combination of multiple scales. We&#10;call features of both homogeneous and heterogeneous scales omni-scale features.&#10;In this paper, a novel deep ReID CNN is designed, termed Omni-Scale Network&#10;(OSNet), for omni-scale feature learning. This is achieved by designing a&#10;residual block composed of multiple convolutional streams, each detecting&#10;features at a certain scale. Importantly, a novel unified aggregation gate is&#10;introduced to dynamically fuse multi-scale features with input-dependent&#10;channel-wise weights. To efficiently learn spatial-channel correlations and&#10;avoid overfitting, the building block uses pointwise and depthwise&#10;convolutions. By stacking such block layer-by-layer, our OSNet is extremely&#10;lightweight and can be trained from scratch on existing ReID benchmarks.&#10;Despite its small model size, OSNet achieves state-of-the-art performance on&#10;six person ReID datasets, outperforming most large-sized models, often by a&#10;clear margin. Code and models are available at:&#10;\url{https://github.com/KaiyangZhou/deep-person-reid}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.03547" label="2004.03547">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Person Re-identification via Softened Similarity Learning" />
-          <attvalue for="1" value="  Person re-identification (re-ID) is an important topic in computer vision.&#10;This paper studies the unsupervised setting of re-ID, which does not require&#10;any labeled information and thus is freely deployed to new scenarios. There are&#10;very few studies under this setting, and one of the best approach till now used&#10;iterative clustering and classification, so that unlabeled images are clustered&#10;into pseudo classes for a classifier to get trained, and the updated features&#10;are used for clustering and so on. This approach suffers two problems, namely,&#10;the difficulty of determining the number of clusters, and the hard quantization&#10;loss in clustering. In this paper, we follow the iterative training mechanism&#10;but discard clustering, since it incurs loss from hard quantization, yet its&#10;only product, image-level similarity, can be easily replaced by pairwise&#10;computation and a softened classification task. With these improvements, our&#10;approach becomes more elegant and is more robust to hyper-parameter changes.&#10;Experiments on two image-based and video-based datasets demonstrate&#10;state-of-the-art performance under the unsupervised re-ID setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.05046" label="2002.05046">
-        <attvalues>
-          <attvalue for="0" value="Intra-Camera Supervised Person Re-Identification" />
-          <attvalue for="1" value="  Existing person re-identification (re-id) methods mostly exploit a large set&#10;of cross-camera identity labelled training data. This requires a tedious data&#10;collection and annotation process, leading to poor scalability in practical&#10;re-id applications. On the other hand unsupervised re-id methods do not need&#10;identity label information, but they usually suffer from much inferior and&#10;insufficient model performance. To overcome these fundamental limitations, we&#10;propose a novel person re-identification paradigm based on an idea of&#10;independent per-camera identity annotation. This eliminates the most&#10;time-consuming and tedious inter-camera identity labelling process,&#10;significantly reducing the amount of human annotation efforts. Consequently, it&#10;gives rise to a more scalable and more feasible setting, which we call&#10;Intra-Camera Supervised (ICS) person re-id, for which we formulate a Multi-tAsk&#10;mulTi-labEl (MATE) deep learning method. Specifically, MATE is designed for&#10;self-discovering the cross-camera identity correspondence in a per-camera&#10;multi-task inference framework. Extensive experiments demonstrate the&#10;cost-effectiveness superiority of our method over the alternative approaches on&#10;three large person re-id datasets. For example, MATE yields 88.7% rank-1 score&#10;on Market-1501 in the proposed ICS person re-id setting, significantly&#10;outperforming unsupervised learning models and closely approaching conventional&#10;fully supervised learning competitors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.13904" label="2107.13904">
-        <attvalues>
-          <attvalue for="0" value="Cross-Camera Feature Prediction for Intra-Camera Supervised Person&#10;  Re-identification across Distant Scenes" />
-          <attvalue for="1" value="  Person re-identification (Re-ID) aims to match person images across&#10;non-overlapping camera views. The majority of Re-ID methods focus on&#10;small-scale surveillance systems in which each pedestrian is captured in&#10;different camera views of adjacent scenes. However, in large-scale surveillance&#10;systems that cover larger areas, it is required to track a pedestrian of&#10;interest across distant scenes (e.g., a criminal suspect escapes from one city&#10;to another). Since most pedestrians appear in limited local areas, it is&#10;difficult to collect training data with cross-camera pairs of the same person.&#10;In this work, we study intra-camera supervised person re-identification across&#10;distant scenes (ICS-DS Re-ID), which uses cross-camera unpaired data with&#10;intra-camera identity labels for training. It is challenging as cross-camera&#10;paired data plays a crucial role for learning camera-invariant features in most&#10;existing Re-ID methods. To learn camera-invariant representation from&#10;cross-camera unpaired training data, we propose a cross-camera feature&#10;prediction method to mine cross-camera self supervision information from&#10;camera-specific feature distribution by transforming fake cross-camera positive&#10;feature pairs and minimize the distances of the fake pairs. Furthermore, we&#10;automatically localize and extract local-level feature by a transformer. Joint&#10;learning of global-level and local-level features forms a global-local&#10;cross-camera feature prediction scheme for mining fine-grained cross-camera&#10;self supervision information. Finally, cross-camera self supervision and&#10;intra-camera supervision are aggregated in a framework. The experiments are&#10;conducted in the ICS-DS setting on Market-SCT, Duke-SCT and MSMT17-SCT&#10;datasets. The evaluation results demonstrate the superiority of our method,&#10;which gains significant improvements of 15.4 Rank-1 and 22.3 mAP on Market-SCT&#10;as compared to the second best method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03832" label="1904.03832">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Person Re-Identification" />
-          <attvalue for="1" value="  In the conventional person re-id setting, it is assumed that the labeled&#10;images are the person images within the bounding box for each individual; this&#10;labeling across multiple nonoverlapping camera views from raw video&#10;surveillance is costly and time-consuming. To overcome this difficulty, we&#10;consider weakly supervised person re-id modeling. The weak setting refers to&#10;matching a target person with an untrimmed gallery video where we only know&#10;that the identity appears in the video without the requirement of annotating&#10;the identity in any frame of the video during the training procedure. Hence,&#10;for a video, there could be multiple video-level labels. We cast this weakly&#10;supervised person re-id challenge into a multi-instance multi-label learning&#10;(MIML) problem. In particular, we develop a Cross-View MIML (CV-MIML) method&#10;that is able to explore potential intraclass person images from all the camera&#10;views by incorporating the intra-bag alignment and the cross-view bag&#10;alignment. Finally, the CV-MIML method is embedded into an existing deep neural&#10;network for developing the Deep Cross-View MIML (Deep CV-MIML) model. We have&#10;performed extensive experiments to show the feasibility of the proposed weakly&#10;supervised setting and verify the effectiveness of our method compared to&#10;related methods on four weakly labeled datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03845" label="1904.03845">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Person Re-ID: Differentiable Graphical Learning and A&#10;  New Benchmark" />
-          <attvalue for="1" value="  Person re-identification (Re-ID) benefits greatly from the accurate&#10;annotations of existing datasets (e.g., CUHK03 [1] and Market-1501 [2]), which&#10;are quite expensive because each image in these datasets has to be assigned&#10;with a proper label. In this work, we ease the annotation of Re-ID by replacing&#10;the accurate annotation with inaccurate annotation, i.e., we group the images&#10;into bags in terms of time and assign a bag-level label for each bag. This&#10;greatly reduces the annotation effort and leads to the creation of a&#10;large-scale Re-ID benchmark called SYSU-30$k$. The new benchmark contains $30k$&#10;individuals, which is about $20$ times larger than CUHK03 ($1.3k$ individuals)&#10;and Market-1501 ($1.5k$ individuals), and $30$ times larger than ImageNet ($1k$&#10;categories). It sums up to 29,606,918 images. Learning a Re-ID model with&#10;bag-level annotation is called the weakly supervised Re-ID problem. To solve&#10;this problem, we introduce a differentiable graphical model to capture the&#10;dependencies from all images in a bag and generate a reliable pseudo label for&#10;each person image. The pseudo label is further used to supervise the learning&#10;of the Re-ID model. When compared with the fully supervised Re-ID models, our&#10;method achieves state-of-the-art performance on SYSU-30$k$ and other datasets.&#10;The code, dataset, and pretrained model will be available at&#10;\url{https://github.com/wanggrun/SYSU-30k}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.09347" label="1804.09347">
-        <attvalues>
-          <attvalue for="0" value="Adaptation and Re-Identification Network: An Unsupervised Deep Transfer&#10;  Learning Approach to Person Re-Identification" />
-          <attvalue for="1" value="  Person re-identification (Re-ID) aims at recognizing the same person from&#10;images taken across different cameras. To address this task, one typically&#10;requires a large amount labeled data for training an effective Re-ID model,&#10;which might not be practical for real-world applications. To alleviate this&#10;limitation, we choose to exploit a sufficient amount of pre-existing labeled&#10;data from a different (auxiliary) dataset. By jointly considering such an&#10;auxiliary dataset and the dataset of interest (but without label information),&#10;our proposed adaptation and re-identification network (ARN) performs&#10;unsupervised domain adaptation, which leverages information across datasets and&#10;derives domain-invariant features for Re-ID purposes. In our experiments, we&#10;verify that our network performs favorably against state-of-the-art&#10;unsupervised Re-ID approaches, and even outperforms a number of baseline Re-ID&#10;methods which require fully supervised data for training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.11658" label="2103.11658">
-        <attvalues>
-          <attvalue for="0" value="Intra-Inter Camera Similarity for Unsupervised Person Re-Identification" />
-          <attvalue for="1" value="  Most of unsupervised person Re-Identification (Re-ID) works produce&#10;pseudo-labels by measuring the feature similarity without considering the&#10;distribution discrepancy among cameras, leading to degraded accuracy in label&#10;computation across cameras. This paper targets to address this challenge by&#10;studying a novel intra-inter camera similarity for pseudo-label generation. We&#10;decompose the sample similarity computation into two stage, i.e., the&#10;intra-camera and inter-camera computations, respectively. The intra-camera&#10;computation directly leverages the CNN features for similarity computation&#10;within each camera. Pseudo-labels generated on different cameras train the&#10;re-id model in a multi-branch network. The second stage considers the&#10;classification scores of each sample on different cameras as a new feature&#10;vector. This new feature effectively alleviates the distribution discrepancy&#10;among cameras and generates more reliable pseudo-labels. We hence train our&#10;re-id model in two stages with intra-camera and inter-camera pseudo-labels,&#10;respectively. This simple intra-inter camera similarity produces surprisingly&#10;good performance on multiple datasets, e.g., achieves rank-1 accuracy of 89.5%&#10;on the Market1501 dataset, outperforming the recent unsupervised works by 9+%,&#10;and is comparable with the latest transfer learning works that leverage extra&#10;annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;This work is closely related to unsupervised person ReID and, domain adaptation and generalization. Recent works on those two topics will be reviewed briefly in following paragraphs.&#10;&#10;Unsupervised person ReID has been studied with three types of methods, i.e., by distribution alignment, training GANs, and generating pseudo-labels, respectively. Distribution alignment based methods follow the traditional domain adaptation methods~\cite{MMD, coral} to align the feature distribution of source and target domains. Wu \etal~\cite{wu2019unsupervised} proposed a Camera-Aware Similarity Consistency Loss to align the pairwise distribution of intra-camera matching and cross-camera matching.&#10;Lin \etal~\cite{lin2018multi} utilized Maximum Mean Discrepancy (MMD) distance \cite{MMD} to align the distribution of mid-level features from source and the target domains. Some other methods use GANs~\cite{CycleGAN2017} to perform image-to-image style translation to transfer source images into target style.&#10;Zou \etal~\cite{zou2020joint} disentangled id-related/unrelated features to enforce the adaptation to work on the id-related feature space.&#10;Wei \etal~\cite{PTGan} proposed person transfer GAN, which can transfer person images with the style of target dataset and keep the identity label of the person.&#10;&#10;Pseudo-labels based methods first generate pseudo-labels by formulating certain rules based on sample similarity, then train the ReID model with those pseudo-labels.&#10;The quality of computed pseudo-labels determines the performance of these methods. Unsupervised clustering method is one of the most commonly used methods to generate pseudo-labels \cite{jin2020global, zhang2019self, multi-label, zhai2020ad, liu2020domain}.&#10;Fan \etal~\cite{fan2018unsupervised} used standard k-means clustering method to generate pseudo-labels and fine-tuned model with these labels.&#10;Lin \etal~\cite{lin2019bottom} proposed a bottom-up clustering approach to generate pseudo-labels. To avoid re-initializing the classifier at each epoch, an extra memory bank was added into the network.&#10;Wang \etal~\cite{multi-label} formulated unsupervised person ReID as multi-label classification task and used memory bank to train the network.&#10;NRMT \cite{zhao2020unsupervised}, MMT \cite{MMT} and MEB-Net \cite{zhai2020multiple} used mutual-training \cite{mutual} to reduce the influence of low-quality pseudo-labels.&#10;&#10;Domain adaptation and generalization are commonly considered to improve the generalization ability of CNN models. Recently, some works have found that Batch Normalization (BN) \cite{BN} and Instance Normalization (IN) \cite{IN} could improve the network's generalization ability on multiple domains~\cite{zhuangrethinking, IBNNet, chang2019domain}. &#10;IBN-Net \cite{IBNNet} integrated IN and BN to enhance the generalization capacity of CNNs to unseen domain without fine-tuning. Chang \etal~\cite{chang2019domain} improved the performance of unsupervised domain adaptation using domain-specific BN. Zhuang \etal~\cite{zhuangrethinking} designed a camera-based BN to alleviate the distribution gap between a camera pair in person ReID. Their method improved the generalization ability of the model across unseen cameras. &#10;&#10;Most pseudo-labels based methods try to mitigate the impact of low-quality pseudo-labels or find high-quality part from generated pseudo-labels.&#10;The work most similar to us is \cite{zhu2021intracamera} which utilizes extra ID labels within each camera as supervision and simply uses classification results to find matching candidates across cameras during inter-camera training.&#10;Different from those works, our work is motivated to seek a reliable similarity by progressively eliminating negative influences of pose variances, illumination, occlusions through intra-camera training, and domain gap through inter-camera&#10;training. This leads to the proposed AIBN and inter-camera similarities.&#10;As shown in our experiments, our method produces surprisingly good performance on multiple datasets.&#10;&#10;" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Person Re-Identification, Unsupervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2304.12634" label="2304.12634">
-        <attvalues>
-          <attvalue for="0" value="Pseudo Labels Refinement with Intra-camera Similarity for Unsupervised&#10;  Person Re-identification" />
-          <attvalue for="1" value="  Unsupervised person re-identification (Re-ID) aims to retrieve person images&#10;across cameras without any identity labels. Most clustering-based methods&#10;roughly divide image features into clusters and neglect the feature&#10;distribution noise caused by domain shifts among different cameras, leading to&#10;inevitable performance degradation. To address this challenge, we propose a&#10;novel label refinement framework with clustering intra-camera similarity.&#10;Intra-camera feature distribution pays more attention to the appearance of&#10;pedestrians and labels are more reliable. We conduct intra-camera training to&#10;get local clusters in each camera, respectively, and refine inter-camera&#10;clusters with local results. We hence train the Re-ID model with refined&#10;reliable pseudo labels in a self-paced way. Extensive experiments demonstrate&#10;that the proposed method surpasses state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Person re-identification (Re-ID) is the task of retrieving images of a specified person under non-overlapping cameras \cite{zheng2016person}. Thanks to the development of deep learning technology, supervised Re-ID methods have led to great performance advances. Unfortunately, purely supervised methods expect a large amount of expensive labelled data, limiting the adaptability in the real world. Therefore, unsupervised settings have gained increasing research attention.&#10;&#10;Unsupervised Re-ID works can be categorized into purely unsupervised learning methods and unsupervised domain adaption (UDA) methods based on whether using external labelled data. The former methods train the feature encoder without any identity labels while UDA methods require a labelled source domain and an unlabeled target domain. This paper addresses the purely unsupervised person Re-ID task, which is more scalable and challenging. Existing unsupervised works usually adopt pseudo-label-based methods, which alternate between generating pseudo labels by clustering \cite{lin2019bottom,dai2022cluster}, softened labels \cite{yu2019unsupervised,lin2020unsupervised} or label refinement \cite{cho2022part} and training the model with the generated labels. The inherent noises in pseudo labels severely degrade the performance of these unsupervised works, and recent studies address the challenge. MMCL \cite{wang2020unsupervised} trains the Re-ID model to recognize each image as multiple labels. SPCL \cite{ge2020self} proposes a unified contrastive learning framework to seek reliable labels. RLCC \cite{zhang2021refining} exploits cluster consensus to estimate pseudo-label similarity and refines pseudo-labels. Although previous methods have achieved state-of-the-art performance, most of them measured the similarity of image pairs without considering domain shifts among different cameras. Due to camera views and resolution gap, feature distribution of the same ID suffers from significant variance. Positive pairs captured from different cameras may have larger dissimilarity than negative samples from the same camera. Hence, it's difficult to identify image pairs of the same ID cross cameras and learn reliable intra-class similarity.&#10;&#10;Recent works have utilized camera labels to address the problem above. SSL \cite{lin2020unsupervised} proposes a cross-camera encouragement term to reduce intra-camera negative pairs by increasing the dissimilarity to image pairs under the same camera. CAP \cite{wang2021camera} splits each cluster into multiple camera-aware proxies according to the camera ID. The proxies can capture the local structure within the clusters and deal with the intra-ID differences and the inter-ID similarities. IICS \cite{xuan2021intra} alternates between intra-camera and inter-camera iterative training to optimize feature encoder. Compared to existing work that directly utilizes camera labels to optimize the re-ID model, this work is motivated to cluster intra-camera similarity to refine global inter-camera pseudo labels, and the refined labels are adopted to train the model. &#10;&#10;In order to take full advantage of the inter-camera similarity, we propose a novel pseudo-label refinement framework to reduce label noise and exploit more reliable labels. Feature distribution in the same camera pays more attention to the appearance of pedestrians and could result in highly reliable intra-camera pseudo labels, which inspires us to make full use of intra-camera pseudo labels to refine noisy inter-camera pseudo labels. Specifically, we adopt a two-stage training strategy to optimize the model. The first stage is intra-camera training, which is conducted in each camera respectively. Features of each camera domain are separately extracted and clustered to generate independent local pseudo labels. The generated pseudo-labels are used as supervision to optimize their specific encoder. This stage ensures that the local pseudo labels are reliable enough to refine the global inter-camera clusters results in the next stage.&#10;&#10;The second stage is inter-camera training conducted across cameras. Because of domain shifts among different cameras, global inter-camera label accuracy suffers greatly. Therefore, we utilize reliable intra-camera labels to handle the label noise. Specifically, we first choose some information nodes for each cluster, which usually have high utility. For each information node, query the relationship with the others in the same cluster and discard the negative samples based on cluster results of the first stage. The remaining pseudo-labels are more reliable for learning. In this way, we train the Re-ID model through self-paced learning. &#10;&#10;Our contributions can be summarized as follows. First, we exploit intra-camera similarity to effectively handle the label noise caused by the domain gap. Second, we propose a novel label refinement method to seek more reliable labels for feature learning. Besides, Extensive experimental results have demonstrated the state-of-the-art performance of our methods on multiple large-scale datasets. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Unsupervised Learning, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="2103.11568" label="2103.11568">
-        <attvalues>
-          <attvalue for="0" value="Cluster Contrast for Unsupervised Person Re-Identification" />
-          <attvalue for="1" value="  State-of-the-art unsupervised re-ID methods train the neural networks using a&#10;memory-based non-parametric softmax loss. Instance feature vectors stored in&#10;memory are assigned pseudo-labels by clustering and updated at instance level.&#10;However, the varying cluster sizes leads to inconsistency in the updating&#10;progress of each cluster. To solve this problem, we present Cluster Contrast&#10;which stores feature vectors and computes contrast loss at the cluster level.&#10;Our approach employs a unique cluster representation to describe each cluster,&#10;resulting in a cluster-level memory dictionary. In this way, the consistency of&#10;clustering can be effectively maintained throughout the pipline and the GPU&#10;memory consumption can be significantly reduced. Thus, our method can solve the&#10;problem of cluster inconsistency and be applicable to larger data sets. In&#10;addition, we adopt different clustering algorithms to demonstrate the&#10;robustness and generalization of our framework. The application of Cluster&#10;Contrast to a standard unsupervised re-ID pipeline achieves considerable&#10;improvements of 9.9%, 8.3%, 12.1% compared to state-of-the-art purely&#10;unsupervised re-ID methods and 5.5%, 4.8%, 4.4% mAP compared to the&#10;state-of-the-art unsupervised domain adaptation re-ID methods on the Market,&#10;Duke, and MSMT17 datasets. Code is available at&#10;https://github.com/alibaba/cluster-contrast.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06325" label="1903.06325">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Person Re-identification by Soft Multilabel Learning" />
-          <attvalue for="1" value="  Although unsupervised person re-identification (RE-ID) has drawn increasing&#10;research attentions due to its potential to address the scalability problem of&#10;supervised RE-ID models, it is very challenging to learn discriminative&#10;information in the absence of pairwise labels across disjoint camera views. To&#10;overcome this problem, we propose a deep model for the soft multilabel learning&#10;for unsupervised RE-ID. The idea is to learn a soft multilabel (real-valued&#10;label likelihood vector) for each unlabeled person by comparing (and&#10;representing) the unlabeled person with a set of known reference persons from&#10;an auxiliary domain. We propose the soft multilabel-guided hard negative mining&#10;to learn a discriminative embedding for the unlabeled target domain by&#10;exploring the similarity consistency of the visual features and the soft&#10;multilabels of unlabeled target pairs. Since most target pairs are cross-view&#10;pairs, we develop the cross-view consistent soft multilabel learning to achieve&#10;the learning goal that the soft multilabels are consistently good across&#10;different camera views. To enable effecient soft multilabel learning, we&#10;introduce the reference agent learning to represent each reference person by a&#10;reference agent in a joint embedding. We evaluate our unified deep model on&#10;Market-1501 and DukeMTMC-reID. Our model outperforms the state-of-the-art&#10;unsupervised RE-ID methods by clear margins. Code is available at&#10;https://github.com/KovenYu/MAR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.14675" label="2203.14675">
-        <attvalues>
-          <attvalue for="0" value="Part-based Pseudo Label Refinement for Unsupervised Person&#10;  Re-identification" />
-          <attvalue for="1" value="  Unsupervised person re-identification (re-ID) aims at learning discriminative&#10;representations for person retrieval from unlabeled data. Recent techniques&#10;accomplish this task by using pseudo-labels, but these labels are inherently&#10;noisy and deteriorate the accuracy. To overcome this problem, several&#10;pseudo-label refinement methods have been proposed, but they neglect the&#10;fine-grained local context essential for person re-ID. In this paper, we&#10;propose a novel Part-based Pseudo Label Refinement (PPLR) framework that&#10;reduces the label noise by employing the complementary relationship between&#10;global and part features. Specifically, we design a cross agreement score as&#10;the similarity of k-nearest neighbors between feature spaces to exploit the&#10;reliable complementary relationship. Based on the cross agreement, we refine&#10;pseudo-labels of global features by ensembling the predictions of part&#10;features, which collectively alleviate the noise in global feature clustering.&#10;We further refine pseudo-labels of part features by applying label smoothing&#10;according to the suitability of given labels for each part. Thanks to the&#10;reliable complementary information provided by the cross agreement score, our&#10;PPLR effectively reduces the influence of noisy labels and learns&#10;discriminative representations with rich local contexts. Extensive experimental&#10;results on Market-1501 and MSMT17 demonstrate the effectiveness of the proposed&#10;method over the state-of-the-art performance. The code is available at&#10;https://github.com/yoonkicho/PPLR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.09228" label="2004.09228">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Person Re-identification via Multi-label Classification" />
-          <attvalue for="1" value="  The challenge of unsupervised person re-identification (ReID) lies in&#10;learning discriminative features without true labels. This paper formulates&#10;unsupervised person ReID as a multi-label classification task to progressively&#10;seek true labels. Our method starts by assigning each person image with a&#10;single-class label, then evolves to multi-label classification by leveraging&#10;the updated ReID model for label prediction. The label prediction comprises&#10;similarity computation and cycle consistency to ensure the quality of predicted&#10;labels. To boost the ReID model training efficiency in multi-label&#10;classification, we further propose the memory-based multi-label classification&#10;loss (MMCL). MMCL works with memory-based non-parametric classifier and&#10;integrates multi-label classification and single-label classification in a&#10;unified framework. Our label prediction and MMCL work iteratively and&#10;substantially boost the ReID performance. Experiments on several large-scale&#10;person ReID datasets demonstrate the superiority of our method in unsupervised&#10;person ReID. Our method also allows to use labeled person images in other&#10;domains. Under this transfer learning setting, our method also achieves&#10;state-of-the-art performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.02713" label="2006.02713">
-        <attvalues>
-          <attvalue for="0" value="Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive&#10;  Object Re-ID" />
-          <attvalue for="1" value="  Domain adaptive object re-ID aims to transfer the learned knowledge from the&#10;labeled source domain to the unlabeled target domain to tackle the open-class&#10;re-identification problems. Although state-of-the-art pseudo-label-based&#10;methods have achieved great success, they did not make full use of all valuable&#10;information because of the domain gap and unsatisfying clustering performance.&#10;To solve these problems, we propose a novel self-paced contrastive learning&#10;framework with hybrid memory. The hybrid memory dynamically generates&#10;source-domain class-level, target-domain cluster-level and un-clustered&#10;instance-level supervisory signals for learning feature representations.&#10;Different from the conventional contrastive learning strategy, the proposed&#10;framework jointly distinguishes source-domain classes, and target-domain&#10;clusters and un-clustered instances. Most importantly, the proposed self-paced&#10;method gradually creates more reliable clusters to refine the hybrid memory and&#10;learning targets, and is shown to be the key to our outstanding performance.&#10;Our method outperforms state-of-the-arts on multiple domain adaptation tasks of&#10;object re-ID and even boosts the performance on the source domain without any&#10;extra annotations. Our generalized version on unsupervised object re-ID&#10;surpasses state-of-the-art algorithms by considerable 16.7% and 7.9% on&#10;Market-1501 and MSMT17 benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.06133" label="2106.06133">
-        <attvalues>
-          <attvalue for="0" value="Refining Pseudo Labels with Clustering Consensus over Generations for&#10;  Unsupervised Object Re-identification" />
-          <attvalue for="1" value="  Unsupervised object re-identification targets at learning discriminative&#10;representations for object retrieval without any annotations. Clustering-based&#10;methods conduct training with the generated pseudo labels and currently&#10;dominate this research direction. However, they still suffer from the issue of&#10;pseudo label noise. To tackle the challenge, we propose to properly estimate&#10;pseudo label similarities between consecutive training generations with&#10;clustering consensus and refine pseudo labels with temporally propagated and&#10;ensembled pseudo labels. To the best of our knowledge, this is the first&#10;attempt to leverage the spirit of temporal ensembling to improve classification&#10;with dynamically changing classes over generations. The proposed pseudo label&#10;refinery strategy is simple yet effective and can be seamlessly integrated into&#10;existing clustering-based unsupervised re-identification methods. With our&#10;proposed approach, state-of-the-art method can be further boosted with up to&#10;8.8% mAP improvements on the challenging MSMT17 dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.10015" label="2403.10015">
-        <attvalues>
-          <attvalue for="0" value="Linear optimal transport subspaces for point set classification" />
-          <attvalue for="1" value="  Learning from point sets is an essential component in many computer vision&#10;and machine learning applications. Native, unordered, and permutation invariant&#10;set structure space is challenging to model, particularly for point set&#10;classification under spatial deformations. Here we propose a framework for&#10;classifying point sets experiencing certain types of spatial deformations, with&#10;a particular emphasis on datasets featuring affine deformations. Our approach&#10;employs the Linear Optimal Transport (LOT) transform to obtain a linear&#10;embedding of set-structured data. Utilizing the mathematical properties of the&#10;LOT transform, we demonstrate its capacity to accommodate variations in point&#10;sets by constructing a convex data space, effectively simplifying point set&#10;classification problems. Our method, which employs a nearest-subspace algorithm&#10;in the LOT space, demonstrates label efficiency, non-iterative behavior, and&#10;requires no hyper-parameter tuning. It achieves competitive accuracies compared&#10;to state-of-the-art methods across various point set classification tasks.&#10;Furthermore, our approach exhibits robustness in out-of-distribution scenarios&#10;where training and test distributions vary in terms of deformation magnitudes.&#10;" />
-          <attvalue for="2" value="&#10;Point sets provide valuable insights about object geometry, making them useful for a variety of applications, including object detection, recognition, segmentation, and tracking in fields such as robotics, autonomous vehicles, virtual reality, and computer vision, among others \cite{qi2017pointnet,zhao2019pointweb,li2018pointcnn,chen2017multi,zhou2018voxelnet}. They represent the surface geometry of an object in an N-dimensional space as a set of points, obtained using various scanning technologies such as LiDAR or photogrammetry \cite{xu2021toward,wang2020computational}, or by sampling a continuous probability density function over an N-D space \cite{zhou20213d}. However, modeling the set structure space for classification presents significant challenges due to the sparsity and noise in data, the accumulation of spatial deformations (such as affine deformations) in real-world point set data, and the high dimensionality of point sets, among other factors \cite{pomerleau2015review,wang2020cascaded,qi2017pointnet,zeng20193d}. Furthermore, defining a metric or distance function for point set classification is challenging due to the permutation invariant nature of point sets resulting from the arbitrary order of points in a set \cite{qi2017pointnet,lu2021slosh}. However, despite these challenges, there has been a growing interest in developing new algorithms and techniques for point set classification. &#10;&#10;In recent years, several research efforts have focused on point set classification, resulting in the development of various methods to address challenges in this area. Over the last few decades, point set classification methods have evolved from relying on feature engineering \cite{radenovic2018fine,wang2020deep,acharya2018covariance,zhang2019fspool} to utilizing deep neural networks to learn representations and use them in classification tasks \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}. Neural networks have emerged as a leading classification framework for point sets, providing end-to-end learning capabilities and eliminating the need for hand-crafted feature engineering. They have demonstrated to achieve high accuracy in several classification tasks, and are also suitable for parallel implementation using graphical processing units (GPUs) \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}. However, the effectiveness of neural network-based methods is often limited by their high data requirements \cite{liu2019deep}, high computational costs \cite{qi2017pointnet}, and vulnerability to out-of-distribution samples, e.g., adversarial attacks \cite{liao2018defense,basu2014detecting,shifat2021radon}. &#10;&#10;While the conventional approach to modeling point sets involves direct processing of their coordinates, an alternative and less commonly used method is to represent a point set as a deformation of another point set \cite{kolouri2017optimal}. To address this challenge, point set deformation models have been developed utilizing the mathematics of optimal mass transport \cite{kolouri2017optimal,wang2013linear}. These models treat a point set as a smooth, nonlinear, and invertible transformation of a reference point set structure. The estimation of such models can be facilitated through the use of the linear optimal transport (LOT) transform, which has found applications in various fields \cite{wang2013linear}. The LOT of a point set provides as a linear embedding for that point set which can be used to compare with other point set data \cite{wang2013linear}. The LOT transform has been combined with various machine learning techniques and has been used in many applications \cite{wang2013linear,basu2014detecting}.&#10;&#10;This paper introduces a new method for classifying point sets by expanding upon the LOT-based modeling frameworks. We start by introducing a transport generative model to define point set classes, where class elements can be conceived as instances of an unknown template point set pattern under the effect of unknown spatial deformations. Using the mathematical properties of the LOT transform, we establish that these point set classes, under our generative model (with certain conditions on spatial deformations), can be constructed as convex subspaces in the LOT space, which are capable of accommodating the variations in point set data. Subsequently, we propose a nearest subspace-based classifier in the LOT space for classifying point sets under the given generative model. Our model is also capable of mathematically encoding invariances by integrating mathematical knowledge of deformations known to be present in the data. In our experiments, we particularly focus on datasets experiencing affine deformations and demonstrate the effectiveness of our method compared to several state-of-the-art methods. Our approach exhibits particular strength in situations characterized by limited training data and in the challenging out-of-distribution setting, where the training and test distributions differ in terms of deformation magnitudes.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Computer Science, Statistics, Point Set Classification, Machine Learning Algorithms" />
-        </attvalues>
-      </node>
-      <node id="2104.03670" label="2104.03670">
-        <attvalues>
-          <attvalue for="0" value="3D Shape Generation and Completion through Point-Voxel Diffusion" />
-          <attvalue for="1" value="  We propose a novel approach for probabilistic generative modeling of 3D&#10;shapes. Unlike most existing models that learn to deterministically translate a&#10;latent vector to a shape, our model, Point-Voxel Diffusion (PVD), is a unified,&#10;probabilistic formulation for unconditional shape generation and conditional,&#10;multi-modal shape completion. PVD marries denoising diffusion models with the&#10;hybrid, point-voxel representation of 3D shapes. It can be viewed as a series&#10;of denoising steps, reversing the diffusion process from observed point cloud&#10;data to Gaussian noise, and is trained by optimizing a variational lower bound&#10;to the (conditional) likelihood function. Experiments demonstrate that PVD is&#10;capable of synthesizing high-fidelity shapes, completing partial point clouds,&#10;and generating multiple completion results from single-view depth scans of real&#10;objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.03327" label="2004.03327">
-        <attvalues>
-          <attvalue for="0" value="Cascaded Refinement Network for Point Cloud Completion" />
-          <attvalue for="1" value="  Point clouds are often sparse and incomplete. Existing shape completion&#10;methods are incapable of generating details of objects or learning the complex&#10;point distributions. To this end, we propose a cascaded refinement network&#10;together with a coarse-to-fine strategy to synthesize the detailed object&#10;shapes. Considering the local details of partial input with the global shape&#10;information together, we can preserve the existing details in the incomplete&#10;point set and generate the missing parts with high fidelity. We also design a&#10;patch discriminator that guarantees every local area has the same pattern with&#10;the ground truth to learn the complicated point distribution. Quantitative and&#10;qualitative experiments on different datasets show that our method achieves&#10;superior results compared to existing state-of-the-art approaches on the 3D&#10;point cloud completion task. Our source code is available at&#10;https://github.com/xiaogangw/cascaded-point-completion.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07252" label="1803.07252">
-        <attvalues>
-          <attvalue for="0" value="3D Point Cloud Denoising using Graph Laplacian Regularization of a Low&#10;  Dimensional Manifold Model" />
-          <attvalue for="1" value="  3D point cloud - a new signal representation of volumetric objects - is a&#10;discrete collection of triples marking exterior object surface locations in 3D&#10;space. Conventional imperfect acquisition processes of 3D point cloud - e.g.,&#10;stereo-matching from multiple viewpoint images or depth data acquired directly&#10;from active light sensors - imply non-negligible noise in the data. In this&#10;paper, we adopt a previously proposed low-dimensional manifold model for the&#10;surface patches in the point cloud and seek self-similar patches to denoise&#10;them simultaneously using the patch manifold prior. Due to discrete&#10;observations of the patches on the manifold, we approximate the manifold&#10;dimension computation defined in the continuous domain with a patch-based graph&#10;Laplacian regularizer and propose a new discrete patch distance measure to&#10;quantify the similarity between two same-sized surface patches for graph&#10;construction that is robust to noise. We show that our graph Laplacian&#10;regularizer has a natural graph spectral interpretation, and has desirable&#10;numerical stability properties via eigenanalysis. Extensive simulation results&#10;show that our proposed denoising scheme can outperform state-of-the-art methods&#10;in objective metrics and can better preserve visually salient structural&#10;features like edges.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.05872" label="2112.05872">
-        <attvalues>
-          <attvalue for="0" value="SLOSH: Set LOcality Sensitive Hashing via Sliced-Wasserstein Embeddings" />
-          <attvalue for="1" value="  Learning from set-structured data is an essential problem with many&#10;applications in machine learning and computer vision. This paper focuses on&#10;non-parametric and data-independent learning from set-structured data using&#10;approximate nearest neighbor (ANN) solutions, particularly locality-sensitive&#10;hashing. We consider the problem of set retrieval from an input set query. Such&#10;retrieval problem requires: 1) an efficient mechanism to calculate the&#10;distances/dissimilarities between sets, and 2) an appropriate data structure&#10;for fast nearest neighbor search. To that end, we propose Sliced-Wasserstein&#10;set embedding as a computationally efficient &quot;set-2-vector&quot; mechanism that&#10;enables downstream ANN, with theoretical guarantees. The set elements are&#10;treated as samples from an unknown underlying distribution, and the&#10;Sliced-Wasserstein distance is used to compare sets. We demonstrate the&#10;effectiveness of our algorithm, denoted as Set-LOcality Sensitive Hashing&#10;(SLOSH), on various set retrieval datasets and compare our proposed embedding&#10;with standard set embedding approaches, including Generalized Mean (GeM)&#10;embedding/pooling, Featurewise Sort Pooling (FSPool), and Covariance Pooling&#10;and show consistent improvement in retrieval results. The code for replicating&#10;our results is available here:&#10;\href{https://github.com/mint-vu/SLOSH}{https://github.com/mint-vu/SLOSH}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06836" label="1904.06836">
-        <attvalues>
-          <attvalue for="0" value="Deep CNNs Meet Global Covariance Pooling: Better Representation and&#10;  Generalization" />
-          <attvalue for="1" value="  Compared with global average pooling in existing deep convolutional neural&#10;networks (CNNs), global covariance pooling can capture richer statistics of&#10;deep features, having potential for improving representation and generalization&#10;abilities of deep CNNs. However, integration of global covariance pooling into&#10;deep CNNs brings two challenges: (1) robust covariance estimation given deep&#10;features of high dimension and small sample size; (2) appropriate usage of&#10;geometry of covariances. To address these challenges, we propose a global&#10;Matrix Power Normalized COVariance (MPN-COV) Pooling. Our MPN-COV conforms to a&#10;robust covariance estimator, very suitable for scenario of high dimension and&#10;small sample size. It can also be regarded as Power-Euclidean metric between&#10;covariances, effectively exploiting their geometry. Furthermore, a global&#10;Gaussian embedding network is proposed to incorporate first-order statistics&#10;into MPN-COV. For fast training of MPN-COV networks, we implement an iterative&#10;matrix square root normalization, avoiding GPU unfriendly eigen-decomposition&#10;inherent in MPN-COV. Additionally, progressive 1x1 convolutions and group&#10;convolution are introduced to compress covariance representations. The proposed&#10;methods are highly modular, readily plugged into existing deep CNNs. Extensive&#10;experiments are conducted on large-scale object classification, scene&#10;categorization, fine-grained visual recognition and texture classification,&#10;showing our methods outperform the counterparts and obtain state-of-the-art&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.04855" label="1805.04855">
-        <attvalues>
-          <attvalue for="0" value="Covariance Pooling For Facial Expression Recognition" />
-          <attvalue for="1" value="  Classifying facial expressions into different categories requires capturing&#10;regional distortions of facial landmarks. We believe that second-order&#10;statistics such as covariance is better able to capture such distortions in&#10;regional facial fea- tures. In this work, we explore the benefits of using a&#10;man- ifold network structure for covariance pooling to improve facial&#10;expression recognition. In particular, we first employ such kind of manifold&#10;networks in conjunction with tradi- tional convolutional networks for spatial&#10;pooling within in- dividual image feature maps in an end-to-end deep learning&#10;manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on&#10;the validation set of Static Facial Expressions in the Wild (SFEW 2.0) and&#10;87.0% on the vali- dation set of Real-World Affective Faces (RAF) Database.&#10;Both of these results are the best results we are aware of. Besides, we&#10;leverage covariance pooling to capture the tem- poral evolution of per-frame&#10;features for video-based facial expression recognition. Our reported results&#10;demonstrate the advantage of pooling image-set features temporally by stacking&#10;the designed manifold network of covariance pool-ing on top of convolutional&#10;network layers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.02795" label="1906.02795">
-        <attvalues>
-          <attvalue for="0" value="FSPool: Learning Set Representations with Featurewise Sort Pooling" />
-          <attvalue for="1" value="  Traditional set prediction models can struggle with simple datasets due to an&#10;issue we call the responsibility problem. We introduce a pooling method for&#10;sets of feature vectors based on sorting features across elements of the set.&#10;This can be used to construct a permutation-equivariant auto-encoder that&#10;avoids this responsibility problem. On a toy dataset of polygons and a set&#10;version of MNIST, we show that such an auto-encoder produces considerably&#10;better reconstructions and representations. Replacing the pooling function in&#10;existing set encoders with FSPool improves accuracy and convergence speed on a&#10;variety of datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.02976" label="1712.02976">
-        <attvalues>
-          <attvalue for="0" value="Defense against Adversarial Attacks Using High-Level Representation&#10;  Guided Denoiser" />
-          <attvalue for="1" value="  Neural networks are vulnerable to adversarial examples, which poses a threat&#10;to their application in security sensitive systems. We propose high-level&#10;representation guided denoiser (HGD) as a defense for image classification.&#10;Standard denoiser suffers from the error amplification effect, in which small&#10;residual adversarial noise is progressively amplified and leads to wrong&#10;classifications. HGD overcomes this problem by using a loss function defined as&#10;the difference between the target model's outputs activated by the clean image&#10;and denoised image. Compared with ensemble adversarial training which is the&#10;state-of-the-art defending method on large images, HGD has three advantages.&#10;First, with HGD as a defense, the target model is more robust to either&#10;white-box or black-box adversarial attacks. Second, HGD can be trained on a&#10;small subset of the images and generalizes well to other images and unseen&#10;classes. Third, HGD can be transferred to defend models other than the one&#10;guiding it. In NIPS competition on defense against adversarial attacks, our HGD&#10;solution won the first place and outperformed other models by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.03669" label="2004.03669">
-        <attvalues>
-          <attvalue for="0" value="Radon cumulative distribution transform subspace modeling for image&#10;  classification" />
-          <attvalue for="1" value="  We present a new supervised image classification method applicable to a broad&#10;class of image deformation models. The method makes use of the previously&#10;described Radon Cumulative Distribution Transform (R-CDT) for image data, whose&#10;mathematical properties are exploited to express the image data in a form that&#10;is more suitable for machine learning. While certain operations such as&#10;translation, scaling, and higher-order transformations are challenging to model&#10;in native image space, we show the R-CDT can capture some of these variations&#10;and thus render the associated image classification problems easier to solve.&#10;The method -- utilizing a nearest-subspace algorithm in R-CDT space -- is&#10;simple to implement, non-iterative, has no hyper-parameters to tune, is&#10;computationally efficient, label efficient, and provides competitive accuracies&#10;to state-of-the-art neural networks for many types of classification problems.&#10;In addition to the test accuracy performances, we show improvements (with&#10;respect to neural network-based methods) in terms of computational efficiency&#10;(it can be implemented without the use of GPUs), number of training samples&#10;needed for training, as well as out-of-distribution generalization. The Python&#10;code for reproducing our results is available at&#10;https://github.com/rohdelab/rcdt_ns_classifier.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.07801" label="2405.07801">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning-Based Object Pose Estimation: A Comprehensive Survey" />
-          <attvalue for="1" value="  Object pose estimation is a fundamental computer vision problem with broad&#10;applications in augmented reality and robotics. Over the past decade, deep&#10;learning models, due to their superior accuracy and robustness, have&#10;increasingly supplanted conventional algorithms reliant on engineered point&#10;pair features. Nevertheless, several challenges persist in contemporary&#10;methods, including their dependency on labeled training data, model&#10;compactness, robustness under challenging conditions, and their ability to&#10;generalize to novel unseen objects. A recent survey discussing the progress&#10;made on different aspects of this area, outstanding challenges, and promising&#10;future directions, is missing. To fill this gap, we discuss the recent advances&#10;in deep learning-based object pose estimation, covering all three formulations&#10;of the problem, \emph{i.e.}, instance-level, category-level, and unseen object&#10;pose estimation. Our survey also covers multiple input data modalities,&#10;degrees-of-freedom of output poses, object properties, and downstream tasks,&#10;providing the readers with a holistic understanding of this field.&#10;Additionally, it discusses training paradigms of different domains, inference&#10;modes, application areas, evaluation metrics, and benchmark datasets, as well&#10;as reports the performance of current state-of-the-art methods on these&#10;benchmarks, thereby facilitating the readers in selecting the most suitable&#10;method for their application. Finally, the survey identifies key challenges,&#10;reviews the prevailing trends along with their pros and cons, and identifies&#10;promising directions for future research. We also keep tracing the latest works&#10;at https://github.com/CNJianLiu/Awesome-Object-Pose-Estimation.&#10;" />
-          <attvalue for="2" value="&#10;O{bject} pose estimation is a fundamental computer vision problem that aims to estimate the pose of an object in a given image relative to the camera that captured the image. &#10;Object pose estimation is a crucial technology for augmented reality\cite{Gen6D, OnePose++, FoundationPose}, robotic manipulation\cite{STG6D, DGPF6D}, hand-object interaction\cite{lin2023harmonious, rezazadeh2023hierarchical}, etc. Depending on the application needs, the object pose is estimated up to varying degrees of freedom (DoF) such as 3DoF that only includes 3D rotation, 6DoF that additionally includes 3D translation, or 9DoF which includes estimating the 3D size of the object besides the 3D rotation and 3D translation. &#10;&#10;In the pre-deep learning era, many hand-crafted feature-based approaches such as SIFT\cite{SIFT}, FPFH\cite{FPFH}, VFH\cite{VFH}, and Point Pair Features (PPF)\cite{PPF, PPF1, PPF2, PPF3} were designed for object pose estimation. However, these methods exhibit deficiencies in accuracy and robustness when confronted with complex scenes\cite{xiang2017posecnn, wang2019densefusion}. These traditional methods have now been supplanted by data driven deep learning-based approaches that harness the power of deep neural networks to learn high-dimensional feature representations from data, leading to improved accuracy and robustness to handle complex environments.&#10;&#10;Deep learning-based object pose estimation methods can be divided into instance-level, category-level, and unseen object methods according to the problem formulation. Fig.~\ref{Fig0} shows a comparison of the three methods. Early methods were mainly instance-level \cite{wang2019densefusion, peng2019pvnet, he2020pvn3d, Li2019CDPN, Zakharov2019DPOD}, trained to estimate the pose of specific object instances. Instance-level methods can be further divided into correspondence-based, template-based, voting-based, and regression-based methods. Since instance-level methods are trained on instance-specific data, they can estimate pose with high precision for the given object instances. However, their generalization performance is poor because they are meant to be applied only to the instances on which they are trained. Moreover, many instance-level methods\cite{he2020pvn3d, he2021ffb6d} require CAD models of the objects. Recognizing these limitations, Wang et al. \cite{NOCS} proposed the first category-level object pose and size estimation method. They generalize to intra-class unseen objects without necessitating retraining and employing CAD models during inference. Subsequent category-level methods \cite{SGPA, DPDN, GPV-Pose, HS-Pose, IST-Net} can be divided into shape prior-based and shape prior-free methods. While improving the generalization ability within a category, these category-level methods still need to collect and label extensive training data for each object category. Moreover, these methods cannot generalize to unseen object categories. To this end, some unseen object pose estimation methods have been recently proposed \cite{Gen6D, MegaPose, GigaPose, SAM-6D, FoundationPose}, which can be further classified into CAD model-based and manual reference view-based methods. These methods further enhance the generalization of object pose estimation, i.e., they can be generalized to unseen objects without retraining. Nevertheless, they still need to obtain the object CAD model or annotate a few reference images of the object.&#10;&#10;Although significant progress has been made in the area of object pose estimation, several challenges persist in current methods, such as the reliance on labeled training data, difficulty in generalizing to novel unseen objects, model compactness, and robustness in challenging scenarios. To enable readers to swiftly grasp the current state-of-the-art (SOTA) in object pose estimation and facilitate further research in this direction, it is crucial to provide a thorough review of all the relevant problem formulations. A close examination of the existing academic literature reveals a significant gap when reviewing the various problem formulations in object pose estimation. Current prevailing reviews \cite{hoque2021comprehensive, Marullo_RGB_review, Fan_monocular_review, Du_Grasping_review, Guan_review} tend to exhibit a narrow focus, either confined to particular input modalities\cite{Marullo_RGB_review, Fan_monocular_review} or tethered to specific application domains\cite{Du_Grasping_review, Guan_review}. Furthermore, these reviews predominantly scrutinize instance-level and category-level methods, thus neglecting the exploration of the most practical problem formulation in the domain which is unseen object pose estimation. This hinders readers from gaining a comprehensive understanding of the area. For instance, Fan et al. \cite{Fan_monocular_review} provided valuable insights into RGB image-based object pose estimation. However, their focus is limited to a singular modality, hindering readers from comprehensively understanding methods across various input modalities. Conversely, Du et al. \cite{Du_Grasping_review} exclusively examined object pose estimation within the context of the robotic grasping task, which limits the readers to understand object pose estimation only from the perspective of a single specific application.&#10;&#10;To address the above problems, we present here a comprehensive survey of recent advancements in deep learning-based methods for object pose estimation. Our survey encompasses all problem formulations, including instance-level, category-level, and unseen object pose estimation, aiming to provide readers with a holistic understanding of this field. Additionally, we discuss different domain training paradigms, application areas, evaluation metrics, and benchmark datasets, as well as report the performance of state-of-the-art methods on these benchmarks, aiding readers in selecting suitable methods for their applications. Furthermore, we also highlight prevailing trends and discuss their strengths and weaknesses, as well as identify key challenges and promising avenues for future research. The taxonomy of this survey is shown in Fig. \ref{Fig1}.&#10;&#10;Our main contributions and highlights are as follows:&#10;\begin{itemize}&#10;\item We present a comprehensive survey of deep learning-based object pose estimation methods. This is the first survey that covers all three problem formulations in the domain, including instance-level, category-level, and unseen object pose estimation.&#10;&#10;\item Our survey covers popular input data modalities (RGB images, depth images, RGBD images), the different degrees of freedom (3DoF, 6DoF, 9DoF) in output poses, object properties (rigid, articulated) for the task of pose estimation as well as tracking. It is crucial to cover all these aspects in a single survey to give a complete picture to readers, an aspect overlooked by existing surveys which only cover a few of these aspects.&#10;&#10;\item We discuss different domain training paradigms, inference modes, application areas, evaluation metrics, and benchmark datasets as well as report the performance of existing SOTA methods on these benchmarks to help readers choose the most appropriate ones for deployment in their application.&#10;&#10;\item We highlight popular trends in the evolution of object pose estimation techniques over the past decade and discuss their strengths and weaknesses. We also identify key challenges that are still outstanding in object pose estimation along with promising research directions to guide future efforts.&#10;\end{itemize} &#10;&#10;The rest of this article is organized as follows. Sec. \ref{Datasets and Metrics} reviews the datasets and metrics used to evaluate the three categories of object pose estimation methods. We then review instance-level methods in Sec. \ref{Instance-Level Object Pose Estimation}, category-level methods in Sec. \ref{Category-Level Object Pose Estimation}, and unseen object pose estimation methods in Sec. \ref{Unseen Object Pose Estimation}. In the aforementioned three sections, we also discuss the training paradigms, inference modes, challenges, and popular trends associated with representative methods in the particular category. Next, Sec. \ref{Applications} reviews the common applications of object pose estimation. Finally, Sec. \ref{Conclusion and Future Direction} summarizes this article and provides an outlook on future research directions based on the challenges in the field.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Computer Vision Applications, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science, Object Pose Estimation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1901.02970" label="1901.02970">
-        <attvalues>
-          <attvalue for="0" value="Normalized Object Coordinate Space for Category-Level 6D Object Pose and&#10;  Size Estimation" />
-          <attvalue for="1" value="  The goal of this paper is to estimate the 6D pose and dimensions of unseen&#10;object instances in an RGB-D image. Contrary to &quot;instance-level&quot; 6D pose&#10;estimation tasks, our problem assumes that no exact object CAD models are&#10;available during either training or testing time. To handle different and&#10;unseen object instances in a given category, we introduce a Normalized Object&#10;Coordinate Space (NOCS)---a shared canonical representation for all possible&#10;object instances within a category. Our region-based neural network is then&#10;trained to directly infer the correspondence from observed pixels to this&#10;shared object representation (NOCS) along with other object information such as&#10;class label and instance mask. These predictions can be combined with the depth&#10;map to jointly estimate the metric 6D pose and dimensions of multiple objects&#10;in a cluttered scene. To train our network, we present a new context-aware&#10;technique to generate large amounts of fully annotated mixed reality data. To&#10;further improve our model and evaluate its performance on real data, we also&#10;provide a fully annotated real-world dataset with large environment and&#10;instance variation. Extensive experiments demonstrate that the proposed method&#10;is able to robustly estimate the pose and size of unseen object instances in&#10;real environments while also achieving state-of-the-art performance on standard&#10;6D pose estimation benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.06658" label="1905.06658">
-        <attvalues>
-          <attvalue for="0" value="Vision-based Robotic Grasping From Object Localization, Object Pose&#10;  Estimation to Grasp Estimation for Parallel Grippers: A Review" />
-          <attvalue for="1" value="  This paper presents a comprehensive survey on vision-based robotic grasping.&#10;We conclude three key tasks during vision-based robotic grasping, which are&#10;object localization, object pose estimation and grasp estimation. In detail,&#10;the object localization task contains object localization without&#10;classification, object detection and object instance segmentation. This task&#10;provides the regions of the target object in the input data. The object pose&#10;estimation task mainly refers to estimating the 6D object pose and includes&#10;correspondence-based methods, template-based methods and voting-based methods,&#10;which affords the generation of grasp poses for known objects. The grasp&#10;estimation task includes 2D planar grasp methods and 6DoF grasp methods, where&#10;the former is constrained to grasp from one direction. These three tasks could&#10;accomplish the robotic grasping with different combinations. Lots of object&#10;pose estimation methods need not object localization, and they conduct object&#10;localization and object pose estimation jointly. Lots of grasp estimation&#10;methods need not object localization and object pose estimation, and they&#10;conduct grasp estimation in an end-to-end manner. Both traditional methods and&#10;latest deep learning-based methods based on the RGB-D image inputs are reviewed&#10;elaborately in this survey. Related datasets and comparisons between&#10;state-of-the-art methods are summarized as well. In addition, challenges about&#10;vision-based robotic grasping and future directions in addressing these&#10;challenges are also pointed out.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.14291" label="2105.14291">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning on Monocular Object Pose Detection and Tracking: A&#10;  Comprehensive Overview" />
-          <attvalue for="1" value="  Object pose detection and tracking has recently attracted increasing&#10;attention due to its wide applications in many areas, such as autonomous&#10;driving, robotics, and augmented reality. Among methods for object pose&#10;detection and tracking, deep learning is the most promising one that has shown&#10;better performance than others. However, survey study about the latest&#10;development of deep learning-based methods is lacking. Therefore, this study&#10;presents a comprehensive review of recent progress in object pose detection and&#10;tracking that belongs to the deep learning technical route. To achieve a more&#10;thorough introduction, the scope of this study is limited to methods taking&#10;monocular RGB/RGBD data as input and covering three kinds of major tasks:&#10;instance-level monocular object pose detection, category-level monocular object&#10;pose detection, and monocular object pose tracking. In our work, metrics,&#10;datasets, and methods of both detection and tracking are presented in detail.&#10;Comparative results of current state-of-the-art methods on several publicly&#10;available datasets are also presented, together with insightful observations&#10;and inspiring future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.11700" label="2003.11700">
-        <attvalues>
-          <attvalue for="0" value="Classification of Chinese Handwritten Numbers with Labeled Projective&#10;  Dictionary Pair Learning" />
-          <attvalue for="1" value="  Dictionary learning is a cornerstone of image classification. We set out to&#10;address a longstanding challenge in using dictionary learning for&#10;classification; that is to simultaneously maximise the discriminability and&#10;sparse-representability power of the learned dictionaries. Upon this premise,&#10;we designed class-specific dictionaries incorporating three factors:&#10;discriminability, sparsity and classification error. We integrated these&#10;metrics into a unified cost function and adopted a new feature space, i.e.,&#10;histogram of oriented gradients (HOG), to generate the dictionary atoms. The&#10;rationale of using HOG features for designing the dictionaries is their&#10;strength in describing fine details of crowded images. The results of applying&#10;the proposed method in the classification of Chinese handwritten numbers&#10;demonstrated enhanced classification performance $(\sim98\%)$ compared to&#10;state-of-the-art deep learning techniques (i.e., SqueezeNet, GoogLeNet and&#10;MobileNetV2), but with a fraction of parameters. Furthermore, combination of&#10;the HOG features with dictionary learning enhances the accuracy by $11\%$&#10;compared to the case where only pixel domain data are used. These results were&#10;supported when the proposed method was applied to both Arabic and English&#10;handwritten number databases.&#10;" />
-          <attvalue for="2" value="&#10;Handwritten number recognition has remained a challenging research topic within pattern recognition \cite{10.3389/fncom.2015.00099}, still attracting many researchers \cite{qiao2018adaptive,9110900,farsiNumber}. A generic handwritten recognition system uses machine learning to interpret and recognise the received handwritten data from different sources, e.g. emails, bank cheque, papers, images, etc. Traditional recognition systems comprise two major stages: feature extraction and classification. The first stage transforms the input data into a space to accurately describe the data while reducing the amount or dimensionality of data, and the second stage assigns the input data to an associated class. Various techniques have been proposed for handwritten numbers classification where the main challenge is to learn efficient and comprehensive model capable of handling a wide diverse range of handwritten styles. A brief overview of the handwritten number recognition literature with focus on two main learning-based methods, i.e., deep learning and dictionary learning, as well as HOG-related techniques is provided in the following.&#10;&#10;Researchers proposed a classifier based on LeNet-5 and support vector machine (SVM) for handwritten number recognition \cite{yu2015handwritten,lauer2007trainable}. Other relevant studies \cite{choudhury2018handwritten}, involved utilising histogram of oriented gradients (HOG) and SVM for feature extraction and classification, respectively. In a recent relevant work \cite{9092067}, HOG and Gabor filter were used as descriptors for feature extraction from Arabic words, leading to promising results using a k-nearest neighbor (kNN) classifier. HOG focuses on the structure of an object and can extract gradient and orientation of edges in a given image. It was initially proposed for human detection, however, it has recently shown great influence for feature extraction from handwritten numbers \cite{choudhury2018handwritten}. Nevertheless, the efficacy of this powerful feature descriptor has not been thoroughly studied in this context, particularly for Chinese handwritten numbers.&#10;&#10;Recently, deep learning-based approaches have also been proposed to classify handwritten numbers. A deep unsupervised network was proposed in \cite{9110900} to learn invariant image representation from unlabeled data. The network architecture comprised a cascade of convolutional layers trained sequentially to represent multiple levels of features. A deep neural network classifier has been proposed in \cite{farsiNumber} for Farsi handwritten phone numbers recognition. In another work \cite{9084035}, Bengali handwritten number detection was performed using a deep structure called region proposal networks (RPN). Despite the promising performance of deep learning-based approaches, they require a large training dataset with numerous parameters to be tuned. Conventional methods, in comparison, are more appropriate for images with lower resolution, such as, handwritten characters \cite{luo2019multi}. &#10;Moreover, in spite of several works on Chinese handwritten characters recognition \cite{8563234,9085435,8869841}, there are no reported works on performance of deep learning for Chinese handwritten numbers. &#10;&#10;Dictionary learning is another learning-based approach that has achieved promising results in image classification \cite{DLbook/3265797}. The key success of this approach originates from the fact that a sample from a class of interest can be efficiently represented as a sparse linear combination of other samples of the same class \cite{wright2008robust}. Most conventional dictionary learning methods involve two major steps in which their performances are highly interdependent; sparse coding and dictionary update. The quality of obtained dictionary is crucial for generating a sufficient sparse representation, e.g., sparsity and grouping. Therefore, some dictionary learning methods attempt to adaptively design dictionaries and to efficiently represent the input data, e.g., gray-scale images \cite{huang2007sparse}. A naive way of building dictionaries is to stack all the training data into a matrix (the so-called dictionary), however, this approach leads to huge and redundant dictionaries; impractical to be stored or utilised for any purposes. Thus, many studies utilise machine learning techniques for obtaining dictionaries by extracting low-dimensional features domain from the given training data. The main aim of such techniques is to obtain a dictionary with approximately independent atoms that convey formative information of the input data. The utilised learning process, however, depends on the structure and nature of the input images too. The efficiency of a dictionary is also dependent on the total number of coefficients contained in the associated sparse vectors \cite{aharon2006k}. These sparse vectors together with dictionary atoms act as a coder for best approximating the data of interest. This idea can be extended from data representation to data classification, i.e., learning class-specific dictionaries. &#10;&#10;Dictionary learning has been applied for face recognition \cite{wright2008robust} and brain signal classification \cite{shin2012sparse}. A well-established work on dictionary learning for image classification is the sparse representation classifier (SRC) \cite{wright2008robust}. This technique uses sparse representation and learned dictionaries for classification of images in pixel domain. It is increasingly being extended and used for a wide variety of image analysis, representation and classification tasks. A recently developed supervised dictionary learning approach constructs image classes using a shared dictionary and discirmintave class models \cite{10.5555/2981780.2981909}.&#10;A limitation associated with the above approach is that the size of the dictionary increases when adding more classes and that degrades the classification performance. To scale to large training sets, researchers have proposed methods that learn a dictionary by merging its atoms by optimising a predefined objective function \cite{fulkerson2008localizing,winn2005object}. This mechanism decreases the mutual information between the dictionary atoms and the class labels \cite{fulkerson2008localizing}. Additionally, it minimises the loss of mutual information between the histogram of dictionary atoms over signal constituents \cite{winn2005object}. Despite the acceptable performance of these dictionaries, they tend to be computationally expensive due to the feature merging stage. &#10;Other approaches involve jointly learning the dictionary and classifier using an optimised objective function. K-singular value decomposition (K-SVD) method was utilised to train the dictionary \cite{aharon2006k}. This method has been applied to a variety of image processing problems, including inpainting missing pixels and image compression. The authors in \cite{zhang2010discriminative}, proposed a method for dictionary learning that jointly learns the classifier parameters and dictionary for face recognition. A method called label consistent K-SVD (LC-KSVD) was proposed to learn a discriminative dictionary for sparse coding \cite{jiang2011learning}, \cite{jiang2013label}. Introducing labels and classification error to the objective function has leveraged the performance of LC-KSVD method.&#10;&#10;Traditional dictionary learning methods merely rely on synthesis dictionaries in which the input data is in a sparse latent subspace. Synthesis dictionaries can well preserve the local structures of the data. In contrast, analysis dictionaries, which was recently introduced, rely on the assumption that the input data can be transformed into a latent sparse subspace by its corresponding dictionary \cite{8745700}. Analysis dictionary can produce sparse representation of data via simple data transformation, i.e. linear projection (simple dot product) without applying $\ell_0$/$\ell_1$ minimisation, which are considered computationally expensive operations due to their non-convex nature. For instance, an analysis discriminative dictionary learning has been proposed in \cite{8038251} to process two-dimensional images. The method imposes a sparse $\ell_{2,1}$-norm constraint on the coding coefficients and attempts to learn dictionaries, representations, and linear classifiers as discriminant as possible. Recently, projective dictionaries pair learning (DPL) was proposed where both types of dictionaries, i.e. analysis (for generating discriminative code by linear projection) and synthesis (for reconstructing the data) were used \cite{gu2014projective}. It benefits from an analysis-synthesis dictionary pair that avoids the need for utilising $\ell_{0}$-norm or $\ell_{1}$-norm minimisation. DPL has shown promising performance on face recognition application over state-of-the-art techniques. We recently applied this method for classification of brain activities in a brain-computer interfacing application \cite{ameri2016projective}. In addition, we proposed an extension to the DPL method, called incoherent dictionary pair learning (InDPL), for the classification of Chinese handwritten numbers \cite{abolghasemi2018incoherent}. InDPL adds a dictionary incoherence penalty to the DPL cost function in order to increase the discriminability and thus improving the classification performance. &#10;&#10;Some recent works have addressed combination of dictionary learning and deep learning. Deep dictionary learning was proposed in \cite{7779008} for building deeper architectures using the layers of dictionary learning. A method called deep micro-dictionary learning and coding network was proposed in \cite{8658671} which includes most of the standard deep learning layers (pooling, fully, connected, input/output, etc.). However, the deep learning architecture is augmented by replacing fundamental convolutional layers with a novel compound dictionary learning and coding layers. In \cite{1446833}, scalability and speed of deep learning were combined with dictionary learning to significantly reduce the number of parameters. This convolutional dictionary learning based auto-encoder was proposed for natural exponential-family distributions such as image denoising and neural spiking data analysis.&#10;&#10;In this paper, we further extend the DPL method and present a Chinese handwritten numbers recognition system that exploits the class labels information within the minimisation process. In our proposed labeled projective DPL (LpDPL), both synthesis and analysis dictionaries as well as class labels are used to calculate the sparse codes. A solution based on alternative minimisation is proposed that provides optimum trade-off between sparsity and grouping effect without using $\ell_0$ and $\ell_1$ regularisers. Instead of using raw pixel domain information as input to dictionary learning stage, here, we propose to use HOG (histogram of oriented gradient) descriptors. To the best of our knowledge, this is the first work reporting the use of HOG features in the context of dictionary learning. The motivation behind opting HOG is its robustness demonstrated in digit/character recognition applications as reviewed earlier in this section. Here, we extract and embed the HOG-based features into dictionary learning process. In addition, we mathematically utilise the classification labels information and propose a novel cost function for simultaneous dictionary learning and discrimination. Finally, to quantitatively evaluate performance, we apply several deep neural network architectures in addition to other well-established dictionary learning methods, and further compare the efficiency of two learning-based methods, i.e.``dictionary learning'' and ``deep learning'' methods. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Handwritten Digit Recognition, Dictionary Learning, Pattern Recognition, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1808.04134" label="1808.04134">
-        <attvalues>
-          <attvalue for="0" value="DenseRAN for Offline Handwritten Chinese Character Recognition" />
-          <attvalue for="1" value="  Recently, great success has been achieved in offline handwritten Chinese&#10;character recognition by using deep learning methods. Chinese characters are&#10;mainly logographic and consist of basic radicals, however, previous research&#10;mostly treated each Chinese character as a whole without explicitly considering&#10;its internal two-dimensional structure and radicals. In this study, we propose&#10;a novel radical analysis network with densely connected architecture (DenseRAN)&#10;to analyze Chinese character radicals and its two-dimensional structures&#10;simultaneously. DenseRAN first encodes input image to high-level visual&#10;features by employing DenseNet as an encoder. Then a decoder based on recurrent&#10;neural networks is employed, aiming at generating captions of Chinese&#10;characters by detecting radicals and two-dimensional structures through&#10;attention mechanism. The manner of treating a Chinese character as a&#10;composition of two-dimensional structures and radicals can reduce the size of&#10;vocabulary and enable DenseRAN to possess the capability of recognizing unseen&#10;Chinese character classes, only if the corresponding radicals have been seen in&#10;training set. Evaluated on ICDAR-2013 competition database, the proposed&#10;approach significantly outperforms whole-character modeling approach with a&#10;relative character error rate (CER) reduction of 18.54%. Meanwhile, for the&#10;case of recognizing 3277 unseen Chinese characters in CASIA-HWDB1.2 database,&#10;DenseRAN can achieve a character accuracy of about 41% while the traditional&#10;whole-character method has no capability to handle them.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.06347" label="1811.06347">
-        <attvalues>
-          <attvalue for="0" value="Deep Template Matching for Offline Handwritten Chinese Character&#10;  Recognition" />
-          <attvalue for="1" value="  Just like its remarkable achievements in many computer vision tasks, the&#10;convolutional neural networks (CNN) provide an end-to-end solution in&#10;handwritten Chinese character recognition (HCCR) with great success. However,&#10;the process of learning discriminative features for image recognition is&#10;difficult in cases where little data is available. In this paper, we propose a&#10;novel method for learning siamese neural network which employ a special&#10;structure to predict the similarity between handwritten Chinese characters and&#10;template images. The optimization of siamese neural network can be treated as a&#10;simple binary classification problem. When the training process has been&#10;finished, the powerful discriminative features help us to generalize the&#10;predictive power not just to new data, but to entirely new classes that never&#10;appear in the training set. Experiments performed on the ICDAR-2013 offline&#10;HCCR datasets have shown that the proposed method has a very promising&#10;generalization ability to the new classes that never appear in the training&#10;set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.04899" label="1807.04899">
-        <attvalues>
-          <attvalue for="0" value="Analysis Dictionary Learning based Classification: Structure for&#10;  Robustness" />
-          <attvalue for="1" value="  A discriminative structured analysis dictionary is proposed for the&#10;classification task. A structure of the union of subspaces (UoS) is integrated&#10;into the conventional analysis dictionary learning to enhance the capability of&#10;discrimination. A simple classifier is also simultaneously included into the&#10;formulated functional to ensure a more complete consistent classification. The&#10;solution of the algorithm is efficiently obtained by the linearized alternating&#10;direction method of multipliers. Moreover, a distributed structured analysis&#10;dictionary learning is also presented to address large scale datasets. It can&#10;group-(class-) independently train the structured analysis dictionaries by&#10;different machines/cores/threads, and therefore avoid a high computational&#10;cost. A consensus structured analysis dictionary and a global classifier are&#10;jointly learned in the distributed approach to safeguard the discriminative&#10;power and the efficiency of classification. Experiments demonstrate that our&#10;method achieves a comparable or better performance than the state-of-the-art&#10;algorithms in a variety of visual classification tasks. In addition, the&#10;training and testing computational complexity are also greatly reduced.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.11543" label="1905.11543">
-        <attvalues>
-          <attvalue for="0" value="Jointly Learning Structured Analysis Discriminative Dictionary and&#10;  Analysis Multiclass Classifier" />
-          <attvalue for="1" value="  In this paper, we propose an analysis mechanism based structured Analysis&#10;Discriminative Dictionary Learning (ADDL) framework. ADDL seamlessly integrates&#10;the analysis discriminative dictionary learning, analysis representation and&#10;analysis classifier training into a unified model. The applied analysis&#10;mechanism can make sure that the learnt dictionaries, representations and&#10;linear classifiers over different classes are independent and discriminating as&#10;much as possible. The dictionary is obtained by minimizing a reconstruction&#10;error and an analytical incoherence promoting term that encourages the&#10;sub-dictionaries associated with different classes to be independent. To obtain&#10;the representation coefficients, ADDL imposes a sparse l2,1-norm constraint on&#10;the coding coefficients instead of using l0 or l1-norm, since the l0 or l1-norm&#10;constraint applied in most existing DL criteria makes the training phase time&#10;consuming. The codes-extraction projection that bridges data with the sparse&#10;codes by extracting special features from the given samples is calculated via&#10;minimizing a sparse codes approximation term. Then we compute a linear&#10;classifier based on the approximated sparse codes by an analysis mechanism to&#10;simultaneously consider the classification and representation powers. Thus, the&#10;classification approach of our model is very efficient, because it can avoid&#10;the extra time-consuming sparse reconstruction process with trained dictionary&#10;for each new test data as most existing DL algorithms. Simulations on real&#10;image databases demonstrate that our ADDL model can obtain superior performance&#10;over other state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.04185" label="1809.04185">
-        <attvalues>
-          <attvalue for="0" value="Deep Micro-Dictionary Learning and Coding Network" />
-          <attvalue for="1" value="  In this paper, we propose a novel Deep Micro-Dictionary Learning and Coding&#10;Network (DDLCN). DDLCN has most of the standard deep learning layers (pooling,&#10;fully, connected, input/output, etc.) but the main difference is that the&#10;fundamental convolutional layers are replaced by novel compound dictionary&#10;learning and coding layers. The dictionary learning layer learns an&#10;over-complete dictionary for the input training data. At the deep coding layer,&#10;a locality constraint is added to guarantee that the activated dictionary bases&#10;are close to each other. Next, the activated dictionary atoms are assembled&#10;together and passed to the next compound dictionary learning and coding layers.&#10;In this way, the activated atoms in the first layer can be represented by the&#10;deeper atoms in the second dictionary. Intuitively, the second dictionary is&#10;designed to learn the fine-grained components which are shared among the input&#10;dictionary atoms. In this way, a more informative and discriminative low-level&#10;representation of the dictionary atoms can be obtained. We empirically compare&#10;the proposed DDLCN with several dictionary learning methods and deep learning&#10;architectures. The experimental results on four popular benchmark datasets&#10;demonstrate that the proposed DDLCN achieves competitive results compared with&#10;state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.03211" label="1907.03211">
-        <attvalues>
-          <attvalue for="0" value="Convolutional dictionary learning based auto-encoders for natural&#10;  exponential-family distributions" />
-          <attvalue for="1" value="  We introduce a class of auto-encoder neural networks tailored to data from&#10;the natural exponential family (e.g., count data). The architectures are&#10;inspired by the problem of learning the filters in a convolutional generative&#10;model with sparsity constraints, often referred to as convolutional dictionary&#10;learning (CDL). Our work is the first to combine ideas from convolutional&#10;generative models and deep learning for data that are naturally modeled with a&#10;non-Gaussian distribution (e.g., binomial and Poisson). This perspective&#10;provides us with a scalable and flexible framework that can be re-purposed for&#10;a wide range of tasks and assumptions on the generative model. Specifically,&#10;the iterative optimization procedure for solving CDL, an unsupervised task, is&#10;mapped to an unfolded and constrained neural network, with iterative&#10;adjustments to the inputs to account for the generative distribution. We also&#10;show that the framework can easily be extended for discriminative training,&#10;appropriate for a supervised task. We demonstrate 1) that fitting the&#10;generative model to learn, in an unsupervised fashion, the latent stimulus that&#10;underlies neural spiking data leads to better goodness-of-fit compared to other&#10;baselines, 2) competitive performance compared to state-of-the-art algorithms&#10;for supervised Poisson image denoising, with significantly fewer parameters,&#10;and 3) gradient dynamics of shallow binomial auto-encoder.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.11810" label="2107.11810">
-        <attvalues>
-          <attvalue for="0" value="Efficient Large Scale Inlier Voting for Geometric Vision Problems" />
-          <attvalue for="1" value="  Outlier rejection and equivalently inlier set optimization is a key&#10;ingredient in numerous applications in computer vision such as filtering&#10;point-matches in camera pose estimation or plane and normal estimation in point&#10;clouds. Several approaches exist, yet at large scale we face a combinatorial&#10;explosion of possible solutions and state-of-the-art methods like RANSAC, Hough&#10;transform or Branch&amp;Bound require a minimum inlier ratio or prior knowledge to&#10;remain practical. In fact, for problems such as camera posing in very large&#10;scenes these approaches become useless as they have exponential runtime growth&#10;if these conditions aren't met. To approach the problem we present a efficient&#10;and general algorithm for outlier rejection based on &quot;intersecting&quot;&#10;$k$-dimensional surfaces in $R^d$. We provide a recipe for casting a variety of&#10;geometric problems as finding a point in $R^d$ which maximizes the number of&#10;nearby surfaces (and thus inliers). The resulting algorithm has linear&#10;worst-case complexity with a better runtime dependency in the approximation&#10;factor than competing algorithms while not requiring domain specific bounds.&#10;This is achieved by introducing a space decomposition scheme that bounds the&#10;number of computations by successively rounding and grouping samples. Our&#10;recipe (and open-source code) enables anybody to derive such fast approaches to&#10;new problems across a wide range of domains. We demonstrate the versatility of&#10;the approach on several camera posing problems with a high number of matches at&#10;low inlier ratio achieving state-of-the-art results at significantly lower&#10;processing times.&#10;" />
-          <attvalue for="2" value="&#10;Whether geometric verification of point matches for absolute pose, homography estimation or normal estimation in point clouds, outlier rejection is a key ingredient in numerous applications in computer vision.&#10;RANSAC~\cite{FB81} is a common choice for the task as it has good quality/runtime tradeoffs for many practical problems.&#10;RANSAC however doesn't provide an optimality guarantee, requires a lower bound on the inlier ratio and its runtime increases exponentially with the outlier ratio making it unusable for problems with few inliers.&#10;In contrast Branch\&amp;Bound methods~\cite{B03,CPKH20} are guaranteed to find the optimal solution, yet their worst-case runtime equals exhaustive search in parameter space. Their practical use depends on the quality of the bounds which are problem specific and notoriously hard to find.&#10;&#10;We propose an alternative approach with linear runtime independent of the inlier ratio and that doesn't require a lower bound on the fraction of inliers. &#10;We base our work on the efficient general algorithm for solving geometric incidence problems proposed in \cite{AKKSZ19}.&#10;In a nutshell the authors show that voting \cite{ZSP15} is equivalent to finding a point of maximum depth, i.e. to find a point in $\reals^d$ which is close to as many surfaces (and thus inliers) as possible among a given set of $k$-surfaces in $\reals^d$.&#10;We demonstrate an effective solution to outlier removal problems by transforming them to the surface incidence framework of~\cite{AKKSZ19}.&#10;Our contributions:&#10;&#10;\begin{itemize}&#10; \item[--] We introduce the concept of general voting and its relation to the approximate incidences of~\cite{AKKSZ19} for outlier removal to a wider computer vision audience and demonstrate its use for camera posing, ray intersectionand geometric model fitting.&#10; \item[--] All approaches have linear complexity and are therefore applicable to very large problems, which are infeasible to solve with RANSAC~\cite{FB81}. The worst case complexity (and performance in practice) is always better than other methods (voting~\cite{HOU62} and B\&amp;B~\cite{HD60}) for sufficiently large inputs.&#10; \item[--] We demonstrate the scalability and versatility using a generalization of \cite{ZSP15,SEKO16} where we remove the requirement on a calibrated camera and known gravity direction, but instead solve for these unknowns.&#10; \item[--] We compare to the state-of-the-art~\cite{CPKH20} for 6 degree-of-freedom (6DoF) camera pose estimation and show that our approach is optimal and faster in practice. In comparison to~\cite{CPKH20} our surface intersectionalgorithm provides a tight upper bound on the score without problem specific knowledge.&#10; \item[--] We provide example-derivations and open-source code which serve as a tutorial for applying~\cite{AKKSZ19} to a range of outlier removal problems in computer vision.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Geometric Algorithms, Computer Vision, Engineering, Mathematics, Computer Science, Geometry, Outlier Rejection" />
-        </attvalues>
-      </node>
-      <node id="1903.07047" label="1903.07047">
-        <attvalues>
-          <attvalue for="0" value="General techniques for approximate incidences and their application to&#10;  the camera posing problem" />
-          <attvalue for="1" value="  We consider the classical camera pose estimation problem that arises in many&#10;computer vision applications, in which we are given n 2D-3D correspondences&#10;between points in the scene and points in the camera image (some of which are&#10;incorrect associations), and where we aim to determine the camera pose (the&#10;position and orientation of the camera in the scene) from this data. We&#10;demonstrate that this posing problem can be reduced to the problem of computing&#10;{\epsilon}-approximate incidences between two-dimensional surfaces (derived&#10;from the input correspondences) and points (on a grid) in a four-dimensional&#10;pose space. Similar reductions can be applied to other camera pose problems, as&#10;well as to similar problems in related application areas. We describe and&#10;analyze three techniques for solving the resulting {\epsilon}-approximate&#10;incidences problem in the context of our camera posing application. The first&#10;is a straightforward assignment of surfaces to the cells of a grid (of&#10;side-length {\epsilon}) that they intersect. The second is a variant of a&#10;primal-dual technique, recently introduced by a subset of the authors [2] for&#10;different (and simpler) applications. The third is a non-trivial generalization&#10;of a data structure Fonseca and Mount [3], originally designed for the case of&#10;hyperplanes. We present and analyze this technique in full generality, and then&#10;apply it to the camera posing problem at hand. We compare our methods&#10;experimentally on real and synthetic data. Our experiments show that for the&#10;typical values of n and {\epsilon}, the primal-dual method is the fastest, also&#10;in practice.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.11365" label="2305.11365">
-        <attvalues>
-          <attvalue for="0" value="Enhancing Transformer Backbone for Egocentric Video Action Segmentation" />
-          <attvalue for="1" value="  Egocentric temporal action segmentation in videos is a crucial task in&#10;computer vision with applications in various fields such as mixed reality,&#10;human behavior analysis, and robotics. Although recent research has utilized&#10;advanced visual-language frameworks, transformers remain the backbone of action&#10;segmentation models. Therefore, it is necessary to improve transformers to&#10;enhance the robustness of action segmentation models. In this work, we propose&#10;two novel ideas to enhance the state-of-the-art transformer for action&#10;segmentation. First, we introduce a dual dilated attention mechanism to&#10;adaptively capture hierarchical representations in both local-to-global and&#10;global-to-local contexts. Second, we incorporate cross-connections between the&#10;encoder and decoder blocks to prevent the loss of local context by the decoder.&#10;We also utilize state-of-the-art visual-language representation learning&#10;techniques to extract richer and more compact features for our transformer. Our&#10;proposed approach outperforms other state-of-the-art methods on the Georgia&#10;Tech Egocentric Activities (GTEA) and HOI4D Office Tools datasets, and we&#10;validate our introduced components with ablation studies. The source code and&#10;supplementary materials are publicly available on&#10;https://www.sail-nu.com/dxformer.&#10;" />
-          <attvalue for="2" value="&#10;Automated detection and segmentation of human activities from an egocentric perspective have numerous applications in fields such as mixed reality, human behavior analysis, and robotics \cite{lea2017temporal,kwon2022context}. However, egocentric action segmentation is particularly challenging due to several factors. First, the videos are untrimmed and can span several minutes, making it difficult to assign an action label to each frame accurately. Second, egocentric videos often have occlusions, where the camera wearer's body or objects in the foreground obstruct the view of the action. Random movement and camera motion can also lead to inconsistent viewpoints, making it challenging to track actions across frames. Third, active hand-object interactions, which are prevalent in egocentric videos, can result in actions slightly outside the frame, further complicating the segmentation task. To address these challenges, action segmentation methods for egocentric videos focus on modeling the temporal relations among frames using pre-extracted frame-wise feature sequences, while also considering the unique characteristics of egocentric videos.&#10;&#10;Recent advances in action segmentation models have shown promising results through the use of cutting-edge visual language feature representations \cite{li2022bridge, wang2021actionclip}. However, there remains scope for improvement in the transformer backbone, which serves as the fundamental component of these models. Specifically, enhancing the transformer backbone can aid in precise action identification by effectively attending to salient regions within the video. A strengthened attention mechanism within the improved transformer plays a pivotal role in capturing temporal dependencies, ultimately leading to improved model accuracy.&#10;&#10;In this paper, we introduce DXFormer, a new transformer-based architecture that improves the state-of-the-art transformer backbone for action segmentation by introducing a dual dilated attention mechanism and cross-connections between encoder and decoder blocks. In addition, we build on advanced visual-language representation learning approaches to extract richer and more compact characteristics for the transformer. We evaluate DXFormer on two challenging egocentric video datasets and show that it outperforms other state-of-the-art action segmentation methods both quantitatively and qualitatively. Furthermore, we conduct an ablation study to validate the effectiveness of newly added components in improving the results with respect to various metrics.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Transformer Models, Mathematics, Artificial Intelligence, Action Segmentation, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1611.05267" label="1611.05267">
-        <attvalues>
-          <attvalue for="0" value="Temporal Convolutional Networks for Action Segmentation and Detection" />
-          <attvalue for="1" value="  The ability to identify and temporally segment fine-grained human actions&#10;throughout a video is crucial for robotics, surveillance, education, and&#10;beyond. Typical approaches decouple this problem by first extracting local&#10;spatiotemporal features from video frames and then feeding them into a temporal&#10;classifier that captures high-level temporal patterns. We introduce a new class&#10;of temporal models, which we call Temporal Convolutional Networks (TCNs), that&#10;use a hierarchy of temporal convolutions to perform fine-grained action&#10;segmentation or detection. Our Encoder-Decoder TCN uses pooling and upsampling&#10;to efficiently capture long-range temporal patterns whereas our Dilated TCN&#10;uses dilated convolutions. We show that TCNs are capable of capturing action&#10;compositions, segment durations, and long-range dependencies, and are over a&#10;magnitude faster to train than competing LSTM-based Recurrent Neural Networks.&#10;We apply these models to three challenging fine-grained datasets and show large&#10;improvements over the state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.12223" label="2204.12223">
-        <attvalues>
-          <attvalue for="0" value="Context-Aware Sequence Alignment using 4D Skeletal Augmentation" />
-          <attvalue for="1" value="  Temporal alignment of fine-grained human actions in videos is important for&#10;numerous applications in computer vision, robotics, and mixed reality.&#10;State-of-the-art methods directly learn image-based embedding space by&#10;leveraging powerful deep convolutional neural networks. While being&#10;straightforward, their results are far from satisfactory, the aligned videos&#10;exhibit severe temporal discontinuity without additional post-processing steps.&#10;The recent advancements in human body and hand pose estimation in the wild&#10;promise new ways of addressing the task of human action alignment in videos. In&#10;this work, based on off-the-shelf human pose estimators, we propose a novel&#10;context-aware self-supervised learning architecture to align sequences of&#10;actions. We name it CASA. Specifically, CASA employs self-attention and&#10;cross-attention mechanisms to incorporate the spatial and temporal context of&#10;human actions, which can solve the temporal discontinuity problem. Moreover, we&#10;introduce a self-supervised learning scheme that is empowered by novel 4D&#10;augmentation techniques for 3D skeleton representations. We systematically&#10;evaluate the key components of our method. Our experiments on three public&#10;datasets demonstrate CASA significantly improves phase progress and Kendall's&#10;Tau scores over the previous state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.14104" label="2203.14104">
-        <attvalues>
-          <attvalue for="0" value="Bridge-Prompt: Towards Ordinal Action Understanding in Instructional&#10;  Videos" />
-          <attvalue for="1" value="  Action recognition models have shown a promising capability to classify human&#10;actions in short video clips. In a real scenario, multiple correlated human&#10;actions commonly occur in particular orders, forming semantically meaningful&#10;human activities. Conventional action recognition approaches focus on analyzing&#10;single actions. However, they fail to fully reason about the contextual&#10;relations between adjacent actions, which provide potential temporal logic for&#10;understanding long videos. In this paper, we propose a prompt-based framework,&#10;Bridge-Prompt (Br-Prompt), to model the semantics across adjacent actions, so&#10;that it simultaneously exploits both out-of-context and contextual information&#10;from a series of ordinal actions in instructional videos. More specifically, we&#10;reformulate the individual action labels as integrated text prompts for&#10;supervision, which bridge the gap between individual action semantics. The&#10;generated text prompts are paired with corresponding video clips, and together&#10;co-train the text encoder and the video encoder via a contrastive approach. The&#10;learned vision encoder has a stronger capability for ordinal-action-related&#10;downstream tasks, e.g. action segmentation and human activity recognition. We&#10;evaluate the performances of our approach on several video datasets: Georgia&#10;Tech Egocentric Activities (GTEA), 50Salads, and the Breakfast dataset.&#10;Br-Prompt achieves state-of-the-art on multiple benchmarks. Code is available&#10;at https://github.com/ttlmh/Bridge-Prompt&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00329" label="1908.00329">
-        <attvalues>
-          <attvalue for="0" value="Physical Cue based Depth-Sensing by Color Coding with Deaberration&#10;  Network" />
-          <attvalue for="1" value="  Color-coded aperture (CCA) methods can physically measure the depth of a&#10;scene given by physical cues from a single-shot image of a monocular camera.&#10;However, they are vulnerable to actual lens aberrations in real scenes because&#10;they assume an ideal lens for simplifying algorithms. In this paper, we propose&#10;physical cue-based deep learning for CCA photography. To address actual lens&#10;aberrations, we developed a deep deaberration network (DDN) that is&#10;additionally equipped with a self-attention mechanism of position and color&#10;channels to efficiently learn the lens aberration. Furthermore, a new Bayes L1&#10;loss function based on Bayesian deep learning enables to handle the uncertainty&#10;of depth estimation more accurately. Quantitative and qualitative comparisons&#10;demonstrate that our method is superior to conventional methods including real&#10;outdoor scenes. Furthermore, compared to a long-baseline stereo camera, the&#10;proposed method provides an error-free depth map at close range, as there is no&#10;blind spot between the left and right cameras.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Compared with multi-shot depth measurement methods such as structure from motion (SfM)~\cite{agarwal2009building} and depth from defocus (DfD)~\cite{subbarao1994depth,tang2017depth,guo2017focal}, a single-shot method is suitable for moving objects. One of the most successful single-shot methods is deep monocular depth estimation. Despite the remarkable progress of deep monocular depth estimation in recent years~\cite{eigen2014depth,godard2017unsupervised,fu2018deep}, it cannot estimate a correct depth map without sufficient contextual information due to the lack of a physical depth cue, for instance, in a scene without the ground. &#10;&#10;Instead of utilizing contextual information, color-coded aperture (CCA) methods can acquire a depth map based on a physical depth cue encoded in a single-shot image by inserting different types of color filters~\cite{bando2008extracting,kim2012multifocusing,chakrabarti2012depth,lee2013single,martinello2015dual,paramonov2016depth,moriuchi201723} into the lens aperture as shown in {Figure~\ref{psf_aberration}}b. As shown in {Figure~\ref{psf_aberration}}a, color and blur radius vary according to the distance from the focus distance. Conventional CCA methods assume an ideal lens for simplifying analytical modeling of defocus blur. However, actual lenses have shift-variant point spread functions (PSFs) distorted according to the position of the image sensor by lens aberrations such as field curvature, coma or lateral chromatic aberration as shown in {Figure~\ref{psf_aberration}}e. Furthermore, the depth cues often disappear because of several uncertainties, such as saturation, soft shadow, dark color and large blur. These uncertainties are not treated distinctly in conventional methods~\cite{bando2008extracting,kim2012multifocusing,chakrabarti2012depth,lee2013single,martinello2015dual,paramonov2016depth,moriuchi201723,haim2018depth}. &#10;&#10;In this paper, we propose a physical cue-based deep learning to overcome the differences between the analytical model and the actual one. In order to estimate a correct depth map under shift-variant PSFs, we add positional information as an additional branch by a self-attention mechanism~\cite{zhang2018self}. It also couples additional color channels to solve dependency on object colors for handling various complex color pattern correctly. To handle various uncertainties, we improve the loss function based on Bayesian deep learning~\cite{kendall2017uncertainties} for stabilizing the training. As shown in {Figure~\ref{title_figure}}, We demonstrate that our method is superior to a conventional method in quantitative and qualitative experiments including various outdoor scenes. Furthermore, compared to a long-baseline stereo camera, the proposed method provides an error-free depth map at close range, as there is no blind spot between the left and right cameras. &#10;&#10;The contributions of this paper are as follows. {(a)} We propose a CNN-based depth estimation network that does not infer the depth from the contextual information but physically measures the depth given by an optical cue. {(b)} We add positional information as additional channels by a self-attention mechanism to handle shift-variant aberrations. {(c)} We train the network with additional color channels using many pictures taken by an actual lens to handle various complex color patterns correctly. {(d)} To handle various uncertainties, we propose Bayes L1 loss instead of the conventional heteroscedastic variance for stabilizing the training. &#10;There are two limitations to this paper. First, our method is not applicable to the regions with small gradients since there are no depth cues. Second, our method requires much computation time, for example, about 50 seconds (NVIDIA Geforce GTX 1080 ti), because of the patch-based architecture~\cite{bailer2018fast}. The improvement is left for future work. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Optics, Mathematics, Physics, Image Processing, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1805.03096" label="1805.03096">
-        <attvalues>
-          <attvalue for="0" value="Fast Feature Extraction with CNNs with Pooling Layers" />
-          <attvalue for="1" value="  In recent years, many publications showed that convolutional neural network&#10;based features can have a superior performance to engineered features. However,&#10;not much effort was taken so far to extract local features efficiently for a&#10;whole image. In this paper, we present an approach to compute patch-based local&#10;feature descriptors efficiently in presence of pooling and striding layers for&#10;whole images at once. Our approach is generic and can be applied to nearly all&#10;existing network architectures. This includes networks for all local feature&#10;extraction tasks like camera calibration, Patchmatching, optical flow&#10;estimation and stereo matching. In addition, our approach can be applied to&#10;other patch-based approaches like sliding window object detection and&#10;recognition. We complete our paper with a speed benchmark of popular CNN based&#10;feature extraction approaches applied on a whole image, with and without our&#10;speedup, and example code (for Torch) that shows how an arbitrary CNN&#10;architecture can be easily converted by our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.10110" label="2105.10110">
-        <attvalues>
-          <attvalue for="0" value="Guidance and Teaching Network for Video Salient Object Detection" />
-          <attvalue for="1" value="  Owing to the difficulties of mining spatial-temporal cues, the existing&#10;approaches for video salient object detection (VSOD) are limited in&#10;understanding complex and noisy scenarios, and often fail in inferring&#10;prominent objects. To alleviate such shortcomings, we propose a simple yet&#10;efficient architecture, termed Guidance and Teaching Network (GTNet), to&#10;independently distil effective spatial and temporal cues with implicit guidance&#10;and explicit teaching at feature- and decision-level, respectively. To be&#10;specific, we (a) introduce a temporal modulator to implicitly bridge features&#10;from motion into the appearance branch, which is capable of fusing cross-modal&#10;features collaboratively, and (b) utilise motion-guided mask to propagate the&#10;explicit cues during the feature aggregation. This novel learning strategy&#10;achieves satisfactory results via decoupling the complex spatial-temporal cues&#10;and mapping informative cues across different modalities. Extensive experiments&#10;on three challenging benchmarks show that the proposed method can run at ~28&#10;fps on a single TITAN Xp GPU and perform competitively against 14 cutting-edge&#10;baselines.&#10;" />
-          <attvalue for="2" value="&#10;Video salient object detection (VSOD) has been a longstanding research topic in the area of computer vision, which aims to predict conspicuous and attractive objects in a given video clip.&#10;It has been applied to autonomous cars, action segmentation, and video captioning.&#10;In recent years, much progresses~\cite{Fan2019ShiftingMA,Feichtenhofer2019SlowFastNF} have been witnessed in handling unconstrained videos, but there remains a large room to improve that has not yet been adequately explored.&#10;&#10;Motion (\eg, optical flow~\cite{Teed2020RAFTRA} and trajectory~\cite{Keuper2015MotionTS}) and appearance features (\eg, color~\cite{Wei2012GeodesicSU} and superpixel~\cite{Wang2015SaliencyawareGV} segmentation) are both crucial cues for understanding the dynamic salient object with an unconstrained background.&#10;Several works have been made to develop spatial-temporal convolution neural networks (CNNs) for learning discriminative appearance and motion features, in which recurrent memory~\cite{Li2018FlowGR,Song2018PyramidDD} and 3D convolution~\cite{ji2021pnsnet} are frequently used.&#10;However, they are hindered by the following problems.&#10;For the former, it is unable to handle spatial and temporal cues simultaneously. &#10;Besides, it only processes the inputs sequentially due to the transmissible temporal memory, and thus, its efficiency is largely limited.&#10;As for the latter, 3D CNNs are difficult to optimise if the number of temporal layers is large, due to exponential growth of the numerical solution space.&#10;In addition, it is overloaded by high computation cost ($\sim$1.5$\times$ memory cost than 2D CNNs).&#10;Thus, it is imperative to separately model a spatial and temporal representation scheme, which can bring considerable benefits for VSOD.&#10;&#10;To efficiently fuse the motion and appearance cues, research has explored different cross-modal strategies with separate branches, and has achieved encouraging results.&#10;Earlier works (\eg, \cite{jain2017fusionseg}) address this issue via encoding motion and appearance features individually and directly aggregating them.&#10;However, it may unintentionally introduce feature conflicts due to different characteristics between them.&#10;The other, more natural idea is to excavate the relationship between motion and appearance cues in a guided fashion.&#10;Thus, recent cutting-edge methods (\eg,\cite{Li2019MotionGA}) encourage utilising the motion-dominated features to guide the encoding/decoding of appearance-dominated features, achieving satisfactory results.&#10;However, these methods serve it as the problem of implicit feature propagation, which is primarily hindered by its inexplicable feature transmission/learning process.&#10;&#10;To alleviate the above concerns, we propose a framework, equipped with implicit guidance and explicit teaching strategies at feature- and decision-level, towards effective and efficient VSOD.&#10;As shown in~Fig.~\ref{fig:aggregation_strategy}, the proposed scheme serves the motion-dominated feature as teacher knowledge. It then uses these features to implicitly guide the encoding of appearance-dominated (\ie, student) features, while explicitly teaching the decoding of student partial decoder via the teacher partial decoder (T-PD).&#10;Attributing to these strategies, this efficient scheme can generate satisfactory results in challenging scenarios.&#10;&#10;To the best of our knowledge, \ourmodel~is the pioneering work to explore both implicit guidance and explicit teaching mechanism for VSOD.&#10;Our main contributions are as follows:&#10;\begin{itemize}&#10;&#10;\item We emphasise the importance of both implicit guidance and explicit teaching strategies for the spatial-temporal representations. It is based on the observation that motion-guided features and masks provide discriminative semantic and temporal cues without redundant structures, contributing to efficient decoding phase of an appearance-dominated branch.&#10;&#10;\item We introduce the temporal modulator that contains two sequential attention mechanisms from channel and spatial view, working in a deeply collaborative manner.&#10;&#10;\item Comprehensive experiments on 3 benchmarks demonstrate that the proposed method can run at $\sim$28 fps on a single TITAN Xp GPU and get competitive performance against 14 cutting-edge VSOD models using 3 metrics, making it a potential solution for practical application.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Architectures, Spatial Temporal Analysis, Computer Science, Video Object Detection" />
-        </attvalues>
-      </node>
-      <node id="1812.03982" label="1812.03982">
-        <attvalues>
-          <attvalue for="0" value="SlowFast Networks for Video Recognition" />
-          <attvalue for="1" value="  We present SlowFast networks for video recognition. Our model involves (i) a&#10;Slow pathway, operating at low frame rate, to capture spatial semantics, and&#10;(ii) a Fast pathway, operating at high frame rate, to capture motion at fine&#10;temporal resolution. The Fast pathway can be made very lightweight by reducing&#10;its channel capacity, yet can learn useful temporal information for video&#10;recognition. Our models achieve strong performance for both action&#10;classification and detection in video, and large improvements are pin-pointed&#10;as contributions by our SlowFast concept. We report state-of-the-art accuracy&#10;on major video recognition benchmarks, Kinetics, Charades and AVA. Code has&#10;been made available at: https://github.com/facebookresearch/SlowFast&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.12943" label="2206.12943">
-        <attvalues>
-          <attvalue for="0" value="Multi-view Feature Augmentation with Adaptive Class Activation Mapping" />
-          <attvalue for="1" value="  We propose an end-to-end-trainable feature augmentation module built for&#10;image classification that extracts and exploits multi-view local features to&#10;boost model performance. Different from using global average pooling (GAP) to&#10;extract vectorized features from only the global view, we propose to sample and&#10;ensemble diverse multi-view local features to improve model robustness. To&#10;sample class-representative local features, we incorporate a simple auxiliary&#10;classifier head (comprising only one 1$\times$1 convolutional layer) which&#10;efficiently and adaptively attends to class-discriminative local regions of&#10;feature maps via our proposed AdaCAM (Adaptive Class Activation Mapping).&#10;Extensive experiments demonstrate consistent and noticeable performance gains&#10;achieved by our multi-view feature augmentation module.&#10;" />
-          <attvalue for="2" value="&#10;Progresses of image classification were dramatically promoted by deep convolutional neural networks. NIN \cite{NIN} for the first time replaced traditional flattening-based fully-connected (FC) layers with global average pooling (GAP), which reduces model size and enables input images of arbitrary size. Since then, GAP is widely used in combination with more complex convolutional backbones. GoogLeNet \cite{googlenet} extracts and fuses multi-scale convolutional features to enrich feature representation. ResNet \cite{resnet} introduces residual learning to facilitate optimization of deep networks, improving model performance noticeably. Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}. Besides, networks were also specially designed for lightweight deployment by using more efficient convolution or convolutional blocks \cite{mobilenets,squeezenet}. &#10;&#10;However, all these advanced deep models only focus on delicate design of convolutional backbone networks, the common ground is applying GAP to obtain vectorized features which are then fed to a linear classifier head for final classification. Though effective and efficient, GAP extracts vectorized image representations simply by averaging cross the entire feature maps. In this way, the extracted image representations could be corrupted by background elements for images where the class-related object takes up only a fraction of the image. As Figure \ref{motivation} displays, even for images of the same class, the scale of class-related object varies a lot in ImageNet dataset. For images with a small scale of class-related object, GAP easily extracts image representations that bias towards features of the background or class-irrelevant objects, making the input features to the subsequent linear classifier head less class-representative. &#10;&#10;Based on this motivation, we replace GAP with a multi-view feature augmentation module (MV-FeaAug), which learns to locate region of class-related object on the final convolutional feature maps, and extracts multi-view image representations by sampling diverse local features around the attended class-discriminative region. The extracted multi-view local representations more precisely represent the class-related object of the image from diverse views, the ensemble of the predictions to all these representations are used as the final prediction of the entire image. Comparison between the image classification architecture with the general GAP and our MV-FeaAug counterpart is illustrated in Figure \ref{comparison_to_GAP}. &#10;&#10;Apart from network architecture design, data augmentation, which acts as a regularizer to reduce overfitting, is also important for image classification. In addition to the basic position and color transformations, some advanced methods were proposed in recent years. Mixup \cite{mixup} blends two images and their corresponding labels with a random proportion as training samples. Cutout \cite{cutout} randomly removes a rectangular patch from images to guide the network to learn from local views. CutMix \cite{cutmix} randomly cuts and pastes rectangular patches among training images with their corresponding labels mixed up in proportion to patch area. Our method shares similar spirit in expanding training samples with these methods. But instead of image-level augmentation, our method extracts diverse multi-view local representations at feature space as augmented samples for subsequent classifier head, for which we call feature augmentation. Besides, our method takes full advantage of the augmented features at inference time by ensembling predictions to all the multi-view features, while such ensemble prediction property is not available for the above-mentioned data augmentation methods.&#10;&#10;Our method also relates to ``visual explanation'' of CNN models. Efforts have been made to visualize the evidence of class predictions made by CNN. CAM \cite{CAM} produces class activation maps that highlight class-discriminative regions of images with GAP and a single-layer classifier head. Grad-CAM \cite{Grad-CAM} extends such object localization ability to arbitrary CNN architectures by computing gradients of the logits with respect to intermediate feature maps. However, both CAM and Grad-CAM operate in a way of postprocessing since they require extraction (or computation) of weights (or gradients) associated to the target-class logit. Moreover, the attention maps produced by these two methods are conditioned on the ground truth label, which is not flexible in cases when the target image label is unknown. In this work, we propose AdaCAM that adaptively generates attention maps with simply a forward pass of the network without conditioning on the class label. The produced attention maps provide cues to sample diverse class-representative local features for feature augmentation.&#10;&#10;Another topic related to our method is attention mechanism, which has been widely applied to image classification recently. SENet \cite{SENet} built a Squeeze-and-Excitation module to selectively enhance or depress features in channel dimension. CBAM \cite{cbam} sequentially stacked two modules for channel-wise and spatial attentions respectively. SKNet \cite{SKNet} proposed a multi-branch attention module for adaptive convolution kernel size selection. GCNet \cite{gcnet} improved model performance by designing an efficient self-attention block that captures global context information. All these methods increase model size for incorporating additional attention modules. By contrast, our method also introduces attention mechanism by locating class-discriminative regions via AdaCAM, however, all the extra cost is an auxiliary classifier head comprised of only one 1$\times$1 convolutional (Conv$_{1\times1}$) layer. Furthermore, we demonstrate that a classification network with a deeper CNN backbone could be surpassed by a network with a shallower backbone combined with our MV-FeaAug module.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Feature Extraction, Deep Learning, Computer Science, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1312.4400" label="1312.4400">
-        <attvalues>
-          <attvalue for="0" value="Network In Network" />
-          <attvalue for="1" value="  We propose a novel deep network structure called &quot;Network In Network&quot; (NIN)&#10;to enhance model discriminability for local patches within the receptive field.&#10;The conventional convolutional layer uses linear filters followed by a&#10;nonlinear activation function to scan the input. Instead, we build micro neural&#10;networks with more complex structures to abstract the data within the receptive&#10;field. We instantiate the micro neural network with a multilayer perceptron,&#10;which is a potent function approximator. The feature maps are obtained by&#10;sliding the micro networks over the input in a similar manner as CNN; they are&#10;then fed into the next layer. Deep NIN can be implemented by stacking mutiple&#10;of the above described structure. With enhanced local modeling via the micro&#10;network, we are able to utilize global average pooling over feature maps in the&#10;classification layer, which is easier to interpret and less prone to&#10;overfitting than traditional fully connected layers. We demonstrated the&#10;state-of-the-art classification performances with NIN on CIFAR-10 and&#10;CIFAR-100, and reasonable performances on SVHN and MNIST datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.07146" label="1605.07146">
-        <attvalues>
-          <attvalue for="0" value="Wide Residual Networks" />
-          <attvalue for="1" value="  Deep residual networks were shown to be able to scale up to thousands of&#10;layers and still have improving performance. However, each fraction of a&#10;percent of improved accuracy costs nearly doubling the number of layers, and so&#10;training very deep residual networks has a problem of diminishing feature&#10;reuse, which makes these networks very slow to train. To tackle these problems,&#10;in this paper we conduct a detailed experimental study on the architecture of&#10;ResNet blocks, based on which we propose a novel architecture where we decrease&#10;depth and increase width of residual networks. We call the resulting network&#10;structures wide residual networks (WRNs) and show that these are far superior&#10;over their commonly used thin and very deep counterparts. For example, we&#10;demonstrate that even a simple 16-layer-deep wide residual network outperforms&#10;in accuracy and efficiency all previous deep residual networks, including&#10;thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR,&#10;SVHN, COCO, and significant improvements on ImageNet. Our code and models are&#10;available at https://github.com/szagoruyko/wide-residual-networks&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.06993" label="1608.06993">
-        <attvalues>
-          <attvalue for="0" value="Densely Connected Convolutional Networks" />
-          <attvalue for="1" value="  Recent work has shown that convolutional networks can be substantially&#10;deeper, more accurate, and efficient to train if they contain shorter&#10;connections between layers close to the input and those close to the output. In&#10;this paper, we embrace this observation and introduce the Dense Convolutional&#10;Network (DenseNet), which connects each layer to every other layer in a&#10;feed-forward fashion. Whereas traditional convolutional networks with L layers&#10;have L connections - one between each layer and its subsequent layer - our&#10;network has L(L+1)/2 direct connections. For each layer, the feature-maps of&#10;all preceding layers are used as inputs, and its own feature-maps are used as&#10;inputs into all subsequent layers. DenseNets have several compelling&#10;advantages: they alleviate the vanishing-gradient problem, strengthen feature&#10;propagation, encourage feature reuse, and substantially reduce the number of&#10;parameters. We evaluate our proposed architecture on four highly competitive&#10;object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet).&#10;DenseNets obtain significant improvements over the state-of-the-art on most of&#10;them, whilst requiring less computation to achieve high performance. Code and&#10;pre-trained models are available at https://github.com/liuzhuang13/DenseNet .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.04861" label="1704.04861">
-        <attvalues>
-          <attvalue for="0" value="MobileNets: Efficient Convolutional Neural Networks for Mobile Vision&#10;  Applications" />
-          <attvalue for="1" value="  We present a class of efficient models called MobileNets for mobile and&#10;embedded vision applications. MobileNets are based on a streamlined&#10;architecture that uses depth-wise separable convolutions to build light weight&#10;deep neural networks. We introduce two simple global hyper-parameters that&#10;efficiently trade off between latency and accuracy. These hyper-parameters&#10;allow the model builder to choose the right sized model for their application&#10;based on the constraints of the problem. We present extensive experiments on&#10;resource and accuracy tradeoffs and show strong performance compared to other&#10;popular models on ImageNet classification. We then demonstrate the&#10;effectiveness of MobileNets across a wide range of applications and use cases&#10;including object detection, finegrain classification, face attributes and large&#10;scale geo-localization.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.09412" label="1710.09412">
-        <attvalues>
-          <attvalue for="0" value="mixup: Beyond Empirical Risk Minimization" />
-          <attvalue for="1" value="  Large deep neural networks are powerful, but exhibit undesirable behaviors&#10;such as memorization and sensitivity to adversarial examples. In this work, we&#10;propose mixup, a simple learning principle to alleviate these issues. In&#10;essence, mixup trains a neural network on convex combinations of pairs of&#10;examples and their labels. By doing so, mixup regularizes the neural network to&#10;favor simple linear behavior in-between training examples. Our experiments on&#10;the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show&#10;that mixup improves the generalization of state-of-the-art neural network&#10;architectures. We also find that mixup reduces the memorization of corrupt&#10;labels, increases the robustness to adversarial examples, and stabilizes the&#10;training of generative adversarial networks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.04552" label="1708.04552">
-        <attvalues>
-          <attvalue for="0" value="Improved Regularization of Convolutional Neural Networks with Cutout" />
-          <attvalue for="1" value="  Convolutional neural networks are capable of learning powerful&#10;representational spaces, which are necessary for tackling complex learning&#10;tasks. However, due to the model capacity required to capture such&#10;representations, they are often susceptible to overfitting and therefore&#10;require proper regularization in order to generalize well. In this paper, we&#10;show that the simple regularization technique of randomly masking out square&#10;regions of input during training, which we call cutout, can be used to improve&#10;the robustness and overall performance of convolutional neural networks. Not&#10;only is this method extremely easy to implement, but we also demonstrate that&#10;it can be used in conjunction with existing forms of data augmentation and&#10;other regularizers to further improve model performance. We evaluate this&#10;method by applying it to current state-of-the-art architectures on the&#10;CIFAR-10, CIFAR-100, and SVHN datasets, yielding new state-of-the-art results&#10;of 2.56%, 15.20%, and 1.30% test error respectively. Code is available at&#10;https://github.com/uoguelph-mlrg/Cutout&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.04899" label="1905.04899">
-        <attvalues>
-          <attvalue for="0" value="CutMix: Regularization Strategy to Train Strong Classifiers with&#10;  Localizable Features" />
-          <attvalue for="1" value="  Regional dropout strategies have been proposed to enhance the performance of&#10;convolutional neural network classifiers. They have proved to be effective for&#10;guiding the model to attend on less discriminative parts of objects (e.g. leg&#10;as opposed to head of a person), thereby letting the network generalize better&#10;and have better object localization capabilities. On the other hand, current&#10;methods for regional dropout remove informative pixels on training images by&#10;overlaying a patch of either black pixels or random noise. Such removal is not&#10;desirable because it leads to information loss and inefficiency during&#10;training. We therefore propose the CutMix augmentation strategy: patches are&#10;cut and pasted among training images where the ground truth labels are also&#10;mixed proportionally to the area of the patches. By making efficient use of&#10;training pixels and retaining the regularization effect of regional dropout,&#10;CutMix consistently outperforms the state-of-the-art augmentation strategies on&#10;CIFAR and ImageNet classification tasks, as well as on the ImageNet&#10;weakly-supervised localization task. Moreover, unlike previous augmentation&#10;methods, our CutMix-trained ImageNet classifier, when used as a pretrained&#10;model, results in consistent performance gains in Pascal detection and MS-COCO&#10;image captioning benchmarks. We also show that CutMix improves the model&#10;robustness against input corruptions and its out-of-distribution detection&#10;performances. Source code and pretrained models are available at&#10;https://github.com/clovaai/CutMix-PyTorch .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.06586" label="1903.06586">
-        <attvalues>
-          <attvalue for="0" value="Selective Kernel Networks" />
-          <attvalue for="1" value="  In standard Convolutional Neural Networks (CNNs), the receptive fields of&#10;artificial neurons in each layer are designed to share the same size. It is&#10;well-known in the neuroscience community that the receptive field size of&#10;visual cortical neurons are modulated by the stimulus, which has been rarely&#10;considered in constructing CNNs. We propose a dynamic selection mechanism in&#10;CNNs that allows each neuron to adaptively adjust its receptive field size&#10;based on multiple scales of input information. A building block called&#10;Selective Kernel (SK) unit is designed, in which multiple branches with&#10;different kernel sizes are fused using softmax attention that is guided by the&#10;information in these branches. Different attentions on these branches yield&#10;different sizes of the effective receptive fields of neurons in the fusion&#10;layer. Multiple SK units are stacked to a deep network termed Selective Kernel&#10;Networks (SKNets). On the ImageNet and CIFAR benchmarks, we empirically show&#10;that SKNet outperforms the existing state-of-the-art architectures with lower&#10;model complexity. Detailed analyses show that the neurons in SKNet can capture&#10;target objects with different scales, which verifies the capability of neurons&#10;for adaptively adjusting their receptive field sizes according to the input.&#10;The code and models are available at https://github.com/implus/SKNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.11492" label="1904.11492">
-        <attvalues>
-          <attvalue for="0" value="GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond" />
-          <attvalue for="1" value="  The Non-Local Network (NLNet) presents a pioneering approach for capturing&#10;long-range dependencies, via aggregating query-specific global context to each&#10;query position. However, through a rigorous empirical analysis, we have found&#10;that the global contexts modeled by non-local network are almost the same for&#10;different query positions within an image. In this paper, we take advantage of&#10;this finding to create a simplified network based on a query-independent&#10;formulation, which maintains the accuracy of NLNet but with significantly less&#10;computation. We further observe that this simplified design shares similar&#10;structure with Squeeze-Excitation Network (SENet). Hence we unify them into a&#10;three-step general framework for global context modeling. Within the general&#10;framework, we design a better instantiation, called the global context (GC)&#10;block, which is lightweight and can effectively model the global context. The&#10;lightweight property allows us to apply it for multiple layers in a backbone&#10;network to construct a global context network (GCNet), which generally&#10;outperforms both simplified NLNet and SENet on major benchmarks for various&#10;recognition tasks. The code and configurations are released at&#10;https://github.com/xvjiarui/GCNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.05278" label="2210.05278">
-        <attvalues>
-          <attvalue for="0" value="EnsembleMOT: A Step towards Ensemble Learning of Multiple Object&#10;  Tracking" />
-          <attvalue for="1" value="  Multiple Object Tracking (MOT) has rapidly progressed in recent years.&#10;Existing works tend to design a single tracking algorithm to perform both&#10;detection and association. Though ensemble learning has been exploited in many&#10;tasks, i.e, classification and object detection, it hasn't been studied in the&#10;MOT task, which is mainly caused by its complexity and evaluation metrics. In&#10;this paper, we propose a simple but effective ensemble method for MOT, called&#10;EnsembleMOT, which merges multiple tracking results from various trackers with&#10;spatio-temporal constraints. Meanwhile, several post-processing procedures are&#10;applied to filter out abnormal results. Our method is model-independent and&#10;doesn't need the learning procedure. What's more, it can easily work in&#10;conjunction with other algorithms, e.g., tracklets interpolation. Experiments&#10;on the MOT17 dataset demonstrate the effectiveness of the proposed method.&#10;Codes are available at https://github.com/dyhBUPT/EnsembleMOT.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Multiple Object Tracking (MOT) aims to detect and track all specific classes of objects frame by frame,&#10;which plays an essential role in video analysis and understanding.&#10;In the past few years, the MOT task is dominated by the tracking-by-detection (TBD) paradigm \cite{wojke2017simple, du2022strongsort},&#10;which performs detection per frame and formulates the MOT problem as a data association task.&#10;Recently, some works integrate the detector and embedding model (i.e., appearance or motion embedding) into a unified framework,&#10;which can benefit from multi-task learning and tend to achieve a better speed-accuracy trade-off \cite{zhou2020tracking, zhang2021fairmot}.&#10;&#10;Ensemble learning \cite{sagi2018ensemble} generally refers to training and/or combining multiple models,&#10;which is widely used in machine learning \cite{efron1992bootstrap, breiman1996bagging, schapire1990strength, breiman1996stacked} &#10;and computer vision \cite{wortsman2022model, laine2016temporal, bodla1704improving, solovyev2021weighted}.&#10;For example, for image classification, Wortsman et al. proposes Model Soups to average weights of multiple models to improve the classification accuracy \cite{wortsman2022model}.&#10;To estimate more stable and accurate pseudo labels for semi-supervised image classification,&#10;Temporal Ensembling \cite{laine2016temporal} aggregates the predictions of multiple previous network evaluations into an ensemble prediction.&#10;For the object detection task, Soft-NMS \cite{bodla1704improving} and WBF \cite{solovyev2021weighted} are widely used to combine results from multiple detectors.&#10;&#10;Ensemble methods are also used in several MOT works.&#10;Peng et al. proposes the Layer-wise Aggregation Discriminative Model (LADM) \cite{peng2020dense},&#10;which uses the weighted average of predictions from three softmax layers to judge whether a detection box represents a person or not.&#10;However, it works in the detection procedure, and is essentially not for the tracking algorithm.&#10;Inspired by SoftNMS, TrackNMS is designed in GIAOTracker \cite{du2021giaotracker} to fuse multiple tracking results.&#10;It first sorts trajectories by the average confidence scores, and then performs non-maximum suppression (NMS) based on the temporal IoU.&#10;Though it is designed for combining multiple trackers, it is evaluated by the score-based metrics mAP \cite{ILSVRC15},&#10;in which redundant low-score results can benefit performance.&#10;Instead, the instance-based metrics, i.e., MOTA \cite{bernardin2008evaluating}, IDF1\cite{ristani2016performance} and HOTA \cite{luiten2021hota},&#10;are more common and reasonable evaluation metrics for the MOT task.&#10;&#10;To sum up, ensemble methods used in the MOT task are still not well exploited.&#10;We summarize the reasons as following:&#10;\begin{itemize}&#10; \item MOT is a complex downstream task. &#10; The diversity and complexity of various tracking algorithms makes it difficult to design a general and effective ensemble algorithm.&#10; \item The tracking results are temporal sequences, not just classification scores or detection bounding boxes (bboxes).&#10; Therefore, intuitive methods like voting can't be directly applied.&#10; \item The widely used metrics are instance-based.&#10; Compared with score-based metrics (e.g., mAP) in image classification and object detection,&#10; the instance-based metrics have no tolerance for redundant results,&#10; which introduces greater risk to ensemble methods.&#10;\end{itemize}&#10;&#10;In this paper, we propose a simple but effective ensemble method for instance-based metrics in the MOT task, called EnsembleMOT.&#10;It mixes tracking results from multiple trackers, and then merges them based on spatio-temporal constraints.&#10;The merged results may contain many redundant trajectories, so two post-processing methods,&#10;bbox-level length-based NMS and sequence-level length-based filtering, are applied to refine the results.&#10;Note that the proposed EnsembleMOT algorithm is model-independent and training-free,&#10;which gives it great flexibility in real applications.&#10;It doesn't need the training/testing data, &#10;which just takes multiple tracking results as input, and then outputs the ensemble results with significant improvements.&#10;&#10;We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline,&#10;i.e., SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}.&#10;Results show that EnsembleMOT can improve metrics MOTA/IDF1/HOTA by 1 to 4.&#10;Moreover, it can be integrated with other post-processing algorithms, e.g., interpolation \cite{du2022strongsort}.&#10;To the best of our knowledge, it is the first ensemble algorithm for MOT with instance-based metrics.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Ensemble Learning, Mathematics, Artificial Intelligence, Object Tracking, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2004.01177" label="2004.01177">
-        <attvalues>
-          <attvalue for="0" value="Tracking Objects as Points" />
-          <attvalue for="1" value="  Tracking has traditionally been the art of following interest points through&#10;space and time. This changed with the rise of powerful deep networks. Nowadays,&#10;tracking is dominated by pipelines that perform object detection followed by&#10;temporal association, also known as tracking-by-detection. In this paper, we&#10;present a simultaneous detection and tracking algorithm that is simpler,&#10;faster, and more accurate than the state of the art. Our tracker, CenterTrack,&#10;applies a detection model to a pair of images and detections from the prior&#10;frame. Given this minimal input, CenterTrack localizes objects and predicts&#10;their associations with the previous frame. That's it. CenterTrack is simple,&#10;online (no peeking into the future), and real-time. It achieves 67.3% MOTA on&#10;the MOT17 challenge at 22 FPS and 89.4% MOTA on the KITTI tracking benchmark at&#10;15 FPS, setting a new state of the art on both datasets. CenterTrack is easily&#10;extended to monocular 3D tracking by regressing additional 3D attributes. Using&#10;monocular video input, it achieves 28.3% AMOTA@0.2 on the newly released&#10;nuScenes 3D tracking benchmark, substantially outperforming the monocular&#10;baseline on this benchmark while running at 28 FPS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.05482" label="2203.05482">
-        <attvalues>
-          <attvalue for="0" value="Model soups: averaging weights of multiple fine-tuned models improves&#10;  accuracy without increasing inference time" />
-          <attvalue for="1" value="  The conventional recipe for maximizing model accuracy is to (1) train&#10;multiple models with various hyperparameters and (2) pick the individual model&#10;which performs best on a held-out validation set, discarding the remainder. In&#10;this paper, we revisit the second step of this procedure in the context of&#10;fine-tuning large pre-trained models, where fine-tuned models often appear to&#10;lie in a single low error basin. We show that averaging the weights of multiple&#10;models fine-tuned with different hyperparameter configurations often improves&#10;accuracy and robustness. Unlike a conventional ensemble, we may average many&#10;models without incurring any additional inference or memory costs -- we call&#10;the results &quot;model soups.&quot; When fine-tuning large pre-trained models such as&#10;CLIP, ALIGN, and a ViT-G pre-trained on JFT, our soup recipe provides&#10;significant improvements over the best model in a hyperparameter sweep on&#10;ImageNet. The resulting ViT-G model, which attains 90.94% top-1 accuracy on&#10;ImageNet, achieved a new state of the art. Furthermore, we show that the model&#10;soup approach extends to multiple image classification and natural language&#10;processing tasks, improves out-of-distribution performance, and improves&#10;zero-shot performance on new downstream tasks. Finally, we analytically relate&#10;the performance similarity of weight-averaging and logit-ensembling to flatness&#10;of the loss and confidence of the predictions, and validate this relation&#10;empirically. Code is available at https://github.com/mlfoundations/model-soups.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.13302" label="1910.13302">
-        <attvalues>
-          <attvalue for="0" value="Weighted boxes fusion: Ensembling boxes from different object detection&#10;  models" />
-          <attvalue for="1" value="  In this work, we present a novel method for combining predictions of object&#10;detection models: weighted boxes fusion. Our algorithm utilizes confidence&#10;scores of all proposed bounding boxes to constructs the averaged boxes. We&#10;tested method on several datasets and evaluated it in the context of the Open&#10;Images and COCO Object Detection tracks, achieving top results in these&#10;challenges. The source code is publicly available at&#10;https://github.com/ZFTurbo/Weighted-Boxes-Fusion&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.15576" label="2007.15576">
-        <attvalues>
-          <attvalue for="0" value="Dense Scene Multiple Object Tracking with Box-Plane Matching" />
-          <attvalue for="1" value="  Multiple Object Tracking (MOT) is an important task in computer vision. MOT&#10;is still challenging due to the occlusion problem, especially in dense scenes.&#10;Following the tracking-by-detection framework, we propose the Box-Plane&#10;Matching (BPM) method to improve the MOT performacne in dense scenes. First, we&#10;design the Layer-wise Aggregation Discriminative Model (LADM) to filter the&#10;noisy detections. Then, to associate remaining detections correctly, we&#10;introduce the Global Attention Feature Model (GAFM) to extract appearance&#10;feature and use it to calculate the appearance similarity between history&#10;tracklets and current detections. Finally, we propose the Box-Plane Matching&#10;strategy to achieve data association according to the motion similarity and&#10;appearance similarity between tracklets and detections. With the effectiveness&#10;of the three modules, our team achieves the 1st place on the Track-1&#10;leaderboard in the ACM MM Grand Challenge HiEve 2020.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.11983" label="2202.11983">
-        <attvalues>
-          <attvalue for="0" value="GIAOTracker: A comprehensive framework for MCMOT with global information&#10;  and optimizing strategies in VisDrone 2021" />
-          <attvalue for="1" value="  In recent years, algorithms for multiple object tracking tasks have benefited&#10;from great progresses in deep models and video quality. However, in challenging&#10;scenarios like drone videos, they still suffer from problems, such as small&#10;objects, camera movements and view changes. In this paper, we propose a new&#10;multiple object tracker, which employs Global Information And some Optimizing&#10;strategies, named GIAOTracker. It consists of three stages, i.e., online&#10;tracking, global link and post-processing. Given detections in every frame, the&#10;first stage generates reliable tracklets using information of camera motion,&#10;object motion and object appearance. Then they are associated into trajectories&#10;by exploiting global clues and refined through four post-processing methods.&#10;With the effectiveness of the three stages, GIAOTracker achieves&#10;state-of-the-art performance on the VisDrone MOT dataset and wins the 3rd place&#10;in the VisDrone2021 MOT Challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;" />
-        </attvalues>
-      </node>
-      <node id="1609.01775" label="1609.01775">
-        <attvalues>
-          <attvalue for="0" value="Performance Measures and a Data Set for Multi-Target, Multi-Camera&#10;  Tracking" />
-          <attvalue for="1" value="  To help accelerate progress in multi-target, multi-camera tracking systems,&#10;we present (i) a new pair of precision-recall measures of performance that&#10;treats errors of all types uniformly and emphasizes correct identification over&#10;sources of error; (ii) the largest fully-annotated and calibrated data set to&#10;date with more than 2 million frames of 1080p, 60fps video taken by 8 cameras&#10;observing more than 2,700 identities over 85 minutes; and (iii) a reference&#10;software system as a comparison baseline. We show that (i) our measures&#10;properly account for bottom-line identity match performance in the multi-camera&#10;setting; (ii) our data set poses realistic challenges to current trackers; and&#10;(iii) the performance of our system is comparable to the state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.07736" label="2009.07736">
-        <attvalues>
-          <attvalue for="0" value="HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking" />
-          <attvalue for="1" value="  Multi-Object Tracking (MOT) has been notoriously difficult to evaluate.&#10;Previous metrics overemphasize the importance of either detection or&#10;association. To address this, we present a novel MOT evaluation metric, HOTA&#10;(Higher Order Tracking Accuracy), which explicitly balances the effect of&#10;performing accurate detection, association and localization into a single&#10;unified metric for comparing trackers. HOTA decomposes into a family of&#10;sub-metrics which are able to evaluate each of five basic error types&#10;separately, which enables clear analysis of tracking performance. We evaluate&#10;the effectiveness of HOTA on the MOTChallenge benchmark, and show that it is&#10;able to capture important aspects of MOT performance not previously taken into&#10;account by established metrics. Furthermore, we show HOTA scores better align&#10;with human visual evaluation of tracking performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.11595" label="2105.11595">
-        <attvalues>
-          <attvalue for="0" value="SiamMOT: Siamese Multi-Object Tracking" />
-          <attvalue for="1" value="  In this paper, we focus on improving online multi-object tracking (MOT). In&#10;particular, we introduce a region-based Siamese Multi-Object Tracking network,&#10;which we name SiamMOT. SiamMOT includes a motion model that estimates the&#10;instance's movement between two frames such that detected instances are&#10;associated. To explore how the motion modelling affects its tracking&#10;capability, we present two variants of Siamese tracker, one that implicitly&#10;models motion and one that models it explicitly. We carry out extensive&#10;quantitative experiments on three different MOT datasets: MOT17, TAO-person and&#10;Caltech Roadside Pedestrians, showing the importance of motion modelling for&#10;MOT and the ability of SiamMOT to substantially outperform the&#10;state-of-the-art. Finally, SiamMOT also outperforms the winners of ACM MM'20&#10;HiEve Grand Challenge on HiEve dataset. Moreover, SiamMOT is efficient, and it&#10;runs at 17 FPS for 720P videos on a single modern GPU. Codes are available in&#10;\url{https://github.com/amazon-research/siam-mot}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.01318" label="2104.01318">
-        <attvalues>
-          <attvalue for="0" value="Efficient DETR: Improving End-to-End Object Detector with Dense Prior" />
-          <attvalue for="1" value="  The recently proposed end-to-end transformer detectors, such as DETR and&#10;Deformable DETR, have a cascade structure of stacking 6 decoder layers to&#10;update object queries iteratively, without which their performance degrades&#10;seriously. In this paper, we investigate that the random initialization of&#10;object containers, which include object queries and reference points, is mainly&#10;responsible for the requirement of multiple iterations. Based on our findings,&#10;we propose Efficient DETR, a simple and efficient pipeline for end-to-end&#10;object detection. By taking advantage of both dense detection and sparse set&#10;detection, Efficient DETR leverages dense prior to initialize the object&#10;containers and brings the gap of the 1-decoder structure and 6-decoder&#10;structure. Experiments conducted on MS COCO show that our method, with only 3&#10;encoder layers and 1 decoder layer, achieves competitive performance with&#10;state-of-the-art object detection methods. Efficient DETR is also robust in&#10;crowded scenes. It outperforms modern detectors on CrowdHuman dataset by a&#10;large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.02404" label="2108.02404">
-        <attvalues>
-          <attvalue for="0" value="Fast Convergence of DETR with Spatially Modulated Co-Attention" />
-          <attvalue for="1" value="  The recently proposed Detection Transformer (DETR) model successfully applies&#10;Transformer to objects detection and achieves comparable performance with&#10;two-stage object detection frameworks, such as Faster-RCNN. However, DETR&#10;suffers from its slow convergence. Training DETR from scratch needs 500 epochs&#10;to achieve a high accuracy. To accelerate its convergence, we propose a simple&#10;yet effective scheme for improving the DETR framework, namely Spatially&#10;Modulated Co-Attention (SMCA) mechanism. The core idea of SMCA is to conduct&#10;location-aware co-attention in DETR by constraining co-attention responses to&#10;be high near initially estimated bounding box locations. Our proposed SMCA&#10;increases DETR's convergence speed by replacing the original co-attention&#10;mechanism in the decoder while keeping other operations in DETR unchanged.&#10;Furthermore, by integrating multi-head and scale-selection attention designs&#10;into SMCA, our fully-fledged SMCA can achieve better performance compared to&#10;DETR with a dilated convolution-based backbone (45.6 mAP at 108 epochs vs. 43.3&#10;mAP at 500 epochs). We perform extensive ablation studies on COCO dataset to&#10;validate SMCA. Code is released at https://github.com/gaopengcuhk/SMCA-DETR .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.10881" label="2011.10881">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Transformer-based Set Prediction for Object Detection" />
-          <attvalue for="1" value="  DETR is a recently proposed Transformer-based method which views object&#10;detection as a set prediction problem and achieves state-of-the-art performance&#10;but demands extra-long training time to converge. In this paper, we investigate&#10;the causes of the optimization difficulty in the training of DETR. Our&#10;examinations reveal several factors contributing to the slow convergence of&#10;DETR, primarily the issues with the Hungarian loss and the Transformer&#10;cross-attention mechanism. To overcome these issues we propose two solutions,&#10;namely, TSP-FCOS (Transformer-based Set Prediction with FCOS) and TSP-RCNN&#10;(Transformer-based Set Prediction with RCNN). Experimental results show that&#10;the proposed methods not only converge much faster than the original DETR, but&#10;also significantly outperform DETR and other baselines in terms of detection&#10;accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.06152" label="2108.06152">
-        <attvalues>
-          <attvalue for="0" value="Conditional DETR for Fast Training Convergence" />
-          <attvalue for="1" value="  The recently-developed DETR approach applies the transformer encoder and&#10;decoder architecture to object detection and achieves promising performance. In&#10;this paper, we handle the critical issue, slow training convergence, and&#10;present a conditional cross-attention mechanism for fast DETR training. Our&#10;approach is motivated by that the cross-attention in DETR relies highly on the&#10;content embeddings for localizing the four extremities and predicting the box,&#10;which increases the need for high-quality content embeddings and thus the&#10;training difficulty. Our approach, named conditional DETR, learns a conditional&#10;spatial query from the decoder embedding for decoder multi-head&#10;cross-attention. The benefit is that through the conditional spatial query,&#10;each cross-attention head is able to attend to a band containing a distinct&#10;region, e.g., one object extremity or a region inside the object box. This&#10;narrows down the spatial range for localizing the distinct regions for object&#10;classification and box regression, thus relaxing the dependence on the content&#10;embeddings and easing the training. Empirical results show that conditional&#10;DETR converges 6.7x faster for the backbones R50 and R101 and 10x faster for&#10;stronger backbones DC5-R50 and DC5-R101. Code is available at&#10;https://github.com/Atten4Vis/ConditionalDETR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14330" label="2111.14330">
-        <attvalues>
-          <attvalue for="0" value="Sparse DETR: Efficient End-to-End Object Detection with Learnable&#10;  Sparsity" />
-          <attvalue for="1" value="  DETR is the first end-to-end object detector using a transformer&#10;encoder-decoder architecture and demonstrates competitive performance but low&#10;computational efficiency on high resolution feature maps. The subsequent work,&#10;Deformable DETR, enhances the efficiency of DETR by replacing dense attention&#10;with deformable attention, which achieves 10x faster convergence and improved&#10;performance. Deformable DETR uses the multiscale feature to ameliorate&#10;performance, however, the number of encoder tokens increases by 20x compared to&#10;DETR, and the computation cost of the encoder attention remains a bottleneck.&#10;In our preliminary experiment, we observe that the detection performance hardly&#10;deteriorates even if only a part of the encoder token is updated. Inspired by&#10;this observation, we propose Sparse DETR that selectively updates only the&#10;tokens expected to be referenced by the decoder, thus help the model&#10;effectively detect objects. In addition, we show that applying an auxiliary&#10;detection loss on the selected tokens in the encoder improves the performance&#10;while minimizing computational overhead. We validate that Sparse DETR achieves&#10;better performance than Deformable DETR even with only 10% encoder tokens on&#10;the COCO dataset. Albeit only the encoder tokens are sparsified, the total&#10;computation cost decreases by 38% and the frames per second (FPS) increases by&#10;42% compared to Deformable DETR.&#10;  Code is available at https://github.com/kakaobrain/sparse-detr&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.01305" label="2203.01305">
-        <attvalues>
-          <attvalue for="0" value="DN-DETR: Accelerate DETR Training by Introducing Query DeNoising" />
-          <attvalue for="1" value="  We present in this paper a novel denoising training method to speedup DETR&#10;(DEtection TRansformer) training and offer a deepened understanding of the slow&#10;convergence issue of DETR-like methods. We show that the slow convergence&#10;results from the instability of bipartite graph matching which causes&#10;inconsistent optimization goals in early training stages. To address this&#10;issue, except for the Hungarian loss, our method additionally feeds&#10;ground-truth bounding boxes with noises into Transformer decoder and trains the&#10;model to reconstruct the original boxes, which effectively reduces the&#10;bipartite graph matching difficulty and leads to a faster convergence. Our&#10;method is universal and can be easily plugged into any DETR-like methods by&#10;adding dozens of lines of code to achieve a remarkable improvement. As a&#10;result, our DN-DETR results in a remarkable improvement ($+1.9$AP) under the&#10;same setting and achieves the best result (AP $43.4$ and $48.6$ with $12$ and&#10;$50$ epochs of training respectively) among DETR-like methods with ResNet-$50$&#10;backbone. Compared with the baseline under the same setting, DN-DETR achieves&#10;comparable performance with $50\%$ training epochs. Code is available at&#10;\url{https://github.com/FengLi-ust/DN-DETR}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.16507" label="2203.16507">
-        <attvalues>
-          <attvalue for="0" value="AdaMixer: A Fast-Converging Query-Based Object Detector" />
-          <attvalue for="1" value="  Traditional object detectors employ the dense paradigm of scanning over&#10;locations and scales in an image. The recent query-based object detectors break&#10;this convention by decoding image features with a set of learnable queries.&#10;However, this paradigm still suffers from slow convergence, limited&#10;performance, and design complexity of extra networks between backbone and&#10;decoder. In this paper, we find that the key to these issues is the&#10;adaptability of decoders for casting queries to varying objects. Accordingly,&#10;we propose a fast-converging query-based detector, named AdaMixer, by improving&#10;the adaptability of query-based decoding processes in two aspects. First, each&#10;query adaptively samples features over space and scales based on estimated&#10;offsets, which allows AdaMixer to efficiently attend to the coherent regions of&#10;objects. Then, we dynamically decode these sampled features with an adaptive&#10;MLP-Mixer under the guidance of each query. Thanks to these two critical&#10;designs, AdaMixer enjoys architectural simplicity without requiring dense&#10;attentional encoders or explicit pyramid networks. On the challenging MS COCO&#10;benchmark, AdaMixer with ResNet-50 as the backbone, with 12 training epochs,&#10;reaches up to 45.0 AP on the validation set along with 27.9 APs in detecting&#10;small objects. With the longer training scheme, AdaMixer with ResNeXt-101-DCN&#10;and Swin-S reaches 49.5 and 51.3 AP. Our work sheds light on a simple,&#10;accurate, and fast converging architecture for query-based object detectors.&#10;The code is made available at https://github.com/MCG-NJU/AdaMixer&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.04632" label="2112.04632">
-        <attvalues>
-          <attvalue for="0" value="Recurrent Glimpse-based Decoder for Detection with Transformer" />
-          <attvalue for="1" value="  Although detection with Transformer (DETR) is increasingly popular, its&#10;global attention modeling requires an extremely long training period to&#10;optimize and achieve promising detection performance. Alternative to existing&#10;studies that mainly develop advanced feature or embedding designs to tackle the&#10;training issue, we point out that the Region-of-Interest (RoI) based detection&#10;refinement can easily help mitigate the difficulty of training for DETR&#10;methods. Based on this, we introduce a novel REcurrent Glimpse-based decOder&#10;(REGO) in this paper. In particular, the REGO employs a multi-stage recurrent&#10;processing structure to help the attention of DETR gradually focus on&#10;foreground objects more accurately. In each processing stage, visual features&#10;are extracted as glimpse features from RoIs with enlarged bounding box areas of&#10;detection results from the previous stage. Then, a glimpse-based decoder is&#10;introduced to provide refined detection results based on both the glimpse&#10;features and the attention modeling outputs of the previous stage. In practice,&#10;REGO can be easily embedded in representative DETR variants while maintaining&#10;their fully end-to-end training and inference pipelines. In particular, REGO&#10;helps Deformable DETR achieve 44.8 AP on the MSCOCO dataset with only 36&#10;training epochs, compared with the first DETR and the Deformable DETR that&#10;require 500 and 50 epochs to achieve comparable performance, respectively.&#10;Experiments also show that REGO consistently boosts the performance of&#10;different DETR detectors by up to 7% relative gain at the same setting of 50&#10;training epochs. Code is available via&#10;https://github.com/zhechen/Deformable-DETR-REGO.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.13498" label="2012.13498">
-        <attvalues>
-          <attvalue for="0" value="1st Place Solution to VisDA-2020: Bias Elimination for Domain Adaptive&#10;  Pedestrian Re-identification" />
-          <attvalue for="1" value="  This paper presents our proposed methods for domain adaptive pedestrian&#10;re-identification (Re-ID) task in Visual Domain Adaptation Challenge&#10;(VisDA-2020). Considering the large gap between the source domain and target&#10;domain, we focused on solving two biases that influenced the performance on&#10;domain adaptive pedestrian Re-ID and proposed a two-stage training procedure.&#10;At the first stage, a baseline model is trained with images transferred from&#10;source domain to target domain and from single camera to multiple camera&#10;styles. Then we introduced a domain adaptation framework to train the model on&#10;source data and target data simultaneously. Different pseudo label generation&#10;strategies are adopted to continuously improve the discriminative ability of&#10;the model. Finally, with multiple models ensembled and additional post&#10;processing approaches adopted, our methods achieve 76.56% mAP and 84.25% rank-1&#10;on the test set. Codes are available at&#10;https://github.com/vimar-gu/Bias-Eliminate-DA-ReID&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pedestrian re-identification (Re-ID) aims to match specific person identities across multiple cameras. &#10;As more and more surveillance cameras are being deployed in cities, pedestrian Re-ID can play an indispensable role in modern security systems. In recent years, deep learning methods have made a significant progress on pedestrian Re-ID task \cite{Luo_2019_Strong_TMM,Luo_2019_CVPR_Workshops,sun2018beyond}. &#10;However, pedestrian Re-ID still faces many challenges, one of which is the large data amount. &#10;As the pedestrian Re-ID task is an open-set problem, it is impossible to manually label all the pedestrian images produced by surveillance cameras day by day. &#10;Based on the situation, domain adaptation has attracted much attention on the pedestrian Re-ID task \cite{deng2018image,zhang2019self,fu2019self,chen2019instance,song2020unsupervised}. &#10;&#10;Compared with traditional domain adaptation tasks, domain adaptive pedestrian Re-ID is much harder, as the source domain and target domain share no identical classes. &#10;Moreover, in VisDA-2020, a synthetic dataset are provided as the labeled source domain \cite{sun2019dissecting} and a real-world dataset is adopted as the target domain, where exists a large domain gap. &#10;In this work, we analyzed the biases in domain adaptive pedestrian Re-ID task introduced by different datasets and different cameras, and proposed a domain adaptation framework to solve the problem. &#10;&#10;The rest of the paper is organized as follows. In Section \ref{methods}, the proposed methods is introduced. The experimental results are presented in Section \ref{experiments}. And finally Section \ref{conclusion} concludes the paper. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Pedestrian Re-identification" />
-        </attvalues>
-      </node>
-      <node id="1906.08332" label="1906.08332">
-        <attvalues>
-          <attvalue for="0" value="A Strong Baseline and Batch Normalization Neck for Deep Person&#10;  Re-identification" />
-          <attvalue for="1" value="  This study explores a simple but strong baseline for person re-identification&#10;(ReID). Person ReID with deep neural networks has progressed and achieved high&#10;performance in recent years. However, many state-of-the-art methods design&#10;complex network structures and concatenate multi-branch features. In the&#10;literature, some effective training tricks briefly appear in several papers or&#10;source codes. The present study collects and evaluates these effective training&#10;tricks in person ReID. By combining these tricks, the model achieves 94.5%&#10;rank-1 and 85.9% mean average precision on Market1501 with only using the&#10;global features of ResNet50. The performance surpasses all existing global- and&#10;part-based baselines in person ReID. We propose a novel neck structure named as&#10;batch normalization neck (BNNeck). BNNeck adds a batch normalization layer&#10;after global pooling layer to separate metric and classification losses into&#10;two different feature spaces because we observe they are inconsistent in one&#10;embedding space. Extended experiments show that BNNeck can boost the baseline,&#10;and our baseline can improve the performance of existing state-of-the-art&#10;methods. Our codes and models are available at:&#10;https://github.com/michuanhaohao/reid-strong-baseline.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.07027" label="1711.07027">
-        <attvalues>
-          <attvalue for="0" value="Image-Image Domain Adaptation with Preserved Self-Similarity and&#10;  Domain-Dissimilarity for Person Re-identification" />
-          <attvalue for="1" value="  Person re-identification (re-ID) models trained on one domain often fail to&#10;generalize well to another. In our attempt, we present a &quot;learning via&#10;translation&quot; framework. In the baseline, we translate the labeled images from&#10;source to target domain in an unsupervised manner. We then train re-ID models&#10;with the translated images by supervised methods. Yet, being an essential part&#10;of this framework, unsupervised image-image translation suffers from the&#10;information loss of source-domain labels during translation.&#10;  Our motivation is two-fold. First, for each image, the discriminative cues&#10;contained in its ID label should be maintained after translation. Second, given&#10;the fact that two domains have entirely different persons, a translated image&#10;should be dissimilar to any of the target IDs. To this end, we propose to&#10;preserve two types of unsupervised similarities, 1) self-similarity of an image&#10;before and after translation, and 2) domain-dissimilarity of a translated&#10;source image and a target image. Both constraints are implemented in the&#10;similarity preserving generative adversarial network (SPGAN) which consists of&#10;an Siamese network and a CycleGAN. Through domain adaptation experiment, we&#10;show that images generated by SPGAN are more suitable for domain adaptation and&#10;yield consistent and competitive re-ID accuracy on two large-scale datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.13315" label="1907.13315">
-        <attvalues>
-          <attvalue for="0" value="Self-training with progressive augmentation for unsupervised&#10;  cross-domain person re-identification" />
-          <attvalue for="1" value="  Person re-identification (Re-ID) has achieved great improvement with deep&#10;learning and a large amount of labelled training data. However, it remains a&#10;challenging task for adapting a model trained in a source domain of labelled&#10;data to a target domain of only unlabelled data available. In this work, we&#10;develop a self-training method with progressive augmentation framework (PAST)&#10;to promote the model performance progressively on the target dataset.&#10;Specially, our PAST framework consists of two stages, namely, conservative&#10;stage and promoting stage. The conservative stage captures the local structure&#10;of target-domain data points with triplet-based loss functions, leading to&#10;improved feature representations. The promoting stage continuously optimizes&#10;the network by appending a changeable classification layer to the last layer of&#10;the model, enabling the use of global information about the data distribution.&#10;Importantly, we propose a new self-training strategy that progressively&#10;augments the model capability by adopting conservative and promoting stages&#10;alternately. Furthermore, to improve the reliability of selected triplet&#10;samples, we introduce a ranking-based triplet loss in the conservative stage,&#10;which is a label-free objective function basing on the similarities between&#10;data pairs. Experiments demonstrate that the proposed method achieves&#10;state-of-the-art person Re-ID performance under the unsupervised cross-domain&#10;setting. Code is available at: https://tinyurl.com/PASTReID&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.10144" label="1811.10144">
-        <attvalues>
-          <attvalue for="0" value="Self-similarity Grouping: A Simple Unsupervised Cross Domain Adaptation&#10;  Approach for Person Re-identification" />
-          <attvalue for="1" value="  Domain adaptation in person re-identification (re-ID) has always been a&#10;challenging task. In this work, we explore how to harness the natural similar&#10;characteristics existing in the samples from the target domain for learning to&#10;conduct person re-ID in an unsupervised manner. Concretely, we propose a&#10;Self-similarity Grouping (SSG) approach, which exploits the potential&#10;similarity (from global body to local parts) of unlabeled samples to&#10;automatically build multiple clusters from different views. These independent&#10;clusters are then assigned with labels, which serve as the pseudo identities to&#10;supervise the training process. We repeatedly and alternatively conduct such a&#10;grouping and training process until the model is stable. Despite the apparent&#10;simplify, our SSG outperforms the state-of-the-arts by more than 4.6% (DukeMTMC&#10;to Market1501) and 4.4% (Market1501 to DukeMTMC) in mAP, respectively. Upon our&#10;SSG, we further introduce a clustering-guided semisupervised approach named SSG&#10;++ to conduct the one-shot domain adaption in an open set setting (i.e. the&#10;number of independent identities from the target domain is unknown). Without&#10;spending much effort on labeling, our SSG ++ can further promote the mAP upon&#10;SSG by 10.7% and 6.9%, respectively. Our Code is available at:&#10;https://github.com/OasisYang/SSG .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.11334" label="1807.11334">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Domain Adaptive Re-Identification: Theory and Practice" />
-          <attvalue for="1" value="  We study the problem of unsupervised domain adaptive re-identification&#10;(re-ID) which is an active topic in computer vision but lacks a theoretical&#10;foundation. We first extend existing unsupervised domain adaptive&#10;classification theories to re-ID tasks. Concretely, we introduce some&#10;assumptions on the extracted feature space and then derive several loss&#10;functions guided by these assumptions. To optimize them, a novel self-training&#10;scheme for unsupervised domain adaptive re-ID tasks is proposed. It iteratively&#10;makes guesses for unlabeled target data based on an encoder and trains the&#10;encoder based on the guessed labels. Extensive experiments on unsupervised&#10;domain adaptive person re-ID and vehicle re-ID tasks with comparisons to the&#10;state-of-the-arts confirm the effectiveness of the proposed theories and&#10;self-training framework. Our code is available at&#10;\url{https://github.com/LcDog/DomainAdaptiveReID}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.02162" label="1812.02162">
-        <attvalues>
-          <attvalue for="0" value="Dissecting Person Re-identification from the Viewpoint of Viewpoint" />
-          <attvalue for="1" value="  Variations in visual factors such as viewpoint, pose, illumination and&#10;background, are usually viewed as important challenges in person&#10;re-identification (re-ID). In spite of acknowledging these factors to be&#10;influential, quantitative studies on how they affect a re-ID system are still&#10;lacking. To derive insights in this scientific campaign, this paper makes an&#10;early attempt in studying a particular factor, viewpoint. We narrow the&#10;viewpoint problem down to the pedestrian rotation angle to obtain focused&#10;conclusions. In this regard, this paper makes two contributions to the&#10;community. First, we introduce a large-scale synthetic data engine, PersonX.&#10;Composed of hand-crafted 3D person models, the salient characteristic of this&#10;engine is &quot;controllable&quot;. That is, we are able to synthesize pedestrians by&#10;setting the visual variables to arbitrary values. Second, on the 3D data&#10;engine, we quantitatively analyze the influence of pedestrian rotation angle on&#10;re-ID accuracy. Comprehensively, the person rotation angles are precisely&#10;customized from 0 to 360, allowing us to investigate its effect on the&#10;training, query, and gallery sets. Extensive experiment helps us have a deeper&#10;understanding of the fundamental problems in person re-ID. Our research also&#10;provides useful insights for dataset building and future practical usage, e.g.,&#10;a person of a side view makes a better query.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04883" label="2003.04883">
-        <attvalues>
-          <attvalue for="0" value="Maximum Likelihood Speed Estimation of Moving Objects in Video Signals" />
-          <attvalue for="1" value="  Video processing solutions for motion analysis are key tasks in many computer&#10;vision applications, ranging from human activity recognition to object&#10;detection. In particular, speed estimation algorithms may be relevant in&#10;contexts such as street monitoring and environment surveillance. In most&#10;realistic scenarios, the projection of a framed object of interest onto the&#10;image plane is likely to be affected by dynamic changes mainly related to&#10;perspectival transformations or periodic behaviours. Therefore, advanced speed&#10;estimation techniques need to rely on robust algorithms for object detection&#10;that are able to deal with potential geometrical modifications. The proposed&#10;method is composed of a sequence of pre-processing operations, that aim to&#10;reduce or neglect perspetival effects affecting the objects of interest,&#10;followed by the estimation phase based on the Maximum Likelihood (ML)&#10;principle, where the speed of the foreground objects is estimated. The ML&#10;estimation method represents, indeed, a consolidated statistical tool that may&#10;be exploited to obtain reliable results. The performance of the proposed&#10;algorithm is evaluated on a set of real video recordings and compared with a&#10;block-matching motion estimation algorithm. The obtained results indicate that&#10;the proposed method shows good and robust performance.&#10;" />
-          <attvalue for="2" value="&#10;Speed estimation systems play nowadays a fundamental role in contexts of traffic control and road monitoring applications. Thanks to the increasing deployment of cameras for surveillance purposes, a significant amount of street-related information is available and may be exploited to build non-intrusive video-based solutions for object speed estimation.&#10;&#10;In many realistic scenarios, the motion of foreground objects is superposed to other dynamic modifications that mainly arise from periodic behaviours (e.g., typical of some human movements such as walking and running) or directly result from the process of image acquisition \cite{Solomon}, \cite{Richard}. A video frame can be indeed defined as a digital image generated by the projection of a three-dimensional (3D) real-world scene onto a two-dimensional (2D) camera plane. For this reason, perspectival effects are likely to affect the image of the framed objects of interest. Hence the analysis of their motion, i.e., speed estimation and periodic feature extraction, may be challenging in some scenarios.&#10;&#10;The goal of this paper is the estimation of the speed of foreground objects in a video sequence, accounting for the perspectival effects that may affect their 2D projection throughout the video duration. To this purpose, we may apply inverse projective transformations \cite{Solomon}, \cite{ViewGeom} to each frame of the video sequence to obtain a new processed sequence where the shape and the size of the framed objects are affected by minor modifications and can be considered constant throughout the video duration. We exploit a sound estimation theory tool, i.e. the Maximum Likelihood (ML) principle \cite{Kay} to obtain an estimator of the speed of the objects of interest.&#10;&#10;An extensive review of video-based solutions for speed estimation can be found in \cite{FernndezLlorca2021VisionbasedVS}, where various methods are classified according to specific criteria. In particular, classical approaches can be mainly categorized as based on a) camera setting and calibration and b) vehicle detection and tracking. The works described in \cite{SOCHOR201787} and \cite{Giannakeris_2018_CVPR_Workshops} are examples in the first category, where vanishing points are detected on the considered 2D scene and the camera parameters are computed to enable the measurement of distances on the road plane, hence the speed of the vehicle of interest. On the other hand, solutions based on category b) vehicle detection and tracking, mainly rely on background removal, feature detection or license plate detection algorithms. In \cite{Wicaksono2017SpeedEO} and \cite{Tourani2019MotionbasedVS}, background subtraction is performed by means of the Gaussian Mixture Model. In \cite{9043732}, Scale Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) algorithms are exploited to detect feature points, whereas in \cite{Vakili}, the licence plate of the vehicle of interest is detected using an open source software and is tracked in consecutive frames to estimate the vehicle speed.&#10;&#10;As a matter of fact, the literature on the extraction of information content from video signals is mainly based on heuristic ad-hoc solutions. To our knowledge, little or no attempts to employ sound approaches from estimation theory have been pursued and this paper wishes to contribute to fill this gap. As exceptions to this general trend, we mention previous contributions in which the ML criterion was successfully employed in the context of video processing for the extraction of periodic features \cite{7886029}, \cite{CATTANI2017158}. Despite a sound and consolidated method, the ML approach continues to be successfully applied to the solution of current problems. To mention a few, examples of recent works on ML parameters estimation include \cite{FASCISTA2021107907} and \cite{HAO2021108111}. In the former, a pseudo ML algorithm is developed to estimate the position of a multi-antenna receiver in dynamic multipath scenarios, whereas, in the latter, the position of multiple emitters is estimated. Another application of the ML approach can be found in \cite{WONG2021108087}, where a novel method for image reconstruction based on ML exposure level estimation is proposed.&#10;&#10;Given a model that describes the observed data, the ML method allows to derive estimators of the unknown parameters of interest. In the application of interest in this paper, once the motion model of the video sequence is defined, the ML criterion can be applied to obtain an expression of the log-likelihood function to be optimized through a maximization process: the coordinates of the maximum of this function indicate the estimated components of the parameters of interest, namely the speed vector in this work. &#10;&#10;The performance of the proposed algorithm is compared with a more conventional solution, i.e. the block-matching method \cite[Ch. 4]{VideoProcessing}, that tends to be subject to several problems mainly related to the presence of noise, repeated patterns in the scene and the block size setting.&#10;This paper expands upon preliminary conference contributions \cite{Eusipco}, \cite{TSP}.&#10;&#10;The remainder of the paper is organized as follows. In Section~\ref{methodology}, preliminary video processing operations are described, along with the mathematical formulation of the dynamic motion model. In Section \ref{ml}, the likelihood function and the resulting speed estimation procedure are proposed. In Section~\ref{performance}, performance results on real videos are presented. Finally, in Section~\ref{conclusion} conclusions are drawn.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Object Detection, Motion Analysis, Computer Science, Statistics, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2105.04213" label="2105.04213">
-        <attvalues>
-          <attvalue for="0" value="Temporal-Spatial Feature Pyramid for Video Saliency Detection" />
-          <attvalue for="1" value="  Multi-level features are important for saliency detection. Better combination&#10;and use of multi-level features with time information can greatly improve the&#10;accuracy of the video saliency model. In order to fully combine multi-level&#10;features and make it serve the video saliency model, we propose a 3D fully&#10;convolutional encoder-decoder architecture for video saliency detection, which&#10;combines scale, space and time information for video saliency modeling. The&#10;encoder extracts multi-scale temporal-spatial features from the input&#10;continuous video frames, and then constructs temporal-spatial feature pyramid&#10;through temporal-spatial convolution and top-down feature integration. The&#10;decoder performs hierarchical decoding of temporal-spatial features from&#10;different scales, and finally produces a saliency map from the integration of&#10;multiple video frames. Our model is simple yet effective, and can run in real&#10;time. We perform abundant experiments, and the results indicate that the&#10;well-designed structure can improve the precision of video saliency detection&#10;significantly. Experimental results on three purely visual video saliency&#10;benchmarks and six audio-video saliency benchmarks demonstrate that our method&#10;outperforms the existing state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Video saliency detection aims to predict the point of fixation for the human eyes while watching videos freely. It is widely applied in a lot of areas such as video compression~\cite{1hadizadeh2013saliency,2zhu2019high}, video surveillance~\cite{3guraya2010predictive,4yubing2011spatiotemporal} and video captioning~\cite{5nguyen2013static}.&#10;&#10;Most of the existing video saliency detection models employ the encoder-decoder structure, and rely on the temporal recurrence to predict video saliency. For example, ACLNet~\cite{7wang2018revisiting} encodes static saliency features through attention mechanism, and then learns dynamic saliency through ConvLSTM~\cite{8shi2015convolutional}. SalEMA~\cite{9linardos2019simple} uses exponential moving average instead of LSTM to extract temporal features for video saliency detection. SalSAC~\cite{10wu2020salsac} proposes a correlation-based ConvLSTM to balance the alteration of saliency caused by the change of image characteristics of past frame and current frame. However, such a saliency modeling approach has the following problems. Firstly, the spatial saliency model is pretrained on the static image saliency datasets before finetuning on the video saliency datasets. However, the effectiveness of this transfer learning mechanism may be limited, since the resolutions of two datasets are different while saliency is greatly influenced by the image shape. Secondly, restricted by memory, the training of video saliency model requires to extract continuous video frames from the datasets randomly. However, the approach based on LSTM needs to utilize backpropagation through time to predict the video saliency of each frame. In this way, the state of LSTM of the first frame for the selected clip must be void, while, during the test, only the state of the LSTM of the first frame of the video is void, such discrepancy makes the modeling of method based on LSTM insufficient. Thirdly, as mentioned by~\cite{11min2019tased}, all the methods based on LSTM overlay the temporal information on top of spatial information, and fail to utilize both kinds of information at the same time, which is crucial for video saliency detection.&#10;&#10;To alleviate above problems, some methods~\cite{11min2019tased, 37tsiami2020stavis, 28bellitto2020video} employ 3D convolutions to continuously aggregate the temporal and spatial clues of videos. While they achieve outstanding performance, there still remains an important issue, that is, lacking the utilization of multi-level features. Multi-level features are essential for the task of saliency detection, since the human visual mechanism is complicated and the concerned region is determined by various factors and from multiple levels. For example, some large objects may be salient, which are captured from the deeper layers with a relatively large receptive fields. Some small but moving at a high speed objects are also salient, which are captured from shallower layers holding more low-level information. Although the use of multi-level features such as FPN \cite{4422} has already shined in the field of 2D object detection, there is currently few methods to fully verify that multi-level features are effective for video saliency. [19] prove that multi-level features are effective for video saliency and achieve excellent performance. However, there is still room for research on how to better use and combine multi-level features and build a fully convolutional model to maximize the accuracy of the model.&#10;&#10;In order to solve the past problems, we propose a new 3D fully convolutional encoder-decoder architecture for video saliency detection.&#10; We consider the influence of time, space and scale and establish a temporal-spatial feature pyramid similar to FPN \cite{4422}.&#10;In this way, the temporal-spatial semantic features of deep layer are aggregated to each layer of the pyramid. In view of the different receptive fields of temporal dimension for the features of various layers, we separately perform independent hierarchical decoding on different levels of the feature pyramid to take the effect of temporal-spatial saliency features with various scales into consideration. Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}. We put away the previous deconvolution and unpooling operation of 3D fully convolutional encoder-decoder~\cite{11min2019tased} and completely adopt the 3D convolution and the trilinear upsampling.&#10;&#10;At the same time, in order to predicting audio-video saliency, audio information and visual information are fused, and the obtained features are integrated in the original visual network in the form of attention. Our network is simple in structure, lower in parameters, and higher in prediction precision, which has obvious difference with other methods of the state-of-the-art. Our method ranks first in the largest and most diverse video saliency dataset, such as DHF1K~\cite{7wang2018revisiting}.&#10;&#10;The main contributions of the paper are as following:&#10;&#10;We develop a new 3D fully convolutional temporal-spatial feature pyramid network called TSFP-Net, which completely consists of 3D convolution and trilinear upsampling and obtain very high accuracy in the case of a small model size.&#10;&#10;We construct feature pyramid of different scales containing rich temporal-spatial semantic features, and build a hierarchical 3D convolutional decoder to decode. We prove that such approach can significantly improve the detection performance of the video saliency. By fusing audio information and visual information and integrating them into the original visual network in the form of attention, we can simultaneously perform audio-video saliency prediction through TSFP-Net (with audio).&#10;&#10;We evaluate our model on three purely visual large-scale video saliency datasets and six audio-video saliency datasets, comparing with the state-of-the-art methods, our model can achieve large gains.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Video Saliency Detection, Cognitive Science, Computer Science, Deep Learning Architecture, Computer Vision Modeling, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1801.07424" label="1801.07424">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Video Saliency: A Large-scale Benchmark and a New Model" />
-          <attvalue for="1" value="  In this work, we contribute to video saliency research in two ways. First, we&#10;introduce a new benchmark for predicting human eye movements during dynamic&#10;scene free-viewing, which is long-time urged in this field. Our dataset, named&#10;DHF1K (Dynamic Human Fixation), consists of 1K high-quality, elaborately&#10;selected video sequences spanning a large range of scenes, motions, object&#10;types and background complexity. Existing video saliency datasets lack variety&#10;and generality of common dynamic scenes and fall short in covering challenging&#10;situations in unconstrained environments. In contrast, DHF1K makes a&#10;significant leap in terms of scalability, diversity and difficulty, and is&#10;expected to boost video saliency modeling. Second, we propose a novel video&#10;saliency model that augments the CNN-LSTM network architecture with an&#10;attention mechanism to enable fast, end-to-end saliency learning. The attention&#10;mechanism explicitly encodes static saliency information, thus allowing LSTM to&#10;focus on learning more flexible temporal saliency representation across&#10;successive frames. Such a design fully leverages existing large-scale static&#10;fixation datasets, avoids overfitting, and significantly improves training&#10;efficiency and testing performance. We thoroughly examine the performance of&#10;our model, with respect to state-of-the-art saliency models, on three&#10;large-scale datasets (i.e., DHF1K, Hollywood2, UCF sports). Experimental&#10;results over more than 1.2K testing videos containing 400K frames demonstrate&#10;that our model outperforms other competitors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.01869" label="1907.01869">
-        <attvalues>
-          <attvalue for="0" value="Simple vs complex temporal recurrences for video saliency prediction" />
-          <attvalue for="1" value="  This paper investigates modifying an existing neural network architecture for&#10;static saliency prediction using two types of recurrences that integrate&#10;information from the temporal domain. The first modification is the addition of&#10;a ConvLSTM within the architecture, while the second is a conceptually simple&#10;exponential moving average of an internal convolutional state. We use weights&#10;pre-trained on the SALICON dataset and fine-tune our model on DHF1K. Our&#10;results show that both modifications achieve state-of-the-art results and&#10;produce similar saliency maps. Source code is available at&#10;https://git.io/fjPiB.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.05786" label="1908.05786">
-        <attvalues>
-          <attvalue for="0" value="TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for&#10;  Video Saliency Detection" />
-          <attvalue for="1" value="  TASED-Net is a 3D fully-convolutional network architecture for video saliency&#10;detection. It consists of two building blocks: first, the encoder network&#10;extracts low-resolution spatiotemporal features from an input clip of several&#10;consecutive frames, and then the following prediction network decodes the&#10;encoded features spatially while aggregating all the temporal information. As a&#10;result, a single prediction map is produced from an input clip of multiple&#10;frames. Frame-wise saliency maps can be predicted by applying TASED-Net in a&#10;sliding-window fashion to a video. The proposed approach assumes that the&#10;saliency map of any frame can be predicted by considering a limited number of&#10;past frames. The results of our extensive experiments on video saliency&#10;detection validate this assumption and demonstrate that our fully-convolutional&#10;model with temporal aggregation method is effective. TASED-Net significantly&#10;outperforms previous state-of-the-art approaches on all three major large-scale&#10;datasets of video saliency detection: DHF1K, Hollywood2, and UCFSports. After&#10;analyzing the results qualitatively, we observe that our model is especially&#10;better at attending to salient moving objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.06612" label="1611.06612">
-        <attvalues>
-          <attvalue for="0" value="RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Recently, very deep convolutional neural networks (CNNs) have shown&#10;outstanding performance in object recognition and have also been the first&#10;choice for dense classification problems such as semantic segmentation.&#10;However, repeated subsampling operations like pooling or convolution striding&#10;in deep CNNs lead to a significant decrease in the initial image resolution.&#10;Here, we present RefineNet, a generic multi-path refinement network that&#10;explicitly exploits all the information available along the down-sampling&#10;process to enable high-resolution prediction using long-range residual&#10;connections. In this way, the deeper layers that capture high-level semantic&#10;features can be directly refined using fine-grained features from earlier&#10;convolutions. The individual components of RefineNet employ residual&#10;connections following the identity mapping mindset, which allows for effective&#10;end-to-end training. Further, we introduce chained residual pooling, which&#10;captures rich background context in an efficient manner. We carry out&#10;comprehensive experiments and set new state-of-the-art results on seven public&#10;datasets. In particular, we achieve an intersection-over-union score of 83.4 on&#10;the challenging PASCAL VOC 2012 dataset, which is the best reported result to&#10;date.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.04044" label="2405.04044">
-        <attvalues>
-          <attvalue for="0" value="DMOFC: Discrimination Metric-Optimized Feature Compression" />
-          <attvalue for="1" value="  Feature compression, as an important branch of video coding for machines&#10;(VCM), has attracted significant attention and exploration. However, the&#10;existing methods mainly focus on intra-feature similarity, such as the Mean&#10;Squared Error (MSE) between the reconstructed and original features, while&#10;neglecting the importance of inter-feature relationships. In this paper, we&#10;analyze the inter-feature relationships, focusing on feature discriminability&#10;in machine vision and underscoring its significance in feature compression. To&#10;maintain the feature discriminability of reconstructed features, we introduce a&#10;discrimination metric for feature compression. The discrimination metric is&#10;designed to ensure that the distance between features of the same category is&#10;smaller than the distance between features of different categories.&#10;Furthermore, we explore the relationship between the discrimination metric and&#10;the discriminability of the original features. Experimental results confirm the&#10;effectiveness of the proposed discrimination metric and reveal there exists a&#10;trade-off between the discrimination metric and the discriminability of the&#10;original features.&#10;" />
-          <attvalue for="2" value="&#10;The exponential growth of image and video data has rendered it impractical for humans to manually analyze all of them. With the remarkable performance of machine vision algorithms, the analysis of image and video data has shifted towards machines by leveraging the capabilities of vision algorithms. &#10;Consequently, an increasing volume of image and video content is being processed by machines for analysis. &#10;However, prior to any analysis, it is crucial to store and transmit these data efficiently, leading to the emergence of the research topic known as video coding for machines (VCM).&#10;&#10;Feature compression, as an important branch in VCM, has attracted considerable attention and exploration. Based on distortion measurement methods, feature encoding can be broadly categorized into three types.&#10;The first kind of method, similar to image compression, measures distortion by the Mean Squared Error (MSE) between the reconstructed feature and the original feature \cite{choi2018deep, suzuki2022deep, kim2023end, liu2023learnt}. For example, in \cite{kim2023end}, the feature MSE is used for multi-layer feature compression.&#10;In the second kind of method, researchers incorporate semantic information into the distortion measurement \cite{choi2021latent, chen2019lossy, chen2020toward, ikusan2021rate, hu2020sensitivity}.&#10;Departing from MSE constraints between reconstructed and original features, this method opts for constraining the semantic similarity between reconstructed and original features. This approach provides a more direct way of constraining the semantic information within the reconstructed features. For instance, in \cite{chen2019lossy}, one-hot vectors of the original and reconstructed features are obtained, and the similarity between them serves as the distortion measure.&#10;In the third kind of method, researchers adopt a more direct semantic information constraint to reconstructed features \cite{singh2020end, yang2021video, zhang2021MSFC, yan2021SSSIC, alvar2019multi}. In this approach, they directly employ the loss function of visual tasks as the distortion measure for feature compression. For example, the machine vision algorithm is treated as the distortion of the compression network.&#10;&#10;Existing methods adhere to the principles of image compression, where each individual feature undergoes independent optimization during the encoding process. For instance, the similarity measure MSE is computed on a single feature and is unrelated to the others. Despite the substantial progress achieved by these methods, this independent compression approach neglects the relationships between different features. However, for features utilized in machine vision, the relationships between different features, such as similarity and discriminability, play a pivotal role in the final machine vision analysis. Current independent compression methods fail to consider these inter-feature relationships, limiting the efficiency of feature compression.&#10;&#10;In this paper, we investigate the inter-feature relationship in the context of the discriminability of features. Discriminability is one of the most important characteristics of features because it is the basis of the classification. Given classification plays an important role in various machine vision tasks, feature discriminability becomes pivotal for machine vision tasks. Our contributions, derived from a thorough investigation of feature discriminability, can be summarized as follows:&#10;\begin{itemize}&#10;&#09;\item We are the first to address the importance of inter-feature relationships in feature compression and study it in the context of feature discriminability&#10;&#09;\item We propose a discrimination metric-optimized feature compression and verified its effectiveness for feature compression in person re-identification (person ReID) and face verification (FV) task &#10;&#09;\item We study the influence of the discriminability of original features on the discrimination metric, and unveil the trade-off between them&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Video Coding Methods, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Feature Compression Techniques, Machine Vision Analysis, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1802.03931" label="1802.03931">
-        <attvalues>
-          <attvalue for="0" value="Deep feature compression for collaborative object detection" />
-          <attvalue for="1" value="  Recent studies have shown that the efficiency of deep neural networks in&#10;mobile applications can be significantly improved by distributing the&#10;computational workload between the mobile device and the cloud. This paradigm,&#10;termed collaborative intelligence, involves communicating feature data between&#10;the mobile and the cloud. The efficiency of such approach can be further&#10;improved by lossy compression of feature data, which has not been examined to&#10;date. In this work we focus on collaborative object detection and study the&#10;impact of both near-lossless and lossy compression of feature data on its&#10;accuracy. We also propose a strategy for improving the accuracy under lossy&#10;feature compression. Experiments indicate that using this strategy, the&#10;communication overhead can be reduced by up to 70% without sacrificing&#10;accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.16670" label="2306.16670">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Learnable Multi-Scale Feature Compression for VCM" />
-          <attvalue for="1" value="  The proliferation of deep learning-based machine vision applications has&#10;given rise to a new type of compression, so called video coding for machine&#10;(VCM). VCM differs from traditional video coding in that it is optimized for&#10;machine vision performance instead of human visual quality. In the feature&#10;compression track of MPEG-VCM, multi-scale features extracted from images are&#10;subject to compression. Recent feature compression works have demonstrated that&#10;the versatile video coding (VVC) standard-based approach can achieve a BD-rate&#10;reduction of up to 96% against MPEG-VCM feature anchor. However, it is still&#10;sub-optimal as VVC was not designed for extracted features but for natural&#10;images. Moreover, the high encoding complexity of VVC makes it difficult to&#10;design a lightweight encoder without sacrificing performance. To address these&#10;challenges, we propose a novel multi-scale feature compression method that&#10;enables both the end-to-end optimization on the extracted features and the&#10;design of lightweight encoders. The proposed model combines a learnable&#10;compressor with a multi-scale feature fusion network so that the redundancy in&#10;the multi-scale features is effectively removed. Instead of simply cascading&#10;the fusion network and the compression network, we integrate the fusion and&#10;encoding processes in an interleaved way. Our model first encodes a&#10;larger-scale feature to obtain a latent representation and then fuses the&#10;latent with a smaller-scale feature. This process is successively performed&#10;until the smallest-scale feature is fused and then the encoded latent at the&#10;final stage is entropy-coded for transmission. The results show that our model&#10;outperforms previous approaches by at least 52% BD-rate reduction and has&#10;$\times5$ to $\times27$ times less encoding time for object detection...&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.10089" label="2105.10089">
-        <attvalues>
-          <attvalue for="0" value="Latent-space scalability for multi-task collaborative intelligence" />
-          <attvalue for="1" value="  We investigate latent-space scalability for multi-task collaborative&#10;intelligence, where one of the tasks is object detection and the other is input&#10;reconstruction. In our proposed approach, part of the latent space can be&#10;selectively decoded to support object detection while the remainder can be&#10;decoded when input reconstruction is needed. Such an approach allows reduced&#10;computational resources when only object detection is required, and this can be&#10;achieved without reconstructing input pixels. By varying the scaling factors of&#10;various terms in the training loss function, the system can be trained to&#10;achieve various trade-offs between object detection accuracy and input&#10;reconstruction quality. Experiments are conducted to demonstrate the adjustable&#10;system performance on the two tasks compared to the relevant benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11797" label="2007.11797">
-        <attvalues>
-          <attvalue for="0" value="End-to-end Learning of Compressible Features" />
-          <attvalue for="1" value="  Pre-trained convolutional neural networks (CNNs) are powerful off-the-shelf&#10;feature generators and have been shown to perform very well on a variety of&#10;tasks. Unfortunately, the generated features are high dimensional and expensive&#10;to store: potentially hundreds of thousands of floats per example when&#10;processing videos. Traditional entropy based lossless compression methods are&#10;of little help as they do not yield desired level of compression, while general&#10;purpose lossy compression methods based on energy compaction (e.g. PCA followed&#10;by quantization and entropy coding) are sub-optimal, as they are not tuned to&#10;task specific objective. We propose a learned method that jointly optimizes for&#10;compressibility along with the task objective for learning the features. The&#10;plug-in nature of our method makes it straight-forward to integrate with any&#10;target objective and trade-off against compressibility. We present results on&#10;multiple benchmarks and demonstrate that our method produces features that are&#10;an order of magnitude more compressible, while having a regularization effect&#10;that leads to a consistent improvement in accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.08870" label="2408.08870">
-        <attvalues>
-          <attvalue for="0" value="SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and&#10;  Medical Image Segmentation" />
-          <attvalue for="1" value="  Image segmentation plays an important role in vision understanding. Recently,&#10;the emerging vision foundation models continuously achieved superior&#10;performance on various tasks. Following such success, in this paper, we prove&#10;that the Segment Anything Model 2 (SAM2) can be a strong encoder for U-shaped&#10;segmentation models. We propose a simple but effective framework, termed&#10;SAM2-UNet, for versatile image segmentation. Specifically, SAM2-UNet adopts the&#10;Hiera backbone of SAM2 as the encoder, while the decoder uses the classic&#10;U-shaped design. Additionally, adapters are inserted into the encoder to allow&#10;parameter-efficient fine-tuning. Preliminary experiments on various downstream&#10;tasks, such as camouflaged object detection, salient object detection, marine&#10;animal segmentation, mirror detection, and polyp segmentation, demonstrate that&#10;our SAM2-UNet can simply beat existing specialized state-of-the-art methods&#10;without bells and whistles. Project page:&#10;\url{https://github.com/WZH0120/SAM2-UNet}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image segmentation is a crucial task in the field of computer vision, serving as the foundation for various visual understanding applications. By dividing an image into meaningful regions based on specific semantic criteria, image segmentation enables a wide array of downstream tasks in both natural and medical domains, such as camouflaged object detection~\cite{IJCAI21_C2FNet,CVPR22_ZoomNet}, salient object detection~\cite{AAAI20_F3Net,CVPR23_FEDER}, marine animal segmentation~\cite{JOE23_MASNet,TCSVT21_MAS3K}, mirror detection~\cite{AAAI23_HetNet,CVPR22_SAMirror}, and polyp segmentation~\cite{MICCAI20_PraNet,PR23_CFANet}. Many specialized architectures have been proposed to achieve superior performance on these different tasks, while it remains an open challenge to design a unified architecture to address the diverse segmentation tasks.&#10;&#10;The emergence of vision foundation models (VFMs)~\cite{ICCV23_SAM1,SAM2,ICCV23_SegGPT,CVPR24_OMGSeg} has introduced significant potential in the field of image segmentation. Among these VFMs, a notable example is the Segment Anything Model (SAM1)~\cite{ICCV23_SAM1} and its successor, Segment Anything 2 (SAM2)~\cite{SAM2}. SAM2 builds upon the foundation laid by SAM1, utilizing a larger dataset for training and incorporating improvements in architectural design. However, despite these advancements, SAM2 still produces class-agnostic segmentation results when no manual prompt is provided. This limitation highlights the ongoing challenge of effectively transferring SAM2 to downstream tasks, where task-specific or class-specific segmentation is often required. Exploring strategies to enhance SAM2's adaptability and performance in these scenarios remains an important area of research.&#10;&#10;To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}. Inspired by the strong segmentation capabilities of U-Net~\cite{MICCAI15_UNet} and its variants~\cite{TMI19_UNetPP,MedIA24_TransUNet,ECCVW22_SwinUNet}, some researchers have explored the possibility of transforming SAM into a U-shaped architecture~\cite{MICCAI24_DeSAM,MLMI23_MammoSAM}. However, these efforts have often been limited by the plain structure of the vanilla ViT encoder~\cite{ICLR21_ViT}, which lacks the hierarchy needed for more sophisticated segmentation tasks. Fortunately, the introduction of SAM2, which features a hierarchical backbone, opens new avenues for designing a U-shaped network with improved effectiveness.&#10;&#10;In this paper, we propose SAM2-UNet, the benefit of which is summarized as follows:&#10;\begin{itemize}&#10; \item Simplicity. SAM2-UNet adopts a classic U-shaped encoder-decoder architecture, known for its ease of use and high extensibility.&#10;&#10;\item Efficiency. Adapters are integrated into the encoder to enable parameter-efficient fine-tuning, allowing the model to be trained even on memory-limited devices.&#10;&#10;\item Effectiveness. Extensive experiments on eighteen public datasets demonstrate that SAM2-UNet delivers powerful performance across five challenging benchmarks.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2105.12555" label="2105.12555">
-        <attvalues>
-          <attvalue for="0" value="Context-aware Cross-level Fusion Network for Camouflaged Object&#10;  Detection" />
-          <attvalue for="1" value="  Camouflaged object detection (COD) is a challenging task due to the low&#10;boundary contrast between the object and its surroundings. In addition, the&#10;appearance of camouflaged objects varies significantly, e.g., object size and&#10;shape, aggravating the difficulties of accurate COD. In this paper, we propose&#10;a novel Context-aware Cross-level Fusion Network (C2F-Net) to address the&#10;challenging COD task. Specifically, we propose an Attention-induced Cross-level&#10;Fusion Module (ACFM) to integrate the multi-level features with informative&#10;attention coefficients. The fused features are then fed to the proposed&#10;Dual-branch Global Context Module (DGCM), which yields multi-scale feature&#10;representations for exploiting rich global context information. In C2F-Net, the&#10;two modules are conducted on high-level features using a cascaded manner.&#10;Extensive experiments on three widely used benchmark datasets demonstrate that&#10;our C2F-Net is an effective COD model and outperforms state-of-the-art models&#10;remarkably. Our code is publicly available at:&#10;https://github.com/thograce/C2FNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.02688" label="2203.02688">
-        <attvalues>
-          <attvalue for="0" value="Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object&#10;  Detection" />
-          <attvalue for="1" value="  The recently proposed camouflaged object detection (COD) attempts to segment&#10;objects that are visually blended into their surroundings, which is extremely&#10;complex and difficult in real-world scenarios. Apart from high intrinsic&#10;similarity between the camouflaged objects and their background, the objects&#10;are usually diverse in scale, fuzzy in appearance, and even severely occluded.&#10;To deal with these problems, we propose a mixed-scale triplet network,&#10;\textbf{ZoomNet}, which mimics the behavior of humans when observing vague&#10;images, i.e., zooming in and out. Specifically, our ZoomNet employs the zoom&#10;strategy to learn the discriminative mixed-scale semantics by the designed&#10;scale integration unit and hierarchical mixed-scale unit, which fully explores&#10;imperceptible clues between the candidate objects and background surroundings.&#10;Moreover, considering the uncertainty and ambiguity derived from&#10;indistinguishable textures, we construct a simple yet effective regularization&#10;constraint, uncertainty-aware loss, to promote the model to accurately produce&#10;predictions with higher confidence in candidate regions. Without bells and&#10;whistles, our proposed highly task-friendly model consistently surpasses the&#10;existing 23 state-of-the-art methods on four public datasets. Besides, the&#10;superior performance over the recent cutting-edge models on the SOD task also&#10;verifies the effectiveness and generality of our model. The code will be&#10;available at \url{https://github.com/lartpang/ZoomNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.15644" label="2211.15644">
-        <attvalues>
-          <attvalue for="0" value="Efficient Mirror Detection via Multi-level Heterogeneous Learning" />
-          <attvalue for="1" value="  We present HetNet (Multi-level \textbf{Het}erogeneous \textbf{Net}work), a&#10;highly efficient mirror detection network. Current mirror detection methods&#10;focus more on performance than efficiency, limiting the real-time applications&#10;(such as drones). Their lack of efficiency is aroused by the common design of&#10;adopting homogeneous modules at different levels, which ignores the difference&#10;between different levels of features. In contrast, HetNet detects potential&#10;mirror regions initially through low-level understandings (\textit{e.g.},&#10;intensity contrasts) and then combines with high-level understandings&#10;(contextual discontinuity for instance) to finalize the predictions. To perform&#10;accurate yet efficient mirror detection, HetNet follows an effective&#10;architecture that obtains specific information at different stages to detect&#10;mirrors. We further propose a multi-orientation intensity-based contrasted&#10;module (MIC) and a reflection semantic logical module (RSL), equipped on&#10;HetNet, to predict potential mirror regions by low-level understandings and&#10;analyze semantic logic in scenarios by high-level understandings, respectively.&#10;Compared to the state-of-the-art method, HetNet runs 664$\%$ faster and draws&#10;an average performance gain of 8.9$\%$ on MAE, 3.1$\%$ on IoU, and 2.0$\%$ on&#10;F-measure on two mirror detection benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.00714" label="2408.00714">
-        <attvalues>
-          <attvalue for="0" value="SAM 2: Segment Anything in Images and Videos" />
-          <attvalue for="1" value="  We present Segment Anything Model 2 (SAM 2), a foundation model towards&#10;solving promptable visual segmentation in images and videos. We build a data&#10;engine, which improves model and data via user interaction, to collect the&#10;largest video segmentation dataset to date. Our model is a simple transformer&#10;architecture with streaming memory for real-time video processing. SAM 2&#10;trained on our data provides strong performance across a wide range of tasks.&#10;In video segmentation, we observe better accuracy, using 3x fewer interactions&#10;than prior approaches. In image segmentation, our model is more accurate and 6x&#10;faster than the Segment Anything Model (SAM). We believe that our data, model,&#10;and insights will serve as a significant milestone for video segmentation and&#10;related perception tasks. We are releasing a version of our model, the dataset&#10;and an interactive demo.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.10229" label="2401.10229">
-        <attvalues>
-          <attvalue for="0" value="OMG-Seg: Is One Model Good Enough For All Segmentation?" />
-          <attvalue for="1" value="  In this work, we address various segmentation tasks, each traditionally&#10;tackled by distinct or partially unified models. We propose OMG-Seg, One Model&#10;that is Good enough to efficiently and effectively handle all the segmentation&#10;tasks, including image semantic, instance, and panoptic segmentation, as well&#10;as their video counterparts, open vocabulary settings, prompt-driven,&#10;interactive segmentation like SAM, and video object segmentation. To our&#10;knowledge, this is the first model to handle all these tasks in one model and&#10;achieve satisfactory performance. We show that OMG-Seg, a transformer-based&#10;encoder-decoder architecture with task-specific queries and outputs, can&#10;support over ten distinct segmentation tasks and yet significantly reduce&#10;computational and parameter overhead across various tasks and datasets. We&#10;rigorously evaluate the inter-task influences and correlations during&#10;co-training. Code and models are available at https://github.com/lxtGH/OMG-Seg.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.00480" label="2406.00480">
-        <attvalues>
-          <attvalue for="0" value="AlignSAM: Aligning Segment Anything Model to Open Context via&#10;  Reinforcement Learning" />
-          <attvalue for="1" value="  Powered by massive curated training data, Segment Anything Model (SAM) has&#10;demonstrated its impressive generalization capabilities in open-world scenarios&#10;with the guidance of prompts. However, the vanilla SAM is class agnostic and&#10;heavily relies on user-provided prompts to segment objects of interest.&#10;Adapting this method to diverse tasks is crucial for accurate target&#10;identification and to avoid suboptimal segmentation results. In this paper, we&#10;propose a novel framework, termed AlignSAM, designed for automatic prompting&#10;for aligning SAM to an open context through reinforcement learning. Anchored by&#10;an agent, AlignSAM enables the generality of the SAM model across diverse&#10;downstream tasks while keeping its parameters frozen. Specifically, AlignSAM&#10;initiates a prompting agent to iteratively refine segmentation predictions by&#10;interacting with the foundational model. It integrates a reinforcement learning&#10;policy network to provide informative prompts to the foundational models.&#10;Additionally, a semantic recalibration module is introduced to provide&#10;fine-grained labels of prompts, enhancing the model's proficiency in handling&#10;tasks encompassing explicit and implicit semantics. Experiments conducted on&#10;various challenging segmentation tasks among existing foundation models&#10;demonstrate the superiority of the proposed AlignSAM over state-of-the-art&#10;approaches. Project page:&#10;\url{https://github.com/Duojun-Huang/AlignSAM-CVPR2024}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.20076" label="2406.20076">
-        <attvalues>
-          <attvalue for="0" value="EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything&#10;  Model" />
-          <attvalue for="1" value="  Segment Anything Model (SAM) has attracted widespread attention for its&#10;superior interactive segmentation capabilities with visual prompts while&#10;lacking further exploration of text prompts. In this paper, we empirically&#10;investigate what text prompt encoders (e.g., CLIP or LLM) are good for adapting&#10;SAM for referring expression segmentation and introduce the Early&#10;Vision-language Fusion-based SAM (EVF-SAM). EVF-SAM is a simple yet effective&#10;referring segmentation method which exploits multimodal prompts (i.e., image&#10;and text) and comprises a pre-trained vision-language model to generate&#10;referring prompts and a SAM model for segmentation. Surprisingly, we observe&#10;that: (1) multimodal prompts and (2) vision-language models with early fusion&#10;(e.g., BEIT-3) are beneficial for prompting SAM for accurate referring&#10;segmentation. Our experiments show that the proposed EVF-SAM based on BEIT-3&#10;can obtain state-of-the-art performance on RefCOCO/+/g for referring expression&#10;segmentation and demonstrate the superiority of prompting SAM with early&#10;vision-language fusion. In addition, the proposed EVF-SAM with 1.32B parameters&#10;achieves remarkably higher performance while reducing nearly 82% of parameters&#10;compared to previous SAM methods based on large multimodal models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.15764" label="2406.15764">
-        <attvalues>
-          <attvalue for="0" value="TP-DRSeg: Improving Diabetic Retinopathy Lesion Segmentation with&#10;  Explicit Text-Prompts Assisted SAM" />
-          <attvalue for="1" value="  Recent advances in large foundation models, such as the Segment Anything&#10;Model (SAM), have demonstrated considerable promise across various tasks.&#10;Despite their progress, these models still encounter challenges in specialized&#10;medical image analysis, especially in recognizing subtle inter-class&#10;differences in Diabetic Retinopathy (DR) lesion segmentation. In this paper, we&#10;propose a novel framework that customizes SAM for text-prompted DR lesion&#10;segmentation, termed TP-DRSeg. Our core idea involves exploiting language cues&#10;to inject medical prior knowledge into the vision-only segmentation network,&#10;thereby combining the advantages of different foundation models and enhancing&#10;the credibility of segmentation. Specifically, to unleash the potential of&#10;vision-language models in the recognition of medical concepts, we propose an&#10;explicit prior encoder that transfers implicit medical concepts into explicit&#10;prior knowledge, providing explainable clues to excavate low-level features&#10;associated with lesions. Furthermore, we design a prior-aligned injector to&#10;inject explicit priors into the segmentation process, which can facilitate&#10;knowledge sharing across multi-modality features and allow our framework to be&#10;trained in a parameter-efficient fashion. Experimental results demonstrate the&#10;superiority of our framework over other traditional models and foundation model&#10;variants.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.03048" label="2305.03048">
-        <attvalues>
-          <attvalue for="0" value="Personalize Segment Anything Model with One Shot" />
-          <attvalue for="1" value="  Driven by large-data pre-training, Segment Anything Model (SAM) has been&#10;demonstrated as a powerful and promptable framework, revolutionizing the&#10;segmentation models. Despite the generality, customizing SAM for specific&#10;visual concepts without man-powered prompting is under explored, e.g.,&#10;automatically segmenting your pet dog in different images. In this paper, we&#10;propose a training-free Personalization approach for SAM, termed as PerSAM.&#10;Given only a single image with a reference mask, PerSAM first localizes the&#10;target concept by a location prior, and segments it within other images or&#10;videos via three techniques: target-guided attention, target-semantic&#10;prompting, and cascaded post-refinement. In this way, we effectively adapt SAM&#10;for private use without any training. To further alleviate the mask ambiguity,&#10;we present an efficient one-shot fine-tuning variant, PerSAM-F. Freezing the&#10;entire SAM, we introduce two learnable weights for multi-scale masks, only&#10;training 2 parameters within 10 seconds for improved performance. To&#10;demonstrate our efficacy, we construct a new segmentation dataset, PerSeg, for&#10;personalized evaluation, and test our methods on video object segmentation with&#10;competitive performance. Besides, our approach can also enhance DreamBooth to&#10;personalize Stable Diffusion for text-to-image generation, which discards the&#10;background disturbance for better target appearance learning. Code is released&#10;at https://github.com/ZrrSkywalker/Personalize-SAM&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.13310" label="2305.13310">
-        <attvalues>
-          <attvalue for="0" value="Matcher: Segment Anything with One Shot Using All-Purpose Feature&#10;  Matching" />
-          <attvalue for="1" value="  Powered by large-scale pre-training, vision foundation models exhibit&#10;significant potential in open-world image understanding. However, unlike large&#10;language models that excel at directly tackling various language tasks, vision&#10;foundation models require a task-specific model structure followed by&#10;fine-tuning on specific tasks. In this work, we present Matcher, a novel&#10;perception paradigm that utilizes off-the-shelf vision foundation models to&#10;address various perception tasks. Matcher can segment anything by using an&#10;in-context example without training. Additionally, we design three effective&#10;components within the Matcher framework to collaborate with these foundation&#10;models and unleash their full potential in diverse perception tasks. Matcher&#10;demonstrates impressive generalization performance across various segmentation&#10;tasks, all without training. For example, it achieves 52.7% mIoU on COCO-20$^i$&#10;with one example, surpassing the state-of-the-art specialist model by 1.6%. In&#10;addition, Matcher achieves 33.0% mIoU on the proposed LVIS-92$^i$ for one-shot&#10;semantic segmentation, outperforming the state-of-the-art generalist model by&#10;14.4%. Our visualization results further showcase the open-world generality and&#10;flexibility of Matcher when applied to images in the wild. Our code can be&#10;found at https://github.com/aim-uofa/Matcher.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.05074" label="1912.05074">
-        <attvalues>
-          <attvalue for="0" value="UNet++: Redesigning Skip Connections to Exploit Multiscale Features in&#10;  Image Segmentation" />
-          <attvalue for="1" value="  The state-of-the-art models for medical image segmentation are variants of&#10;U-Net and fully convolutional networks (FCN). Despite their success, these&#10;models have two limitations: (1) their optimal depth is apriori unknown,&#10;requiring extensive architecture search or inefficient ensemble of models of&#10;varying depths; and (2) their skip connections impose an unnecessarily&#10;restrictive fusion scheme, forcing aggregation only at the same-scale feature&#10;maps of the encoder and decoder sub-networks. To overcome these two&#10;limitations, we propose UNet++, a new neural architecture for semantic and&#10;instance segmentation, by (1) alleviating the unknown network depth with an&#10;efficient ensemble of U-Nets of varying depths, which partially share an&#10;encoder and co-learn simultaneously using deep supervision; (2) redesigning&#10;skip connections to aggregate features of varying semantic scales at the&#10;decoder sub-networks, leading to a highly flexible feature fusion scheme; and&#10;(3) devising a pruning scheme to accelerate the inference speed of UNet++. We&#10;have evaluated UNet++ using six different medical image segmentation datasets,&#10;covering multiple imaging modalities such as computed tomography (CT), magnetic&#10;resonance imaging (MRI), and electron microscopy (EM), and demonstrating that&#10;(1) UNet++ consistently outperforms the baseline models for the task of&#10;semantic segmentation across different datasets and backbone architectures; (2)&#10;UNet++ enhances segmentation quality of varying-size objects -- an improvement&#10;over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design)&#10;outperforms the original Mask R-CNN for the task of instance segmentation; and&#10;(4) pruned UNet++ models achieve significant speedup while showing only modest&#10;performance degradation. Our implementation and pre-trained models are&#10;available at https://github.com/MrGiovanni/UNetPlusPlus.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.05537" label="2105.05537">
-        <attvalues>
-          <attvalue for="0" value="Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation" />
-          <attvalue for="1" value="  In the past few years, convolutional neural networks (CNNs) have achieved&#10;milestones in medical image analysis. Especially, the deep neural networks&#10;based on U-shaped architecture and skip-connections have been widely applied in&#10;a variety of medical image tasks. However, although CNN has achieved excellent&#10;performance, it cannot learn global and long-range semantic information&#10;interaction well due to the locality of the convolution operation. In this&#10;paper, we propose Swin-Unet, which is an Unet-like pure Transformer for medical&#10;image segmentation. The tokenized image patches are fed into the&#10;Transformer-based U-shaped Encoder-Decoder architecture with skip-connections&#10;for local-global semantic feature learning. Specifically, we use hierarchical&#10;Swin Transformer with shifted windows as the encoder to extract context&#10;features. And a symmetric Swin Transformer-based decoder with patch expanding&#10;layer is designed to perform the up-sampling operation to restore the spatial&#10;resolution of the feature maps. Under the direct down-sampling and up-sampling&#10;of the inputs and outputs by 4x, experiments on multi-organ and cardiac&#10;segmentation tasks demonstrate that the pure Transformer-based U-shaped&#10;Encoder-Decoder network outperforms those methods with full-convolution or the&#10;combination of transformer and convolution. The codes and trained models will&#10;be publicly available at https://github.com/HuCaoFighting/Swin-Unet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.02270" label="2309.02270">
-        <attvalues>
-          <attvalue for="0" value="SAM-Deblur: Let Segment Anything Boost Image Deblurring" />
-          <attvalue for="1" value="  Image deblurring is a critical task in the field of image restoration, aiming&#10;to eliminate blurring artifacts. However, the challenge of addressing&#10;non-uniform blurring leads to an ill-posed problem, which limits the&#10;generalization performance of existing deblurring models. To solve the problem,&#10;we propose a framework SAM-Deblur, integrating prior knowledge from the Segment&#10;Anything Model (SAM) into the deblurring task for the first time. In&#10;particular, SAM-Deblur is divided into three stages. First, we preprocess the&#10;blurred images, obtain segment masks via SAM, and propose a mask dropout method&#10;for training to enhance model robustness. Then, to fully leverage the&#10;structural priors generated by SAM, we propose a Mask Average Pooling (MAP)&#10;unit specifically designed to average SAM-generated segmented areas, serving as&#10;a plug-and-play component which can be seamlessly integrated into existing&#10;deblurring networks. Finally, we feed the fused features generated by the MAP&#10;Unit into the deblurring model to obtain a sharp image. Experimental results on&#10;the RealBlurJ, ReloBlur, and REDS datasets reveal that incorporating our&#10;methods improves GoPro-trained NAFNet's PSNR by 0.05, 0.96, and 7.03,&#10;respectively. Project page is available at GitHub&#10;\href{https://hplqaq.github.io/projects/sam-deblur}{HPLQAQ/SAM-Deblur}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;When capturing images, factors like camera shake, defocusing, and high-speed motion can cause image blurring, impacting various practical applications \cite{d1}. Developing image deblurring models to transform blurred images into sharp ones can greatly improve downstream tasks such as object detection, autonomous driving, and visual navigation\cite{d2}. Therefore, image deblurring is a vital task in computer vision.&#10;&#10;Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks. MIRNet-v2 integrates multi-scale contextual information, while Uformer, a Transformer-based model, optimizes computational efficiency for high-resolution images. These models achieve impressive PSNR scores on benchmarks like SIDD \cite{d5} and GoPro \cite{d6}. However, they are computationally expensive. To address this issue, Chen et al. introduced NAFNet \cite{d7}, a streamlined and nonlinear activation free network that maintains state-of-the-art (SOTA) performance while reducing computational overhead.&#10;&#10;However, image deblurring is an ill-posed problem with a large solution space for non-uniform blur \cite{d1}, limiting model generalization. For example, the SOTA deblurring model NAFNet with 32 channels, trained on the GoPro dataset, can yield a high PSNR of 32.85 on in-distribution (ID) test sets. Nevertheless, its performance deteriorates on out-of-distribution (OOD) datasets, even experiencing &quot;mode collapse&quot; (shown in Fig. \ref{fig1}). This severely limits the practical potential of deblurring models.&#10;&#10;In this paper, to regularize the solution space of latent sharp images in deblurring tasks and thereby improve the generalization performance of existing models, we pioneer the use of semantic-aware priors from the pretrained Segment Anything Model (SAM) \cite{d8} to regularize the solution space in deblurring tasks. The motivation for using SAM as a semantic prior is based on its robustness to blurred images, which is due to SAM's pretraining on the expansive SA-1B dataset \cite{d8}, encompassing a staggering 10 billion masks and 11 million images. As shown in Fig. \ref{fig2}, even under suboptimal image quality, SAM succeeds in accurately segmenting most of objects. &#10;&#10;Some studies have applied SAM priors to image restoration tasks like dehazing \cite{hazing} by using grayscale coding and channel expansion, highlighting their utility in low-level visual tasks. However, due to the complex causes and larger solution space for non-uniform blur in deblurring, these methods are not directly transferable. To better adapt SAM for the task of deblurring, we propose a framework, SAM-Deblur, which for the first time incorporates SAM-generated priors to enhance image deblurring. Specifically, SAM-Deblur consists of three main stages. First, we preprocess blurred images and use SAM to obtain segment masks. For model training, we propose a mask dropout method during preprocessing to improve model robustness. Then, we propose a plug-and-play Mask Average Pooling (MAP) Unit designed for SAM, which averages segmented regions and can be readily integrated into existing deblurring networks. Finally, we use the fused features created by the MAP Unit as input to the deblurring model to achieve a sharp image. Extensively experiment on RealBlurJ \cite{realblur}, REDS \cite{reds}, and ReLoBlur \cite{relo} show that SAM priors improve deblurring performance and mitigate &quot;mode collapse&quot;.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Deep Learning, Computer Science, Image Restoration, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2201.10700" label="2201.10700">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Deblurring: A Survey" />
-          <attvalue for="1" value="  Image deblurring is a classic problem in low-level computer vision with the&#10;aim to recover a sharp image from a blurred input image. Advances in deep&#10;learning have led to significant progress in solving this problem, and a large&#10;number of deblurring networks have been proposed. This paper presents a&#10;comprehensive and timely survey of recently published deep-learning based image&#10;deblurring approaches, aiming to serve the community as a useful literature&#10;review. We start by discussing common causes of image blur, introduce benchmark&#10;datasets and performance metrics, and summarize different problem formulations.&#10;Next, we present a taxonomy of methods using convolutional neural networks&#10;(CNN) based on architecture, loss function, and application, offering a&#10;detailed review and comparison. In addition, we discuss some domain-specific&#10;deblurring applications including face images, text, and stereo image pairs. We&#10;conclude by discussing key challenges and future research directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.11745" label="2111.11745">
-        <attvalues>
-          <attvalue for="0" value="Intriguing Findings of Frequency Selection for Image Deblurring" />
-          <attvalue for="1" value="  Blur was naturally analyzed in the frequency domain, by estimating the latent&#10;sharp image and the blur kernel given a blurry image. Recent progress on image&#10;deblurring always designs end-to-end architectures and aims at learning the&#10;difference between blurry and sharp image pairs from pixel-level, which&#10;inevitably overlooks the importance of blur kernels. This paper reveals an&#10;intriguing phenomenon that simply applying ReLU operation on the frequency&#10;domain of a blur image followed by inverse Fourier transform, i.e., frequency&#10;selection, provides faithful information about the blur pattern (e.g., the blur&#10;direction and blur level, implicitly shows the kernel pattern). Based on this&#10;observation, we attempt to leverage kernel-level information for image&#10;deblurring networks by inserting Fourier transform, ReLU operation, and inverse&#10;Fourier transform to the standard ResBlock. 1x1 convolution is further added to&#10;let the network modulate flexible thresholds for frequency selection. We term&#10;our newly built block as Res FFT-ReLU Block, which takes advantages of both&#10;kernel-level and pixel-level features via learning frequency-spatial&#10;dual-domain representations. Extensive experiments are conducted to acquire a&#10;thorough analysis on the insights of the method. Moreover, after plugging the&#10;proposed block into NAFNet, we can achieve 33.85 dB in PSNR on GoPro dataset.&#10;Our method noticeably improves backbone architectures without introducing many&#10;parameters, while maintaining low computational complexity. Code is available&#10;at https://github.com/DeepMed-Lab/DeepRFT-AAAI2023.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.04676" label="2204.04676">
-        <attvalues>
-          <attvalue for="0" value="Simple Baselines for Image Restoration" />
-          <attvalue for="1" value="  Although there have been significant advances in the field of image&#10;restoration recently, the system complexity of the state-of-the-art (SOTA)&#10;methods is increasing as well, which may hinder the convenient analysis and&#10;comparison of methods. In this paper, we propose a simple baseline that exceeds&#10;the SOTA methods and is computationally efficient. To further simplify the&#10;baseline, we reveal that the nonlinear activation functions, e.g. Sigmoid,&#10;ReLU, GELU, Softmax, etc. are not necessary: they could be replaced by&#10;multiplication or removed. Thus, we derive a Nonlinear Activation Free Network,&#10;namely NAFNet, from the baseline. SOTA results are achieved on various&#10;challenging benchmarks, e.g. 33.69 dB PSNR on GoPro (for image deblurring),&#10;exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs;&#10;40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28&#10;dB with less than half of its computational costs. The code and the pre-trained&#10;models are released at https://github.com/megvii-research/NAFNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.15870" label="2306.15870">
-        <attvalues>
-          <attvalue for="0" value="Let Segment Anything Help Image Dehaze" />
-          <attvalue for="1" value="  The large language model and high-level vision model have achieved impressive&#10;performance improvements with large datasets and model sizes. However,&#10;low-level computer vision tasks, such as image dehaze and blur removal, still&#10;rely on a small number of datasets and small-sized models, which generally&#10;leads to overfitting and local optima. Therefore, we propose a framework to&#10;integrate large-model prior into low-level computer vision tasks. Just as with&#10;the task of image segmentation, the degradation of haze is also&#10;texture-related. So we propose to detect gray-scale coding, network channel&#10;expansion, and pre-dehaze structures to integrate large-model prior knowledge&#10;into any low-level dehazing network. We demonstrate the effectiveness and&#10;applicability of large models in guiding low-level visual tasks through&#10;different datasets and algorithms comparison experiments. Finally, we&#10;demonstrate the effect of grayscale coding, network channel expansion, and&#10;recurrent network structures through ablation experiments. Under the conditions&#10;where additional data and training resources are not required, we successfully&#10;prove that the integration of large-model prior knowledge will improve the&#10;dehaze performance and save training time for low-level visual tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08179" label="2204.08179">
-        <attvalues>
-          <attvalue for="0" value="Real-World Deep Local Motion Deblurring" />
-          <attvalue for="1" value="  Most existing deblurring methods focus on removing global blur caused by&#10;camera shake, while they cannot well handle local blur caused by object&#10;movements. To fill the vacancy of local deblurring in real scenes, we establish&#10;the first real local motion blur dataset (ReLoBlur), which is captured by a&#10;synchronized beam-splitting photographing system and corrected by a&#10;post-progressing pipeline. Based on ReLoBlur, we propose a Local Blur-Aware&#10;Gated network (LBAG) and several local blur-aware techniques to bridge the gap&#10;between global and local deblurring: 1) a blur detection approach based on&#10;background subtraction to localize blurred regions; 2) a gate mechanism to&#10;guide our network to focus on blurred regions; and 3) a blur-aware patch&#10;cropping strategy to address data imbalance problem. Extensive experiments&#10;prove the reliability of ReLoBlur dataset, and demonstrate that LBAG achieves&#10;better performance than state-of-the-art global deblurring methods without our&#10;proposed local blur-aware techniques.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.06281" label="2108.06281">
-        <attvalues>
-          <attvalue for="0" value="Modal-Adaptive Gated Recoding Network for RGB-D Salient Object Detection" />
-          <attvalue for="1" value="  The multi-modal salient object detection model based on RGB-D information has&#10;better robustness in the real world. However, it remains nontrivial to better&#10;adaptively balance effective multi-modal information in the feature fusion&#10;phase. In this letter, we propose a novel gated recoding network (GRNet) to&#10;evaluate the information validity of the two modes, and balance their&#10;influence. Our framework is divided into three phases: perception phase,&#10;recoding mixing phase and feature integration phase. First, A perception&#10;encoder is adopted to extract multi-level single-modal features, which lays the&#10;foundation for multi-modal semantic comparative analysis. Then, a&#10;modal-adaptive gate unit (MGU) is proposed to suppress the invalid information&#10;and transfer the effective modal features to the recoding mixer and the hybrid&#10;branch decoder. The recoding mixer is responsible for recoding and mixing the&#10;balanced multi-modal information. Finally, the hybrid branch decoder completes&#10;the multi-level feature integration under the guidance of an optional edge&#10;guidance stream (OEGS). Experiments and analysis on eight popular benchmarks&#10;verify that our framework performs favorably against 9 state-of-art methods.&#10;" />
-          <attvalue for="2" value="&#10;Salient object detection (SOD)~\cite{2019-SPL-EACNNet}\cite{2018-SPL-BGFANet}\cite{2016-SPL-DCMC} aims to identify object-level regions with strong visual impact that attract the attention of the human visual system most in an image~\cite{what-is-SOD1}\cite{2009-CVPR-Fm}. With the development of deep learning, deep features have replaced hand-crafted features~\cite{2014-ICIMCS-DES}\cite{2016-SPL-DCMC}\cite{2017-ICCVW-CDCP} as a powerful tool for SOD, and promote it to better serve application-oriented tasks~\cite{2010-ACMTOG-RepFinder}\cite{2009-ACMTOG-Sketch2Photo}\cite{2009-CVPR-tracking-SBDT}.&#10;&#10;The existing RGB-D SOD methods have proposed a variety of multi-level feature and multi-modal feature fusion techniques.&#10;\cite{2020-TIP-DPANet} uses Ostu algorithm~\cite{1979-TSMC-Ostu} to design supervised gate modules to suppress invalid depth information.&#10;\cite{2020-ECCV-GateNet} adopts the gated mechanism to suppress different level features in different degrees.&#10;\cite{2020-TIP-DPANet} proposes a three-stream structure and uses an attention mechanism to better fuse multi-modal features.&#10;However, there is still room for improvement.&#10;Different from the above methods, we provide more structure attempts for the gate unit and propose a four-stream structure with a dual recoding mixer.&#10;Besides, our gate unit does not need to specially design supervision labels like~\cite{2020-TIP-DPANet}.&#10;Let's introduce our network in order of phases.&#10;&#10;In this letter, from the perspective of multi-modal information balance, we propose a novel saliency detection method to suppress the invalid modal information and recode the balanced features.&#10;The whole network is a 4-stream structure and includes three phases: perception phase, recoding mixing phase, and feature integration phase.&#10;In the first phase, as shown in Fig.\ref{pipline}, the perception encoder consists of two parts (Encoder-A and Encoder-B), which are only responsible for processing single-modal data and providing multi-level features for the two-phase recoding mixer.&#10;MGU is the bridge between perception encoder, recoding mixer, and hybrid branch decoder.&#10;It senses and analyzes the corresponding multi-level features of the two modes, suppresses the inaccurate features, and transfers the balanced features to the recoding mixer.&#10;In the second phase, the recoding mixer (Mixer-A and Mixer-B) recodes the balanced multi-level multi-modal features in the way of step-by-step insertion.&#10;In the third phase, the hybrid branch decoder uses three classical feature integration structures for reference to further fuse the multi-level features provided by the mixer.&#10;The accurate low-level detail features in the encoder are used for cross-phase multi-modal edge guidance under the regulation of OEGS.&#10;&#10;Our contributions are summarized as follows:&#10;\begin{itemize}&#10;\item We propose MGU to compare and analyze the multi-level features obtained by two single-modal encoders to complete the feature validity evaluation.&#10; According to the evaluation results, MGU suppresses negative modal information to achieve multi-modal balance.&#10;\item We adopt a recoding mixer to recode the new features balanced by MGU to obtain more accurate multi-level features.&#10;The hybrid branch decoder integrates multi-level features under the guidance of OEGS.&#10;\item Sufficient experiments conducted on 8 RGB-D SOD datasets demonstrate that the proposed method outperforms 26 state-of-the-art methods.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science, Multi-Modal Fusion" />
-        </attvalues>
-      </node>
-      <node id="1809.10821" label="1809.10821">
-        <attvalues>
-          <attvalue for="0" value="Boundary-guided Feature Aggregation Network for Salient Object Detection" />
-          <attvalue for="1" value="  Fully convolutional networks (FCN) has significantly improved the performance&#10;of many pixel-labeling tasks, such as semantic segmentation and depth&#10;estimation. However, it still remains non-trivial to thoroughly utilize the&#10;multi-level convolutional feature maps and boundary information for salient&#10;object detection. In this paper, we propose a novel FCN framework to integrate&#10;multi-level convolutional features recurrently with the guidance of object&#10;boundary information. First, a deep convolutional network is used to extract&#10;multi-level feature maps and separately aggregate them into multiple&#10;resolutions, which can be used to generate coarse saliency maps. Meanwhile,&#10;another boundary information extraction branch is proposed to generate boundary&#10;features. Finally, an attention-based feature fusion module is designed to fuse&#10;boundary information into salient regions to achieve accurate boundary&#10;inference and semantic enhancement. The final saliency maps are the combination&#10;of the predicted boundary maps and integrated saliency maps, which are more&#10;closer to the ground truths. Experiments and analysis on four large-scale&#10;benchmarks verify that our framework achieves new state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.05174" label="1710.05174">
-        <attvalues>
-          <attvalue for="0" value="Saliency Detection for Stereoscopic Images Based on Depth Confidence&#10;  Analysis and Multiple Cues Fusion" />
-          <attvalue for="1" value="  Stereoscopic perception is an important part of human visual system that&#10;allows the brain to perceive depth. However, depth information has not been&#10;well explored in existing saliency detection models. In this letter, a novel&#10;saliency detection method for stereoscopic images is proposed. Firstly, we&#10;propose a measure to evaluate the reliability of depth map, and use it to&#10;reduce the influence of poor depth map on saliency detection. Then, the input&#10;image is represented as a graph, and the depth information is introduced into&#10;graph construction. After that, a new definition of compactness using color and&#10;depth cues is put forward to compute the compactness saliency map. In order to&#10;compensate the detection errors of compactness saliency when the salient&#10;regions have similar appearances with background, foreground saliency map is&#10;calculated based on depth-refined foreground seeds selection mechanism and&#10;multiple cues contrast. Finally, these two saliency maps are integrated into a&#10;final saliency map through weighted-sum method according to their importance.&#10;Experiments on two publicly available stereo datasets demonstrate that the&#10;proposed method performs better than other 10 state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.08074" label="2007.08074">
-        <attvalues>
-          <attvalue for="0" value="Suppress and Balance: A Simple Gated Network for Salient Object&#10;  Detection" />
-          <attvalue for="1" value="  Most salient object detection approaches use U-Net or feature pyramid&#10;networks (FPN) as their basic structures. These methods ignore two key problems&#10;when the encoder exchanges information with the decoder: one is the lack of&#10;interference control between them, the other is without considering the&#10;disparity of the contributions of different encoder blocks. In this work, we&#10;propose a simple gated network (GateNet) to solve both issues at once. With the&#10;help of multilevel gate units, the valuable context information from the&#10;encoder can be optimally transmitted to the decoder. We design a novel gated&#10;dual branch structure to build the cooperation among different levels of&#10;features and improve the discriminability of the whole network. Through the&#10;dual branch design, more details of the saliency map can be further restored.&#10;In addition, we adopt the atrous spatial pyramid pooling based on the proposed&#10;&quot;Fold&quot; operation (Fold-ASPP) to accurately localize salient objects of various&#10;scales. Extensive experiments on five challenging datasets demonstrate that the&#10;proposed model performs favorably against most state-of-the-art methods under&#10;different evaluation metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.05109" label="2303.05109">
-        <attvalues>
-          <attvalue for="0" value="Updated version: A Video Anomaly Detection Framework based on&#10;  Appearance-Motion Semantics Representation Consistency" />
-          <attvalue for="1" value="  Video anomaly detection is an essential but challenging task. The prevalent&#10;methods mainly investigate the reconstruction difference between normal and&#10;abnormal patterns but ignore the semantics consistency between appearance and&#10;motion information of behavior patterns, making the results highly dependent on&#10;the local context of frame sequences and lacking the understanding of behavior&#10;semantics. To address this issue, we propose a framework of Appearance-Motion&#10;Semantics Representation Consistency that uses the gap of appearance and motion&#10;semantic representation consistency between normal and abnormal data. The&#10;two-stream structure is designed to encode the appearance and motion&#10;information representation of normal samples, and a novel consistency loss is&#10;proposed to enhance the consistency of feature semantics so that anomalies with&#10;low consistency can be identified. Moreover, the lower consistency features of&#10;anomalies can be used to deteriorate the quality of the predicted frame, which&#10;makes anomalies easier to spot. Experimental results demonstrate the&#10;effectiveness of the proposed method.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Video anomaly detection (VAD) refers to identifying events that do not conform to expected behavior \cite{chandola2009anomaly}, which is of great practical value in public safety scenarios. In addition to much effort devoted in \cite{luo2017revisit, liu2018future}, VAD remains an extremely challenging task due to the rarity and ambiguity of anomalies \cite{chandola2009anomaly}. It is infeasible to collect balanced normal and abnormal samples and tackle this task with a supervised binary classification model. Therefore, a typical solution to VAD is often formulated as an unsupervised learning problem, where the goal is to train a model by using only normal data to mine regular patterns. The events that do not conform to this model are viewed as anomalies. &#10;&#10;Existing prevalent VAD methods almost follow a reconstruction or future frame prediction mode. Reconstruction-based methods \cite{hasan2016learning, luo2017remembering, gong2019memorizing, park2020learning} usually train AutoEncoders (AEs) on normal data and expect abnormal data to incur larger reconstruction errors at test time, making abnormal data detectable from normal ones. Prediction-based method \cite{liu2018future} uses the temporal characteristics of video frames to predict the next frame based on a given sequence of previous frames, then uses the prediction errors for anomaly measuring. However, such approaches are highly dependent on the local context of frame sequence, and some studies \cite{gong2019memorizing, liu2021hybrid} have shown that AEs trained only on normal data sometimes reconstruct anomalies well, which leads to poor performance. So some researchers \cite{xu2017detecting, yan2018abnormal, vu2019robust} try to utilize the motion information of activities, which contains a lot of semantics representing behavioral properties, to achieve good performance for VAD. Nevertheless, these methods only combine the information of appearance and motion to detect anomalies in the test phase and do not jointly model the two types of information in the same space during the training phase \cite{xu2017detecting, yan2018abnormal, vu2019robust, nguyen2019anomaly, ye2019anopcn}, which makes it difficult to capture the correlation between the two modalities for VAD. Recently, these state-of-the-art (SOTA) methods \cite{liu2021hybrid, cai2021appearance} are proposed to model the relationship of two modalities and can detect anomalies in most cases, but the results are still far from expectations due to the unstable relationship modeling method. For example, Cai et al. \cite{cai2021appearance} use a memory network to store the relationship of two modalities so that unsuitable-sized memory modules easily limit the network’s performance. Furthermore, Liu et al. \cite{liu2021hybrid} use a hybrid framework in a combination of flow reconstruction and frame prediction, but the result is highly dependent on the quality of flow reconstruction in the previous phase, which makes it difficult to train a stable model. Moreover, the correlations modeled by these methods are essentially designed for recovering the pixel information but still lack the understanding of behavior semantics.&#10;&#10;In this paper, we take a step forward in making full use of the multi-modal knowledge from normal events to detect anomalies via a simple yet novel network based on Appearance-Motion Semantics Representation Consistency, termed AMSRC-Net. Specifically, inspired by SOTA methods that use multiple modalities \cite{liu2021hybrid, yan2018abnormal, nguyen2019anomaly, cai2021appearance}, we first extract the representative features of appearance and motion in normal events by a universal two-stream encoder. Unlike previous works, we observe the consistency between the features of two modalities and propose a novel consistency loss to model the semantics consistency in the feature space explicitly. Then, the proposed network generates lower consistent features for abnormal samples, which typically reflect irregular behavior semantics and can be used to detect anomalies. Moreover, we design a simple flow-guided fusion module, which utilizes the above feature semantics consistency gap to augment the prediction quality gap. Extensive experiments on three public VAD datasets show that our proposed AMSRC-Net achieves better performance than SOTA methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Multimodal Representation Learning, Artificial Intelligence, Video Anomaly Detection, Computer Science, Semantic Consistency Analysis" />
-        </attvalues>
-      </node>
-      <node id="1604.04574" label="1604.04574">
-        <attvalues>
-          <attvalue for="0" value="Learning Temporal Regularity in Video Sequences" />
-          <attvalue for="1" value="  Perceiving meaningful activities in a long video sequence is a challenging&#10;problem due to ambiguous definition of 'meaningfulness' as well as clutters in&#10;the scene. We approach this problem by learning a generative model for regular&#10;motion patterns, termed as regularity, using multiple sources with very limited&#10;supervision. Specifically, we propose two methods that are built upon the&#10;autoencoders for their ability to work with little to no supervision. We first&#10;leverage the conventional handcrafted spatio-temporal local features and learn&#10;a fully connected autoencoder on them. Second, we build a fully convolutional&#10;feed-forward autoencoder to learn both the local features and the classifiers&#10;as an end-to-end learning framework. Our model can capture the regularities&#10;from multiple datasets. We evaluate our methods in both qualitative and&#10;quantitative ways - showing the learned regularity of videos in various aspects&#10;and demonstrating competitive performance on anomaly detection datasets as an&#10;application.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.02639" label="1904.02639">
-        <attvalues>
-          <attvalue for="0" value="Memorizing Normality to Detect Anomaly: Memory-augmented Deep&#10;  Autoencoder for Unsupervised Anomaly Detection" />
-          <attvalue for="1" value="  Deep autoencoder has been extensively used for anomaly detection. Training on&#10;the normal data, the autoencoder is expected to produce higher reconstruction&#10;error for the abnormal inputs than the normal ones, which is adopted as a&#10;criterion for identifying anomalies. However, this assumption does not always&#10;hold in practice. It has been observed that sometimes the autoencoder&#10;&quot;generalizes&quot; so well that it can also reconstruct anomalies well, leading to&#10;the miss detection of anomalies. To mitigate this drawback for autoencoder&#10;based anomaly detector, we propose to augment the autoencoder with a memory&#10;module and develop an improved autoencoder called memory-augmented autoencoder,&#10;i.e. MemAE. Given an input, MemAE firstly obtains the encoding from the encoder&#10;and then uses it as a query to retrieve the most relevant memory items for&#10;reconstruction. At the training stage, the memory contents are updated and are&#10;encouraged to represent the prototypical elements of the normal data. At the&#10;test stage, the learned memory will be fixed, and the reconstruction is&#10;obtained from a few selected memory records of the normal data. The&#10;reconstruction will thus tend to be close to a normal sample. Thus the&#10;reconstructed errors on anomalies will be strengthened for anomaly detection.&#10;MemAE is free of assumptions on the data type and thus general to be applied to&#10;different tasks. Experiments on various datasets prove the excellent&#10;generalization and high effectiveness of the proposed MemAE.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.06852" label="2108.06852">
-        <attvalues>
-          <attvalue for="0" value="A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow&#10;  Reconstruction and Flow-Guided Frame Prediction" />
-          <attvalue for="1" value="  In this paper, we propose $\text{HF}^2$-VAD, a Hybrid framework that&#10;integrates Flow reconstruction and Frame prediction seamlessly to handle Video&#10;Anomaly Detection. Firstly, we design the network of ML-MemAE-SC (Multi-Level&#10;Memory modules in an Autoencoder with Skip Connections) to memorize normal&#10;patterns for optical flow reconstruction so that abnormal events can be&#10;sensitively identified with larger flow reconstruction errors. More&#10;importantly, conditioned on the reconstructed flows, we then employ a&#10;Conditional Variational Autoencoder (CVAE), which captures the high correlation&#10;between video frame and optical flow, to predict the next frame given several&#10;previous frames. By CVAE, the quality of flow reconstruction essentially&#10;influences that of frame prediction. Therefore, poorly reconstructed optical&#10;flows of abnormal events further deteriorate the quality of the final predicted&#10;future frame, making the anomalies more detectable. Experimental results&#10;demonstrate the effectiveness of the proposed method. Code is available at&#10;\href{https://github.com/LiUzHiAn/hf2vad}{https://github.com/LiUzHiAn/hf2vad}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.10192" label="2105.10192">
-        <attvalues>
-          <attvalue for="0" value="Pyramid Fusion Dark Channel Prior for Single Image Dehazing" />
-          <attvalue for="1" value="  In this paper, we propose the pyramid fusion dark channel prior (PF-DCP) for&#10;single image dehazing. Based on the well-known Dark Channel Prior (DCP), we&#10;introduce an easy yet effective approach PF-DCP by employing the DCP algorithm&#10;at a pyramid of multi-scale images to alleviate the problem of patch size&#10;selection. In this case, we obtain the final transmission map by fusing&#10;transmission maps at each level to recover a high-quality haze-free image.&#10;Experiments on RESIDE SOTS show that PF-DCP not only outperforms the&#10;traditional prior-based methods with a large margin but also achieves&#10;comparable or even better results of state-of-art deep learning approaches.&#10;Furthermore, the visual quality is also greatly improved with much fewer color&#10;distortions and halo artifacts.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Haze is ubiquitous in life, which comes from fog, sandstorms, or pollutants in the air. Haze in the air largely degrades the quality of the photo. Many details of the scene become blurry and the visibility is reduced. From the aspect of aesthetic, the images without haze are much clearer and visually pleasing than the hazy ones. For these reasons, haze removal has been studied for decades, yet still a challenging task in computer vision.&#10;&#10;The key problem of haze removal is the strategy to dehaze an image. Hautiére et al. \cite{hautiere2006automatic} proposed an equation to model hazy images captured by the camera:&#10;\begin{equation}&#10; I(x) = J(x)t(x) + A(1-t(x)),&#10;\end{equation}&#10;where $I$ is the observed hazy image, $J$ is the real scene radiance, $A$ is the atmospheric light, $t$ is the transmission rate. The goal of the image dehazing is to recover haze-free scene $J$, with only the knowledge of $I$. If we are able to estimate $A$ and $t$, we can recover the real scene $J$ as follows:&#10;\begin{equation}&#10; J(x) = \frac{I(x)-A}{t(x)}+A.&#10;\end{equation}&#10;&#10;However, $A$ and $t$ are both unknown, and it is impossible to estimate their values from a single hazy image without any constraint. Early approaches developed a strong prior or assumption of the difference between the haze-free and hazy images. Tan \cite{tan2008visibility} maximized the local contrast to restore the clear images given the fact that clear images have higher contrast compared to hazy ones. He et al. \cite{he2010single} utilized the prior knowledge that clear images have a dark channel whereas hazy images do not. Meng et al. \cite{meng2013efficient} introduced a boundary constraint to help formulate the transmission map. \cite{zhu2015fast} proposed a color attenuation prior (CAP), which models a linear relationship between the scene depth, brightness, and saturation. Berman et al. \cite{berman2016non} suggested a non-local prior (NLP), using only 500 distinct colors to quantize the image and clustering different pixels according to their colors into haze-lines for dehazing.&#10;&#10;In the emerging years of deep learning, researchers focus on building models to generate the transmission map, $t(x)$, and atmospheric light, $A$, separately. Cai et al. \cite{cai2016dehazenet} used a convolutional neural network (CNN) to estimate the transmission rate and use the atmospheric scattering model to dehaze the image. Ren et al. \cite{ren2016single} introduced a multi-scale CNN model to improve the quality of the transmission map. However, estimating two parameters individually might cause larger errors. Li et al. \cite{li2017aod} unified the transmission map and the atmospheric light into one parameter and use it to estimate the clear image. Later, Ren et al. \cite{ren2018gated} designed a coarse-to-fine fusion network, which recovers the clear image directly from the hazy one. Deep learning has gained huge success by surpassing the physical models built by humans and exploring the tremendous power of computation. Nevertheless, it is troublesome to train a deep learning model so that it performs well. Not only a large labeled dataset is required, but also the training process is time-consuming, let alone the problems of interpretability and robustness \cite{marcus2018deep}, which is required to be employed for productization. &#10;&#10;Among all traditional algorithms, due to the effectiveness and robustness, DCP serves as a building block to many haze-removal tasks \cite{xu2012fast,pan2016blind}. Besides, DCP is also broadly used as a standard baseline of the performance evaluation in recent works using deep learning \cite{cai2016dehazenet, li2017aod, ren2018gated, qu2019enhanced, zhang2018densely, dong2020fd}. However, DCP still suffers from the problem of patch size selection. Inappropriate patch size would introduce an inaccurate estimation of the global atmospheric light and a poor transmission map \cite{lee2016review}, which leads to dis-satisfactory haze removal results.&#10;&#10;In this paper, we propose the pyramid fusion dark channel prior (PF-DCP) using an iterative weighted fusion strategy to address the problem of patch size selection. Through incorporating multi-scale information of the transmission map, we demonstrate that the proposed method outperforms DCP by a large margin with a similar computational cost. The halo artifacts of the original DCP are greatly reduced, and the dehazed images become much smoother in colors and visually pleasing. Although the idea of using multi-scale information is well-established in various computer vision tasks, its application as a traditional algorithm in this field has not been proposed yet. The novelty of this paper lies in two aspects by solving two critical problems. First, there is no evaluation metric to select the best patch size. Thus, we adopt pyramid fusion among different sizes to alleviate the problem. Second, the process of transmission map refinement helps maintain a similar level of sharpness at the final dehazed images, which prevents the algorithm from degrading the image quality. Furthermore, as a traditional algorithm that is interpretable and robust, PF-DCP manages to produce superior results compared to other prior-based traditional algorithms and commensurable to the latest deep learning methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Image Processing, Physics, Deep Learning, Computer Science, Image Dehazing" />
-        </attvalues>
-      </node>
-      <node id="1601.07661" label="1601.07661">
-        <attvalues>
-          <attvalue for="0" value="DehazeNet: An End-to-End System for Single Image Haze Removal" />
-          <attvalue for="1" value="  Single image haze removal is a challenging ill-posed problem. Existing&#10;methods use various constraints/priors to get plausible dehazing solutions. The&#10;key to achieve haze removal is to estimate a medium transmission map for an&#10;input hazy image. In this paper, we propose a trainable end-to-end system&#10;called DehazeNet, for medium transmission estimation. DehazeNet takes a hazy&#10;image as input, and outputs its medium transmission map that is subsequently&#10;used to recover a haze-free image via atmospheric scattering model. DehazeNet&#10;adopts Convolutional Neural Networks (CNN) based deep architecture, whose&#10;layers are specially designed to embody the established assumptions/priors in&#10;image dehazing. Specifically, layers of Maxout units are used for feature&#10;extraction, which can generate almost all haze-relevant features. We also&#10;propose a novel nonlinear activation function in DehazeNet, called Bilateral&#10;Rectified Linear Unit (BReLU), which is able to improve the quality of&#10;recovered haze-free image. We establish connections between components of the&#10;proposed DehazeNet and those used in existing methods. Experiments on benchmark&#10;images show that DehazeNet achieves superior performance over existing methods,&#10;yet keeps efficient and easy to use.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.00631" label="1801.00631">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning: A Critical Appraisal" />
-          <attvalue for="1" value="  Although deep learning has historical roots going back decades, neither the&#10;term &quot;deep learning&quot; nor the approach was popular just over five years ago,&#10;when the field was reignited by papers such as Krizhevsky, Sutskever and&#10;Hinton's now classic (2012) deep network model of Imagenet. What has the field&#10;discovered in the five subsequent years? Against a background of considerable&#10;progress in areas such as speech recognition, image recognition, and game&#10;playing, and considerable enthusiasm in the popular press, I present ten&#10;concerns for deep learning, and suggest that deep learning must be supplemented&#10;by other techniques if we are to reach artificial general intelligence.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.08531" label="1912.08531">
-        <attvalues>
-          <attvalue for="0" value="GlobalTrack: A Simple and Strong Baseline for Long-term Tracking" />
-          <attvalue for="1" value="  A key capability of a long-term tracker is to search for targets in very&#10;large areas (typically the entire image) to handle possible target absences or&#10;tracking failures. However, currently there is a lack of such a strong baseline&#10;for global instance search. In this work, we aim to bridge this gap.&#10;Specifically, we propose GlobalTrack, a pure global instance search based&#10;tracker that makes no assumption on the temporal consistency of the target's&#10;positions and scales. GlobalTrack is developed based on two-stage object&#10;detectors, and it is able to perform full-image and multi-scale search of&#10;arbitrary instances with only a single query as the guide. We further propose a&#10;cross-query loss to improve the robustness of our approach against distractors.&#10;With no online learning, no punishment on position or scale changes, no scale&#10;smoothing and no trajectory refinement, our pure global instance search based&#10;tracker achieves comparable, sometimes much better performance on four&#10;large-scale tracking benchmarks (i.e., 52.1% AUC on LaSOT, 63.8% success rate&#10;on TLP, 60.3% MaxGM on OxUvA and 75.4% normalized precision on TrackingNet),&#10;compared to state-of-the-art approaches that typically require complex&#10;post-processing. More importantly, our tracker runs without cumulative errors,&#10;i.e., any type of temporary tracking failures will not affect its performance&#10;on future frames, making it ideal for long-term tracking. We hope this work&#10;will be a strong baseline for long-term tracking and will stimulate future&#10;works in this area. Code is available at&#10;https://github.com/huanglianghua/GlobalTrack.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Long-term tracking refers to the task of continuously locating an arbitrary target in a relatively long video, where the target may temporarily disappear~\cite{oxuva2018}.&#10;A key challenge of long-term tracking is to retrieve the target after a period of target absence or tracking failures.&#10;To our knowledge, only limited works focus on this task~\cite{tld2012,lct2015,bmvc19,icip19}.&#10;Representive approaches include TLD~\cite{tld2012}, SPL~\cite{spl2013}, LCT~\cite{lct2015}, EBT~\cite{ebt2016} and DaSiam\_LT~\cite{dasiamrpn2018}.&#10;TLD, SPL, LCT and EBT are capable of performing full-image search, which is an important ability, especially when the target can disappear. TLD, SPL and LCT searches globally for the target when they detect tracking failures, while EBT always perform full-image search.&#10;DaSiam\_LT~\cite{dasiamrpn2018} is a variant of DaSiamRPN~\cite{dasiamrpn2018} for long-term tracking, which expands the search area as it detects tracking failures.&#10;Similar to these trackers, our approach GlobalTrack can perform full-image search for the target. Differently, we use no online learning and impose no constraints on temporal consistency of the target's location or scale to avoid cumulative errors.&#10;&#10;In terms of the tracking framework, our approach share some similarities with SiamRPN~\cite{siamrpn2018} and ATOM~\cite{atom2019}, where they all use shared backbones for extracting features of the query and search images, and they use correlation to encode the relationship between them.&#10;However, our work has significant differences with these methods: (1) Our approach performs global search for the target in all frames, without relying on any locality assumption, while ATOM and SiamRPN search only locally and they use complex post-processing to impose constraints on the target's location and scale changes; (2) We reuse the RPN and RCNN heads of Faster-RCNN and apply them on the modulated features to generate proposals and final predictions. However, SiamRPN uses the query to generate classification and localization weights, while ATOM learns a hand-crafted classifier from scratch during tracking.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Computer Science, Image Search" />
-        </attvalues>
-      </node>
-      <node id="2107.03000" label="2107.03000">
-        <attvalues>
-          <attvalue for="0" value="PoseRN: A 2D pose refinement network for bias-free multi-view 3D human&#10;  pose estimation" />
-          <attvalue for="1" value="  We propose a new 2D pose refinement network that learns to predict the human&#10;bias in the estimated 2D pose. There are biases in 2D pose estimations that are&#10;due to differences between annotations of 2D joint locations based on&#10;annotators' perception and those defined by motion capture (MoCap) systems.&#10;These biases are crafted into publicly available 2D pose datasets and cannot be&#10;removed with existing error reduction approaches. Our proposed pose refinement&#10;network allows us to efficiently remove the human bias in the estimated 2D&#10;poses and achieve highly accurate multi-view 3D human pose estimation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Early works on 3D human pose estimation have using convolutional neural networks (CNN)~\cite{kolotouros2019learning, cheng2019occlusion, Sun2018integral, Martinez_2017_ICCV,zhou2017towards, NibaliHMP19} have focused on using a single image as input.&#10;This is an ill-posed problem and as a consequence, such CNN-based methods rely much on the knowledge from the dataset. However, unlike 2D human pose datasets, publicly available MoCap-based 3D human pose datasets only contain images of a limited number of persons taken in a controlled environment and with tight clothes.&#10;&#10;To take advantage of the publicly available large 2D datasets, methods were proposed to estimate 3D human poses from multi-view 2D images~\cite{iskakov2019learnable, multiviewpose, Joo2017panoptic, zhou2019hemlets, pavlakos2017harvesting, tome2018rethinking}. &#10;The core idea is to first estimate 2D poses in each view by using a 2D pose estimation network and then triangulate them to generate a 3D pose. To cope with noise in the 2D pose estimate two strategies exist: (1) defining a robust 3D triangulation algorithm~\cite{iskakov2019learnable, Joo2017panoptic, zhou2019hemlets} or (2) refining the 2D pose estimates prior to doing the 3D triangulation~\cite{multiviewpose, pavlakos2017harvesting, tome2018rethinking}.&#10;&#10;In this work, we take the second approach. We observed in the commonly used datasets like Human3.6M \cite{ionescu2013human3} that there are consistent errors between projections of 3D poses and the corresponding 2D estimates (or even annotations).&#10;We reason that errors in 2D pose estimates do not come solely from computational errors but also come from consistent displacements due to human perception bias that depends on the viewpoint and the pose. Unlike computational errors, biases cannot be removed by using averaging strategies. &#10;&#10;We propose a new 2D pose refinement network that leverages both 2D and 3D information to un-bias the initial 2D pose estimates and achieve accurate 3D pose estimation. Our core idea is to build a network, dubbed PoseRN~(\ref{fig:PoseRN}), that learns to predict the 2D bias between the initial multi-view 2D poses and the anatomical ones defined by the (few) available MoCap-based 3D datasets. The 2D human biases depend on the camera viewpoint and the human pose. We reason that the camera viewpoint information is contained into the estimated 2D pose, while the human pose information is contained in the initial 3D pose estimate. &#10;&#10;In summary, our main contributions are (1) a marker-less multi-view 3D pose estimation method, which outperforms previous methods; (2) a pose-refining network, PoseRN, to rectify human perceptional 2D annotations to anatomical ones, which is defined by MoCap systems.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Engineering, Biomechanics, Human-Computer Interaction, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="1905.05754" label="1905.05754">
-        <attvalues>
-          <attvalue for="0" value="Learnable Triangulation of Human Pose" />
-          <attvalue for="1" value="  We present two novel solutions for multi-view 3D human pose estimation based&#10;on new learnable triangulation methods that combine 3D information from&#10;multiple 2D views. The first (baseline) solution is a basic differentiable&#10;algebraic triangulation with an addition of confidence weights estimated from&#10;the input images. The second solution is based on a novel method of volumetric&#10;aggregation from intermediate 2D backbone feature maps. The aggregated volume&#10;is then refined via 3D convolutions that produce final 3D joint heatmaps and&#10;allow modelling a human pose prior. Crucially, both approaches are end-to-end&#10;differentiable, which allows us to directly optimize the target metric. We&#10;demonstrate transferability of the solutions across datasets and considerably&#10;improve the multi-view state of the art on the Human3.6M dataset. Video&#10;demonstration, annotations and additional materials will be posted on our&#10;project page (https://saic-violet.github.io/learnable-triangulation).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06039" label="1804.06039">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Rotation-Invariant Face Detection with Progressive Calibration&#10;  Networks" />
-          <attvalue for="1" value="  Rotation-invariant face detection, i.e. detecting faces with arbitrary&#10;rotation-in-plane (RIP) angles, is widely required in unconstrained&#10;applications but still remains as a challenging task, due to the large&#10;variations of face appearances. Most existing methods compromise with speed or&#10;accuracy to handle the large RIP variations. To address this problem more&#10;efficiently, we propose Progressive Calibration Networks (PCN) to perform&#10;rotation-invariant face detection in a coarse-to-fine manner. PCN consists of&#10;three stages, each of which not only distinguishes the faces from non-faces,&#10;but also calibrates the RIP orientation of each face candidate to upright&#10;progressively. By dividing the calibration process into several progressive&#10;steps and only predicting coarse orientations in early stages, PCN can achieve&#10;precise and fast calibration. By performing binary classification of face vs.&#10;non-face with gradually decreasing RIP ranges, PCN can accurately detect faces&#10;with full $360^{\circ}$ RIP angles. Such designs lead to a real-time&#10;rotation-invariant face detector. The experiments on multi-oriented FDDB and a&#10;challenging subset of WIDER FACE containing rotated faces in the wild show that&#10;our PCN achieves quite promising performance.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Face detection serves as an important component in computer vision systems which aim to extract information from face images. Practical applications, such as face recognition and face animation, all need to quickly and accurately detect faces on input images in advance. Same as many other vision tasks, the performance of face detection has been substantially improved by Convolutional Neural Network (CNN) \cite{farfade2015multi, ren2015faster, Liu2015SSD, li2015convolutional, yang2015facial, zhang2016joint, jiang2017face, Hu_2017_CVPR, Najibi_2017_ICCV}. The CNN-based detectors enjoy the natural advantage of strong capability in non-linear feature learning. However, most works focus on designing an effective detector for generic faces without considerations for specific scenarios, such as detecting faces with full rotation-in-plane (RIP) angles as shown in Figure \ref{fig1}. They become less satisfactory in such complex applications. Face detection in full RIP, i.e. rotation-invariant face detection, is quite challenging, because faces can be captured almost from any RIP angle, leading to significant divergence in face appearances. An accurate rotation-invariant face detector can greatly boost the performance of subsequent process, e.g. face alignment and face recognition.&#10;&#10;Generally, there are three strategies for dealing with the rotation variations including data augmentation, divide-and-conquer, and rotation router \cite{rowley1998rotation}, detailed as follows.&#10;&#10;Data Augmentation is the most straightforward solution for training a rotation-invariant face detector, which augments the training data by uniformly rotating the upright faces to full RIP angles. The advantage of this strategy is that the same scheme as that of the upright face detectors can be directly used without extra operations. However, to characterize such large variations of face appearances in single detector, one usually needs to use large neural networks with high time cost, which is not practical in many applications.&#10;&#10;Divide-and-Conquer is another commonly used method for dealing with this problem which trains multiple detectors, one for a small range of RIP angles, to cover the full RIP range such as \cite{huang2007high}. For example, four detectors covering the faces facing up, down, left and right respectively are constructed to detect the faces in full RIP angles, as shown in Figure \ref{fig2b}. As one detector only deals with a small range of face appearance variations, and thus a small neural network with low time cost is enough for each detector. However, the overall time cost of running multiple detectors grows and more false alarms are easily introduced.&#10;&#10;Rotation Router The large appearance variations of rotated faces come from their diverse RIP angles. Thus, a natural way is to estimate the faces' RIP angles explicitly, and then rotate them to upright, significantly reducing appearance variations of faces. In \cite{rowley1998rotation}, a router network is firstly used to estimate each face candidate's RIP angle, and then the candidates are calibrated to face upright, as shown in Figure \ref{fig2c}. After this step, an upright face detector can easily process the calibrated face candidates. Obviously, an inaccurate estimation of the RIP angle will cause miss detection of faces, leading to a lower recall. However, precisely estimating the RIP angles of faces is quite challenging, so a large neural network is usually used as the router network, resulting in high time cost.&#10;&#10;To solve the problems above, we propose a real-time and accurate rotation-invariant face detector with progressive calibration networks (PCN), as shown in Figure \ref{fig3}. Our PCN progressively calibrates the RIP orientation of each face candidate to upright for better distinguishing faces from non-faces. Specifically, PCN first identifies face candidates and calibrates those facing down to facing up, halving the range of RIP angles from $[-180^{\circ}, 180^{\circ}]$ to $[-90^{\circ}, 90^{\circ}]$. Then the rotated face candidates are further distinguished and calibrated to an upright range of $[-45^{\circ}, 45^{\circ}]$, shrinking the RIP range by half again. Finally, PCN makes the accurate final decision for each face candidate to determine whether it is a face and predict the precise RIP angle. By dividing the calibration process into several progressive steps and only predicting coarse orientations in early stages, PCN can achieve precise calibration. And the calibration process can be implemented as rotating original image by $-90^{\circ}$, $90^{\circ}$, and $180^{\circ}$ with quite low time cost. By performing binary classification of face vs. non-face with gradually decreasing RIP ranges, PCN can accurately detect faces with full $360^{\circ}$ RIP angles. Such designs lead to a real-time rotation-invariant face detector.&#10;&#10;Briefly, the novelties and advantages of our proposed method are listed as below:&#10;\begin{itemize}&#10;\item Our PCN divides the calibration process into several progressive steps, each of which is an easy task, resulting in accurate calibration with low time cost. And the range of RIP angles is gradually decreasing, which helps distinguish faces from non-faces.&#10;\item In the first two stages of our PCN, only coarse calibrations are conducted, such as calibrations from facing down to facing up, and from facing left to facing right. On the one hand, a robust and accurate RIP angle prediction for this coarse calibration is easier to attain without extra time cost, by jointly learning calibration task with the classification task and bounding box regression task in a multi-task learning manner. On the other hand, the calibration can be easier to implement as flipping original image with quite low time cost.&#10;\item As evaluated on the face detection datasets including multi-oriented FDDB \cite{jain2010fddb} and a challenging subset of WIDER FACE \cite{yang2016wider} containing rotated faces in the wild, the PCN detector achieves quite promising performance with extremely fast speed.&#10;\end{itemize}&#10;&#10;The rest of the paper is organized as follows. Section ~\ref{sec:2} describes the proposed PCN detector in detail, explaining the design of different stages. Section ~\ref{sec:3} presents the experimental results on two challenging face detection datasets together with analysis on the stage-wise calibration. The final Section ~\ref{sec:4} concludes this work.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Image Processing, Face Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1502.02766" label="1502.02766">
-        <attvalues>
-          <attvalue for="0" value="Multi-view Face Detection Using Deep Convolutional Neural Networks" />
-          <attvalue for="1" value="  In this paper we consider the problem of multi-view face detection. While&#10;there has been significant research on this problem, current state-of-the-art&#10;approaches for this task require annotation of facial landmarks, e.g. TSM [25],&#10;or annotation of face poses [28, 22]. They also require training dozens of&#10;models to fully capture faces in all orientations, e.g. 22 models in HeadHunter&#10;method [22]. In this paper we propose Deep Dense Face Detector (DDFD), a method&#10;that does not require pose/landmark annotation and is able to detect faces in a&#10;wide range of orientations using a single model based on deep convolutional&#10;neural networks. The proposed method has minimal complexity; unlike other&#10;recent deep learning object detection methods [9], it does not require&#10;additional components such as segmentation, bounding-box regression, or SVM&#10;classifiers. Furthermore, we analyzed scores of the proposed face detector for&#10;faces in different orientations and found that 1) the proposed method is able&#10;to detect faces from different angles and can handle occlusion to some extent,&#10;2) there seems to be a correlation between dis- tribution of positive examples&#10;in the training set and scores of the proposed face detector. The latter&#10;suggests that the proposed methods performance can be further improved by using&#10;better sampling strategies and more sophisticated data augmentation techniques.&#10;Evaluations on popular face detection benchmark datasets show that our&#10;single-model face detector algorithm has similar or better performance compared&#10;to the previous methods, which are more complex and require annotations of&#10;either different poses or facial landmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.13769" label="2210.13769">
-        <attvalues>
-          <attvalue for="0" value="GlobalFlowNet: Video Stabilization using Deep Distilled Global Motion&#10;  Estimates" />
-          <attvalue for="1" value="  Videos shot by laymen using hand-held cameras contain undesirable shaky&#10;motion. Estimating the global motion between successive frames, in a manner not&#10;influenced by moving objects, is central to many video stabilization&#10;techniques, but poses significant challenges. A large body of work uses 2D&#10;affine transformations or homography for the global motion. However, in this&#10;work, we introduce a more general representation scheme, which adapts any&#10;existing optical flow network to ignore the moving objects and obtain a&#10;spatially smooth approximation of the global motion between video frames. We&#10;achieve this by a knowledge distillation approach, where we first introduce a&#10;low pass filter module into the optical flow network to constrain the predicted&#10;optical flow to be spatially smooth. This becomes our student network, named as&#10;\textsc{GlobalFlowNet}. Then, using the original optical flow network as the&#10;teacher network, we train the student network using a robust loss function.&#10;Given a trained \textsc{GlobalFlowNet}, we stabilize videos using a two stage&#10;process. In the first stage, we correct the instability in affine parameters&#10;using a quadratic programming approach constrained by a user-specified cropping&#10;limit to control loss of field of view. In the second stage, we stabilize the&#10;video further by smoothing global motion parameters, expressed using a small&#10;number of discrete cosine transform coefficients. In extensive experiments on a&#10;variety of different videos, our technique outperforms state of the art&#10;techniques in terms of subjective quality and different quantitative measures&#10;of video stability. The source code is publicly available at&#10;\href{https://github.com/GlobalFlowNet/GlobalFlowNet}{https://github.com/GlobalFlowNet/GlobalFlowNet}&#10;" />
-          <attvalue for="2" value="&#10;Videos acquired by amateur photographers or lay users from hand-held cameras or mobile phones are subject to a large magnitude of undesirable and discontinuous motion. The process of eliminating or reducing this undesirable motion is called video stabilization. In some setups, the camera can be mounted on stable stands or dollies, but this is infeasible in many commonplace scenarios. Some cameras are equipped with hardware such as gyroscopes for stabilization, but the state of the art in video stabilization still adopts software-based approaches due to the gyroscope's cost, weight and error-pone motion estimation \cite{Milanovic2021, Shi_2022_WACV}. Apart from casual hand-held photography, the need for video stabilization also arises in endoscopy \cite{Karargyris2014}, underwater imaging \cite{Oreifej2011} and aerial photography from drones/helicopters \cite{Lim2019}. Many video stabilization techniques consist of three broad steps: (1) estimation of the motion between consecutive or temporally neighboring frames assuming a suitable motion model, (2) temporal motion smoothing assuming an appropriate motion model for the underlying stable video, and (3) re-targeting or warping of the frames of the unstable video so as to generate a stabilized video. There exists a large body of literature on video stabilization, with differences in the way these three steps are executed. Several of these techniques are summarized below.&#10;&#10;Related work (Classical Approaches): Many traditional techniques assume that the motion between consecutive frames can be approximated using 2D affine transformations or homographies \cite{Matsushita2006,Grundmann2011}, and seek to smooth a sequence of such parameters to render a stabilized video. For computing the parameterized motion, many of these techniques make use of robust point tracking methods \cite{Matsushita2006,Grundmann2011,Liu2013}. However, 2D motion models cannot accurately account for the motion between consecutive video frames for scenes with significant depth variation or significant camera motion. Some methods such as \cite{Liu2013} approximate the motion between consecutive frames by means of patch-wise 2D models or homographies. The method in \cite{Liu2014} performs three tasks in an iterated fashion: determining the smooth global background motion between consecutive frames by detecting moving objects, inpainting the flow in those regions, and smoothing per-pixel optical flow vectors across time. There also exist methods which make use of epipolar geometry \cite{Goldstein2012} or various geometrically motivated subspace constraints \cite{Liu2011}. The latter technique requires fairly long feature tracks which may not be available in many real-world videos. Finally, many techniques which use 3D information have also been proposed, for example methods that use structure from motion \cite{Liu2009}, a depth camera \cite{Sun2012} or a light field camera \cite{Smith2009}.&#10;&#10;Related work (Deep Learning Approaches): Deep neural network (DNN) based approaches for video stabilization have become very popular in recent years. The work in \cite{Yu2019} represents the warp fields using the weights of an unsupervised DNN, which minimizes the sum of two terms: a regularizer that encourages the warp fields to be piece-wise linear, and a fidelity term which minimizes the distance between corresponding pixels in consecutive frames in the stabilized video. This approach, though elegant, must evolve the network weights afresh for every video, and has very high computational cost. The work in \cite{Yu2020} trains a DNN offline on a large video dataset with synthetic unstable motion. In an unsupervised fashion, the weights of the DNN are evolved so as to generate warp fields that (1) have dominant low-frequency content in the Fourier domain, and (2) yield minimal distance between corresponding pixels in consecutive frames of the stabilized video. The work uses frame-to-frame optical flow as initial input and requires a number of pre-processing steps to: (1) identify regions with moving objects from the optical flow fields using a variety of segmentation masks for typical foreground objects obtained from \cite{Zhou2018_ACM}, (2) identify regions of inaccurate optical flow, and (3) inpaint all such regions using the PCA-based approach from \cite{Wulff2015}. The work in \cite{Choi2020} trains two DNNs for performing video stabilization via frame interpolation to smooth the motion between consecutive frames. The $(i-1)^{\textrm{th}}$ and $(i+1)^{\textrm{th}}$ frames are linearly warped mid-way toward each other using the bidirectional optical flow between them. The resulting warped frames are passed through a U-Net \cite{Ronneberger2015} to generate the $i^{\textrm{th}}$ intermediate `stabilized' frame. This interpolation process is carried out iteratively which may accumulate blur. To prevent this, the intermediate stabilized frames are also passed through a Resnet \cite{He2016}. The motion smoothing in this approach is always linear without any adaptation of the smoothing parameters to the motion at different time instants or at different depths. Similar in spirit to \cite{Choi2020}, work in \cite{Liu_2021_ICCV, Xu_2021_ICCV} perform full-frame video stabilization by bringing in border-based frame inpainting. However, the approach in \cite{Liu_2021_ICCV} is computationally expensive. The approach in \cite{Wang2019} trains a Siamese network to generate a warp grid for video stabilization using stable and unstable video pairs from the Deepstab dataset \cite{Xu2018}. Their approach is based purely on color without using any motion parameters and does not perform very well. The approach in \cite{Xu2018} uses spatial transformer networks along with adversarial networks for video stabilization, but suffers from problems due to inadequate training data. The work in \cite{Zhao2020}, which is called PWStableNet, uses a supervised training approach based on a cascade of encoder-decoder units to optimize a combination of criteria such as fidelity w.r.t. the underlying stable video, and various motion and feature-based characteristics. This approach has limitations in terms of training data scarcity and generalizability.&#10;&#10;Overview of Proposed Approach: &#10;A major contribution of our work is a novel method of estimating the global motion between frames of an unstable video, proposed in Sec.~\ref{sec:theory_global_motion}. Our method involves training a network GlobalFlowNet in a teacher-student fashion in such a way that it imposes a smooth and compact representation for the global motion and is designed to not be influenced by the motion in regions containing moving objects. Our method yields global motion representations that are more general than 2D affine or homography transformation and does not require any salient feature point tracking. Given a pre-trained GlobalFlowNet, we achieve video stabilization using a two-stage process comprising a novel global affine parameter smoothing step (Sec.~\ref{subsec:globalStabilization}) and a novel residual level smoothing step (Sec.~\ref{subsec:smooth_dct}) involving low frequency discrete cosine transform (DCT) coefficients of the residual flow. Both these steps smooth the parameters in the temporal direction. The first step acts as a very useful initial condition, whereas the second step is necessary to significantly improve stabilization performance. This is because it works with a global motion model that despite being very compact, is much more general than just 2D affine or homography. Our overall approach for video stabilization is simple, computationally efficient and interpretable. In extensive experiments (see Sec.~\ref{sec:experiments}), it outperforms state of the art techniques in terms of stability measures. We also propose a new video stabilization measure which uses the low frequency representation from Sec.~\ref{subsec:low_pass_module} to quantify the temporal smoothness of the global motion between successive pairs of frames. Our measure uses a more general motion model than existing measures which largely use affine transformations. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Optical Flow Estimation, Computer Science, Video Stabilization, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1701.03572" label="1701.03572">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Optical flow-based Video Stabilization for Unmanned Aerial&#10;  Vehicles" />
-          <attvalue for="1" value="  This paper describes the development of a novel algorithm to tackle the&#10;problem of real-time video stabilization for unmanned aerial vehicles (UAVs).&#10;There are two main components in the algorithm: (1) By designing a suitable&#10;model for the global motion of UAV, the proposed algorithm avoids the necessity&#10;of estimating the most general motion model, projective transformation, and&#10;considers simpler motion models, such as rigid transformation and similarity&#10;transformation. (2) To achieve a high processing speed, optical-flow based&#10;tracking is employed in lieu of conventional tracking and matching methods used&#10;by state-of-the-art algorithms. These two new ideas resulted in a real-time&#10;stabilization algorithm, developed over two phases. Stage I considers&#10;processing the whole sequence of frames in the video while achieving an average&#10;processing speed of 50fps on several publicly available benchmark videos. Next,&#10;Stage II undertakes the task of real-time video stabilization using a&#10;multi-threading implementation of the algorithm designed in Stage I.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.02641" label="1909.02641">
-        <attvalues>
-          <attvalue for="0" value="Deep Iterative Frame Interpolation for Full-frame Video Stabilization" />
-          <attvalue for="1" value="  Video stabilization is a fundamental and important technique for higher&#10;quality videos. Prior works have extensively explored video stabilization, but&#10;most of them involve cropping of the frame boundaries and introduce moderate&#10;levels of distortion. We present a novel deep approach to video stabilization&#10;which can generate video frames without cropping and low distortion. The&#10;proposed framework utilizes frame interpolation techniques to generate in&#10;between frames, leading to reduced inter-frame jitter. Once applied in an&#10;iterative fashion, the stabilization effect becomes stronger. A major advantage&#10;is that our framework is end-to-end trainable in an unsupervised manner. In&#10;addition, our method is able to run in near real-time (15 fps). To the best of&#10;our knowledge, this is the first work to propose an unsupervised deep approach&#10;to full-frame video stabilization. We show the advantages of our method through&#10;quantitative and qualitative evaluations comparing to the state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.06205" label="2102.06205">
-        <attvalues>
-          <attvalue for="0" value="Hybrid Neural Fusion for Full-frame Video Stabilization" />
-          <attvalue for="1" value="  Existing video stabilization methods often generate visible distortion or&#10;require aggressive cropping of frame boundaries, resulting in smaller field of&#10;views. In this work, we present a frame synthesis algorithm to achieve&#10;full-frame video stabilization. We first estimate dense warp fields from&#10;neighboring frames and then synthesize the stabilized frame by fusing the&#10;warped contents. Our core technical novelty lies in the learning-based&#10;hybrid-space fusion that alleviates artifacts caused by optical flow inaccuracy&#10;and fast-moving objects. We validate the effectiveness of our method on the&#10;NUS, selfie, and DeepStab video datasets. Extensive experiment results&#10;demonstrate the merits of our approach over prior video stabilization methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.02944" label="1906.02944">
-        <attvalues>
-          <attvalue for="0" value="Learning Adaptive Classifiers Synthesis for Generalized Few-Shot&#10;  Learning" />
-          <attvalue for="1" value="  Object recognition in the real-world requires handling long-tailed or even&#10;open-ended data. An ideal visual system needs to recognize the populated head&#10;visual concepts reliably and meanwhile efficiently learn about emerging new&#10;tail categories with a few training instances. Class-balanced many-shot&#10;learning and few-shot learning tackle one side of this problem, by either&#10;learning strong classifiers for head or learning to learn few-shot classifiers&#10;for the tail. In this paper, we investigate the problem of generalized few-shot&#10;learning (GFSL) -- a model during the deployment is required to learn about&#10;tail categories with few shots and simultaneously classify the head classes. We&#10;propose the ClAssifier SynThesis LEarning (CASTLE), a learning framework that&#10;learns how to synthesize calibrated few-shot classifiers in addition to the&#10;multi-class classifiers of head classes with a shared neural dictionary,&#10;shedding light upon the inductive GFSL. Furthermore, we propose an adaptive&#10;version of CASTLE (ACASTLE) that adapts the head classifiers conditioned on the&#10;incoming tail training examples, yielding a framework that allows effective&#10;backward knowledge transfer. As a consequence, ACASTLE can handle GFSL with&#10;classes from heterogeneous domains effectively. CASTLE and ACASTLE demonstrate&#10;superior performances than existing GFSL algorithms and strong baselines on&#10;MiniImageNet as well as TieredImageNet datasets. More interestingly, they&#10;outperform previous state-of-the-art methods when evaluated with standard&#10;few-shot learning criteria.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Visual recognition for objects in the ``long tail'' has been an important challenge to address~\cite{WangRH17Learning,Liu2019Large,Kang2020Decoupling,Zhou2020BBN}. We often have a very limited amount of data on those objects as they are infrequently observed and/or visual exemplars of them are hard to collect. As such, state-of-the-art methods (\eg., deep learning) can not be directly applied due to their notorious demand of a large number of annotated data~\cite{simonyan2014very,he2016deep,Krizhevsky2017ImageNet}.&#10;&#10;Few-shot learning~(FSL)~\cite{VinyalsBLKW16Matching,chen2019closer} is mindful of the limited data per tail concept (\ie., shots), which attempts to address this challenging problem by distinguishing between the data-rich head categories as seen classes and data-scarce tail categories as unseen classes. While it is difficult to build classifiers with data from unseen classes, FSL mimics the test scenarios by sampling few-shot tasks from seen class data, and extracts inductive biases for effective classifiers acquisition on unseen ones. Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL.&#10;&#10;This type of learning makes the classifier from few-shot learning for unseen classes difficult to be combined directly with the classifier from many-shot learning for seen classes, however, the demand to recognize all object categories simultaneously in object recognition is essential as well. &#10;&#10;In this paper, we study the problem of Generalized Few-Shot Learning~(GFSL), which focuses on the {joint} classification of both data-rich and data-poor categories. Figure~\ref{fig:diagram} illustrates the high-level idea of the GFSL, contrasting the standard FSL. &#10;In particular, our goal is for the model trained on the seen categories to be capable of incorporating the limited unseen class examples, and make predictions for test data in both the head and tail of the entire distribution of categories. &#10;&#10;One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}. &#10;One main advantage of such a joint learning objective over all classes is that it characterizes both seen and unseen classes simultaneously. In other words, training of one part (\eg., head) naturally takes the other part (\eg., tail) into consideration, and promotes the knowledge transfer between classes.&#10;However, such a transductive learning paradigm requires collecting the limited tail data in advance, which is violated in many real-world tasks.&#10;In contrast to it, our learning setup requires an {inductive} modeling of the tail, which is therefore more challenging as we assume no knowledge about the unseen tail categories is available during the model learning phase. &#10;&#10;There are two main challenges in the inductive GFSL problem, including how to construct the many-shot and few-shot classifiers in the GFSL scenario and how to calibrate their predictions.&#10;&#10;First, the head and tail classifiers for a GFSL model should encode different properties of all classes towards high discerning ability, and the classifiers for the many-shot part should be adapted based on the tail concepts accordingly. &#10;For example, if the unseen classes come from different domains, the same single seen classifier is difficult to handle their diverse properties and should not be left alone in this dynamic process.&#10;Furthermore, as observed in the generalized zero-shot learning scenario~\cite{Chao2016Generalized}, a classifier performs over-confident with its familiar concepts and fear to make predictions for those unseen ones, which leads to a confidence gap when predicting seen and unseen classes. The calibration issue appears in the generalized few-shot learning as well, \ie., seen and unseen classifiers have different confidence ranges. We empirically find that directly optimizing two objectives together could not resolve the problem completely.&#10;&#10;To this end, we propose ClAssifier SynThesis LEarning~(Castle), where the few-shot classifiers are synthesized based on a neural dictionary with common characteristics across classes. &#10;Such synthesized few-shot classifiers are then used together with the many-shot classifiers, and learned end-to-end.&#10;To this purpose, we create a learning scenario by sampling a set of data instances from seen categories and pretend that they come from unseen categories, and apply the synthesized classifiers (based on the above instances) as if they are many-shot classifiers to optimize multi-class classification together with the remaining many-shot seen classifiers.&#10;In other words, we construct few-shot classifiers to not only perform well on the few-shot classes but also to be competitive when used in conjunction with many-shot classifiers of populated classes. &#10;We argue that such highly contrastive learning can benefit the few-shot classification in two aspects: (1) it provides high discernibility for its synthesized classifiers. (2) it makes the synthesized classifier automatically calibrated with the many-shot classifiers.&#10;&#10;Taking steps further, we then propose the Adaptive ClAssifier SynThesis LEarning~(aCastle), with additional flexibility to adapt the many-shot classifiers based on few-shot training examples. As a result, it allows backward knowledge transfer~\cite{lopez2017gradient} --- new knowledge learned from novel few-shot training examples can benefit the existing many-shot classifiers. In aCastle, the neural dictionary is the concatenation of the shared and the task-specific neural bases, whose elements summarize the generality of all visual classes and the specialty of current few-shot categories. This improved neural dictionary facilitates the adaptation of the many-shot classifiers conditioned on the limited tail training examples. &#10;The adapted many-shot classifiers in aCastle are then used together with the (jointly) synthesized few-shot classifiers for GFSL classification.&#10;&#10;We first verify the effectiveness of the synthesized GFSL classifiers over multi-domain GFSL tasks, where the unseen classes would come from diverse domains. {aCastle} can best handle such task heterogeneity due to its ability to adapt the head classifiers.&#10;Next, we empirically validate our approach on two standard benchmark datasets --- {Mini}ImageNet~\cite{VinyalsBLKW16Matching} and {Tiered}ImageNet~\cite{Ren2018Meta}. &#10;The proposed approach retains competitive tail concept recognition performances while outperforming existing approaches on generalized few-shot learning with criteria from different aspects. By carefully selecting a prediction bias from the validation set, those miscalibrated FSL approaches or other baselines perform well in the GFSL scenario. The implicit confidence calibration in {Castle} and {aCastle} works as well as or even better than the post-calibration techniques.&#10;We note that {Castle} and {aCastle} are applicable for standard few-shot learning, which stays competitive with and sometimes even outperforms state-of-the-art methods when evaluated on two popular FSL benchmarks.&#10;&#10;Our contributions are summarized as what follows:&#10;\begin{itemize}[leftmargin=*]&#10;&#09;\item We propose a framework that synthesizes few-shot classifiers for GFSL with a shared neural dictionary, as well as its adaptive variant that modifies seen many-shot classifiers to allow the backward knowledge transfer.&#10;&#09;\item We extend an existing GFSL learning framework into an end-to-end counterpart that learns and contrasts the few-shot and the many-shot classifiers simultaneously, which is observed beneficial to the confidence calibration of these two types of classifiers.&#10;&#09;\item We empirically demonstrate that aCastle is effective in backward transferring knowledge when learning novel classes under the setting of multi-domain GFSL. Meanwhile, we perform a comprehensive evaluation of both existing and our approaches with criteria from various perspectives on multiple GFSL benchmarks.&#10;\end{itemize}&#10;&#10;In the rest sections of this paper, we first describe the problem formulation of GFSL in \S~\ref{sec:preliminary}, and then introduce our {Castle}/{aCastle} approach in \S~\ref{sec:method}. We conduct thorough experiments (see \S~\ref{sec:ext_setup} for the setups) to verify the the proposed {Castle} and {aCastle} across multiple benchmarks. We first conduct a pivot study on multi-domain GFSL benchmarks~(\S~\ref{sec:exp_cross}) to study the backward transfer capability of different methods. Then we evaluate both aCastle and Castle on popular GFSL~(\S~\ref{sec:exp_gfsl}), and FSL benchmarks~(\S~\ref{sec:exp_fsl}). Eventually, we review existing related works in \S~\ref{sec:related} and discuss the connections to our work.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Few-Shot Learning, Neural Network Frameworks, Object Recognition, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1904.05160" label="1904.05160">
-        <attvalues>
-          <attvalue for="0" value="Large-Scale Long-Tailed Recognition in an Open World" />
-          <attvalue for="1" value="  Real world data often have a long-tailed and open-ended distribution. A&#10;practical recognition system must classify among majority and minority classes,&#10;generalize from a few known instances, and acknowledge novelty upon a never&#10;seen instance. We define Open Long-Tailed Recognition (OLTR) as learning from&#10;such naturally distributed data and optimizing the classification accuracy over&#10;a balanced test set which include head, tail, and open classes. OLTR must&#10;handle imbalanced classification, few-shot learning, and open-set recognition&#10;in one integrated algorithm, whereas existing classification approaches focus&#10;only on one aspect and deliver poorly over the entire class spectrum. The key&#10;challenges are how to share visual knowledge between head and tail classes and&#10;how to reduce confusion between tail and open classes. We develop an integrated&#10;OLTR algorithm that maps an image to a feature space such that visual concepts&#10;can easily relate to each other based on a learned metric that respects the&#10;closed-world classification while acknowledging the novelty of the open world.&#10;Our so-called dynamic meta-embedding combines a direct image feature and an&#10;associated memory feature, with the feature norm indicating the familiarity to&#10;known classes. On three large-scale OLTR datasets we curate from object-centric&#10;ImageNet, scene-centric Places, and face-centric MS1M data, our method&#10;consistently outperforms the state-of-the-art. Our code, datasets, and models&#10;enable future OLTR research and are publicly available at&#10;https://liuziwei7.github.io/projects/LongTail.html.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.09217" label="1910.09217">
-        <attvalues>
-          <attvalue for="0" value="Decoupling Representation and Classifier for Long-Tailed Recognition" />
-          <attvalue for="1" value="  The long-tail distribution of the visual world poses great challenges for&#10;deep learning based classification models on how to handle the class imbalance&#10;problem. Existing solutions usually involve class-balancing strategies, e.g.,&#10;by loss re-weighting, data re-sampling, or transfer learning from head- to&#10;tail-classes, but most of them adhere to the scheme of jointly learning&#10;representations and classifiers. In this work, we decouple the learning&#10;procedure into representation learning and classification, and systematically&#10;explore how different balancing strategies affect them for long-tailed&#10;recognition. The findings are surprising: (1) data imbalance might not be an&#10;issue in learning high-quality representations; (2) with representations&#10;learned with the simplest instance-balanced (natural) sampling, it is also&#10;possible to achieve strong long-tailed recognition ability by adjusting only&#10;the classifier. We conduct extensive experiments and set new state-of-the-art&#10;performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and&#10;iNaturalist, showing that it is possible to outperform carefully designed&#10;losses, sampling strategies, even complex modules with memory, by using a&#10;straightforward approach that decouples representation and classification. Our&#10;code is available at https://github.com/facebookresearch/classifier-balancing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.02413" label="1912.02413">
-        <attvalues>
-          <attvalue for="0" value="BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed&#10;  Visual Recognition" />
-          <attvalue for="1" value="  Our work focuses on tackling the challenging but natural visual recognition&#10;task of long-tailed data distribution (i.e., a few classes occupy most of the&#10;data, while most classes have rarely few samples). In the literature, class&#10;re-balancing strategies (e.g., re-weighting and re-sampling) are the prominent&#10;and effective methods proposed to alleviate the extreme imbalance for dealing&#10;with long-tailed problems. In this paper, we firstly discover that these&#10;re-balancing methods achieving satisfactory recognition accuracy owe to that&#10;they could significantly promote the classifier learning of deep networks.&#10;However, at the same time, they will unexpectedly damage the representative&#10;ability of the learned deep features to some extent. Therefore, we propose a&#10;unified Bilateral-Branch Network (BBN) to take care of both representation&#10;learning and classifier learning simultaneously, where each branch does perform&#10;its own duty separately. In particular, our BBN model is further equipped with&#10;a novel cumulative learning strategy, which is designed to first learn the&#10;universal patterns and then pay attention to the tail data gradually. Extensive&#10;experiments on four benchmark datasets, including the large-scale iNaturalist&#10;ones, justify that the proposed BBN can significantly outperform&#10;state-of-the-art methods. Furthermore, validation experiments can demonstrate&#10;both our preliminary discovery and effectiveness of tailored designs in BBN for&#10;long-tailed problems. Our method won the first place in the iNaturalist 2019&#10;large scale species classification competition, and our code is open-source and&#10;available at https://github.com/Megvii-Nanjing/BBN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.04232" label="1904.04232">
-        <attvalues>
-          <attvalue for="0" value="A Closer Look at Few-shot Classification" />
-          <attvalue for="1" value="  Few-shot classification aims to learn a classifier to recognize unseen&#10;classes during training with limited labeled examples. While significant&#10;progress has been made, the growing complexity of network designs,&#10;meta-learning algorithms, and differences in implementation details make a fair&#10;comparison difficult. In this paper, we present 1) a consistent comparative&#10;analysis of several representative few-shot classification algorithms, with&#10;results showing that deeper backbones significantly reduce the performance&#10;differences among methods on datasets with limited domain differences, 2) a&#10;modified baseline method that surprisingly achieves competitive performance&#10;when compared with the state-of-the-art on both the \miniI and the CUB&#10;datasets, and 3) a new experimental setting for evaluating the cross-domain&#10;generalization ability for few-shot classification algorithms. Our results&#10;reveal that reducing intra-class variation is an important factor when the&#10;feature backbone is shallow, but not as critical when using deeper backbones.&#10;In a realistic cross-domain evaluation setting, we show that a baseline method&#10;with a standard fine-tuning practice compares favorably against other&#10;state-of-the-art few-shot learning algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.05960" label="1807.05960">
-        <attvalues>
-          <attvalue for="0" value="Meta-Learning with Latent Embedding Optimization" />
-          <attvalue for="1" value="  Gradient-based meta-learning techniques are both widely applicable and&#10;proficient at solving challenging few-shot learning and fast adaptation&#10;problems. However, they have practical difficulties when operating on&#10;high-dimensional parameter spaces in extreme low-data regimes. We show that it&#10;is possible to bypass these limitations by learning a data-dependent latent&#10;generative representation of model parameters, and performing gradient-based&#10;meta-learning in this low-dimensional latent space. The resulting approach,&#10;latent embedding optimization (LEO), decouples the gradient-based adaptation&#10;procedure from the underlying high-dimensional space of model parameters. Our&#10;evaluation shows that LEO can achieve state-of-the-art performance on the&#10;competitive miniImageNet and tieredImageNet few-shot classification tasks.&#10;Further analysis indicates LEO is able to capture uncertainty in the data, and&#10;can perform adaptation more effectively by optimizing in latent space.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.02999" label="1803.02999">
-        <attvalues>
-          <attvalue for="0" value="On First-Order Meta-Learning Algorithms" />
-          <attvalue for="1" value="  This paper considers meta-learning problems, where there is a distribution of&#10;tasks, and we would like to obtain an agent that performs well (i.e., learns&#10;quickly) when presented with a previously unseen task sampled from this&#10;distribution. We analyze a family of algorithms for learning a parameter&#10;initialization that can be fine-tuned quickly on a new task, using only&#10;first-order derivatives for the meta-learning updates. This family includes and&#10;generalizes first-order MAML, an approximation to MAML obtained by ignoring&#10;second-order derivatives. It also includes Reptile, a new algorithm that we&#10;introduce here, which works by repeatedly sampling a task, training on it, and&#10;moving the initialization towards the trained weights on that task. We expand&#10;on the results from Finn et al. showing that first-order meta-learning&#10;algorithms perform well on some well-established benchmarks for few-shot&#10;classification, and we provide theoretical analysis aimed at understanding why&#10;these algorithms work.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.05401" label="1801.05401">
-        <attvalues>
-          <attvalue for="0" value="Low-Shot Learning from Imaginary Data" />
-          <attvalue for="1" value="  Humans can quickly learn new visual concepts, perhaps because they can easily&#10;visualize or imagine what novel objects look like from different views.&#10;Incorporating this ability to hallucinate novel instances of new concepts might&#10;help machine vision systems perform better low-shot learning, i.e., learning&#10;concepts from few examples. We present a novel approach to low-shot learning&#10;that uses this idea. Our approach builds on recent progress in meta-learning&#10;(&quot;learning to learn&quot;) by combining a meta-learner with a &quot;hallucinator&quot; that&#10;produces additional training examples, and optimizing both models jointly. Our&#10;hallucinator can be incorporated into a variety of meta-learners and provides&#10;significant gains: up to a 6 point boost in classification accuracy when only a&#10;single training example is available, yielding state-of-the-art performance on&#10;the challenging ImageNet low-shot classification benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03758" label="1904.03758">
-        <attvalues>
-          <attvalue for="0" value="Meta-Learning with Differentiable Convex Optimization" />
-          <attvalue for="1" value="  Many meta-learning approaches for few-shot learning rely on simple base&#10;learners such as nearest-neighbor classifiers. However, even in the few-shot&#10;regime, discriminatively trained linear predictors can offer better&#10;generalization. We propose to use these predictors as base learners to learn&#10;representations for few-shot learning and show they offer better tradeoffs&#10;between feature size and performance across a range of few-shot recognition&#10;benchmarks. Our objective is to learn feature embeddings that generalize well&#10;under a linear classification rule for novel categories. To efficiently solve&#10;the objective, we exploit two properties of linear classifiers: implicit&#10;differentiation of the optimality conditions of the convex problem and the dual&#10;formulation of the optimization problem. This allows us to use high-dimensional&#10;embeddings with improved generalization at a modest increase in computational&#10;overhead. Our approach, named MetaOptNet, achieves state-of-the-art performance&#10;on miniImageNet, tieredImageNet, CIFAR-FS, and FC100 few-shot learning&#10;benchmarks. Our code is available at https://github.com/kjunelee/MetaOptNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.02819" label="1606.02819">
-        <attvalues>
-          <attvalue for="0" value="Low-shot Visual Recognition by Shrinking and Hallucinating Features" />
-          <attvalue for="1" value="  Low-shot visual learning---the ability to recognize novel object categories&#10;from very few examples---is a hallmark of human visual intelligence. Existing&#10;machine learning approaches fail to generalize in the same way. To make&#10;progress on this foundational problem, we present a low-shot learning benchmark&#10;on complex images that mimics challenges faced by recognition systems in the&#10;wild. We then propose a) representation regularization techniques, and b)&#10;techniques to hallucinate additional training examples for data-starved&#10;classes. Together, our methods improve the effectiveness of convolutional&#10;networks in low-shot learning, improving the one-shot accuracy on novel classes&#10;by 2.3x on the challenging ImageNet dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.07413" label="1906.07413">
-        <attvalues>
-          <attvalue for="0" value="Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss" />
-          <attvalue for="1" value="  Deep learning algorithms can fare poorly when the training dataset suffers&#10;from heavy class-imbalance but the testing criterion requires good&#10;generalization on less frequent classes. We design two novel methods to improve&#10;performance in such scenarios. First, we propose a theoretically-principled&#10;label-distribution-aware margin (LDAM) loss motivated by minimizing a&#10;margin-based generalization bound. This loss replaces the standard&#10;cross-entropy objective during training and can be applied with prior&#10;strategies for training with class-imbalance such as re-weighting or&#10;re-sampling. Second, we propose a simple, yet effective, training schedule that&#10;defers re-weighting until after the initial stage, allowing the model to learn&#10;an initial representation while avoiding some of the complications associated&#10;with re-weighting or re-sampling. We test our methods on several benchmark&#10;vision tasks including the real-world imbalanced dataset iNaturalist 2018. Our&#10;experiments show that either of these methods alone can already improve over&#10;existing techniques and their combination achieves even better performance&#10;gains.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.08840" label="1706.08840">
-        <attvalues>
-          <attvalue for="0" value="Gradient Episodic Memory for Continual Learning" />
-          <attvalue for="1" value="  One major obstacle towards AI is the poor ability of models to solve new&#10;problems quicker, and without forgetting previously acquired knowledge. To&#10;better understand this issue, we study the problem of continual learning, where&#10;the model observes, once and one by one, examples concerning a sequence of&#10;tasks. First, we propose a set of metrics to evaluate models learning over a&#10;continuum of data. These metrics characterize models not only by their test&#10;accuracy, but also in terms of their ability to transfer knowledge across&#10;tasks. Second, we propose a model for continual learning, called Gradient&#10;Episodic Memory (GEM) that alleviates forgetting, while allowing beneficial&#10;transfer of knowledge to previous tasks. Our experiments on variants of the&#10;MNIST and CIFAR-100 datasets demonstrate the strong performance of GEM when&#10;compared to the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.00676" label="1803.00676">
-        <attvalues>
-          <attvalue for="0" value="Meta-Learning for Semi-Supervised Few-Shot Classification" />
-          <attvalue for="1" value="  In few-shot classification, we are interested in learning algorithms that&#10;train a classifier from only a handful of labeled examples. Recent progress in&#10;few-shot classification has featured meta-learning, in which a parameterized&#10;model for a learning algorithm is defined and trained on episodes representing&#10;different classification problems, each with a small labeled training set and&#10;its corresponding test set. In this work, we advance this few-shot&#10;classification paradigm towards a scenario where unlabeled examples are also&#10;available within each episode. We consider two situations: one where all&#10;unlabeled examples are assumed to belong to the same set of classes as the&#10;labeled examples of the episode, as well as the more challenging situation&#10;where examples from other distractor classes are also provided. To address this&#10;paradigm, we propose novel extensions of Prototypical Networks (Snell et al.,&#10;2017) that are augmented with the ability to use unlabeled examples when&#10;producing prototypes. These models are trained in an end-to-end way on&#10;episodes, to learn to leverage the unlabeled examples successfully. We evaluate&#10;these methods on versions of the Omniglot and miniImageNet benchmarks, adapted&#10;to this new framework augmented with unlabeled examples. We also propose a new&#10;split of ImageNet, consisting of a large set of classes, with a hierarchical&#10;structure. Our experiments confirm that our Prototypical Networks can learn to&#10;improve their predictions due to unlabeled examples, much like a&#10;semi-supervised algorithm would.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.02296" label="2405.02296">
-        <attvalues>
-          <attvalue for="0" value="M\&quot;obius Transform for Mitigating Perspective Distortions in&#10;  Representation Learning" />
-          <attvalue for="1" value="  Perspective distortion (PD) causes unprecedented changes in shape, size,&#10;orientation, angles, and other spatial relationships of visual concepts in&#10;images. Precisely estimating camera intrinsic and extrinsic parameters is a&#10;challenging task that prevents synthesizing perspective distortion.&#10;Non-availability of dedicated training data poses a critical barrier to&#10;developing robust computer vision methods. Additionally, distortion correction&#10;methods make other computer vision tasks a multi-step approach and lack&#10;performance. In this work, we propose mitigating perspective distortion (MPD)&#10;by employing a fine-grained parameter control on a specific family of M\&quot;obius&#10;transform to model real-world distortion without estimating camera intrinsic&#10;and extrinsic parameters and without the need for actual distorted data. Also,&#10;we present a dedicated perspectively distorted benchmark dataset, ImageNet-PD,&#10;to benchmark the robustness of deep learning models against this new dataset.&#10;The proposed method outperforms existing benchmarks, ImageNet-E and ImageNet-X.&#10;Additionally, it significantly improves performance on ImageNet-PD while&#10;consistently performing on standard data distribution. Notably, our method&#10;shows improved performance on three PD-affected real-world applications crowd&#10;counting, fisheye image recognition, and person re-identification and one&#10;PD-affected challenging CV task: object detection. The source code, dataset,&#10;and models are available on the project webpage at&#10;https://prakashchhipa.github.io/projects/mpd.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Perspective distortion (PD) in real-world imagery is omnipresent and poses challenges in developing computer vision applications. PD arises from various factors, including camera positioning, depth, intrinsic parameters such as focal length and lens distortion, and extrinsic parameters such as rotation and translation. These factors collectively influence the projection of 3D scenes onto 2D planes \cite{rahman2011efficient}, altering semantic interpretation and local geometry. Precisely estimating these parameters for perspective distortion correction is challenging, presenting a critical barrier to developing robust computer vision (CV) methods.&#10;&#10;Previous studies in distortion correction predominantly concentrated on the correction of PD, with limited emphasis on enhancing the robustness of CV applications. Consequently, these correction methods transform CV tasks, such as image recognition and scene understanding, into a two-stage process: firstly, rectifying the distortion, and secondly, engaging in task-specific learning. GeoNet\cite{li2019blind} employs CNN to predict distortion flow in images without prior distortion-type knowledge. PCL\cite{yu2021pcls} uses perspective crop layers in CNNs to correct perspective distortions for 3D pose estimation. A cascaded deep structure network\cite{tan2021practical} corrects wide-angle portrait distortions without calibrated camera parameters. Another study\cite{zhao2019learning} predicts per-pixel displacement for face portrait undistortion. PerspectiveNet\cite{jin2023perspective} and ParamNet\cite{jin2023perspective} predict perspective fields and derive camera calibration parameters, respectively. Methods for fisheye image distortion correction are also developed\cite{yang2023innovating,yin2018fisheyerecnet}. &#10; Zolly\cite{wang2023zolly} corrects perspective distortion by adjusting focal lengths in human mesh reconstruction tasks.&#10;&#10;Since the correction methods are fundamentally not focused on robust representation learning for CV tasks, they essentially follow an inefficient two-step process. &#10;One possible approach is to focus on developing task-specific methods where PD is noticeable, such as crowd counting\cite{yang2020weakly, idrees2018composition}, object re-identification\cite{ayala2019vehicle}, autonomous vehicle\cite{wang2020pillar}, and recognizing texts in scene\cite{phan2013recognizing}. &#10;Further, \cite{cho2021camera} meta-learning based approach to adapt 3D human pose estimation to varying degrees of camera distortion. SPEC\cite{kocabas2021spec} tackled PD by estimating the camera parameters directly from in-the-wild images to improve the accuracy of 3D human pose predictions. &#10;These task-specific approaches fall short in providing generalizable and robust representation learning for PD mitigation, a crucial step for advancing CV research towards distortion robustness.&#10;&#10;The core reason behind the lack of generalizable and robust representation learning CV methods to mitigate PD is the absence of a real-world PD-affected training dataset. Both capturing real-world PD data and synthesizing PD-affected image data using parameter estimation are far from reality. Although existing benchmarks (ImageNet-X \cite{idrissi2022imagenet}, ImageNet-E \cite{li2023imagenet}) are partially affected by PD, a dedicated benchmark is needed. Therefore, there is a need to artificially synthesize PD in existing data without estimating camera intrinsic and extrinsic parameters. Since PD is non-linear in nature, the focus should be shifted from affine to potential non-linear transformation.&#10;&#10;One potential non-linear family of transformation is Möbius transform \cite{arnold2008mobius,olsen2010geometry} in complex theory. Due to its conformal nature (preserving angles during transformation), \cite{zhou2021data} utilized the Möbius transform for eight specific transformations, including various twists and spreads, as a data augmentation strategy to enhance model performance on standard data distributions. In other works, Möbius convolution\cite{mitchel2022mobius}, the authors design a Möbius-equivalent convolution operator in spherical CNN for shape classification and image segmentation tasks. OmniZoomer\cite{cao2023omnizoomer} integrates Möbius transformations into a neural network, enabling movement and variable zoom on omnidirectional images while addressing the challenges of blur and aliasing associated with such transformations. MöbiusGCN\cite{azizi20223d} addresses the limitation of existing graph convolution network {GCNs} in explicitly encoding transformations between joints in 3D human pose estimation. None of these works focus on perspective distortion.&#10;&#10;Due to the non-linear characteristics of Möbius transforming geometry in complex space while ensuring conformality, it is a potential approach to imitate perspective distortion when adapted to image data. Therefore, in the proposed mitigating perspective distortion (MPD) method, we mathematically model PD in the images by employing fine-grained parameter control on Möbius. It is governed by four parameters $a,b,c$ and $d$ (Eq. \ref{eq_mobius}). MPD focuses on a specific family of transforms corresponding to the parameter $c$ to synthesize PD artificially. The core contribution of our method, MPD, is controlled parametrization for orientation (Fig. \ref{fig:shematic}) and varying intensity (Fig. \ref{fig:intensity}), which is capable of representing real-world perspective distortion. &#10;&#10;MPD transforms input image coordinates into complex vectors and performs a PD-specific family of Möbius transforms. Transformed complex vectors are then remapped to real-valued pixels, later discretized to obtain images mimicking perspective distortion while maintaining the integrity of visual concepts. Optionally, padding is applied to replace the black background with boundary pixel values to achieve another variant called integrated padding background variant (refer Fig. \ref{fig:padding} in suppl. material). Notably, MPD achieves this without estimating intrinsic and extrinsic parameters, thus bypassing the need for actual distorted data. &#10;&#10;Incorporating PD in computer vision is beneficial for robustness since it can be used as an augmentation, resulting in improved performance on different computer vision tasks. To address the non-availability of a suitable benchmark, we have developed a dedicated perspectively distorted benchmark dataset, {ImageNet-PD}, derived from the ImageNet\cite{deng2009imagenet} validation set to evaluate the robustness of deep learning models against perspective distortion. We showcase a lack of robustness of standard deep learning models against perspective distortion by benchmarking them on ImageNet-PD.&#10;&#10;We employ ImageNet-PD in our proposed MPD method and extensively investigate it across supervised and self-supervised learning methods (SimCLR \cite{chen2020simple}, DINO \cite{caron2021emerging}). MPD demonstrates improved performance on existing benchmarks, ImagNet-E \cite{li2023imagenet}, and ImageNet-X \cite{idrissi2022imagenet}. MPD improves $10\%$ on ImageNet-PD while consistently maintaining the performance on the original ImageNet validation set. MPD shows performance improvements when adapted to multiple real-world CV applications. Specifically, MPD advances crowd counting with a novel method, MPD-AutoCrowd, achieving the highest performance (to the best of our knowledge) with a mean absolute error of 50.81 on ShanghaiTech-Part-A\cite{7780439} and 96.80 on UCFCC50\cite{idrees2013multi}. It demonstrates effective knowledge transfer in fish-eye image recognition, improving performance by $3\%$ on the VOC-360\cite{fu2019datasets} dataset. MPD combined with a transformer-based self-supervised method, Clip-ReIdent\cite{habel2022clip}, excels in person re-identification, achieving mean average precision of 97.02 and 98.30 with and without re-ranking, respectively, on the DeepSportradar dataset\cite{van2022deepsportradar}. Also, MPD adapted for object detection with FasterRCNN \cite{ren2015faster} and FCOS \cite{9010746} methods gain up to 3\% performance improvement on COCO \cite{lin2014microsoft} dataset. The main contributions of this paper are as follows: &#10;\begin{enumerate}&#10; \item We mathematically model perspective distortion within deep learning by developing MPD, a method that successfully mimics perspective distortion without estimating the intrinsic and extrinsic camera parameters.&#10; \item We develop a dedicated perspectively distorted dataset (ImageNet-PD) derived from the ImageNet\cite{deng2009imagenet} validation set to benchmark the robustness of computer vision models against PD to address the non-availability of a suitable benchmark. &#10; \item We demonstrate MPD's generalizability, transferability, and adaptability across different learning paradigms, architectures, and real-world applications. &#10; \item Additionally, we develop the MPD-AutoCrowd method in crowd counting to augment crowd and corresponding ground truth in scene images. &#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Deep Learning Benchmarking, Mathematics, Physics, Image Processing, Deep Learning, Computer Science, Perspective Distortion Correction, Computer Vision Methods" />
-        </attvalues>
-      </node>
-      <node id="2011.13607" label="2011.13607">
-        <attvalues>
-          <attvalue for="0" value="PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective&#10;  Crop Layers" />
-          <attvalue for="1" value="  Local processing is an essential feature of CNNs and other neural network&#10;architectures - it is one of the reasons why they work so well on images where&#10;relevant information is, to a large extent, local. However, perspective effects&#10;stemming from the projection in a conventional camera vary for different global&#10;positions in the image. We introduce Perspective Crop Layers (PCLs) - a form of&#10;perspective crop of the region of interest based on the camera geometry - and&#10;show that accounting for the perspective consistently improves the accuracy of&#10;state-of-the-art 3D pose reconstruction methods. PCLs are modular neural&#10;network layers, which, when inserted into existing CNN and MLP architectures,&#10;deterministically remove the location-dependent perspective effects while&#10;leaving end-to-end training and the number of parameters of the underlying&#10;neural network unchanged. We demonstrate that PCL leads to improved 3D human&#10;pose reconstruction accuracy for CNN architectures that use cropping&#10;operations, such as spatial transformer networks (STN), and, somewhat&#10;surprisingly, MLPs used for 2D-to-3D keypoint lifting. Our conclusion is that&#10;it is important to utilize camera calibration information when available, for&#10;classical and deep-learning-based computer vision alike. PCL offers an easy way&#10;to improve the accuracy of existing 3D reconstruction networks by making them&#10;geometry aware. Our code is publicly available at&#10;github.com/yu-frank/PerspectiveCropLayers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.12464" label="2104.12464">
-        <attvalues>
-          <attvalue for="0" value="Practical Wide-Angle Portraits Correction with Deep Structured Models" />
-          <attvalue for="1" value="  Wide-angle portraits often enjoy expanded views. However, they contain&#10;perspective distortions, especially noticeable when capturing group portrait&#10;photos, where the background is skewed and faces are stretched. This paper&#10;introduces the first deep learning based approach to remove such artifacts from&#10;freely-shot photos. Specifically, given a wide-angle portrait as input, we&#10;build a cascaded network consisting of a LineNet, a ShapeNet, and a transition&#10;module (TM), which corrects perspective distortions on the background, adapts&#10;to the stereographic projection on facial regions, and achieves smooth&#10;transitions between these two projections, accordingly. To train our network,&#10;we build the first perspective portrait dataset with a large diversity in&#10;identities, scenes and camera modules. For the quantitative evaluation, we&#10;introduce two novel metrics, line consistency and face congruence. Compared to&#10;the previous state-of-the-art approach, our method does not require camera&#10;distortion parameters. We demonstrate that our approach significantly&#10;outperforms the previous state-of-the-art approach both qualitatively and&#10;quantitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.07515" label="1905.07515">
-        <attvalues>
-          <attvalue for="0" value="Learning Perspective Undistortion of Portraits" />
-          <attvalue for="1" value="  Near-range portrait photographs often contain perspective distortion&#10;artifacts that bias human perception and challenge both facial recognition and&#10;reconstruction techniques. We present the first deep learning based approach to&#10;remove such artifacts from unconstrained portraits. In contrast to the previous&#10;state-of-the-art approach, our method handles even portraits with extreme&#10;perspective distortion, as we avoid the inaccurate and error-prone step of&#10;first fitting a 3D face model. Instead, we predict a distortion correction flow&#10;map that encodes a per-pixel displacement that removes distortion artifacts&#10;when applied to the input image. Our method also automatically infers missing&#10;facial features, i.e. occluded ears caused by strong perspective distortion,&#10;with coherent details. We demonstrate that our approach significantly&#10;outperforms the previous state-of-the-art both qualitatively and&#10;quantitatively, particularly for portraits with extreme perspective distortion&#10;or facial expressions. We further show that our technique benefits a number of&#10;fundamental tasks, significantly improving the accuracy of both face&#10;recognition and 3D reconstruction and enables a novel camera calibration&#10;technique from a single portrait. Moreover, we also build the first perspective&#10;portrait database with a large diversity in identities, expression and poses,&#10;which will benefit the related research in this area.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.03239" label="2212.03239">
-        <attvalues>
-          <attvalue for="0" value="Perspective Fields for Single Image Camera Calibration" />
-          <attvalue for="1" value="  Geometric camera calibration is often required for applications that&#10;understand the perspective of the image. We propose perspective fields as a&#10;representation that models the local perspective properties of an image.&#10;Perspective Fields contain per-pixel information about the camera view,&#10;parameterized as an up vector and a latitude value. This representation has a&#10;number of advantages as it makes minimal assumptions about the camera model and&#10;is invariant or equivariant to common image editing operations like cropping,&#10;warping, and rotation. It is also more interpretable and aligned with human&#10;perception. We train a neural network to predict Perspective Fields and the&#10;predicted Perspective Fields can be converted to calibration parameters easily.&#10;We demonstrate the robustness of our approach under various scenarios compared&#10;with camera calibration-based methods and show example applications in image&#10;compositing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.04784" label="1804.04784">
-        <attvalues>
-          <attvalue for="0" value="FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye&#10;  Image Rectification" />
-          <attvalue for="1" value="  Images captured by fisheye lenses violate the pinhole camera assumption and&#10;suffer from distortions. Rectification of fisheye images is therefore a crucial&#10;preprocessing step for many computer vision applications. In this paper, we&#10;propose an end-to-end multi-context collaborative deep network for removing&#10;distortions from single fisheye images. In contrast to conventional approaches,&#10;which focus on extracting hand-crafted features from input images, our method&#10;learns high-level semantics and low-level appearance features simultaneously to&#10;estimate the distortion parameters. To facilitate training, we construct a&#10;synthesized dataset that covers various scenes and distortion parameter&#10;settings. Experiments on both synthesized and real-world datasets show that the&#10;proposed model significantly outperforms current state of the art methods. Our&#10;code and synthesized dataset will be made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.13796" label="2303.13796">
-        <attvalues>
-          <attvalue for="0" value="Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh&#10;  Reconstruction" />
-          <attvalue for="1" value="  As it is hard to calibrate single-view RGB images in the wild, existing 3D&#10;human mesh reconstruction (3DHMR) methods either use a constant large focal&#10;length or estimate one based on the background environment context, which can&#10;not tackle the problem of the torso, limb, hand or face distortion caused by&#10;perspective camera projection when the camera is close to the human body. The&#10;naive focal length assumptions can harm this task with the incorrectly&#10;formulated projection matrices. To solve this, we propose Zolly, the first&#10;3DHMR method focusing on perspective-distorted images. Our approach begins with&#10;analysing the reason for perspective distortion, which we find is mainly caused&#10;by the relative location of the human body to the camera center. We propose a&#10;new camera model and a novel 2D representation, termed distortion image, which&#10;describes the 2D dense distortion scale of the human body. We then estimate the&#10;distance from distortion scale features rather than environment context&#10;features. Afterwards, we integrate the distortion feature with image features&#10;to reconstruct the body mesh. To formulate the correct projection matrix and&#10;locate the human body position, we simultaneously use perspective and&#10;weak-perspective projection loss. Since existing datasets could not handle this&#10;task, we propose the first synthetic dataset PDHuman and extend two real-world&#10;datasets tailored for this task, all containing perspective-distorted human&#10;images. Extensive experiments show that Zolly outperforms existing&#10;state-of-the-art methods on both perspective-distorted datasets and the&#10;standard benchmark (3DPW).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.10323" label="2007.10323">
-        <attvalues>
-          <attvalue for="0" value="Pillar-based Object Detection for Autonomous Driving" />
-          <attvalue for="1" value="  We present a simple and flexible object detection framework optimized for&#10;autonomous driving. Building on the observation that point clouds in this&#10;application are extremely sparse, we propose a practical pillar-based approach&#10;to fix the imbalance issue caused by anchors. In particular, our algorithm&#10;incorporates a cylindrical projection into multi-view feature learning,&#10;predicts bounding box parameters per pillar rather than per point or per&#10;anchor, and includes an aligned pillar-to-point projection module to improve&#10;the final prediction. Our anchor-free approach avoids hyperparameter search&#10;associated with past methods, simplifying 3D object detection while&#10;significantly improving upon state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.15056" label="2111.15056">
-        <attvalues>
-          <attvalue for="0" value="Camera Distortion-aware 3D Human Pose Estimation in Video with&#10;  Optimization-based Meta-Learning" />
-          <attvalue for="1" value="  Existing 3D human pose estimation algorithms trained on distortion-free&#10;datasets suffer performance drop when applied to new scenarios with a specific&#10;camera distortion. In this paper, we propose a simple yet effective model for&#10;3D human pose estimation in video that can quickly adapt to any distortion&#10;environment by utilizing MAML, a representative optimization-based&#10;meta-learning algorithm. We consider a sequence of 2D keypoints in a particular&#10;distortion as a single task of MAML. However, due to the absence of a&#10;large-scale dataset in a distorted environment, we propose an efficient method&#10;to generate synthetic distorted data from undistorted 2D keypoints. For the&#10;evaluation, we assume two practical testing situations depending on whether a&#10;motion capture sensor is available or not. In particular, we propose Inference&#10;Stage Optimization using bone-length symmetry and consistency. Extensive&#10;evaluation shows that our proposed method successfully adapts to various&#10;degrees of distortion in the testing phase and outperforms the existing&#10;state-of-the-art approaches. The proposed method is useful in practice because&#10;it does not require camera calibration and additional computations in a testing&#10;set-up.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.01866" label="2211.01866">
-        <attvalues>
-          <attvalue for="0" value="ImageNet-X: Understanding Model Mistakes with Factor of Variation&#10;  Annotations" />
-          <attvalue for="1" value="  Deep learning vision systems are widely deployed across applications where&#10;reliability is critical. However, even today's best models can fail to&#10;recognize an object when its pose, lighting, or background varies. While&#10;existing benchmarks surface examples challenging for models, they do not&#10;explain why such mistakes arise. To address this need, we introduce ImageNet-X,&#10;a set of sixteen human annotations of factors such as pose, background, or&#10;lighting the entire ImageNet-1k validation set as well as a random subset of&#10;12k training images. Equipped with ImageNet-X, we investigate 2,200 current&#10;recognition models and study the types of mistakes as a function of model's (1)&#10;architecture, e.g. transformer vs. convolutional, (2) learning paradigm, e.g.&#10;supervised vs. self-supervised, and (3) training procedures, e.g., data&#10;augmentation. Regardless of these choices, we find models have consistent&#10;failure modes across ImageNet-X categories. We also find that while data&#10;augmentation can improve robustness to certain factors, they induce spill-over&#10;effects to other factors. For example, strong random cropping hurts robustness&#10;on smaller objects. Together, these insights suggest to advance the robustness&#10;of modern vision models, future research should focus on collecting additional&#10;data and understanding data augmentation schemes. Along with these insights, we&#10;release a toolkit based on ImageNet-X to spur further study into the mistakes&#10;image recognition systems make.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.02917" label="2002.02917">
-        <attvalues>
-          <attvalue for="0" value="Data augmentation with Mobius transformations" />
-          <attvalue for="1" value="  Data augmentation has led to substantial improvements in the performance and&#10;generalization of deep models, and remain a highly adaptable method to evolving&#10;model architectures and varying amounts of data---in particular, extremely&#10;scarce amounts of available training data. In this paper, we present a novel&#10;method of applying Mobius transformations to augment input images during&#10;training. Mobius transformations are bijective conformal maps that generalize&#10;image translation to operate over complex inversion in pixel space. As a&#10;result, Mobius transformations can operate on the sample level and preserve&#10;data labels. We show that the inclusion of Mobius transformations during&#10;training enables improved generalization over prior sample-level data&#10;augmentation techniques such as cutout and standard crop-and-flip&#10;transformations, most notably in low data regimes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.12212" label="2201.12212">
-        <attvalues>
-          <attvalue for="0" value="M\&quot;obius Convolutions for Spherical CNNs" />
-          <attvalue for="1" value="  M\&quot;obius transformations play an important role in both geometry and&#10;spherical image processing - they are the group of conformal automorphisms of&#10;2D surfaces and the spherical equivalent of homographies. Here we present a&#10;novel, M\&quot;obius-equivariant spherical convolution operator which we call&#10;M\&quot;obius convolution, and with it, develop the foundations for&#10;M\&quot;obius-equivariant spherical CNNs. Our approach is based on a simple&#10;observation: to achieve equivariance, we only need to consider the&#10;lower-dimensional subgroup which transforms the positions of points as seen in&#10;the frames of their neighbors. To efficiently compute M\&quot;obius convolutions at&#10;scale we derive an approximation of the action of the transformations on&#10;spherical filters, allowing us to compute our convolutions in the spectral&#10;domain with the fast Spherical Harmonic Transform. The resulting framework is&#10;both flexible and descriptive, and we demonstrate its utility by achieving&#10;promising results in both shape classification and image segmentation tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.08114" label="2308.08114">
-        <attvalues>
-          <attvalue for="0" value="OmniZoomer: Learning to Move and Zoom in on Sphere at High-Resolution" />
-          <attvalue for="1" value="  Omnidirectional images (ODIs) have become increasingly popular, as their&#10;large field-of-view (FoV) can offer viewers the chance to freely choose the&#10;view directions in immersive environments such as virtual reality. The M\&quot;obius&#10;transformation is typically employed to further provide the opportunity for&#10;movement and zoom on ODIs, but applying it to the image level often results in&#10;blurry effect and aliasing problem. In this paper, we propose a novel deep&#10;learning-based approach, called \textbf{OmniZoomer}, to incorporate the&#10;M\&quot;obius transformation into the network for movement and zoom on ODIs. By&#10;learning various transformed feature maps under different conditions, the&#10;network is enhanced to handle the increasing edge curvatures, which alleviates&#10;the blurry effect. Moreover, to address the aliasing problem, we propose two&#10;key components. Firstly, to compensate for the lack of pixels for describing&#10;curves, we enhance the feature maps in the high-resolution (HR) space and&#10;calculate the transformed index map with a spatial index generation module.&#10;Secondly, considering that ODIs are inherently represented in the spherical&#10;space, we propose a spherical resampling module that combines the index map and&#10;HR feature maps to transform the feature maps for better spherical correlation.&#10;The transformed feature maps are decoded to output a zoomed ODI. Experiments&#10;show that our method can produce HR and high-quality ODIs with the flexibility&#10;to move and zoom in to the object of interest. Project page is available at&#10;http://vlislab22.github.io/OmniZoomer/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.10554" label="2203.10554">
-        <attvalues>
-          <attvalue for="0" value="3D Human Pose Estimation Using M\&quot;obius Graph Convolutional Networks" />
-          <attvalue for="1" value="  3D human pose estimation is fundamental to understanding human behavior.&#10;Recently, promising results have been achieved by graph convolutional networks&#10;(GCNs), which achieve state-of-the-art performance and provide rather&#10;light-weight architectures. However, a major limitation of GCNs is their&#10;inability to encode all the transformations between joints explicitly. To&#10;address this issue, we propose a novel spectral GCN using the M\&quot;obius&#10;transformation (M\&quot;obiusGCN). In particular, this allows us to directly and&#10;explicitly encode the transformation between joints, resulting in a&#10;significantly more compact representation. Compared to even the lightest&#10;architectures so far, our novel approach requires 90-98% fewer parameters, i.e.&#10;our lightest M\&quot;obiusGCN uses only 0.042M trainable parameters. Besides the&#10;drastic parameter reduction, explicitly encoding the transformation of joints&#10;also enables us to achieve state-of-the-art results. We evaluate our approach&#10;on the two challenging pose estimation benchmarks, Human3.6M and MPI-INF-3DHP,&#10;demonstrating both state-of-the-art results and the generalization capabilities&#10;of M\&quot;obiusGCN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11942" label="1906.11942">
-        <attvalues>
-          <attvalue for="0" value="Datasets for Face and Object Detection in Fisheye Images" />
-          <attvalue for="1" value="  We present two new fisheye image datasets for training face and object&#10;detection models: VOC-360 and Wider-360. The fisheye images are created by&#10;post-processing regular images collected from two well-known datasets, VOC2012&#10;and Wider Face, using a model for mapping regular to fisheye images implemented&#10;in Matlab. VOC-360 contains 39,575 fisheye images for object detection,&#10;segmentation, and classification. Wider-360 contains 63,897 fisheye images for&#10;face detection. These datasets will be useful for developing face and object&#10;detectors as well as segmentation modules for fisheye images while the efforts&#10;to collect and manually annotate true fisheye images are underway.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.11855" label="2303.11855">
-        <attvalues>
-          <attvalue for="0" value="CLIP-ReIdent: Contrastive Training for Player Re-Identification" />
-          <attvalue for="1" value="  Sports analytics benefits from recent advances in machine learning providing&#10;a competitive advantage for teams or individuals. One important task in this&#10;context is the performance measurement of individual players to provide reports&#10;and log files for subsequent analysis. During sport events like basketball,&#10;this involves the re-identification of players during a match either from&#10;multiple camera viewpoints or from a single camera viewpoint at different&#10;times. In this work, we investigate whether it is possible to transfer the&#10;out-standing zero-shot performance of pre-trained CLIP models to the domain of&#10;player re-identification. For this purpose we reformulate the contrastive&#10;language-to-image pre-training approach from CLIP to a contrastive&#10;image-to-image training approach using the InfoNCE loss as training objective.&#10;Unlike previous work, our approach is entirely class-agnostic and benefits from&#10;large-scale pre-training. With a fine-tuned CLIP ViT-L/14 model we achieve&#10;98.44 % mAP on the MMSports 2022 Player Re-Identification challenge.&#10;Furthermore we show that the CLIP Vision Transformers have already strong OCR&#10;capabilities to identify useful player features like shirt numbers in a&#10;zero-shot manner without any fine-tuning on the dataset. By applying the&#10;Score-CAM algorithm we visualise the most important image regions that our&#10;fine-tuned model identifies when calculating the similarity score between two&#10;images of a player.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.08190" label="2208.08190">
-        <attvalues>
-          <attvalue for="0" value="DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with&#10;  High Quality Annotations" />
-          <attvalue for="1" value="  With the recent development of Deep Learning applied to Computer Vision,&#10;sport video understanding has gained a lot of attention, providing much richer&#10;information for both sport consumers and leagues. This paper introduces&#10;DeepSportradar-v1, a suite of computer vision tasks, datasets and benchmarks&#10;for automated sport understanding. The main purpose of this framework is to&#10;close the gap between academic research and real world settings. To this end,&#10;the datasets provide high-resolution raw images, camera parameters and high&#10;quality annotations. DeepSportradar currently supports four challenging tasks&#10;related to basketball: ball 3D localization, camera calibration, player&#10;instance segmentation and player re-identification. For each of the four tasks,&#10;a detailed description of the dataset, objective, performance metrics, and the&#10;proposed baseline method are provided. To encourage further research on&#10;advanced methods for sport understanding, a competition is organized as part of&#10;the MMSports workshop from the ACM Multimedia 2022 conference, where&#10;participants have to develop state-of-the-art methods to solve the above tasks.&#10;The four datasets, development kits and baselines are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.09801" label="2303.09801">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Graph Convolution Module for Salient Object Detection" />
-          <attvalue for="1" value="  Salient object detection (SOD) is a task that involves identifying and&#10;segmenting the most visually prominent object in an image. Existing solutions&#10;can accomplish this use a multi-scale feature fusion mechanism to detect the&#10;global context of an image. However, as there is no consideration of the&#10;structures in the image nor the relations between distant pixels, conventional&#10;methods cannot deal with complex scenes effectively. In this paper, we propose&#10;an adaptive graph convolution module (AGCM) to overcome these limitations.&#10;Prototype features are initially extracted from the input image using a&#10;learnable region generation layer that spatially groups features in the image.&#10;The prototype features are then refined by propagating information between them&#10;based on a graph architecture, where each feature is regarded as a node.&#10;Experimental results show that the proposed AGCM dramatically improves the SOD&#10;performance both quantitatively and quantitatively.&#10;" />
-          <attvalue for="2" value="&#10;Salient object detection (SOD) identifies and segments an image's most visually prominent object. The primary purpose of SOD is to automatically highlight the regions of an image that are most noticeable to a human observer. SOD is a vital pre-processing step for various computer vision tasks such as object detection and segmentation.&#10;&#10;Because the goal is to find the most salient object of a scene, extracting global context from the input image is a crucial factor in SOD. Existing methods ~\cite{FPN,wang2019salient,park2022saliency,qin2020u2} focus on leveraging multi-scale features to obtain comprehensive representations of an image. For instance, FPN~\cite{FPN} used a pyramid structure with bottom-up and top-down pathways, and lateral connections to extract multi-scale features. Similarly, Page-Net~\cite{wang2019salient} proposed an attentive pyramid module to increase the receptive field size and used multi-scale features by repeatedly downsampling the feature maps. Park et al.~\cite{park2022saliency} introduces a context fusion decoder network consisting of a context module to extract global information of an image and a feature fusion module to refine the features from encoder and decoder. U2-Net~\cite{qin2020u2} employs a residual U-block consisting of a nested U-structure to extract multi-scale properties and enable the network to extract the feature map without degrading the feature map resolution.&#10;&#10;Although leveraging multi-scale features may help capture global context, the previous methods cannot effectively deal with complex scenes where solid understanding of the structure is required. To overcome this limitation, we propose an adaptive graph convolution module (AGCM). Inspired by a previously published graph convolution network~\cite{gnn}, we designed AGCM to preserve permutation invariance even if the order of the nodes is altered. The proposed AGCM can be divided into two stages: 1) prototype feature generation; 2) prototype feature refinement. First, given the input features, spatial regions are first generated to construct prototype feature vectors, each containing global feature representations for respective regions. In other words, the input features are semantically and spatially grouped to provide explicit structural prior of an image. This process is completely differentiable, and therefore, it can be learned in an end-to-end manner. Second, AGCM adaptively creates a weight kernel from the generated prototypes and forms an adjacency matrix between them. In particular, when forming an adaptive graph, the constructed prototype feature vectors are refined using a graph architecture (more specifically, EdgeConv~\cite{EdgeConv}) to generate refined feature vectors, where each prototype feature vector is considered as a node. Through this process, contextual information of both near and distant pixels can be effectively leveraged. By plugging the proposed AGCM into the model, structures of a scene can be better appreciated, leading to a better SOD performance.&#10;&#10;We validate our proposed AGCM on public benchmark datasets, i.e., DUTS~\cite{wang2017learning}, ECSSD~\cite{yan2013hierarchical}, and DUT-OMRON~\cite{yang2013saliency}. Quantitative and qualitative comparison with other state-of-the-art methods demonstrates the effectiveness of our proposed AGCM. Especially, on complex scenes where multiple objects exist, using AGCM can dramatically boost the performance of a model. A solid ablation study on benchmark datasets confirms the efficacy of AGCM. &#10;&#10;Our contributions can be summarized as follows:&#10;\begin{itemize}&#10;\item We propose an AGCM to capture the local structural information and long-range dependencies between distant pixels, enabling the network to leverage global context in an image effectively. &#10;\item The proposed AGCM can be easily plugged into existing segmentation models without the need for architecture modifications. &#10;\item Our approach outperforms other state-of-the-art methods on public benchmark datasets quantitatively and qualitatively.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Graph Convolution Networks, Object Detection, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2110.06550" label="2110.06550">
-        <attvalues>
-          <attvalue for="0" value="Saliency Detection via Global Context Enhanced Feature Fusion and Edge&#10;  Weighted Loss" />
-          <attvalue for="1" value="  UNet-based methods have shown outstanding performance in salient object&#10;detection (SOD), but are problematic in two aspects. 1) Indiscriminately&#10;integrating the encoder feature, which contains spatial information for&#10;multiple objects, and the decoder feature, which contains global information of&#10;the salient object, is likely to convey unnecessary details of non-salient&#10;objects to the decoder, hindering saliency detection. 2) To deal with ambiguous&#10;object boundaries and generate accurate saliency maps, the model needs&#10;additional branches, such as edge reconstructions, which leads to increasing&#10;computational cost. To address the problems, we propose a context fusion&#10;decoder network (CFDN) and near edge weighted loss (NEWLoss) function. The CFDN&#10;creates an accurate saliency map by integrating global context information and&#10;thus suppressing the influence of the unnecessary spatial information. NEWLoss&#10;accelerates learning of obscure boundaries without additional modules by&#10;generating weight maps on object boundaries. Our method is evaluated on four&#10;benchmarks and achieves state-of-the-art performance. We prove the&#10;effectiveness of the proposed method through comparative experiments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.05258" label="2012.05258">
-        <attvalues>
-          <attvalue for="0" value="ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic&#10;  Segmentation" />
-          <attvalue for="1" value="  In this paper, we present ViP-DeepLab, a unified model attempting to tackle&#10;the long-standing and challenging inverse projection problem in vision, which&#10;we model as restoring the point clouds from perspective image sequences while&#10;providing each point with instance-level semantic interpretations. Solving this&#10;problem requires the vision models to predict the spatial location, semantic&#10;class, and temporally consistent instance label for each 3D point. ViP-DeepLab&#10;approaches it by jointly performing monocular depth estimation and video&#10;panoptic segmentation. We name this joint task as Depth-aware Video Panoptic&#10;Segmentation, and propose a new evaluation metric along with two derived&#10;datasets for it, which will be made available to the public. On the individual&#10;sub-tasks, ViP-DeepLab also achieves state-of-the-art results, outperforming&#10;previous methods by 5.1% VPQ on Cityscapes-VPS, ranking 1st on the KITTI&#10;monocular depth estimation benchmark, and 1st on KITTI MOTS pedestrian. The&#10;datasets and the evaluation codes are made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;\slimparagraph{Panoptic Segmentation}&#10;Recent methods for image panoptic segmentation can be grouped into two types: top-down (proposal-based) methods and bottom-up (box-free) methods.&#10;Top-down methods employ a two-stage approach which generates object proposals followed by outputting panoptic predictions based on regional computations \cite{sofiiuk2019adaptis,li2020unifying,qiao2020detectors,wu2020bidirectional,lazarow2020learning,chen2020banet,li2018learning,li2019attention,xiong2019upsnet}.&#10;For example, Panoptic FPN~\cite{kirillov2019panoptic} incorporates a semantic segmentation head into Mask R-CNN~\cite{he2017mask}.&#10;Porzi \etal~\cite{porzi2019seamless} proposes a novel segmentation head to integrate FPN~\cite{lin2017feature} features by a lightweight DeepLab-like module~\cite{chen2017deeplab}.&#10;Bottom-up panoptic segmentation methods group pixels to form instances on top of semantic segmentation prediction~\cite{yang2019deeperlab,wang2020axial,wang2020pixel}.&#10;For example, SSAP~\cite{gao2019ssap} uses pixel-pair affinity pyramid~\cite{liu2018affinity} and a cascaded graph partition module~\cite{keuper2015efficient} to generate instances from coarse to fine.&#10;BBFNet~\cite{bonde2020towards} uses Hough-voting~\cite{ballard1981generalizing,leibe2004combined} and Watershed transform~\cite{vincent1991watersheds,bai2017deep} to generate instance segmentation predictions.&#10;Panoptic-DeepLab~\cite{cheng2020panoptic} employs class-agnostic instance center regression~\cite{kendall2018multi,uhrig2018box2pix,neven2019instance} on top of semantic segmentation outputs from DeepLab~\cite{chen2014semantic,chen2018encoder}.&#10;&#10;\slimparagraph{Object Tracking}&#10;One of the major tasks in video panoptic segmentation is object tracking.&#10;Many trackers use tracking-by-detection,&#10;which divides the task into two sub-tasks where an object detector (\eg \cite{felzenszwalb2009object,ren2015faster}) finds all objects and then an algorithm associates them~\cite{bewley2016simple,fang2018recurrent,leal2016learning,schulter2017deep,sharma2018beyond,tang2017multiple,wojke2017simple,xu2019spatial,zhu2018online,porzi2020learning}.&#10;Another design is transforming object detectors to object trackers which detect and track objects at the same time~\cite{feichtenhofer2017detect,zhang2018integrated,bergmann2019tracking,peng2020chained,wang2020towards,zhang2020fair}.&#10;For example, &#10;CenterTrack~\cite{zhou2020tracking} extends CenterNet~\cite{zhou2019objects} to predict offsets from the object center to its center in the previous frame.&#10;STEm-Seg~\cite{athar2020stem} proposes to group all instance pixels in a video clip by learning a spatio-temporal embedding.&#10;By contrast, our ViP-DeepLab implicitly performs object tracking by clustering all instance pixels in two consecutive video frames. Additionally, our method simply uses center&#10;regression and achieves better results on MOTS~\cite{voigtlaender2019mots}.&#10;&#10;\slimparagraph{Monocular Depth Estimation}&#10;Monocular depth estimation predicts depth from a single image.&#10;It can be learned in a supervised way~\cite{saxena2006learning,eigen2015predicting,li2015depth,cao2017estimating,fu2018deep,xu2017multi,gan2018monocular,yin2019enforcing,lee2019big},&#10;by reconstructing images in the stereo setting~\cite{garg2016unsupervised,godard2017unsupervised,godard2019digging,xie2016deep3d,kuznietsov2017semi}, from videos~\cite{yin2018geonet,mahjourian2018unsupervised,wang2018learning}, or in relative order~\cite{chen2016single}.&#10;ViP-DeepLab models monocular depth estimation as a dense regression problem, and we train it in a fully-supervised manner.&#10;&#10;" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Segmentation, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2303.12217" label="2303.12217">
-        <attvalues>
-          <attvalue for="0" value="Image Reconstruction without Explicit Priors" />
-          <attvalue for="1" value="  We consider solving ill-posed imaging inverse problems without access to an&#10;explicit image prior or ground-truth examples. An overarching challenge in&#10;inverse problems is that there are many undesired images that fit to the&#10;observed measurements, thus requiring image priors to constrain the space of&#10;possible solutions to more plausible reconstructions. However, in many&#10;applications it is difficult or potentially impossible to obtain ground-truth&#10;images to learn an image prior. Thus, inaccurate priors are often used, which&#10;inevitably result in biased solutions. Rather than solving an inverse problem&#10;using priors that encode the explicit structure of any one image, we propose to&#10;solve a set of inverse problems jointly by incorporating prior constraints on&#10;the collective structure of the underlying images.The key assumption of our&#10;work is that the ground-truth images we aim to reconstruct share common,&#10;low-dimensional structure. We show that such a set of inverse problems can be&#10;solved simultaneously by learning a shared image generator with a&#10;low-dimensional latent space. The parameters of the generator and latent&#10;embedding are learned by maximizing a proxy for the Evidence Lower Bound&#10;(ELBO). Once learned, the generator and latent embeddings can be combined to&#10;provide reconstructions for each inverse problem. The framework we propose can&#10;handle general forward model corruptions, and we show that measurements derived&#10;from only a few ground-truth images (O(10)) are sufficient for image&#10;reconstruction without explicit priors.&#10;" />
-          <attvalue for="2" value="&#10;In imaging inverse problems, the goal is to recover the ground-truth image from corrupted measurements, where the measurements and image are related via a forward model: $y = f(x) + \eta$. Here, $y$ are our measurements, $x$ is the ground-truth image, $f$ is a known forward model, and $\eta$ is noise. Such problems are ubiquitous and include denoising \cite{Burger2012}, super-resolution \cite{CandesGranda}, compressed sensing \cite{CandesRombergTao2006, Dono2007}, and phase retrieval \cite{Numerics-of-PR}. These problems are often ill-posed: there are many images that are consistent with the observed measurements. Thus, traditionally structural assumptions on the image are required to reduce the space of possible solutions. We encode these structural assumptions in an image generation model (IGM), which could take the form of either an implicit or explicit image prior. &#10;&#10;In order to define an IGM it is necessary to have knowledge of the structure of the underlying image distribution. If ground-truth images are available, then an IGM can be learned directly \cite{venkatakrishnan2013pnp, Boraetal17}. However, this approach requires access to an abundance of clean data and there are many scientific applications (e.g., geophysical imaging and astronomical imaging) where we do not have access to ground-truth images. Collecting ground-truth images in these domains can be extremely invasive, time-consuming, expensive, or even impossible. For instance, how should we define an IGM for black hole imaging without having ever seen a direct image of a black hole or knowing what one should look like?&#10;Moreover, classical approaches that utilize hand-crafted IGMs \cite{TV-ROF} are prone to human bias \cite{akiyama2019first4}.&#10;&#10;In this work, we show how one can solve a set of ill-posed image reconstruction tasks without access to an explicit image prior. The key insight of our work is that knowledge of common structure across multiple problems is sufficient regularization alone. In particular, we suppose we have access to a collection of measurements $\{y^{(i)}\}_{i=1}^N$ which are observed through a forward model $y^{(i)} := f(x^{(i)}) + \eta^{(i)}$. An important assumption we make is that the ground-truth images $\{x^{(i)}\}_{i=1}^N$ are drawn from the same distribution (unknown {a priori}) and share common, low-dimensional structure. This assumption is satisfied in a number of applications where ground-truth images are not available. For instance, although we might not know what a black hole looks like, we might expect it to be similar in appearance over time. Our main result is that one can exploit this common structure by jointly inferring 1) a single generator $G_{\theta}$ and 2) $N$ low-dimensional latent distributions $q_{\phi^{(i)}}$, such that the distribution induced by the push-forward of $q_{\phi^{(i)}}$ through $G_{\theta}$ approximates the posterior $p(x| y^{(i)})$ for each example $i \in [N]$. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Inverse Imaging Problems, Machine Learning, Computer Vision, Mathematics, Latent Space Modeling, Physics, Image Prior Learning, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1906.11241" label="1906.11241">
-        <attvalues>
-          <attvalue for="0" value="First M87 Event Horizon Telescope Results. IV. Imaging the Central&#10;  Supermassive Black Hole" />
-          <attvalue for="1" value="  We present the first Event Horizon Telescope (EHT) images of M87, using&#10;observations from April 2017 at 1.3 mm wavelength. These images show a&#10;prominent ring with a diameter of ~40 micro-as, consistent with the size and&#10;shape of the lensed photon orbit encircling the &quot;shadow&quot; of a supermassive&#10;black hole. The ring is persistent across four observing nights and shows&#10;enhanced brightness in the south. To assess the reliability of these results,&#10;we implemented a two-stage imaging procedure. In the first stage, four teams,&#10;each blind to the others' work, produced images of M87 using both an&#10;established method (CLEAN) and a newer technique (regularized maximum&#10;likelihood). This stage allowed us to avoid shared human bias and to assess&#10;common features among independent reconstructions. In the second stage, we&#10;reconstructed synthetic data from a large survey of imaging parameters and then&#10;compared the results with the corresponding ground truth images. This stage&#10;allowed us to select parameters objectively to use when reconstructing images&#10;of M87. Across all tests in both stages, the ring diameter and asymmetry&#10;remained stable, insensitive to the choice of imaging technique. We describe&#10;the EHT imaging procedures, the primary image features in M87, and the&#10;dependence of these features on imaging assumptions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.06401" label="2112.06401">
-        <attvalues>
-          <attvalue for="0" value="Deep Attentional Guided Image Filtering" />
-          <attvalue for="1" value="  Guided filter is a fundamental tool in computer vision and computer graphics&#10;which aims to transfer structure information from guidance image to target&#10;image. Most existing methods construct filter kernels from the guidance itself&#10;without considering the mutual dependency between the guidance and the target.&#10;However, since there typically exist significantly different edges in the two&#10;images, simply transferring all structural information of the guidance to the&#10;target would result in various artifacts. To cope with this problem, we propose&#10;an effective framework named deep attentional guided image filtering, the&#10;filtering process of which can fully integrate the complementary information&#10;contained in both images. Specifically, we propose an attentional kernel&#10;learning module to generate dual sets of filter kernels from the guidance and&#10;the target, respectively, and then adaptively combine them by modeling the&#10;pixel-wise dependency between the two images. Meanwhile, we propose a&#10;multi-scale guided image filtering module to progressively generate the&#10;filtering result with the constructed kernels in a coarse-to-fine manner.&#10;Correspondingly, a multi-scale fusion strategy is introduced to reuse the&#10;intermediate results in the coarse-to-fine process. Extensive experiments show&#10;that the proposed framework compares favorably with the state-of-the-art&#10;methods in a wide range of guided image filtering applications, such as guided&#10;super-resolution, cross-modality restoration, texture removal, and semantic&#10;segmentation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;G{uided} filter (GF), also named joint filter, is tailored to transfer structural information from a guidance image to a target one. The popularity of GF can be attributed to its ability in handling visual signals in various domains and modalities, where one modal signal serves as the guidance to improve the quality of the other one. It has been a useful tool for many image processing and computer vision tasks, such as depth map super-resolution~\cite{JBU, DJFR}, scale-space filtering~\cite{sdf, RGF}, cross-modality image restoration~\cite{GF, mugf, guide-filter-SR2019}, structure-texture separation~\cite{texture2012, texture2013, RTV}, image semantic segmentation~\cite{FBS, DKN} and so on.&#10;&#10;In the literature, GF has been extensively studied, ranging from the classical bilateral filter to the emerging deep learning-based ones. The pioneer bilateral filter \cite{bilateral} constructs spatially-varying kernels, where local image structures of the guidance image are explicitly involved into filtering process through the photometric similarity. The guided image filtering scheme proposed by He et al.~\cite{GF} takes a more rigorous manner to exploit the structure information of the guidance, which computes a locally linear model over the guidance image for filtering. These filters consider only the information contained in the guidance image in filtering. However, since there typically exist significantly different edges in the two images, simply transferring all patterns of the guidance to the target would introduce various artifacts. Some works~\cite{sdf, mugf} propose to utilize the optimization-based manner to find mutual structures for propagation while suppressing inconsistent ones. However, it is challenging to select reference structures and propagate them properly by hand-crafted objective functions. In addition, the computational complexity of these methods is usually high.&#10;&#10;In recent years, learning-based approaches for GF design are becoming increasingly popular, which derive GF in a purely data-driven manner. They allow the networks to learn how to adaptively select structures to transfer, and thus have the ability to handle more complicated scenarios. For instance, in~\cite{DFN}, a dynamic filter network (DFN) is proposed where pixel-wise filters are generated dynamically using a separate sub-network conditioned on the guidance. Unlike DFN, Su et al.~\cite{PanNet} adapts a standard spatially invariant kernel at each pixel by multiplying it with a spatially varying filter. Although with increased flexibility thanks to their adaptive nature, ~\cite{DFN} and \cite{PanNet} still suffer from the same drawback as~\cite{GF, bilateral} that only the guidance information is considered in filters design. Some recent methods attempt to exploit the target and guidance information jointly. For instance, Li et al.~\cite{DJFR} propose to leverage two sub-networks to extract informative features from both the target and guidance images, which are then concatenated as inputs for the fusion network to selectively transfer salient structures from the guidance to the target. Instead of regressing the filtering results directly from the network, Kim et al.~\cite{DKN} proposes to use spatially variant weighted averages, where the set of neighbors and the corresponding kernel weights are learned in an end-to-end manner. However, in the designed networks of these methods, the simple concatenation or element-wise multiplication is exploited to combine multi-modal information, which is not that effective. There is no mechanism to distinguish the contributions of the guidance and the target to the final filtering result, and thus would also lead to erroneous structure propagation. In addition, the guidance and target images are treated as independent information since existing methods typically utilize two separate networks for feature extraction, thus the complementary information contained in the two images cannot be fully exploited.&#10;&#10;By reviewing existing GF methods, it can be found that most of them concentrate their efforts on how to transfer structural information from the guidance to the target. However, for some scenarios, such as cross-modality image restoration~\cite{CUNet} and guided super-resolution~\cite{GSPRT}, multi-modal data has significantly different characteristics due to the difference of sensing principle, making the guidance not always trustworthy. In view of this, we argue that the purpose of GF should be two-folds: 1) apply the guidance as a prior for reconstruction of regions in the target where there are structure-consistent contents; and 2) derive a plausible prediction for regions in the target with inconsistent contents of the guidance. The latter represents the case that the guidance is no longer reliable, so we have to rely on the target itself for reconstruction. Most existing GF methods only concern structure transferring from the guidance, but neglect structure prediction from the target, leading to erroneous or extraneous artifacts in the output. It implies that instead of performing regression on guidance only, as done in ~\cite{GF, bilateral}, we should perform dual regression on both the guidance and the target, and combine them adaptively in a smarter manner instead of simple concatenation or element-wise multiplication, as done in ~\cite{DJFR, DKN}. ``Dual regression&quot; and ``smart combination&quot; bring the main motivations of our proposed method. &#10;&#10;Accordingly, in this paper, we propose an effective deep attentional guided image filtering scheme, which constructs filter kernels by fully considering information from both guidance and target images. Specifically, an attentional kernel learning module is proposed to generate dual sets of filter kernels from the guidance and the target, respectively. Moreover, pixel-wise contributions of the guidance and the target to the final filtering result are automatically learned. In this way, we can adaptively apply the guidance as a prior for reconstruction of target regions where there are structure-consistent contents with the guidance; and derive a prediction for target regions with inconsistent contents by regression on the target itself. We show an illustrated example in Fig.~\ref{fig:fm}, which presents the visual filtering results comparison of our scheme with the state-of-the-art guided depth super-resolution methods.&#10;It can be found that our proposed method is capable of producing high-resolution depth image with clear boundaries as well as avoiding texture-copying artifacts. &#10;&#10;The main contributions of the proposed method are summarized as follows:&#10;&#10;\begin{itemize}&#10; \item We propose an attentional kernel learning (AKL) module for guided image filtering, which generates dual sets of filter kernels from both guidance and target, and then adaptively combines these kernels by modeling the pixel-wise dependencies between the two images in a learning manner. Compared with existing kernel generation approaches, the proposed method is more robust when there are inconsistent structures&#10; between the guidance and the target.&#10; \item We propose a multi-scale guided filtering module, which generates the filtering result in a coarse-to-fine manner. Correspondingly, we propose a multi-scale fusion strategy with deep supervision to fully explore the intermediate results in the coarse-to-fine process. To the best of our knowledge, this is the first guided filter framework that learns the multi-scale kernels to filter the target image at different scales in the embedding space.&#10; \item We evaluate the performance of the proposed method on various computational photography and computer vision tasks, such as guided image super-resolution, cross-modality image restoration, texture removal, and semantic segmentation. The quantitative and qualitative results demonstrate the effectiveness and universality of the proposed method. &#10; \item Considering that there is no standard protocol to train and evaluate the performance of guided image filtering algorithms, we reimplement eight recently proposed state-of-the-art deep learning-based guided filtering models and unify their settings to facilitate fair comparison. All of the codes and trained models are publicly available to encourage reproducible research.&#10;\end{itemize}&#10;&#10;The remainder of this paper is organized as follows. Sect.~\ref{gfr} gives a brief introduction to the relevant works of guided filter. Sect.~\ref{sec_3} introduces the proposed method for guided image filtering. Sect.~\ref{exp} provides experimental comparisons with existing state-of-the-art methods for a varied range of guided filtering tasks. Ablation experiments are presented in Sect.~\ref{abl} to analyze the network hyper-parameters and verify the advantage of each components proposed in our model. We conclude the paper in Sect.~\ref{con}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Deep Learning, Computer Science, Image Filtering, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1605.09673" label="1605.09673">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Filter Networks" />
-          <attvalue for="1" value="  In a traditional convolutional layer, the learned filters stay fixed after&#10;training. In contrast, we introduce a new framework, the Dynamic Filter&#10;Network, where filters are generated dynamically conditioned on an input. We&#10;show that this architecture is a powerful one, with increased flexibility&#10;thanks to its adaptive nature, yet without an excessive increase in the number&#10;of model parameters. A wide variety of filtering operations can be learned this&#10;way, including local spatial transformations, but also others like selective&#10;(de)blurring or adaptive feature extraction. Moreover, multiple such layers can&#10;be combined, e.g. in a recurrent architecture. We demonstrate the effectiveness&#10;of the dynamic filter network on the tasks of video and stereo prediction, and&#10;reach state-of-the-art performance on the moving MNIST dataset with a much&#10;smaller model. By visualizing the learned filters, we illustrate that the&#10;network has picked up flow information by only looking at unlabelled training&#10;data. This suggests that the network can be used to pretrain networks for&#10;various supervised tasks in an unsupervised way, like optical flow and depth&#10;estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.05373" label="1904.05373">
-        <attvalues>
-          <attvalue for="0" value="Pixel-Adaptive Convolutional Neural Networks" />
-          <attvalue for="1" value="  Convolutions are the fundamental building block of CNNs. The fact that their&#10;weights are spatially shared is one of the main reasons for their widespread&#10;use, but it also is a major limitation, as it makes convolutions content&#10;agnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yet&#10;effective modification of standard convolutions, in which the filter weights&#10;are multiplied with a spatially-varying kernel that depends on learnable, local&#10;pixel features. PAC is a generalization of several popular filtering techniques&#10;and thus can be used for a wide range of use cases. Specifically, we&#10;demonstrate state-of-the-art performance when PAC is used for deep joint image&#10;upsampling. PAC also offers an effective alternative to fully-connected CRF&#10;(Full-CRF), called PAC-CRF, which performs competitively, while being&#10;considerably faster. In addition, we also demonstrate that PAC can be used as a&#10;drop-in replacement for convolution layers in pre-trained networks, resulting&#10;in consistent performance improvements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.04200" label="1710.04200">
-        <attvalues>
-          <attvalue for="0" value="Joint Image Filtering with Deep Convolutional Networks" />
-          <attvalue for="1" value="  Joint image filters leverage the guidance image as a prior and transfer the&#10;structural details from the guidance image to the target image for suppressing&#10;noise or enhancing spatial resolution. Existing methods either rely on various&#10;explicit filter constructions or hand-designed objective functions, thereby&#10;making it difficult to understand, improve, and accelerate these filters in a&#10;coherent framework. In this paper, we propose a learning-based approach for&#10;constructing joint filters based on Convolutional Neural Networks. In contrast&#10;to existing methods that consider only the guidance image, the proposed&#10;algorithm can selectively transfer salient structures that are consistent with&#10;both guidance and target images. We show that the model trained on a certain&#10;type of data, e.g., RGB and depth images, generalizes well to other modalities,&#10;e.g., flash/non-Flash and RGB/NIR images. We validate the effectiveness of the&#10;proposed joint filter through extensive experimental evaluations with&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.04066" label="1910.04066">
-        <attvalues>
-          <attvalue for="0" value="Deep Convolutional Neural Network for Multi-modal Image Restoration and&#10;  Fusion" />
-          <attvalue for="1" value="  In this paper, we propose a novel deep convolutional neural network to solve&#10;the general multi-modal image restoration (MIR) and multi-modal image fusion&#10;(MIF) problems. Different from other methods based on deep learning, our&#10;network architecture is designed by drawing inspirations from a new proposed&#10;multi-modal convolutional sparse coding (MCSC) model. The key feature of the&#10;proposed network is that it can automatically split the common information&#10;shared among different modalities, from the unique information that belongs to&#10;each single modality, and is therefore denoted with CU-Net, i.e., Common and&#10;Unique information splitting network. Specifically, the CU-Net is composed of&#10;three modules, i.e., the unique feature extraction module (UFEM), common&#10;feature preservation module (CFPM), and image reconstruction module (IRM). The&#10;architecture of each module is derived from the corresponding part in the MCSC&#10;model, which consists of several learned convolutional sparse coding (LCSC)&#10;blocks. Extensive numerical results verify the effectiveness of our method on a&#10;variety of MIR and MIF tasks, including RGB guided depth image&#10;super-resolution, flash guided non-flash image denoising, multi-focus and&#10;multi-exposure image fusion.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.01501" label="1904.01501">
-        <attvalues>
-          <attvalue for="0" value="Guided Super-Resolution as Pixel-to-Pixel Transformation" />
-          <attvalue for="1" value="  Guided super-resolution is a unifying framework for several computer vision&#10;tasks where the inputs are a low-resolution source image of some target&#10;quantity (e.g., perspective depth acquired with a time-of-flight camera) and a&#10;high-resolution guide image from a different domain (e.g., a grey-scale image&#10;from a conventional camera); and the target output is a high-resolution version&#10;of the source (in our example, a high-res depth map). The standard way of&#10;looking at this problem is to formulate it as a super-resolution task, i.e.,&#10;the source image is upsampled to the target resolution, while transferring the&#10;missing high-frequency details from the guide. Here, we propose to turn that&#10;interpretation on its head and instead see it as a pixel-to-pixel mapping of&#10;the guide image to the domain of the source image. The pixel-wise mapping is&#10;parametrised as a multi-layer perceptron, whose weights are learned by&#10;minimising the discrepancies between the source image and the downsampled&#10;target image. Importantly, our formulation makes it possible to regularise only&#10;the mapping function, while avoiding regularisation of the outputs; thus&#10;producing crisp, natural-looking images. The proposed method is unsupervised,&#10;using only the specific source and guide images to fit the mapping. We evaluate&#10;our method on two different tasks, super-resolution of depth maps and of tree&#10;height maps. In both cases, we clearly outperform recent baselines in&#10;quantitative comparisons, while delivering visually much sharper outputs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.04706" label="2108.04706">
-        <attvalues>
-          <attvalue for="0" value="BIDCD -- Bosch Industrial Depth Completion Dataset" />
-          <attvalue for="1" value="  We introduce BIDCD -- the Bosch Industrial Depth Completion Dataset. BIDCD is&#10;a new RGBD dataset of metallic industrial objects, collected with a depth&#10;camera mounted on a robotic manipulator. The main purpose of this dataset is to&#10;facilitate the training of domain-specific depth completion models, to be used&#10;in logistics and manufacturing tasks. We trained a State-of-the-Art depth&#10;completion model on this dataset, and report the results, setting an initial&#10;benchmark. Further, we propose to use this dataset for learning&#10;synthetic-to-depth-camera domain adaptation. Modifying synthetic RGBD data to&#10;mimic characteristics of real-world depth acquisition could potentially enhance&#10;training on synthetic data. For this end, we trained a Generative Adversarial&#10;Network (GAN) on a synthetic industrial dataset and our real-world data.&#10;Finally, to address geometric distortions in the generated images, we introduce&#10;an auxiliary loss that promotes preservation of the original shape. The BIDCD&#10;data is publicly available at https://zenodo.org/communities/bidcd.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Robotic manipulation is an important component of the industry 4.0 revolution, both for logistic tasks such as bin-picking and assembly tasks such as peg-in-hole. Operation in unstructured environments requires perception, most commonly implemented with a camera for data acquisition feeding a computer vision model. Perception models need to provide 3D information for task and motion planning, which is typically done in six or more Degree-of-Freedom (DOF). &#10;Digital RGB cameras have been used to estimate 3D geometry using Structure-from-Motion (SfM) \cite{sfm_2017}, single-frame monocular depth estimation \cite{monocular_Godard_2019_ICCV}, and 3D pose estimation of known objects \cite{tremblay2018corl:dope}. &#10;&#10;In contrast to RGB cameras, distance sensors can provide a direct measurement of 3D geometry. In outdoor applications, the most notable one being self-driving cars, multiple different modalities have been utilized for depth sensing, including LiDAR, Radar, and Ultra Sound (US). &#10;For indoor applications, a highly prevalent modality for 3D acquisition is depth cameras, which provide RGBD images \cite{rgbd_datasets}. These dense depth maps have been successfully employed for grasp prediction of unknown objects \cite{mahler2019_dexnet4, zeng2019tossingbot}. &#10;Depth cameras can be implemented in several different ways, including stereo vision, structured light, and Time-of-Flight (ToF), but all these methods suffer from large errors and loss of information (&quot;holes&quot;) under optical reflections and deflection. This pitfall is prominent in industrial settings, which routinely include reflective metallic objects. &#10;&#10;The shortcomings of depth sensors have been previously addressed with depth completion \cite{depth_comp_unit,Ma19icra,NLSPN,plugNplay19,sajjan2019cleargrasp} on single frames. &#10;Robotic applications often use mounted cameras, which present the opportunity to utilize multiple views and applying volumetric fusion methods, such as the Truncated Signed Distance Function (TSDF) \cite{tsdf_1996, yak}. &#10;A preliminary requirement for TSDF is the registration between the different points of view.&#10;However, industrial applications tend to be confounded with a multitude of artifacts, which may in turn cause the registration to fail. &#10;For this reason and additional operational considerations, single-frame solutions seem to be a more viable alternative.&#10;&#10;Depth completion can be thought of as a type of guided interpolation. The goal is to generate a dense depth map using a partial sampling (raw depth), guided by visual RGB cues such as perspective, occlusions, object boundaries, and surface normals. &#10;Depth completion datasets include RGB and raw-depth inputs and an estimated dense Ground-Truth (GT) depth map. In driving applications, the depth is typically obtained with LiDAR \cite{kitti}. In contrast, depth cameras are predominant when imaging indoor scenes \cite{Silberman:NYUv2,dai2017scannet}, household objects \cite{ycb}, and other applications \cite{rgbd_datasets}. &#10;These two acquisition systems pose distinct challenges for depth completion, suggesting that they warrant different solutions.&#10;LiDAR provides sparse depth maps where the values of the samples are typically considered to be sufficiently accurate. Thus, generating a dense depth map is largely a question of partitioning the influence regions around the depth samples using RGB cues such as boundaries.&#10;Depth cameras, on the other hand, may yield large connected regions with missing or invalid values, for example at window panes \cite{Silberman:NYUv2,dai2017scannet}. Moreover, specular reflections and transparent recyclable objects \cite{sajjan2019cleargrasp} may induce large regions of erroneous values that need to be identified and corrected.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Depth Completion Models, Computer Vision, Engineering, Machine Learning, Domain Adaptation Techniques, Mathematics, Industrial Object Recognition, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1809.10790" label="1809.10790">
-        <attvalues>
-          <attvalue for="0" value="Deep Object Pose Estimation for Semantic Robotic Grasping of Household&#10;  Objects" />
-          <attvalue for="1" value="  Using synthetic data for training deep neural networks for robotic&#10;manipulation holds the promise of an almost unlimited amount of pre-labeled&#10;training data, generated safely out of harm's way. One of the key challenges of&#10;synthetic data, to date, has been to bridge the so-called reality gap, so that&#10;networks trained on synthetic data operate correctly when exposed to real-world&#10;data. We explore the reality gap in the context of 6-DoF pose estimation of&#10;known objects from a single RGB image. We show that for this problem the&#10;reality gap can be successfully spanned by a simple combination of domain&#10;randomized and photorealistic data. Using synthetic data generated in this&#10;manner, we introduce a one-shot deep neural network that is able to perform&#10;competitively against a state-of-the-art network trained on a combination of&#10;real and synthetic data. To our knowledge, this is the first deep network&#10;trained only on synthetic data that is able to achieve state-of-the-art&#10;performance on 6-DoF object pose estimation. Our network also generalizes&#10;better to novel environments including extreme lighting conditions, for which&#10;we show qualitative results. Using this network we demonstrate a real-time&#10;system estimating object poses with sufficient accuracy for real-world semantic&#10;grasping of known household objects in clutter by a real robot.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.00999" label="1604.00999">
-        <attvalues>
-          <attvalue for="0" value="RGBD Datasets: Past, Present and Future" />
-          <attvalue for="1" value="  Since the launch of the Microsoft Kinect, scores of RGBD datasets have been&#10;released. These have propelled advances in areas from reconstruction to gesture&#10;recognition. In this paper we explore the field, reviewing datasets across&#10;eight categories: semantics, object pose estimation, camera tracking, scene&#10;reconstruction, object tracking, human actions, faces and identification. By&#10;extracting relevant information in each category we help researchers to find&#10;appropriate data for their needs, and we consider which datasets have succeeded&#10;in driving computer vision forward and why.&#10;  Finally, we examine the future of RGBD datasets. We identify key areas which&#10;are currently underexplored, and suggest that future directions may include&#10;synthetic data and dense reconstructions of static and dynamic scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.00769" label="1808.00769">
-        <attvalues>
-          <attvalue for="0" value="Sparse and Dense Data with CNNs: Depth Completion and Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  Convolutional neural networks are designed for dense data, but vision data is&#10;often sparse (stereo depth, point clouds, pen stroke, etc.). We present a&#10;method to handle sparse depth data with optional dense RGB, and accomplish&#10;depth completion and semantic segmentation changing only the last layer. Our&#10;proposal efficiently learns sparse features without the need of an additional&#10;validity mask. We show how to ensure network robustness to varying input&#10;sparsities. Our method even works with densities as low as 0.8% (8 layer&#10;lidar), and outperforms all published state-of-the-art on the Kitti depth&#10;completion benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.00275" label="1807.00275">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from&#10;  LiDAR and Monocular Camera" />
-          <attvalue for="1" value="  Depth completion, the technique of estimating a dense depth image from sparse&#10;depth measurements, has a variety of applications in robotics and autonomous&#10;driving. However, depth completion faces 3 main challenges: the irregularly&#10;spaced pattern in the sparse depth input, the difficulty in handling multiple&#10;sensor modalities (when color images are available), as well as the lack of&#10;dense, pixel-level ground truth depth labels. In this work, we address all&#10;these challenges. Specifically, we develop a deep regression model to learn a&#10;direct mapping from sparse depth (and color images) to dense depth. We also&#10;propose a self-supervised training framework that requires only sequences of&#10;color and sparse depth images, without the need for dense depth labels. Our&#10;experiments demonstrate that our network, when trained with semi-dense&#10;annotations, attains state-of-the- art accuracy and is the winning approach on&#10;the KITTI depth completion benchmark at the time of submission. Furthermore,&#10;the self-supervised framework outperforms a number of existing solutions&#10;trained with semi- dense annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.10042" label="2007.10042">
-        <attvalues>
-          <attvalue for="0" value="Non-Local Spatial Propagation Network for Depth Completion" />
-          <attvalue for="1" value="  In this paper, we propose a robust and efficient end-to-end non-local spatial&#10;propagation network for depth completion. The proposed network takes RGB and&#10;sparse depth images as inputs and estimates non-local neighbors and their&#10;affinities of each pixel, as well as an initial depth map with pixel-wise&#10;confidences. The initial depth prediction is then iteratively refined by its&#10;confidence and non-local spatial propagation procedure based on the predicted&#10;non-local neighbors and corresponding affinities. Unlike previous algorithms&#10;that utilize fixed-local neighbors, the proposed algorithm effectively avoids&#10;irrelevant local neighbors and concentrates on relevant non-local neighbors&#10;during propagation. In addition, we introduce a learnable affinity&#10;normalization to better learn the affinity combinations compared to&#10;conventional methods. The proposed algorithm is inherently robust to the&#10;mixed-depth problem on depth boundaries, which is one of the major issues for&#10;existing depth estimation/completion algorithms. Experimental results on indoor&#10;and outdoor datasets demonstrate that the proposed algorithm is superior to&#10;conventional algorithms in terms of depth completion accuracy and robustness to&#10;the mixed-depth problem. Our implementation is publicly available on the&#10;project page.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.02550" label="1910.02550">
-        <attvalues>
-          <attvalue for="0" value="ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation" />
-          <attvalue for="1" value="  Transparent objects are a common part of everyday life, yet they possess&#10;unique visual properties that make them incredibly difficult for standard 3D&#10;sensors to produce accurate depth estimates for. In many cases, they often&#10;appear as noisy or distorted approximations of the surfaces that lie behind&#10;them. To address these challenges, we present ClearGrasp -- a deep learning&#10;approach for estimating accurate 3D geometry of transparent objects from a&#10;single RGB-D image for robotic manipulation. Given a single RGB-D image of&#10;transparent objects, ClearGrasp uses deep convolutional networks to infer&#10;surface normals, masks of transparent surfaces, and occlusion boundaries. It&#10;then uses these outputs to refine the initial depth estimates for all&#10;transparent surfaces in the scene. To train and test ClearGrasp, we construct a&#10;large-scale synthetic dataset of over 50,000 RGB-D images, as well as a&#10;real-world test benchmark with 286 RGB-D images of transparent objects and&#10;their ground truth geometries. The experiments demonstrate that ClearGrasp is&#10;substantially better than monocular depth estimation baselines and is capable&#10;of generalizing to real-world images and novel objects. We also demonstrate&#10;that ClearGrasp can be applied out-of-the-box to improve grasping algorithms'&#10;performance on transparent objects. Code, data, and benchmarks will be&#10;released. Supplementary materials available on the project website:&#10;https://sites.google.com/view/cleargrasp&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.13820" label="2003.13820">
-        <attvalues>
-          <attvalue for="0" value="When to Use Convolutional Neural Networks for Inverse Problems" />
-          <attvalue for="1" value="  Reconstruction tasks in computer vision aim fundamentally to recover an&#10;undetermined signal from a set of noisy measurements. Examples include&#10;super-resolution, image denoising, and non-rigid structure from motion, all of&#10;which have seen recent advancements through deep learning. However, earlier&#10;work made extensive use of sparse signal reconstruction frameworks (e.g&#10;convolutional sparse coding). While this work was ultimately surpassed by deep&#10;learning, it rested on a much more developed theoretical framework. Recent work&#10;by Papyan et. al provides a bridge between the two approaches by showing how a&#10;convolutional neural network (CNN) can be viewed as an approximate solution to&#10;a convolutional sparse coding (CSC) problem. In this work we argue that for&#10;some types of inverse problems the CNN approximation breaks down leading to&#10;poor performance. We argue that for these types of problems the CSC approach&#10;should be used instead and validate this argument with empirical evidence.&#10;Specifically we identify JPEG artifact reduction and non-rigid trajectory&#10;reconstruction as challenging inverse problems for CNNs and demonstrate state&#10;of the art performance on them using a CSC method. Furthermore, we offer some&#10;practical improvements to this model and its application, and also show how&#10;insights from the CSC model can be used to make CNNs effective in tasks where&#10;their naive application fails.&#10;" />
-          <attvalue for="2" value="&#10;In computer vision we often deal with signals which contain local spatial or temporal statistical dependencies. Convolution has been demonstrated to be an effective tool for modeling this type of inherent redundancy; notable examples include convolutional sparse coding (CSC) and convolutional neural networks (CNNs). In CSC in particular, one can say that a signal is convolutionally compressible if there exists a convolutional dictionary and a sparse vector such that, &#10;\begin{equation}&#10; \z \approx \D\x \;\;.&#10;\end{equation}&#10;Here $\z$ is the signal, $\D \in \R^{N \times K}$ is the convolutional dictionary, and~$\x$ is the sparse signal such that $\norm{\x}_0 &lt;&lt; K$. The convolutionally compressible assumption has been notably applied to many inverse reconstruction problem in vision such as image inpainting, super resolution, and trajectory reconstruction\cite{Hu_14,Yang_2008,Zhu_15}. &#10;&#10;Inverse problems which use the convolutionally compressible assumption can be expressed in a general form;&#10;\begin{equation}&#10; \arg\min_{\x} \norm{\y - \M \D \x}_2^2 + \lambda\mathbf{\Omega}(\x),&#10;\end{equation}&#10;where $\y$ is the degraded signal such that $\y = \M\z$, and $\mathbf{\Omega}$ provides a prior on the sparsity of $\x$. Throughout this paper we shall assume we know the degradation matrix~$\M$ a priori. &#10;&#10;Recently, however, CNNs have demonstrated their utility for such inverse problems\cite{Xie_2012,Dong_16}. Unlike CSC methods, however, CNNs make no explicit assumptions about the signal ($\z$) being convolutionally compressible; but are yet considered state-of-the-art for most inversion problems in vision. In this paper we want to explore if CNNs are always the correct tool for inverse problems in computer vision. In particular, we hypothesize that for inverse problems with particular types of degradation matrices ($\M$), CNNs are substantially sub-optimal in comparison to CSC inspired approaches. Notable examples include JPEG artifact removal\cite{Fu_2019}, and non-rigid trajectory reconstruction\cite{Zhu_15}. &#10;&#10;Bridge between CSC and CNNs: In this paper we shall heavily rely upon a recently established theoretical connection between CSC and CNNs by Papyan \etal~\cite{Papyan_2017}. Their work shows that a convolutional neural network (CNN) can be interpreted as approximately solving a multi-layer CSC objective. Specifically the authors state that Equation (\ref{eq:opt}) can be approximated by a feed forward CNN when $\mathbf{\Omega}$ represents a hierarchical sparsity model. That is, instead of just assuming that $\z = \D\x$, we further assume that the sparse code $\x$ also has a convolutional structure with respect to a different dictionary, and so on. Given this observation, we can view CNNs as implicitly including the sparsity prior which is made explicit in Equation (\ref{eq:opt}). In this work we use this connection to hypothesize that for certain types of degradation matricies, the implicit convolutional assumptions of CNNs fail and in these cases CSC based methods are expected to perform better.&#10;&#10;When will CNNs fail? Traditional applications of CNNs to inverse problems involve feeding the corrupted signal $\y$ into the network and predicting the original signal $\z$. As shown in the work of Papyan \etal~\cite{Papyan_2017}, this implicitly encodes the assumption that $\y$ is convolutionally compressible. This is a valid assumption for tasks such as super resolution and in painting. However, in this work, we make the observation that it is possible for $\M$ to have a structure that destroys the convolutional compressibility assumption for $\y$, despite it still holding for the original signal $\z$. We predict that for inverse problems where $\M$ has this structure, CSC based models will outperform CNNs through their ability to explicitly model the convolutional structure of the predicted signal separate from the non convolutional structure of the degradation.&#10;&#10;Contributions: Our main contribution is bridging the gap between the sparse coding based analysis of CNNs and its practical application, by predicting which types of inverse problems CNNs struggle with and showing how these issues can be overcome using CSC. To validate our claim we provide four pieces of experimental evidence. First, we perform a synthetic inverse problem experiment which highlights how the structure of $\M$ affects CNN performance. Second, we show how JPEG artifact removal (AR) can be expressed as an inverse problem with non-convolutional $\M$ and achieve state of the art performance with our method. In particular we greatly outperform a recent optimization based approach~\cite{Fu_2019} to JPEG AR by explicitly modeling the non-convolutional nature of $\M$. Third, we identify non-rigid trajectory reconstruction as another inverse problem with non-convolutional $\M$, again demonstrate state of the art performance on it. With this task we also show how the CSC model can be used to design CNNs which elegantly include non-convolutional $\M$. Finally, we propose several practical contributions to the application of multi layer CSC models which improve performance in all tasks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Signal Reconstruction, Deep Learning, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1607.08194" label="1607.08194">
-        <attvalues>
-          <attvalue for="0" value="Convolutional Neural Networks Analyzed via Convolutional Sparse Coding" />
-          <attvalue for="1" value="  Convolutional neural networks (CNN) have led to many state-of-the-art results&#10;spanning through various fields. However, a clear and profound theoretical&#10;understanding of the forward pass, the core algorithm of CNN, is still lacking.&#10;In parallel, within the wide field of sparse approximation, Convolutional&#10;Sparse Coding (CSC) has gained increasing attention in recent years. A&#10;theoretical study of this model was recently conducted, establishing it as a&#10;reliable and stable alternative to the commonly practiced patch-based&#10;processing. Herein, we propose a novel multi-layer model, ML-CSC, in which&#10;signals are assumed to emerge from a cascade of CSC layers. This is shown to be&#10;tightly connected to CNN, so much so that the forward pass of the CNN is in&#10;fact the thresholding pursuit serving the ML-CSC model. This connection brings&#10;a fresh view to CNN, as we are able to attribute to this architecture&#10;theoretical claims such as uniqueness of the representations throughout the&#10;network, and their stable estimation, all guaranteed under simple local&#10;sparsity conditions. Lastly, identifying the weaknesses in the above pursuit&#10;scheme, we propose an alternative to the forward pass, which is connected to&#10;deconvolutional, recurrent and residual networks, and has better theoretical&#10;guarantees.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.07932" label="1911.07932">
-        <attvalues>
-          <attvalue for="0" value="Copy-Move Forgery Classification via Unsupervised Domain Adaptation" />
-          <attvalue for="1" value="  In the current era, image manipulation is becoming increasingly easier,&#10;yielding more natural looking images, owing to the modern tools in image&#10;processing and computer vision techniques. The task of the segregation of&#10;forged images has become very challenging. To tackle such problems, publicly&#10;available datasets are insufficient. In this paper, we propose to create a&#10;synthetic forged dataset using deep semantic image inpainting algorithm.&#10;Furthermore, we use an unsupervised domain adaptation network to detect&#10;copy-move forgery in images. Our approach can be helpful in those cases, where&#10;the classification of data is unavailable.&#10;" />
-          <attvalue for="2" value="&#10;With advancement of new image editing technologies the number of forgery cases is increasing manifold. As a result, in recent years, several deep learning algorithms such as Image Region Forgery Detection \cite{Zhang2016ImageRF}, Augment and Adapt \cite{augment} and BusterNet \cite{busternet} have been proposed to counterattack the problem of image forgery. There are diverse ways of forging images, of which Copy-Move and Splicing forgery are the most common ones. In this paper, we mainly focus on Copy-Move forgery (CMF). &#10;&#10;CMF is a type of passive image forgery technique in which a sectionof an image is copied and pasted within the same image. Many post-image processing operations such as rescaling, affine transformations, resizing, and blurring can be applied to the copied region. &#10;As the source and target image remains the same, the photometric characteristics of the image remain largely invariable. Thus, the detection of this type of forgery becomes even more difficult. &#10;&#10;Contributions In this paper, the primary task considered is that of classification of forged and authentic images, for which we employ Unsupervised Domain Adaptation (DA). Moreover, as the publicly available datasets are small, we generate a new dataset of forged images using deep semantic inpainting algorithm from COCO \cite{coco} dataset. To the best of our knowledge, the use of unsupervised learning has not been exploited for the classification of CMF.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Computer Vision Applications, Mathematics, Image Processing, Image Manipulation Detection, Computer Science, Deep Learning Techniques" />
-        </attvalues>
-      </node>
-      <node id="2303.17123" label="2303.17123">
-        <attvalues>
-          <attvalue for="0" value="Masked and Adaptive Transformer for Exemplar Based Image Translation" />
-          <attvalue for="1" value="  We present a novel framework for exemplar based image translation. Recent&#10;advanced methods for this task mainly focus on establishing cross-domain&#10;semantic correspondence, which sequentially dominates image generation in the&#10;manner of local style control. Unfortunately, cross-domain semantic matching is&#10;challenging; and matching errors ultimately degrade the quality of generated&#10;images. To overcome this challenge, we improve the accuracy of matching on the&#10;one hand, and diminish the role of matching in image generation on the other&#10;hand. To achieve the former, we propose a masked and adaptive transformer (MAT)&#10;for learning accurate cross-domain correspondence, and executing context-aware&#10;feature augmentation. To achieve the latter, we use source features of the&#10;input and global style codes of the exemplar, as supplementary information, for&#10;decoding an image. Besides, we devise a novel contrastive style learning&#10;method, for acquire quality-discriminative style representations, which in turn&#10;benefit high-quality image generation. Experimental results show that our&#10;method, dubbed MATEBIT, performs considerably better than state-of-the-art&#10;methods, in diverse image translation tasks. The codes are available at&#10;\url{https://github.com/AiArt-HDU/MATEBIT}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image-to-image translation aims at transfer images in a source domain to a target domain \cite{isola2017image, zhu2017unpaired}. Early studies learn mappings directly by Generating Adversarial Networks (GANs), and have shown great success in various applications \cite{wang2018esrgan, chang2018pairedcyclegan}. &#10;Recently, exemplar based image translation \cite{ma2018exemplar,zhang2020cocosnet,luo2022memory}, where an exemplar image is used to control the style of translated images, has attracted a lot of attention. &#10;Such methods allow high flexibility and controllability, and have a wide range of potential applications in social networks and metaverse. For example, people can transfer a facial sketch to an artistic portrait, in the style of oil paintings or avatars. Despite the remarkable progress, yielding high-fidelity images with consistent semantic and faithful styles remains a grand challenge.&#10;&#10;Early pioneering works \cite{kipf2016variational, park2019SPADE,huang2018multimodal} attempt to globally control the style of generated images. However, such methods ignore spatial correlations between an input image and an exemplar, and may fail to produce faithful details.&#10;Recently, some advanced methods \cite{zhang2020cocosnet, zhou2021cocosnetv2, zhan2022marginal, DynaST} first establish the cross-domain semantic correspondence between an input image and an exemplar, and then use it to warp the exemplar for controlling local style patterns. &#10;In these methods, the quality of generated images relies heavily on the learned correspondence \cite{seo2022midms}. &#10;Unfortunately, cross-domain semantic matching is challenging, since there is no reliable supervision on correspondence learning \cite{zhang2020cocosnet}. As a result, potential matching errors ultimately lead to degraded artifacts in generated images. &#10;&#10;To combat this challenge, we propose to boost the matching accuracy on one hand, and to diminish the role of matching in image generation on the other hand. &#10;Inspired by the great success of Transformers \cite{vaswani2017attention,dosovitskiy2020image,liu2021swin,heo2021rethinking}, we first devise a Masked and Adaptive Transformer (MAT) for learning accurate cross-domain correspondence and executing context-aware feature augmentation. &#10;Previous works \cite{zhang2020cocosnet, zhou2021cocosnetv2, zhan2022marginal} have used the vanilla attention mechanism \cite{vaswani2017attention} for learning full correspondence. However, the initial attention typically involves ambiguous correspondences (2nd row in Fig. \ref{fig:visatt}). &#10;To mitigate these limitations, in MAT, we use a masked attention to distinguish the correspondence as reliable or not, and then reliability-adaptively aggregate representations. &#10;Besides, the Feed-Forward Network (FFN) \cite{vaswani2017attention} in vanilla transformers neglects contextual correlations inside an image. &#10;We thus replace FFN by an adaptive convolution block \cite{liu2022convnext}, where the coordinate attention \cite{hou2021coordinate} and depthwise separable convolution \cite{chollet2017xception} are used to capture contextual correlations and to improve efficiency. With a joint consideration of matching reliability and contextual correlations, MAT gradually focuses on accurate correspondences and emphasizes on features of interest (3rd row in Fig. \ref{fig:visatt}).&#10;&#10;In addition, to boost both the semantic consistency and style faithfulness, we supplementally use semantic features of the input image and global style codes of the exemplar for decoding an image. &#10;To this end, we first design our whole network following the U-Net architecture \cite{isola2017image}. &#10;Besides, we devise a novel contrastive style learning (CSL) framework for acquiring discriminative style representations.&#10;Recently, Zhang et al. \cite{zhang2020cast} propose a similar CSL method, where the target exemplar is used as a positive sample, and the other exemplars as negative ones. &#10;Differently, we use low-quality images, generated during early training stages, as negative samples. In this way, our style codes are desired to discriminate not only subtle differences in style, but also those in perceptual quality. &#10;Ultimately, the learned global style codes, cooperating with the local style control induced by MAT, in turn benefit high-quality image generation. &#10;&#10;With the proposed techniques above, our full model, dubbed MATEBIT, diminishes the impact of position-wise matching on image quality, and integrates both local and global style control for image generation. &#10;Experimental results show that MATEBIT generates considerably more plausible images than previous state-of-the-art methods, in diverse image translation tasks.&#10;In addition, comprehensive ablation studies demonstrate the effectiveness of our proposed components. &#10;Finally, we perform interesting applications of photo-to-painting translation and Chinese ink paintings generation. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Translation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1805.11145" label="1805.11145">
-        <attvalues>
-          <attvalue for="0" value="Exemplar Guided Unsupervised Image-to-Image Translation with Semantic&#10;  Consistency" />
-          <attvalue for="1" value="  Image-to-image translation has recently received significant attention due to&#10;advances in deep learning. Most works focus on learning either a one-to-one&#10;mapping in an unsupervised way or a many-to-many mapping in a supervised way.&#10;However, a more practical setting is many-to-many mapping in an unsupervised&#10;way, which is harder due to the lack of supervision and the complex inner- and&#10;cross-domain variations. To alleviate these issues, we propose the Exemplar&#10;Guided &amp; Semantically Consistent Image-to-image Translation (EGSC-IT) network&#10;which conditions the translation process on an exemplar image in the target&#10;domain. We assume that an image comprises of a content component which is&#10;shared across domains, and a style component specific to each domain. Under the&#10;guidance of an exemplar from the target domain we apply Adaptive Instance&#10;Normalization to the shared content component, which allows us to transfer the&#10;style information of the target domain to the source domain. To avoid semantic&#10;inconsistencies during translation that naturally appear due to the large&#10;inner- and cross-domain variations, we introduce the concept of feature masks&#10;that provide coarse semantic guidance without requiring the use of any semantic&#10;labels. Experimental results on various datasets show that EGSC-IT does not&#10;only translate the source image to diverse instances in the target domain, but&#10;also preserves the semantic consistency during the process.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07308" label="1611.07308">
-        <attvalues>
-          <attvalue for="0" value="Variational Graph Auto-Encoders" />
-          <attvalue for="1" value="  We introduce the variational graph auto-encoder (VGAE), a framework for&#10;unsupervised learning on graph-structured data based on the variational&#10;auto-encoder (VAE). This model makes use of latent variables and is capable of&#10;learning interpretable latent representations for undirected graphs. We&#10;demonstrate this model using a graph convolutional network (GCN) encoder and a&#10;simple inner product decoder. Our model achieves competitive results on a link&#10;prediction task in citation networks. In contrast to most existing models for&#10;unsupervised learning on graph-structured data and link prediction, our model&#10;can naturally incorporate node features, which significantly improves&#10;predictive performance on a number of benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.11047" label="2209.11047">
-        <attvalues>
-          <attvalue for="0" value="MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image&#10;  Translation" />
-          <attvalue for="1" value="  We present a novel method for exemplar-based image translation, called&#10;matching interleaved diffusion models (MIDMs). Most existing methods for this&#10;task were formulated as GAN-based matching-then-generation framework. However,&#10;in this framework, matching errors induced by the difficulty of semantic&#10;matching across cross-domain, e.g., sketch and photo, can be easily propagated&#10;to the generation step, which in turn leads to degenerated results. Motivated&#10;by the recent success of diffusion models overcoming the shortcomings of GANs,&#10;we incorporate the diffusion models to overcome these limitations.&#10;Specifically, we formulate a diffusion-based matching-and-generation framework&#10;that interleaves cross-domain matching and diffusion steps in the latent space&#10;by iteratively feeding the intermediate warp into the noising process and&#10;denoising it to generate a translated image. In addition, to improve the&#10;reliability of the diffusion process, we design a confidence-aware process&#10;using cycle-consistency to consider only confident regions during translation.&#10;Experimental results show that our MIDMs generate more plausible images than&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.16302" label="2103.16302">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Spatial Dimensions of Vision Transformers" />
-          <attvalue for="1" value="  Vision Transformer (ViT) extends the application range of transformers from&#10;language processing to computer vision tasks as being an alternative&#10;architecture against the existing convolutional neural networks (CNN). Since&#10;the transformer-based architecture has been innovative for computer vision&#10;modeling, the design convention towards an effective architecture has been less&#10;studied yet. From the successful design principles of CNN, we investigate the&#10;role of spatial dimension conversion and its effectiveness on transformer-based&#10;architecture. We particularly attend to the dimension reduction principle of&#10;CNNs; as the depth increases, a conventional CNN increases channel dimension&#10;and decreases spatial dimensions. We empirically show that such a spatial&#10;dimension reduction is beneficial to a transformer architecture as well, and&#10;propose a novel Pooling-based Vision Transformer (PiT) upon the original ViT&#10;model. We show that PiT achieves the improved model capability and&#10;generalization performance against ViT. Throughout the extensive experiments,&#10;we further show PiT outperforms the baseline on several tasks such as image&#10;classification, object detection, and robustness evaluation. Source codes and&#10;ImageNet models are available at https://github.com/naver-ai/pit&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.03545" label="2201.03545">
-        <attvalues>
-          <attvalue for="0" value="A ConvNet for the 2020s" />
-          <attvalue for="1" value="  The &quot;Roaring 20s&quot; of visual recognition began with the introduction of Vision&#10;Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art&#10;image classification model. A vanilla ViT, on the other hand, faces&#10;difficulties when applied to general computer vision tasks such as object&#10;detection and semantic segmentation. It is the hierarchical Transformers (e.g.,&#10;Swin Transformers) that reintroduced several ConvNet priors, making&#10;Transformers practically viable as a generic vision backbone and demonstrating&#10;remarkable performance on a wide variety of vision tasks. However, the&#10;effectiveness of such hybrid approaches is still largely credited to the&#10;intrinsic superiority of Transformers, rather than the inherent inductive&#10;biases of convolutions. In this work, we reexamine the design spaces and test&#10;the limits of what a pure ConvNet can achieve. We gradually &quot;modernize&quot; a&#10;standard ResNet toward the design of a vision Transformer, and discover several&#10;key components that contribute to the performance difference along the way. The&#10;outcome of this exploration is a family of pure ConvNet models dubbed ConvNeXt.&#10;Constructed entirely from standard ConvNet modules, ConvNeXts compete favorably&#10;with Transformers in terms of accuracy and scalability, achieving 87.8%&#10;ImageNet top-1 accuracy and outperforming Swin Transformers on COCO detection&#10;and ADE20K segmentation, while maintaining the simplicity and efficiency of&#10;standard ConvNets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.02907" label="2103.02907">
-        <attvalues>
-          <attvalue for="0" value="Coordinate Attention for Efficient Mobile Network Design" />
-          <attvalue for="1" value="  Recent studies on mobile network design have demonstrated the remarkable&#10;effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention)&#10;for lifting model performance, but they generally neglect the positional&#10;information, which is important for generating spatially selective attention&#10;maps. In this paper, we propose a novel attention mechanism for mobile networks&#10;by embedding positional information into channel attention, which we call&#10;&quot;coordinate attention&quot;. Unlike channel attention that transforms a feature&#10;tensor to a single feature vector via 2D global pooling, the coordinate&#10;attention factorizes channel attention into two 1D feature encoding processes&#10;that aggregate features along the two spatial directions, respectively. In this&#10;way, long-range dependencies can be captured along one spatial direction and&#10;meanwhile precise positional information can be preserved along the other&#10;spatial direction. The resulting feature maps are then encoded separately into&#10;a pair of direction-aware and position-sensitive attention maps that can be&#10;complementarily applied to the input feature map to augment the representations&#10;of the objects of interest. Our coordinate attention is simple and can be&#10;flexibly plugged into classic mobile networks, such as MobileNetV2, MobileNeXt,&#10;and EfficientNet with nearly no computational overhead. Extensive experiments&#10;demonstrate that our coordinate attention is not only beneficial to ImageNet&#10;classification but more interestingly, behaves better in down-stream tasks,&#10;such as object detection and semantic segmentation. Code is available at&#10;https://github.com/Andrew-Qibin/CoordAttention.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.02357" label="1610.02357">
-        <attvalues>
-          <attvalue for="0" value="Xception: Deep Learning with Depthwise Separable Convolutions" />
-          <attvalue for="1" value="  We present an interpretation of Inception modules in convolutional neural&#10;networks as being an intermediate step in-between regular convolution and the&#10;depthwise separable convolution operation (a depthwise convolution followed by&#10;a pointwise convolution). In this light, a depthwise separable convolution can&#10;be understood as an Inception module with a maximally large number of towers.&#10;This observation leads us to propose a novel deep convolutional neural network&#10;architecture inspired by Inception, where Inception modules have been replaced&#10;with depthwise separable convolutions. We show that this architecture, dubbed&#10;Xception, slightly outperforms Inception V3 on the ImageNet dataset (which&#10;Inception V3 was designed for), and significantly outperforms Inception V3 on a&#10;larger image classification dataset comprising 350 million images and 17,000&#10;classes. Since the Xception architecture has the same number of parameters as&#10;Inception V3, the performance gains are not due to increased capacity but&#10;rather to a more efficient use of model parameters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.09542" label="2205.09542">
-        <attvalues>
-          <attvalue for="0" value="Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning" />
-          <attvalue for="1" value="  In this work, we tackle the challenging problem of arbitrary image style&#10;transfer using a novel style feature representation learning method. A suitable&#10;style representation, as a key component in image stylization tasks, is&#10;essential to achieve satisfactory results. Existing deep neural network based&#10;approaches achieve reasonable results with the guidance from second-order&#10;statistics such as Gram matrix of content features. However, they do not&#10;leverage sufficient style information, which results in artifacts such as local&#10;distortions and style inconsistency. To address these issues, we propose to&#10;learn style representation directly from image features instead of their&#10;second-order statistics, by analyzing the similarities and differences between&#10;multiple styles and considering the style distribution. Specifically, we&#10;present Contrastive Arbitrary Style Transfer (CAST), which is a new style&#10;representation learning and style transfer method via contrastive learning. Our&#10;framework consists of three key components, i.e., a multi-layer style projector&#10;for style code encoding, a domain enhancement module for effective learning of&#10;style distribution, and a generative network for image style transfer. We&#10;conduct qualitative and quantitative evaluations comprehensively to demonstrate&#10;that our approach achieves significantly better results compared to those&#10;obtained via state-of-the-art methods. Code and models are available at&#10;https://github.com/zyxElsa/CAST_pytorch&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.08242" label="1901.08242">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Image-to-Image Translation with Self-Attention Networks" />
-          <attvalue for="1" value="  Unsupervised image translation aims to learn the transformation from a source&#10;domain to another target domain given unpaired training data. Several&#10;state-of-the-art works have yielded impressive results in the GANs-based&#10;unsupervised image-to-image translation. It fails to capture strong geometric&#10;or structural changes between domains, or it produces unsatisfactory result for&#10;complex scenes, compared to local texture mapping tasks such as style transfer.&#10;Recently, SAGAN (Han Zhang, 2018) showed that the self-attention network&#10;produces better results than the convolution-based GAN. However, the&#10;effectiveness of the self-attention network in unsupervised image-to-image&#10;translation tasks have not been verified. In this paper, we propose an&#10;unsupervised image-to-image translation with self-attention networks, in which&#10;long range dependency helps to not only capture strong geometric change but&#10;also generate details using cues from all feature locations. In experiments, we&#10;qualitatively and quantitatively show superiority of the proposed method&#10;compared to existing state-of-the-art unsupervised image-to-image translation&#10;task. The source code and our results are online:&#10;https://github.com/itsss/img2img_sa and&#10;http://itsc.kr/2019/01/24/2019_img2img_sa&#10;" />
-          <attvalue for="2" value="&#10;&#10;In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on. This cross-domain image-to-image translation topic has become a major concern of researchers. &#10;&#10;In many cases, given a paired dataset, it is possible to solve the problem with conditional image translation \cite{isola2017image,wang2017high,li2017alice}. However, it is difficult and expensive to obtain the paired samples. In addition, there are cases where supervision is not possible. &#10;&#10;The goal of the unsupervised image translation is to learn the transformation from a source domain to a target domain given unpaired training data. Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}. It can be largely classified into two types. The first is the style transfer task. This problem is to change low-level information such as color or texture while maintaining high-level information such as content or geometric structure. Style transfer and conditional GANs-based methods have yielded excellent results in this research area. &#10;&#10;The second is the object transfiguration task. Unlike the style transfer task, this focuses on changing high-level information while keeping the low-level information. CycleGAN \cite{zhu2017unpaired}, the most representative unsupervised image translation method, failed to change the high-level semantic meaning due to the network structure specialized for style transfer. &#10;&#10;To solve the unsupervised image-to-image translation problem, UNIT \cite{liu2017unsupervised} made a shared-latent space assumption. It assumes a pair of corresponding images in different domains can be mapped to a same latent code in a shared-latent space. MUNIT \cite{huang2018multimodal} proposed a multimodal unsupervised image-to-image translation framework. &#10;&#10;To achieve many-to-many cross domain mapping, it mitigates a fully shared latent space assumption in UNIT by decomposing a shared-latent space across domains and each domain-specific part for the style code. UNIT and MUNIT experimentally showed impressive animal image translation from a cropped dataset centered on the head. When the training image dataset is spatially unnormalized, it makes the problem more difficult because the absence of correspondences between the shared semantic parts. &#10;&#10;In our experiments, we show that these methods often fail in various image-to-image translation applications with strong geometric change. Recently, SAGAN \cite{zhang2018self} showed that the self-attention module is complementary to convolutions and helps with modeling long range, multi-level dependencies across image regions. Despite the success of the self-attention module in non-conditional GANs, the effectiveness of the self-attention module for unsupervised image-to-image translation has not been validated. &#10;&#10;In this paper, we propose a unpaired image-to-image translation model with self-attention networks which allows long range dependency modeling for image translation task with strong geometry change. In experiments, we show superiority of the proposed method compared to existing state-of-the-art unsupervised image-to-image translation tasks.&#10;&#10;The source code and our results are online:&#10;\\&#10; and.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Translation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1603.08511" label="1603.08511">
-        <attvalues>
-          <attvalue for="0" value="Colorful Image Colorization" />
-          <attvalue for="1" value="  Given a grayscale photograph as input, this paper attacks the problem of&#10;hallucinating a plausible color version of the photograph. This problem is&#10;clearly underconstrained, so previous approaches have either relied on&#10;significant user interaction or resulted in desaturated colorizations. We&#10;propose a fully automatic approach that produces vibrant and realistic&#10;colorizations. We embrace the underlying uncertainty of the problem by posing&#10;it as a classification task and use class-rebalancing at training time to&#10;increase the diversity of colors in the result. The system is implemented as a&#10;feed-forward pass in a CNN at test time and is trained on over a million color&#10;images. We evaluate our algorithm using a &quot;colorization Turing test,&quot; asking&#10;human participants to choose between a generated and ground truth color image.&#10;Our method successfully fools humans on 32% of the trials, significantly higher&#10;than previous methods. Moreover, we show that colorization can be a powerful&#10;pretext task for self-supervised feature learning, acting as a cross-channel&#10;encoder. This approach results in state-of-the-art performance on several&#10;feature learning benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.02999" label="1705.02999">
-        <attvalues>
-          <attvalue for="0" value="Real-Time User-Guided Image Colorization with Learned Deep Priors" />
-          <attvalue for="1" value="  We propose a deep learning approach for user-guided image colorization. The&#10;system directly maps a grayscale image, along with sparse, local user &quot;hints&quot;&#10;to an output colorization with a Convolutional Neural Network (CNN). Rather&#10;than using hand-defined rules, the network propagates user edits by fusing&#10;low-level cues along with high-level semantic information, learned from&#10;large-scale data. We train on a million images, with simulated user inputs. To&#10;guide the user towards efficient input selection, the system recommends likely&#10;colors based on the input image and current user inputs. The colorization is&#10;performed in a single feed-forward pass, enabling real-time use. Even with&#10;randomly simulated user inputs, we show that the proposed system helps novice&#10;users quickly create realistic colorizations, and offers large improvements in&#10;colorization quality with just a minute of use. In addition, we demonstrate&#10;that the framework can incorporate other user &quot;hints&quot; to the desired&#10;colorization, showing an application to color histogram transfer. Our code and&#10;models are available at https://richzhang.github.io/ideepcolor.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.02342" label="1812.02342">
-        <attvalues>
-          <attvalue for="0" value="Arbitrary Style Transfer with Style-Attentional Networks" />
-          <attvalue for="1" value="  Arbitrary style transfer aims to synthesize a content image with the style of&#10;an image to create a third image that has never been seen before. Recent&#10;arbitrary style transfer algorithms find it challenging to balance the content&#10;structure and the style patterns. Moreover, simultaneously maintaining the&#10;global and local style patterns is difficult due to the patch-based mechanism.&#10;In this paper, we introduce a novel style-attentional network (SANet) that&#10;efficiently and flexibly integrates the local style patterns according to the&#10;semantic spatial distribution of the content image. A new identity loss&#10;function and multi-level feature embeddings enable our SANet and decoder to&#10;preserve the content structure as much as possible while enriching the style&#10;patterns. Experimental results demonstrate that our algorithm synthesizes&#10;stylized images in real-time that are higher in quality than those produced by&#10;the state-of-the-art algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.11585" label="1711.11585">
-        <attvalues>
-          <attvalue for="0" value="High-Resolution Image Synthesis and Semantic Manipulation with&#10;  Conditional GANs" />
-          <attvalue for="1" value="  We present a new method for synthesizing high-resolution photo-realistic&#10;images from semantic label maps using conditional generative adversarial&#10;networks (conditional GANs). Conditional GANs have enabled a variety of&#10;applications, but the results are often limited to low-resolution and still far&#10;from realistic. In this work, we generate 2048x1024 visually appealing results&#10;with a novel adversarial loss, as well as new multi-scale generator and&#10;discriminator architectures. Furthermore, we extend our framework to&#10;interactive visual manipulation with two additional features. First, we&#10;incorporate object instance segmentation information, which enables object&#10;manipulations such as removing/adding objects and changing the object category.&#10;Second, we propose a method to generate diverse results given the same input,&#10;allowing users to edit the object appearance interactively. Human opinion&#10;studies demonstrate that our method significantly outperforms existing methods,&#10;advancing both the quality and the resolution of deep image synthesis and&#10;editing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.01215" label="1709.01215">
-        <attvalues>
-          <attvalue for="0" value="ALICE: Towards Understanding Adversarial Learning for Joint Distribution&#10;  Matching" />
-          <attvalue for="1" value="  We investigate the non-identifiability issues associated with bidirectional&#10;adversarial training for joint distribution matching. Within a framework of&#10;conditional entropy, we propose both adversarial and non-adversarial approaches&#10;to learn desirable matched joint distributions for unsupervised and supervised&#10;tasks. We unify a broad family of adversarial models as joint distribution&#10;matching problems. Our approach stabilizes learning of unsupervised&#10;bidirectional adversarial learning methods. Further, we introduce an extension&#10;for semi-supervised learning tasks. Theoretical results are validated in&#10;synthetic data and real-world applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.02510" label="1704.02510">
-        <attvalues>
-          <attvalue for="0" value="DualGAN: Unsupervised Dual Learning for Image-to-Image Translation" />
-          <attvalue for="1" value="  Conditional Generative Adversarial Networks (GANs) for cross-domain&#10;image-to-image translation have made much progress recently. Depending on the&#10;task complexity, thousands to millions of labeled image pairs are needed to&#10;train a conditional GAN. However, human labeling is expensive, even&#10;impractical, and large quantities of data may not always be available. Inspired&#10;by dual learning from natural language translation, we develop a novel dual-GAN&#10;mechanism, which enables image translators to be trained from two sets of&#10;unlabeled images from two domains. In our architecture, the primal GAN learns&#10;to translate images from domain U to those in domain V, while the dual GAN&#10;learns to invert the task. The closed loop made by the primal and dual tasks&#10;allows images from either domain to be translated and then reconstructed. Hence&#10;a loss function that accounts for the reconstruction error of images can be&#10;used to train the translators. Experiments on multiple image translation tasks&#10;with unlabeled data show considerable performance gain of DualGAN over a single&#10;GAN. For some tasks, DualGAN can even achieve comparable or slightly better&#10;results than conditional GAN trained on fully labeled data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.05192" label="1703.05192">
-        <attvalues>
-          <attvalue for="0" value="Learning to Discover Cross-Domain Relations with Generative Adversarial&#10;  Networks" />
-          <attvalue for="1" value="  While humans easily recognize relations between data from different domains&#10;without any supervision, learning to automatically discover them is in general&#10;very challenging and needs many ground-truth pairs that illustrate the&#10;relations. To avoid costly pairing, we address the task of discovering&#10;cross-domain relations given unpaired data. We propose a method based on&#10;generative adversarial networks that learns to discover relations between&#10;different domains (DiscoGAN). Using the discovered relations, our proposed&#10;network successfully transfers style from one domain to another while&#10;preserving key attributes such as orientation and face identity. Source code&#10;for official implementation is publicly available&#10;https://github.com/SKTBrain/DiscoGAN&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.02200" label="1611.02200">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Cross-Domain Image Generation" />
-          <attvalue for="1" value="  We study the problem of transferring a sample in one domain to an analog&#10;sample in another domain. Given two related domains, S and T, we would like to&#10;learn a generative function G that maps an input sample from S to the domain T,&#10;such that the output of a given function f, which accepts inputs in either&#10;domains, would remain unchanged. Other than the function f, the training data&#10;is unsupervised and consist of a set of samples from each domain. The Domain&#10;Transfer Network (DTN) we present employs a compound loss function that&#10;includes a multiclass GAN loss, an f-constancy component, and a regularizing&#10;component that encourages G to map samples from T to themselves. We apply our&#10;method to visual domains including digits and face images and demonstrate its&#10;ability to generate convincing novel images of previously unseen entities,&#10;while preserving their identity.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.00848" label="1703.00848">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Image-to-Image Translation Networks" />
-          <attvalue for="1" value="  Unsupervised image-to-image translation aims at learning a joint distribution&#10;of images in different domains by using images from the marginal distributions&#10;in individual domains. Since there exists an infinite set of joint&#10;distributions that can arrive the given marginal distributions, one could infer&#10;nothing about the joint distribution from the marginal distributions without&#10;additional assumptions. To address the problem, we make a shared-latent space&#10;assumption and propose an unsupervised image-to-image translation framework&#10;based on Coupled GANs. We compare the proposed framework with competing&#10;approaches and present high quality image translation results on various&#10;challenging unsupervised image translation tasks, including street scene image&#10;translation, animal image translation, and face image translation. We also&#10;apply the proposed framework to domain adaptation and achieve state-of-the-art&#10;performance on benchmark datasets. Code and additional results are available in&#10;https://github.com/mingyuliutw/unit .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.05139" label="1711.05139">
-        <attvalues>
-          <attvalue for="0" value="XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings" />
-          <attvalue for="1" value="  Style transfer usually refers to the task of applying color and texture&#10;information from a specific style image to a given content image while&#10;preserving the structure of the latter. Here we tackle the more generic problem&#10;of semantic style transfer: given two unpaired collections of images, we aim to&#10;learn a mapping between the corpus-level style of each collection, while&#10;preserving semantic content shared across the two domains. We introduce XGAN&#10;(&quot;Cross-GAN&quot;), a dual adversarial autoencoder, which captures a shared&#10;representation of the common domain semantic content in an unsupervised way,&#10;while jointly learning the domain-to-domain image translations in both&#10;directions. We exploit ideas from the domain adaptation literature and define a&#10;semantic consistency loss which encourages the model to preserve semantics in&#10;the learned embedding space. We report promising qualitative results for the&#10;task of face-to-cartoon translation. The cartoon dataset, CartoonSet, we&#10;collected for this purpose is publicly available at&#10;google.github.io/cartoonset/ as a new benchmark for semantic style transfer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.09020" label="1711.09020">
-        <attvalues>
-          <attvalue for="0" value="StarGAN: Unified Generative Adversarial Networks for Multi-Domain&#10;  Image-to-Image Translation" />
-          <attvalue for="1" value="  Recent studies have shown remarkable success in image-to-image translation&#10;for two domains. However, existing approaches have limited scalability and&#10;robustness in handling more than two domains, since different models should be&#10;built independently for every pair of image domains. To address this&#10;limitation, we propose StarGAN, a novel and scalable approach that can perform&#10;image-to-image translations for multiple domains using only a single model.&#10;Such a unified model architecture of StarGAN allows simultaneous training of&#10;multiple datasets with different domains within a single network. This leads to&#10;StarGAN's superior quality of translated images compared to existing models as&#10;well as the novel capability of flexibly translating an input image to any&#10;desired target domain. We empirically demonstrate the effectiveness of our&#10;approach on a facial attribute transfer and a facial expression synthesis&#10;tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.06909" label="1712.06909">
-        <attvalues>
-          <attvalue for="0" value="ComboGAN: Unrestrained Scalability for Image Domain Translation" />
-          <attvalue for="1" value="  This year alone has seen unprecedented leaps in the area of learning-based&#10;image translation, namely CycleGAN, by Zhu et al. But experiments so far have&#10;been tailored to merely two domains at a time, and scaling them to more would&#10;require an quadratic number of models to be trained. And with two-domain models&#10;taking days to train on current hardware, the number of domains quickly becomes&#10;limited by the time and resources required to process them. In this paper, we&#10;propose a multi-component image translation model and training scheme which&#10;scales linearly - both in resource consumption and time required - with the&#10;number of domains. We demonstrate its capabilities on a dataset of paintings by&#10;14 different artists and on images of the four different seasons in the Alps.&#10;Note that 14 data groups would need (14 choose 2) = 91 different CycleGAN&#10;models: a total of 182 generator/discriminator pairs; whereas our model&#10;requires only 14 generator/discriminator pairs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.03903" label="2309.03903">
-        <attvalues>
-          <attvalue for="0" value="Tracking Anything with Decoupled Video Segmentation" />
-          <attvalue for="1" value="  Training data for video segmentation are expensive to annotate. This impedes&#10;extensions of end-to-end algorithms to new video segmentation tasks, especially&#10;in large-vocabulary settings. To 'track anything' without training on video&#10;data for every individual task, we develop a decoupled video segmentation&#10;approach (DEVA), composed of task-specific image-level segmentation and&#10;class/task-agnostic bi-directional temporal propagation. Due to this design, we&#10;only need an image-level model for the target task (which is cheaper to train)&#10;and a universal temporal propagation model which is trained once and&#10;generalizes across tasks. To effectively combine these two modules, we use&#10;bi-directional propagation for (semi-)online fusion of segmentation hypotheses&#10;from different frames to generate a coherent segmentation. We show that this&#10;decoupled formulation compares favorably to end-to-end approaches in several&#10;data-scarce tasks including large-vocabulary video panoptic segmentation,&#10;open-world video segmentation, referring video segmentation, and unsupervised&#10;video object segmentation. Code is available at:&#10;https://hkchengrex.github.io/Tracking-Anything-with-DEVA&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;End-to-End Video Segmentation.&#10;Recent end-to-end video segmentation approaches~\cite{qiao2021vip,hwang2021video,wang2021end,bertasius2020classifying,cheng2021,ChoudhuriCVPR2023,ChoudhuriICCV2021} have made significant progress in tasks like Video Instance Segmentation (VIS) and Video Panoptic Segmentation (VPS), especially in closed and small vocabulary datasets like YouTube-VIS~\cite{yang2019video} and Cityscape-VPS~\cite{kim2020video}.&#10;However, these methods require end-to-end training and their scalability to larger vocabularies, where video data and annotations are expensive, is questionable. &#10;MaskProp~\cite{bertasius2020classifying} uses mask propagation to provide temporal information, but still needs to be trained end-to-end on the target task. This is because their mask propagation is not class-agnostic.&#10;We circumvent this training requirement and instead decouple the task into image segmentation and temporal propagation, each of which is easier to train with image-only data and readily available class-agnostic mask propagation data respectively. &#10;&#10;Open-World Video Segmentation.&#10;Recently, an open-world video segmentation dataset BURST~\cite{athar2023burst} has been proposed. &#10;It contains 482 object classes in diverse scenarios and evaluates open-world performance by computing metrics for the common classes (78, overlap with COCO~\cite{lin2014microsoft}) and uncommon classes (404) separately. &#10;The baseline in BURST~\cite{athar2023burst} predicts a set of object proposals using an image instance segmentation model trained on COCO~\cite{lin2014microsoft} and associates the proposals frame-by-frame using either box IoU or STCN~\cite{cheng2021stcn}. OWTB~\cite{liu2022opening} additionally associates proposals using optical flow and pre-trained Re-ID features. &#10;Differently, we use bi-directional propagation that generates segmentations instead of simply associating existing segmentations -- this reduces sensitivity to image segmentation errors.&#10;UVO~\cite{du2021uvo} is another open-world video segmentation dataset and focuses on human actions. We mainly evaluate on BURST~\cite{athar2023burst} as it is much more diverse and allows separate evaluation for common/uncommon classes.&#10;&#10;Decoupled Video Segmentation.&#10;`Tracking-by-detection' approaches~\cite{kim2015multiple,tang2017multiple,bergmann2019tracking} often consider image-level detections immutable and use a short-term temporal tracking model to associate detected objects. This formulation depends heavily on the quality of per-image detections and is sensitive to image-level errors.&#10;Related long-term temporal propagation works exist~\cite{goel2021msn,garg2021mask}, but they consider a single task and do not filter the image-level segmentation. We instead propose a general framework, with a bi-directional propagation mechanism that denoises the image segmentations and allows our result to potentially perform better than the image-level model.&#10;&#10;Video Object Segmentation.&#10;Semi-supervised Video Object Segmentation (VOS) aims to propagate an initial ground-truth segmentation through a video~\cite{perazzi2016benchmark,oh2019videoSTM,yang2021associating,cheng2022xmem}. &#10;However, it does not account for any errors in the initial segmentation, and cannot incorporate new segmentation given by the image model at later frames.&#10;SAM-PT~\cite{rajivc2023segment} combines point tracking with SAM~\cite{cheng2023segment} to create a video object segmentation pipeline, while our method tracks masks directly.&#10;We find a recent VOS algorithm~\cite{cheng2022xmem} works well for our temporal propagation model. Our proposed bi-directional propagation is essential for bringing image segmentation models and propagation models together as a unified video segmentation framework.&#10;&#10;Unified Video Segmentation.&#10;Recent Video-K-Net~\cite{li2022video} uses a unified framework for multiple video tasks but requires separate end-to-end training for each task. Unicorn~\cite{yan2022towards}, TarViS~\cite{athar2023tarvis}, and UNINEXT~\cite{yan2023universal} share model parameters for different tasks, and train on all the target tasks end-to-end. &#10;They report lower tracking accuracy for objects that are not in the target tasks during training compared with class-agnostic VOS approaches, which might be caused by joint learning with class-specific features.&#10;In contrast, we only train an image segmentation model for the target task, while the temporal propagation model is always fully class-agnostic for generalization across tasks.&#10;&#10;Segmenting/Tracking Anything.&#10;Concurrent to our work, Segment Anything (SAM)~\cite{kirillov2023segment} demonstrates the effectiveness and generalizability of large-scale training for universal image segmentation, serving as an important foundation for open-world segmentation. &#10;Follow-up works~\cite{yang2023track,cheng2023segment} extend SAM to video data by propagating the masks generated by SAM with video object segmentation algorithms.&#10;However, they rely on single-frame segmentation and lack the denoising capability of our proposed in-clip consensus approach.&#10; " />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2311.02815" label="2311.02815">
-        <attvalues>
-          <attvalue for="0" value="Efficient, Self-Supervised Human Pose Estimation with Inductive Prior&#10;  Tuning" />
-          <attvalue for="1" value="  The goal of 2D human pose estimation (HPE) is to localize anatomical&#10;landmarks, given an image of a person in a pose. SOTA techniques make use of&#10;thousands of labeled figures (finetuning transformers or training deep CNNs),&#10;acquired using labor-intensive crowdsourcing. On the other hand,&#10;self-supervised methods re-frame the HPE task as a reconstruction problem,&#10;enabling them to leverage the vast amount of unlabeled visual data, though at&#10;the present cost of accuracy. In this work, we explore ways to improve&#10;self-supervised HPE. We (1) analyze the relationship between reconstruction&#10;quality and pose estimation accuracy, (2) develop a model pipeline that&#10;outperforms the baseline which inspired our work, using less than one-third the&#10;amount of training data, and (3) offer a new metric suitable for&#10;self-supervised settings that measures the consistency of predicted body part&#10;length proportions. We show that a combination of well-engineered&#10;reconstruction losses and inductive priors can help coordinate pose learning&#10;alongside reconstruction in a self-supervised paradigm.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Applications of human pose estimation span a wide range, including predicting pedestrian behavior and trajectory on roads with autonomous vehicles \cite{bauer_2023_weakly, kress_2019_human, wang_leverage_2019, zanfir_2023_hum3dil, Zheng_2022_multimodal}. SOTA works \cite{geng_human_2023, xu_vitpose_2022} have explored larger models or tackled specific failure modes (\eg occlusion). Broadly speaking, some of the latest HPE models address one or more of the following five questions. (1) Can we speed up prediction to enable real-time pose estimation \cite{garau_capsulepose_2023, li_human_2022}? (2) Can we create lightweight models with smaller memory footprints \cite{li_human_2022, osokin_real-time_2018, santavas_attention_2021, xu_can_2023, zhang_simple_2020}? (3) Pose estimation models lack robustness under situation $X$. How can we address this \cite{geng_human_2023, yang_camerapose_2023, zhang_neuromorphic_2023}? (4) Can we use transformers \cite{einfalt_uplift_2023, qian_hstformer_2023, xu_vitpose_2022, zhao_dpit_2022}? (5) Vision-based pose estimation is unreliable. Can we use other more robust modes of data \cite{geng_densepose_2022, lee_hupr_2023}?&#10;&#10;An important caveat is that many of these methods require labor-intensive pose labeling, which limits scalability in training data. To illustrate, COCO's training and validation set alone contain 1.7 million labeled keypoints \cite{lin2014microsoft}. MPII Human Pose contains more than 600,000 \cite{andriluka_2d_2014}.&#10;&#10;To make use of the vast amount of unlabeled data, we look to self-supervised models, which frame classification and regression as reconstruction problems \cite{cao_-bed_2022, li_geometry-driven_2020, schmidtke_unsupervised_2021, wan_self-supervised_2019}, where given some parts of the input space (source), the model reconstructs other parts of the input space (target). These methods are engineered such that classification or regression are necessary to reason about the signals absent in the source but present in the target to be recovered. Hence, rather than optimizing directly for the task, self-supervision learns indirectly by optimizing for reconstruction, yielding representations that surpass the generalizability of those learned via supervised learning \cite{tendle_study_2021}. Self-supervised learning holds much potential for autonomous driving, facilitating greater scalability in training data, improving robustness, and enabling lifelong-learning \cite{berscheid2020self, deng2020self}. SOTA methods in this space typically rely on multi-view geometry \cite{bouazizi2021self, gholami2022self, Kocabas_2019_self, Wandt_2021_CVPR}, unpaired pose data \cite{Kundu_2020_self, sosa2023self}, or synthetic datasets with pose labels \cite{Kundu_2022_uncertainty}.&#10;&#10;Recently, Schmidtke et al.~\cite{schmidtke_unsupervised_2021} use a template-based, self-supervised approach to estimate pose in the Human3.6M dataset \cite{ionescu_human36m_2014}, without relying on multi-view geometry, unpaired pose data, or synthetic datasets with pose labels. By guiding the model with templates of Gaussians, each representing a body part, the model is able to render feasible pose estimates. However, without labels, there is no guarantee that predictions will reflect ground truth \cite{shu_feature-metric_2020}. &#10;&#10;In this work, we analyze the relationship between reconstruction optimization and pose estimate accuracy in the absence of ground truth, using a carefully engineered combination of reconstruction loss and inductive prior (in the form of a new template). Using these insights, we develop an efficient model pipeline that exceeds the baseline \cite{schmidtke_unsupervised_2021} performance, using a training set that is less than one-third the size of the original, along with a new formulation of matrix transformation that implements coarse-to-fine learning and data augmentation. Further, we propose a metric of consistency for body part lengths that is suitable for self-supervised settings, where ground truth is not available. Our code is available at.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Deep Learning Techniques, Human Pose Estimation, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2307.14889" label="2307.14889">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for&#10;  Autonomous Driving" />
-          <attvalue for="1" value="  Accurate 3D human pose estimation (3D HPE) is crucial for enabling autonomous&#10;vehicles (AVs) to make informed decisions and respond proactively in critical&#10;road scenarios. Promising results of 3D HPE have been gained in several domains&#10;such as human-computer interaction, robotics, sports and medical analytics,&#10;often based on data collected in well-controlled laboratory environments.&#10;Nevertheless, the transfer of 3D HPE methods to AVs has received limited&#10;research attention, due to the challenges posed by obtaining accurate 3D pose&#10;annotations and the limited suitability of data from other domains.&#10;  We present a simple yet efficient weakly supervised approach for 3D HPE in&#10;the AV context by employing a high-level sensor fusion between camera and LiDAR&#10;data. The weakly supervised setting enables training on the target datasets&#10;without any 2D/3D keypoint labels by using an off-the-shelf 2D joint extractor&#10;and pseudo labels generated from LiDAR to image projections. Our approach&#10;outperforms state-of-the-art results by up to $\sim$ 13% on the Waymo Open&#10;Dataset in the weakly supervised setting and achieves state-of-the-art results&#10;in the supervised setting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.06110" label="2210.06110">
-        <attvalues>
-          <attvalue for="0" value="Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting&#10;  Transformers" />
-          <attvalue for="1" value="  The state-of-the-art for monocular 3D human pose estimation in videos is&#10;dominated by the paradigm of 2D-to-3D pose uplifting. While the uplifting&#10;methods themselves are rather efficient, the true computational complexity&#10;depends on the per-frame 2D pose estimation. In this paper, we present a&#10;Transformer-based pose uplifting scheme that can operate on temporally sparse&#10;2D pose sequences but still produce temporally dense 3D pose estimates. We show&#10;how masked token modeling can be utilized for temporal upsampling within&#10;Transformer blocks. This allows to decouple the sampling rate of input 2D poses&#10;and the target frame rate of the video and drastically decreases the total&#10;computational complexity. Additionally, we explore the option of pre-training&#10;on large motion capture archives, which has been largely neglected so far. We&#10;evaluate our method on two popular benchmark datasets: Human3.6M and&#10;MPI-INF-3DHP. With an MPJPE of 45.0 mm and 46.9 mm, respectively, our proposed&#10;method can compete with the state-of-the-art while reducing inference time by a&#10;factor of 12. This enables real-time throughput with variable consumer hardware&#10;in stationary and mobile applications. We release our code and models at&#10;https://github.com/goldbricklemon/uplift-upsample-3dhpe&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.00250" label="2301.00250">
-        <attvalues>
-          <attvalue for="0" value="DensePose From WiFi" />
-          <attvalue for="1" value="  Advances in computer vision and machine learning techniques have led to&#10;significant development in 2D and 3D human pose estimation from RGB cameras,&#10;LiDAR, and radars. However, human pose estimation from images is adversely&#10;affected by occlusion and lighting, which are common in many scenarios of&#10;interest. Radar and LiDAR technologies, on the other hand, need specialized&#10;hardware that is expensive and power-intensive. Furthermore, placing these&#10;sensors in non-public areas raises significant privacy concerns. To address&#10;these limitations, recent research has explored the use of WiFi antennas (1D&#10;sensors) for body segmentation and key-point body detection. This paper further&#10;expands on the use of the WiFi signal in combination with deep learning&#10;architectures, commonly used in computer vision, to estimate dense human pose&#10;correspondence. We developed a deep neural network that maps the phase and&#10;amplitude of WiFi signals to UV coordinates within 24 human regions. The&#10;results of the study reveal that our model can estimate the dense pose of&#10;multiple subjects, with comparable performance to image-based approaches, by&#10;utilizing WiFi signals as the only input. This paves the way for low-cost,&#10;broadly accessible, and privacy-preserving algorithms for human sensing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.15124" label="2111.15124">
-        <attvalues>
-          <attvalue for="0" value="In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image&#10;  Domains" />
-          <attvalue for="1" value="  Medical applications have benefited greatly from the rapid advancement in&#10;computer vision. Considering patient monitoring in particular, in-bed human&#10;posture estimation offers important health-related metrics with potential value&#10;in medical condition assessments. Despite great progress in this domain, it&#10;remains challenging due to substantial ambiguity during occlusions, and the&#10;lack of large corpora of manually labeled data for model training, particularly&#10;with domains such as thermal infrared imaging which are privacy-preserving, and&#10;thus of great interest. Motivated by the effectiveness of self-supervised&#10;methods in learning features directly from data, we propose a multi-modal&#10;conditional variational autoencoder (MC-VAE) capable of reconstructing features&#10;from missing modalities seen during training. This approach is used with HRNet&#10;to enable single modality inference for in-bed pose estimation. Through&#10;extensive evaluations, we demonstrate that body positions can be effectively&#10;recognized from the available modality, achieving on par results with baseline&#10;models that are highly dependent on having access to multiple modes at&#10;inference time. The proposed framework supports future research towards&#10;self-supervised learning that generates a robust model from a single source,&#10;and expects it to generalize over many unknown distributions in clinical&#10;environments.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.09150" label="2109.09150">
-        <attvalues>
-          <attvalue for="0" value="A Study of the Generalizability of Self-Supervised Representations" />
-          <attvalue for="1" value="  Recent advancements in self-supervised learning (SSL) made it possible to&#10;learn generalizable visual representations from unlabeled data. The performance&#10;of Deep Learning models fine-tuned on pretrained SSL representations is on par&#10;with models fine-tuned on the state-of-the-art supervised learning (SL)&#10;representations. Irrespective of the progress made in SSL, its generalizability&#10;has not been studied extensively. In this article, we perform a deeper analysis&#10;of the generalizability of pretrained SSL and SL representations by conducting&#10;a domain-based study for transfer learning classification tasks. The&#10;representations are learned from the ImageNet source data, which are then&#10;fine-tuned using two types of target datasets: similar to the source dataset,&#10;and significantly different from the source dataset. We study generalizability&#10;of the SSL and SL-based models via their prediction accuracy as well as&#10;prediction confidence. In addition to this, we analyze the attribution of the&#10;final convolutional layer of these models to understand how they reason about&#10;the semantic identity of the data. We show that the SSL representations are&#10;more generalizable as compared to the SL representations. We explain the&#10;generalizability of the SSL representations by investigating its invariance&#10;property, which is shown to be better than that observed in the SL&#10;representations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.08373" label="2006.08373">
-        <attvalues>
-          <attvalue for="0" value="Self-supervised Learning for Precise Pick-and-place without Object Model" />
-          <attvalue for="1" value="  Flexible pick-and-place is a fundamental yet challenging task within&#10;robotics, in particular due to the need of an object model for a simple target&#10;pose definition. In this work, the robot instead learns to pick-and-place&#10;objects using planar manipulation according to a single, demonstrated goal&#10;state. Our primary contribution lies within combining robot learning of&#10;primitives, commonly estimated by fully-convolutional neural networks, with&#10;one-shot imitation learning. Therefore, we define the place reward as a&#10;contrastive loss between real-world measurements and a task-specific noise&#10;distribution. Furthermore, we design our system to learn in a self-supervised&#10;manner, enabling real-world experiments with up to 25000 pick-and-place&#10;actions. Then, our robot is able to place trained objects with an average&#10;placement error of 2.7 (0.2) mm and 2.6 (0.8){\deg}. As our approach does not&#10;require an object model, the robot is able to generalize to unknown objects&#10;while keeping a precision of 5.9 (1.1) mm and 4.1 (1.2){\deg}. We further show&#10;a range of emerging behaviors: The robot naturally learns to select the correct&#10;object in the presence of multiple object types, precisely inserts objects&#10;within a peg game, picks screws out of dense clutter, and infers multiple&#10;pick-and-place actions from a single goal state.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.07777" label="2108.07777">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry" />
-          <attvalue for="1" value="  We present a self-supervised learning algorithm for 3D human pose estimation&#10;of a single person based on a multiple-view camera system and 2D body pose&#10;estimates for each view. To train our model, represented by a deep neural&#10;network, we propose a four-loss function learning algorithm, which does not&#10;require any 2D or 3D body pose ground-truth. The proposed loss functions make&#10;use of the multiple-view geometry to reconstruct 3D body pose estimates and&#10;impose body pose constraints across the camera views. Our approach utilizes all&#10;available camera views during training, while the inference is single-view. In&#10;our evaluations, we show promising performance on Human3.6M and HumanEva&#10;benchmarks, while we also present a generalization study on MPI-INF-3DHP&#10;dataset, as well as several ablation results. Overall, we outperform all&#10;self-supervised learning methods and reach comparable results to supervised and&#10;weakly-supervised learning approaches. Our code and models are publicly&#10;available&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.15293" label="2203.15293">
-        <attvalues>
-          <attvalue for="0" value="Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose&#10;  Estimation" />
-          <attvalue for="1" value="  The advances in monocular 3D human pose estimation are dominated by&#10;supervised techniques that require large-scale 2D/3D pose annotations. Such&#10;methods often behave erratically in the absence of any provision to discard&#10;unfamiliar out-of-distribution data. To this end, we cast the 3D human pose&#10;learning as an unsupervised domain adaptation problem. We introduce MRP-Net&#10;that constitutes a common deep network backbone with two output heads&#10;subscribing to two diverse configurations; a) model-free joint localization and&#10;b) model-based parametric regression. Such a design allows us to derive&#10;suitable measures to quantify prediction uncertainty at both pose and joint&#10;level granularity. While supervising only on labeled synthetic samples, the&#10;adaptation process aims to minimize the uncertainty for the unlabeled target&#10;images while maximizing the same for an extreme out-of-distribution dataset&#10;(backgrounds). Alongside synthetic-to-real 3D pose adaptation, the&#10;joint-uncertainties allow expanding the adaptation to work on in-the-wild&#10;images even in the presence of occlusion and truncation scenarios. We present a&#10;comprehensive evaluation of the proposed approach and demonstrate&#10;state-of-the-art performance on benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.04154" label="2105.04154">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Human Pose Estimation through Transforming Shape Templates" />
-          <attvalue for="1" value="  Human pose estimation is a major computer vision problem with applications&#10;ranging from augmented reality and video capture to surveillance and movement&#10;tracking. In the medical context, the latter may be an important biomarker for&#10;neurological impairments in infants. Whilst many methods exist, their&#10;application has been limited by the need for well annotated large datasets and&#10;the inability to generalize to humans of different shapes and body&#10;compositions, e.g. children and infants. In this paper we present a novel&#10;method for learning pose estimators for human adults and infants in an&#10;unsupervised fashion. We approach this as a learnable template matching problem&#10;facilitated by deep feature extractors. Human-interpretable landmarks are&#10;estimated by transforming a template consisting of predefined body parts that&#10;are characterized by 2D Gaussian distributions. Enforcing a connectivity prior&#10;guides our model to meaningful human shape representations. We demonstrate the&#10;effectiveness of our approach on two different datasets including adults and&#10;infants.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.18092" label="2403.18092">
-        <attvalues>
-          <attvalue for="0" value="OCAI: Improving Optical Flow Estimation by Occlusion and Consistency&#10;  Aware Interpolation" />
-          <attvalue for="1" value="  The scarcity of ground-truth labels poses one major challenge in developing&#10;optical flow estimation models that are both generalizable and robust. While&#10;current methods rely on data augmentation, they have yet to fully exploit the&#10;rich information available in labeled video sequences. We propose OCAI, a&#10;method that supports robust frame interpolation by generating intermediate&#10;video frames alongside optical flows in between. Utilizing a forward warping&#10;approach, OCAI employs occlusion awareness to resolve ambiguities in pixel&#10;values and fills in missing values by leveraging the forward-backward&#10;consistency of optical flows. Additionally, we introduce a teacher-student&#10;style semi-supervised learning method on top of the interpolated frames. Using&#10;a pair of unlabeled frames and the teacher model's predicted optical flow, we&#10;generate interpolated frames and flows to train a student model. The teacher's&#10;weights are maintained using Exponential Moving Averaging of the student. Our&#10;evaluations demonstrate perceptually superior interpolation quality and&#10;enhanced optical flow accuracy on established benchmarks such as Sintel and&#10;KITTI.&#10;" />
-          <attvalue for="2" value="&#10;&#10;&#10;&#10;Optical flow estimation and Video Frame Interpolation (VFI) share a complementary relationship. Accurate optical flow contributes significantly to various downstream tasks such as video compression~\cite{wu2018video, lu2019dvc}, video denoising and blur removal~\cite{buades2016patch, yuan2020efficient, zhang2021deep}, action recognition~\cite{lee2018motion, cai2019temporal}, and VFI stands as one of these applications. Pixel-level correspondence by optical flow enables estimating pixel-level movement and generating intermediate frames (or inter-frames). While utilizing flow-based methods is a common practice in VFI~\cite{kong2022ifrnet, lu2022video, huang2022real, li2023amt}, leveraging inter-frames to train optical flow models is relatively less explored.&#10;&#10;While the scarcity of ground-truth data has long been a critical challenge in learning-based optical flow estimation~\cite{dosovitskiy2015flownet, ilg2017flownet, ranjan2017optical, sun2018pwc, teed2020raft, huang2022flowformer}, there has been little attention into leveraging Video Frame Interpolation (VFI) to augment the training of optical flow networks. State-of-the-art VFI models predominantly employ deep networks trained to interpolate the exact middle frame between two consecutive time steps within a video sequence, which restricts their capability to generate frames at other intermediate time instances thus hampering their ability to produce optical flows between existing and intermediate frames reliably. Besides, these models lack generalizability across new domains without necessitating finetuning or retraining. These limitations hinder the potential use of existing VFI models for augmenting the training data for optical flow models.&#10;&#10;To address data scarcity in optical flow training: data augmentation \cite{sun2021autoflow, jeong2023distractflow}, data generation \cite{dosovitskiy2015flownet, sun2021autoflow, han2022realflow}, and semi-supervised learning \cite{jeong2022imposing, im2022semi, han2022realflow,jeong2023distractflow} have been explored. Most of the current data augmentation methods prioritize modifying the existing frames, e.g.,~\cite{jeong2023distractflow}, as illustrated in Fig.~\ref{fig:teaser} (top). Notably, RealFlow\cite{han2022realflow} stands out by synthesizing a new second frame via forward warping. Given a pair of frames and the model's prediction, the original first and second frames form a new training pair, used to update model weights. Model prediction, frame synthesis, and model update are iterated in an Expectation-Maximization (EM) framework, as depicted in Fig.~\ref{fig:teaser} (middle). While it is possible to use forward warping to interpolate frames, RealFlow focuses only on making the model's prediction consistent with frame synthesis. Moreover, the EM steps significantly increase the training workload.&#10;&#10;In this paper, we introduce {OCAI}, a novel approach for training optical flow networks within a semi-supervised framework using readily available unlabeled pairs. Our method leverages video frames and flow interpolation to achieve this goal. {OCAI} implements an occlusion-aware forward warping technique, facilitating the interpolation of both inter-frames and intermediate flows, and can effectively tackle pixel value ambiguities during the warping process without needing any depth information and can do confidence-aware estimation of missing values utilizing forward-backward consistency. Our algorithm can perform interpolation at any intermediate time step, thus offering a broad diversity of data and motion ranges essential for training optical flow models. Our innovative teacher-student style semi-supervised learning scheme utilizes these resulting inter-frames to train optical flow networks effectively.&#10;&#10;In summary, our main contributions are as follows:&#10;\begin{itemize}&#10;&#10;\item We propose a novel approach, {OCAI}, that tackles the data scarcity challenge in training optical flow models, by exploiting useful, hidden information in existing videos. Specifically, we interpolating frames and flows to generate supplementary data, and use them in a semi-supervised learning framework.&#10;&#10;\item To do this, we propose a new, effective video interpolation method that derives occlusion information to address ambiguous pixels and fill in holes by exploiting optical flow consistency. Our algorithm flexibly generates high-quality intermediate frames and reliable intermediate flows along with corresponding confidence maps at any intermediate time step.&#10;&#10;\item We devise a new teacher-student semi-supervised learning strategy leveraging VFI to train an optical flow network, incorporating exponential moving averaging (EMA) to enhance training stability.&#10;&#10;\item &#10;We demonstrate that {OCAI} achieves higher quality in video interpolation than existing SOTA methods on standard datasets including Sintel and KITTI.&#10;By incorporating interpolated video information, our semi-supervised learning scheme significantly improves the optical flow estimation performance, e.g., 0.5+ Fl-all reduction on KITTI test set when comparing to latest SOTA.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Optical Flow Estimation, Video Frame Interpolation, Computer Science, Semi-Supervised Learning, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1804.06919" label="1804.06919">
-        <attvalues>
-          <attvalue for="0" value="Video Compression through Image Interpolation" />
-          <attvalue for="1" value="  An ever increasing amount of our digital communication, media consumption,&#10;and content creation revolves around videos. We share, watch, and archive many&#10;aspects of our lives through them, all of which are powered by strong video&#10;compression. Traditional video compression is laboriously hand designed and&#10;hand optimized. This paper presents an alternative in an end-to-end deep&#10;learning codec. Our codec builds on one simple idea: Video compression is&#10;repeated image interpolation. It thus benefits from recent advances in deep&#10;image interpolation and generation. Our deep video codec outperforms today's&#10;prevailing codecs, such as H.261, MPEG-4 Part 2, and performs on par with&#10;H.264.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.10037" label="1807.10037">
-        <attvalues>
-          <attvalue for="0" value="Motion Feature Network: Fixed Motion Filter for Action Recognition" />
-          <attvalue for="1" value="  Spatio-temporal representations in frame sequences play an important role in&#10;the task of action recognition. Previously, a method of using optical flow as a&#10;temporal information in combination with a set of RGB images that contain&#10;spatial information has shown great performance enhancement in the action&#10;recognition tasks. However, it has an expensive computational cost and requires&#10;two-stream (RGB and optical flow) framework. In this paper, we propose MFNet&#10;(Motion Feature Network) containing motion blocks which make it possible to&#10;encode spatio-temporal information between adjacent frames in a unified network&#10;that can be trained end-to-end. The motion block can be attached to any&#10;existing CNN-based action recognition frameworks with only a small additional&#10;cost. We evaluated our network on two of the action recognition datasets&#10;(Jester and Something-Something) and achieved competitive performances for both&#10;datasets by training the networks from scratch.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.14620" label="2205.14620">
-        <attvalues>
-          <attvalue for="0" value="IFRNet: Intermediate Feature Refine Network for Efficient Frame&#10;  Interpolation" />
-          <attvalue for="1" value="  Prevailing video frame interpolation algorithms, that generate the&#10;intermediate frames from consecutive inputs, typically rely on complex model&#10;architectures with heavy parameters or large delay, hindering them from diverse&#10;real-time applications. In this work, we devise an efficient encoder-decoder&#10;based network, termed IFRNet, for fast intermediate frame synthesizing. It&#10;first extracts pyramid features from given inputs, and then refines the&#10;bilateral intermediate flow fields together with a powerful intermediate&#10;feature until generating the desired output. The gradually refined intermediate&#10;feature can not only facilitate intermediate flow estimation, but also&#10;compensate for contextual details, making IFRNet do not need additional&#10;synthesis or refinement module. To fully release its potential, we further&#10;propose a novel task-oriented optical flow distillation loss to focus on&#10;learning the useful teacher knowledge towards frame synthesizing. Meanwhile, a&#10;new geometry consistency regularization term is imposed on the gradually&#10;refined intermediate features to keep better structure layout. Experiments on&#10;various benchmarks demonstrate the excellent performance and fast inference&#10;speed of proposed approaches. Code is available at&#10;https://github.com/ltkong218/IFRNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14544" label="2104.14544">
-        <attvalues>
-          <attvalue for="0" value="AutoFlow: Learning a Better Training Set for Optical Flow" />
-          <attvalue for="1" value="  Synthetic datasets play a critical role in pre-training CNN models for&#10;optical flow, but they are painstaking to generate and hard to adapt to new&#10;applications. To automate the process, we present AutoFlow, a simple and&#10;effective method to render training data for optical flow that optimizes the&#10;performance of a model on a target dataset. AutoFlow takes a layered approach&#10;to render synthetic data, where the motion, shape, and appearance of each layer&#10;are controlled by learnable hyperparameters. Experimental results show that&#10;AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and&#10;RAFT. Our code and data are available at https://autoflow-google.github.io .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.14078" label="2303.14078">
-        <attvalues>
-          <attvalue for="0" value="DistractFlow: Improving Optical Flow Estimation via Realistic&#10;  Distractions and Pseudo-Labeling" />
-          <attvalue for="1" value="  We propose a novel data augmentation approach, DistractFlow, for training&#10;optical flow estimation models by introducing realistic distractions to the&#10;input frames. Based on a mixing ratio, we combine one of the frames in the pair&#10;with a distractor image depicting a similar domain, which allows for inducing&#10;visual perturbations congruent with natural objects and scenes. We refer to&#10;such pairs as distracted pairs. Our intuition is that using semantically&#10;meaningful distractors enables the model to learn related variations and attain&#10;robustness against challenging deviations, compared to conventional&#10;augmentation schemes focusing only on low-level aspects and modifications. More&#10;specifically, in addition to the supervised loss computed between the estimated&#10;flow for the original pair and its ground-truth flow, we include a second&#10;supervised loss defined between the distracted pair's flow and the original&#10;pair's ground-truth flow, weighted with the same mixing ratio. Furthermore,&#10;when unlabeled data is available, we extend our augmentation approach to&#10;self-supervised settings through pseudo-labeling and cross-consistency&#10;regularization. Given an original pair and its distracted version, we enforce&#10;the estimated flow on the distracted pair to agree with the flow of the&#10;original pair. Our approach allows increasing the number of available training&#10;pairs significantly without requiring additional annotations. It is agnostic to&#10;the model architecture and can be applied to training any optical flow&#10;estimation models. Our extensive evaluations on multiple benchmarks, including&#10;Sintel, KITTI, and SlowFlow, show that DistractFlow improves existing models&#10;consistently, outperforming the latest state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.08192" label="2406.08192">
-        <attvalues>
-          <attvalue for="0" value="2nd Place Solution for MOSE Track in CVPR 2024 PVUW workshop: Complex&#10;  Video Object Segmentation" />
-          <attvalue for="1" value="  Complex video object segmentation serves as a fundamental task for a wide&#10;range of downstream applications such as video editing and automatic data&#10;annotation. Here we present the 2nd place solution in the MOSE track of PVUW&#10;2024. To mitigate problems caused by tiny objects, similar objects and fast&#10;movements in MOSE. We use instance segmentation to generate extra pretraining&#10;data from the valid and test set of MOSE. The segmented instances are combined&#10;with objects extracted from COCO to augment the training data and enhance&#10;semantic representation of the baseline model. Besides, motion blur is added&#10;during training to increase robustness against image blur induced by motion.&#10;Finally, we apply test time augmentation (TTA) and memory strategy to the&#10;inference stage. Our method ranked 2nd in the MOSE track of PVUW 2024, with a&#10;$\mathcal{J}$ of 0.8007, a $\mathcal{F}$ of 0.8683 and a&#10;$\mathcal{J}$\&amp;$\mathcal{F}$ of 0.8345.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. The pixel-level Video Understanding in the Wild Challenge (PVUW) shop challenge advances the segmentation task from images to videos, aiming at enabling challenging and practical realistic applications. The PVUW 2024 workshop challenge includes two new tracks, Complex Video Object Segmentation Track based on MOSE\cite{r9} and Motion Expression guided Video Segmentation track based on MeViS\cite{ding2023mevis}. The Complex Video Object Segmentation Track focuses on semi-supervised video object segmentation (VOS) under complex environments. As an important branch of the VOS task, semi-supervised VOS aims at tracking and segmenting agnostic objects given only the first-frame annotations, which has been widely applied in autonomous driving\cite{r1}, video editing\cite{r2}, automatic data annotation\cite{r3}, and universal video segmentation\cite{r4}.&#10;&#10;Recent memory-based approaches have become the main stream data driven VOS methods. Memory-based approaches store past segmented frames in a memory bank, when a new query frame comes, it will read from the memory bank through cross attention, which is more robust to drifting and occlusions. Due to the advantages over other VOS methods\cite{other1,other2,other3,other4}, the memory-based paradigm has been paid much attention by the research community. As one of the most successful early attempts, Space-Time Memory network (STM)\cite{r5} stores the past frames with object masks into the memory and performs pixel-level matching between the encoded key of query frame and memory. &#10;&#10;Space-Time Correspondence Network (STCN)\cite{r6} is developed from STM, it encodes the key features from frames without masks and replace dot product with L2 similarity in the affinity for memory reading. STCN achieves better efficiency and effectiveness than STM. XMem\cite{r7} introduces three indendent memory banks: a sensory memory, a working memory and a long-term memory. The three-level memories are inspired by the Atkinson–Shiffrin memory model of Human. Xmem performs especially well on long-video datasets because of the short-term to long-term memories. To solve the mismatch problem in pixel-level matching, Cutie\cite{r8} proposes the object memory and object transformer for bidirectional information interaction. The object memory and object transformer improve robustness in challenging scenes with heavy occlusions and similarity. Due to the state-of-the-art performance of Cutie, we choose Cutie as our baseline model. &#10;&#10;However, the challenging nature of MOSE\cite{r9} still &#10;&#10;poses several obstacles to overcome. Apart from higher disappearance-reappearance rates and heavier occlusions than previous VOS datasets such as DAVIS\cite{r10} and YouTubeVOS\cite{r11}, MOSE has many tiny and similar objects, which may confuse VOS models. Besides, some videos in MOSE contain objects with fast movements, which are hard to track in consecutive frames. &#10;&#10;To mitigate the above issues, we combine Mask2Former\cite{r12} and motion blur as data augmentation to enhance semantic representation during the training process. We exploit images from the valid and test set of MOSE, and use Mask2Former to segment instance masks and generate pretraining data for Cutie. We also introduce COCO\cite{r13} to further enrich the pretraining data, with the purpose of enhancing semantic learning in the early stage of training and improve the segmentation accuracy of tiny and similar objects. At inference time, we employ test time augmentation (TTA) and memory strategy to optimize the results. Our solution reaches a $\mathcal{J}$ of 0.8007, a $\mathcal{F}$ of 0.8683 and a $\mathcal{J}$\&amp;$\mathcal{F}$ of 0.8345, which achieved the 2nd place in the MOSE track of the PVUW challenge in CVPR 2024. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Information Technology, Artificial Intelligence, Computer Science, Deep Learning Techniques, Video Object Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2308.08544" label="2308.08544">
-        <attvalues>
-          <attvalue for="0" value="MeViS: A Large-scale Benchmark for Video Segmentation with Motion&#10;  Expressions" />
-          <attvalue for="1" value="  This paper strives for motion expressions guided video segmentation, which&#10;focuses on segmenting objects in video content based on a sentence describing&#10;the motion of the objects. Existing referring video object datasets typically&#10;focus on salient objects and use language expressions that contain excessive&#10;static attributes that could potentially enable the target object to be&#10;identified in a single frame. These datasets downplay the importance of motion&#10;in video content for language-guided video object segmentation. To investigate&#10;the feasibility of using motion expressions to ground and segment objects in&#10;videos, we propose a large-scale dataset called MeViS, which contains numerous&#10;motion expressions to indicate target objects in complex environments. We&#10;benchmarked 5 existing referring video object segmentation (RVOS) methods and&#10;conducted a comprehensive comparison on the MeViS dataset. The results show&#10;that current RVOS methods cannot effectively address motion expression-guided&#10;video segmentation. We further analyze the challenges and propose a baseline&#10;approach for the proposed MeViS dataset. The goal of our benchmark is to&#10;provide a platform that enables the development of effective language-guided&#10;video segmentation algorithms that leverage motion expressions as a primary cue&#10;for object segmentation in complex video scenes. The proposed MeViS dataset has&#10;been released at https://henghuiding.github.io/MeViS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1512.06735" label="1512.06735">
-        <attvalues>
-          <attvalue for="0" value="Instance-Level Segmentation for Autonomous Driving with Deep Densely&#10;  Connected MRFs" />
-          <attvalue for="1" value="  Our aim is to provide a pixel-wise instance-level labeling of a monocular&#10;image in the context of autonomous driving. We build on recent work [Zhang et&#10;al., ICCV15] that trained a convolutional neural net to predict instance&#10;labeling in local image patches, extracted exhaustively in a stride from an&#10;image. A simple Markov random field model using several heuristics was then&#10;proposed in [Zhang et al., ICCV15] to derive a globally consistent instance&#10;labeling of the image. In this paper, we formulate the global labeling problem&#10;with a novel densely connected Markov random field and show how to encode&#10;various intuitive potentials in a way that is amenable to efficient mean field&#10;inference [Kr\&quot;ahenb\&quot;uhl et al., NIPS11]. Our potentials encode the&#10;compatibility between the global labeling and the patch-level predictions,&#10;contrast-sensitive smoothness as well as the fact that separate regions form&#10;different instances. Our experiments on the challenging KITTI benchmark [Geiger&#10;et al., CVPR12] demonstrate that our method achieves a significant performance&#10;boost over the baseline [Zhang et al., ICCV15].&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.07941" label="2103.07941">
-        <attvalues>
-          <attvalue for="0" value="Modular Interactive Video Object Segmentation: Interaction-to-Mask,&#10;  Propagation and Difference-Aware Fusion" />
-          <attvalue for="1" value="  We present Modular interactive VOS (MiVOS) framework which decouples&#10;interaction-to-mask and mask propagation, allowing for higher generalizability&#10;and better performance. Trained separately, the interaction module converts&#10;user interactions to an object mask, which is then temporally propagated by our&#10;propagation module using a novel top-$k$ filtering strategy in reading the&#10;space-time memory. To effectively take the user's intent into account, a novel&#10;difference-aware module is proposed to learn how to properly fuse the masks&#10;before and after each interaction, which are aligned with the target frames by&#10;employing the space-time memory. We evaluate our method both qualitatively and&#10;quantitatively with different forms of user interactions (e.g., scribbles,&#10;clicks) on DAVIS to show that our method outperforms current state-of-the-art&#10;algorithms while requiring fewer frame interactions, with the additional&#10;advantage in generalizing to different types of user interactions. We&#10;contribute a large-scale synthetic VOS dataset with pixel-accurate segmentation&#10;of 4.8M frames to accompany our source codes to facilitate future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.12118" label="2209.12118">
-        <attvalues>
-          <attvalue for="0" value="BURST: A Benchmark for Unifying Object Recognition, Segmentation and&#10;  Tracking in Video" />
-          <attvalue for="1" value="  Multiple existing benchmarks involve tracking and segmenting objects in video&#10;e.g., Video Object Segmentation (VOS) and Multi-Object Tracking and&#10;Segmentation (MOTS), but there is little interaction between them due to the&#10;use of disparate benchmark datasets and metrics (e.g. J&amp;F, mAP, sMOTSA). As a&#10;result, published works usually target a particular benchmark, and are not&#10;easily comparable to each another. We believe that the development of&#10;generalized methods that can tackle multiple tasks requires greater cohesion&#10;among these research sub-communities. In this paper, we aim to facilitate this&#10;by proposing BURST, a dataset which contains thousands of diverse videos with&#10;high-quality object masks, and an associated benchmark with six tasks involving&#10;object tracking and segmentation in video. All tasks are evaluated using the&#10;same data and comparable metrics, which enables researchers to consider them in&#10;unison, and hence, more effectively pool knowledge from different methods&#10;across different tasks. Additionally, we demonstrate several baselines for all&#10;tasks and show that approaches for one task can be applied to another with a&#10;quantifiable and explainable performance difference. Dataset annotations and&#10;evaluation code is available at: https://github.com/Ali2500/BURST-benchmark.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.12982" label="2310.12982">
-        <attvalues>
-          <attvalue for="0" value="Putting the Object Back into Video Object Segmentation" />
-          <attvalue for="1" value="  We present Cutie, a video object segmentation (VOS) network with object-level&#10;memory reading, which puts the object representation from memory back into the&#10;video object segmentation result. Recent works on VOS employ bottom-up&#10;pixel-level memory reading which struggles due to matching noise, especially in&#10;the presence of distractors, resulting in lower performance in more challenging&#10;data. In contrast, Cutie performs top-down object-level memory reading by&#10;adapting a small set of object queries. Via those, it interacts with the&#10;bottom-up pixel features iteratively with a query-based object transformer (qt,&#10;hence Cutie). The object queries act as a high-level summary of the target&#10;object, while high-resolution feature maps are retained for accurate&#10;segmentation. Together with foreground-background masked attention, Cutie&#10;cleanly separates the semantics of the foreground object from the background.&#10;On the challenging MOSE dataset, Cutie improves by 8.7 J&amp;F over XMem with a&#10;similar running time and improves by 4.2 J&amp;F over DeAOT while being three times&#10;faster. Code is available at: https://hkchengrex.github.io/Cutie&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.08936" label="1912.08936">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Weakly Supervised Video Object Segmentation" />
-          <attvalue for="1" value="  Conventional few-shot object segmentation methods learn object segmentation&#10;from a few labelled support images with strongly labelled segmentation masks.&#10;Recent work has shown to perform on par with weaker levels of supervision in&#10;terms of scribbles and bounding boxes. However, there has been limited&#10;attention given to the problem of few-shot object segmentation with image-level&#10;supervision. We propose a novel multi-modal interaction module for few-shot&#10;object segmentation that utilizes a co-attention mechanism using both visual&#10;and word embeddings. It enables our model to achieve 5.1% improvement over&#10;previously proposed image-level few-shot object segmentation. Our method&#10;compares relatively close to the state of the art methods that use strong&#10;supervision, while ours use the least possible supervision. We further propose&#10;a novel setup for few-shot weakly supervised video object segmentation(VOS)&#10;that relies on image-level labels for the first frame. The proposed setup uses&#10;weak annotation unlike semi-supervised VOS setting that utilizes strongly&#10;labelled segmentation masks. The setup evaluates the effectiveness of&#10;generalizing to novel classes in the VOS setting. The setup splits the VOS data&#10;into multiple folds with different categories per fold. It provides a potential&#10;setup to evaluate how few-shot object segmentation methods can benefit from&#10;additional object poses, or object interactions that is not available in static&#10;frames as in PASCAL-5i benchmark.&#10;" />
-          <attvalue for="2" value="&#10;The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}. Recently, solutions for few-shot object segmentation which learns pixel-wise classification based on few labelled samples have emerged~\cite{shaban2017one}\cite{rakelly2018conditional}\cite{dong2018few}\cite{wang2019panet}\cite{zhang2019pyramid}\cite{zhang2019canet}\cite{siam2019amp}. Previous literature in few-shot object segmentation relied on manually labelled segmentation masks. A few recent works~\cite{rakelly2018conditional}\cite{zhang2019canet}\cite{wang2019panet} started to conduct experiments using weak annotations in terms of scribbles, bounding boxes. However, limited research was conducted on using image-level supervision for few-shot object segmentation with one sole recent work~\cite{raza2019weakly}.&#10;&#10;In order to improve image-level few-shot object segmentation we propose a multi-modal interaction module that leverages the interaction between support and query visual features and word embeddings. The use of neural word embeddings pretrained on GoogleNews~\cite{mikolov2013distributed} and visual embeddings from pretrained networks on ImageNet~\cite{deng2009imagenet} allows to build such a model. We propose a novel approach that uses Stacked Co-Attention to leverage the interaction among visual and word embeddings. We mainly inspire from~\cite{lu2019see} which proposed a co-attention siamese network for unsupervised video object segmentation. However, our setup mainly focuses on the few-shot object segmentation aspect to assess its ability to generalize to novel classes. This aspect motivates why we meta-learned a multi-modal interaction module that leverages the interaction between support and query with image-level supervision.&#10;&#10;Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}. Two main categories for video object segmentation are studied which are semi-supervised VOS and unsupervised VOS. The semi-supervised VOS also named as few-shot requires an initial strongly labelled segmentation mask to be provided. In a recent work~\cite{khoreva2018video} video object segmentation using language expression has been studied. However, their work does not focus on the aspect of segmenting novel categories which we try to address since it can be of potential benefit to the few-shot learning community. The additional temporal information in VOS can provide extra unlabelled data about the object poses or its interactions with other objects to the few-shot object segmentation method. It has the potential to be closer to human-like learning rather than learning from a single frame. To the best of our knowledge we are the first to propose a novel setup for video object segmentation that focuses on the ability to generalize to novel classes by splitting the categories to different folds which would better assess the generalization ability. The setup as well only requires image-level labels for the first frame to segment the corresponding objects unlike conventional semi-supervised VOS. Although Youtube-VOS~\cite{xu2018youtube} provided a way to evaluate on certain unseen categories but it does not utilize any of the category label information in the segmentation model. However, in order to ensure the evaluation for the few-shot method is not biased to a certain category, it is best to split into multiple folds and evaluate on different ones. To this end we use Youtube-VOS dataset~\cite{xu2018youtube} which is a large-scale VOS dataset with 65 different object categories, the originally provided training data is further split into 5 different folds with 13 novel classes per fold. The novel setup opens the door towards studying how few-shot object segmentation can benefit from temporal information for the different object poses or its interactions. The main contributions in this paper are summarized as follows: &#10;\begin{itemize}&#10; \item A novel formulation for learning image-level labelled few-shot segmentation based on a multi-modal interaction module is presented. It utilizes neural word embeddings and attention mechanisms that relates the support and query images. It enables attention to the most relevant spatial locations in the query feature maps.&#10; \item We propose a novel setup called few-shot weakly supervised video object segmentation that can be of potential benefit to few-shot object segmentation.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Weak Supervision, Machine Learning, Computer Vision, Few-Shot Learning, Mathematics, Artificial Intelligence, Object Segmentation, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1706.03466" label="1706.03466">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Image Recognition by Predicting Parameters from Activations" />
-          <attvalue for="1" value="  In this paper, we are interested in the few-shot learning problem. In&#10;particular, we focus on a challenging scenario where the number of categories&#10;is large and the number of examples per novel category is very limited, e.g. 1,&#10;2, or 3. Motivated by the close relationship between the parameters and the&#10;activations in a neural network associated with the same category, we propose a&#10;novel method that can adapt a pre-trained neural network to novel categories by&#10;directly predicting the parameters from the activations. Zero training is&#10;required in adaptation to novel categories, and fast inference is realized by a&#10;single forward pass. We evaluate our method by doing few-shot image recognition&#10;on the ImageNet dataset, which achieves the state-of-the-art classification&#10;accuracy on novel categories by a significant margin while keeping comparable&#10;performance on the large-scale categories. We also test our method on the&#10;MiniImageNet dataset and it strongly outperforms the previous state-of-the-art&#10;methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.06391" label="1908.06391">
-        <attvalues>
-          <attvalue for="0" value="PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment" />
-          <attvalue for="1" value="  Despite the great progress made by deep CNNs in image semantic segmentation,&#10;they typically require a large number of densely-annotated images for training&#10;and are difficult to generalize to unseen object categories. Few-shot&#10;segmentation has thus been developed to learn to perform segmentation from only&#10;a few annotated examples. In this paper, we tackle the challenging few-shot&#10;segmentation problem from a metric learning perspective and present PANet, a&#10;novel prototype alignment network to better utilize the information of the&#10;support set. Our PANet learns class-specific prototype representations from a&#10;few support images within an embedding space and then performs segmentation&#10;over the query images through matching each pixel to the learned prototypes.&#10;With non-parametric metric learning, PANet offers high-quality prototypes that&#10;are representative for each semantic class and meanwhile discriminative for&#10;different classes. Moreover, PANet introduces a prototype alignment&#10;regularization between support and query. With this, PANet fully exploits&#10;knowledge from the support and provides better generalization on few-shot&#10;segmentation. Significantly, our model achieves the mIoU score of 48.1% and&#10;55.7% on PASCAL-5i for 1-shot and 5-shot settings respectively, surpassing the&#10;state-of-the-art method by 1.8% and 8.6%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1310.4546" label="1310.4546">
-        <attvalues>
-          <attvalue for="0" value="Distributed Representations of Words and Phrases and their&#10;  Compositionality" />
-          <attvalue for="1" value="  The recently introduced continuous Skip-gram model is an efficient method for&#10;learning high-quality distributed vector representations that capture a large&#10;number of precise syntactic and semantic word relationships. In this paper we&#10;present several extensions that improve both the quality of the vectors and the&#10;training speed. By subsampling of the frequent words we obtain significant&#10;speedup and also learn more regular word representations. We also describe a&#10;simple alternative to the hierarchical softmax called negative sampling. An&#10;inherent limitation of word representations is their indifference to word order&#10;and their inability to represent idiomatic phrases. For example, the meanings&#10;of &quot;Canada&quot; and &quot;Air&quot; cannot be easily combined to obtain &quot;Air Canada&quot;.&#10;Motivated by this example, we present a simple method for finding phrases in&#10;text, and show that learning good vector representations for millions of&#10;phrases is possible.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.07217" label="1612.07217">
-        <attvalues>
-          <attvalue for="0" value="Learning Motion Patterns in Videos" />
-          <attvalue for="1" value="  The problem of determining whether an object is in motion, irrespective of&#10;camera motion, is far from being solved. We address this challenging task by&#10;learning motion patterns in videos. The core of our approach is a fully&#10;convolutional network, which is learned entirely from synthetic video&#10;sequences, and their ground-truth optical flow and motion segmentation. This&#10;encoder-decoder style architecture first learns a coarse representation of the&#10;optical flow field features, and then refines it iteratively to produce motion&#10;labels at the original high-resolution. We further improve this labeling with&#10;an objectness map and a conditional random field, to account for errors in&#10;optical flow, and also to focus on moving &quot;things&quot; rather than &quot;stuff&quot;. The&#10;output label of each pixel denotes whether it has undergone independent motion,&#10;i.e., irrespective of camera motion. We demonstrate the benefits of this&#10;learning framework on the moving object segmentation task, where the goal is to&#10;segment all objects in motion. Our approach outperforms the top method on the&#10;recently released DAVIS benchmark dataset, comprising real-world sequences, by&#10;5.6%. We also evaluate on the Berkeley motion segmentation database, achieving&#10;state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.05384" label="1701.05384">
-        <attvalues>
-          <attvalue for="0" value="FusionSeg: Learning to combine motion and appearance for fully automatic&#10;  segmention of generic objects in videos" />
-          <attvalue for="1" value="  We propose an end-to-end learning framework for segmenting generic objects in&#10;videos. Our method learns to combine appearance and motion information to&#10;produce pixel level segmentation masks for all prominent objects in videos. We&#10;formulate this task as a structured prediction problem and design a two-stream&#10;fully convolutional neural network which fuses together motion and appearance&#10;in a unified framework. Since large-scale video datasets with pixel level&#10;segmentations are problematic, we show how to bootstrap weakly annotated videos&#10;together with existing image recognition datasets for training. Through&#10;experiments on three challenging video segmentation benchmarks, our method&#10;substantially improves the state-of-the-art for segmenting generic (unseen)&#10;objects. Code and pre-trained models are available on the project website.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08006" label="1803.08006">
-        <attvalues>
-          <attvalue for="0" value="Video Object Segmentation with Language Referring Expressions" />
-          <attvalue for="1" value="  Most state-of-the-art semi-supervised video object segmentation methods rely&#10;on a pixel-accurate mask of a target object provided for the first frame of a&#10;video. However, obtaining a detailed segmentation mask is expensive and&#10;time-consuming. In this work we explore an alternative way of identifying a&#10;target object, namely by employing language referring expressions. Besides&#10;being a more practical and natural way of pointing out a target object, using&#10;language specifications can help to avoid drift as well as make the system more&#10;robust to complex dynamics and appearance variations. Leveraging recent&#10;advances of language grounding models designed for images, we propose an&#10;approach to extend them to video data, ensuring temporally coherent&#10;predictions. To evaluate our method we augment the popular video object&#10;segmentation benchmarks, DAVIS'16 and DAVIS'17 with language descriptions of&#10;target objects. We show that our language-supervised approach performs on par&#10;with the methods which have access to a pixel-level mask of the target object&#10;on DAVIS'16 and is competitive to methods using scribbles on the challenging&#10;DAVIS'17 dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.12670" label="2102.12670">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Ellipse Detection for Robotics Applications" />
-          <attvalue for="1" value="  We propose a new algorithm for real-time detection and tracking of elliptic&#10;patterns suitable for real-world robotics applications. The method fits&#10;ellipses to each contour in the image frame and rejects ellipses that do not&#10;yield a good fit. The resulting detection and tracking method is lightweight&#10;enough to be used on robots' resource-limited onboard computers, can deal with&#10;lighting variations and detect the pattern even when the view is partial. The&#10;method is tested on an example application of an autonomous UAV landing on a&#10;fast-moving vehicle to show its performance indoors, outdoors, and in&#10;simulation on a real-world robotics task. The comparison with other well-known&#10;ellipse detection methods shows that our proposed algorithm outperforms other&#10;methods with the F1 score of 0.981 on a dataset with over 1500 frames. The&#10;videos of experiments, the source codes, and the collected dataset are provided&#10;with the paper at https://theairlab.org/landing-on-vehicle .&#10;" />
-          <attvalue for="2" value=" &#10;&#10;Real-time detection and tracking of circular and elliptic shapes using an onboard vision system are essential for many real-world (mainly robotics) applications. For example, many aerial robot' landing zones consist of an elliptical shape~\cite{cargozone}, and autonomous cars need to detect the circular road signs~\cite{roadsign}. &#10;&#10;Detecting ellipses in images has been a topic of interest for researchers for a long time \cite{tsuji1977detection, tsuji1978detection}. In general, it is possible to classify the available ellipse detection approaches into four classes. &#10;&#10;The first class contains voting-based algorithms, including Hough Transform (HT) \cite{cite:hough} and the methods based on it. The HT algorithm uses a 5-dimensional parametric space for the ellipse detection task and is too slow for real-time applications. Other methods try to enhance HT by reducing the dimensionality in parametric space \cite{cite:houghdim1, cite:houghdim2, cite:zhang}, performing piecewise-linear approximation for the curved segments \cite{cite:houghapprox1} or randomizing the method \cite{cite:houghrand1, cite:houghrand2, cite:houghrand3}. Some of these modified HT-based methods are fast but generally less accurate and unsuitable for many robotics applications.&#10;&#10;The second class contains optimization-based methods. Most popular methods convert the ellipse fitting problem into a least-squares optimization problem and solve the problem to find the best fit \cite{cite:fitls, fitzgibbon1995buyer}. These methods are generally not robust and tend to produce many false positives. However, their high processing speed is useful for quickly estimating the ellipse as the first step in other methods. Another group of optimization-based methods try to solve the non-linear optimization problem of fitting an ellipse using a genetic algorithm \cite{cite:genetic1, cite:genetic2}. These algorithms perform well in noisy images with multiple ellipses, but their processing time makes them impractical for real-time applications.&#10;&#10;The third class consists of methods based on edge linking and curve grouping \cite{cite:mai}. These methods can detect ellipses from complex and noisy images but are generally computationally expensive and cannot be used in real-time applications.&#10;&#10;The last class contains the methods that use an ad-hoc approach or combine the methods from the first three classes to achieve a faster and more accurate ellipse detection. A real-time method proposed by Nguyen et al. \cite{cite:nguyen} detects arc-segments from the image and groups them into elliptical arcs to estimate the ellipse parameters using a least-square optimization. A method proposed by Fornaciari et al.~\cite{fornaciari2014ellipsedetect} combines arc grouping with Hough Transform in a decomposed parameter space to estimate the ellipse parameters. In this way, it achieves a real-time performance comparable to slower, more robust algorithms.&#10;&#10;Currently, in many robotics applications, the detection of the previously-known elliptic objects is done by general learning-based object detection methods~\cite{bhattacharya2021}. However, this approach requires collection and enhancement of large amount of data for each individual object and access to powerful computing systems for training. Recently, there has been an effort to implement a learning-based method specifically for ellipse detection~\cite{cnnellipse}. The method has been shown to work well for occlusions and partial views. However, it requires a powerful processor and is not yet suitable for real-time execution on onboard computers.&#10;&#10;While many ellipse detection algorithms are proposed for computer vision tasks (\cite{WANG2021107741, Liu2020, 8972900}), the performance of these methods drops when used in real-world robotics tasks. Some of the challenges in these applications include:&#10;&#10;\begin{itemize}[leftmargin=*]&#10;\item The algorithm should work online (with a frequency greater than 10 Hz), generally on the robot's resource-limited onboard computer.&#10;&#10;\item The elliptical object should be detected when it is far from or close to the robot.&#10;&#10;\item The shape of the ellipse is transformed by a projective distortion, which occurs when the shape is seen from different views.&#10;&#10;\item There is a wide range of possible illumination conditions (e.g., cloudy, sunny, morning, evening, indoor lighting). &#10;&#10;\item Due to reflections (e.g., from sources like sun, bulbs), the pattern may not always be seen in all the frames, even when the camera is close.&#10;&#10;\item In some frames, there may be shadows on the pattern (e.g., the shadow of the robot or trees around).&#10;&#10;\item In some frames, the pattern may be seen only partially (due to occlusion or being outside of the camera view).&#10;&#10;\item In some frames, the pattern may not be present.&#10;&#10;\end{itemize}&#10;&#10;Different approaches have been devised in the literature to detect and track an elliptic pattern in real applications to address the challenges above. For example, \cite{beul2018} uses the circular Hough-transform algorithm for the initial detection, which is a slow algorithm, and then uses other features in their pattern for the tracking. The authors of~\cite{jin2018} have developed a Convolutional Neural Network trained with over 5,000 images collected from the pattern moving at a maximum speed of $15$\,km/h at various heights to select (validate) the desired ellipse from all the ellipses detected using the method in~\cite{jia2017}. The authors of~\cite{li2018} use the method proposed in~\cite{fornaciari2014ellipsedetect} for ellipse detection only when their robot is far from the pattern and switch to other features in their pattern for the closer frames. &#10;&#10;It is possible to use a predictor filter along with the detector to narrow down the region of interest (ROI) and improve the detection speed of ellipses across the frames. The authors of~\cite{ellipsekalman1} and \cite{ellipsekalman2} use Kalman filter to predict the direction of the ellipse movement in the frames, increase the detection speed and reduce the false positives. In many robotics applications, the combined simultaneous motions of the elliptic pattern, the robot, and the camera complicate the prediction provided by such predictors, resulting in the loss of tracking on many frames, triggering a detection on the entire frame and slowing the tracking speed.&#10;&#10;This paper proposes a novel ellipse detection method that performs on resource-limited onboard computers in real-time. Our proposed ad-hoc method first extracts all contours from the input image and then uses a least-square method to fit an ellipse to each contour. It tests how well the estimated ellipse fits the contour and starts rejecting the contours using several criteria. The remaining contours are accepted as resulted ellipses, and if there are no contours left (e.g., due to the absence of ellipses in the frame), the algorithm returns an empty set of ellipses. The detection method is combined with a simple tracking algorithm that changes detection parameters as necessary and significantly increases the elliptical pattern detection's precision and performance. This simple tracker has been shown to be faster than using predictor filters due to the reduced tracking losses. The resulting detector and tracking method can deal with lighting variations and detect the pattern even when the view is partial. By comparing the results of our experiments on a collected dataset to the other methods tested on similar datasets we show that our method outperforms all the other real-time ellipse detection methods proposed so far.&#10;&#10;Section~\ref{sec:approach} explains the developed method for detection and tracking of the elliptical targets; Section~\ref{sec:tests} shows an example real-time application that uses the proposed algorithm and compares the performance with some other available well-known methods. Finally, Section~\ref{sec:conclusion} discusses how the proposed method can be further improved in the future.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Pattern Detection, Robotics Applications, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2001.11584" label="2001.11584">
-        <attvalues>
-          <attvalue for="0" value="Ellipse R-CNN: Learning to Infer Elliptical Object from Clustering and&#10;  Occlusion" />
-          <attvalue for="1" value="  Images of heavily occluded objects in cluttered scenes, such as fruit&#10;clusters in trees, are hard to segment. To further retrieve the 3D size and 6D&#10;pose of each individual object in such cases, bounding boxes are not reliable&#10;from multiple views since only a little portion of the object's geometry is&#10;captured. We introduce the first CNN-based ellipse detector, called Ellipse&#10;R-CNN, to represent and infer occluded objects as ellipses. We first propose a&#10;robust and compact ellipse regression based on the Mask R-CNN architecture for&#10;elliptical object detection. Our method can infer the parameters of multiple&#10;elliptical objects even they are occluded by other neighboring objects. For&#10;better occlusion handling, we exploit refined feature regions for the&#10;regression stage, and integrate the U-Net structure for learning different&#10;occlusion patterns to compute the final detection score. The correctness of&#10;ellipse regression is validated through experiments performed on synthetic data&#10;of clustered ellipses. We further quantitatively and qualitatively demonstrate&#10;that our approach outperforms the state-of-the-art model (i.e., Mask R-CNN&#10;followed by ellipse fitting) and its three variants on both synthetic and real&#10;datasets of occluded and clustered elliptical objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.03496" label="2104.03496">
-        <attvalues>
-          <attvalue for="0" value="Prototypical Region Proposal Networks for Few-Shot Localization and&#10;  Classification" />
-          <attvalue for="1" value="  Recently proposed few-shot image classification methods have generally&#10;focused on use cases where the objects to be classified are the central subject&#10;of images. Despite success on benchmark vision datasets aligned with this use&#10;case, these methods typically fail on use cases involving densely-annotated,&#10;busy images: images common in the wild where objects of relevance are not the&#10;central subject, instead appearing potentially occluded, small, or among other&#10;incidental objects belonging to other classes of potential interest. To&#10;localize relevant objects, we employ a prototype-based few-shot segmentation&#10;model which compares the encoded features of unlabeled query images with&#10;support class centroids to produce region proposals indicating the presence and&#10;location of support set classes in a query image. These region proposals are&#10;then used as additional conditioning input to few-shot image classifiers. We&#10;develop a framework to unify the two stages (segmentation and classification)&#10;into an end-to-end classification model -- PRoPnet -- and empirically&#10;demonstrate that our methods improve accuracy on image datasets with natural&#10;scenes containing multiple object classes.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Metric learning approaches for image classification have shown impressive accuracy in the few-shot setting, where models must learn to detect unobserved classes from few labeled examples. The most successful demonstrations have been on datasets such as mini-ImageNet \cite{finn2017model}, ImageNet \cite{deng2009imagenet}, and Omniglot \cite{lake2019omniglot}---datasets comprised of images typically containing centered, non-occluded objects commonly appearing in the foreground. However, performance of state-of-the-art few-shot classification methods can severely degrade on naturally-occurring, so-called ``busy image'' scenes wherein the object to classify is small or occluded, or accompanied by several other incidental objects from other classes. In the common use case where query images have neither image-level annotations nor any indication of where objects of interest are in the image, the few-shot classification task becomes considerably more challenging.&#10; This work presents and assesses methods for addressing these limitations by conditioning model predictions on regions of interest for improved classification performance in use cases involving busy, unlabeled query images.&#10;&#10;Our presented methods assume the use of an annotated support set---images that define each new few-shot class---wherein each support image is accompanied not only with labels indicating their corresponding class, but also with an annotation mask indicating the objects of interest that belong to the class in question. However, we also explore the use case in which query images are {entirely unlabeled}---no image-level annotations or annotation masks are available---and attempt to gain insight into how this can impact model performance. We address three basic research questions related to few-shot classification with busy images:&#10;&#10;\begin{enumerate}[noitemsep]&#10; \item Can annotation information be leveraged for enhanced classification of busy images?&#10; \item How crucial are query image localization annotations for few-shot busy image classification?&#10; \item Can few-shot models be leveraged to generate localization proposals for busy, unlabeled query images?&#10; \end{enumerate}&#10;&#10;To address the first question, we compare the accuracy of few-shot classifiers with and without localization, finding that including localization features is beneficial to performance, particularly on densely-annotated datasets. Then, in comparing performance with and without query images with region annotations, we find that for datasets containing busy images which benefit most from localization, few-shot classification methods which incorporate localization yield sub-par performance when annotations are not available for query images.&#10;&#10;As a solution to localizing and classifying busy, unlabeled query images under the few-shot paradigm, we present an $n$-shot, one-way segmentation network which produces query region proposal masks conditioned on a set of annotated support images. &#10; We develop an end-to-end classifier we call Prototypical Region Proposal Networks (PRoPnet), whose first stage generates region proposals for query images given a set of support images with class labels and object annotations, and whose second stage augments the standard ResNet-50 architecture with a four-channel input (color channels plus annotation mask) for localization-conditioned classification.&#10;&#10;We conduct localization conditioning experiments on seven datasets&#10; which span a range of classification difficulties---difficult datasets like Visual Genome which contain images of very busy scenes, as well as those which fall on the other end of the spectrum of difficulty, such as ImageNet, where most images depict only objects from a single class of interest in the foreground. &#10; Our experiments show that query image annotation information can make few-shot classification a tractable problem for difficult scenes characteristic of Visual Genome, increasing 5-shot, 5-way accuracy from 59 to 76\%. We further demonstrate PRoPnet's improved 5-shot, 5-way classification for the FSS-1000 and PASCAL-5$^i$ datasets. Together, we conclude that i) localization conditioning can be crucial for adequate performance on busy natural image scenes and ii) two-stage networks like PRoPnet which incorporate generated query region proposals as input to a Prototypical Network are a promising approach to providing this essential conditioning. &#10;&#10;Related Work&#10; Given saturation of performance on the standard few-shot benchmarks, recent works have extended few-shot tasks to more difficult settings, e.g. object classes with confounding texture \cite{azad2020texture}, cluttered scene segmentation \cite{michaelis2018one, fortin2019few}, long tailed distributions~\cite{wertheimer_few-shot_2019}, and segmentation of homogeneous object clusters ~\cite{wu2018annotation}.&#10; Enhancements to Prototypical Networks (ProtoNets)~\cite{snell_prototypical_2017} in particular have been proposed for difficult few-shot tasks involving, for instance, inhomogeneous noisy datasets~\cite{fort2017gaussian}, domain adaptation~\cite{Pan_2019_CVPR}, and relation classification in text~\cite{gao2019hybrid}.&#10; Several works have incorporated localization conditioning into a few-shot classification architecture~\cite{karlinsky2020starnet, lifchitz2020few, lin2020fewshot, selvaraju2017grad, wertheimer_few-shot_2019}. We include comparison with the Few-Shot Localization (FSL) method presented in \cite{wertheimer_few-shot_2019} in our experimental evaluations.&#10; Numerous contemporaneous papers have presented few-shot semantic segmentation methods~\cite{lin2020fewshot, feyjie2020semi, liu2020prototype, wang2019panet, yang2020new, nguyen2019feature, siam2019amp, michaelis2018one, liu2020crnet, zhang2019canet, zhang2020sg, bhat2020learning}, which differ from our approach along several axes of variation including comparison metric, support set aggregation, feature map backbone, and mask refinement.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Few-Shot Learning, Mathematics, Artificial Intelligence, Object Segmentation, Cognitive Science, Computer Science, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1902.03477" label="1902.03477">
-        <attvalues>
-          <attvalue for="0" value="The Omniglot challenge: a 3-year progress report" />
-          <attvalue for="1" value="  Three years ago, we released the Omniglot dataset for one-shot learning,&#10;along with five challenge tasks and a computational model that addresses these&#10;tasks. The model was not meant to be the final word on Omniglot; we hoped that&#10;the community would build on our work and develop new approaches. In the time&#10;since, we have been pleased to see wide adoption of the dataset. There has been&#10;notable progress on one-shot classification, but researchers have adopted new&#10;splits and procedures that make the task easier. There has been less progress&#10;on the other four tasks. We conclude that recent approaches are still far from&#10;human-like concept learning on Omniglot, a challenge that requires performing&#10;many tasks with a single model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04052" label="2003.04052">
-        <attvalues>
-          <attvalue for="0" value="On the Texture Bias for Few-Shot CNN Segmentation" />
-          <attvalue for="1" value="  Despite the initial belief that Convolutional Neural Networks (CNNs) are&#10;driven by shapes to perform visual recognition tasks, recent evidence suggests&#10;that texture bias in CNNs provides higher performing models when learning on&#10;large labeled training datasets. This contrasts with the perceptual bias in the&#10;human visual cortex, which has a stronger preference towards shape components.&#10;Perceptual differences may explain why CNNs achieve human-level performance&#10;when large labeled datasets are available, but their performance significantly&#10;degrades in lowlabeled data scenarios, such as few-shot semantic segmentation.&#10;To remove the texture bias in the context of few-shot learning, we propose a&#10;novel architecture that integrates a set of Difference of Gaussians (DoG) to&#10;attenuate high-frequency local components in the feature space. This produces a&#10;set of modified feature maps, whose high-frequency components are diminished at&#10;different standard deviation values of the Gaussian distribution in the spatial&#10;domain. As this results in multiple feature maps for a single image, we employ&#10;a bi-directional convolutional long-short-term-memory to efficiently merge the&#10;multi scale-space representations. We perform extensive experiments on three&#10;well-known few-shot segmentation benchmarks -- Pascal i5, COCO-20i and FSS-1000&#10;-- and demonstrate that our method outperforms state-of-the-art approaches in&#10;two datasets under the same conditions. The code is available at:&#10;https://github.com/rezazad68/fewshot-segmentation&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.09597" label="1803.09597">
-        <attvalues>
-          <attvalue for="0" value="One-Shot Segmentation in Clutter" />
-          <attvalue for="1" value="  We tackle the problem of one-shot segmentation: finding and segmenting a&#10;previously unseen object in a cluttered scene based on a single instruction&#10;example. We propose a novel dataset, which we call $\textit{cluttered&#10;Omniglot}$. Using a baseline architecture combining a Siamese embedding for&#10;detection with a U-net for segmentation we show that increasing levels of&#10;clutter make the task progressively harder. Using oracle models with access to&#10;various amounts of ground-truth information, we evaluate different aspects of&#10;the problem and show that in this kind of visual search task, detection and&#10;segmentation are two intertwined problems, the solution to each of which helps&#10;solving the other. We therefore introduce $\textit{MaskNet}$, an improved model&#10;that attends to multiple candidate locations, generates segmentation proposals&#10;to mask out background clutter and selects among the segmented objects. Our&#10;findings suggest that such image recognition models based on an iterative&#10;refinement of object detection and foreground segmentation may provide a way to&#10;deal with highly cluttered scenes.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.08502" label="1904.08502">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Learning with Localization in Realistic Settings" />
-          <attvalue for="1" value="  Traditional recognition methods typically require large,&#10;artificially-balanced training classes, while few-shot learning methods are&#10;tested on artificially small ones. In contrast to both extremes, real world&#10;recognition problems exhibit heavy-tailed class distributions, with cluttered&#10;scenes and a mix of coarse and fine-grained class distinctions. We show that&#10;prior methods designed for few-shot learning do not work out of the box in&#10;these challenging conditions, based on a new &quot;meta-iNat&quot; benchmark. We&#10;introduce three parameter-free improvements: (a) better training procedures&#10;based on adapting cross-validation to meta-learning, (b) novel architectures&#10;that localize objects using limited bounding box annotations before&#10;classification, and (c) simple parameter-free expansions of the feature space&#10;based on bilinear pooling. Together, these improvements double the accuracy of&#10;state-of-the-art models on meta-iNat while generalizing to prior benchmarks,&#10;complex neural architectures, and settings with substantial domain shift.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.00383" label="1802.00383">
-        <attvalues>
-          <attvalue for="0" value="Annotation-Free and One-Shot Learning for Instance Segmentation of&#10;  Homogeneous Object Clusters" />
-          <attvalue for="1" value="  We propose a novel approach for instance segmen- tation given an image of&#10;homogeneous object clus- ter (HOC). Our learning approach is one-shot be- cause&#10;a single video of an object instance is cap- tured and it requires no human&#10;annotation. Our in- tuition is that images of homogeneous objects can be&#10;effectively synthesized based on structure and illumination priors derived from&#10;real images. A novel solver is proposed that iteratively maximizes our&#10;structured likelihood to generate realistic im- ages of HOC. Illumination&#10;transformation scheme is applied to make the real and synthetic images share&#10;the same illumination condition. Extensive experiments and comparisons are&#10;performed to ver- ify our method. We build a dataset consisting of pixel-level&#10;annotated images of HOC. The dataset and code will be published with the paper.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.02735" label="1708.02735">
-        <attvalues>
-          <attvalue for="0" value="Gaussian Prototypical Networks for Few-Shot Learning on Omniglot" />
-          <attvalue for="1" value="  We propose a novel architecture for $k$-shot classification on the Omniglot&#10;dataset. Building on prototypical networks, we extend their architecture to&#10;what we call Gaussian prototypical networks. Prototypical networks learn a map&#10;between images and embedding vectors, and use their clustering for&#10;classification. In our model, a part of the encoder output is interpreted as a&#10;confidence region estimate about the embedding point, and expressed as a&#10;Gaussian covariance matrix. Our network then constructs a direction and class&#10;dependent distance metric on the embedding space, using uncertainties of&#10;individual data points as weights. We show that Gaussian prototypical networks&#10;are a preferred architecture over vanilla prototypical networks with an&#10;equivalent number of parameters. We report state-of-the-art performance in&#10;1-shot and 5-shot classification both in 5-way and 20-way regime (for 5-shot&#10;5-way, we are comparable to previous state-of-the-art) on the Omniglot dataset.&#10;We explore artificially down-sampling a fraction of images in the training set,&#10;which improves our performance even further. We therefore hypothesize that&#10;Gaussian prototypical networks might perform better in less homogeneous,&#10;noisier datasets, which are commonplace in real world applications.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.11227" label="1904.11227">
-        <attvalues>
-          <attvalue for="0" value="Transferrable Prototypical Networks for Unsupervised Domain Adaptation" />
-          <attvalue for="1" value="  In this paper, we introduce a new idea for unsupervised domain adaptation via&#10;a remold of Prototypical Networks, which learn an embedding space and perform&#10;classification via a remold of the distances to the prototype of each class.&#10;Specifically, we present Transferrable Prototypical Networks (TPN) for&#10;adaptation such that the prototypes for each class in source and target domains&#10;are close in the embedding space and the score distributions predicted by&#10;prototypes separately on source and target data are similar. Technically, TPN&#10;initially matches each target example to the nearest prototype in the source&#10;domain and assigns an example a &quot;pseudo&quot; label. The prototype of each class&#10;could then be computed on source-only, target-only and source-target data,&#10;respectively. The optimization of TPN is end-to-end trained by jointly&#10;minimizing the distance across the prototypes on three types of data and&#10;KL-divergence of score distributions output by each pair of the prototypes.&#10;Extensive experiments are conducted on the transfers across MNIST, USPS and&#10;SVHN datasets, and superior results are reported when comparing to&#10;state-of-the-art approaches. More remarkably, we obtain an accuracy of 80.4% of&#10;single model on VisDA 2017 dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.08650" label="2106.08650">
-        <attvalues>
-          <attvalue for="0" value="Shuffle Transformer with Feature Alignment for Video Face Parsing" />
-          <attvalue for="1" value="  This is a short technical report introducing the solution of the Team&#10;TCParser for Short-video Face Parsing Track of The 3rd Person in Context (PIC)&#10;Workshop and Challenge at CVPR 2021. In this paper, we introduce a strong&#10;backbone which is cross-window based Shuffle Transformer for presenting&#10;accurate face parsing representation. To further obtain the finer segmentation&#10;results, especially on the edges, we introduce a Feature Alignment Aggregation&#10;(FAA) module. It can effectively relieve the feature misalignment issue caused&#10;by multi-resolution feature aggregation. Benefiting from the stronger backbone&#10;and better feature aggregation, the proposed method achieves 86.9519% score in&#10;the Short-video Face Parsing track of the 3rd Person in Context (PIC) Workshop&#10;and Challenge, ranked the first place.&#10;" />
-          <attvalue for="2" value="&#10;Face parsing has been applied in a variety of scenarios such as face understanding, editing, synthesis, and animation. As a particular task in semantic segmentation, Face parsing assigns different labels to the corresponding regions on human faces, e.g., hair, facial skins, eyes, nose, mouth and etc., which relies highly on the accuracy of facial components' representation. As a consequence, this is very important to find a way to improve the representation ability. &#10;&#10;Traditionally, ~\cite{smith2013exemplar,warrell2009labelfaces,kae2013augmenting} use hand crafted features including SIFT or machine learning method including Restricted Boltzmann Machine (RBM) to extract local and global features. Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress.&#10;For face parsing, CNN-based features are imported into network to extract multi-scale or independent-part facial features~\cite{liu2015multi,luo2012hierarchical,zhou2015inter}. Lin et al. propose two branches with the local-based for inner facial components and the global based for outer facial ones~\cite{lin2019face}. Te et al. propose to learn graph representations over facial images, which model the relations between regions~\cite{te2020edge}. These local-based methods almost adopt the coarse-to-fine strategy, training separated models for various facial components (e.g. eyes, nose etc.) to extract features for each part individually. However, this kind of method achieves good performance at the expense of large memory and computation consumption.&#10;&#10;In this paper, we introduce a cross-window based Shuffle Transformer with feature alignment for better improving the face parsing representation ability. Transformers~\cite{dosovitskiy2020image,nicolas2020end,hugo2020training,liu2021swin} have achieve excellent performance on a wide range of visual tasks including image-level classification, object detection, and semantic segmentation. And Shuffle Transformer~\cite{huang2021shuffle} is confirmed as a strong backbone for many vision tasks. Furthermore we import a feature alignment aggregation module to pass information between high-resolution and low-resolution feature maps precisely. The proposed method achieves a good performance in the Short-video Face Parsing track (SFP) of the 3rd Person in Context (PIC) Workshop and achieved the accuracy of 86.9519$\%$, ranked the first place.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Face Parsing Techniques, Mathematics, Artificial Intelligence, Deep Learning Architectures, Computer Science, Computer Vision Challenges" />
-        </attvalues>
-      </node>
-      <node id="1804.09337" label="1804.09337">
-        <attvalues>
-          <attvalue for="0" value="Learning a Discriminative Feature Network for Semantic Segmentation" />
-          <attvalue for="1" value="  Most existing methods of semantic segmentation still suffer from two aspects&#10;of challenges: intra-class inconsistency and inter-class indistinction. To&#10;tackle these two problems, we propose a Discriminative Feature Network (DFN),&#10;which contains two sub-networks: Smooth Network and Border Network.&#10;Specifically, to handle the intra-class inconsistency problem, we specially&#10;design a Smooth Network with Channel Attention Block and global average pooling&#10;to select the more discriminative features. Furthermore, we propose a Border&#10;Network to make the bilateral features of boundary distinguishable with deep&#10;semantic boundary supervision. Based on our proposed DFN, we achieve&#10;state-of-the-art performance 86.2% mean IOU on PASCAL VOC 2012 and 80.3% mean&#10;IOU on Cityscapes dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.07122" label="1511.07122">
-        <attvalues>
-          <attvalue for="0" value="Multi-Scale Context Aggregation by Dilated Convolutions" />
-          <attvalue for="1" value="  State-of-the-art models for semantic segmentation are based on adaptations of&#10;convolutional networks that had originally been designed for image&#10;classification. However, dense prediction and image classification are&#10;structurally different. In this work, we develop a new convolutional network&#10;module that is specifically designed for dense prediction. The presented module&#10;uses dilated convolutions to systematically aggregate multi-scale contextual&#10;information without losing resolution. The architecture is based on the fact&#10;that dilated convolutions support exponential expansion of the receptive field&#10;without loss of resolution or coverage. We show that the presented context&#10;module increases the accuracy of state-of-the-art semantic segmentation&#10;systems. In addition, we examine the adaptation of image classification&#10;networks to dense prediction and show that simplifying the adapted network can&#10;increase accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09798" label="1908.09798">
-        <attvalues>
-          <attvalue for="0" value="SPGNet: Semantic Prediction Guidance for Scene Parsing" />
-          <attvalue for="1" value="  Multi-scale context module and single-stage encoder-decoder structure are&#10;commonly employed for semantic segmentation. The multi-scale context module&#10;refers to the operations to aggregate feature responses from a large spatial&#10;extent, while the single-stage encoder-decoder structure encodes the high-level&#10;semantic information in the encoder path and recovers the boundary information&#10;in the decoder path. In contrast, multi-stage encoder-decoder networks have&#10;been widely used in human pose estimation and show superior performance than&#10;their single-stage counterpart. However, few efforts have been attempted to&#10;bring this effective design to semantic segmentation. In this work, we propose&#10;a Semantic Prediction Guidance (SPG) module which learns to re-weight the local&#10;features through the guidance from pixel-wise semantic prediction. We find that&#10;by carefully re-weighting features across stages, a two-stage encoder-decoder&#10;network coupled with our proposed SPG module can significantly outperform its&#10;one-stage counterpart with similar parameters and computations. Finally, we&#10;report experimental results on the semantic segmentation benchmark Cityscapes,&#10;in which our SPGNet attains 81.1% on the test set using only 'fine'&#10;annotations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.02479" label="1806.02479">
-        <attvalues>
-          <attvalue for="0" value="Interlinked Convolutional Neural Networks for Face Parsing" />
-          <attvalue for="1" value="  Face parsing is a basic task in face image analysis. It amounts to labeling&#10;each pixel with appropriate facial parts such as eyes and nose. In the paper,&#10;we present a interlinked convolutional neural network (iCNN) for solving this&#10;problem in an end-to-end fashion. It consists of multiple convolutional neural&#10;networks (CNNs) taking input in different scales. A special interlinking layer&#10;is designed to allow the CNNs to exchange information, enabling them to&#10;integrate local and contextual information efficiently. The hallmark of iCNN is&#10;the extensive use of downsampling and upsampling in the interlinking layers,&#10;while traditional CNNs usually uses downsampling only. A two-stage pipeline is&#10;proposed for face parsing and both stages use iCNN. The first stage localizes&#10;facial parts in the size-reduced image and the second stage labels the pixels&#10;in the identified facial parts in the original image. On a benchmark dataset we&#10;have obtained better results than the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.01342" label="1906.01342">
-        <attvalues>
-          <attvalue for="0" value="Face Parsing with RoI Tanh-Warping" />
-          <attvalue for="1" value="  Face parsing computes pixel-wise label maps for different semantic components&#10;(e.g., hair, mouth, eyes) from face images. Existing face parsing literature&#10;have illustrated significant advantages by focusing on individual regions of&#10;interest (RoIs) for faces and facial components. However, the traditional&#10;crop-and-resize focusing mechanism ignores all contextual area outside the&#10;RoIs, and thus is not suitable when the component area is unpredictable, e.g.&#10;hair. Inspired by the physiological vision system of human, we propose a novel&#10;RoI Tanh-warping operator that combines the central vision and the peripheral&#10;vision together. It addresses the dilemma between a limited sized RoI for&#10;focusing and an unpredictable area of surrounding context for peripheral&#10;information. To this end, we propose a novel hybrid convolutional neural&#10;network for face parsing. It uses hierarchical local based method for inner&#10;facial components and global methods for outer facial components. The whole&#10;framework is simple and principled, and can be trained end-to-end. To&#10;facilitate future research of face parsing, we also manually relabel the&#10;training data of the HELEN dataset and will make it public. Experiments on both&#10;HELEN and LFW-PL benchmarks demonstrate that our method surpasses&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11240" label="2007.11240">
-        <attvalues>
-          <attvalue for="0" value="Edge-aware Graph Representation Learning and Reasoning for Face Parsing" />
-          <attvalue for="1" value="  Face parsing infers a pixel-wise label to each facial component, which has&#10;drawn much attention recently. Previous methods have shown their efficiency in&#10;face parsing, which however overlook the correlation among different face&#10;regions. The correlation is a critical clue about the facial appearance, pose,&#10;expression etc., and should be taken into account for face parsing. To this&#10;end, we propose to model and reason the region-wise relations by learning graph&#10;representations, and leverage the edge information between regions for&#10;optimized abstraction. Specifically, we encode a facial image onto a global&#10;graph representation where a collection of pixels (&quot;regions&quot;) with similar&#10;features are projected to each vertex. Our model learns and reasons over&#10;relations between the regions by propagating information across vertices on the&#10;graph. Furthermore, we incorporate the edge information to aggregate the&#10;pixel-wise features onto vertices, which emphasizes on the features around&#10;edges for fine segmentation along edges. The finally learned graph&#10;representation is projected back to pixel grids for parsing. Experiments&#10;demonstrate that our model outperforms state-of-the-art methods on the widely&#10;used Helen dataset, and also exhibits the superior performance on the&#10;large-scale CelebAMask-HQ and LaPa dataset. The code is available at&#10;https://github.com/tegusi/EAGRNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.03650" label="2106.03650">
-        <attvalues>
-          <attvalue for="0" value="Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer" />
-          <attvalue for="1" value="  Very recently, Window-based Transformers, which computed self-attention&#10;within non-overlapping local windows, demonstrated promising results on image&#10;classification, semantic segmentation, and object detection. However, less&#10;study has been devoted to the cross-window connection which is the key element&#10;to improve the representation ability. In this work, we revisit the spatial&#10;shuffle as an efficient way to build connections among windows. As a result, we&#10;propose a new vision transformer, named Shuffle Transformer, which is highly&#10;efficient and easy to implement by modifying two lines of code. Furthermore,&#10;the depth-wise convolution is introduced to complement the spatial shuffle for&#10;enhancing neighbor-window connections. The proposed architectures achieve&#10;excellent performance on a wide range of visual tasks including image-level&#10;classification, object detection, and semantic segmentation. Code will be&#10;released for reproduction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.16190" label="2408.16190">
-        <attvalues>
-          <attvalue for="0" value="Estimating Dynamic Flow Features in Groups of Tracked Objects" />
-          <attvalue for="1" value="  Interpreting motion captured in image sequences is crucial for a wide range&#10;of computer vision applications. Typical estimation approaches include optical&#10;flow (OF), which approximates the apparent motion instantaneously in a scene,&#10;and multiple object tracking (MOT), which tracks the motion of subjects over&#10;time. Often, the motion of objects in a scene is governed by some underlying&#10;dynamical system which could be inferred by analyzing the motion of groups of&#10;objects. Standard motion analyses, however, are not designed to intuit flow&#10;dynamics from trajectory data, making such measurements difficult in practice.&#10;The goal of this work is to extend gradient-based dynamical systems analyses to&#10;real-world applications characterized by complex, feature-rich image sequences&#10;with imperfect tracers. The tracer trajectories are tracked using deep vision&#10;networks and gradients are approximated using Lagrangian gradient regression&#10;(LGR), a tool designed to estimate spatial gradients from sparse data. From&#10;gradients, dynamical features such as regions of coherent rotation and&#10;transport barriers are identified. The proposed approach is affordably&#10;implemented and enables advanced studies including the motion analysis of two&#10;distinct object classes in a single image sequence. Two examples of the method&#10;are presented on data sets for which standard gradient-based analyses do not&#10;apply.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Understanding dynamical behavior in sequences of images is crucial for many applications within science, engineering, and policy. Because the motion of objects in a scene is often governed by an underlying flow, it conveys information of the system's dynamics and can be used for modelling, estimation, and control. A prominent example of this type of analysis comes from the world of experimental fluid mechanics, where image sequences of illuminated tracers supply the basis for estimating underlying vector fields~\cite{Raffel.Kompenhans_PIVbook_2018} and associated flow metrics~\cite{Haller_LCS_2015}. However, technological barriers have inhibited the generalization of such analyses to arbitrary systems of dynamic objects. The algorithms that are effective with carefully structured fluids experiments are not, for instance, effective when applied to videos of herds of sheep or of traffic at an interchange. Now, however, it is no longer difficult to identify arbitrary objects in images. Moreover, recent developments in the data-driven analysis of dynamical systems enable estimation of vector-field gradients from sparse data. Synthesizing these advances to estimate dynamical features in crowds of arbitrary objects is the goal of this work; if it is successful, there could be impact across broad disciplines, including drone surveillance, microfluidics, herds and swarms, traffic flows, crowd analysis, geophysical flows, and more. &#10;&#10;In the vision community, estimation of dynamical behavior is often pursued through one of two primary approaches. The first of these is optical flow (OF), which identifies the apparent motion from one image to the next in a sequence. A variety of surveys have been published to categorize the optical flow algorithms that have proliferated in recent years (for instance, \cite{Fortun.Kervrann_OFSurvey_2015, Zhai.Kong_OFSurvey_2021}). Traditional approaches such as Lucas-Kanade \cite{BakerMatthews_L-KSurvey_2004} and Horn-Schunk \cite{HornSchunk_HSOF_1981} are knowledge-driven in the sense that they do not need to be trained on data to be implemented. In recent years, deep, or data-driven, OF models such as FlowNet \cite{Dosovitskiy.Brox_FlowNet_2015, Ilg.Brox_FlowNet2_2017}, PWC-net \cite{Sun.Kautz_PWC-Net_2018}, and RAFT \cite{TeedDeng_RAFT_2020} have seen significant improvement in accuracy and computation time over their knowledge-driven counterparts \cite{Zhai.Kong_OFSurvey_2021}. Scene flow---the extension of optical flow to three dimensions using either disparity (depth) estimation in frame or 3D point clouds obtained via LiDAR---has also been heavily researched in recent years due to breadth of applications \cite{Zhai.Kong_OFSurvey_2021}. Though OF methods have become adept at estimating motion in scene, they are not well equipped to estimate the dynamic behavior of specific groups. OF does not discriminate objects in a scene, so any analysis of the motion is done on all motion within the scene. Moreover, OF analysis is limited to instantaneous measurements of velocity that are sensitive to the motion of the observer. Thus it is not possible using OF alone to objectively study the group behavior of entities over time. &#10;&#10;Rather than broadly estimating the motion of an entire scene, multiple object tracking (MOT), examines the movement of particular objects over time. It can be used to estimate the velocity of specific subjects along their trajectories rather than the velocity field over the entire scene. Many reviews have been written on the subject (for instance, \cite{Xu.Li_DeepMOTSurvey_2019, Luo.Kim_MOTReview_2021}). In its most basic form, MOT consists of an object detection step and a target association step. Object detection may be accomplished through the use of large vision models \cite{Voigtlaender.Leibe_MOTS_2019}, and association achieved using estimation filters like Kalman filters or particle filters, the Hungarian algorithm, or one of many other approaches \cite{Luo.Kim_MOTReview_2021}. Recent years have seen tracking frameworks built around vision transformers \cite{Su.Luo_TransTrack_2020, Meinhardt.Feichtenhofer_Trackformer_2022}, graph models \cite{Liu.Yu_GraphGSM_2020, Quach.Luu_DyGlip_2021}, and attention \cite{Guo.Shen_GraphAttention_2021, Chu.Yu_OnlineMOTAttention_2017}, to name a few alternative approaches. &#10;&#10;Traditional MOT approaches are also limited as a tool for identifying dynamical patterns in crowd behavior. MOT analyses are designed to give information about the behavior of individual entities, rather than interpret the underlying systems driving their motion. A field of work related to this is crowd analysis \cite{Zhan.Xu_CrowdSurvey_2008, Sanchez.Herrera_DeepCrowdAnalysis_2020}, which seeks to understand people's behavior in crowds. MOT is sometimes used for such studies \cite{cheriyadatRadke_DominantMotionInCrowds_2008}, which has seen recent interest as a result of social distancing protocols during the COVID-19 pandemic \cite{Verma.Kulkarni_COVIDCrowdDetection_2021, Pouw.Corbetta_CrowdManPhysicalDistancing_2020}. Crowd analyses, however, are typically not directed towards understanding the dynamic flow patterns espoused by the studied objects, but rather towards identifying anomalies or perceiving the intent of the crowd \cite{Sanchez.Herrera_DeepCrowdAnalysis_2020}. Understanding flow patterns in crowds is still largely unaddressed.&#10;&#10;Interpreting the dynamical systems governing the motion of groups of tracked objects is, however, an important aspect of experimental analysis of fluids, where decades of research has been devoted to extracting physical meaning from sequences of illuminated particle images. The most common method for measuring fluidic flow fields is particle image velocimetry (PIV) \cite{Raffel.Kompenhans_PIVbook_2018}, which characterizes velocity fields through windowed correlations between frames of artificial particles illuminated in the flow. Recently, however, algorithmic improvements have made Lagrangian particle tracking (LPT) \cite{Schanz.Schroder_ShakeTheBox_2016, SchroderSchanz_PTVReview_2023} a popular approach to flow field measurement. Instead of examining the aggregate motion of particles in a window, trajectories of individual particles are traced and flow fields interpolated. &#10;{While PIV and LPT are robust and effective for analyzing flow fields in highly controlled experiments, there remains a need for measurement technologies that leverage naturally occurring and imperfect tracers (in the sense that they may be sampled from a broad distribution) and for methods which can be implemented successful using non-specialist imaging equipment. The current state-of-the-art methods in flow measurement require expensive instrumentation and the addition of artificial tracers in order to be viable.}&#10;&#10;Identifying dominant flow structures from the LPT particle trajectories is accomplished through the theory of Lagrangian coherent structures (LCS) \cite{Haller_LCS_2015, AllshousePeacock_LagrangianBasedMethods_2015}. Analyses in this field are typically designed to be objective \cite{Haller_LCStextbook_2023, Gurtin.Anand_ContinuumMechanics_2010}, such that the relative motion of the observer to the flow does not influence the results. Because of this property and others, LCS have been influential for many applications in fluids literature, including forecasting of pollution patterns \cite{OlascoagaHaller_ForecastingPollutionPatterns_2012, Nolan.Ross_app.bio_2020}, understanding biological feeding mechanisms \cite{PengDabiri_app.bio_2009}, characterizing heart flows \cite{Shadden.Gerbeau_app.bio_2010}, and engineering improved fluid systems \cite{Ahmed.Hanifatu_app.aero_2023}. &#10;&#10;{Many LCS analyses are based on high-resolution computation of flow gradients~\cite{AllshousePeacock_LagrangianBasedMethods_2015}. This can pose serious limitations when working with naturally observed systems, where representative tracers are often sparsely distributed. Some methods approximate true gradient-based metrics using few or even single trajectories~\cite{Haller.Encinas-Bartos_Quasi-ObjectiveDiagnostics_2021, Encinos-Bartos.Haller_objectiveEddyViz_2022}. Recently, a regression-based approach to gradient estimation, known as Lagrangian gradient regression (LGR), was introduced to approximate complete gradients from sparse trajectories (\cite{Harms.McKeon_LGR_2023}, under review). LGR enables typical LCS analyses and velocity gradient analyses from tracer distribution densities that are reasonable for naturally observed systems. In the original paper, the algorithm performance was demonstrated on numerical benchmark data with known tracer trajectories.}&#10;&#10;{This work leverages modern capabilities in computer vision in tandem with LGR to provide a framework for extracting dynamical information beyond velocity from groups of observed objects. Specifically, large vision models are used to identify objects of interest and record their motion as a series of trajectories. These are provided as inputs to the Lagrangian gradient regression (LGR) algorithm which enables reliable gradient estimation from sparse trajectories such as are characteristic of MOT studies. Using the flow gradients, quantitative information of the flow including rotation and deformation rates for instantaneous and finite times can be computed. While such gradient-based analyses have been useful in fluidic applications \cite{Haller_LCS_2015, AllshousePeacock_LagrangianBasedMethods_2015}, this work aims to extend their use to real-world scenarios where dynamical object classes and the scenes they inhabit can be visually complex and difficult to analyze by standard approaches.}&#10;&#10;Specific attributes of the proposed method include:&#10;\begin{itemize}&#10; \item The technique reveals physically interpretable dynamical structures such as transport barriers and regions of coherent rotation directly from groups of tracked objects. &#10; \item The approach is general, and therefore it can be applied to any system that is characterized by the motion of multiple objects captured in sequences of images. Moreover, the method is effective even when the spacing of observed tracers is sparse. &#10; \item The three stages of the approach---detection, tracking, and sparse gradient estimation---are modular, which allows new developments in technology to be seamlessly integrated into the process.&#10; \item Because multiple classes may be detected in a sequence, dynamical features in groups of disparate object classes can be simultaneously identified. &#10; \item The tool is built to enable objective analyses, for which the motion of the observer (drone, vehicle, etc.) will not alter the results. &#10;\end{itemize}&#10;&#10;The rest of the paper is organized as follows: &#10;{Section \ref{sec:detection and tracking} discusses the modern vision tools which are implemented to identify the sets of object trajectories required by LGR. Then, section\ref{sec:estimating dynamical features from trajectories} provides the theoretical framework for LGR, which is used to identify dynamical features from the sparse trajectories which MOT detects}&#10;. Because appropriate benchmarks are not available for this type of analysis, the tool is demonstrated on a case study using data from an experimental debris flow in section\ref{sec:case study: experimental debris flow} and on field data in section\ref{sec:field test: pond debris}, with discussion in section\ref{sec:discussion}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Physics, Motion Analysis, Dynamical Systems, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2202.07474" label="2202.07474">
-        <attvalues>
-          <attvalue for="0" value="Do Lessons from Metric Learning Generalize to Image-Caption Retrieval?" />
-          <attvalue for="1" value="  The triplet loss with semi-hard negatives has become the de facto choice for&#10;image-caption retrieval (ICR) methods that are optimized from scratch. Recent&#10;progress in metric learning has given rise to new loss functions that&#10;outperform the triplet loss on tasks such as image retrieval and representation&#10;learning. We ask whether these findings generalize to the setting of ICR by&#10;comparing three loss functions on two ICR methods. We answer this question&#10;negatively: the triplet loss with semi-hard negative mining still outperforms&#10;newly introduced loss functions from metric learning on the ICR task. To gain a&#10;better understanding of these outcomes, we introduce an analysis method to&#10;compare loss functions by counting how many samples contribute to the gradient&#10;w.r.t. the query representation during optimization. We find that loss&#10;functions that result in lower evaluation scores on the ICR task, in general,&#10;take too many (non-informative) samples into account when computing a gradient&#10;w.r.t. the query representation, which results in sub-optimal performance. The&#10;triplet loss with semi-hard negatives is shown to outperform the other loss&#10;functions, as it only takes one (hard) negative into account when computing the&#10;gradient.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Given a query item in one modality, cross-modal retrieval is the task of retrieving similar items in another modality~\cite{zeng2020deep}. &#10;We focus on \acfi{ICR}~\cite{li2019visual, diao2021similarity, lee2018stacked,verma-2020-using}. &#10;For the ICR task, given an image or a caption as a query, systems have to retrieve the positive (e.g., matching or similar) item(s) in the other modality.&#10;Most ICR methods work with a separate encoder for each modality to map the input data to a representation in a shared latent space~\cite{diao2021similarity, li2019visual, faghri2018vse++, jia2021scaling, lee2018stacked}. &#10;The encoders are optimized by using a contrastive-loss criterion, so as to enforce a high degree of similarity between representations of matching items in the latent space.&#10;For retrieval, a similarity score between a query and each candidate in a candidate set is computed to produce a ranking with the top-$k$ best matching items. &#10;A lot of recent work on ICR relies on &#10;\begin{enumerate*}[label=(\arabic*)]&#10;&#09;\item pre-training on large amounts of data \cite{jia2021scaling, radford2021learning, li2020oscar}, and &#10;&#09;\item more sophisticated (and data-hungry) model architectures \cite{messina2021transformer, faghri2018vse++, li2019visual, diao2021similarity, lee2018stacked, chen2020intriguing}.&#10;\end{enumerate*}&#10;However, pre-training on large-scale datasets is not always an option, either due to a lack of compute power, a lack of data, or both. &#10;Hence, it is important to continue to develop effective ICR methods that only rely on a modest amount of data.&#10;&#10;To learn the similarity between a query and candidate representations, most ICR work relies on the standard Triplet loss with semi-hard negatives (Triplet SH)~\cite{faghri2018vse++, li2019visual, messina2021transformer, diao2021similarity, lee2018stacked, chen2020intriguing, chen2020imram} or on the cross-entropy based NT-Xent~\cite{chen2020adaptive, jia2021scaling} loss.&#10;In metric learning, the focus is on loss functions that result in more accurate item representations (in terms of a given evaluation metric) that can distinguish between similar and dissimilar items in a low-dimensional latent space~\cite{musgrave2020metric}.&#10;There has been important progress in metric learning, with the introduction of new loss functions that result in better evaluation scores on a specific (evaluation) task. &#10;For example SmoothAP~\cite{brown2020smooth}, it is a smooth approximation of the discrete evaluation metric Average Precision. By using SmoothAP, a retrieval method can be optimized with a discrete ranking evaluation metric and can handle multiple positive candidates simultaneously, which is not possible for the standard Triplet loss. &#10;Loss functions such as SmoothAP narrow the gap between the training setting and a discrete evaluation objective and thereby improve evaluation scores. &#10;&#10;Research goal.&#10;Most metric learning functions work with general representations of similar/dissimilar candidates and, in principle, there is no clear argument why obtained results on a specific task/method should not generalize to other tasks or methods.&#10;Hence, can newly introduced metric learning approaches, that is, alternative loss functions, be used to increase the performance of ICR methods? &#10;We compare three loss function for the ICR task: &#10;\begin{enumerate*}[label=(\arabic*)]&#10;\item the Triplet loss \cite{kiros2014unifying}, including semi-hard negative mining, &#10;\item NT-Xent loss~\cite{chen2020simple}, and&#10;\item SmoothAP~\cite{brown2020smooth}.&#10;\end{enumerate*}&#10;We expect SmoothAP to result in the highest performance based on the findings in in the context of image retrieval \cite{brown2020smooth} and in representation learning~\cite{varamesh2020self}.&#10;&#10;Main findings.&#10;Following~\cite{musgrave2020metric}, we evaluate the three loss functions on fixed methods, with different datasets, and with a fixed training regime (i.e., training hyper-parameters) to verify which loss function uses the given training data as effectively as possible.&#10;Surprisingly, the lessons from metric learning do not generalize to ICR.&#10;The Triplet loss with semi-hard negative mining still outperforms the other loss functions that we consider. &#10;The promising results obtained by SmoothAP and the NT-Xent loss in other fields do not generalize to the ICR task. &#10;&#10;To get a better grasp of this unexpected outcome, we propose counting contributing samples (COCOS), a method for analyzing contrastive loss functions. &#10;The gradient w.r.t.\ the query for the Triplet loss, NT-Xent and SmoothAP can be formulated as a sum over the representations of the positive and negative candidates in the training batch. &#10;The main difference between the loss functions lies in the number of samples used when computing the gradient w.r.t.\ the query and how each sample is weighted.&#10;We compare loss functions by counting how many samples contribute to the gradient w.r.t.\ the query representation at their convergence points. &#10;This yields an explanation of why one loss function outperforms another on the ICR task.&#10;&#10;Main contributions.&#10;\begin{enumerate*}[label=(\arabic*)]&#10;&#09;\item We experimentally compare three loss functions from the metric learning domain to determine if promising results from metric learning generalize to the ICR task, and find that the Triplet loss semi-hard (SH) still results in the highest evaluation scores.&#10;&#09;\item We propose COCOS, a way of analyzing contrastive loss functions, by defining a count that tells us how many candidates in the batch contribute to the gradient w.r.t.\ the query. On average, the best performing loss function takes at most one (semi-hard) negative sample into account when computing the gradient.&#10;\end{enumerate*}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Metric Learning, Computer Science, Image Retrieval, Loss Functions" />
-        </attvalues>
-      </node>
-      <node id="1908.03737" label="1908.03737">
-        <attvalues>
-          <attvalue for="0" value="Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual&#10;  Cross-modal Retrieval" />
-          <attvalue for="1" value="  Cross-modal retrieval aims to retrieve data in one modality by a query in&#10;another modality, which has been a very interesting research issue in the field&#10;of multimedia, information retrieval, and computer vision, and database. Most&#10;existing works focus on cross-modal retrieval between text-image, text-video,&#10;and lyrics-audio.Little research addresses cross-modal retrieval between audio&#10;and video due to limited audio-video paired datasets and semantic information.&#10;The main challenge of audio-visual cross-modal retrieval task focuses on&#10;learning joint embeddings from a shared subspace for computing the similarity&#10;across different modalities, where generating new representations is to&#10;maximize the correlation between audio and visual modalities space. In this&#10;work, we propose a novel deep triplet neural network with cluster canonical&#10;correlation analysis(TNN-C-CCA), which is an end-to-end supervised learning&#10;architecture with audio branch and video branch.We not only consider the&#10;matching pairs in the common space but also compute the mismatching pairs when&#10;maximizing the correlation. In particular, two significant contributions are&#10;made: i) a better representation by constructing deep triplet neural network&#10;with triplet loss for optimal projections can be generated to maximize&#10;correlation in the shared subspace. ii) positive examples and negative examples&#10;are used in the learning stage to improve the capability of embedding learning&#10;between audio and video. Our experiment is run over 5-fold cross-validation,&#10;where average performance is applied to demonstrate the performance of&#10;audio-video cross-modal retrieval. The experimental results achieved on two&#10;different audio-visual datasets show the proposed learning architecture with&#10;two branches outperforms existing six CCA-based methods and four&#10;state-of-the-art based cross-modal retrieval methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.02701" label="1909.02701">
-        <attvalues>
-          <attvalue for="0" value="Visual Semantic Reasoning for Image-Text Matching" />
-          <attvalue for="1" value="  Image-text matching has been a hot research topic bridging the vision and&#10;language areas. It remains challenging because the current representation of&#10;image usually lacks global semantic concepts as in its corresponding text&#10;caption. To address this issue, we propose a simple and interpretable reasoning&#10;model to generate visual representation that captures key objects and semantic&#10;concepts of a scene. Specifically, we first build up connections between image&#10;regions and perform reasoning with Graph Convolutional Networks to generate&#10;features with semantic relationships. Then, we propose to use the gate and&#10;memory mechanism to perform global semantic reasoning on these&#10;relationship-enhanced features, select the discriminative information and&#10;gradually generate the representation for the whole scene. Experiments validate&#10;that our method achieves a new state-of-the-art for the image-text matching on&#10;MS-COCO and Flickr30K datasets. It outperforms the current best method by 6.8%&#10;relatively for image retrieval and 4.8% relatively for caption retrieval on&#10;MS-COCO (Recall@1 using 1K test set). On Flickr30K, our model improves image&#10;retrieval by 12.6% relatively and caption retrieval by 5.8% relatively&#10;(Recall@1). Our code is available at https://github.com/KunpengLi1994/VSRN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.00708" label="2003.00708">
-        <attvalues>
-          <attvalue for="0" value="Using Image Captions and Multitask Learning for Recommending Query&#10;  Reformulations" />
-          <attvalue for="1" value="  Interactive search sessions often contain multiple queries, where the user&#10;submits a reformulated version of the previous query in response to the&#10;original results. We aim to enhance the query recommendation experience for a&#10;commercial image search engine. Our proposed methodology incorporates current&#10;state-of-the-art practices from relevant literature -- the use of&#10;generation-based sequence-to-sequence models that capture session context, and&#10;a multitask architecture that simultaneously optimizes the ranking of results.&#10;We extend this setup by driving the learning of such a model with captions of&#10;clicked images as the target, instead of using the subsequent query within the&#10;session. Since these captions tend to be linguistically richer, the&#10;reformulation mechanism can be seen as assistance to construct more descriptive&#10;queries. In addition, via the use of a pairwise loss for the secondary ranking&#10;task, we show that the generated reformulations are more diverse.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.01368" label="2101.01368">
-        <attvalues>
-          <attvalue for="0" value="Similarity Reasoning and Filtration for Image-Text Matching" />
-          <attvalue for="1" value="  Image-text matching plays a critical role in bridging the vision and&#10;language, and great progress has been made by exploiting the global alignment&#10;between image and sentence, or local alignments between regions and words.&#10;However, how to make the most of these alignments to infer more accurate&#10;matching scores is still underexplored. In this paper, we propose a novel&#10;Similarity Graph Reasoning and Attention Filtration (SGRAF) network for&#10;image-text matching. Specifically, the vector-based similarity representations&#10;are firstly learned to characterize the local and global alignments in a more&#10;comprehensive manner, and then the Similarity Graph Reasoning (SGR) module&#10;relying on one graph convolutional neural network is introduced to infer&#10;relation-aware similarities with both the local and global alignments. The&#10;Similarity Attention Filtration (SAF) module is further developed to integrate&#10;these alignments effectively by selectively attending on the significant and&#10;representative alignments and meanwhile casting aside the interferences of&#10;non-meaningful alignments. We demonstrate the superiority of the proposed&#10;method with achieving state-of-the-art performances on the Flickr30K and MSCOCO&#10;datasets, and the good interpretability of SGR and SAF modules with extensive&#10;qualitative experiments and analyses.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.09144" label="2004.09144">
-        <attvalues>
-          <attvalue for="0" value="Transformer Reasoning Network for Image-Text Matching and Retrieval" />
-          <attvalue for="1" value="  Image-text matching is an interesting and fascinating task in modern AI&#10;research. Despite the evolution of deep-learning-based image and text&#10;processing systems, multi-modal matching remains a challenging problem. In this&#10;work, we consider the problem of accurate image-text matching for the task of&#10;multi-modal large-scale information retrieval. State-of-the-art results in&#10;image-text matching are achieved by inter-playing image and text features from&#10;the two different processing pipelines, usually using mutual attention&#10;mechanisms. However, this invalidates any chance to extract separate visual and&#10;textual features needed for later indexing steps in large-scale retrieval&#10;systems. In this regard, we introduce the Transformer Encoder Reasoning Network&#10;(TERN), an architecture built upon one of the modern relationship-aware&#10;self-attentive architectures, the Transformer Encoder (TE). This architecture&#10;is able to separately reason on the two different modalities and to enforce a&#10;final common abstract concept space by sharing the weights of the deeper&#10;transformer layers. Thanks to this design, the implemented network is able to&#10;produce compact and very rich visual and textual features available for the&#10;successive indexing step. Experiments are conducted on the MS-COCO dataset, and&#10;we evaluate the results using a discounted cumulative gain metric with&#10;relevance computed exploiting caption similarities, in order to assess possibly&#10;non-exact but relevant search results. We demonstrate that on this metric we&#10;are able to achieve state-of-the-art results in the image retrieval task. Our&#10;code is freely available at https://github.com/mesnico/TERN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.05612" label="1707.05612">
-        <attvalues>
-          <attvalue for="0" value="VSE++: Improving Visual-Semantic Embeddings with Hard Negatives" />
-          <attvalue for="1" value="  We present a new technique for learning visual-semantic embeddings for&#10;cross-modal retrieval. Inspired by hard negative mining, the use of hard&#10;negatives in structured prediction, and ranking loss functions, we introduce a&#10;simple change to common loss functions used for multi-modal embeddings. That,&#10;combined with fine-tuning and use of augmented data, yields significant gains&#10;in retrieval performance. We showcase our approach, VSE++, on MS-COCO and&#10;Flickr30K datasets, using ablation studies and comparisons with existing&#10;methods. On MS-COCO our approach outperforms state-of-the-art methods by 8.8%&#10;in caption retrieval and 11.3% in image retrieval (at R@1).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.03669" label="2003.03669">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Offline Quintuplet Loss for Image-Text Matching" />
-          <attvalue for="1" value="  Existing image-text matching approaches typically leverage triplet loss with&#10;online hard negatives to train the model. For each image or text anchor in a&#10;training mini-batch, the model is trained to distinguish between a positive and&#10;the most confusing negative of the anchor mined from the mini-batch (i.e.&#10;online hard negative). This strategy improves the model's capacity to discover&#10;fine-grained correspondences and non-correspondences between image and text&#10;inputs. However, the above approach has the following drawbacks: (1) the&#10;negative selection strategy still provides limited chances for the model to&#10;learn from very hard-to-distinguish cases. (2) The trained model has weak&#10;generalization capability from the training set to the testing set. (3) The&#10;penalty lacks hierarchy and adaptiveness for hard negatives with different&#10;&quot;hardness&quot; degrees. In this paper, we propose solutions by sampling negatives&#10;offline from the whole training set. It provides &quot;harder&quot; offline negatives&#10;than online hard negatives for the model to distinguish. Based on the offline&#10;hard negatives, a quintuplet loss is proposed to improve the model's&#10;generalization capability to distinguish positives and negatives. In addition,&#10;a novel loss function that combines the knowledge of positives, offline hard&#10;negatives and online hard negatives is created. It leverages offline hard&#10;negatives as the intermediary to adaptively penalize them based on their&#10;distance relations to the anchor. We evaluate the proposed training approach on&#10;three state-of-the-art image-text models on the MS-COCO and Flickr30K datasets.&#10;Significant performance improvements are observed for all the models, proving&#10;the effectiveness and generality of our approach. Code is available at&#10;https://github.com/sunnychencool/AOQ&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.12163" label="2007.12163">
-        <attvalues>
-          <attvalue for="0" value="Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval" />
-          <attvalue for="1" value="  Optimising a ranking-based metric, such as Average Precision (AP), is&#10;notoriously challenging due to the fact that it is non-differentiable, and&#10;hence cannot be optimised directly using gradient-descent methods. To this end,&#10;we introduce an objective that optimises instead a smoothed approximation of&#10;AP, coined Smooth-AP. Smooth-AP is a plug-and-play objective function that&#10;allows for end-to-end training of deep networks with a simple and elegant&#10;implementation. We also present an analysis for why directly optimising the&#10;ranking based metric of AP offers benefits over other deep metric learning&#10;losses. We apply Smooth-AP to standard retrieval benchmarks: Stanford Online&#10;products and VehicleID, and also evaluate on larger-scale datasets: INaturalist&#10;for fine-grained category retrieval, and VGGFace2 and IJB-C for face retrieval.&#10;In all cases, we improve the performance over the state-of-the-art, especially&#10;for larger-scale datasets, thus demonstrating the effectiveness and scalability&#10;of Smooth-AP to real-world scenarios.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.2539" label="1411.2539">
-        <attvalues>
-          <attvalue for="0" value="Unifying Visual-Semantic Embeddings with Multimodal Neural Language&#10;  Models" />
-          <attvalue for="1" value="  Inspired by recent advances in multimodal learning and machine translation,&#10;we introduce an encoder-decoder pipeline that learns (a): a multimodal joint&#10;embedding space with images and text and (b): a novel language model for&#10;decoding distributed representations from our space. Our pipeline effectively&#10;unifies joint image-text embedding models with multimodal neural language&#10;models. We introduce the structure-content neural language model that&#10;disentangles the structure of a sentence to its content, conditioned on&#10;representations produced by the encoder. The encoder allows one to rank images&#10;and sentences while the decoder can generate novel descriptions from scratch.&#10;Using LSTM to encode sentences, we match the state-of-the-art performance on&#10;Flickr8K and Flickr30K without using object detections. We also set new best&#10;results when using the 19-layer Oxford convolutional network. Furthermore we&#10;show that with linear encoders, the learned embedding space captures multimodal&#10;regularities in terms of vector space arithmetic e.g. *image of a blue car* -&#10;&quot;blue&quot; + &quot;red&quot; is near images of red cars. Sample captions generated for 800&#10;images are made available for comparison.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.07258" label="2010.07258">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Ranking for Representation Learning" />
-          <attvalue for="1" value="  We present a new framework for self-supervised representation learning by&#10;formulating it as a ranking problem in an image retrieval context on a large&#10;number of random views (augmentations) obtained from images. Our work is based&#10;on two intuitions: first, a good representation of images must yield a&#10;high-quality image ranking in a retrieval task; second, we would expect random&#10;views of an image to be ranked closer to a reference view of that image than&#10;random views of other images. Hence, we model representation learning as a&#10;learning to rank problem for image retrieval. We train a representation encoder&#10;by maximizing average precision (AP) for ranking, where random views of an&#10;image are considered positively related, and that of the other images&#10;considered negatives. The new framework, dubbed S2R2, enables computing a&#10;global objective on multiple views, compared to the local objective in the&#10;popular contrastive learning framework, which is calculated on pairs of views.&#10;In principle, by using a ranking criterion, we eliminate reliance on&#10;object-centric curated datasets. When trained on STL10 and MS-COCO, S2R2&#10;outperforms SimCLR and the clustering-based contrastive learning model, SwAV,&#10;while being much simpler both conceptually and at implementation. On MS-COCO,&#10;S2R2 outperforms both SwAV and SimCLR with a larger margin than on STl10. This&#10;indicates that S2R2 is more effective on diverse scenes and could eliminate the&#10;need for an object-centric large training dataset for self-supervised&#10;representation learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.04833" label="2201.04833">
-        <attvalues>
-          <attvalue for="0" value="SnapshotNet: Self-supervised Feature Learning for Point Cloud Data&#10;  Segmentation Using Minimal Labeled Data" />
-          <attvalue for="1" value="  Manually annotating complex scene point cloud datasets is both costly and&#10;error-prone. To reduce the reliance on labeled data, a new model called&#10;SnapshotNet is proposed as a self-supervised feature learning approach, which&#10;directly works on the unlabeled point cloud data of a complex 3D scene. The&#10;SnapshotNet pipeline includes three stages. In the snapshot capturing stage,&#10;snapshots, which are defined as local collections of points, are sampled from&#10;the point cloud scene. A snapshot could be a view of a local 3D scan directly&#10;captured from the real scene, or a virtual view of such from a large 3D point&#10;cloud dataset. Snapshots could also be sampled at different sampling rates or&#10;fields of view (FOVs), thus multi-FOV snapshots, to capture scale information&#10;from the scene. In the feature learning stage, a new pre-text task called&#10;multi-FOV contrasting is proposed to recognize whether two snapshots are from&#10;the same object or not, within the same FOV or across different FOVs. Snapshots&#10;go through two self-supervised learning steps: the contrastive learning step&#10;with both part and scale contrasting, followed by a snapshot clustering step to&#10;extract higher level semantic features. Then a weakly-supervised segmentation&#10;stage is implemented by first training a standard SVM classifier on the learned&#10;features with a small fraction of labeled snapshots. The trained SVM is used to&#10;predict labels for input snapshots and predicted labels are converted into&#10;point-wise label assignments for semantic segmentation of the entire scene&#10;using a voting procedure. The experiments are conducted on the Semantic3D&#10;dataset and the results have shown that the proposed method is capable of&#10;learning effective features from snapshots of complex scene data without any&#10;labels. Moreover, the proposed method has shown advantages when comparing to&#10;the SOA method on weakly-supervised point cloud semantic segmentation.&#10;" />
-          <attvalue for="2" value="&#10;Studies on 3D point cloud data have been gaining momentum in the field of computer vision.&#10;Deep neural networks such as PointNet\cite{pointnet}, DGCNN\cite{DGCNN} have been proposed for better performances on point cloud related tasks, with the help of larger datasets such as the ModelNet\cite{modelnet} and Semantic3D\cite{semantic3d}.&#10;The collective effort between deep neural networks and dedicated datasets continues to push the state of the art performance on the point cloud object classification.&#10;&#10;On the other hand, point cloud semantic segmentation is of great interests in the applications of autonomous driving, robotics and remote sensing\cite{xie2020linking}.&#10;So far most of the deep learning driven point cloud semantic segmentation methods follow the supervised workflow, which requires densely labeled datasets, such as the 1.6 million points Oakland Dataset\cite{oakland}, the 215 million points Stanford Large-scale 3D Indoor Spaces Dataset (S3DIS) dataset\cite{S3DIS} and the 4 billion points Semantic3D\cite{semantic3d}.&#10;However, annotating large scale datasets is at a very high cost both in time and human labors.&#10;This issue is becoming more prominent in applications such as hazard assessment where drive-by and fly-by LiDAR mapping systems have been used to collect massive windstorm damage data sets in recent hurricane events \cite{lidar, terrestrial, gong2013mobile, hu2018advancing}. The fact that LiDAR is starting to be integrated into smaller mobile devices\cite{ipad}, which could lead to a boom in the scale of real life complex point cloud data.&#10;&#10;To alleviate the dependence on the labels of large datasets, unsupervised learning methods have drawn increasing attention.&#10;Among the unsupervised methods, one form known as ``self-supervised learning&quot; has been popular in the studies of image data understanding.&#10;This self-supervised approach has found success in designing ``pretext&quot; tasks, such as jigsaw puzzle reassembly\cite{jigsaw}, image clustering\cite{imageClustering} and image rotation prediction\cite{rotation} etc, by training deep learning models for feature extraction without labels being involved.&#10;Based on the idea of solving pretext tasks, we previously developed the model of Contrast-ClusterNet\cite{contrastnet}, which works on unlabeled point cloud datasets by part contrasting and object clustering.&#10;While this work has shown comparable performance to its supervised counterparts on synthetic point cloud objects classification, it inherits the problem of other pretext-driven models used on image data: pretext tasks must be defined regarding the prior knowledge embedded in the data.&#10;In the context of point cloud understanding, the part contrasting and object clustering tasks assume the input data are well separated as individual objects. &#10;This assumption limits the model's power on real life scene data or where 3D data of single objects cannot be easily obtained.&#10;We addressed this limitation posed on scene point cloud by proposing a snapshot-based method\cite{visapp21}, which captures local clusters of points called snapshots from the scene as input samples to fulfill the tasks of part contrasting and object clustering.&#10;Formally, a snapshot is defined as a collection of points, sampled from a point cloud scene, without knowing their labels or even assuming they are from the the same objects. (Figure \ref{fig:snapshots}). &#10;It could be a real view of a local 3D scan directly captured from the real scene, or a &quot;virtual&quot; view of such a local 3D scan from a large 3D point cloud dataset.&#10;The effectiveness of this method is evaluated and approved by conducting classification on the captured snapshots with a single FOV in \cite{visapp21}. In this paper, we extend the idea to capture multi-FOV snapshots to further improve the performance of classification.&#10;&#10;Besides from the limitation of making assumptions on the training data, another weakness of the Contrast-ClusterNet is that dense labels are still needed for the downstream tasks.&#10;The full supervision involved in the object classification contradicts the main goal of self-supervision, that to save labeling efforts on training data.&#10;To extend the idea of reducing labeled data usage to the downstream tasks, we seek solutions from weak supervision.&#10;&#10;Therefore, based on the two pieces of our previous work, we further propose the SnapshotNet, which integrates multi-FOV snapshot generation, contrastive feature learning, and a weakly-supervised technique for point-wise scene segmentation using a voting mechanism. &#10;First of all, inspired by the observation that, humans are able to distinguish objects at different scales, we present a new pre-text task for contrastive learning, namely multi-FOV contrasting.&#10;When capturing a sample, we take multiple snapshots in different field-of-views (FOVs).&#10;Assuming these multi-FOV snapshots are small enough so they still represent the same object, the task of scale contrasting is to consider whether two snapshots, within one FOV or across multiple FOVs, are of the same object or not. Thus the multi-FOV contrasting includes two parts: part contrasting and scale contrasting.&#10;We will compare the performance of the scale contrasting against the part contrasting, as well as the combination of the two - the multi-FOV contrasting.&#10;&#10;For the complete model pipeline, the captured snapshots first go through a two-step self-supervised pipeline using ContrastNet and ClusterNet consecutively for feature learning. &#10;Then a weakly-supervised approach is implemented by training an SVM classifier on the learned features of a small portion of labeled snapshot samples (mostly cluster centers) combined with the samples automatically labeled from the clusters of the samples generated in the pipeline.&#10;Finally, the entire 3D point cloud scene is repeatedly scanned as random snapshots to go through the feature extractor and classifier. &#10;The predicted snapshot-wise label is assigned to each point of a snapshot, followed by a voting-based mechanism for the final label for each point.&#10;&#10;This work makes the following contributions:&#10;&#10;(1) We propose a new contrastive learning method called multi-FOV contrasting, by leveraging point cloud samples at different scales. This task devotes on predicting if two snapshots are of the same object, regardless of their sampling FOVs.&#10;&#10;(2) We develop a three-stage approach for semantic segmentation: snapshot generation, self-supervised feature learning, and point-wise segmentation by integrating multiple weakly-supervised classified results.&#10;&#10;(3) We study the &quot;purity&quot; of snapshots, and show that the self-supervised learning with impure snapshots can still effectively obtain highly useful semantic features for object classification and scene segmentation. This includes cases when some of the classes do not have well-sampled snapshots.&#10;&#10;(4) By using the learned features and clustering to obtain larger pseudo labels with a small number of labels (thus weakly-supervised) to train a simple classifier, we design a simple voting procedure to integrating labels of randomly sample snapshots, which leads to point-wise point cloud scene segmentation performance comparable to the state of the art weakly-supervised methods.&#10;&#10;The rest of the paper is organized as the following.&#10;Section \ref{sec:relatedwork} discusses related work on self-supervised learning methods on point cloud, and point cloud semantic segmentation.&#10;Section \ref{sec:method} describes the theory and design of the SnapshotNet for self-supervised feature learning and point cloud semantic segmentation with fewer labeled data.&#10;Section \ref{sec:results} details the experimental results, including the designs and evaluations of data capturing, feature learning, and segmentation. &#10;Finally Section \ref{sec:conclusion} concludes the work with discussions of a few ideas for future work.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Geometric Computing, Computer Science, Point Cloud Analysis, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="1406.5670" label="1406.5670">
-        <attvalues>
-          <attvalue for="0" value="3D ShapeNets: A Deep Representation for Volumetric Shapes" />
-          <attvalue for="1" value="  3D shape is a crucial but heavily underutilized cue in today's computer&#10;vision systems, mostly due to the lack of a good generic shape representation.&#10;With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft&#10;Kinect), it is becoming increasingly important to have a powerful 3D shape&#10;representation in the loop. Apart from category recognition, recovering full 3D&#10;shapes from view-based 2.5D depth maps is also a critical part of visual&#10;understanding. To this end, we propose to represent a geometric 3D shape as a&#10;probability distribution of binary variables on a 3D voxel grid, using a&#10;Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the&#10;distribution of complex 3D shapes across different object categories and&#10;arbitrary poses from raw CAD data, and discovers hierarchical compositional&#10;part representations automatically. It naturally supports joint object&#10;recognition and shape completion from 2.5D depth maps, and it enables active&#10;object recognition through view planning. To train our 3D deep learning model,&#10;we construct ModelNet -- a large-scale 3D CAD model dataset. Extensive&#10;experiments show that our 3D deep representation enables significant&#10;performance improvement over the-state-of-the-arts in a variety of tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.03847" label="1704.03847">
-        <attvalues>
-          <attvalue for="0" value="Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark" />
-          <attvalue for="1" value="  This paper presents a new 3D point cloud classification benchmark data set&#10;with over four billion manually labelled points, meant as input for data-hungry&#10;(deep) learning methods. We also discuss first submissions to the benchmark&#10;that use deep convolutional neural networks (CNNs) as a work horse, which&#10;already show remarkable performance improvements over state-of-the-art. CNNs&#10;have become the de-facto standard for many tasks in computer vision and machine&#10;learning like semantic segmentation or object detection in images, but have no&#10;yet led to a true breakthrough for 3D point cloud labelling tasks due to lack&#10;of training data. With the massive data set presented in this paper, we aim at&#10;closing this data gap to help unleash the full potential of deep learning&#10;methods for 3D labelling tasks. Our semantic3D.net data set consists of dense&#10;point clouds acquired with static terrestrial laser scanners. It contains 8&#10;semantic classes and covers a wide range of urban outdoor scenes: churches,&#10;streets, railroad tracks, squares, villages, soccer fields and castles. We&#10;describe our labelling interface and show that our data set provides more dense&#10;and complete point clouds with much higher overall number of labelled points&#10;compared to those already available to the research community. We further&#10;provide baseline method descriptions and comparison between methods submitted&#10;to our online system. We hope semantic3D.net will pave the way for deep&#10;learning methods in 3D point cloud labelling to learn richer, more general 3D&#10;representations, and first submissions after only a few months indicate that&#10;this might indeed be the case.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.09246" label="1603.09246">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Visual Representations by Solving Jigsaw&#10;  Puzzles" />
-          <attvalue for="1" value="  In this paper we study the problem of image representation learning without&#10;human annotation. By following the principles of self-supervision, we build a&#10;convolutional neural network (CNN) that can be trained to solve Jigsaw puzzles&#10;as a pretext task, which requires no manual labeling, and then later repurposed&#10;to solve object classification and detection. To maintain the compatibility&#10;across tasks we introduce the context-free network (CFN), a siamese-ennead CNN.&#10;The CFN takes image tiles as input and explicitly limits the receptive field&#10;(or context) of its early processing units to one tile at a time. We show that&#10;the CFN includes fewer parameters than AlexNet while preserving the same&#10;semantic learning capabilities. By training the CFN to solve Jigsaw puzzles, we&#10;learn both a feature mapping of object parts as well as their correct spatial&#10;arrangement. Our experimental evaluations show that the learned features&#10;capture semantically relevant content. Our proposed method for learning visual&#10;representations outperforms state of the art methods in several transfer&#10;learning benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.11387" label="1811.11387">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Spatiotemporal Feature Learning via Video Rotation&#10;  Prediction" />
-          <attvalue for="1" value="  The success of deep neural networks generally requires a vast amount of&#10;training data to be labeled, which is expensive and unfeasible in scale,&#10;especially for video collections. To alleviate this problem, in this paper, we&#10;propose 3DRotNet: a fully self-supervised approach to learn spatiotemporal&#10;features from unlabeled videos. A set of rotations are applied to all videos,&#10;and a pretext task is defined as prediction of these rotations. When&#10;accomplishing this task, 3DRotNet is actually trained to understand the&#10;semantic concepts and motions in videos. In other words, it learns a&#10;spatiotemporal video representation, which can be transferred to improve video&#10;understanding tasks in small datasets. Our extensive experiments successfully&#10;demonstrate the effectiveness of the proposed framework on action recognition,&#10;leading to significant improvements over the state-of-the-art self-supervised&#10;methods. With the self-supervised pre-trained 3DRotNet from large datasets, the&#10;recognition accuracy is boosted up by 20.4% on UCF101 and 16.7% on HMDB51&#10;respectively, compared to the models trained from scratch.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.09153" label="2005.09153">
-        <attvalues>
-          <attvalue for="0" value="Associating Multi-Scale Receptive Fields for Fine-grained Recognition" />
-          <attvalue for="1" value="  Extracting and fusing part features have become the key of fined-grained&#10;image recognition. Recently, Non-local (NL) module has shown excellent&#10;improvement in image recognition. However, it lacks the mechanism to model the&#10;interactions between multi-scale part features, which is vital for fine-grained&#10;recognition. In this paper, we propose a novel cross-layer non-local (CNL)&#10;module to associate multi-scale receptive fields by two operations. First, CNL&#10;computes correlations between features of a query layer and all response&#10;layers. Second, all response features are weighted according to the&#10;correlations and are added to the query features. Due to the interactions of&#10;cross-layer features, our model builds spatial dependencies among multi-level&#10;layers and learns more discriminative features. In addition, we can reduce the&#10;aggregation cost if we set low-dimensional deep layer as query layer.&#10;Experiments are conducted to show our model achieves or surpasses&#10;state-of-the-art results on three benchmark datasets of fine-grained&#10;classification. Our codes can be found at github.com/FouriYe/CNL-ICIP2020.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Fine-grained image classification focuses on categorizing very similar subgroups, e.g. the Husky and Alaska in dogs. &#10;It has long been considered as a challenging task due to discriminative region localization and discriminative feature learning from those regions.&#10;Early works on localization-based methods often adopt part detectors trained on part annotations and then extract part-specific features for fine-grained classification \cite{xu2015augmenting, xiao2015application,simon2015neural,zhang2016picking}.&#10;However, these strongly-supervised methods heavily rely on manual object part annotations, which is too expensive to be prevalently applied in practice.&#10;Therefore, the attention mechanism~\cite{fu2017look,sun2018multi, ye2019unsupervised,ye2018daugan} have received increasing attention in recent researches, which can be implemented as one kind of weakly-supervised frameworks that do not need part annotations.&#10;\cite{fu2017look} proposes a recurrent attention convolutional neural network that progressively localizes discriminative areas and extracts features from coarse to fine scale.&#10;\cite{sun2018multi} proposes a soft attenion-based model that generates semantic region features.&#10;&#10;Recently, non-local (NL) module~\cite{wang2018non} is proposed to model the long-range dependencies in one layer, via self-attention mechanism to form an attention map.&#10;For each position, the NL module first computes the pairwise relations between current position and all positions, and then aggregates the features of all positions by weighted sum.&#10;The aggregated features are added to the feature of each position to form the output.&#10;\cite{yue2018compact,cao2019gcnet} show the NL module could bring excellent improvement in image recognition.&#10;However, for fine-grained classification task, multi-scale features are vital because object parts have various sizes and shapes in images~\cite{zhao2017diversified,luo2019cross,he2015spatial}.&#10;One NL module models the spatial dependencies in only one convolutional layer, in which neurons have fixed-size receptive fields.&#10;The mismatch between the sizes of receptive fields and parts of objects might undermines the feature extract.&#10;For example, as shown in Fig.\ref{fig:RF1} (a), an over-sized receptive fields contain more background region, and sequentially bring more noises.&#10;The noises brought by meaningless background have been proven to be harmful for classification~\cite{zhang2018occluded,luo2015multiview}.&#10;&#10;To address the issue, we propose a novel Cross-layer Non-Local module (CNL).&#10;We first decompose traditional non-local module into two basic components, query layer and response layer.&#10;Then, based on the observation that neurons in different layers has various receptive fields as shown in Fig.\ref{fig:RF1} (b), we associate one query layer with several response layers.&#10;Consequently, the query layer could learn more discriminative multi-scale feature from distant response layers.&#10;The explicitly learned correlations among all related layers improve the representation power of network.&#10;&#10;To the best of our knowledge, we are the first that explore the non-local operation into cross-layer feature fusion for fined-grained classification.&#10;We certify that the cross-layer operation could ease the computation complexity of associating receptive fields in theory.&#10;We evaluate our CNL method on three datasets of fine-grained classification, i.e. CUB-200-2011~\cite{wah2011caltech}, Stanford Dogs~\cite{khosla2011novel} and Stanford Cars~\cite{he2016deep}.&#10;Experimental results show that: (1) as a plug-and-play module, CNL needs less cost than NL module and other methods, but surpasses or achieves state-of-the-art performance, (2) CNL can adaptively focus on various parts in multi-scale.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Image Recognition, Mathematics, Artificial Intelligence, Feature Extraction, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1504.08289" label="1504.08289">
-        <attvalues>
-          <attvalue for="0" value="Neural Activation Constellations: Unsupervised Part Model Discovery with&#10;  Convolutional Networks" />
-          <attvalue for="1" value="  Part models of object categories are essential for challenging recognition&#10;tasks, where differences in categories are subtle and only reflected in&#10;appearances of small parts of the object. We present an approach that is able&#10;to learn part models in a completely unsupervised manner, without part&#10;annotations and even without given bounding boxes during learning. The key idea&#10;is to find constellations of neural activation patterns computed using&#10;convolutional neural networks. In our experiments, we outperform existing&#10;approaches for fine-grained recognition on the CUB200-2011, NA birds, Oxford&#10;PETS, and Oxford Flowers dataset in case no part or bounding box annotations&#10;are available and achieve state-of-the-art performance for the Stanford Dog&#10;dataset. We also show the benefits of neural constellation models as a data&#10;augmentation technique for fine-tuning. Furthermore, our paper unites the areas&#10;of generic and fine-grained classification, since our approach is suitable for&#10;both scenarios. The source code of our method is available online at&#10;http://www.inf-cv.uni-jena.de/part_discovery&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.13125" label="1810.13125">
-        <attvalues>
-          <attvalue for="0" value="Compact Generalized Non-local Network" />
-          <attvalue for="1" value="  The non-local module is designed for capturing long-range spatio-temporal&#10;dependencies in images and videos. Although having shown excellent performance,&#10;it lacks the mechanism to model the interactions between positions across&#10;channels, which are of vital importance in recognizing fine-grained objects and&#10;actions. To address this limitation, we generalize the non-local module and&#10;take the correlations between the positions of any two channels into account.&#10;This extension utilizes the compact representation for multiple kernel&#10;functions with Taylor expansion that makes the generalized non-local module in&#10;a fast and low-complexity computation flow. Moreover, we implement our&#10;generalized non-local method within channel groups to ease the optimization.&#10;Experimental results illustrate the clear-cut improvements and practical&#10;applicability of the generalized non-local module on both fine-grained object&#10;recognition and video classification. Code is available at:&#10;https://github.com/KaiyuYue/cgnl-network.pytorch.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.08572" label="1606.08572">
-        <attvalues>
-          <attvalue for="0" value="Diversified Visual Attention Networks for Fine-Grained Object&#10;  Classification" />
-          <attvalue for="1" value="  Fine-grained object classification is a challenging task due to the subtle&#10;inter-class difference and large intra-class variation. Recently, visual&#10;attention models have been applied to automatically localize the discriminative&#10;regions of an image for better capturing critical difference and demonstrated&#10;promising performance. However, without consideration of the diversity in&#10;attention process, most of existing attention models perform poorly in&#10;classifying fine-grained objects. In this paper, we propose a diversified&#10;visual attention network (DVAN) to address the problems of fine-grained object&#10;classification, which substan- tially relieves the dependency on&#10;strongly-supervised information for learning to localize discriminative regions&#10;compared with attentionless models. More importantly, DVAN explicitly pursues&#10;the diversity of attention and is able to gather discriminative information to&#10;the maximal extent. Multiple attention canvases are generated to extract&#10;convolutional features for attention. An LSTM recurrent unit is employed to&#10;learn the attentiveness and discrimination of attention canvases. The proposed&#10;DVAN has the ability to attend the object from coarse to fine granularity, and&#10;a dynamic internal representation for classification is built up by&#10;incrementally combining the information from different locations and scales of&#10;the image. Extensive experiments con- ducted on CUB-2011, Stanford Dogs and&#10;Stanford Cars datasets have demonstrated that the proposed diversified visual&#10;attention networks achieve competitive performance compared to the state-&#10;of-the-art approaches, without using any prior knowledge, user interaction or&#10;external resource in training or testing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.04412" label="1909.04412">
-        <attvalues>
-          <attvalue for="0" value="Cross-X Learning for Fine-Grained Visual Categorization" />
-          <attvalue for="1" value="  Recognizing objects from subcategories with very subtle differences remains a&#10;challenging task due to the large intra-class and small inter-class variation.&#10;Recent work tackles this problem in a weakly-supervised manner: object parts&#10;are first detected and the corresponding part-specific features are extracted&#10;for fine-grained classification. However, these methods typically treat the&#10;part-specific features of each image in isolation while neglecting their&#10;relationships between different images. In this paper, we propose Cross-X&#10;learning, a simple yet effective approach that exploits the relationships&#10;between different images and between different network layers for robust&#10;multi-scale feature learning. Our approach involves two novel components: (i) a&#10;cross-category cross-semantic regularizer that guides the extracted features to&#10;represent semantic parts and, (ii) a cross-layer regularizer that improves the&#10;robustness of multi-scale features by matching the prediction distribution&#10;across multiple layers. Our approach can be easily trained end-to-end and is&#10;scalable to large datasets like NABirds. We empirically analyze the&#10;contributions of different components of our approach and demonstrate its&#10;robustness, effectiveness and state-of-the-art performance on five benchmark&#10;datasets. Code is available at \url{https://github.com/cswluo/CrossX}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.4729" label="1406.4729">
-        <attvalues>
-          <attvalue for="0" value="Spatial Pyramid Pooling in Deep Convolutional Networks for Visual&#10;  Recognition" />
-          <attvalue for="1" value="  Existing deep convolutional neural networks (CNNs) require a fixed-size&#10;(e.g., 224x224) input image. This requirement is &quot;artificial&quot; and may reduce&#10;the recognition accuracy for the images or sub-images of an arbitrary&#10;size/scale. In this work, we equip the networks with another pooling strategy,&#10;&quot;spatial pyramid pooling&quot;, to eliminate the above requirement. The new network&#10;structure, called SPP-net, can generate a fixed-length representation&#10;regardless of image size/scale. Pyramid pooling is also robust to object&#10;deformations. With these advantages, SPP-net should in general improve all&#10;CNN-based image classification methods. On the ImageNet 2012 dataset, we&#10;demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures&#10;despite their different designs. On the Pascal VOC 2007 and Caltech101&#10;datasets, SPP-net achieves state-of-the-art classification results using a&#10;single full-image representation and no fine-tuning.&#10;  The power of SPP-net is also significant in object detection. Using SPP-net,&#10;we compute the feature maps from the entire image only once, and then pool&#10;features in arbitrary regions (sub-images) to generate fixed-length&#10;representations for training the detectors. This method avoids repeatedly&#10;computing the convolutional features. In processing test images, our method is&#10;24-102x faster than the R-CNN method, while achieving better or comparable&#10;accuracy on Pascal VOC 2007.&#10;  In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our&#10;methods rank #2 in object detection and #3 in image classification among all 38&#10;teams. This manuscript also introduces the improvement made for this&#10;competition.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.13458" label="2006.13458">
-        <attvalues>
-          <attvalue for="0" value="IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency" />
-          <attvalue for="1" value="  Multiple object tracking (MOT) is a crucial task in computer vision society.&#10;However, most tracking-by-detection MOT methods, with available detected&#10;bounding boxes, cannot effectively handle static, slow-moving and fast-moving&#10;camera scenarios simultaneously due to ego-motion and frequent occlusion. In&#10;this work, we propose a novel tracking framework, called &quot;instance-aware MOT&quot;&#10;(IA-MOT), that can track multiple objects in either static or moving cameras by&#10;jointly considering the instance-level features and object motions. First,&#10;robust appearance features are extracted from a variant of Mask R-CNN detector&#10;with an additional embedding head, by sending the given detections as the&#10;region proposals. Meanwhile, the spatial attention, which focuses on the&#10;foreground within the bounding boxes, is generated from the given instance&#10;masks and applied to the extracted embedding features. In the tracking stage,&#10;object instance masks are aligned by feature similarity and motion consistency&#10;using the Hungarian association algorithm. Moreover, object re-identification&#10;(ReID) is incorporated to recover ID switches caused by long-term occlusion or&#10;missing detection. Overall, when evaluated on the MOTS20 and KITTI-MOTS&#10;dataset, our proposed method won the first place in Track 3 of the BMTT&#10;Challenge in CVPR2020 workshops.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Multi-Object Tracking (MOT) leveraging detected bounding box locations has been the mainstream and well-studied, a.k.a., tracking-by-detection. Most tracking-by-detection methods firstly extract the objects' deep features from the detected bounding boxes. With these extracted features, a tracking algorithm is followed to associate the detections along consecutive frames. However, the bounding-box-based features contain significant background noises, which are usually redundant and would downgrade the tracking performance. &#10;Recently, Multi-Object Tracking and Segmentation (MOTS) \cite{voigtlaender2019mots} is posed as a new task to track multiple objects with the corresponding instance segmentation. MOTS is more informative and spatially precise, and the masks also introduce potential promotions to tracking accuracy. &#10;&#10;Typically, there are three key components in MOTS: object detection and segmentation, feature extraction, and multi-object tracking. Some related works are briefly reviewed as follows. Recent studies on object detection and segmentation achieve impressive performance with various advanced structures \cite{ren2015faster,he2017mask}. &#10;Especially, for mask generation in MOTS, Luiten et al. \cite{luiten2018premvos} present the Proposal-generation, Refinement and Merging for Video Object Segmentation algorithm (PReMVOS). First, a set of object segmentation mask proposals are generated from Mask R-CNN and refined by a refinement network frame-by-frame. Then, with the assistance of optical-flow and ReID information, these selected proposals are merged into pixel-wise object tracks over a video sequence. &#10;Feature extraction with deep neural networks is widely used in the appearance similarity comparison in tracking. FaceNet \cite{schroff2015facenet} proposes a metric learning strategy. Moreover, a multi-scaled pedestrian detector \cite{jde} is proposed to allow target detection and appearance features to be learned in a shared model. &#10;On top of that, several MOT methods are studied to obtain the object trajectories. Wang et al. \cite{wang2019exploit} propose the TrackletNet Tracking (TNT) framework that jointly considers appearance and geometric features to form a tracklet-based graph model. &#10;To incorporate segmentation into MOT, Voigtlaende et al. \cite{voigtlaender2019mots} propose a new baseline approach, Track R-CNN, which addresses detection, tracking, and segmentation via a unified convolutional neural network. Milan et al. \cite{milan2015joint} use superpixel information and consider detection and segmentation jointly with a conditional random field model. CAMOT \cite{ovsep2018track} exploits stereo information to perform mask-based tracking of generic objects on the KITTI dataset. &#10;&#10;Instance segmentation is trained and inferred in parallel with the bounding box detection in the previous studies. In this paper, to better utilize the instance masks for embedding feature extraction, we propose a novel MOTS framework, called instance-aware multi-object tracking (IA-MOT) with motion consistency (IA-MOT) to integrate the segmentation information with instance features. With the given detection results, the sequences are sent into a variant of Mask R-CNN \cite{he2017mask} to obtain the embedding features as region proposals of RPN. Here, to handle instance-aware features, we apply the spatial attention generated from the corresponding instance masks to weight more on the foreground of each bounding box. Second, the extracted instance-aware features are utilized in the following tracking module based on the Hungarian assignment algorithm. In this tracking module, object motion consistency, i.e., the similarity of object motion, including object sizes, moving direction, and moving speed, are jointly considered. Moreover, discontinuity caused by missing detection is recovered by the short-term retrieval (STR) module and object re-identification (ReID) is further incorporated to reduce ID switches caused by long-term occlusion. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Tracking, Deep Learning, Computer Science, Pattern Recognition" />
-        </attvalues>
-      </node>
-      <node id="1811.07258" label="1811.07258">
-        <attvalues>
-          <attvalue for="0" value="Exploit the Connectivity: Multi-Object Tracking with TrackletNet" />
-          <attvalue for="1" value="  Multi-object tracking (MOT) is an important and practical task related to&#10;both surveillance systems and moving camera applications, such as autonomous&#10;driving and robotic vision. However, due to unreliable detection, occlusion and&#10;fast camera motion, tracked targets can be easily lost, which makes MOT very&#10;challenging. Most recent works treat tracking as a re-identification (Re-ID)&#10;task, but how to combine appearance and temporal features is still not well&#10;addressed. In this paper, we propose an innovative and effective tracking&#10;method called TrackletNet Tracker (TNT) that combines temporal and appearance&#10;information together as a unified framework. First, we define a graph model&#10;which treats each tracklet as a vertex. The tracklets are generated by&#10;appearance similarity with CNN features and intersection-over-union (IOU) with&#10;epipolar constraints to compensate camera movement between adjacent frames.&#10;Then, for every pair of two tracklets, the similarity is measured by our&#10;designed multi-scale TrackletNet. Afterwards, the tracklets are clustered into&#10;groups which represent individual object IDs. Our proposed TNT has the ability&#10;to handle most of the challenges in MOT, and achieve promising results on MOT16&#10;and MOT17 benchmark datasets compared with other state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.07920" label="1712.07920">
-        <attvalues>
-          <attvalue for="0" value="Track, then Decide: Category-Agnostic Vision-based Multi-Object Tracking" />
-          <attvalue for="1" value="  The most common paradigm for vision-based multi-object tracking is&#10;tracking-by-detection, due to the availability of reliable detectors for&#10;several important object categories such as cars and pedestrians. However,&#10;future mobile systems will need a capability to cope with rich human-made&#10;environments, in which obtaining detectors for every possible object category&#10;would be infeasible. In this paper, we propose a model-free multi-object&#10;tracking approach that uses a category-agnostic image segmentation method to&#10;track objects. We present an efficient segmentation mask-based tracker which&#10;associates pixel-precise masks reported by the segmentation. Our approach can&#10;utilize semantic information whenever it is available for classifying objects&#10;at the track level, while retaining the capability to track generic unknown&#10;objects in the absence of such information. We demonstrate experimentally that&#10;our approach achieves performance comparable to state-of-the-art&#10;tracking-by-detection methods for popular object categories such as cars and&#10;pedestrians. Additionally, we show that the proposed method can discover and&#10;robustly track a large variety of other objects.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.12533" label="2201.12533">
-        <attvalues>
-          <attvalue for="0" value="Light field Rectification based on relative pose estimation" />
-          <attvalue for="1" value="  Hand-held light field (LF) cameras have unique advantages in computer vision&#10;such as 3D scene reconstruction and depth estimation. However, the related&#10;applications are limited by the ultra-small baseline, e.g., leading to the&#10;extremely low depth resolution in reconstruction. To solve this problem, we&#10;propose to rectify LF to obtain a large baseline. Specifically, the proposed&#10;method aligns two LFs captured by two hand-held LF cameras with a random&#10;relative pose, and extracts the corresponding row-aligned sub-aperture images&#10;(SAIs) to obtain an LF with a large baseline. For an accurate rectification, a&#10;method for pose estimation is also proposed, where the relative rotation and&#10;translation between the two LF cameras are estimated. The proposed pose&#10;estimation minimizes the degree of freedom (DoF) in the LF-point-LF-point&#10;correspondence model and explicitly solves this model in a linear way. The&#10;proposed pose estimation outperforms the state-of-the-art algorithms by&#10;providing more accurate results to support rectification. The significantly&#10;improved depth resolution in 3D reconstruction demonstrates the effectiveness&#10;of the proposed LF rectification.&#10;" />
-          <attvalue for="2" value="&#10;The commercialization of light field (LF) cameras, such as Lytro~\cite{Lytroorg} and Raytrix~\cite{Raytrixorg}, have accelerated the development of LF technologies. Unlike the classic pinhole camera, an LF camera can capture both spatial and angular information of rays in the environment by a single 2D snapshot, which is the reason why the LF camera has gained significant interest in computer vision. Typical applications of the LF camera include simultaneous localization and mapping (SLAM)~\cite{dong2013plenoptic,zeller2017calibration}, 3D reconstruction~\cite{johannsen2016layered,Wu_2017_CVPR,Nousias_2019_CVPR}, digital refocusing ~\cite{ng2005fourier}, virtual viewpoint synthesis~\cite{10.1145/2980179.2980251}, super resolution~\cite{Jin_2020_CVPR,meng2020high}, etc.&#10;&#10;One of the most important tasks of the LF camera is depth estimation~\cite{jeon2015accurate,tao2013depth,wanner2012globally,8985549,Huang_2017_ICCV,rogge2020depth,8263242}. Compared with traditional stereo image pairs, LFs extend disparity to a continuous space. This advantage is apparent when considering epipolar plane images (EPIs)~\cite{bolles1987epipolarp}. Due to a dense sampling in the angular direction (e.g., 15 15 sub-views in Lytro Illum), corresponding pixels of a scene point in sub-aperture images (SAIs) can be projected onto a slope line in EPIs, and line parameters can be encoded into dense stereo matching to obtain a more robust depth estimation~\cite{wanner2012globally}. However, a single LF has a weakness which cannot be ignored: the extremely small baseline between SAIs (e.g., 14 in Lytro Illum). With this structural imperfection, even if LF depth estimation methods~\cite{jeon2015accurate,8985549,Huang_2017_ICCV,rogge2020depth,8263242} achieve a high disparity resolution, a low depth resolution is still inevitable, which generally leads to a layered 3D reconstruction, as shown in Fig.~\ref{fig:intro}. To obtain an LF with a larger baseline, on one hand, using the camera array~\cite{wilburn2005high,1315176} is a candidate choice. However, it is complicated to align all cameras in the camera array and high in cost to reach the identical angular resolution as that of an LF camera. On the other hand, manually aligning two hand-held LF cameras is another straightforward solution, whereas it is also hard to make the image planes reside in the same plane and the rows of corresponding SAIs completely aligned. Hence it is critical to mathematically align two hand-held LF cameras with random relative pose to obtain an LF with a large baseline.&#10;&#10;In order to achieve an accurate alignment, the relative pose is first estimated. Relative pose estimation has attracted a lot attention in research. The most general framework~\cite{1211520} defined a generalized camera model as a collection of rays, and established the correspondences between rays intersecting the same 3D point, denoted as the generalized epipolar constraint (GEC). Taking all kinds of degenerated camera configurations into consideration, Li et al.~\cite{4587545} used a linear algorithm to solve the relative pose from the GEC efficiently without ambiguities, which was also applicable to LF cameras. Johannsen et al.~\cite{johannsen2015on} first introduced the GEC into the LF camera and considered the geometric constraints between projections within a single LF, which effectively utilized the characteristics of the LF camera.&#10;&#10;However, the rays reconstructed by the small baseline are very dense and easy to be corrupted by noises in the above mentioned methods~\cite{9320357}. Accordingly, Nousias et al.~\cite{9320357} proposed an LF projection matrix to represent the correspondence between LF features and 3D points, and used the direct linear transformation (DLT) to estimate the absolute pose of LF cameras. In our previous work~\cite{9694504}, the LF-point-LF-point correspondence (LLC) model was established, which described the correspondence between corresponding LF features (LF-points) from a pair of LFs. Rotation and translation were directly decoupled from the model and solved separately. In this paper, we explicitly solved the parameters in the LLC model, from which more accurate of relative pose is extracted. Based on the extracted relative pose, we establish a pairwise rays warping function, and construct the rectification matrixes and vectors to align LFs captured by two hand-held LF cameras so that an LF with a large baseline can be obtained. A subsequent quadrilinear interpolation is implemented to acquire the row-aligned SAI sets. By rectification, the dimension of searching space in depth estimation is reduced from two to one, which directly makes stereo matching in different LFs feasible. A commonly used depth estimation algorithm designed for a single LF captured by an LF camera is performed on the rectified LF to output the depth in a high depth resolution.&#10;&#10;The main contributions include:&#10;&#10;(1) We propose a linear approach without redundant variables to solve the LF-point-LF-point correspondence and recover the relative pose from it. Experimental results demonstrate that, compared with other state-of-the-art relative pose estimation algorithms, our estimated relative poses are both accurate and robust, which prepares for the proposed LF rectification.&#10;&#10;(2) To the best of our knowledge, our work is the first attempt to rectify two 4D LFs based on the estimated relative pose of LF cameras to obtain an LF with row-aligned SAIs and a larger baseline.&#10;&#10;(3) We build a rectification pipeline on LFs captured by two LF cameras which can support to generate 3D reconstructions in high depth resolution.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Optics, Light Field Imaging, Mathematics, Physics, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2004.02215" label="2004.02215">
-        <attvalues>
-          <attvalue for="0" value="Light Field Spatial Super-resolution via Deep Combinatorial Geometry&#10;  Embedding and Structural Consistency Regularization" />
-          <attvalue for="1" value="  Light field (LF) images acquired by hand-held devices usually suffer from low&#10;spatial resolution as the limited sampling resources have to be shared with the&#10;angular dimension. LF spatial super-resolution (SR) thus becomes an&#10;indispensable part of the LF camera processing pipeline. The&#10;high-dimensionality characteristic and complex geometrical structure of LF&#10;images make the problem more challenging than traditional single-image SR. The&#10;performance of existing methods is still limited as they fail to thoroughly&#10;explore the coherence among LF views and are insufficient in accurately&#10;preserving the parallax structure of the scene. In this paper, we propose a&#10;novel learning-based LF spatial SR framework, in which each view of an LF image&#10;is first individually super-resolved by exploring the complementary information&#10;among views with combinatorial geometry embedding. For accurate preservation of&#10;the parallax structure among the reconstructed views, a regularization network&#10;trained over a structure-aware loss function is subsequently appended to&#10;enforce correct parallax relationships over the intermediate estimation. Our&#10;proposed approach is evaluated over datasets with a large number of testing&#10;images including both synthetic and real-world scenes. Experimental results&#10;demonstrate the advantage of our approach over state-of-the-art methods, i.e.,&#10;our method not only improves the average PSNR by more than 1.0 dB but also&#10;preserves more accurate parallax details, at a lower computational cost.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.13094" label="2003.13094">
-        <attvalues>
-          <attvalue for="0" value="High-Order Residual Network for Light Field Super-Resolution" />
-          <attvalue for="1" value="  Plenoptic cameras usually sacrifice the spatial resolution of their SAIs to&#10;acquire geometry information from different viewpoints. Several methods have&#10;been proposed to mitigate such spatio-angular trade-off, but seldom make use of&#10;the structural properties of the light field (LF) data efficiently. In this&#10;paper, we propose a novel high-order residual network to learn the geometric&#10;features hierarchically from the LF for reconstruction. An important component&#10;in the proposed network is the high-order residual block (HRB), which learns&#10;the local geometric features by considering the information from all input&#10;views. After fully obtaining the local features learned from each HRB, our&#10;model extracts the representative geometric features for spatio-angular&#10;upsampling through the global residual learning. Additionally, a refinement&#10;network is followed to further enhance the spatial details by minimizing a&#10;perceptual loss. Compared with previous work, our model is tailored to the rich&#10;structure inherent in the LF, and therefore can reduce the artifacts near&#10;non-Lambertian and occlusion regions. Experimental results show that our&#10;approach enables high-quality reconstruction even in challenging regions and&#10;outperforms state-of-the-art single image or LF reconstruction methods with&#10;both quantitative measurements and visual evaluation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.10847" label="2104.10847">
-        <attvalues>
-          <attvalue for="0" value="Localization of Ice-Rink for Broadcast Hockey Videos" />
-          <attvalue for="1" value="  In this work, an automatic and simple framework for hockey ice-rink&#10;localization from broadcast videos is introduced. First, video is broken into&#10;video-shots by a hierarchical partitioning of the video frames, and&#10;thresholding based on their histograms. To localize the frames on the ice-rink&#10;model, a ResNet18-based regressor is implemented and trained, which regresses&#10;to four control points on the model in a frame-by-frame fashion. This leads to&#10;the projection jittering problem in the video. To overcome this, in the&#10;inference phase, the trajectory of the control points on the ice-rink model are&#10;smoothed, for all the consecutive frames of a given video-shot, by convolving a&#10;Hann window with the achieved coordinates. Finally, the smoothed homography&#10;matrix is computed by using the direct linear transform on the four pairs of&#10;corresponding points. A hockey dataset for training and testing the regressor&#10;is gathered. The results show success of this simple and comprehensive&#10;procedure for localizing the hockey ice-rink and addressing the problem of&#10;jittering without affecting the accuracy of homography estimation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In computer vision based sports video analytics, one of the fundamental challenges is precise localization of the sport field. Which is determining where the broadcast camera is looking, at any video frame. Result of this can be used for different purposes, such as determining the players’ coordinates on the sport field, camera calibration \cite{chen2019sports}, placement of the visual ads on the sport field. &#10;&#10;In the literature, this problem is addressed by finding the homography between video frames and the top-view model of the sports field. Generally, the state-of-the-art procedures for localizing the sports fields \cite{citraro2020realtime, jiang2020optimizing, sha2020endtoend} &#10;utilize deep structures for registering the video frames on the model of the sport filed, by minimizing a distance metric between the ground-truth and predicted points (or areas) of a frame on the model (or vice versa). These predictions are usually performed in a frame-by-frame manner, so, the result of registration (localization) for a given video sequence usually suffers from jittering. Therefore, an additional smoothing step on predicted homography of the consecutive frames is required. For smoothing or refinement of the initial homography estimation, some &#10;works in the literature such as\cite{chen2019sports}, use Lucas-Kanade algorithm \cite{baker2004lucas}, which is based on tracking feature patches in the video frames. So, the success of this procedure highly depends on the detecting precision of the corresponding features.&#10;&#10;In this work, a shot boundary detection procedure for broadcast hockey videos is implemented by adopting the cut transition method in \cite{yazdi2016shot}. A frame-based hockey ice-rink localization is performed by inspiration from \cite{jiang2020optimizing}. A ResNet18-based regressor is implemented which, for an input frame, regresses to the four control points on the hockey ice-rink model. In the inference phase, to remove the jittering of warped frames on the ice-rink model, for consecutive video-frames, the trajectories of the control points on the ice-rink model are smoothed by using a simple moving average operation based on Hann window. Then, the homography matrix is computed by using the direct linear transform (DLT) method \cite{dubrofsky2008combining}.&#10;To the best of our knowledge there is no publicly available dataset on hockey videos for ice-rink localization. Therefore, a dataset of 72 hockey video-shots, along with their key-frames and their corresponding homography matrices, is generated and used to train, evaluate and test our method.&#10;&#10;Our contributions in this paper are:&#10;\begin{itemize}&#10; \item Adopting a cut transition detection procedure, for fast and accurate shot boundary detection in broadcast hockey videos.&#10; \item Implementing a frame-by-frame regressor for localizing the video-shot frames on the ice-rink model.&#10; \item Introducing a new smoothing procedure by convolving a Hann window to the trajectory of the predicted control points, for the successive video-shot frames.&#10; \item Generating a new hockey data set for ice-rink localization.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Video Analysis, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2003.14109" label="2003.14109">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Camera Pose Estimation for Sports Fields" />
-          <attvalue for="1" value="  Given an image sequence featuring a portion of a sports field filmed by a&#10;moving and uncalibrated camera, such as the one of the smartphones, our goal is&#10;to compute automatically in real time the focal length and extrinsic camera&#10;parameters for each image in the sequence without using a priori knowledges of&#10;the position and orientation of the camera. To this end, we propose a novel&#10;framework that combines accurate localization and robust identification of&#10;specific keypoints in the image by using a fully convolutional deep&#10;architecture. Our algorithm exploits both the field lines and the players'&#10;image locations, assuming their ground plane positions to be given, to achieve&#10;accuracy and robustness that is beyond the current state of the art. We will&#10;demonstrate its effectiveness on challenging soccer, basketball, and volleyball&#10;benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.08034" label="1909.08034">
-        <attvalues>
-          <attvalue for="0" value="Optimizing Through Learned Errors for Accurate Sports Field Registration" />
-          <attvalue for="1" value="  We propose an optimization-based framework to register sports field templates&#10;onto broadcast videos. For accurate registration we go beyond the prevalent&#10;feed-forward paradigm. Instead, we propose to train a deep network that&#10;regresses the registration error, and then register images by finding the&#10;registration parameters that minimize the regressed error. We demonstrate the&#10;effectiveness of our method by applying it to real-world sports broadcast&#10;videos, outperforming the state of the art. We further apply our method on a&#10;synthetic toy example and demonstrate that our method brings significant gains&#10;even when the problem is simplified and unlimited training data is available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.04238" label="2211.04238">
-        <attvalues>
-          <attvalue for="0" value="HDRfeat: A Feature-Rich Network for High Dynamic Range Image&#10;  Reconstruction" />
-          <attvalue for="1" value="  A major challenge for high dynamic range (HDR) image reconstruction from&#10;multi-exposed low dynamic range (LDR) images, especially with dynamic scenes,&#10;is the extraction and merging of relevant contextual features in order to&#10;suppress any ghosting and blurring artifacts from moving objects. To tackle&#10;this, in this work we propose a novel network for HDR reconstruction with deep&#10;and rich feature extraction layers, including residual attention blocks with&#10;sequential channel and spatial attention. For the compression of the&#10;rich-features to the HDR domain, a residual feature distillation block (RFDB)&#10;based architecture is adopted. In contrast to earlier deep-learning methods for&#10;HDR, the above contributions shift focus from merging/compression to feature&#10;extraction, the added value of which we demonstrate with ablation experiments.&#10;We present qualitative and quantitative comparisons on a public benchmark&#10;dataset, showing that our proposed method outperforms the state-of-the-art.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Dynamic range is the luminance ratio between the brightest and darkest areas in a scene. &#10;Natural scenes have a much higher luminance range than digital cameras can represent, resulting in captured low dynamic range (LDR) images to often have over- or under-exposed regions~\cite{reinhard2010high}. &#10;High dynamic range (HDR) images can represent a wide range of illuminations. &#10;&#10;The typical way to generate an HDR image is to merge a series of LDR images with different exposures of the same scene captured by a positionally-fixed camera. &#10;Given the acquired LDR images, some methods focus on recovering the camera response function~\cite{debevec2008recovering,mann1994being,granados2010optimal,reinhard2010high,yan2017high,salih2012tone}, which is the relation between the irradiance map and the HDR-image pixel values.&#10;Others adopt a multiple exposure fusion approach~\cite{jinno2012multiple, pradeep2012robust,xu2022multi,zhang2021benchmarking}, which combines the multi-exposed LDR images to estimate the irradiance value for each pixel. &#10;However, if the input LDR images contain large foreground motions, the merged HDR image will suffer from ghosting and blurring artifacts due to the misalignment among the LDR images. &#10;&#10;Several deep learning approaches have been proposed to address the aforementioned problems~\cite{kalantari2017deep,wu2018deep,yan2019attention,yan2019multi,yan2022dual,wang2021exposure, niu2021HDRGAN}. &#10;For example, Kalantari et al.~\cite{kalantari2017deep} used optical flow to align input images and then merged the aligned images with a four-layer fully convolutional neural network (CNN) to generate a restored HDR image. &#10;However, such a simple neural network structure cannot handle the artifacts and distortion caused by errors from an unreliable optical flow. &#10;Wu et al.~\cite{wu2018deep} proposed the first end-to-end deep neural network without optical flow alignment for HDR reconstruction, which includes three encoder networks, a merger network, and a decoder network. &#10;However, due to the lack of an explicit alignment mechanism among LDR images, this approach suffers from occlusion and ghosting. &#10;In their seminal work, Yan et al.~\cite{yan2019attention} were the first to use attention networks (AHDRNet) aiming to highlight or align large motions in the features prior to the merger network, which can then suppress any ghosting artifacts.&#10;Besides the attention network dramatically increasing the computational cost, there still existed blurring or artifacts in large motion scenarios as shown in Fig. \ref{fig:visual_comparison_first}.&#10;Nevertheless, this work inspired several following works, which adapted their proposed architecture; for example, see several solutions in the recent NTIRE 2021 and 2022 challenges on HDR~\cite{eduardo2021ntire, perez2022ntire}.&#10;In \cite{yan2022dual}, Yan et al.\ proposed DAHDRNet with recursive channel and spatial attention for more effective de-ghosting. &#10;Most deep-learning solutions to HDR (e.g.\ AHDRNet and its derivatives) involve two consecutive stages overall: extraction of features from LDR images, and based on these merging/compression into an HDR image.&#10;The latter stage in essence is a function of LDR images, e.g.\ how to scale and merge them locally.&#10;It is the features from the former stage that parametrizes such merging function, e.g.\ that should highlight/suppress misaligned regions, occlusions, etc.&#10;In other words, one can expect the former task to be far more complex than the latter task.&#10;Based on this motivation, we propose a network architecture (HDRfeat) which is rich in feature extraction with attention and channel-wise hierarchical layers, while lighter on the latter reconstruction side, compared to most state-of-the-art solutions.&#10;Accordingly, our main contributions herein include: &#10;$\bullet$~a novel channel-wise hierarchical feature extraction network to efficiently extract rich contextual information from multi-exposure images, with a channel bottleneck structure to compress these LDR feature representations purposed for subsequent HDR reconstruction;&#10;$\bullet$~a residual attention block with sequential channel and spatial attention to allow the features to focus where needed, e.g.\ large motions;&#10;$\bullet$~the state-of-the-art on HDR reconstruction, demonstrated via extensive evaluations on a public benchmark dataset.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Image Reconstruction, Mathematics, Feature Extraction, Deep Learning, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2007.15156" label="2007.15156">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking and Comparing Multi-exposure Image Fusion Algorithms" />
-          <attvalue for="1" value="  Multi-exposure image fusion (MEF) is an important area in computer vision and&#10;has attracted increasing interests in recent years. Apart from conventional&#10;algorithms, deep learning techniques have also been applied to multi-exposure&#10;image fusion. However, although much efforts have been made on developing MEF&#10;algorithms, the lack of benchmark makes it difficult to perform fair and&#10;comprehensive performance comparison among MEF algorithms, thus significantly&#10;hindering the development of this field. In this paper, we fill this gap by&#10;proposing a benchmark for multi-exposure image fusion (MEFB) which consists of&#10;a test set of 100 image pairs, a code library of 16 algorithms, 20 evaluation&#10;metrics, 1600 fused images and a software toolkit. To the best of our&#10;knowledge, this is the first benchmark in the field of multi-exposure image&#10;fusion. Extensive experiments have been conducted using MEFB for comprehensive&#10;performance evaluation and for identifying effective algorithms. We expect that&#10;MEFB will serve as an effective platform for researchers to compare&#10;performances and investigate MEF algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.08937" label="1711.08937">
-        <attvalues>
-          <attvalue for="0" value="Deep High Dynamic Range Imaging with Large Foreground Motions" />
-          <attvalue for="1" value="  This paper proposes the first non-flow-based deep framework for high dynamic&#10;range (HDR) imaging of dynamic scenes with large-scale foreground motions. In&#10;state-of-the-art deep HDR imaging, input images are first aligned using optical&#10;flows before merging, which are still error-prone due to occlusion and large&#10;motions. In stark contrast to flow-based methods, we formulate HDR imaging as&#10;an image translation problem without optical flows. Moreover, our simple&#10;translation network can automatically hallucinate plausible HDR details in the&#10;presence of total occlusion, saturation and under-exposure, which are otherwise&#10;almost impossible to recover by conventional optimization approaches. Our&#10;framework can also be extended for different reference images. We performed&#10;extensive qualitative and quantitative comparisons to show that our approach&#10;produces excellent results where color artifacts and geometric distortions are&#10;significantly reduced compared to existing state-of-the-art methods, and is&#10;robust across various inputs, including images without radiometric calibration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.01439" label="2106.01439">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and&#10;  Results" />
-          <attvalue for="1" value="  This paper reviews the first challenge on high-dynamic range (HDR) imaging&#10;that was part of the New Trends in Image Restoration and Enhancement (NTIRE)&#10;workshop, held in conjunction with CVPR 2021. This manuscript focuses on the&#10;newly introduced dataset, the proposed methods and their results. The challenge&#10;aims at estimating a HDR image from one or multiple respective low-dynamic&#10;range (LDR) observations, which might suffer from under- or over-exposed&#10;regions and different sources of noise. The challenge is composed by two&#10;tracks: In Track 1 only a single LDR image is provided as input, whereas in&#10;Track 2 three differently-exposed LDR images with inter-frame motion are&#10;available. In both tracks, the ultimate goal is to achieve the best objective&#10;HDR reconstruction in terms of PSNR with respect to a ground-truth image,&#10;evaluated both directly and with a canonical tonemapping operation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.04536" label="2403.04536">
-        <attvalues>
-          <attvalue for="0" value="A stochastic optimisation unadjusted Langevin method for empirical&#10;  Bayesian estimation in semi-blind image deblurring problems" />
-          <attvalue for="1" value="  This paper presents a novel stochastic optimisation methodology to perform&#10;empirical Bayesian inference in semi-blind image deconvolution problems. Given&#10;a blurred image and a parametric class of possible operators, the proposed&#10;optimisation approach automatically calibrates the parameters of the blur model&#10;by maximum marginal likelihood estimation, followed by (non-blind) image&#10;deconvolution by maximum-a-posteriori estimation conditionally to the estimated&#10;model parameters. In addition to the blur model, the proposed approach also&#10;automatically calibrates the noise variance as well as any regularisation&#10;parameters. The marginal likelihood of the blur, noise variance, and&#10;regularisation parameters is generally computationally intractable, as it&#10;requires calculating several integrals over the entire solution space. Our&#10;approach addresses this difficulty by using a stochastic approximation proximal&#10;gradient optimisation scheme, which iteratively solves such integrals by using&#10;a Moreau-Yosida regularised unadjusted Langevin Markov chain Monte Carlo&#10;algorithm. This optimisation strategy can be easily and efficiently applied to&#10;any model that is log-concave, and by using the same gradient and proximal&#10;operators that are required to compute the maximum-a-posteriori solution by&#10;convex optimisation. We provide convergence guarantees for the proposed&#10;optimisation scheme under realistic and easily verifiable conditions and&#10;subsequently demonstrate the effectiveness of the approach with a series of&#10;deconvolution experiments and comparisons with alternative strategies from the&#10;state of the art.&#10;" />
-          <attvalue for="2" value="&#10;We consider image deconvolution problems that seek to recover an unknown image $x^\star\in \R^d$ from a blurred and noisy observation $y\in \R^d$ related to $x^\star$ by the following linear model&#10;\begin{equation} &#10;&#09;y = H x^\star + w\, &#10;\end{equation}&#10;where $w$ is a realisation of zero-mean white Gaussian noise with covariance variance $\sigma^2 \mathbb{I}_d$, and $H$ is a circulant matrix of size $d\times d$ representing the discrete convolution with a blur kernel $h$ (also commonly known in the image deconvolution literature as the point-spread-function (PSF)). Image deconvolution problems are central to imaging sciences (see, e.g., \cite{almeida2009blind, fergus2006removing, jefferies2002blind, bertero2000image,markham1999parametric}, for examples in photographic imaging, astronomical imaging, medical imaging).&#10;&#10;A main challenge in performing image deblurring is that the estimation of $x^\star$ from $y$ by direct inversion of $H$ is often either ill-conditioned or ill-posed \cite{kaipio2006statistical,idier2013bayesian}. This difficulty can be addressed by exploiting prior knowledge about $x^\star$ in order to regularise the estimation problem and derive solutions that are well-posed. Following several decades of active research on the topic, the image deconvolution literature proposes a wide range of methods to tackle these problems. This paper considers image deconvolution methodology for situations in which there is no suitable training data available to use as ground truth (with regards to methodology for situations with abundant training data available, we refer the reader to the recent works \cite{dong2015image, kupyn2018deblurgan, albluwi2018image,monga2021algorithm,pesquet2021learning,li2020efficient,bertocchi2020deep} based on deep learning strategies). Within the class of image deconvolution methods that do not involve training data, we broadly classify the methods as non-blind, blind and semi-blind based on their assumptions on $h$. &#10;&#10;Non-blind image deblurring methods seek to estimate $x^\star$ from $y$ under the assumption that the blur kernel $h$ is perfectly known. There are many regularisation strategies available to make these problems well-posed, offering a wide range of possibilities in terms of estimation accuracy and computational efficiency (see, e.g., \cite{afonso2010fast,beck2009fast,laumont2022bayesian,chantas2006bayesian, guerrero2006deblurring, bar2006image}). By comparison to blind or semi-blind methods, state-of-the-art non-blind methods are fast and accurate. However, deploying non-blind image deconvolution methods in real-world settings often require expert supervision and onerous calibration experiments to determine $h$. In some applications, regular recalibration experiments are also required.&#10;&#10;In contrast to non-blind methods, blind image deblurring methods assume that $h$ is completely unknown. This leads to a challenging estimation problem that is severely ill-posed and requires significant regularisation to deliver well-posed solutions (we refer the reader to \cite{campisi2017blind} for an excellent introduction to the topic and an overview of the state-of-the-art until 2017). Blind image deconvolution methods often seek to address this difficulty in one of the following three ways. One approach is to construct estimators to determine $x^\star$ and $h$ jointly from $y$, such as joint penalised least-squares estimators \cite{krishnan2011blind,chan1998total}, or joint maximum-a-posteriori (MAP) estimators formulated in a hierarchical Bayesian framework \cite{kotera2013blind,almeida2009blind,michaeli2014blind}. Another hierarchical Bayesian approach is to estimate $x^\star$ directly from $y$ without assigning a value to $h$, e.g., by marginal MAP or minimum mean squared error (MMSE) estimation (with the marginalisation of $h$) \cite{chantas2006bayesian, huang2022unrolled}. A third approach is to estimate $h$ from $y$, without involving the unknown image, followed by inference on $x^\star$ in a non-blind manner by using the estimate of $h$ as the truth. This arises, for example, in empirical Bayesian methods that estimate $h$ from $y$ by maximum marginal likelihood estimation (MMLE), followed by MAP estimation of $x^\star$ with the (pseudo)-posterior density (see, e.g., \cite{levin2009understanding,levin2011efficient,babacan2010variational,abdulaziz2021blind}. Naturally, blind methods are often noticeably more computationally expensive and inaccurate than non-blind methods that are correctly calibrated. However, blind methods can be applied more widely, as they do not assume prior knowledge of $h$ or require calibration experiments to determine $h$. Blind methods are also more robust than non-blind methods to errors in the estimation of $h$: in most applications, the blur $h$ is not known a priori with good accuracy; and the non-blind methods are sensitive to mismatches between the PSF used in the method and the true PSF. It is worth mentioning at this point that blind deconvolution is a longstanding focus of effort by the community and there are many excellent classical blind deconvolution strategies based on spectral methods and filtering (see, e.g., \cite{kundur1996blind,kundur1996blindrev} for an overview of the topic).&#10;&#10;Semi-blind methods are an intermediate approach which stems from the observation that, in many applications of image deconvolution, there is enough information to specify $h$ partially. Indeed, in many disciplines, while practitioners do not know $h$ exactly a priori, they do have the expertise required to approximate $h$ by using their knowledge of the imaging system (see \cite{morin2013semi, michailovich2007blind,pantin2007deconvolution, holmes2006blind} for examples in medical imaging, astronomy, and microscopy). Formally, semi-blind image deconvolution methods posit that $h$ belongs to some appropriate parametric family of blur operators. Common choices to model $h$ are parametric functions (e.g., the Gaussian, Laplace, Moffat, and Cauchy blur models), or polynomial sequences such as the Zernike polynomials \cite{Niu_2022}. Constraining $h$ to a given parametric family significantly regularises the estimation problem, at the expense of introducing non-linearity in the relationship to $y$. Moreover, in a manner akin to blind problems, semi-blind problems can be solved by joint MAP estimation to determine the unknown image and the parameters of $h$ from $y$ \cite{almeida2009blind}, by marginal MAP or MMSE estimation (following the marginalisation of the parameters of $h$) \cite{orieux2010bayesian,Park2014}, or by estimating the parameter of $h$ by MMLE followed by empirical Bayesian MAP estimation of $x^\star$ given $y$ and the estimated blur parameters. From an implementation viewpoint, most blind deconvolution methods can be adapted for semi-blind inference by parametrising $h$ and either optimising or marginalising w.r.t. to the parameter of $h$, instead of $h$ itself. For example, in Section \ref{results} we report comparisons with two semi-blind methods \cite{almeida2009blind,orieux2010bayesian}, as well with two blind methods \cite{levin2011efficient,abdulaziz2021blind} that can be straightforwardly adapted for semi-blind inference. We have chosen to report comparisons with these methods because they provide a wide range of complementary alternative strategies to perform inference: the method \cite{almeida2009blind} is based on joint MAP estimation, \cite{orieux2010bayesian} implements a hierarchical Bayesian formulation by using a Markov chain Monte Carlo (MCMC) algorithm, \cite{levin2011efficient} adopts an empirical Bayesian approach by using a variational Bayesian approximation, and \cite{abdulaziz2021blind} also adopts an approximation of empirical Bayesian inference by using an expectation propagation algorithm. &#10;&#10;{Performing Bayesian inference in semi-blind image deconvolution problems is computationally challenging because of the high dimensionality involved. Prior work such as \cite{levin2011efficient,abdulaziz2021blind,orieux2010bayesian} addresses this difficulty by making careful modelling choices and by leveraging Bayesian computation algorithms that are specialised to these models. This approach has traditionally relied strongly on Gaussian approximations, Gaussian models and conjugate priors. However, as we demonstrate empirically in Section \ref{results}, these approximations and modelling simplifications limit the performance of the resulting semi-blind inference methods. Moreover, from a Bayesian computation viewpoint, the joint MAP approach of \cite{almeida2009blind,almeida2013parameter} is significantly less challenging because it can be tackled by alternating optimisation and without resorting to approximations. This approach is particularly effective in semi-blind problems that are convex w.r.t. the unknown image $x$, where it can leverage modern proximal optimisation algorithms \cite{chambolle2016introduction}. However, the experiments reported in Section \ref{results} suggest that the resulting inference methods are less accurate than the empirical Bayesian estimation strategies previously mentioned. This is in agreement with the analysis of \cite{levin2011efficient} for blind deconvolution problems, as well as with the experiments reported in \cite{vidal2020maximum} for estimators of regularisation parameters in non-blind image deconvolution problems.}&#10;&#10;This paper presents a new and general method for performing empirical Bayesian inference in semi-blind image deblurring problems that are convex w.r.t. the unknown image $x$. The proposed method estimates the parameters of the blur $h$ directly from $y$ by MMLE, as well as other parameters such as the regularisation parameter and the noise variance. This is then followed by (non-blind) empirical Bayesian MAP inference on the unknown image $x^\star$ conditionally on the estimated parameters by proximal convex optimisation. Unlike existing empirical Bayesian strategies that rely on deterministic algorithms and approximations, the proposed method is based on a state-of-the-art stochastic approximation proximal gradient (SAPG) algorithm \cite{vidal2020maximum} that performs the required computations efficiently without resorting to Gaussian approximations.&#10; The wide range of numerical experiments presented in \ref{results} suggest that the method is remarkably accurate as a result. In addition to experimental results, we also present detailed theoretical convergence results for the proposed method. To the best of our knowledge, this is the first provably convergent empirical Bayesian semi-blind image deconvolution method, as well as the first method of this kind that can be straightforwardly applied to any image deconvolution model with an underlying convex geometry.&#10;&#10;The remainder of the paper is organised as follows: \ref{problem statement} introduces notation and defines the class of semi-blind image deconvolution problems considered in the paper. \ref{methodology} presents the proposed SAPG algorithm for the semi-blind image deconvolution, establishes convergence guarantees, and provides implementation guidelines. \ref{results} demonstrates the proposed method with a range of experiments involving three classes of blur operators (Gaussian, Laplace and Moffat) and a selection of noise levels, as well as via comparisons with alternative approaches from the state-of-the-art. The conclusions and perspectives for future work are finally reported in \ref{conclusion}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Deconvolution, Machine Learning, Computer Vision, Mathematics, Computer Science, Statistics, Stochastic Optimisation, Bayesian Inference, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1812.04276" label="1812.04276">
-        <attvalues>
-          <attvalue for="0" value="Deep Unfolding of a Proximal Interior Point Method for Image Restoration" />
-          <attvalue for="1" value="  Variational methods are widely applied to ill-posed inverse problems for they&#10;have the ability to embed prior knowledge about the solution. However, the&#10;level of performance of these methods significantly depends on a set of&#10;parameters, which can be estimated through computationally expensive and&#10;time-consuming methods. In contrast, deep learning offers very generic and&#10;efficient architectures, at the expense of explainability, since it is often&#10;used as a black-box, without any fine control over its output. Deep unfolding&#10;provides a convenient approach to combine variational-based and deep learning&#10;approaches. Starting from a variational formulation for image restoration, we&#10;develop iRestNet, a neural network architecture obtained by unfolding a&#10;proximal interior point algorithm. Hard constraints, encoding desirable&#10;properties for the restored image, are incorporated into the network thanks to&#10;a logarithmic barrier, while the barrier parameter, the stepsize, and the&#10;penalization weight are learned by the network. We derive explicit expressions&#10;for the gradient of the proximity operator for various choices of constraints,&#10;which allows training iRestNet with gradient descent and backpropagation. In&#10;addition, we provide theoretical results regarding the stability of the network&#10;for a common inverse problem example. Numerical experiments on image deblurring&#10;problems show that the proposed approach compares favorably with both&#10;state-of-the-art variational and machine learning methods in terms of image&#10;quality.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="0910.4887" label="0910.4887">
-        <attvalues>
-          <attvalue for="0" value="Fast Image Recovery Using Variable Splitting and Constrained&#10;  Optimization" />
-          <attvalue for="1" value="  We propose a new fast algorithm for solving one of the standard formulations&#10;of image restoration and reconstruction which consists of an unconstrained&#10;optimization problem where the objective includes an $\ell_2$ data-fidelity&#10;term and a non-smooth regularizer. This formulation allows both wavelet-based&#10;(with orthogonal or frame-based representations) regularization or&#10;total-variation regularization. Our approach is based on a variable splitting&#10;to obtain an equivalent constrained optimization formulation, which is then&#10;addressed with an augmented Lagrangian method. The proposed algorithm is an&#10;instance of the so-called &quot;alternating direction method of multipliers&quot;, for&#10;which convergence has been proved. Experiments on a set of image restoration&#10;and reconstruction benchmark problems show that the proposed algorithm is&#10;faster than the current state of the art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1303.3866" label="1303.3866">
-        <attvalues>
-          <attvalue for="0" value="Variational Semi-blind Sparse Deconvolution with Orthogonal Kernel Bases&#10;  and its Application to MRFM" />
-          <attvalue for="1" value="  We present a variational Bayesian method of joint image reconstruction and&#10;point spread function (PSF) estimation when the PSF of the imaging device is&#10;only partially known. To solve this semi-blind deconvolution problem, prior&#10;distributions are specified for the PSF and the 3D image. Joint image&#10;reconstruction and PSF estimation is then performed within a Bayesian&#10;framework, using a variational algorithm to estimate the posterior&#10;distribution. The image prior distribution imposes an explicit atomic measure&#10;that corresponds to image sparsity. Importantly, the proposed Bayesian&#10;deconvolution algorithm does not require hand tuning. Simulation results&#10;clearly demonstrate that the semi-blind deconvolution algorithm compares&#10;favorably with previous Markov chain Monte Carlo (MCMC) version of myopic&#10;sparse reconstruction. It significantly outperforms mismatched non-blind&#10;algorithms that rely on the assumption of the perfect knowledge of the PSF. The&#10;algorithm is illustrated on real data from magnetic resonance force microscopy&#10;(MRFM).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.11964" label="2304.11964">
-        <attvalues>
-          <attvalue for="0" value="3rd Place Solution to Meta AI Video Similarity Challenge" />
-          <attvalue for="1" value="  This paper presents our 3rd place solution in both Descriptor Track and&#10;Matching Track of the Meta AI Video Similarity Challenge (VSC2022), a&#10;competition aimed at detecting video copies. Our approach builds upon existing&#10;image copy detection techniques and incorporates several strategies to exploit&#10;on the properties of video data, resulting in a simple yet powerful solution.&#10;By employing our proposed method, we achieved substantial improvements in&#10;accuracy compared to the baseline results (Descriptor Track: 38% improvement,&#10;Matching Track: 60% improvement). Our code is publicly available here:&#10;https://github.com/line/Meta-AI-Video-Similarity-Challenge-3rd-Place-Solution&#10;" />
-          <attvalue for="2" value="&#10;In recent years, with the rapid development of social media, issues such as plagiarism and unauthorized modification have become increasingly severe. Consequently, there is a growing demand for technologies capable of accurately and automatically detecting illegal content. In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention. Notably, the Image Similarity Challenge at NeurIPS'21 (ISC2021) was organized as a competition for image copy detection~\cite{ISC2021,papakipos2022resultsISC}, followed by the recent organization of the Meta AI Video Similarity Challenge at CVPR’23 (VSC2022) focusing on video copy detection.&#10;&#10;VSC2022 has two tracks: descriptor and matching. In the descriptor track, the task is to compute embeddings (up to 512 dimensions) for each video. In the matching track, the task is to generate predicted matches containing the starting and ending timestamps of query-reference video pairs.&#10;&#10;In this paper, we describe our approach that achieved third place in both tracks of the VSC2022 competition. Our solution can be summarized as follows:&#10;&#10;\begin{enumerate}&#10; \item We propose a simple yet powerful video copy detection pipeline, based on an image copy detection model (ISC-dt1~\cite{ISC-dt1}).&#10; \item Our approach utilizes test time augmentation based on the predictions of an editing prediction model.&#10; \item We exploit the properties of videos through various techniques: emphasizing copy videos more in search results by calculating frame consistency, concatenating adjacent frames followed by dimensionality reduction using PCA, and localizing copied frames by employing Temporal Network~\cite{Tan2009ScalableTN}.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Processing Techniques, Machine Learning, Engineering, Computer Vision, Mathematics, Video Copy Detection, Computer Science, Information Retrieval, Deep Learning Solutions" />
-        </attvalues>
-      </node>
-      <node id="2202.10261" label="2202.10261">
-        <attvalues>
-          <attvalue for="0" value="A Self-Supervised Descriptor for Image Copy Detection" />
-          <attvalue for="1" value="  Image copy detection is an important task for content moderation. We&#10;introduce SSCD, a model that builds on a recent self-supervised contrastive&#10;training objective. We adapt this method to the copy detection task by changing&#10;the architecture and training objective, including a pooling operator from the&#10;instance matching literature, and adapting contrastive learning to&#10;augmentations that combine images.&#10;  Our approach relies on an entropy regularization term, promoting consistent&#10;separation between descriptor vectors, and we demonstrate that this&#10;significantly improves copy detection accuracy. Our method produces a compact&#10;descriptor vector, suitable for real-world web scale applications. Statistical&#10;information from a background image distribution can be incorporated into the&#10;descriptor.&#10;  On the recent DISC2021 benchmark, SSCD is shown to outperform both baseline&#10;copy detection models and self-supervised architectures designed for image&#10;classification by huge margins, in all settings. For example, SSCD out-performs&#10;SimCLR descriptors by 48% absolute. Code is available at&#10;https://github.com/facebookresearch/sscd-copy-detection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.12358" label="2205.12358">
-        <attvalues>
-          <attvalue for="0" value="A Benchmark and Asymmetrical-Similarity Learning for Practical Image&#10;  Copy Detection" />
-          <attvalue for="1" value="  Image copy detection (ICD) aims to determine whether a query image is an&#10;edited copy of any image from a reference set. Currently, there are very&#10;limited public benchmarks for ICD, while all overlook a critical challenge in&#10;real-world applications, i.e., the distraction from hard negative queries.&#10;Specifically, some queries are not edited copies but are inherently similar to&#10;some reference images. These hard negative queries are easily false recognized&#10;as edited copies, significantly compromising the ICD accuracy. This observation&#10;motivates us to build the first ICD benchmark featuring this characteristic.&#10;Based on existing ICD datasets, this paper constructs a new dataset by&#10;additionally adding 100, 000 and 24, 252 hard negative pairs into the training&#10;and test set, respectively. Moreover, this paper further reveals a unique&#10;difficulty for solving the hard negative problem in ICD, i.e., there is a&#10;fundamental conflict between current metric learning and ICD. This conflict is:&#10;the metric learning adopts symmetric distance while the edited copy is an&#10;asymmetric (unidirectional) process, e.g., a partial crop is close to its&#10;holistic reference image and is an edited copy, while the latter cannot be the&#10;edited copy of the former (in spite the distance is equally small). This&#10;insight results in an Asymmetrical-Similarity Learning (ASL) method, which&#10;allows the similarity in two directions (the query &lt;-&gt; the reference image) to&#10;be different from each other. Experimental results show that ASL outperforms&#10;state-of-the-art methods by a clear margin, confirming that solving the&#10;symmetric-asymmetric conflict is critical for ICD. The NDEC dataset and code&#10;are available at https://github.com/WangWenhao0716/ASL.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.10620" label="2210.10620">
-        <attvalues>
-          <attvalue for="0" value="Active Image Indexing" />
-          <attvalue for="1" value="  Image copy detection and retrieval from large databases leverage two&#10;components. First, a neural network maps an image to a vector representation,&#10;that is relatively robust to various transformations of the image. Second, an&#10;efficient but approximate similarity search algorithm trades scalability (size&#10;and speed) against quality of the search, thereby introducing a source of&#10;error. This paper improves the robustness of image copy detection with active&#10;indexing, that optimizes the interplay of these two components. We reduce the&#10;quantization loss of a given image representation by making imperceptible&#10;changes to the image before its release. The loss is back-propagated through&#10;the deep neural network back to the image, under perceptual constraints. These&#10;modifications make the image more retrievable. Our experiments show that the&#10;retrieval and copy detection of activated images is significantly improved. For&#10;instance, activation improves by $+40\%$ the Recall1@1 on various image&#10;transformations, and for several popular indexing structures based on product&#10;quantization and locality sensitivity hashing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.02654" label="2203.02654">
-        <attvalues>
-          <attvalue for="0" value="A Large-scale Comprehensive Dataset and Copy-overlap Aware Evaluation&#10;  Protocol for Segment-level Video Copy Detection" />
-          <attvalue for="1" value="  In this paper, we introduce VCSL (Video Copy Segment Localization), a new&#10;comprehensive segment-level annotated video copy dataset. Compared with&#10;existing copy detection datasets restricted by either video-level annotation or&#10;small-scale, VCSL not only has two orders of magnitude more segment-level&#10;labelled data, with 160k realistic video copy pairs containing more than 280k&#10;localized copied segment pairs, but also covers a variety of video categories&#10;and a wide range of video duration. All the copied segments inside each&#10;collected video pair are manually extracted and accompanied by precisely&#10;annotated starting and ending timestamps. Alongside the dataset, we also&#10;propose a novel evaluation protocol that better measures the prediction&#10;accuracy of copy overlapping segments between a video pair and shows improved&#10;adaptability in different scenarios. By benchmarking several baseline and&#10;state-of-the-art segment-level video copy detection methods with the proposed&#10;dataset and evaluation metric, we provide a comprehensive analysis that&#10;uncovers the strengths and weaknesses of current approaches, hoping to open up&#10;promising directions for future works. The VCSL dataset, metric and benchmark&#10;codes are all publicly available at https://github.com/alipay/VCSL.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.13090" label="2211.13090">
-        <attvalues>
-          <attvalue for="0" value="TransVCL: Attention-enhanced Video Copy Localization Network with&#10;  Flexible Supervision" />
-          <attvalue for="1" value="  Video copy localization aims to precisely localize all the copied segments&#10;within a pair of untrimmed videos in video retrieval applications. Previous&#10;methods typically start from frame-to-frame similarity matrix generated by&#10;cosine similarity between frame-level features of the input video pair, and&#10;then detect and refine the boundaries of copied segments on similarity matrix&#10;under temporal constraints. In this paper, we propose TransVCL: an&#10;attention-enhanced video copy localization network, which is optimized directly&#10;from initial frame-level features and trained end-to-end with three main&#10;components: a customized Transformer for feature enhancement, a correlation and&#10;softmax layer for similarity matrix generation, and a temporal alignment module&#10;for copied segments localization. In contrast to previous methods demanding the&#10;handcrafted similarity matrix, TransVCL incorporates long-range temporal&#10;information between feature sequence pair using self- and cross- attention&#10;layers. With the joint design and optimization of three components, the&#10;similarity matrix can be learned to present more discriminative copied&#10;patterns, leading to significant improvements over previous methods on&#10;segment-level labeled datasets (VCSL and VCDB). Besides the state-of-the-art&#10;performance in fully supervised setting, the attention architecture facilitates&#10;TransVCL to further exploit unlabeled or simply video-level labeled data.&#10;Additional experiments of supplementing video-level labeled datasets including&#10;SVD and FIVR reveal the high flexibility of TransVCL from full supervision to&#10;semi-supervision (with or without video-level annotation). Code is publicly&#10;available at https://github.com/transvcl/TransVCL.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.04007" label="2202.04007">
-        <attvalues>
-          <attvalue for="0" value="Results and findings of the 2021 Image Similarity Challenge" />
-          <attvalue for="1" value="  The 2021 Image Similarity Challenge introduced a dataset to serve as a new&#10;benchmark to evaluate recent image copy detection methods. There were 200&#10;participants to the competition. This paper presents a quantitative and&#10;qualitative analysis of the top submissions. It appears that the most difficult&#10;image transformations involve either severe image crops or hiding into&#10;unrelated images, combined with local pixel perturbations. The key algorithmic&#10;elements in the winning submissions are: training on strong augmentations,&#10;self-supervised learning, score normalization, explicit overlay detection, and&#10;global descriptor matching followed by pairwise image comparison.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.11559" label="2209.11559">
-        <attvalues>
-          <attvalue for="0" value="Query-based Hard-Image Retrieval for Object Detection at Test Time" />
-          <attvalue for="1" value="  There is a longstanding interest in capturing the error behaviour of object&#10;detectors by finding images where their performance is likely to be&#10;unsatisfactory. In real-world applications such as autonomous driving, it is&#10;also crucial to characterise potential failures beyond simple requirements of&#10;detection performance. For example, a missed detection of a pedestrian close to&#10;an ego vehicle will generally require closer inspection than a missed detection&#10;of a car in the distance. The problem of predicting such potential failures at&#10;test time has largely been overlooked in the literature and conventional&#10;approaches based on detection uncertainty fall short in that they are agnostic&#10;to such fine-grained characterisation of errors. In this work, we propose to&#10;reformulate the problem of finding &quot;hard&quot; images as a query-based hard image&#10;retrieval task, where queries are specific definitions of &quot;hardness&quot;, and offer&#10;a simple and intuitive method that can solve this task for a large family of&#10;queries. Our method is entirely post-hoc, does not require ground-truth&#10;annotations, is independent of the choice of a detector, and relies on an&#10;efficient Monte Carlo estimation that uses a simple stochastic model in place&#10;of the ground-truth. We show experimentally that it can be applied successfully&#10;to a wide variety of queries for which it can reliably identify hard images for&#10;a given detector without any labelled data. We provide results on ranking and&#10;classification tasks using the widely used RetinaNet, Faster-RCNN, Mask-RCNN,&#10;and Cascade Mask-RCNN object detectors. The code for this project is available&#10;at https://github.com/fiveai/hardest.&#10;" />
-          <attvalue for="2" value="&#10;How difficult (or hard) an image can be for a model at inference? A large body of work has already paid significant attention in defining the notion of hard (or useful) images for standard classification and object detection tasks. The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}. All these approaches attempt to improve the performance or robustness of object detectors by focusing on their training and generally assume that ground-truth labels are available. In contrast, our objective is to identify hard images for a given trained detector, that is images for which the detector is likely to make a mistake, without having access to ground-truth labels.&#10;&#10;First note that, in contrast to classification where there is a well-defined definition of error (a misclassification), there are multiple ways to define mistakes in object detection.&#10;For example, an image can be considered hard if a detector produces many false-positives on it, but this by far not the only possibility. Another one is to say that an image is hard if the detector makes many mistakes on close objects rather than on distant ones. Therefore, what makes an image hard depends on the situation at hand and one should be able to quantify ``hardness'' accordingly by defining the associated notion of a mistake. We illustrate this point on Figure~\ref{fig:introexamplefig}, where we show two example images that exhibit a variety of possible definitions of errors for object detection.&#10;Note that the widely used uncertainty metrics that are often used to quantify ``hardness'', such as entropy~\cite{ShannonEntropy} and Dempster-Shafer~\cite{sensoy2018evidential}, to name a few, are completely agnostic to such requirements. In this work, we revisit these notions and take a step towards defining and quantifying requirement-specific hardness metrics without having access to the ground-truth labels. &#10;&#10;Our main contributions are as follows. First, we provide a framework that allows to define a large family of requirement-specific hardness metrics that define a notion of error. We call them hardness queries. The central idea is to provide fine-grained error behaviours of a given detector at test time (e.g., set of false-negative (FN) and false-positive (FP) bounding boxes) to allow users to compose a large family of complex queries. For example, if provided with the set of potential false positive and false negative bounding boxes, a user may compose a query and define hardness as a metric that quantifies the mistake severity on these sets. Let us say that the user's query is to ``find images where the number of false-positive boxes strictly dominate the number of false-negative ones''. Then, the metric quantifying such query would simply be $\mathbb{1}_{|FN|&gt;|FP|}$, where each image for which the indicator turns out to be one would be classified as the hard image under the the given query. In such a way, many queries, much more expressive than this one (discussed in the main paper), can be composed depending on the requirements of the application. &#10;&#10;Since, in the absence of ground-truth annotations, it is not possible to define the set of fine-grained errors mentioned above, we propose a simple approach to define a distribution of pseudo ground-truth labels (a multivariate Bernoulli distribution) that is then used to define a distribution over these error sets. This distribution is then employed via an efficient Monte Carlo sampling procedure to estimate the expected query-based hardness per image. Through extensive experiments using a suite of widely used object detectors (RetinaNet, Faster-RCNN, Mask-RCNN, and Cascade Mask-RCNN), we show that our method provides state-of-the-art performance (with significant margins in many cases) on ranking and classification tasks that tackle the problem of finding query-based hard images at test time.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Error Analysis, Mathematics, Artificial Intelligence, Object Detection, Autonomous Systems, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1703.02910" label="1703.02910">
-        <attvalues>
-          <attvalue for="0" value="Deep Bayesian Active Learning with Image Data" />
-          <attvalue for="1" value="  Even though active learning forms an important pillar of machine learning,&#10;deep learning tools are not prevalent within it. Deep learning poses several&#10;difficulties when used in an active learning setting. First, active learning&#10;(AL) methods generally rely on being able to learn and update models from small&#10;amounts of data. Recent advances in deep learning, on the other hand, are&#10;notorious for their dependence on large amounts of data. Second, many AL&#10;acquisition functions rely on model uncertainty, yet deep learning methods&#10;rarely represent such model uncertainty. In this paper we combine recent&#10;advances in Bayesian deep learning into the active learning framework in a&#10;practical way. We develop an active learning framework for high dimensional&#10;data, a task which has been extremely challenging so far, with very sparse&#10;existing literature. Taking advantage of specialised models such as Bayesian&#10;convolutional neural networks, we demonstrate our active learning techniques&#10;with image data, obtaining a significant improvement on existing active&#10;learning approaches. We demonstrate this on both the MNIST dataset, as well as&#10;for skin cancer diagnosis from lesion images (ISIC2016 task).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.16130" label="2103.16130">
-        <attvalues>
-          <attvalue for="0" value="Active Learning for Deep Object Detection via Probabilistic Modeling" />
-          <attvalue for="1" value="  Active learning aims to reduce labeling costs by selecting only the most&#10;informative samples on a dataset. Few existing works have addressed active&#10;learning for object detection. Most of these methods are based on multiple&#10;models or are straightforward extensions of classification methods, hence&#10;estimate an image's informativeness using only the classification head. In this&#10;paper, we propose a novel deep active learning approach for object detection.&#10;Our approach relies on mixture density networks that estimate a probabilistic&#10;distribution for each localization and classification head's output. We&#10;explicitly estimate the aleatoric and epistemic uncertainty in a single forward&#10;pass of a single model. Our method uses a scoring function that aggregates&#10;these two types of uncertainties for both heads to obtain every image's&#10;informativeness score. We demonstrate the efficacy of our approach in PASCAL&#10;VOC and MS-COCO datasets. Our approach outperforms single-model based methods&#10;and performs on par with multi-model based methods at a fraction of the&#10;computing cost.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.04699" label="2004.04699">
-        <attvalues>
-          <attvalue for="0" value="Scalable Active Learning for Object Detection" />
-          <attvalue for="1" value="  Deep Neural Networks trained in a fully supervised fashion are the dominant&#10;technology in perception-based autonomous driving systems. While collecting&#10;large amounts of unlabeled data is already a major undertaking, only a subset&#10;of it can be labeled by humans due to the effort needed for high-quality&#10;annotation. Therefore, finding the right data to label has become a key&#10;challenge. Active learning is a powerful technique to improve data efficiency&#10;for supervised learning methods, as it aims at selecting the smallest possible&#10;training set to reach a required performance. We have built a scalable&#10;production system for active learning in the domain of autonomous driving. In&#10;this paper, we describe the resulting high-level design, sketch some of the&#10;challenges and their solutions, present our current results at scale, and&#10;briefly describe the open problems and future directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.10374" label="2103.10374">
-        <attvalues>
-          <attvalue for="0" value="Consistency-based Active Learning for Object Detection" />
-          <attvalue for="1" value="  Active learning aims to improve the performance of task model by selecting&#10;the most informative samples with a limited budget. Unlike most recent works&#10;that focused on applying active learning for image classification, we propose&#10;an effective Consistency-based Active Learning method for object Detection&#10;(CALD), which fully explores the consistency between original and augmented&#10;data. CALD has three appealing benefits. (i) CALD is systematically designed by&#10;investigating the weaknesses of existing active learning methods, which do not&#10;take the unique challenges of object detection into account. (ii) CALD unifies&#10;box regression and classification with a single metric, which is not concerned&#10;by active learning methods for classification. CALD also focuses on the most&#10;informative local region rather than the whole image, which is beneficial for&#10;object detection. (iii) CALD not only gauges individual information for sample&#10;selection, but also leverages mutual information to encourage a balanced data&#10;distribution. Extensive experiments show that CALD significantly outperforms&#10;existing state-of-the-art task-agnostic and detection-specific active learning&#10;methods on general object detection datasets. Based on the Faster R-CNN&#10;detector, CALD consistently surpasses the baseline method (random selection) by&#10;2.9/2.8/0.8 mAP on average on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO.&#10;Code is available at \url{https://github.com/we1pingyu/CALD}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.10609" label="1901.10609">
-        <attvalues>
-          <attvalue for="0" value="Deep Active Learning for Efficient Training of a LiDAR 3D Object&#10;  Detector" />
-          <attvalue for="1" value="  Training a deep object detector for autonomous driving requires a huge amount&#10;of labeled data. While recording data via on-board sensors such as camera or&#10;LiDAR is relatively easy, annotating data is very tedious and time-consuming,&#10;especially when dealing with 3D LiDAR points or radar data. Active learning has&#10;the potential to minimize human annotation efforts while maximizing the object&#10;detector's performance. In this work, we propose an active learning method to&#10;train a LiDAR 3D object detector with the least amount of labeled training data&#10;necessary. The detector leverages 2D region proposals generated from the RGB&#10;images to reduce the search space of objects and speed up the learning process.&#10;Experiments show that our proposed method works under different uncertainty&#10;estimations and query functions, and can save up to 60% of the labeling efforts&#10;while reaching the same network performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.07441" label="1608.07441">
-        <attvalues>
-          <attvalue for="0" value="Hard Negative Mining for Metric Learning Based Zero-Shot Classification" />
-          <attvalue for="1" value="  Zero-Shot learning has been shown to be an efficient strategy for domain&#10;adaptation. In this context, this paper builds on the recent work of Bucher et&#10;al. [1], which proposed an approach to solve Zero-Shot classification problems&#10;(ZSC) by introducing a novel metric learning based objective function. This&#10;objective function allows to learn an optimal embedding of the attributes&#10;jointly with a measure of similarity between images and attributes. This paper&#10;extends their approach by proposing several schemes to control the generation&#10;of the negative pairs, resulting in a significant improvement of the&#10;performance and giving above state-of-the-art results on three challenging ZSC&#10;datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.05036" label="2101.05036">
-        <attvalues>
-          <attvalue for="0" value="Estimating and Evaluating Regression Predictive Uncertainty in Deep&#10;  Object Detectors" />
-          <attvalue for="1" value="  Predictive uncertainty estimation is an essential next step for the reliable&#10;deployment of deep object detectors in safety-critical tasks. In this work, we&#10;focus on estimating predictive distributions for bounding box regression output&#10;with variance networks. We show that in the context of object detection,&#10;training variance networks with negative log likelihood (NLL) can lead to high&#10;entropy predictive distributions regardless of the correctness of the output&#10;mean. We propose to use the energy score as a non-local proper scoring rule and&#10;find that when used for training, the energy score leads to better calibrated&#10;and lower entropy predictive distributions than NLL. We also address the&#10;widespread use of non-proper scoring metrics for evaluating predictive&#10;distributions from deep object detectors by proposing an alternate evaluation&#10;approach founded on proper scoring rules. Using the proposed evaluation tools,&#10;we show that although variance networks can be used to produce high quality&#10;predictive distributions, ad-hoc approaches used by seminal object detectors&#10;for choosing regression targets during training do not provide wide enough data&#10;support for reliable variance learning. We hope that our work helps shift&#10;evaluation in probabilistic object detection to better align with predictive&#10;uncertainty evaluation in other machine learning domains. Code for all models,&#10;evaluation, and datasets is available at:&#10;https://github.com/asharakeh/probdet.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01768" label="1806.01768">
-        <attvalues>
-          <attvalue for="0" value="Evidential Deep Learning to Quantify Classification Uncertainty" />
-          <attvalue for="1" value="  Deterministic neural nets have been shown to learn effective predictors on a&#10;wide range of machine learning problems. However, as the standard approach is&#10;to train the network to minimize a prediction loss, the resultant model remains&#10;ignorant to its prediction confidence. Orthogonally to Bayesian neural nets&#10;that indirectly infer prediction uncertainty through weight uncertainties, we&#10;propose explicit modeling of the same using the theory of subjective logic. By&#10;placing a Dirichlet distribution on the class probabilities, we treat&#10;predictions of a neural net as subjective opinions and learn the function that&#10;collects the evidence leading to these opinions by a deterministic neural net&#10;from data. The resultant predictor for a multi-class classification problem is&#10;another Dirichlet distribution whose parameters are set by the continuous&#10;output of a neural net. We provide a preliminary analysis on how the&#10;peculiarities of our new loss function drive improved uncertainty estimation.&#10;We observe that our method achieves unprecedented success on detection of&#10;out-of-distribution queries and endurance against adversarial perturbations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.00922" label="2310.00922">
-        <attvalues>
-          <attvalue for="0" value="How Close are Other Computer Vision Tasks to Deepfake Detection?" />
-          <attvalue for="1" value="  In this paper, we challenge the conventional belief that supervised&#10;ImageNet-trained models have strong generalizability and are suitable for use&#10;as feature extractors in deepfake detection. We present a new measurement,&#10;&quot;model separability,&quot; for visually and quantitatively assessing a model's raw&#10;capacity to separate data in an unsupervised manner. We also present a&#10;systematic benchmark for determining the correlation between deepfake detection&#10;and other computer vision tasks using pre-trained models. Our analysis shows&#10;that pre-trained face recognition models are more closely related to deepfake&#10;detection than other models. Additionally, models trained using self-supervised&#10;methods are more effective in separation than those trained using supervised&#10;methods. After fine-tuning all models on a small deepfake dataset, we found&#10;that self-supervised models deliver the best results, but there is a risk of&#10;overfitting. Our results provide valuable insights that should help researchers&#10;and practitioners develop more effective deepfake detection models.&#10;" />
-          <attvalue for="2" value="&#10;Deepfake detection (DFD) has become an increasingly important task in recent years with the proliferation of high-quality generative models~\cite{tolosana2020deepfakes, rathgeb2022handbook}. DFD is a challenging task due to the sophistication of the generated content. As a result, most detection methods are deep-learning-based~\cite{malik2022deepfake} ones utilizing high-capacity models. However, the lack of large-scale and diverse deepfake datasets makes it difficult to effectively train models from scratch. To overcome this challenge, the use of transfer learning and fine-tuning has become a commonly used approach~\cite{rossler2019faceforensics++, nguyen2019capsule, ferrer2020deepfake, zhao2021multi, sun2022dual} in which researchers and practitioners leverage pre-trained backbones from computer vision (CV)-related models trained on ImageNet~\cite{deng2009imagenet}. In this way, the learned features from the pre-training can be transferred to the DFD task, making it possible to build more effective models even with limited training data. &#10;&#10;This paper challenges this conventional practice of using pre-trained image classification backbones for deepfake detection. While ImageNet has played a vital role in the success of several deep learning models, such as VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, XceptionNet~\cite{chollet2017xception}, and EfficientNet~\cite{tan2019efficientnet}, the effectiveness of the pre-trained backbones for the DFD task remains uncertain. Given that (facial) DFD is focused on the human face, we hypothesize that pre-trained backbones from facial-related tasks, such as face recognition and age estimation, may also be suitable. Additionally, the potential of self-supervised learning in learning effective representations suitable for various tasks has recently become recognized~\cite{schmarje2021survey}. Hence, there is a need for a systematic benchmark for evaluating the suitability of these backbones for the DFD task.&#10;&#10;To ensure the accuracy of our systematic benchmark, we have established the following principles for conducting experiments in this paper:&#10;\begin{itemize}&#10; \item Type of DFD models: Since our benchmark focus on the backbones, we choose the most basic DFD approach, which takes images (video frames in the case of videos) as input. Such models from this approach consist of a backbone and a fully connected (FC) classifier. More complex models may utilize additional features or have complex modules, which may introduce nuisance factors making the benchmark inaccurate or biased. If a backbone demonstrates good separability on a simple model, it will likely perform well on more complex models.&#10; \item Type of deepfakes: We focus on popular types of facial deepfakes~\cite{tolosana2020deepfakes}, comprising identity swaps, expression swaps, and entire face synthesis, whose datasets can be found or easily generated. We crop the faces from images or video frames during pre-processing.&#10; \item Datasets: Since our objective is to measure the closeness between tasks, we use a small dataset for fine-tuning to assess whether the backbones can easily adapt to the DFD task and avoid catastrophic forgetting. The closeness measurement becomes meaningless if a model almost forgets its previously learned representations. A small dataset can also help reveal the model under-fitting or over-fitting, indicating that the original and DFD tasks are not closely related.&#10; \item Metrics: Most benchmark methods for DFD rely on quantitative scores such as accuracy, equal error rate (EER), area under the curve (AUC), or half total error rate (HTER)~\cite{abdullakutty2021review}, which often lack explainability and rely heavily on the construction of the test set, especially when evaluating generalizability. Herefore, adding a more intuitive measurement is needed for DFD evaluation.&#10;\end{itemize}&#10;&#10;In summary, our contributions are threefold:&#10;\begin{itemize}&#10; \item We present a new measurement, ``backbone separability,'' for visually and quantitatively assessing a backbone's ability to separate data into pre-defined classes in an unsupervised manner using dimensional reduction and clustering.&#10; \item We present a systematic benchmark for measuring the similarity between DFD tasks and other CV tasks by evaluating the separability of various pre-trained and fine-tuned backbones in detecting different kinds of deepfakes.&#10; \item We demonstrate that fine-tuning and transfer learning with self-supervised backbones provide the best performance when dealing with a small deepfake dataset, although with a potential risk of overfitting. These insights can inform the development of more effective countermeasures against deepfakes, particularly for detecting unseen ones.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Model Generalizability, Mathematics, Deepfake Detection, Artificial Intelligence, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2001.00179" label="2001.00179">
-        <attvalues>
-          <attvalue for="0" value="DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection" />
-          <attvalue for="1" value="  The free access to large-scale public databases, together with the fast&#10;progress of deep learning techniques, in particular Generative Adversarial&#10;Networks, have led to the generation of very realistic fake content with its&#10;corresponding implications towards society in this era of fake news. This&#10;survey provides a thorough review of techniques for manipulating face images&#10;including DeepFake methods, and methods to detect such manipulations. In&#10;particular, four types of facial manipulation are reviewed: i) entire face&#10;synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv)&#10;expression swap. For each manipulation group, we provide details regarding&#10;manipulation techniques, existing public databases, and key benchmarks for&#10;technology evaluation of fake detection methods, including a summary of results&#10;from those evaluations. Among all the aspects discussed in the survey, we pay&#10;special attention to the latest generation of DeepFakes, highlighting its&#10;improvements and challenges for fake detection.&#10;  In addition to the survey information, we also discuss open issues and future&#10;trends that should be considered to advance in the field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.08971" label="1901.08971">
-        <attvalues>
-          <attvalue for="0" value="FaceForensics++: Learning to Detect Manipulated Facial Images" />
-          <attvalue for="1" value="  The rapid progress in synthetic image generation and manipulation has now&#10;come to a point where it raises significant concerns for the implications&#10;towards society. At best, this leads to a loss of trust in digital content, but&#10;could potentially cause further harm by spreading false information or fake&#10;news. This paper examines the realism of state-of-the-art image manipulations,&#10;and how difficult it is to detect them, either automatically or by humans. To&#10;standardize the evaluation of detection methods, we propose an automated&#10;benchmark for facial manipulation detection. In particular, the benchmark is&#10;based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent&#10;representatives for facial manipulations at random compression level and size.&#10;The benchmark is publicly available and contains a hidden test set as well as a&#10;database of over 1.8 million manipulated images. This dataset is over an order&#10;of magnitude larger than comparable, publicly available, forgery datasets.&#10;Based on this data, we performed a thorough analysis of data-driven forgery&#10;detectors. We show that the use of additional domainspecific knowledge improves&#10;forgery detection to unprecedented accuracy, even in the presence of strong&#10;compression, and clearly outperforms human observers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.11946" label="1905.11946">
-        <attvalues>
-          <attvalue for="0" value="EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks" />
-          <attvalue for="1" value="  Convolutional Neural Networks (ConvNets) are commonly developed at a fixed&#10;resource budget, and then scaled up for better accuracy if more resources are&#10;available. In this paper, we systematically study model scaling and identify&#10;that carefully balancing network depth, width, and resolution can lead to&#10;better performance. Based on this observation, we propose a new scaling method&#10;that uniformly scales all dimensions of depth/width/resolution using a simple&#10;yet highly effective compound coefficient. We demonstrate the effectiveness of&#10;this method on scaling up MobileNets and ResNet.&#10;  To go even further, we use neural architecture search to design a new&#10;baseline network and scale it up to obtain a family of models, called&#10;EfficientNets, which achieve much better accuracy and efficiency than previous&#10;ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.3%&#10;top-1 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on&#10;inference than the best existing ConvNet. Our EfficientNets also transfer well&#10;and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%),&#10;and 3 other transfer learning datasets, with an order of magnitude fewer&#10;parameters. Source code is at&#10;https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.02445" label="2203.02445">
-        <attvalues>
-          <attvalue for="0" value="SFPN: Synthetic FPN for Object Detection" />
-          <attvalue for="1" value="  FPN (Feature Pyramid Network) has become a basic component of most SoTA one&#10;stage object detectors. Many previous studies have repeatedly proved that FPN&#10;can caputre better multi-scale feature maps to more precisely describe objects&#10;if they are with different sizes. However, for most backbones such VGG, ResNet,&#10;or DenseNet, the feature maps at each layer are downsized to their quarters due&#10;to the pooling operation or convolutions with stride 2. The gap of&#10;down-scaling-by-2 is large and makes its FPN not fuse the features smoothly.&#10;This paper proposes a new SFPN (Synthetic Fusion Pyramid Network) arichtecture&#10;which creates various synthetic layers between layers of the original FPN to&#10;enhance the accuracy of light-weight CNN backones to extract objects' visual&#10;features more accurately. Finally, experiments prove the SFPN architecture&#10;outperforms either the large backbone VGG16, ResNet50 or light-weight backbones&#10;such as MobilenetV2 based on AP score.&#10;" />
-          <attvalue for="2" value="&#10;Many past studies\cite{ssd,fpn,retinanet,yolov2} have shown that feature maps in a feature pyramid can capture an object's visual features at different scales. The shallow layers retain details, such as texture, corner, and so on; the deep layers cover a broader range of semantic features. In real scenes, objects with different sizes often appear together, and how to detect them simultaneously becomes a critical problem. This way makes FPN~\cite{fpn} significantly improve object detection performance and become a standard component of most SoTA object detectors~\cite{yolov1,prb}. However,&#10;the feaure maps in this FPN are scaled to 1/2, 1/4, 1/8, and so on in both the $x$ and $y$ directions. The scale gap between two adjacent layers of FPN is large and causes two objects with similar sizes to be predicted and categorized to different layers. For example, two objects with the dimensions 32$\times$32 and 31$\times$31 are at different prediction maps in the FPN. This scale truncation problem can be improved by adding a synthetic layer to this FPN.&#10;&#10;This paper proposes a new SFPN (Syntethic Fusion Pyramid Network) (see Fig. \ref{SFPN}) to make the density map scaled to 1/2, 1/3, 1/4, 1/6 and so on for reducing the effect of scaling truncation. We believe that the adding of middle-scale feature map can make the transition of different scales smoother for better object detection especitally on a lightweight architecture. We build a baseline with only three outputs layers based on this idea, then gradually synthetically insert the middle-scale layers. The FPNs is called Syntethic Fusion Pyramid Network (SFPN). Then, the SFPN is integrated on VGG-16~\cite{vgg} and MobileNetV2~\cite{mobilenetv2}, respectively, and evaluated in the MS-COCO~\cite{mscoco} dataset. Unsurprisingly, we find that both large and lightweight backbones can benefit from the synthetic layers. The same scheme is applied again to separate the final output layers to more layers and surprisingly still outperforms the baseline. Furthermore, we visualize the confidence maps of the SFPN to show how the synthetic layers can help the model fit the objects better. The synthetic layers can make the original layers retain more information for object representation. The findings confirm the performance gain of synthetic layers, and we hope that more variants of this promising method can be derived.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Deep Learning Architectures, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2012.01724" label="2012.01724">
-        <attvalues>
-          <attvalue for="0" value="Parallel Residual Bi-Fusion Feature Pyramid Network for Accurate&#10;  Single-Shot Object Detection" />
-          <attvalue for="1" value="  This paper proposes the Parallel Residual Bi-Fusion Feature Pyramid Network&#10;(PRB-FPN) for fast and accurate single-shot object detection. Feature Pyramid&#10;(FP) is widely used in recent visual detection, however the top-down pathway of&#10;FP cannot preserve accurate localization due to pooling shifting. The advantage&#10;of FP is weakened as deeper backbones with more layers are used. In addition,&#10;it cannot keep up accurate detection of both small and large objects at the&#10;same time. To address these issues, we propose a new parallel FP structure with&#10;bi-directional (top-down and bottom-up) fusion and associated improvements to&#10;retain high-quality features for accurate localization. We provide the&#10;following design improvements: (1) A parallel bifusion FP structure with a&#10;bottom-up fusion module (BFM) to detect both small and large objects at once&#10;with high accuracy. (2) A concatenation and re-organization (CORE) module&#10;provides a bottom-up pathway for feature fusion, which leads to the&#10;bi-directional fusion FP that can recover lost information from lower-layer&#10;feature maps. (3) The CORE feature is further purified to retain richer&#10;contextual information. Such CORE purification in both top-down and bottom-up&#10;pathways can be finished in only a few iterations. (4) The adding of a residual&#10;design to CORE leads to a new Re-CORE module that enables easy training and&#10;integration with a wide range of deeper or lighter backbones. The proposed&#10;network achieves state-of-the-art performance on the UAVDT17 and MS COCO&#10;datasets. Code is available at https://github.com/pingyang1117/PRBNet_PyTorch.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.04381" label="1801.04381">
-        <attvalues>
-          <attvalue for="0" value="MobileNetV2: Inverted Residuals and Linear Bottlenecks" />
-          <attvalue for="1" value="  In this paper we describe a new mobile architecture, MobileNetV2, that&#10;improves the state of the art performance of mobile models on multiple tasks&#10;and benchmarks as well as across a spectrum of different model sizes. We also&#10;describe efficient ways of applying these mobile models to object detection in&#10;a novel framework we call SSDLite. Additionally, we demonstrate how to build&#10;mobile semantic segmentation models through a reduced form of DeepLabv3 which&#10;we call Mobile DeepLabv3.&#10;  The MobileNetV2 architecture is based on an inverted residual structure where&#10;the input and output of the residual block are thin bottleneck layers opposite&#10;to traditional residual models which use expanded representations in the input&#10;an MobileNetV2 uses lightweight depthwise convolutions to filter features in&#10;the intermediate expansion layer. Additionally, we find that it is important to&#10;remove non-linearities in the narrow layers in order to maintain&#10;representational power. We demonstrate that this improves performance and&#10;provide an intuition that led to this design. Finally, our approach allows&#10;decoupling of the input/output domains from the expressiveness of the&#10;transformation, which provides a convenient framework for further analysis. We&#10;measure our performance on Imagenet classification, COCO object detection, VOC&#10;image segmentation. We evaluate the trade-offs between accuracy, and number of&#10;operations measured by multiply-adds (MAdd), as well as the number of&#10;parameters&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.06683" label="1912.06683">
-        <attvalues>
-          <attvalue for="0" value="LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic image segmentation plays a pivotal role in many vision applications&#10;including autonomous driving and medical image analysis. Most of the former&#10;approaches move towards enhancing the performance in terms of accuracy with a&#10;little awareness of computational efficiency. In this paper, we introduce&#10;LiteSeg, a lightweight architecture for semantic image segmentation. In this&#10;work, we explore a new deeper version of Atrous Spatial Pyramid Pooling module&#10;(ASPP) and apply short and long residual connections, and depthwise separable&#10;convolution, resulting in a faster and efficient model. LiteSeg architecture is&#10;introduced and tested with multiple backbone networks as Darknet19, MobileNet,&#10;and ShuffleNet to provide multiple trade-offs between accuracy and&#10;computational cost. The proposed model LiteSeg, with MobileNetV2 as a backbone&#10;network, achieves an accuracy of 67.81% mean intersection over union at 161&#10;frames per second with $640 \times 360$ resolution on the Cityscapes dataset.&#10;" />
-          <attvalue for="2" value="&#10;Semantic image segmentation is defined as the assigning of every pixel in a given image to a specific categorical label. Semantic segmentation~\cite{long2015fully, deeplabv3plus, zhao2017pspnet,deeplabv3}, and similar to image classification~\cite{xie2017aggregated,resnet,szegedy2015going} and object detection~\cite{yolo,lin2017focal}, has seen considerable progress due to the employment of deep learning architectures, especially convolutional neural networks (CNN). This progress has resulted in a much better quality of real-world applications, such as autonomous driving, medical diagnosis~\cite{ronneberger2015u}, and aerial image segmentation~\cite{marmanis2016semantic}. &#10;&#10;Despite the high accuracy achieved by recent proposed architectures~\cite{deeplabv3plus,zhao2017pspnet} for semantic segmentation, they are not computationally efficient especially for the applications that are needed to be run on edge devices, such as autonomous driving cars, robots, or augmented reality kits. Numerous attempts have been investigated in providing lightweight semantic segmentation architectures, such as ERFNet~\cite{erfnet}, ESPNet~\cite{espnet}, Enet~\cite{enet}, CCC~\cite{ccc2}, and DSNet~\cite{dsnet}. Some of these lightweight architectures attempts aimed at obtaining real-time performance with a considerable reduction in network parameters which significantly causes a loss in accuracy measures~\cite{espnet, enet, rtseg,ccc2}. Other methods paid more attention to both accuracy and real-time performance which leads to gain a better real-time performance when compared to complex networks and a better accuracy than the first group~\cite{erfnet, dsnet}. &#10;\\&#10;Semantic segmentation. The Fully Convolutional Network (FCN)~\cite{long2015fully} is a pivotal approach which paves the way to employ deep learning methods into the semantic segmentation problem. In FCN, a classification model such as, GoogleNet~\cite{szegedy2015going} or VGG~\cite{simonyan2014very} was used as an encoder to extract features from tested images and then these feature maps were upsampled to pixelwise dense predictions by cascaded layers of unpooling and deconvolution operations. Accuracy of this architecture was improved by using skip architecture, in which semantic information from deep layer and spatial information from the earlier layers were combined to get better results. Despite the breakthrough of FCN architecture, it has suffered from low resolution prediction. Many variant of FCN are proposed to solve this problem. For example, the work in~\cite{eigen2015predicting} proposed a multi-scale network which employed a different three scale to generate a fine, high resolution predictions. Another solution was proposed by ~\cite{noh2015learning}, in which a more complex deconvolution network was used to produce high resolution predictions instead of the used one in~\cite{long2015fully} which used a single bilinear interpolation layer. A different approach~\cite{chen2018deeplab} employed dilated convolution to increase the receptive field without any increase the in number of parameters and computational cost, followed by bilinear interpolation layers to scale up the feature maps to the input image size. Then a conditional random field (CRF)~\cite{krahenbuhl2011efficient} was used as a post processing to refine the result image.&#10;PSPNet~\cite{zhao2017pspnet}, Deeplabv3~\cite{deeplabv3}, Deeplabv3+~\cite{deeplabv3plus} capture information at multiple scales by either applying pooling operations with different kernel size and they called it pyramid pooling module (PPM) or employing dilated convolution with different rate and that was called Atrous Spatial Pyramid Pooling (ASPP).&#10;\\&#10;&#10;Real-Time Segmentation. Most of the mentioned approaches are not efficient for real-time applications as they employed large backbone networks such as GoogleNet~\cite{szegedy2015going}, Xcepetion~\cite{simonyan2014very}, or ResNet~\cite{resnet}, or employed a large CNN architectures for both the encoder or decoder sides. This has lead to having a large number of parameters to be tuned and a large floating point operations (FLOPS), even though they are efficient form accuracy perspective.&#10;Many approaches have been proposed to deal with this problem, e.g., ERFNet~\cite{erfnet} employed a residual connection and depthwise separable convolution to increase receptive field to achieve high accuracy with a reasonable performance. Alternatively, ESPNet~\cite{espnet} proposed an efficient module called efficient spatial pyramid (ESP), which uses point wise convolution and spatial pyramid of dilated convolution. ESPnet along with Enet provide a lightweight architectures but with a degradation in accuracy. RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (i.e., VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (i.e., UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently. They have found out that using SkipNet architecture along with MobileNet and ShuffleNet provided the best trade-off between accuracy and performance.&#10;\\Motivated by the encoder-decoder architecture, Atours Spatial Pyramid Pooling (ASPP), dilated convolution, and depthwise separable convolution, we design a novel architecture called LiteSeg which is capable of adapting any backbone network. This capability would allow a variety in trade-offs between computational cost and accuracy to fit multiple needs by choosing different backbone networks.&#10;\\&#10;In summary, our main contributions are:&#10;\begin{itemize}&#10;\item LiteSeg, a real time competitive architecture is presented and tested with three different backbone networks, Darknet19~\cite{yolo}, MobileNetV2~\cite{mobilenetv2}, and ShuffleNet~\cite{shufflenet}, achieving performance 70.75\%, 67.81\%, and 65.17\%, respectively on Cityscapes dataset.&#10;\item A new deeper version of ASPP module is adapted to improve the results along with using long and short residual connection.&#10;\end{itemize}&#10;The rest of the paper is organized as follows. Section 2 describes the proposed architecture, LiteSeg, in details. In Section 3, both the accuracy and the computational efficiency of the proposed model is evaluated and the paper is finally concluded in Section 4.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1803.06815" label="1803.06815">
-        <attvalues>
-          <attvalue for="0" value="ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  We introduce a fast and efficient convolutional neural network, ESPNet, for&#10;semantic segmentation of high resolution images under resource constraints.&#10;ESPNet is based on a new convolutional module, efficient spatial pyramid (ESP),&#10;which is efficient in terms of computation, memory, and power. ESPNet is 22&#10;times faster (on a standard GPU) and 180 times smaller than the&#10;state-of-the-art semantic segmentation network PSPNet, while its category-wise&#10;accuracy is only 8% less. We evaluated ESPNet on a variety of semantic&#10;segmentation datasets including Cityscapes, PASCAL VOC, and a breast biopsy&#10;whole slide image dataset. Under the same constraints on memory and&#10;computation, ESPNet outperforms all the current efficient CNN networks such as&#10;MobileNet, ShuffleNet, and ENet on both standard metrics and our newly&#10;introduced performance metrics that measure efficiency on edge devices. Our&#10;network can process high resolution images at a rate of 112 and 9 frames per&#10;second on a standard GPU and edge device, respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.02147" label="1606.02147">
-        <attvalues>
-          <attvalue for="0" value="ENet: A Deep Neural Network Architecture for Real-Time Semantic&#10;  Segmentation" />
-          <attvalue for="1" value="  The ability to perform pixel-wise semantic segmentation in real-time is of&#10;paramount importance in mobile applications. Recent deep neural networks aimed&#10;at this task have the disadvantage of requiring a large number of floating&#10;point operations and have long run-times that hinder their usability. In this&#10;paper, we propose a novel deep neural network architecture named ENet&#10;(efficient neural network), created specifically for tasks requiring low&#10;latency operation. ENet is up to 18$\times$ faster, requires 75$\times$ less&#10;FLOPs, has 79$\times$ less parameters, and provides similar or better accuracy&#10;to existing models. We have tested it on CamVid, Cityscapes and SUN datasets&#10;and report on comparisons with existing state-of-the-art methods, and the&#10;trade-offs between accuracy and processing time of a network. We present&#10;performance measurements of the proposed architecture on embedded systems and&#10;suggest possible software improvements that could make ENet even faster.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.07049" label="1812.07049">
-        <attvalues>
-          <attvalue for="0" value="DSNet for Real-Time Driving Scene Semantic Segmentation" />
-          <attvalue for="1" value="  We focus on the very challenging task of semantic segmentation for autonomous&#10;driving system. It must deliver decent semantic segmentation result for traffic&#10;critical objects real-time. In this paper, we propose a very efficient yet&#10;powerful deep neural network for driving scene semantic segmentation termed as&#10;Driving Segmentation Network (DSNet). DSNet achieves state-of-the-art balance&#10;between accuracy and inference speed through efficient units and architecture&#10;design inspired by ShuffleNet V2 and ENet. More importantly, DSNet highlights&#10;classes most critical with driving decision making through our novel Driving&#10;Importance-weighted Loss. We evaluate DSNet on Cityscapes dataset, our DSNet&#10;achieves 71.8% mean Intersection-over-Union (IoU) on validation set and 69.3%&#10;on test set. Class-wise IoU scores show that Driving Importance-weighted Loss&#10;could improve most driving critical classes by a large margin. Compared with&#10;ENet, DSNet is 18.9% more accurate and 1.1+ times faster which implies great&#10;potential for autonomous driving application.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.04366" label="1505.04366">
-        <attvalues>
-          <attvalue for="0" value="Learning Deconvolution Network for Semantic Segmentation" />
-          <attvalue for="1" value="  We propose a novel semantic segmentation algorithm by learning a&#10;deconvolution network. We learn the network on top of the convolutional layers&#10;adopted from VGG 16-layer net. The deconvolution network is composed of&#10;deconvolution and unpooling layers, which identify pixel-wise class labels and&#10;predict segmentation masks. We apply the trained network to each proposal in an&#10;input image, and construct the final semantic segmentation map by combining the&#10;results from all proposals in a simple manner. The proposed algorithm mitigates&#10;the limitations of the existing methods based on fully convolutional networks&#10;by integrating deep deconvolution network and proposal-wise prediction; our&#10;segmentation method typically identifies detailed structures and handles&#10;objects in multiple scales naturally. Our network demonstrates outstanding&#10;performance in PASCAL VOC 2012 dataset, and we achieve the best accuracy&#10;(72.5%) among the methods trained with no external data through ensemble with&#10;the fully convolutional network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1210.5644" label="1210.5644">
-        <attvalues>
-          <attvalue for="0" value="Efficient Inference in Fully Connected CRFs with Gaussian Edge&#10;  Potentials" />
-          <attvalue for="1" value="  Most state-of-the-art techniques for multi-class image segmentation and&#10;labeling use conditional random fields defined over pixels or image regions.&#10;While region-level models often feature dense pairwise connectivity,&#10;pixel-level models are considerably larger and have only permitted sparse graph&#10;structures. In this paper, we consider fully connected CRF models defined on&#10;the complete set of pixels in an image. The resulting graphs have billions of&#10;edges, making traditional inference algorithms impractical. Our main&#10;contribution is a highly efficient approximate inference algorithm for fully&#10;connected CRF models in which the pairwise edge potentials are defined by a&#10;linear combination of Gaussian kernels. Our experiments demonstrate that dense&#10;connectivity at the pixel level substantially improves segmentation and&#10;labeling accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.02758" label="1803.02758">
-        <attvalues>
-          <attvalue for="0" value="RTSeg: Real-time Semantic Segmentation Comparative Study" />
-          <attvalue for="1" value="  Semantic segmentation benefits robotics related applications especially&#10;autonomous driving. Most of the research on semantic segmentation is only on&#10;increasing the accuracy of segmentation models with little attention to&#10;computationally efficient solutions. The few work conducted in this direction&#10;does not provide principled methods to evaluate the different design choices&#10;for segmentation. In this paper, we address this gap by presenting a real-time&#10;semantic segmentation benchmarking framework with a decoupled design for&#10;feature extraction and decoding methods. The framework is comprised of&#10;different network architectures for feature extraction such as VGG16, Resnet18,&#10;MobileNet, and ShuffleNet. It is also comprised of multiple meta-architectures&#10;for segmentation that define the decoding methodology. These include SkipNet,&#10;UNet, and Dilation Frontend. Experimental results are presented on the&#10;Cityscapes dataset for urban scenes. The modular design allows novel&#10;architectures to emerge, that lead to 143x GFLOPs reduction in comparison to&#10;SegNet. This benchmarking framework is publicly available at&#10;&quot;https://github.com/MSiam/TFSegmentation&quot;.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1707.01083" label="1707.01083">
-        <attvalues>
-          <attvalue for="0" value="ShuffleNet: An Extremely Efficient Convolutional Neural Network for&#10;  Mobile Devices" />
-          <attvalue for="1" value="  We introduce an extremely computation-efficient CNN architecture named&#10;ShuffleNet, which is designed specially for mobile devices with very limited&#10;computing power (e.g., 10-150 MFLOPs). The new architecture utilizes two new&#10;operations, pointwise group convolution and channel shuffle, to greatly reduce&#10;computation cost while maintaining accuracy. Experiments on ImageNet&#10;classification and MS COCO object detection demonstrate the superior&#10;performance of ShuffleNet over other structures, e.g. lower top-1 error&#10;(absolute 7.8%) than recent MobileNet on ImageNet classification task, under&#10;the computation budget of 40 MFLOPs. On an ARM-based mobile device, ShuffleNet&#10;achieves ~13x actual speedup over AlexNet while maintaining comparable&#10;accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.03098" label="2107.03098">
-        <attvalues>
-          <attvalue for="0" value="Greedy Offset-Guided Keypoint Grouping for Human Pose Estimation" />
-          <attvalue for="1" value="  We propose a simple yet reliable bottom-up approach with a good trade-off&#10;between accuracy and efficiency for the problem of multi-person pose&#10;estimation. Given an image, we employ an Hourglass Network to infer all the&#10;keypoints from different persons indiscriminately as well as the guiding&#10;offsets connecting the adjacent keypoints belonging to the same persons. Then,&#10;we greedily group the candidate keypoints into multiple human poses (if any),&#10;utilizing the predicted guiding offsets. And we refer to this process as greedy&#10;offset-guided keypoint grouping (GOG). Moreover, we revisit the&#10;encoding-decoding method for the multi-person keypoint coordinates and reveal&#10;some important facts affecting accuracy. Experiments have demonstrated the&#10;obvious performance improvements brought by the introduced components. Our&#10;approach is comparable to the state of the art on the challenging COCO dataset&#10;under fair conditions. The source code and our pre-trained model are publicly&#10;available online.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The problem of multi-person pose estimation is to localize the 2D skeleton keypoints (body joints) for all persons given a single image \cite{Ronchi2017Benchmarking, Papandreou2018PersonLab, li2020simple}. It is a fundamental task with many applications.&#10;Some recent work can address the problem of single-person pose estimation extremely well thanks to the development of convolutional neural networks (CNNs) specially designed for human pose estimation such as Hourglass Network \cite{Newell2016Stacked, law2018cornernet, li2020simple} and HRNet \cite{sun2019deep}. But the challenge becomes much more difficult when multiple persons appear in the scene at the same time. And it has not been solved well so far considering accuracy, speed and simplicity.&#10;&#10;Approach taxonomy. Existing approaches for this problem can be roughly divided into two categories: top-down and bottom-up. Some recent top-down approaches \cite{zhang2020distribution, Chen2017Cascaded, Papandreou2017Towards, sun2019deep, He2017Mask} have achieved high accuracy. However, most of them have low prediction efficiency especially when many people exist in the scene, for they must rely on an advanced person detector to detect all the persons and estimate all the single-person poses one by one. The bottom-up approaches \cite{Cao2017Realtime, Newell2017Associative, kreiss2019pifpaf, li2020simple, cheng2019higherhrnet} instead infer the keypoint positions and corresponding grouping cues of all persons indiscriminately using a feed-forward network (here, we mainly discuss CNN-based approaches) and then group the detected keypoints into individual human poses. The run time of the grouping process may be very fast and nearly constant regardless of the person number in the image.&#10;Therefore, developing bottom-up approaches is challenging but attractive for a better trade-off between accuracy and efficiency. In this paper, we mainly focus on the bottom-up approaches.&#10;&#10;Keypoint coordinate encoding. Here, we only review the most important work. With the arrival of the deep learning era, DeepPose \cite{Toshev2013DeepPose} for the first time uses CNNs to regress the Cartesian coordinates of a fixed number of person keypoints directly. As a result, this approach cannot handle the situation of multiple persons. PersonLab \cite{Papandreou2018PersonLab}, CenterNet \cite{Zhou:2019ta} and PifPaf \cite{kreiss2019pifpaf} decompose the task of keypoint localization into two subproblems at each position: binary classification to discriminate whether this current pixel is a keypoint, and offset regression to the ground-truth keypoint position. However, special techniques are essential to make their approaches work well, e.g., leveraging focal loss \cite{Lin2017Focal} for the classification task and Laplace-based L1 loss \cite{kendall2017uncertainties} for the offset regression task \cite{kreiss2019pifpaf}. By contrast, Tompson et al. \cite{Tompson2014Joint} and many later researchers use CNNs to predict the Gaussian response heatmaps of person keypoints and then obtain the keypoint coordinates by finding the local maximum responses in the heatmaps. But high-res feature maps are required to alleviate the precision decline of keypoint localization.&#10;&#10;Keypoint grouping encoding.&#10;The encoding of keypoint grouping (or association) information is critical for the post-processing in bottom-up approaches. &#10;Here, we conclude the existing approaches for keypoint grouping into two categories: global grouping and greedy grouping. Prior work, such as DeeperCut \cite{Insafutdinov2016DeeperCut}, Associative Embedding \cite{Newell2017Associative} and CenterNet \cite{Zhou:2019ta}, presents different approaches to encode the global keypoint grouping information. For instance, Associative Embedding employs Hourglass Networks \cite{Newell2016Stacked} to infer the identity tags for all candidate keypoints. These tags are grouping cues to cluster keypoints into individual human poses. And CenterNet \cite{Zhou:2019ta} proposes center offsets away from person centers as keypoint grouping cues. By contrast, Part Affinity Fields \cite{Cao2017Realtime}, PersonLab \cite{Papandreou2018PersonLab} and PIFPAF \cite{kreiss2019pifpaf} are greedy grouping approaches, associating adjacent nodes in a human skeleton tree independently rather than finding the global solution of a global graph matching problem. &#10;&#10;Some state-of-the-art (SOTA) bottom-up approaches are remarkable in estimation accuracy but have so complicated structure and many hyper-parameters affecting results that we cannot clearly figure out the contributions of introduced components or compare these approaches equally. &#10;In this work, we propose a bottom-up approach, which is easy to follow and works well in terms of accuracy, speed and clarity. We use Hourglass-104 \cite{law2018cornernet} as the inference model and select CenterNet \cite{Zhou:2019ta} as the baseline approach. The main contributions of this paper are as follows:&#10;&#10;(1) We present a greedy keypoint grouping method, which we refer to as greedy offset-guided keypoint grouping (GOG). The adjacent keypoints of each person are connected by guiding offsets. The proposed GOG algorithm is robust and fast.&#10;&#10;(2) We follow and improve the existing Gaussian heatmap encoding-decoding method for keypoint coordinates of the multi-person poses, ensuring precise keypoint localization. &#10;&#10;(3) Our preliminary system has obtained obvious performance increases compared with the baseline. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Multi-Person Pose Estimation, Computer Vision Techniques, Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning Architectures, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1910.06278" label="1910.06278">
-        <attvalues>
-          <attvalue for="0" value="Distribution-Aware Coordinate Representation for Human Pose Estimation" />
-          <attvalue for="1" value="  While being the de facto standard coordinate representation in human pose&#10;estimation, heatmap is never systematically investigated in the literature, to&#10;our best knowledge. This work fills this gap by studying the coordinate&#10;representation with a particular focus on the heatmap. Interestingly, we found&#10;that the process of decoding the predicted heatmaps into the final joint&#10;coordinates in the original image space is surprisingly significant for human&#10;pose estimation performance, which nevertheless was not recognised before. In&#10;light of the discovered importance, we further probe the design limitations of&#10;the standard coordinate decoding method widely used by existing methods, and&#10;propose a more principled distribution-aware decoding method. Meanwhile, we&#10;improve the standard coordinate encoding process (i.e. transforming&#10;ground-truth coordinates to heatmaps) by generating accurate heatmap&#10;distributions for unbiased model training. Taking the two together, we&#10;formulate a novel Distribution-Aware coordinate Representation of Keypoint&#10;(DARK) method. Serving as a model-agnostic plug-in, DARK significantly improves&#10;the performance of a variety of state-of-the-art human pose estimation models.&#10;Extensive experiments show that DARK yields the best results on two common&#10;benchmarks, MPII and COCO, consistently validating the usefulness and&#10;effectiveness of our novel coordinate representation idea.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.03170" label="1605.03170">
-        <attvalues>
-          <attvalue for="0" value="DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation&#10;  Model" />
-          <attvalue for="1" value="  The goal of this paper is to advance the state-of-the-art of articulated pose&#10;estimation in scenes with multiple people. To that end we contribute on three&#10;fronts. We propose (1) improved body part detectors that generate effective&#10;bottom-up proposals for body parts; (2) novel image-conditioned pairwise terms&#10;that allow to assemble the proposals into a variable number of consistent body&#10;part configurations; and (3) an incremental optimization strategy that explores&#10;the search space more efficiently thus leading both to better performance and&#10;significant speed-up factors. Evaluation is done on two single-person and two&#10;multi-person pose estimation benchmarks. The proposed approach significantly&#10;outperforms best known multi-person pose estimation results while demonstrating&#10;competitive performance on the task of single person pose estimation. Models&#10;and code available at http://pose.mpi-inf.mpg.de&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.01244" label="1808.01244">
-        <attvalues>
-          <attvalue for="0" value="CornerNet: Detecting Objects as Paired Keypoints" />
-          <attvalue for="1" value="  We propose CornerNet, a new approach to object detection where we detect an&#10;object bounding box as a pair of keypoints, the top-left corner and the&#10;bottom-right corner, using a single convolution neural network. By detecting&#10;objects as paired keypoints, we eliminate the need for designing a set of&#10;anchor boxes commonly used in prior single-stage detectors. In addition to our&#10;novel formulation, we introduce corner pooling, a new type of pooling layer&#10;that helps the network better localize corners. Experiments show that CornerNet&#10;achieves a 42.2% AP on MS COCO, outperforming all existing one-stage detectors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.14931" label="2112.14931">
-        <attvalues>
-          <attvalue for="0" value="Dense Depth Estimation from Multiple 360-degree Images Using Virtual&#10;  Depth" />
-          <attvalue for="1" value="  In this paper, we propose a dense depth estimation pipeline for multiview&#10;360{\deg} images. The proposed pipeline leverages a spherical camera model that&#10;compensates for radial distortion in 360{\deg} images. The key contribution of&#10;this paper is the extension of a spherical camera model to multiview by&#10;introducing a translation scaling scheme. Moreover, we propose an effective&#10;dense depth estimation method by setting virtual depth and minimizing photonic&#10;reprojection error. We validate the performance of the proposed pipeline using&#10;the images of natural scenes as well as the synthesized dataset for quantitive&#10;evaluation. The experimental results verify that the proposed pipeline improves&#10;estimation accuracy compared to the current state-of-art dense depth estimation&#10;methods.&#10;" />
-          <attvalue for="2" value="&#10;Dense depth estimation is an essential process that inferences scene structure from two-dimensional~(2D) images. It has been a major topic of research in computer vision for the past few decades due to the significant role it plays in various machine vision applications, including scene reconstruction~\cite{da2019dense, kim20133d, ma20153d, pathak20163d, pathak2016dense, schonbein2014omnidirectional, Yang_2019_CVPR, Fernandez_RAL_2020, Fangzheng_CSVT_2021}, view synthesis~\cite{lie2017key, jiang2017light}, navigation~\cite{yang2020d3vo, zhan2020visual, xue2019beyond}, and tracking~\cite{carrio2018drone, kart2019object}.&#10;&#10;In recent years, 360\degree\: cameras have gained more popularity in applications that require a wide field of view~(FoV) such as human-robot-interaction, intelligent vehicles, surveillance, virtual reality, and augmented reality. Unlike ordinary (perspective) cameras, typical 360\degree\: cameras are equipped with fisheye lenses covering wide angles of view~(usually, $&gt;180$\degree\:), and field of view can substantially increase using 360\degree\: cameras. However, the captured images have non-rectilinear appearances, yielding directionally dependent distortions. Due to such distortion, dense depth estimation from 360\degree\: images remains a challenge, although remarkable progress has been made in dense depth estimation from the images captured by perspective cameras~\cite{hirschmuller2007stereo, chen2001fast, wang2020fadnet, gao2020compact}. &#10;&#10;To extract depth from different view images, the images should be registered into one reference coordinate system using a camera model and image geometry. Due to the abovementioned radial distortion of 360\degree\: cameras, conventional camera models and image geometry with perspective projection cannot be directly applied to them; thereby, previous studies developed the camera models to depict the relationship between 3-dimensional~(3D) points based on spherical projection~\cite{geyer2000unifying, ying2004can, courbon2007generic, li2008binocular, pagani2011structure, ma20153d, kim20133d, im2016all, da2019dense}. Also, to accurately estimate relative camera positions from different views, the work in \cite{da2019dense} investigated a way to obtain better dense correspondences using optical flow~\cite{weinzaepfel2013deepflow} along with visual features~(e.g., SIFT~\cite{zhao2015sphorb}, SPHORB~\cite{rublee2011orb}, etc.), similar to the works in \cite{pathak2016dense, schonbein2014omnidirectional}. &#10;&#10;Toward 3D dense reconstruction from spherical images, Arican and Frossard~\cite{arican2007dense} obtained disparity maps from two views by optimizing a pixel-wise energy function using graph cuts. The work in \cite{schonbein2014omnidirectional} estimated disparity based on the plane-based prior models from two temporally and two spatially views. Kim and Hilton~\cite{kim20133d} used a partial differential equation-based disparity estimation algorithm under the assumption that image pairs were acquired in the limited configuration -- stereo spherical image pairs aligned with a vertical displacement between camera views like the setting proposed in \cite{li2008binocular}. Im et al. ~\cite{im2016all} estimated a dense depth map using the spherical sweeping algorithm inspired by the plane sweeping algorithm~\cite{collins1996space}.&#10;Owing to the development of deep learning frameworks in stereo depth estimation~\cite{chang2018pyramid, kendall2017end, Zhang_2019_CVPR}, the recent work in \cite{wang2020360sd} employed a convolutional neural network framework based on two camera configurations in a top-bottom manner~\cite{li2008binocular, kim20133d}, showing satisfactory results. Nevertheless, these works were developed in limited camera configurations, making them restricted to applying unarranged settings.&#10;Furthermore, Zioulis et al. ~\cite{zioulis2019spherical} designed a self-supervised learning framework that synthesis three views using estimated disparity maps from the convolutional neural network. More recently, the framework to fuse features from equirectangular and cubemap projections was proposed to estimate depth in monocular 360\degree\: images~\cite{Wang_2020_CVPR, 9353978}. Although the framework only requires an image to estimate depth, additional information such as cube map projection is necessary, and the estimation performance is not guaranteed.&#10;&#10;Although the previous researches have provided decent results, most approaches have been restricted to either two-view analysis or limited camera configurations. This indicates that typical spherical camera models are inadequate and possibly inferior to multiview settings. Consequently, in this paper, we develop a practical pipeline that estimates dense depth maps from multiple 360\degree\: images, as described in \fref{fig:intro}. We expand the two-view spherical camera model to multiview and estimate camera poses by taking scaling into account. The proposed pose estimation employs a scaling factor to estimate actual relative translation between cameras, in contrast to that typical pose estimation methods find solutions with unknown scale. The camera coordinates are then registered based on the estimated poses. Finally, dense depth maps are estimated by setting virtual depth and finding the optimal depth that minimizes reprojection error in corresponding regions. &#10;&#10;The remainder of this paper is organized as follows. In \sref{sec:method}, we develop the proposed dense depth estimation pipeline for multiview 360\degree\: images. Then, experimental results are shown in \sref{sec:results}, and concluding remarks are made in \sref{sec:conclusion}. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Mathematics, Camera Modeling, Physics, Image Processing, Depth Estimation, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2003.01060" label="2003.01060">
-        <attvalues>
-          <attvalue for="0" value="D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual&#10;  Odometry" />
-          <attvalue for="1" value="  We propose D3VO as a novel framework for monocular visual odometry that&#10;exploits deep networks on three levels -- deep depth, pose and uncertainty&#10;estimation. We first propose a novel self-supervised monocular depth estimation&#10;network trained on stereo videos without any external supervision. In&#10;particular, it aligns the training image pairs into similar lighting condition&#10;with predictive brightness transformation parameters. Besides, we model the&#10;photometric uncertainties of pixels on the input images, which improves the&#10;depth estimation accuracy and provides a learned weighting function for the&#10;photometric residuals in direct (feature-less) visual odometry. Evaluation&#10;results show that the proposed network outperforms state-of-the-art&#10;self-supervised depth estimation networks. D3VO tightly incorporates the&#10;predicted depth, pose and uncertainty into a direct visual odometry method to&#10;boost both the front-end tracking as well as the back-end non-linear&#10;optimization. We evaluate D3VO in terms of monocular visual odometry on both&#10;the KITTI odometry benchmark and the EuRoC MAV dataset.The results show that&#10;D3VO outperforms state-of-the-art traditional monocular VO methods by a large&#10;margin. It also achieves comparable results to state-of-the-art stereo/LiDAR&#10;odometry on KITTI and to the state-of-the-art visual-inertial odometry on EuRoC&#10;MAV, while using only a single camera.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.00259" label="1808.00259">
-        <attvalues>
-          <attvalue for="0" value="Drone Detection Using Depth Maps" />
-          <attvalue for="1" value="  Obstacle avoidance is a key feature for safe Unmanned Aerial Vehicle (UAV)&#10;navigation. While solutions have been proposed for static obstacle avoidance,&#10;systems enabling avoidance of dynamic objects, such as drones, are hard to&#10;implement due to the detection range and field-of-view (FOV) requirements, as&#10;well as the constraints for integrating such systems on-board small UAVs. In&#10;this work, a dataset of 6k synthetic depth maps of drones has been generated&#10;and used to train a state-of-the-art deep learning-based drone detection model.&#10;While many sensing technologies can only provide relative altitude and azimuth&#10;of an obstacle, our depth map-based approach enables full 3D localization of&#10;the obstacle. This is extremely useful for collision avoidance, as 3D&#10;localization of detected drones is key to perform efficient collision-free path&#10;planning. The proposed detection technique has been validated in several real&#10;depth map sequences, with multiple types of drones flying at up to 2 m/s,&#10;achieving an average precision of 98.7%, an average recall of 74.7% and a&#10;record detection range of 9.5 meters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08669" label="1803.08669">
-        <attvalues>
-          <attvalue for="0" value="Pyramid Stereo Matching Network" />
-          <attvalue for="1" value="  Recent work has shown that depth estimation from a stereo pair of images can&#10;be formulated as a supervised learning task to be resolved with convolutional&#10;neural networks (CNNs). However, current architectures rely on patch-based&#10;Siamese networks, lacking the means to exploit context information for finding&#10;correspondence in illposed regions. To tackle this problem, we propose PSMNet,&#10;a pyramid stereo matching network consisting of two main modules: spatial&#10;pyramid pooling and 3D CNN. The spatial pyramid pooling module takes advantage&#10;of the capacity of global context information by aggregating context in&#10;different scales and locations to form a cost volume. The 3D CNN learns to&#10;regularize cost volume using stacked multiple hourglass networks in conjunction&#10;with intermediate supervision. The proposed approach was evaluated on several&#10;benchmark datasets. Our method ranked first in the KITTI 2012 and 2015&#10;leaderboards before March 18, 2018. The codes of PSMNet are available at:&#10;https://github.com/JiaRenChang/PSMNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.08112" label="1909.08112">
-        <attvalues>
-          <attvalue for="0" value="Spherical View Synthesis for Self-Supervised 360 Depth Estimation" />
-          <attvalue for="1" value="  Learning based approaches for depth perception are limited by the&#10;availability of clean training data. This has led to the utilization of view&#10;synthesis as an indirect objective for learning depth estimation using&#10;efficient data acquisition procedures. Nonetheless, most research focuses on&#10;pinhole based monocular vision, with scarce works presenting results for&#10;omnidirectional input. In this work, we explore spherical view synthesis for&#10;learning monocular 360 depth in a self-supervised manner and demonstrate its&#10;feasibility. Under a purely geometrically derived formulation we present&#10;results for horizontal and vertical baselines, as well as for the trinocular&#10;case. Further, we show how to better exploit the expressiveness of traditional&#10;CNNs when applied to the equirectangular domain in an efficient manner.&#10;Finally, given the availability of ground truth depth data, our work is&#10;uniquely positioned to compare view synthesis against direct supervision in a&#10;consistent and fair manner. The results indicate that alternative research&#10;directions might be better suited to enable higher quality depth perception.&#10;Our data, models and code are publicly available at&#10;https://vcl3d.github.io/SphericalViewSynthesis/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2405.09923" label="2405.09923">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2024 Restore Any Image Model (RAIM) in the Wild Challenge" />
-          <attvalue for="1" value="  In this paper, we review the NTIRE 2024 challenge on Restore Any Image Model&#10;(RAIM) in the Wild. The RAIM challenge constructed a benchmark for image&#10;restoration in the wild, including real-world images with/without reference&#10;ground truth in various scenarios from real applications. The participants were&#10;required to restore the real-captured images from complex and unknown&#10;degradation, where generative perceptual quality and fidelity are desired in&#10;the restoration result. The challenge consisted of two tasks. Task one employed&#10;real referenced data pairs, where quantitative evaluation is available. Task&#10;two used unpaired images, and a comprehensive user study was conducted. The&#10;challenge attracted more than 200 registrations, where 39 of them submitted&#10;results with more than 400 submissions. Top-ranked methods improved the&#10;state-of-the-art restoration performance and obtained unanimous recognition&#10;from all 18 judges. The proposed datasets are available at&#10;https://drive.google.com/file/d/1DqbxUoiUqkAIkExu3jZAqoElr_nu1IXb/view?usp=sharing&#10;and the homepage of this challenge is at&#10;https://codalab.lisn.upsaclay.fr/competitions/17632.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image restoration, aiming at recovering high-quality images from their low-quality counterparts, is one of the most popular low-level vision tasks in the research community. However, there has been a large gap between Academic research and Industrial application for a long time. For example, the image signal processing (ISP) systems on digital cameras always face mixed and complex degradations, yet most methods in academic research are designed and evaluated based on simulated and limited degradation. How to design and train a model that can be generalized to practical applications is a challenging yet highly valuable problem.&#10;&#10;The deep learning techniques have significantly advanced the performance of image restoration. Recently, generative adversarial networks show good performance in approximating distributions of real photos in image restoration tasks, while the large-scale pre-trained generative diffusion models have provided powerful priors to further improve the quality of image restoration outputs. &#10;&#10;This challenge aims to provide a platform for industrial and academic participants to test and evaluate their algorithms and models on real-world imaging scenarios, bridging the gap between academic research and practical photography. The objectives of this RAIM challenge are:&#10;&#10;\begin{itemize}&#10; \item Construct a benchmark for image restoration in the wild, including real-world images with/without reference ground-truth in various scenarios and objective/subjective evaluation methods;&#10; \item Promote the research and development of RAIMs with strong generalization performance to images in the wild.&#10;\end{itemize}&#10;&#10;This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Deep Learning Challenges, Mathematics, Computer Vision Benchmarking, Computer Science, Image Restoration, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2406.13007" label="2406.13007">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2024 Challenge on Night Photography Rendering" />
-          <attvalue for="1" value="  This paper presents a review of the NTIRE 2024 challenge on night photography&#10;rendering. The goal of the challenge was to find solutions that process raw&#10;camera images taken in nighttime conditions, and thereby produce a&#10;photo-quality output images in the standard RGB (sRGB) space. Unlike the&#10;previous year's competition, the challenge images were collected with a mobile&#10;phone and the speed of algorithms was also measured alongside the quality of&#10;their output. To evaluate the results, a sufficient number of viewers were&#10;asked to assess the visual quality of the proposed solutions, considering the&#10;subjective nature of the task. There were 2 nominations: quality and&#10;efficiency. Top 5 solutions in terms of output quality were sorted by&#10;evaluation time (see Fig. 1). The top ranking participants' solutions&#10;effectively represent the state-of-the-art in nighttime photography rendering.&#10;More results can be found at https://nightimaging.org.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.10343" label="2404.10343">
-        <attvalues>
-          <attvalue for="0" value="The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report" />
-          <attvalue for="1" value="  This paper provides a comprehensive review of the NTIRE 2024 challenge,&#10;focusing on efficient single-image super-resolution (ESR) solutions and their&#10;outcomes. The task of this challenge is to super-resolve an input image with a&#10;magnification factor of x4 based on pairs of low and corresponding&#10;high-resolution images. The primary objective is to develop networks that&#10;optimize various aspects such as runtime, parameters, and FLOPs, while still&#10;maintaining a peak signal-to-noise ratio (PSNR) of approximately 26.90 dB on&#10;the DIV2K_LSDIR_valid dataset and 26.99 dB on the DIV2K_LSDIR_test dataset. In&#10;addition, this challenge has 4 tracks including the main track (overall&#10;performance), sub-track 1 (runtime), sub-track 2 (FLOPs), and sub-track 3&#10;(parameters). In the main track, all three metrics (ie runtime, FLOPs, and&#10;parameter count) were considered. The ranking of the main track is calculated&#10;based on a weighted sum-up of the scores of all other sub-tracks. In sub-track&#10;1, the practical runtime performance of the submissions was evaluated, and the&#10;corresponding score was used to determine the ranking. In sub-track 2, the&#10;number of FLOPs was considered. The score calculated based on the corresponding&#10;FLOPs was used to determine the ranking. In sub-track 3, the number of&#10;parameters was considered. The score calculated based on the corresponding&#10;parameters was used to determine the ranking. RLFN is set as the baseline for&#10;efficiency measurement. The challenge had 262 registered participants, and 34&#10;teams made valid submissions. They gauge the state-of-the-art in efficient&#10;single-image super-resolution. To facilitate the reproducibility of the&#10;challenge and enable other researchers to build upon these findings, the code&#10;and the pre-trained model of validated solutions are made publicly available at&#10;https://github.com/Amazingren/NTIRE2024_ESR/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.11159" label="2404.11159">
-        <attvalues>
-          <attvalue for="0" value="Deep Portrait Quality Assessment. A NTIRE 2024 Challenge Survey" />
-          <attvalue for="1" value="  This paper reviews the NTIRE 2024 Portrait Quality Assessment Challenge,&#10;highlighting the proposed solutions and results. This challenge aims to obtain&#10;an efficient deep neural network capable of estimating the perceptual quality&#10;of real portrait photos. The methods must generalize to diverse scenes and&#10;diverse lighting conditions (indoor, outdoor, low-light), movement, blur, and&#10;other challenging conditions. In the challenge, 140 participants registered,&#10;and 35 submitted results during the challenge period. The performance of the&#10;top 5 submissions is reviewed and provided here as a gauge for the current&#10;state-of-the-art in Portrait Quality Assessment.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.16687" label="2404.16687">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2024 Quality Assessment of AI-Generated Content Challenge" />
-          <attvalue for="1" value="  This paper reports on the NTIRE 2024 Quality Assessment of AI-Generated&#10;Content Challenge, which will be held in conjunction with the New Trends in&#10;Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2024. This challenge&#10;is to address a major challenge in the field of image and video processing,&#10;namely, Image Quality Assessment (IQA) and Video Quality Assessment (VQA) for&#10;AI-Generated Content (AIGC). The challenge is divided into the image track and&#10;the video track. The image track uses the AIGIQA-20K, which contains 20,000&#10;AI-Generated Images (AIGIs) generated by 15 popular generative models. The&#10;image track has a total of 318 registered participants. A total of 1,646&#10;submissions are received in the development phase, and 221 submissions are&#10;received in the test phase. Finally, 16 participating teams submitted their&#10;models and fact sheets. The video track uses the T2VQA-DB, which contains&#10;10,000 AI-Generated Videos (AIGVs) generated by 9 popular Text-to-Video (T2V)&#10;models. A total of 196 participants have registered in the video track. A total&#10;of 991 submissions are received in the development phase, and 185 submissions&#10;are received in the test phase. Finally, 12 participating teams submitted their&#10;models and fact sheets. Some methods have achieved better results than baseline&#10;methods, and the winning methods in both tracks have demonstrated superior&#10;prediction performance on AIGC.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.16223" label="2404.16223">
-        <attvalues>
-          <attvalue for="0" value="Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey" />
-          <attvalue for="1" value="  This paper reviews the NTIRE 2024 RAW Image Super-Resolution Challenge,&#10;highlighting the proposed solutions and results. New methods for RAW&#10;Super-Resolution could be essential in modern Image Signal Processing (ISP)&#10;pipelines, however, this problem is not as explored as in the RGB domain. Th&#10;goal of this challenge is to upscale RAW Bayer images by 2x, considering&#10;unknown degradations such as noise and blur. In the challenge, a total of 230&#10;participants registered, and 45 submitted results during thee challenge period.&#10;The performance of the top-5 submissions is reviewed and provided here as a&#10;gauge for the current state-of-the-art in RAW Image Super-Resolution.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.11313" label="2404.11313">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods&#10;  and Results" />
-          <attvalue for="1" value="  This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality&#10;Assessment (S-UGC VQA), where various excellent solutions are submitted and&#10;evaluated on the collected dataset KVQ from popular short-form video platform,&#10;i.e., Kuaishou/Kwai Platform. The KVQ database is divided into three parts,&#10;including 2926 videos for training, 420 videos for validation, and 854 videos&#10;for testing. The purpose is to build new benchmarks and advance the development&#10;of S-UGC VQA. The competition had 200 participants and 13 teams submitted valid&#10;solutions for the final testing phase. The proposed solutions achieved&#10;state-of-the-art performances for S-UGC VQA. The project can be found at&#10;https://github.com/lixinustc/KVQChallenge-CVPR-NTIRE2024.&#10;" />
-          <attvalue for="2" value="&#10;Short-from UGC video platforms, \eg, Kwai, and Tiktok have attained significant success and widespread popularity, attracting billions of users globally. In contrast to long-form UGC videos, short-form UGC (S-UGC) videos present several advantages, including mobile-friendly broadcasting, user-friendly engagement, kaleidoscope content creation, and snackable content, etc.~\cite{lu2024kvq}. However, S-UGC videos inevitably suffer from inconsistent and even poor subjective quality due to unprofessional creation, improper collection environments, or limited processing workflow. Building the quality assessment benchmarks tailed for S-UGC videos is crucial and urgent to advance the fast development of associated video processing techniques and quality control mechanisms for user-uploaded S-UGC videos. &#10;&#10;Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}. Since the scarcity of the VQA database, some well-designed sampling strategies~\cite{wu2022fastFastVQA,Dover}, \egno, fragment sampling, are introduced to reduce the computational complexity of the VQA method and enhance the diversity of VQA database. Apart from that, there are also some works exploring the potential of large cross-modality foundation models~\cite{ADAQA,maxwell,wu2023qQ-Align,CLIP} for VQA. However, the above works are only designed and validated on general long UGC VQA databases, \eg, LSVQ~\cite{ying2021patchLSVQ}, YouTube-UGC~\cite{youtubeUGC}, and KoNViD-1k~\cite{KoNViD-1k}, lacking the perception capability for short-form UGC VQA. &#10;&#10;This NTIRE 2024 Short-form UGC Video Quality Assessment Challenge is held to promote related research on S-UGC VQA, and develop the powerful S-UGC VQA benchmark to assist the technique evolution of S-UGC video creation, compression, or processing, etc. In this competition, we utilize the KVQ dataset~\cite{lu2024kvq} collected from the Kwai platform to evaluate the submitted methods. The KVQ dataset contains 4200 S-UGC videos with distinct creation modes, content scenarios, and video processing workflows, satisfying the data distribution of practical short-form video platforms. For evaluation, the KVQ dataset is divided into three parts with a proportion of 7:1:2 for training, validation, and testing. Instead of only quality scores, we also provide the ranking labels for indistinguishable samples in the validation and testing parts, ensuring the fine-grained perception evaluation for S-UGC VQA methods. The evaluation metric is composed of four parts, including PLCC, SROCC, the ranking accuracies between homogeneous video pairs (\ieno, with the same content) and non-homogeneous video pairs. Notably, we have released the quality labels and ranking scores of the validation data, and reopened the test submission on Codalab~\cite{codalab_competitions_JMLR} after this competition, intending to facilitate the development of algorithms. &#10;&#10;The competition has two phases, \ie, the development and testing phases, attracting 200 registered participants in total. There are 49 teams, and 56 teams submitting their predictions in the development and testing phases, respectively. Finally, 13 teams submitted their fact sheets and validation codes for competition ranking. Their algorithms will be summarized in the Section~\ref{sec:teams_methods}. &#10;&#10;This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge.&#10;&#10;" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.14248" label="2404.14248">
-        <attvalues>
-          <attvalue for="0" value="NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results" />
-          <attvalue for="1" value="  This paper reviews the NTIRE 2024 low light image enhancement challenge,&#10;highlighting the proposed solutions and results. The aim of this challenge is&#10;to discover an effective network design or solution capable of generating&#10;brighter, clearer, and visually appealing results when dealing with a variety&#10;of conditions, including ultra-high resolution (4K and beyond), non-uniform&#10;illumination, backlighting, extreme darkness, and night scenes. A notable total&#10;of 428 participants registered for the challenge, with 22 teams ultimately&#10;making valid submissions. This paper meticulously evaluates the&#10;state-of-the-art advancements in enhancing low-light images, reflecting the&#10;significant progress and creativity in this field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.15779" label="2012.15779">
-        <attvalues>
-          <attvalue for="0" value="Illumination Estimation Challenge: experience of past two years" />
-          <attvalue for="1" value="  Illumination estimation is the essential step of computational color&#10;constancy, one of the core parts of various image processing pipelines of&#10;modern digital cameras. Having an accurate and reliable illumination estimation&#10;is important for reducing the illumination influence on the image colors. To&#10;motivate the generation of new ideas and the development of new algorithms in&#10;this field, the 2nd Illumination estimation challenge~(IEC\#2) was conducted.&#10;The main advantage of testing a method on a challenge over testing in on some&#10;of the known datasets is the fact that the ground-truth illuminations for the&#10;challenge test images are unknown up until the results have been submitted,&#10;which prevents any potential hyperparameter tuning that may be biased.&#10;  The challenge had several tracks: general, indoor, and two-illuminant with&#10;each of them focusing on different parameters of the scenes. Other main&#10;features of it are a new large dataset of images (about 5000) taken with the&#10;same camera sensor model, a manual markup accompanying each image, diverse&#10;content with scenes taken in numerous countries under a huge variety of&#10;illuminations extracted by using the SpyderCube calibration object, and a&#10;contest-like markup for the images from the Cube+ dataset that was used in&#10;IEC\#1.&#10;  This paper focuses on the description of the past two challenges, algorithms&#10;which won in each track, and the conclusions that were drawn based on the&#10;results obtained during the 1st and 2nd challenge that can be useful for&#10;similar future developments.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In the modern world of technology, a lot of modern devices, including mobile phones, tablets, and laptops are equipped with digital cameras. One of the essential parts of the image processing pipelines of these cameras is to remove the influence of the scene illumination on the image colors. To do that properly, it is first required to perform an accurate illumination estimation in an image or a video, and then this information is used for the actual color correction step~\cite{gijsenij2011color}.&#10;Human color perception system performs a similar task by means of a feature known as color constancy~\cite{ebner2007color} that allows it to recognize object coloration regardless of the scene illumination. &#10;While the mechanism of color constancy in human color perception is not fully understood, there are numerous proposed implementations of the analogous feature in digital cameras known as auto white balance (AWB), which is supposed to achieve computational color constancy. &#10;Some of the simplest illumination estimation methods such as max-RGB~\cite{land1977retinex,funt2010rehabilitation,banic2014improving} or Gray-World~\cite{buchsbaum1980spatial} and its upgrades~\cite{finlayson2004shades,van2007edge} are based on simple image statistics and their main advantage are their speed and simplicity of implementation. &#10;Over time, the best accuracy used to be achieved by learning-based methods such as Bayesian learning~\cite{gehler2008bayesian}, spatio-spectral learning~\cite{chakrabarti2012color}, illumination solution space restriction~\cite{banic2015color,banic2015using,banic2015acolor}, using color moments~\cite{finlayson2013corrected}, regression trees with simple features from color distribution statistics~\cite{cheng2015effective}, spatial localizations~\cite{barron2015convolutional,barron2017fast}, convolutional neural networks~\cite{bianco2015color,shi2016deep,hu2017fc4,oh2017approaching}, genetic algorithms~\cite{koscevic2019color}, etc.&#10;Thousands of other scientific papers about the problem of illumination estimation have been written, but it is still not solved to a satisfactory level due to the following reasons:&#10;\begin{itemize}&#10; \item Volumes of publicly available scientific datasets are not large enough to cover all various cases of illumination. &#10; \item Images in existing datasets have ground-truth scene illumination, but information about the scene is usually not provided. This excludes or complicates the process of a deeper study of the problem and its division into subtasks. For example, an efficient illumination estimation process can vary considering different times of the day. &#10; \item Considering oversimplified formulation by evaluating only a single dominant light source (there are only few publicly available datasets with multiple light sources). The uniform illumination assumption is true for images taken on a cloudy day, or in a room with a single lamp. However, this assumption is not valid on a sunny day when there are at least two light sources: the sun and the sky. Another example includes a street at night and a closed room.&#10;\end{itemize}&#10;&#10;To take a step forward towards overcoming these three obstacles, two challenges for the task of illumination estimation were organized. The First Illumination Estimation Challenge (IEC\#1) was organized as a part of the \nth{11} International Symposium on Image and Signal Processing and Analysis (ISPA 2019, September 23-25, 2019, Dubrovnik, Croatia). A year later, the Second Illumination Estimation Challenge (IEC\#2) was organized as a part of the \nth{13} International Conference on Machine Vision (ICMV 2020, November 2-6, 2020, Rome, Italy). A large illumination estimation dataset was created as a key component of the challenges. The dataset contains around 5000 images captured with two Canon cameras (Canon 600D and Canon 550D), which share the same image sensor model. For each image, ground-truth illuminations with up to two light sources per image were extracted. Additionally, various information about the scene content was provided for each image. &#10;&#10;For both challenges, the ground-truth illuminations for the images from the test set were made publicly available only after the authors have submitted their solutions. This was done in order to prevent possible hyperparameter tuning that could compromise the testing fairness. Such testing with previously unknown ground-truth illumination can be seen as a potential advantage over testing on publicly available datasets.&#10;&#10;The paper is structured as follows: Section~\ref{sec:iec_one} describes IEC\#1, Section~\ref{sec:challenge_description} describes IEC\#2, Section~\ref{sec:discussion} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Color Constancy, Illumination Estimation, Image Processing, Physics, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1608.07951" label="1608.07951">
-        <attvalues>
-          <attvalue for="0" value="Approaching the Computational Color Constancy as a Classification&#10;  Problem through Deep Learning" />
-          <attvalue for="1" value="  Computational color constancy refers to the problem of computing the&#10;illuminant color so that the images of a scene under varying illumination can&#10;be normalized to an image under the canonical illumination. In this paper, we&#10;adopt a deep learning framework for the illumination estimation problem. The&#10;proposed method works under the assumption of uniform illumination over the&#10;scene and aims for the accurate illuminant color computation. Specifically, we&#10;trained the convolutional neural network to solve the problem by casting the&#10;color constancy problem as an illumination classification problem. We designed&#10;the deep learning architecture so that the output of the network can be&#10;directly used for computing the color of the illumination. Experimental results&#10;show that our deep network is able to extract useful features for the&#10;illumination estimation and our method outperforms all previous color constancy&#10;methods on multiple test datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.02419" label="2007.02419">
-        <attvalues>
-          <attvalue for="0" value="Attention-based Joint Detection of Object and Semantic Part" />
-          <attvalue for="1" value="  In this paper, we address the problem of joint detection of objects like dog&#10;and its semantic parts like face, leg, etc. Our model is created on top of two&#10;Faster-RCNN models that share their features to perform a novel Attention-based&#10;feature fusion of related Object and Part features to get enhanced&#10;representations of both. These representations are used for final&#10;classification and bounding box regression separately for both models. Our&#10;experiments on the PASCAL-Part 2010 dataset show that joint detection can&#10;simultaneously improve both object detection and part detection in terms of&#10;mean Average Precision (mAP) at IoU=0.5.&#10;" />
-          <attvalue for="2" value="&#10;With the advances in camera and imaging technology, the amount of available online content with images and videos is ever-increasing. This makes it impossible for any human to manually parse and make sense out of images and motivates the development of automated techniques to parse images. One such popular and immensely important task is automated object detection in images. Recently, there have been major developments in this domain (\cite{yolov3}, \cite{ren2015faster}, \cite{he2017mask}) but the diversity of this problem poses new challenges motivating active research in this direction.&#10;&#10;Humans identify objects not only by looking at the object as a whole, but looking at various parts of the object to gain confidence. Taking inspiration from this idea, there have been some recent researches (\cite{felzenszwalb2009object}, \cite{yao2018exploiting}). A concrete example is, identifying a human in an image by identifying parts like hands, faces etc. to add confidence to classification. Another example is identifying an aeroplane by using information of its wings. \cite{yao2018exploiting} propose a LSTM-based architecture for joint modelling of semantic-part information with full object detection. Recently, self-attention based transformer architectures \cite{vaswani2017attention} have gained popularity which eliminated the use of Recurrent Model like RNN for modeling contextual information. We aim to borrow ideas of self-attention in our setting where semantic parts associated with an object will act as different contexts which will be weighted based on the attention score.&#10;&#10;In this project, we propose a novel model for joint detection of an object as a whole and its semantic parts using an attention-based feature fusion of object and its related parts. We use the PASCAL-Part 2010 Dataset (\cite{chen2014detect}) that contains segmentation masks for object and its various semantic parts. From our limited experiments for Animal objects (bird, cat, cow, dog, horse, sheep) and part (face, leg, neck, tail, torso, wings) detection, we find that our joint detection model can give slight improvement for object detection and a reasonable improvement for part detection in terms of mean Average Precision (mAP) at Intersectionover Union (IoU) threshold of 0.5. We believe a thorough hyperparameter tuning can lead to even better results.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science, Pattern Recognition" />
-        </attvalues>
-      </node>
-      <node id="2401.04921" label="2401.04921">
-        <attvalues>
-          <attvalue for="0" value="Diffusion-based Pose Refinement and Muti-hypothesis Generation for 3D&#10;  Human Pose Estimaiton" />
-          <attvalue for="1" value="  Previous probabilistic models for 3D Human Pose Estimation (3DHPE) aimed to&#10;enhance pose accuracy by generating multiple hypotheses. However, most of the&#10;hypotheses generated deviate substantially from the true pose. Compared to&#10;deterministic models, the excessive uncertainty in probabilistic models leads&#10;to weaker performance in single-hypothesis prediction. To address these two&#10;challenges, we propose a diffusion-based refinement framework called DRPose,&#10;which refines the output of deterministic models by reverse diffusion and&#10;achieves more suitable multi-hypothesis prediction for the current pose&#10;benchmark by multi-step refinement with multiple noises. To this end, we&#10;propose a Scalable Graph Convolution Transformer (SGCT) and a Pose Refinement&#10;Module (PRM) for denoising and refining. Extensive experiments on Human3.6M and&#10;MPI-INF-3DHP datasets demonstrate that our method achieves state-of-the-art&#10;performance on both single and multi-hypothesis 3DHPE. Code is available at&#10;https://github.com/KHB1698/DRPose.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In recent years, 3D Human Pose Estimation (3DHPE) has garnered widespread attention due to its significant applications in fields such as action recognition\cite{liu2018recognizing}, virtual reality\cite{mehta2017vnect}, and human-computer interaction\cite{preece1994human}. The primary objective of 3DHPE is to accurately predict the 3D poses of the human body from images or videos. Substantial progress has been achieved in 3D pose estimation from 2D pose inputs, facilitated by powerful 2D pose estimators\cite{newell2016stacked,chen2018cascaded,sun2019deep}.&#10;&#10;A considerable body of work has focused on deterministic models\cite{pavllo20193d,zou2021modulated,zhang2022mixste,zhao2023poseformerv2,cai2023htnet,kang2023double} aimed at inferring the most likely pose directly from given input data. However, these models often grapple with the inherent ambiguity present in the data, leading to suboptimal results, particularly in complex and challenging scenarios. To address this issue, probabilistic models\cite{wehrbein2021probabilistic,oikarinen2021graphmdn,li2022mhformer,ci2023gfpose} have been introduced, aiming to capture the uncertainty in pose estimation by generating multiple pose hypotheses. By encompassing a broader range of potential solutions, this approach holds promise for more accurate pose predictions.&#10;&#10;Although probabilistic models offer advantages in handling uncertainty, excessive uncertainty makes them face two main problems: ($\romannumeral1$) most generated hypotheses often exhibit significant deviations from the ground truth pose, resulting in averaged outcomes considerably inferior to the optimal solution; ($\romannumeral2$) under the context of single-pose hypothesis prediction, probabilistic models demonstrate weaker performance compared to deterministic models, impeding their widespread adoption. We contend that the key to addressing these challenges lies in aligning the average distribution of all hypotheses closer to the true values, enabling multi-hypothesis average outcomes to achieve comparable or even superior performance to deterministic models.&#10;&#10;Consequently, we propose a diffusion-based\cite{ho2020denoising,song2020denoising} refinement framework, called DRPose, aimed at refining 3D poses generated by deterministic models\cite{cai2023htnet,kang2023double} to bring the average predictions of probabilistic models closer to the real distribution. As depicted in Fig. \ref{fig:inference}, we combine initial 3D poses (deterministic) predicted by the deterministic model with pure noise (probabilistic) to capture their underlying 3D features, achieving denoising and refinement effects. By introducing different noises, we obtain predictive results for multiple hypotheses. Importantly, each hypothesis's distribution still adheres to the characteristics of single-hypothesis predictions. This ensures that the multi-hypothesis average distribution obtained through DRPose is closer to the true values, and the iterative nature of the diffusion model further enhances differences between distinct hypotheses.&#10;&#10;The core success of DRPose lies in two key components: a Scalable Graph Convolution Transformer (SGCT) and a Pose Refinement Module (PRM). SGCT is primarily used for denoising and learning the distribution of initial 3D poses and their latent features. Ultimately, through PRM, a balance is struck between certain and uncertain poses, yielding a refined final pose. The effective integration of these two models results in more robust and accurate 3D pose estimation.&#10;&#10;Contributions in this paper can be summarized as follows: &#10;&#10;\begin{itemize}&#10;&#09;\item We propose a DRPose framework for refining 3D poses and achieving more accurate multi-hypothesis extensions.&#10;&#09;\item We design two innovative components: SGCT for denoising and learning the distribution of initial 3D poses and latent features, and PRM for balancing certain and uncertain poses.&#10;&#09;\item Extensive experiments validate the efficacy of DRPose. Our approach achieves state-of-the-art performance in both single-hypothesis and multi-hypothesis 3DHPE scenarios on the Human3.6M and MPI-INF-3DHP datasets.&#10; \end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Probabilistic Modeling, Deep Learning Architectures, Computer Science, Human Pose Estimation" />
-        </attvalues>
-      </node>
-      <node id="2303.17472" label="2303.17472">
-        <attvalues>
-          <attvalue for="0" value="PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D&#10;  Human Pose Estimation" />
-          <attvalue for="1" value="  Recently, transformer-based methods have gained significant success in&#10;sequential 2D-to-3D lifting human pose estimation. As a pioneering work,&#10;PoseFormer captures spatial relations of human joints in each video frame and&#10;human dynamics across frames with cascaded transformer layers and has achieved&#10;impressive performance. However, in real scenarios, the performance of&#10;PoseFormer and its follow-ups is limited by two factors: (a) The length of the&#10;input joint sequence; (b) The quality of 2D joint detection. Existing methods&#10;typically apply self-attention to all frames of the input sequence, causing a&#10;huge computational burden when the frame number is increased to obtain advanced&#10;estimation accuracy, and they are not robust to noise naturally brought by the&#10;limited capability of 2D joint detectors. In this paper, we propose&#10;PoseFormerV2, which exploits a compact representation of lengthy skeleton&#10;sequences in the frequency domain to efficiently scale up the receptive field&#10;and boost robustness to noisy 2D joint detection. With minimum modifications to&#10;PoseFormer, the proposed method effectively fuses features both in the time&#10;domain and frequency domain, enjoying a better speed-accuracy trade-off than&#10;its precursor. Extensive experiments on two benchmark datasets (i.e., Human3.6M&#10;and MPI-INF-3DHP) demonstrate that the proposed approach significantly&#10;outperforms the original PoseFormer and other transformer-based variants. Code&#10;is released at \url{https://github.com/QitaoZhao/PoseFormerV2}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.09790" label="2302.09790">
-        <attvalues>
-          <attvalue for="0" value="HTNet: Human Topology Aware Network for 3D Human Pose Estimation" />
-          <attvalue for="1" value="  3D human pose estimation errors would propagate along the human body topology&#10;and accumulate at the end joints of limbs. Inspired by the backtracking&#10;mechanism in automatic control systems, we design an Intra-Part Constraint&#10;module that utilizes the parent nodes as the reference to build topological&#10;constraints for end joints at the part level. Further considering the hierarchy&#10;of the human topology, joint-level and body-level dependencies are captured via&#10;graph convolutional networks and self-attentions, respectively. Based on these&#10;designs, we propose a novel Human Topology aware Network (HTNet), which adopts&#10;a channel-split progressive strategy to sequentially learn the structural&#10;priors of the human topology from multiple semantic levels: joint, part, and&#10;body. Extensive experiments show that the proposed method improves the&#10;estimation accuracy by 18.7% on the end joints of limbs and achieves&#10;state-of-the-art results on Human3.6M and MPI-INF-3DHP datasets. Code is&#10;available at https://github.com/vefalun/HTNet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.05298" label="2308.05298">
-        <attvalues>
-          <attvalue for="0" value="Double-chain Constraints for 3D Human Pose Estimation in Images and&#10;  Videos" />
-          <attvalue for="1" value="  Reconstructing 3D poses from 2D poses lacking depth information is&#10;particularly challenging due to the complexity and diversity of human motion.&#10;The key is to effectively model the spatial constraints between joints to&#10;leverage their inherent dependencies. Thus, we propose a novel model, called&#10;Double-chain Graph Convolutional Transformer (DC-GCT), to constrain the pose&#10;through a double-chain design consisting of local-to-global and global-to-local&#10;chains to obtain a complex representation more suitable for the current human&#10;pose. Specifically, we combine the advantages of GCN and Transformer and design&#10;a Local Constraint Module (LCM) based on GCN and a Global Constraint Module&#10;(GCM) based on self-attention mechanism as well as a Feature Interaction Module&#10;(FIM). The proposed method fully captures the multi-level dependencies between&#10;human body joints to optimize the modeling capability of the model. Moreover,&#10;we propose a method to use temporal information into the single-frame model by&#10;guiding the video sequence embedding through the joint embedding of the target&#10;frame, with negligible increase in computational cost. Experimental results&#10;demonstrate that DC-GCT achieves state-of-the-art performance on two&#10;challenging datasets (Human3.6M and MPI-INF-3DHP). Notably, our model achieves&#10;state-of-the-art performance on all action categories in the Human3.6M dataset&#10;using detected 2D poses from CPN, and our code is available at:&#10;https://github.com/KHB1698/DC-GCT.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.13788" label="2107.13788">
-        <attvalues>
-          <attvalue for="0" value="Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows" />
-          <attvalue for="1" value="  3D human pose estimation from monocular images is a highly ill-posed problem&#10;due to depth ambiguities and occlusions. Nonetheless, most existing works&#10;ignore these ambiguities and only estimate a single solution. In contrast, we&#10;generate a diverse set of hypotheses that represents the full posterior&#10;distribution of feasible 3D poses. To this end, we propose a normalizing flow&#10;based method that exploits the deterministic 3D-to-2D mapping to solve the&#10;ambiguous inverse 2D-to-3D problem. Additionally, uncertain detections and&#10;occlusions are effectively modeled by incorporating uncertainty information of&#10;the 2D detector as condition. Further keys to success are a learned 3D pose&#10;prior and a generalization of the best-of-M loss. We evaluate our approach on&#10;the two benchmark datasets Human3.6M and MPI-INF-3DHP, outperforming all&#10;comparable methods in most metrics. The implementation is available on GitHub.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.13668" label="2010.13668">
-        <attvalues>
-          <attvalue for="0" value="GraphMDN: Leveraging graph structure and deep learning to solve inverse&#10;  problems" />
-          <attvalue for="1" value="  The recent introduction of Graph Neural Networks (GNNs) and their growing&#10;popularity in the past few years has enabled the application of deep learning&#10;algorithms to non-Euclidean, graph-structured data. GNNs have achieved&#10;state-of-the-art results across an impressive array of graph-based machine&#10;learning problems. Nevertheless, despite their rapid pace of development, much&#10;of the work on GNNs has focused on graph classification and embedding&#10;techniques, largely ignoring regression tasks over graph data. In this paper,&#10;we develop a Graph Mixture Density Network (GraphMDN), which combines graph&#10;neural networks with mixture density network (MDN) outputs. By combining these&#10;techniques, GraphMDNs have the advantage of naturally being able to incorporate&#10;graph structured information into a neural architecture, as well as the ability&#10;to model multi-modal regression targets. As such, GraphMDNs are designed to&#10;excel on regression tasks wherein the data are graph structured, and target&#10;statistics are better represented by mixtures of densities rather than singular&#10;values (so-called ``inverse problems&quot;). To demonstrate this, we extend an&#10;existing GNN architecture known as Semantic GCN (SemGCN) to a GraphMDN&#10;structure, and show results from the Human3.6M pose estimation task. The&#10;extended model consistently outperforms both GCN and MDN architectures on their&#10;own, with a comparable number of parameters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.12707" label="2111.12707">
-        <attvalues>
-          <attvalue for="0" value="MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation" />
-          <attvalue for="1" value="  Estimating 3D human poses from monocular videos is a challenging task due to&#10;depth ambiguity and self-occlusion. Most existing works attempt to solve both&#10;issues by exploiting spatial and temporal relationships. However, those works&#10;ignore the fact that it is an inverse problem where multiple feasible solutions&#10;(i.e., hypotheses) exist. To relieve this limitation, we propose a&#10;Multi-Hypothesis Transformer (MHFormer) that learns spatio-temporal&#10;representations of multiple plausible pose hypotheses. In order to effectively&#10;model multi-hypothesis dependencies and build strong relationships across&#10;hypothesis features, the task is decomposed into three stages: (i) Generate&#10;multiple initial hypothesis representations; (ii) Model self-hypothesis&#10;communication, merge multiple hypotheses into a single converged representation&#10;and then partition it into several diverged hypotheses; (iii) Learn&#10;cross-hypothesis communication and aggregate the multi-hypothesis features to&#10;synthesize the final 3D pose. Through the above processes, the final&#10;representation is enhanced and the synthesized pose is much more accurate.&#10;Extensive experiments show that MHFormer achieves state-of-the-art results on&#10;two challenging datasets: Human3.6M and MPI-INF-3DHP. Without bells and&#10;whistles, its performance surpasses the previous best result by a large margin&#10;of 3% on Human3.6M. Code and models are available at&#10;\url{https://github.com/Vegetebird/MHFormer}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.12448" label="2403.12448">
-        <attvalues>
-          <attvalue for="0" value="Do Generated Data Always Help Contrastive Learning?" />
-          <attvalue for="1" value="  Contrastive Learning (CL) has emerged as one of the most successful paradigms&#10;for unsupervised visual representation learning, yet it often depends on&#10;intensive manual data augmentations. With the rise of generative models,&#10;especially diffusion models, the ability to generate realistic images close to&#10;the real data distribution has been well recognized. These generated&#10;high-equality images have been successfully applied to enhance contrastive&#10;representation learning, a technique termed ``data inflation''. However, we&#10;find that the generated data (even from a good diffusion model like DDPM) may&#10;sometimes even harm contrastive learning. We investigate the causes behind this&#10;failure from the perspective of both data inflation and data augmentation. For&#10;the first time, we reveal the complementary roles that stronger data inflation&#10;should be accompanied by weaker augmentations, and vice versa. We also provide&#10;rigorous theoretical explanations for these phenomena via deriving its&#10;generalization bounds under data inflation. Drawing from these insights, we&#10;propose Adaptive Inflation (AdaInf), a purely data-centric strategy without&#10;introducing any extra computation cost. On benchmark datasets, AdaInf can bring&#10;significant improvements for various contrastive learning methods. Notably,&#10;without using external data, AdaInf obtains 94.70% linear accuracy on CIFAR-10&#10;with SimCLR, setting a new record that surpasses many sophisticated methods.&#10;Code is available at https://github.com/PKU-ML/adainf.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}. However, there still remains a noticeable gap in performance compared to its supervised counterparts \cite{mocov3,cui2023rethinking}. Among many attempts to close this gap, a recent surge of interest lies in leveraging high-quality generative models to boost contrastive learning \cite{wu2022synthetic,wang2022unified,azizi2023synthetic,tian2023stablerep}. Given an unlabeled dataset, \eg CIFAR-10, one can train a generative model on it (\eg DDPM \cite{ho2020denoising}) to generate a lot of synthetic samples, and then perform contrastive learning on the combination of the real and generated data. This simplest way for using generated data is called ``data inflation''. Notably, it is orthogonal to the data augmentation process, wherein an image--be it raw or generated--is subjected to manual augmentations (\eg cropping) to yield positive and negative pairs used in contrastive learning (see Figure \ref{fig:flow} for an illustration of the pipeline).&#10;&#10;Though one may expect that generated data will benefit contrastive learning with more diverse data, we find that it is not always the case. As shown in Figure \ref{fig:counterexample}, simply inflating CIFAR-10 with 1M images generated by DDPM (the vanilla setting) leads to even worse linear probing accuracy (91.33\% \vvs 90.27\%). We investigate this unexpected performance degradation from two aspects: data inflation (how inflated data are constructed) and data augmentation (how to craft augmented samples with inflated data). For the former, we find that better generation quality is of limited help, while reweighting real and generated data can attain larger gains. For the latter, we discover an intriguing phenomenon that weaker data augmentation, although harmful in standard contrastive learning \cite{nips/Tian0PKSI20,luo2023rethinking}, can be very helpful with data inflation. To uncover the mysteries behind these observations, we establish the first generalization guarantees for inflated contrastive learning, and explain the benefits of weak augmentations by revealing the complementary roles between data inflation and data augmentation. Based on these insights, we propose an Adaptive Inflation (AdaInf) strategy that adaptively adjusts data augmentation strength and mixing ratio for data inflation, which can bring significant improvements in downstream performance without any introducing computation overhead. We summarize our contributions as follows:&#10;\begin{itemize}&#10;\item We discover a failure mode of data inflation for contrastive learning, and reveal the causes of this failure from the perspective of both data inflation and data augmentation. In particular, we find that data reweighting and weak augmentation contribute significantly to improving final performance.&#10;\item To understand these phenomena, we establish the first theoretical guarantees for inflated contrastive learning, which not only rigorously explain previous phenomena, but also reveal the complementary roles between data inflation and data augmentation. &#10;\item We propose the Adaptive Inflation (AdaInf) strategy to adaptively adjust data augmentation strength and mixing ratio for data inflation. Extensive experiments show that the proposed approach improves downstream accuracy significantly at no extra cost, and it is particularly beneficial for data-scarce scenarios.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Visual Representation, Mathematics, Artificial Intelligence, Contrastive Learning, Generative Models, Computer Science, Statistics" />
-        </attvalues>
-      </node>
-      <node id="2110.15348" label="2110.15348">
-        <attvalues>
-          <attvalue for="0" value="Residual Relaxation for Multi-view Representation Learning" />
-          <attvalue for="1" value="  Multi-view methods learn representations by aligning multiple views of the&#10;same image and their performance largely depends on the choice of data&#10;augmentation. In this paper, we notice that some other useful augmentations,&#10;such as image rotation, are harmful for multi-view methods because they cause a&#10;semantic shift that is too large to be aligned well. This observation motivates&#10;us to relax the exact alignment objective to better cultivate stronger&#10;augmentations. Taking image rotation as a case study, we develop a generic&#10;approach, Pretext-aware Residual Relaxation (Prelax), that relaxes the exact&#10;alignment by allowing an adaptive residual vector between different views and&#10;encoding the semantic shift through pretext-aware learning. Extensive&#10;experiments on different backbones show that our method can not only improve&#10;multi-view methods with existing augmentations, but also benefit from stronger&#10;image augmentations like rotation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.06562" label="2303.06562">
-        <attvalues>
-          <attvalue for="0" value="ContraNorm: A Contrastive Learning Perspective on Oversmoothing and&#10;  Beyond" />
-          <attvalue for="1" value="  Oversmoothing is a common phenomenon in a wide range of Graph Neural Networks&#10;(GNNs) and Transformers, where performance worsens as the number of layers&#10;increases. Instead of characterizing oversmoothing from the view of complete&#10;collapse in which representations converge to a single point, we dive into a&#10;more general perspective of dimensional collapse in which representations lie&#10;in a narrow cone. Accordingly, inspired by the effectiveness of contrastive&#10;learning in preventing dimensional collapse, we propose a novel normalization&#10;layer called ContraNorm. Intuitively, ContraNorm implicitly shatters&#10;representations in the embedding space, leading to a more uniform distribution&#10;and a slighter dimensional collapse. On the theoretical analysis, we prove that&#10;ContraNorm can alleviate both complete collapse and dimensional collapse under&#10;certain conditions. Our proposed normalization layer can be easily integrated&#10;into GNNs and Transformers with negligible parameter overhead. Experiments on&#10;various real-world datasets demonstrate the effectiveness of our proposed&#10;ContraNorm. Our implementation is available at&#10;https://github.com/PKU-ML/ContraNorm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.04272" label="2306.04272">
-        <attvalues>
-          <attvalue for="0" value="On the Generalization of Multi-modal Contrastive Learning" />
-          <attvalue for="1" value="  Multi-modal contrastive learning (MMCL) has recently garnered considerable&#10;interest due to its superior performance in visual tasks, achieved by embedding&#10;multi-modal data, such as visual-language pairs. However, there still lack&#10;theoretical understandings of how MMCL extracts useful visual representation&#10;from multi-modal pairs, and particularly, how MMCL outperforms previous&#10;approaches like self-supervised contrastive learning (SSCL). In this paper, by&#10;drawing an intrinsic connection between MMCL and asymmetric matrix&#10;factorization, we establish the first generalization guarantees of MMCL for&#10;visual downstream tasks. Based on this framework, we further unify MMCL and&#10;SSCL by showing that MMCL implicitly performs SSCL with (pseudo) positive pairs&#10;induced by text pairs. Through this unified perspective, we characterize the&#10;advantage of MMCL by showing that text pairs induce more semantically&#10;consistent and diverse positive pairs, which, according to our analysis,&#10;provably benefit downstream generalization. Inspired by this finding, we&#10;propose CLIP-guided resampling methods to significantly improve the downstream&#10;performance of SSCL on ImageNet by leveraging multi-modal information. Code is&#10;available at https://github.com/PKU-ML/CLIP-Help-SimCLR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.18904" label="2310.18904">
-        <attvalues>
-          <attvalue for="0" value="Identifiable Contrastive Learning with Automatic Feature Importance&#10;  Discovery" />
-          <attvalue for="1" value="  Existing contrastive learning methods rely on pairwise sample contrast&#10;$z_x^\top z_{x'}$ to learn data representations, but the learned features often&#10;lack clear interpretability from a human perspective. Theoretically, it lacks&#10;feature identifiability and different initialization may lead to totally&#10;different features. In this paper, we study a new method named tri-factor&#10;contrastive learning (triCL) that involves a 3-factor contrast in the form of&#10;$z_x^\top S z_{x'}$, where $S=\text{diag}(s_1,\dots,s_k)$ is a learnable&#10;diagonal matrix that automatically captures the importance of each feature. We&#10;show that by this simple extension, triCL can not only obtain identifiable&#10;features that eliminate randomness but also obtain more interpretable features&#10;that are ordered according to the importance matrix $S$. We show that features&#10;with high importance have nice interpretability by capturing common classwise&#10;features, and obtain superior performance when evaluated for image retrieval&#10;using a few features. The proposed triCL objective is general and can be&#10;applied to different contrastive learning methods like SimCLR and CLIP. We&#10;believe that it is a better alternative to existing 2-factor contrastive&#10;learning by improving its identifiability and interpretability with minimal&#10;overhead. Code is available at&#10;https://github.com/PKU-ML/Tri-factor-Contrastive-Learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.02057" label="2104.02057">
-        <attvalues>
-          <attvalue for="0" value="An Empirical Study of Training Self-Supervised Vision Transformers" />
-          <attvalue for="1" value="  This paper does not describe a novel method. Instead, it studies a&#10;straightforward, incremental, yet must-know baseline given the recent progress&#10;in computer vision: self-supervised learning for Vision Transformers (ViT).&#10;While the training recipes for standard convolutional networks have been highly&#10;mature and robust, the recipes for ViT are yet to be built, especially in the&#10;self-supervised scenarios where training becomes more challenging. In this&#10;work, we go back to basics and investigate the effects of several fundamental&#10;components for training self-supervised ViT. We observe that instability is a&#10;major issue that degrades accuracy, and it can be hidden by apparently good&#10;results. We reveal that these results are indeed partial failure, and they can&#10;be improved when training is made more stable. We benchmark ViT results in MoCo&#10;v3 and several other self-supervised frameworks, with ablations in various&#10;aspects. We discuss the currently positive evidence as well as challenges and&#10;open questions. We hope that this work will provide useful data points and&#10;experience for future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.04160" label="2306.04160">
-        <attvalues>
-          <attvalue for="0" value="Rethinking Weak Supervision in Helping Contrastive Learning" />
-          <attvalue for="1" value="  Contrastive learning has shown outstanding performances in both supervised&#10;and unsupervised learning, and has recently been introduced to solve weakly&#10;supervised learning problems such as semi-supervised learning and noisy label&#10;learning. Despite the empirical evidence showing that semi-supervised labels&#10;improve the representations of contrastive learning, it remains unknown if&#10;noisy supervised information can be directly used in training instead of after&#10;manual denoising. Therefore, to explore the mechanical differences between&#10;semi-supervised and noisy-labeled information in helping contrastive learning,&#10;we establish a unified theoretical framework of contrastive learning under weak&#10;supervision. Specifically, we investigate the most intuitive paradigm of&#10;jointly training supervised and unsupervised contrastive losses. By translating&#10;the weakly supervised information into a similarity graph under the framework&#10;of spectral clustering based on the posterior probability of weak labels, we&#10;establish the downstream classification error bound. We prove that&#10;semi-supervised labels improve the downstream error bound whereas noisy labels&#10;have limited effects under such a paradigm. Our theoretical findings here&#10;provide new insights for the community to rethink the role of weak supervision&#10;in helping contrastive learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.06464" label="2202.06464">
-        <attvalues>
-          <attvalue for="0" value="Synthetic Data Can Also Teach: Synthesizing Effective Data for&#10;  Unsupervised Visual Representation Learning" />
-          <attvalue for="1" value="  Contrastive learning (CL), a self-supervised learning approach, can&#10;effectively learn visual representations from unlabeled data. Given the CL&#10;training data, generative models can be trained to generate synthetic data to&#10;supplement the real data. Using both synthetic and real data for CL training&#10;has the potential to improve the quality of learned representations. However,&#10;synthetic data usually has lower quality than real data, and using synthetic&#10;data may not improve CL compared with using real data. To tackle this problem,&#10;we propose a data generation framework with two methods to improve CL training&#10;by joint sample generation and contrastive learning. The first approach&#10;generates hard samples for the main model. The generator is jointly learned&#10;with the main model to dynamically customize hard samples based on the training&#10;state of the main model. Besides, a pair of data generators are proposed to&#10;generate similar but distinct samples as positive pairs. In joint learning, the&#10;hardness of a positive pair is progressively increased by decreasing their&#10;similarity. Experimental results on multiple datasets show superior accuracy&#10;and data efficiency of the proposed data generation methods applied to CL. For&#10;example, about 4.0%, 3.5%, and 2.6% accuracy improvements for linear&#10;classification are observed on ImageNet-100, CIFAR-100, and CIFAR-10,&#10;respectively. Besides, up to 2x data efficiency for linear classification and&#10;up to 5x data efficiency for transfer learning are achieved.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.13455" label="2203.13455">
-        <attvalues>
-          <attvalue for="0" value="A Unified Contrastive Energy-based Model for Understanding the&#10;  Generative Ability of Adversarial Training" />
-          <attvalue for="1" value="  Adversarial Training (AT) is known as an effective approach to enhance the&#10;robustness of deep neural networks. Recently researchers notice that robust&#10;models with AT have good generative ability and can synthesize realistic&#10;images, while the reason behind it is yet under-explored. In this paper, we&#10;demystify this phenomenon by developing a unified probabilistic framework,&#10;called Contrastive Energy-based Models (CEM). On the one hand, we provide the&#10;first probabilistic characterization of AT through a unified understanding of&#10;robustness and generative ability. On the other hand, our unified framework can&#10;be extended to the unsupervised scenario, which interprets unsupervised&#10;contrastive learning as an important sampling of CEM. Based on these, we&#10;propose a principled method to develop adversarial learning and sampling&#10;methods. Experiments show that the sampling methods derived from our framework&#10;improve the sample quality in both supervised and unsupervised learning.&#10;Notably, our unsupervised adversarial sampling method achieves an Inception&#10;score of 9.61 on CIFAR-10, which is superior to previous energy-based models&#10;and comparable to state-of-the-art generative models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.08466" label="2304.08466">
-        <attvalues>
-          <attvalue for="0" value="Synthetic Data from Diffusion Models Improves ImageNet Classification" />
-          <attvalue for="1" value="  Deep generative models are becoming increasingly powerful, now generating&#10;diverse high fidelity photo-realistic samples given text prompts. Have they&#10;reached the point where models of natural images can be used for generative&#10;data augmentation, helping to improve challenging discriminative tasks? We show&#10;that large-scale text-to image diffusion models can be fine-tuned to produce&#10;class conditional models with SOTA FID (1.76 at 256x256 resolution) and&#10;Inception Score (239 at 256x256). The model also yields a new SOTA in&#10;Classification Accuracy Scores (64.96 for 256x256 generative samples, improving&#10;to 69.24 for 1024x1024 samples). Augmenting the ImageNet training set with&#10;samples from the resulting models yields significant improvements in ImageNet&#10;classification accuracy over strong ResNet and Vision Transformer baselines.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.00984" label="2306.00984">
-        <attvalues>
-          <attvalue for="0" value="StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual&#10;  Representation Learners" />
-          <attvalue for="1" value="  We investigate the potential of learning visual representations using&#10;synthetic images generated by text-to-image models. This is a natural question&#10;in the light of the excellent performance of such models in generating&#10;high-quality images. We consider specifically the Stable Diffusion, one of the&#10;leading open source text-to-image models. We show that (1) when the generative&#10;model is configured with proper classifier-free guidance scale, training&#10;self-supervised methods on synthetic images can match or beat the real image&#10;counterpart; (2) by treating the multiple images generated from the same text&#10;prompt as positives for each other, we develop a multi-positive contrastive&#10;learning method, which we call StableRep. With solely synthetic images, the&#10;representations learned by StableRep surpass the performance of representations&#10;learned by SimCLR and CLIP using the same set of text prompts and corresponding&#10;real images, on large scale datasets. When we further add language supervision,&#10;StableRep trained with 20M synthetic images achieves better accuracy than CLIP&#10;trained with 50M real images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.10243" label="2005.10243">
-        <attvalues>
-          <attvalue for="0" value="What Makes for Good Views for Contrastive Learning?" />
-          <attvalue for="1" value="  Contrastive learning between multiple views of the data has recently achieved&#10;state of the art performance in the field of self-supervised representation&#10;learning. Despite its success, the influence of different view choices has been&#10;less studied. In this paper, we use theoretical and empirical analysis to&#10;better understand the importance of view selection, and argue that we should&#10;reduce the mutual information (MI) between views while keeping task-relevant&#10;information intact. To verify this hypothesis, we devise unsupervised and&#10;semi-supervised frameworks that learn effective views by aiming to reduce their&#10;MI. We also consider data augmentation as a way to reduce MI, and show that&#10;increasing data augmentation indeed leads to decreasing MI and improves&#10;downstream classification accuracy. As a by-product, we achieve a new&#10;state-of-the-art accuracy on unsupervised pre-training for ImageNet&#10;classification ($73\%$ top-1 linear readout with a ResNet-50). In addition,&#10;transferring our models to PASCAL VOC object detection and COCO instance&#10;segmentation consistently outperforms supervised pre-training.&#10;Code:http://github.com/HobbitLong/PyContrast&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.01289" label="2303.01289">
-        <attvalues>
-          <attvalue for="0" value="Rethinking the Effect of Data Augmentation in Adversarial Contrastive&#10;  Learning" />
-          <attvalue for="1" value="  Recent works have shown that self-supervised learning can achieve remarkable&#10;robustness when integrated with adversarial training (AT). However, the&#10;robustness gap between supervised AT (sup-AT) and self-supervised AT (self-AT)&#10;remains significant. Motivated by this observation, we revisit existing self-AT&#10;methods and discover an inherent dilemma that affects self-AT robustness:&#10;either strong or weak data augmentations are harmful to self-AT, and a medium&#10;strength is insufficient to bridge the gap. To resolve this dilemma, we propose&#10;a simple remedy named DYNACL (Dynamic Adversarial Contrastive Learning). In&#10;particular, we propose an augmentation schedule that gradually anneals from a&#10;strong augmentation to a weak one to benefit from both extreme cases. Besides,&#10;we adopt a fast post-processing stage for adapting it to downstream tasks.&#10;Through extensive experiments, we show that DYNACL can improve state-of-the-art&#10;self-AT robustness by 8.84% under Auto-Attack on the CIFAR-10 dataset, and can&#10;even outperform vanilla supervised adversarial training for the first time. Our&#10;code is available at \url{https://github.com/PKU-ML/DYNACL}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.10766" label="2201.10766">
-        <attvalues>
-          <attvalue for="0" value="A Comprehensive Study of Image Classification Model Sensitivity to&#10;  Foregrounds, Backgrounds, and Visual Attributes" />
-          <attvalue for="1" value="  While datasets with single-label supervision have propelled rapid advances in&#10;image classification, additional annotations are necessary in order to&#10;quantitatively assess how models make predictions. To this end, for a subset of&#10;ImageNet samples, we collect segmentation masks for the entire object and $18$&#10;informative attributes. We call this dataset RIVAL10 (RIch Visual Attributes&#10;with Localization), consisting of roughly $26k$ instances over $10$ classes.&#10;Using RIVAL10, we evaluate the sensitivity of a broad set of models to noise&#10;corruptions in foregrounds, backgrounds and attributes. In our analysis, we&#10;consider diverse state-of-the-art architectures (ResNets, Transformers) and&#10;training procedures (CLIP, SimCLR, DeiT, Adversarial Training). We find that,&#10;somewhat surprisingly, in ResNets, adversarial training makes models more&#10;sensitive to the background compared to foreground than standard training.&#10;Similarly, contrastively-trained models also have lower relative foreground&#10;sensitivity in both transformers and ResNets. Lastly, we observe intriguing&#10;adaptive abilities of transformers to increase relative foreground sensitivity&#10;as corruption level increases. Using saliency methods, we automatically&#10;discover spurious features that drive the background sensitivity of models and&#10;assess alignment of saliency maps with foregrounds. Finally, we quantitatively&#10;study the attribution problem for neural features by comparing feature saliency&#10;with ground-truth localization of semantic attributes.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Large scale benchmark datasets like ImageNet\cite{deng2009imagenet} that were constructed with single class label annotation have propelled rapid advances in the image classification task\cite{resnets, huang2017densely, tan2019efficientnet, zhai2021scaling}. Over the last decade, several network architectures and training procedures were proposed to yield very high classification accuracies \cite{simonyan2014very, resnets, tan2019efficientnet, vits}. However, methods to interpret these model predictions and to diagnose undesirable behaviors are fairly limited. One of the most popular class of approaches are saliency methods\cite{gradcam,cam,smoothgrad, integrated-grads} that use model gradients to produce a saliency map corresponding to the most influential input regions that yielded the resulting prediction. However, these methods are qualitative, require human supervision, and can be noisy, thus making their judgements potentially unreliable when made in isolation of other supporting analysis.&#10;&#10;In this paper, we argue that to obtain a proper understanding of how specific input regions impact the prediction, we need additional ground truth annotations beyond a single class label. To this end, we introduce a novel dataset, RIVAL10, whose samples include {RI}ch {V}isual {A}ttributions with {L}ocalization. RIVAL10 consists of images from $20$ categories of ImageNet-1k \cite{deng2009imagenet}, with a total of $26k$ high resolution images organized into $10$ classes, matching those of CIFAR10 \cite{cifar10}. The main contribution of our dataset is instance wise labels for $18$ informative visual attributes, as well as segmentation masks for each attribute and the entire object. We present our dataset as a general resource for understanding models trained on ImageNet. We then provide a study of the sensitivity of a diverse set of models to foregrounds, backgrounds, and attributes. &#10;&#10;Our study of background and foreground model sensitivity is motivated by some counter-intuitive model behaviors on images whose background and foreground regions were corrupted with Gaussian noise: Figure \ref{fig:examples} shows instances where highly accurate models have performance degraded much more due to the background noise than the foreground noise. While this is not the norm (i.e. models are more sensitive to foregrounds on average), the existence of these examples warrants greater investigation, as they expose a stark difference in how deep models and humans perform object recognition. Quantifying the degree to which different architectures and training procedures admit these examples can shed new insight on how models incorporate foreground and background information. &#10;&#10;To this end, we conduct a noise analysis that leverages object segmentation masks to quantitatively assess model sensitivity to foregrounds relative to backgrounds. We proxy sensitivity to a region by observing model performance under corruption of that region. We propose a normalized metric, {relative foreground &#10;sensitivity} ($RFS$), to compare models with various general noise robustness. A high $RFS$ value indicates that the model uses foreground features in its inferences more than background ones since corrupting them result in higher performance degradation.&#10;&#10;In Figure \ref{fig:scatter_fg_bg}, we see different architectures and training procedures lead to variations in both general noise robustness (projection onto the main diagonal) and relative foreground sensitivity (normalized distance orthogonal to the diagonal). Notably, we find that adversarially training ResNets significantly reduces $RFS$, surprisingly suggesting that robust ResNet models make greater use of background information. We also observe contrastive training to reduce $RFS$, and transformers to uniquely be able to adjust $RFS$ across noise levels, reducing their sensitivity to backgrounds as corruption level increases. Lastly, we find object classes strongly affect $RFS$ across models. &#10;&#10;We couple our noise analysis with saliency methods to add a second perspective of model sensitivity to different input regions. Using RIVAL10 segmentations, we can quantitatively assess the alignment of saliency maps to foregrounds. We also show how we can discover spurious background features by sorting images based on the saliency alignment scores. We observe that performance trends that our noise analysis reveals are not captured using qualitative saliency methods alone, suggesting our noise analysis can provide new insights on model sensitivity to foregrounds and backgrounds.&#10;&#10;Lastly, we utilize RIVAL10 attribute segmentations to systematically investigate the generalizability of neural feature attribution: for a neural feature (i.e., a neuron in the penultimate layer of the network) that achieves the highest intersection-over-union (IOU) score with a specific attribute mask on top-$k$ images within a class, how the IOU scores of that neural feature behave on other samples in that class. For some class-attribute pairs (e.g. dog, floppy-ears), we indeed observe generalizability of neural feature attributions, in the sense that test set IOUs are also high.&#10;&#10;In summary, we present a novel dataset with rich annotations of object and attribute segmentation masks that can be used for a myriad of applications including model interpretability. We then present a study involving three quantitative methods to analyze the sensitivity of models to different regions in inputs. We hope that our richly annotated RIVAL10 dataset helps studying failure modes of current deep classifiers and paves the way for building more reliable models in the future. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Model Sensitivity Analysis, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning Architectures, Cognitive Science, Computer Science, Image Classification" />
-        </attvalues>
-      </node>
-      <node id="1706.03825" label="1706.03825">
-        <attvalues>
-          <attvalue for="0" value="SmoothGrad: removing noise by adding noise" />
-          <attvalue for="1" value="  Explaining the output of a deep network remains a challenge. In the case of&#10;an image classifier, one type of explanation is to identify pixels that&#10;strongly influence the final decision. A starting point for this strategy is&#10;the gradient of the class score function with respect to the input image. This&#10;gradient can be interpreted as a sensitivity map, and there are several&#10;techniques that elaborate on this basic idea. This paper makes two&#10;contributions: it introduces SmoothGrad, a simple method that can help visually&#10;sharpen gradient-based sensitivity maps, and it discusses lessons in the&#10;visualization of these maps. We publish the code for our experiments and a&#10;website with our results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.06841" label="2304.06841">
-        <attvalues>
-          <attvalue for="0" value="Video alignment using unsupervised learning of local and global features" />
-          <attvalue for="1" value="  In this paper, we tackle the problem of video alignment, the process of&#10;matching the frames of a pair of videos containing similar actions. The main&#10;challenge in video alignment is that accurate correspondence should be&#10;established despite the differences in the execution processes and appearances&#10;between the two videos. We introduce an unsupervised method for alignment that&#10;uses global and local features of the frames. In particular, we introduce&#10;effective features for each video frame by means of three machine vision tools:&#10;person detection, pose estimation, and VGG network. Then the features are&#10;processed and combined to construct a multidimensional time series that&#10;represent the video. The resulting time series are used to align videos of the&#10;same actions using a novel version of dynamic time warping named Diagonalized&#10;Dynamic Time Warping(DDTW). The main advantage of our approach is that no&#10;training is required, which makes it applicable for any new type of action&#10;without any need to collect training samples for it. Additionally, our approach&#10;can be used for framewise labeling of action phases in a dataset with only a&#10;few labeled videos. For evaluation, we considered video synchronization and&#10;phase classification tasks on the Penn action and subset of UCF101 datasets.&#10;Also, for an effective evaluation of the video synchronization task, we present&#10;a new metric called Enclosed Area Error(EAE). The results show that our method&#10;outperforms previous state-of-the-art methods, such as TCC, and other&#10;self-supervised and weakly supervised methods.&#10;" />
-          <attvalue for="2" value="&#10;Many sequential processes happen daily in the world. Waking up, drinking water, and growing a plant are examples of sequential processes that are always happening. Although these processes are performed with different varieties and qualities, all the processes that show a specific action have common time points. For example, drinking a glass of water may happen at different speeds, places, and containers, but all the processes that indicate drinking water consist of 3 main steps: lifting the glass, drinking water, and lowering the glass. As a result, each process, or in other words, each action, consists of one or more phases, which are the same in terms of the order of occurrence in all similar processes.&#10;Video alignment is a method in which the frames of the videos of two identical actions that differ in things such as scene, camera angle, and speed are matched to each other.&#10;&#10;The main challenge in the task of video alignment is the difference in the execution process and the appearance of the frames in videos containing the same action. For example, an action such as picking up an object from the ground can be done in various ways while recorded by the camera. This action may be done once in $5$ seconds and another time in $10$ seconds. The object that is risen from the ground may be a ball or a cup, big or small, red or blue. Also, the camera may record this action from the front, side, or any other angle. All these differences cause video alignment and choosing the correct method for video modeling to face many challenges.&#10;&#10;In recent years, much research has been done in action recognition, anomaly detection, tracking, etc. However, video alignment has received less attention, while video alignment can be used to improve all of the above. For example, in \cite{normalized_pose}, novel normalized pose features invariant to video subjects’ anthropometric characteristics are introduced. The method was evaluated in the task of action recognition, and significant results were achieved. In \cite{few-shot-action-recognition}, the action recognition problem is considered as two separate problems, action duration misalignment, and action evolution misalignment. Based on this assumption, a two-stage action alignment network is presented in their work. Video Classification \cite{few-shot-classification, few-shot-classiciation2} and action detection \cite{action-detection, action-detection2, action-detection3} are other examples of the applications of video alignment that have received attention in recent years.&#10;&#10;In the field of video alignment, self-supervised \cite{TCC, aligning_in_space_and_time, learning-by-aligning-videos-in-time, learning-to-align-sequential-actions, ctw} and weakly-supervised methods \cite{D3TW, representation-learning-via-global-temporal-alignment, weakly3, weakly4} have been presented in recent years. In some works, video alignment has been tried to solve using Dynamic Time Warping(DTW) \cite{few-shot-classiciation2, D3TW, ctw}. Since DTW is not derivable and cannot be implemented using neural networks, in these works, some modified types of DTW, such as soft-dtw, which are derivable, have been used \cite{few-shot-classiciation2, D3TW }. Another category of video alignment methods is based on cycle consistency loss \cite{TCC, aligning_in_space_and_time, representation-learning-via-global-temporal-alignment, cycle-consistency}. In \cite{TCC}, they presented a self-supervised method for learning correspondences between frames in the time domain. In this article, the network is trained based on the cycle-consistency cost function, and then the trained network is used to match the frames of a pair of videos with each other. Unlike \cite{TCC}, \cite{aligning_in_space_and_time} deals with correspondence learning in both time and space domains based on cross-cycle stability. In \cite{learning-by-aligning-videos-in-time, representation-learning-via-global-temporal-alignment}, the network is trained based on frame level and video level simultaneously. In \cite{learning-by-aligning-videos-in-time}, the network is trained based on a cost function including two terms, soft-dtw and temporal regularization. In \cite{representation-learning-via-global-temporal-alignment}, a weakly supervised method is presented based on a cost function consisting of dtw and cycle-consistency. In \cite{learning-to-align-sequential-actions}, another look at the subject of video alignment is given. In this work, video representation is learned to align two videos while the possibility of background frames, redundant frames, and non-monotonic frames are considered.&#10;&#10;One of the shortcomings of the existing methods is the need to train deep networks for each class of action, which requires a lot of training samples from each action. In this work, we present an unsupervised method that can be used to align pairs of videos containing any action without any need to train a network.&#10;&#10;The main contribution of this article is, using an unsupervised approach for representing a video as a multidimensional time series representing features of its frames over time. To construct these features, we simultaneously employ box and pose features to extract local details, while using the VGG network to capture global features.&#10;&#10;This combination of local and global features provides an effective representation that models the details of each phase of an action, enabling accurate video alignment. These features can be used individually for phase classification tasks or can be aligned using Dynamic Time Warping (DTW) to synchronize the time series of videos. As shown in Figure \ref{header}, the time series of videos are aligned using a modified version of DTW, known as DDTW, which is introduced in this study.&#10;&#10;To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets. Additionally, a new evaluation metric is introduced in this work to compare the performance of different alignment methods.&#10;&#10;In summary, our contributions include the following:&#10;\begin{itemize}&#10; \item Presenting an unsupervised method to align two videos.&#10; \item Modeling videos using their global and local features as well as their static and dynamic features.&#10; \item Presenting a modified DTW method for aligning time series with limited deviation.&#10; \item Presenting a new metric to compare the performance of alignment methods.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Video Alignment, Time Series Analysis, Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Machine Vision, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2107.04782" label="2107.04782">
-        <attvalues>
-          <attvalue for="0" value="TA2N: Two-Stage Action Alignment Network for Few-shot Action Recognition" />
-          <attvalue for="1" value="  Few-shot action recognition aims to recognize novel action classes (query)&#10;using just a few samples (support). The majority of current approaches follow&#10;the metric learning paradigm, which learns to compare the similarity between&#10;videos. Recently, it has been observed that directly measuring this similarity&#10;is not ideal since different action instances may show distinctive temporal&#10;distribution, resulting in severe misalignment issues across query and support&#10;videos. In this paper, we arrest this problem from two distinct aspects --&#10;action duration misalignment and action evolution misalignment. We address them&#10;sequentially through a Two-stage Action Alignment Network (TA2N). The first&#10;stage locates the action by learning a temporal affine transform, which warps&#10;each video feature to its action duration while dismissing the&#10;action-irrelevant feature (e.g. background). Next, the second stage coordinates&#10;query feature to match the spatial-temporal action evolution of support by&#10;performing temporally rearrange and spatially offset prediction. Extensive&#10;experiments on benchmark datasets show the potential of the proposed method in&#10;achieving state-of-the-art performance for few-shot action recognition.The code&#10;of this project can be found at https://github.com/R00Kie-Liu/TA2N&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.04823" label="2105.04823">
-        <attvalues>
-          <attvalue for="0" value="Learning Implicit Temporal Alignment for Few-shot Video Classification" />
-          <attvalue for="1" value="  Few-shot video classification aims to learn new video categories with only a&#10;few labeled examples, alleviating the burden of costly annotation in real-world&#10;applications. However, it is particularly challenging to learn a&#10;class-invariant spatial-temporal representation in such a setting. To address&#10;this, we propose a novel matching-based few-shot learning strategy for video&#10;sequences in this work. Our main idea is to introduce an implicit temporal&#10;alignment for a video pair, capable of estimating the similarity between them&#10;in an accurate and robust manner. Moreover, we design an effective context&#10;encoding module to incorporate spatial and feature channel context, resulting&#10;in better modeling of intra-class variations. To train our model, we develop a&#10;multi-task loss for learning video matching, leading to video features with&#10;better generalization. Extensive experimental results on two challenging&#10;benchmarks, show that our method outperforms the prior arts with a sizable&#10;margin on SomethingSomething-V2 and competitive results on Kinetics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.00558" label="1612.00558">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Human Action Detection by Action Matching" />
-          <attvalue for="1" value="  We propose a new task of unsupervised action detection by action matching.&#10;Given two long videos, the objective is to temporally detect all pairs of&#10;matching video segments. A pair of video segments are matched if they share the&#10;same human action. The task is category independent---it does not matter what&#10;action is being performed---and no supervision is used to discover such video&#10;segments. Unsupervised action detection by action matching allows us to align&#10;videos in a meaningful manner. As such, it can be used to discover new action&#10;categories or as an action proposal technique within, say, an action detection&#10;pipeline. Moreover, it is a useful pre-processing step for generating video&#10;highlights, e.g., from sports videos.&#10;  We present an effective and efficient method for unsupervised action&#10;detection. We use an unsupervised temporal encoding method and exploit the&#10;temporal consistency in human actions to obtain candidate action segments. We&#10;evaluate our method on this challenging task using three activity recognition&#10;benchmarks, namely, the MPII Cooking activities dataset, the THUMOS15 action&#10;detection benchmark and a new dataset called the IKEA dataset. On the MPII&#10;Cooking dataset we detect action segments with a precision of 21.6% and recall&#10;of 11.7% over 946 long video pairs and over 5000 ground truth action segments.&#10;Similarly, on THUMOS dataset we obtain 18.4% precision and 25.1% recall over&#10;5094 ground truth action segment pairs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.07846" label="1904.07846">
-        <attvalues>
-          <attvalue for="0" value="Temporal Cycle-Consistency Learning" />
-          <attvalue for="1" value="  We introduce a self-supervised representation learning method based on the&#10;task of temporal alignment between videos. The method trains a network using&#10;temporal cycle consistency (TCC), a differentiable cycle-consistency loss that&#10;can be used to find correspondences across time in multiple videos. The&#10;resulting per-frame embeddings can be used to align videos by simply matching&#10;frames using the nearest-neighbors in the learned embedding space.&#10;  To evaluate the power of the embeddings, we densely label the Pouring and&#10;Penn Action video datasets for action phases. We show that (i) the learned&#10;embeddings enable few-shot classification of these action phases, significantly&#10;reducing the supervised training requirements; and (ii) TCC is complementary to&#10;other methods of self-supervised learning in videos, such as Shuffle and Learn&#10;and Time-Contrastive Networks. The embeddings are also used for a number of&#10;applications based on alignment (dense temporal correspondence) between video&#10;pairs, including transfer of metadata of synchronized modalities between videos&#10;(sounds, temporal semantic labels), synchronized playback of multiple videos,&#10;and anomaly detection. Project webpage:&#10;https://sites.google.com/view/temporal-cycle-consistency .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.02598" label="1901.02598">
-        <attvalues>
-          <attvalue for="0" value="D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly&#10;  Supervised Action Alignment and Segmentation" />
-          <attvalue for="1" value="  We address weakly supervised action alignment and segmentation in videos,&#10;where only the order of occurring actions is available during training. We&#10;propose Discriminative Differentiable Dynamic Time Warping (D3TW), the first&#10;discriminative model using weak ordering supervision. The key technical&#10;challenge for discriminative modeling with weak supervision is that the loss&#10;function of the ordering supervision is usually formulated using dynamic&#10;programming and is thus not differentiable. We address this challenge with a&#10;continuous relaxation of the min-operator in dynamic programming and extend the&#10;alignment loss to be differentiable. The proposed D3TW innovatively solves&#10;sequence alignment with discriminative modeling and end-to-end training, which&#10;substantially improves the performance in weakly supervised action alignment&#10;and segmentation tasks. We show that our model is able to bypass the&#10;degenerated sequence problem usually encountered in previous work and&#10;outperform the current state-of-the-art across three evaluation metrics in two&#10;challenging datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.11415" label="1906.11415">
-        <attvalues>
-          <attvalue for="0" value="Few-Shot Video Classification via Temporal Alignment" />
-          <attvalue for="1" value="  There is a growing interest in learning a model which could recognize novel&#10;classes with only a few labeled examples. In this paper, we propose Temporal&#10;Alignment Module (TAM), a novel few-shot learning framework that can learn to&#10;classify a previous unseen video. While most previous works neglect long-term&#10;temporal ordering information, our proposed model explicitly leverages the&#10;temporal ordering information in video data through temporal alignment. This&#10;leads to strong data-efficiency for few-shot learning. In concrete, TAM&#10;calculates the distance value of query video with respect to novel class&#10;proxies by averaging the per frame distances along its alignment path. We&#10;introduce continuous relaxation to TAM so the model can be learned in an&#10;end-to-end fashion to directly optimize the few-shot learning objective. We&#10;evaluate TAM on two challenging real-world datasets, Kinetics and&#10;Something-Something-V2, and show that our model leads to significant&#10;improvement of few-shot video classification over a wide range of competitive&#10;baselines.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.04515" label="2007.04515">
-        <attvalues>
-          <attvalue for="0" value="Aligning Videos in Space and Time" />
-          <attvalue for="1" value="  In this paper, we focus on the task of extracting visual correspondences&#10;across videos. Given a query video clip from an action class, we aim to align&#10;it with training videos in space and time. Obtaining training data for such a&#10;fine-grained alignment task is challenging and often ambiguous. Hence, we&#10;propose a novel alignment procedure that learns such correspondence in space&#10;and time via cross video cycle-consistency. During training, given a pair of&#10;videos, we compute cycles that connect patches in a given frame in the first&#10;video by matching through frames in the second video. Cycles that connect&#10;overlapping patches together are encouraged to score higher than cycles that&#10;connect non-overlapping patches. Our experiments on the Penn Action and Pouring&#10;datasets demonstrate that the proposed method can successfully learn to&#10;correspond semantically similar patches across videos, and learns&#10;representations that are sensitive to object and action states.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.17260" label="2103.17260">
-        <attvalues>
-          <attvalue for="0" value="Learning by Aligning Videos in Time" />
-          <attvalue for="1" value="  We present a self-supervised approach for learning video representations&#10;using temporal video alignment as a pretext task, while exploiting both&#10;frame-level and video-level information. We leverage a novel combination of&#10;temporal alignment loss and temporal regularization terms, which can be used as&#10;supervision signals for training an encoder network. Specifically, the temporal&#10;alignment loss (i.e., Soft-DTW) aims for the minimum cost for temporally&#10;aligning videos in the embedding space. However, optimizing solely for this&#10;term leads to trivial solutions, particularly, one where all frames get mapped&#10;to a small cluster in the embedding space. To overcome this problem, we propose&#10;a temporal regularization term (i.e., Contrastive-IDM) which encourages&#10;different frames to be mapped to different points in the embedding space.&#10;Extensive evaluations on various tasks, including action phase classification,&#10;action phase progression, and fine-grained frame retrieval, on three datasets,&#10;namely Pouring, Penn Action, and IKEA ASM, show superior performance of our&#10;approach over state-of-the-art methods for self-supervised representation&#10;learning from videos. In addition, our method provides significant performance&#10;gain where labeled data is lacking. Our code and labels are available on our&#10;research website: https://retrocausal.ai/research/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.05217" label="2105.05217">
-        <attvalues>
-          <attvalue for="0" value="Representation Learning via Global Temporal Alignment and&#10;  Cycle-Consistency" />
-          <attvalue for="1" value="  We introduce a weakly supervised method for representation learning based on&#10;aligning temporal sequences (e.g., videos) of the same process (e.g., human&#10;action). The main idea is to use the global temporal ordering of latent&#10;correspondences across sequence pairs as a supervisory signal. In particular,&#10;we propose a loss based on scoring the optimal sequence alignment to train an&#10;embedding network. Our loss is based on a novel probabilistic path finding view&#10;of dynamic time warping (DTW) that contains the following three key features:&#10;(i) the local path routing decisions are contrastive and differentiable, (ii)&#10;pairwise distances are cast as probabilities that are contrastive as well, and&#10;(iii) our formulation naturally admits a global cycle consistency loss that&#10;verifies correspondences. For evaluation, we consider the tasks of fine-grained&#10;action classification, few shot learning, and video synchronization. We report&#10;significant performance increases over previous methods. In addition, we report&#10;two applications of our temporal alignment framework, namely 3D pose&#10;reconstruction and fine-grained audio/visual retrieval.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.09301" label="2111.09301">
-        <attvalues>
-          <attvalue for="0" value="Learning to Align Sequential Actions in the Wild" />
-          <attvalue for="1" value="  State-of-the-art methods for self-supervised sequential action alignment rely&#10;on deep networks that find correspondences across videos in time. They either&#10;learn frame-to-frame mapping across sequences, which does not leverage temporal&#10;information, or assume monotonic alignment between each video pair, which&#10;ignores variations in the order of actions. As such, these methods are not able&#10;to deal with common real-world scenarios that involve background frames or&#10;videos that contain non-monotonic sequence of actions.&#10;  In this paper, we propose an approach to align sequential actions in the wild&#10;that involve diverse temporal variations. To this end, we propose an approach&#10;to enforce temporal priors on the optimal transport matrix, which leverages&#10;temporal consistency, while allowing for variations in the order of actions.&#10;Our model accounts for both monotonic and non-monotonic sequences and handles&#10;background frames that should not be aligned. We demonstrate that our approach&#10;consistently outperforms the state-of-the-art in self-supervised sequential&#10;action representation learning on four different benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.08561" label="1603.08561">
-        <attvalues>
-          <attvalue for="0" value="Shuffle and Learn: Unsupervised Learning using Temporal Order&#10;  Verification" />
-          <attvalue for="1" value="  In this paper, we present an approach for learning a visual representation&#10;from the raw spatiotemporal signals in videos. Our representation is learned&#10;without supervision from semantic labels. We formulate our method as an&#10;unsupervised sequential verification task, i.e., we determine whether a&#10;sequence of frames from a video is in the correct temporal order. With this&#10;simple task and no semantic labels, we learn a powerful visual representation&#10;using a Convolutional Neural Network (CNN). The representation contains&#10;complementary information to that learned from supervised image datasets like&#10;ImageNet. Qualitative results show that our method captures information that is&#10;temporally varying, such as human pose. When used as pre-training for action&#10;recognition, our method gives significant gains over learning without external&#10;data on benchmark datasets like UCF101 and HMDB51. To demonstrate its&#10;sensitivity to human pose, we show results for pose estimation on the FLIC and&#10;MPII datasets that are competitive, or better than approaches using&#10;significantly more supervision. Our method can be combined with supervised&#10;representations to provide an additional boost in accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.06888" label="1704.06888">
-        <attvalues>
-          <attvalue for="0" value="Time-Contrastive Networks: Self-Supervised Learning from Video" />
-          <attvalue for="1" value="  We propose a self-supervised approach for learning representations and&#10;robotic behaviors entirely from unlabeled videos recorded from multiple&#10;viewpoints, and study how this representation can be used in two robotic&#10;imitation settings: imitating object interactions from videos of humans, and&#10;imitating human poses. Imitation of human behavior requires a&#10;viewpoint-invariant representation that captures the relationships between&#10;end-effectors (hands or robot grippers) and the environment, object attributes,&#10;and body pose. We train our representations using a metric learning loss, where&#10;multiple simultaneous viewpoints of the same observation are attracted in the&#10;embedding space, while being repelled from temporal neighbors which are often&#10;visually similar but functionally different. In other words, the model&#10;simultaneously learns to recognize what is common between different-looking&#10;images, and what is different between similar-looking images. This signal&#10;causes our model to discover attributes that do not change across viewpoint,&#10;but do change across time, while ignoring nuisance variables such as&#10;occlusions, motion blur, lighting and background. We demonstrate that this&#10;representation can be used by a robot to directly mimic human poses without an&#10;explicit correspondence, and that it can be used as a reward function within a&#10;reinforcement learning algorithm. While representations are learned from an&#10;unlabeled collection of task-related videos, robot behaviors such as pouring&#10;are learned by watching a single 3rd-person demonstration by a human. Reward&#10;functions obtained by following the human demonstrations under the learned&#10;representation enable efficient reinforcement learning that is practical for&#10;real-world robotic systems. Video results, open-source code and dataset are&#10;available at https://sermanet.github.io/imitate&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1212.0402" label="1212.0402">
-        <attvalues>
-          <attvalue for="0" value="UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild" />
-          <attvalue for="1" value="  We introduce UCF101 which is currently the largest dataset of human actions.&#10;It consists of 101 action classes, over 13k clips and 27 hours of video data.&#10;The database consists of realistic user uploaded videos containing camera&#10;motion and cluttered background. Additionally, we provide baseline action&#10;recognition results on this new dataset using standard bag of words approach&#10;with overall performance of 44.5%. To the best of our knowledge, UCF101 is&#10;currently the most challenging dataset of actions due to its large number of&#10;classes, large number of clips and also unconstrained nature of such clips.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.02728" label="1903.02728">
-        <attvalues>
-          <attvalue for="0" value="Graphical Contrastive Losses for Scene Graph Parsing" />
-          <attvalue for="1" value="  Most scene graph parsers use a two-stage pipeline to detect visual&#10;relationships: the first stage detects entities, and the second predicts the&#10;predicate for each entity pair using a softmax distribution. We find that such&#10;pipelines, trained with only a cross entropy loss over predicate classes,&#10;suffer from two common errors. The first, Entity Instance Confusion, occurs&#10;when the model confuses multiple instances of the same type of entity (e.g.&#10;multiple cups). The second, Proximal Relationship Ambiguity, arises when&#10;multiple subject-predicate-object triplets appear in close proximity with the&#10;same predicate, and the model struggles to infer the correct subject-object&#10;pairings (e.g. mis-pairing musicians and their instruments). We propose a set&#10;of contrastive loss formulations that specifically target these types of errors&#10;within the scene graph parsing problem, collectively termed the Graphical&#10;Contrastive Losses. These losses explicitly force the model to disambiguate&#10;related and unrelated instances through margin constraints specific to each&#10;type of confusion. We further construct a relationship detector, called RelDN,&#10;using the aforementioned pipeline to demonstrate the efficacy of our proposed&#10;losses. Our model outperforms the winning method of the OpenImages Relationship&#10;Detection Challenge by 4.7\% (16.5\% relative) on the test set. We also show&#10;improved results over the best previous methods on the Visual Genome and Visual&#10;Relationship Detection datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="&#10;&#10;Scene Graph Parsing: A large number of scene graph parsing approaches have emerged during the last couple of years. They use the same pipeline that first either uses off-the-shelf detectors \cite{lu2016visual,Zhuang_2017_ICCV,zhang2017visual,dai2017detecting,yu17iccv,Xu_2018_ECCV} or detectors fine-tuned with relationship datasets \cite{LiCVPR2017,xu2017scenegraph,zellers2018neural,zhang2017relationship,zhang2018large,Yin_2018_ECCV,Yang_2018_ECCV} to detect entities, then predicts the predicate using proposed methods. Most of them \cite{lu2016visual,Zhuang_2017_ICCV,zhang2017visual,dai2017detecting,yu17iccv,Yin_2018_ECCV,LiCVPR2017,xu2017scenegraph,zellers2018neural,zhang2017relationship,zhang2018neurips,zhang2018introduction} model the second step as a classification task that takes features of each entity pair as input and output a label independently from other pairs. \cite{zhang2018large} instead learn embeddings for subjects, predicates and objects and use nearest neighbor searching during testing to predict predicates. Nevertheless, the prediction is still done on each entity pair individually. We show that this pipeline struggles with two major scenarios. We find that ignoring the intrinsic graph structure of relationships and predicting each predicate separately is the main cause. Our proposed losses compensate for such drawback by contrasting positive against negative edges for each node, providing global supervision to the classifier and significantly alleviating those two issues.&#10;&#10;The scene graph parsing work most related to ours is Associative Embedding \cite{newell2017pixels}. They use use a push and pull contrastive loss to train embeddings for entities within a visual genome scene graph. Our work differs in that we propose to have different sets of hard negatives to target specific error types within scene graph parsing.&#10;&#10;Phrase Grounding and Referring Expressions:&#10;Phrase grounding and referring expression models aim to localize the region described by a given expression, with the latter focusing more on cases of possible reference confusion \cite{yu2018mattnet,mao2016generation,yu2016modeling,nagaraja2016modeling,hu2016natural,luo2017comprehension,rohrbach2016grounding,wang2016learning,liu2017referring,chen2017query,hu2017modeling, plummer2015flickr30k}. It can be abstracted as a bipartite graph matching problem, where nodes on the visual side are the regions and nodes on the language side are the expressions, and the goal is to find all matched pairs. In contrast, scene graphs are arbitrarily connected graphs whose nodes are visual entities and edges are predicates with rich semantic information. Our losses are designed to leverage that information to better discriminate between related and non-related entities.&#10;&#10;Contrastive Training: Contrastive training using a triplet loss \cite{kiros2014unifying} has wide application in both computer vision and natural language processing. Representative works include Negative Sampling \cite{mikolov2013distributed} and Noise Contrastive Sampling \cite{mnih2013learning}. More recent work also utilizes it to solve multi-modal tasks such as phrase grounding, image captioning, VQA, and vector embeddings \cite{wang2016learning,gupta2017aligned,yu2018mattnet,newell2017pixels}.&#10;Our setting differs in that we define hard negative contrastive margins along the known structure of the annotated scene graph, allowing us to specifically target entity instance and proximal relationship confusion. By adding our losses as additional supervision on top of the N-way cross-entropy loss, we are able to improve the model by significant margins.&#10;&#10;" />
-          <attvalue for="4" value="Visual Relationship Detection, Machine Learning, Computer Vision, Scene Graph Parsing, Mathematics, Artificial Intelligence, Contrastive Loss Formulations, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2407.21596" label="2407.21596">
-        <attvalues>
-          <attvalue for="0" value="Evaluating SAM2's Role in Camouflaged Object Detection: From SAM to SAM2" />
-          <attvalue for="1" value="  The Segment Anything Model (SAM), introduced by Meta AI Research as a generic&#10;object segmentation model, quickly garnered widespread attention and&#10;significantly influenced the academic community. To extend its application to&#10;video, Meta further develops Segment Anything Model 2 (SAM2), a unified model&#10;capable of both video and image segmentation. SAM2 shows notable improvements&#10;over its predecessor in terms of applicable domains, promptable segmentation&#10;accuracy, and running speed. However, this report reveals a decline in SAM2's&#10;ability to perceive different objects in images without prompts in its auto&#10;mode, compared to SAM. Specifically, we employ the challenging task of&#10;camouflaged object detection to assess this performance decrease, hoping to&#10;inspire further exploration of the SAM model family by researchers. The results&#10;of this paper are provided in \url{https://github.com/luckybird1994/SAMCOD}.&#10;" />
-          <attvalue for="2" value="&#10;In recent years, large language models (LLMs)~\cite{chowdhery2023palm,touvron2023llama,zhang2022opt} have sparked a revolution in natural language processing (NLP). These foundational models exhibit remarkable transfer capabilities, extending far beyond their initial training objectives. LLMs showcase robust generalization abilities and excel in a multitude of open-world language tasks, including language comprehension, generation, interaction, and reasoning. Inspired by the success of LLMs, vision foundational models such as CLIP~\cite{DBLP:conf/icml/RadfordKHRGASAM21}, DINOv2~\cite{DBLP:journals/corr/abs-2304-07193}, BLIP~\cite{DBLP:conf/icml/0001LXH22}, and SAM~\cite{kirillov2023segment} have also emerged. The introduction of these foundational models continues to drive researchers’ exploration in the field of computer vision.&#10;&#10;Among these foundational models, SAM stands out as an influential model in the domain of image segmentation. Upon its introduction, it quickly receives widespread attention and is utilized across various fields. To further extend SAM’s applicability, SAM2~\cite{ravi2024sam2} is introduced, designed to handle both image and video segmentation tasks within a unified architecture. Compared to SAM, SAM2 has significantly improved in terms of promptable segmentation accuracy and running speed, further enhancing the capabilities of the SAM model family.&#10;&#10;In this technical report, we use the task of camouflaged object detection (COD) as a case study to analyze the progress and compromises involved in transitioning from SAM to SAM2. We observe the following two points: 1) When the SAM2 model is provided with prompts for segmentation, its performance shows substantial improvement compared to SAM. 2) However, when operating in auto mode, where SAM2 and SAM are both tasked with perceiving all objects in an image without prompts, the performance of SAM2 significantly deteriorates. We hope that our findings will further encourage researchers to explore both SAM and SAM2.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Object Segmentation, Deep Learning Models, Computer Science, Video Analysis" />
-        </attvalues>
-      </node>
-      <node id="2204.02311" label="2204.02311">
-        <attvalues>
-          <attvalue for="0" value="PaLM: Scaling Language Modeling with Pathways" />
-          <attvalue for="1" value="  Large language models have been shown to achieve remarkable performance&#10;across a variety of natural language tasks using few-shot learning, which&#10;drastically reduces the number of task-specific training examples needed to&#10;adapt the model to a particular application. To further our understanding of&#10;the impact of scale on few-shot learning, we trained a 540-billion parameter,&#10;densely activated, Transformer language model, which we call Pathways Language&#10;Model PaLM. We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML&#10;system which enables highly efficient training across multiple TPU Pods. We&#10;demonstrate continued benefits of scaling by achieving state-of-the-art&#10;few-shot learning results on hundreds of language understanding and generation&#10;benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough&#10;performance, outperforming the finetuned state-of-the-art on a suite of&#10;multi-step reasoning tasks, and outperforming average human performance on the&#10;recently released BIG-bench benchmark. A significant number of BIG-bench tasks&#10;showed discontinuous improvements from model scale, meaning that performance&#10;steeply increased as we scaled to our largest model. PaLM also has strong&#10;capabilities in multilingual tasks and source code generation, which we&#10;demonstrate on a wide array of benchmarks. We additionally provide a&#10;comprehensive analysis on bias and toxicity, and study the extent of training&#10;data memorization with respect to model scale. Finally, we discuss the ethical&#10;considerations related to large language models and discuss potential&#10;mitigation strategies.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.01068" label="2205.01068">
-        <attvalues>
-          <attvalue for="0" value="OPT: Open Pre-trained Transformer Language Models" />
-          <attvalue for="1" value="  Large language models, which are often trained for hundreds of thousands of&#10;compute days, have shown remarkable capabilities for zero- and few-shot&#10;learning. Given their computational cost, these models are difficult to&#10;replicate without significant capital. For the few that are available through&#10;APIs, no access is granted to the full model weights, making them difficult to&#10;study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only&#10;pre-trained transformers ranging from 125M to 175B parameters, which we aim to&#10;fully and responsibly share with interested researchers. We show that OPT-175B&#10;is comparable to GPT-3, while requiring only 1/7th the carbon footprint to&#10;develop. We are also releasing our logbook detailing the infrastructure&#10;challenges we faced, along with code for experimenting with all of the released&#10;models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.12086" label="2201.12086">
-        <attvalues>
-          <attvalue for="0" value="BLIP: Bootstrapping Language-Image Pre-training for Unified&#10;  Vision-Language Understanding and Generation" />
-          <attvalue for="1" value="  Vision-Language Pre-training (VLP) has advanced the performance for many&#10;vision-language tasks. However, most existing pre-trained models only excel in&#10;either understanding-based tasks or generation-based tasks. Furthermore,&#10;performance improvement has been largely achieved by scaling up the dataset&#10;with noisy image-text pairs collected from the web, which is a suboptimal&#10;source of supervision. In this paper, we propose BLIP, a new VLP framework&#10;which transfers flexibly to both vision-language understanding and generation&#10;tasks. BLIP effectively utilizes the noisy web data by bootstrapping the&#10;captions, where a captioner generates synthetic captions and a filter removes&#10;the noisy ones. We achieve state-of-the-art results on a wide range of&#10;vision-language tasks, such as image-text retrieval (+2.7% in average&#10;recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score).&#10;BLIP also demonstrates strong generalization ability when directly transferred&#10;to video-language tasks in a zero-shot manner. Code, models, and datasets are&#10;released at https://github.com/salesforce/BLIP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.07529" label="2111.07529">
-        <attvalues>
-          <attvalue for="0" value="Object Propagation via Inter-Frame Attentions for Temporally Stable&#10;  Video Instance Segmentation" />
-          <attvalue for="1" value="  Video instance segmentation aims to detect, segment, and track objects in a&#10;video. Current approaches extend image-level segmentation algorithms to the&#10;temporal domain. However, this results in temporally inconsistent masks. In&#10;this work, we identify the mask quality due to temporal stability as a&#10;performance bottleneck. Motivated by this, we propose a video instance&#10;segmentation method that alleviates the problem due to missing detections.&#10;Since this cannot be solved simply using spatial information, we leverage&#10;temporal context using inter-frame attentions. This allows our network to&#10;refocus on missing objects using box predictions from the neighbouring frame,&#10;thereby overcoming missing detections. Our method significantly outperforms&#10;previous state-of-the-art algorithms using the Mask R-CNN backbone, by&#10;achieving 36.0% mAP on the YouTube-VIS benchmark. Additionally, our method is&#10;completely online and requires no future frames. Our code is publicly available&#10;at https://github.com/anirudh-chakravarthy/ObjProp.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In this work, we propose a video instance segmentation algorithm based on the Mask R-CNN pipeline~\cite{he2017Mask}.&#10;We focus on the problem of temporal instability in video instance segmentation (Figure~\ref{fig:teaser}). There are many reasons for temporal instability: missing proposals from a region proposal network, misclassification of the object's class, or aliasing from small visual displacements. &#10;We address temporal instability by propagating masks using object boxes to neighbouring frames to complement missing detections. &#10;Mask propagation through bounding boxes enables tracking of objects even when the detector misses the object's bounding box in the current frame.&#10;To this end, we use the attention mechanism. Our propagation network predicts an attention map propagated from previous frames to the current frame. We apply the attention map to current frame features, which allows us to fill in absent instance masks and overcome temporal instability. &#10;&#10;We have three main contributions in this work. First, we identify the temporal instability for video instance segmentation. Second, by propagating object masks through an inter-frame attention mechanism, we generate temporally coherent and spatially accurate mask tracks. Third, our method outperforms the conventional Mask R-CNN methods on the on the YouTube-VIS dataset~\cite{yang2019Video}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2301.12053" label="2301.12053">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Image Segmentation Beyond Tight Bounding Box&#10;  Annotations" />
-          <attvalue for="1" value="  Weakly supervised image segmentation approaches in the literature usually&#10;achieve high segmentation performance using tight bounding box supervision and&#10;decrease the performance greatly when supervised by loose bounding boxes.&#10;However, compared with loose bounding box, it is much more difficult to acquire&#10;tight bounding box due to its strict requirements on the precise locations of&#10;the four sides of the box. To resolve this issue, this study investigates&#10;whether it is possible to maintain good segmentation performance when loose&#10;bounding boxes are used as supervision. For this purpose, this work extends our&#10;previous parallel transformation based multiple instance learning (MIL) for&#10;tight bounding box supervision by integrating an MIL strategy based on polar&#10;transformation to assist image segmentation. The proposed polar transformation&#10;based MIL formulation works for both tight and loose bounding boxes, in which a&#10;positive bag is defined as pixels in a polar line of a bounding box with one&#10;endpoint located inside the object enclosed by the box and the other endpoint&#10;located at one of the four sides of the box. Moreover, a weighted smooth&#10;maximum approximation is introduced to incorporate the observation that pixels&#10;closer to the origin of the polar transformation are more likely to belong to&#10;the object in the box. The proposed approach was evaluated on two public&#10;datasets using dice coefficient when bounding boxes at different precision&#10;levels were considered in the experiments. The results demonstrate that the&#10;proposed approach achieves state-of-the-art performance for bounding boxes at&#10;all precision levels and is robust to mild and moderate errors in the loose&#10;bounding box annotations. The codes are available at&#10;\url{https://github.com/wangjuan313/wsis-beyond-tightBB}.&#10;" />
-          <attvalue for="2" value="&#10;Image segmentation is the process of partitioning a digital image into multiple image segments such that pixels in an image segment share certain characteristics and are assigned the same category label. It has been widely studied in all kinds of applications \cite{minaee2021image} since 1965. In recent years, with the development of the deep learning in medical image analysis \cite{wang2017detecting,esteva2017dermatologist,wang2018context,wang2020simultaneous}, deep neural networks (DNNs) have been used to successfully tackle a variety of image segmentation problems in a fully-supervised manner \cite{ronneberger2015u,long2015fully,chen2018encoder}. However, it is labor-intensive and expensive to collect large-scale dataset with precise pixel-wise annotations for DNN training, thus limiting the value of the image segmentation in real applications. This is especially true in medical image analysis due to the difficulty of the segmentation problems and the difficulty in recruiting qualified annotators.&#10;&#10;To resolve the problem mentioned above, great interests have been made in the literature to develop weakly supervised image segmentation (WSIS), the purpose of which is to substitute costly pixel-wise annotations into cost-effective annotations as supervision for image segmentation. Several types of annotations have been investigated, including image labels \cite{ahn2019weakly,wang2020self}, points \cite{bearman2016s}, scribbles \cite{lin2016scribblesup}, and bounding boxes \cite{rajchl2016deepcut,kervadec2020bounding}. This work considers bounding boxes as supervision for image segmentation. In the literature, some efforts have been made to develop WSIS adopting bounding box supervision. For example, Rajchl et al. \cite{rajchl2016deepcut} designed an iterative optimization approach for image segmentation, in which a neural network classifier was trained using bounding box supervision. Hsu et al. \cite{hsu2019weakly} exploited mask R-CNN to simultaneously conduct object detection and image segmentation, in which the bounding box supervision was formulated as multiple instance learning (MIL) for image segmentation. Kervadec et al. \cite{kervadec2020bounding} imposed a set of constraints on the network output by leveraging the tightness prior of bounding boxes as supervision for image segmentation. &#10;&#10;Depending on the relationship between an object and its bounding box annotation, as shown in Figure \ref{fig:bbox_demonstration}, bounding box annotations can be divided into two categories: tight bounding box and loose bounding box. The tight bounding box has its four sides touching the object, thus the size of the tight bounding box is same as the size of the object; in contrast, the size of the loose bounding box is larger than the size of the object, thus at least one side of the loose bounding box does not touch the object. &#10;Compared with the loose bounding boxes, the tight bounding boxes have strict requirements on the precise locations of the four sides of the bounding box annotations, thus it is much more difficult and time-consuming to annotate tight bounding boxes. However, most methods in the literature, if not all, achieve high segmentation performance using tight bounding box supervision, and decrease the performance greatly when supervised by loose bounding boxes \cite{kervadec2020bounding}. Such inconsistency in the difficulty of annotation acquisition and the segmentation performance for tight and loose bounding boxes poses a problem in bounding box supervision. To conquer this issue, in this paper we investigate whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision.&#10;&#10;Recently, in our previous study we developed a WSIS approach using tight bounding box supervision by exploiting the properties of tight bounding boxes for MIL formulation \cite{wang2021bounding} and achieved state-of-the-art segmentation performance \cite{wang2021bounding,wang2022cdrnet}. Building on our previous success in \cite{wang2021bounding}, this study extends tight bounding box supervision to loose bounding box supervision for image segmentation. For this purpose, we propose an MIL formulation based on polar transformation of the image region in the bounding box to assist the approach in \cite{wang2021bounding} for image segmentation, and develop a weighted smooth maximum approximation to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the bounding box. In the end, the segmentation assistance is conducted by combining the loss derived from the polar transformation based MIL into the loss in \cite{wang2021bounding}. &#10;The polar transformation based MIL strategy developed in this study is valid for both tight and loose bounding boxes, contributing on the good segmentation performance of the proposed approach for both tight and loose bounding box supervision. &#10;In the experiments, the proposed approach is evaluated by two public datasets when both tight bounding boxes and loose bounding boxes at different precision levels are used as supervision. The results demonstrate that the proposed approach outperforms several existing methods in all precision levels of bounding boxes; and more importantly, it is robust to mild and moderate errors in the loose bounding box annotations.&#10;&#10;In summary, the contributions of this study are as follows:&#10;\begin{enumerate}&#10;\item First, we develop a WSIS approach beyond tight bounding box supervision. It achieves state-of-the-art performance for bounding boxes at all precision levels and is robust to mild and moderate errors in the loose bounding box annotations.&#10;\item Second, we propose an MIL strategy based on polar transformation of the image regions of the bounding boxes to incorporate the bounding box supervision into the network output to assist the image segmentation. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes, contributing on the good segmentation performance of the proposed approach for both tight and loose bounding box supervision. &#10;\item Third, a weighted smooth maximum approximation is introduced for the bag prediction in the proposed polar transformation based MIL to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the box.&#10;\item Finally, the proposed approach is evaluated on two public datasets when bounding boxes at different precision levels are used as supervision. The results demonstrate the effectiveness of the proposed approach for image segmentation in all precision levels of bounding boxes.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Weak Supervision, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1904.05044" label="1904.05044">
-        <attvalues>
-          <attvalue for="0" value="Weakly Supervised Learning of Instance Segmentation with Inter-pixel&#10;  Relations" />
-          <attvalue for="1" value="  This paper presents a novel approach for learning instance segmentation with&#10;image-level class labels as supervision. Our approach generates pseudo instance&#10;segmentation labels of training images, which are used to train a fully&#10;supervised model. For generating the pseudo labels, we first identify confident&#10;seed areas of object classes from attention maps of an image classification&#10;model, and propagate them to discover the entire instance areas with accurate&#10;boundaries. To this end, we propose IRNet, which estimates rough areas of&#10;individual instances and detects boundaries between different object classes.&#10;It thus enables to assign instance labels to the seeds and to propagate them&#10;within the boundaries so that the entire areas of instances can be estimated&#10;accurately. Furthermore, IRNet is trained with inter-pixel relations on the&#10;attention maps, thus no extra supervision is required. Our method with IRNet&#10;achieves an outstanding performance on the PASCAL VOC 2012 dataset, surpassing&#10;not only previous state-of-the-art trained with the same level of supervision,&#10;but also some of previous models relying on stronger supervision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.07866" label="1605.07866">
-        <attvalues>
-          <attvalue for="0" value="DeepCut: Object Segmentation from Bounding Box Annotations using&#10;  Convolutional Neural Networks" />
-          <attvalue for="1" value="  In this paper, we propose DeepCut, a method to obtain pixelwise object&#10;segmentations given an image dataset labelled with bounding box annotations. It&#10;extends the approach of the well-known GrabCut method to include machine&#10;learning by training a neural network classifier from bounding box annotations.&#10;We formulate the problem as an energy minimisation problem over a&#10;densely-connected conditional random field and iteratively update the training&#10;targets to obtain pixelwise object segmentations. Additionally, we propose&#10;variants of the DeepCut method and compare those to a naive approach to CNN&#10;training under weak supervision. We test its applicability to solve brain and&#10;lung segmentation problems on a challenging fetal magnetic resonance dataset&#10;and obtain encouraging results in terms of accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.06816" label="2004.06816">
-        <attvalues>
-          <attvalue for="0" value="Bounding boxes for weakly supervised segmentation: Global constraints&#10;  get close to full supervision" />
-          <attvalue for="1" value="  We propose a novel weakly supervised learning segmentation based on several&#10;global constraints derived from box annotations. Particularly, we leverage a&#10;classical tightness prior to a deep learning setting via imposing a set of&#10;constraints on the network outputs. Such a powerful topological prior prevents&#10;solutions from excessive shrinking by enforcing any horizontal or vertical line&#10;within the bounding box to contain, at least, one pixel of the foreground&#10;region. Furthermore, we integrate our deep tightness prior with a global&#10;background emptiness constraint, guiding training with information outside the&#10;bounding box. We demonstrate experimentally that such a global constraint is&#10;much more powerful than standard cross-entropy for the background class. Our&#10;optimization problem is challenging as it takes the form of a large set of&#10;inequality constraints on the outputs of deep networks. We solve it with&#10;sequence of unconstrained losses based on a recent powerful extension of the&#10;log-barrier method, which is well-known in the context of interior-point&#10;methods. This accommodates standard stochastic gradient descent (SGD) for&#10;training deep networks, while avoiding computationally expensive and unstable&#10;Lagrangian dual steps and projections. Extensive experiments over two different&#10;public data sets and applications (prostate and brain lesions) demonstrate that&#10;the synergy between our global tightness and emptiness priors yield very&#10;competitive performances, approaching full supervision and outperforming&#10;significantly DeepCut. Furthermore, our approach removes the need for&#10;computationally expensive proposal generation. Our code is shared anonymously.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.00934" label="2110.00934">
-        <attvalues>
-          <attvalue for="0" value="Bounding Box Tightness Prior for Weakly Supervised Image Segmentation" />
-          <attvalue for="1" value="  This paper presents a weakly supervised image segmentation method that adopts&#10;tight bounding box annotations. It proposes generalized multiple instance&#10;learning (MIL) and smooth maximum approximation to integrate the bounding box&#10;tightness prior into the deep neural network in an end-to-end manner. In&#10;generalized MIL, positive bags are defined by parallel crossing lines with a&#10;set of different angles, and negative bags are defined as individual pixels&#10;outside of any bounding boxes. Two variants of smooth maximum approximation,&#10;i.e., $\alpha$-softmax function and $\alpha$-quasimax function, are exploited&#10;to conquer the numeral instability introduced by maximum function of bag&#10;prediction. The proposed approach was evaluated on two pubic medical datasets&#10;using Dice coefficient. The results demonstrate that it outperforms the&#10;state-of-the-art methods. The codes are available at&#10;\url{https://github.com/wangjuan313/wsis-boundingbox}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2110.00943" label="2110.00943">
-        <attvalues>
-          <attvalue for="0" value="CDRNet: Accurate Cup-to-Disc Ratio Measurement with Tight Bounding Box&#10;  Supervision in Fundus Photography Using Deep Learning" />
-          <attvalue for="1" value="  The cup-to-disc ratio (CDR) is one of the most significant indicator for&#10;glaucoma diagnosis. Different from the use of costly fully supervised learning&#10;formulation with pixel-wise annotations in the literature, this study&#10;investigates the feasibility of accurate CDR measurement in fundus images using&#10;only tight bounding box supervision. For this purpose, we develop a two-task&#10;network named as CDRNet for accurate CDR measurement, one for weakly supervised&#10;image segmentation, and the other for bounding-box regression. The weakly&#10;supervised image segmentation task is implemented based on generalized multiple&#10;instance learning formulation and smooth maximum approximation, and the&#10;bounding-box regression task outputs class-specific bounding box prediction in&#10;a single scale at the original image resolution. To get accurate bounding box&#10;prediction, a class-specific bounding-box normalizer and an expected&#10;intersection-over-union are proposed. In the experiments, the proposed approach&#10;was evaluated by a testing set with 1200 images using CDR error and $F_1$ score&#10;for CDR measurement and dice coefficient for image segmentation. A grader study&#10;was conducted to compare the performance of the proposed approach with those of&#10;individual graders. The experimental results indicate that the proposed&#10;approach outperforms the state-of-the-art performance obtained from the fully&#10;supervised image segmentation (FSIS) approach using pixel-wise annotation for&#10;CDR measurement. Its performance is also better than those of individual&#10;graders. In addition, the proposed approach gets performance close to the&#10;state-of-the-art obtained from FSIS and the performance of individual graders&#10;for optic cup and disc segmentation. The codes are available at&#10;\url{https://github.com/wangjuan313/CDRNet}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.09997" label="2302.09997">
-        <attvalues>
-          <attvalue for="0" value="A Large Scale Homography Benchmark" />
-          <attvalue for="1" value="  We present a large-scale dataset of Planes in 3D, Pi3D, of roughly 1000&#10;planes observed in 10 000 images from the 1DSfM dataset, and HEB, a large-scale&#10;homography estimation benchmark leveraging Pi3D. The applications of the Pi3D&#10;dataset are diverse, e.g. training or evaluating monocular depth, surface&#10;normal estimation and image matching algorithms. The HEB dataset consists of&#10;226 260 homographies and includes roughly 4M correspondences. The homographies&#10;link images that often undergo significant viewpoint and illumination changes.&#10;As applications of HEB, we perform a rigorous evaluation of a wide range of&#10;robust estimators and deep learning-based correspondence filtering methods,&#10;establishing the current state-of-the-art in robust homography estimation. We&#10;also evaluate the uncertainty of the SIFT orientations and scales w.r.t. the&#10;ground truth coming from the underlying homographies and provide codes for&#10;comparing uncertainty of custom detectors. The dataset is available at&#10;\url{https://github.com/danini/homography-benchmark}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The planar homography is a projective mapping between images of co-planar 3D points. &#10;The homography induced by a plane is unique up to a scale and has eight degrees-of-freedom (DoF). &#10;It encodes the intrinsic and extrinsic camera parameters and the parameters of the underlying 3D plane.&#10;&#10;The homography plays an important role in the geometry of multiple views~\cite{hartley2003multiple} with hundreds of papers published in the last few decades about its theory and applications.&#10;&#10;Estimating planar homographies from image pairs is an important task in computer vision with a number of applications. &#10;For instance, monocular SLAM systems~\cite{taketomi2017visual,younes2017keyframe,saputra2018visual} rely on homographies when detecting pure rotational camera movements, planar scenes, and scenes with far objects. &#10;As a homography induced by a plane at infinity represents rotation-only camera motion, it is one of the most important tools for stitching images~\cite{brown2007automatic,adel2014image}.&#10;The generated images cover a larger field-of-view and are useful in various applications, \eg image-based localization~\cite{Arth2011}, SLAM~\cite{Lemaire2007,Ji2020}, autonomous driving~\cite{Wang2020}, sport broadcasting~\cite{chen2018two}, surveillance~\cite{yang2019panoramic}, and augmented and virtual reality~\cite{jethwa1998real,macquarrie2017cinematic}.&#10;Homographies play an important role in calibration~\cite{zhang2000flexible,chuan2003planar}, metric rectification~\cite{collins1993matching,liebowitz1998metric}, augmented reality~\cite{simon2000markerless,zhou2012robust}, optical flow based on piece-wise planar scene modeling~\cite{yang2015dense}, video stabilization~\cite{grundmann2012calibration,zhou2013plane}, and incremental~\cite{schonberger2016structure} and global~\cite{theia-manual,moulon2016openmvg} Structure-from-Motion.&#10;&#10;The traditional approach of finding homographies in image pairs consists of two main stages.&#10;First, similarly as in most algorithms working with pairs, feature points are detected and matched~\cite{SIFT2004,ORB2011,brown2007automatic,schoenberger2016vote,IMC2020}. &#10;They are then often filtered by the widely-used second nearest neighbors (SNN) ratio~\cite{lowe1999object,SIFT2004} or by deep learned filtering methods~\cite{cne2018,acne2020,dfe2018,clnet2021}, to remove gross outliers and, therefore, improve the robust estimation procedure that follows. &#10;The found tentative point correspondences are contaminated by various sources of noise due to, \eg, measurement and quantization, and a large proportion of them are still outliers -- correspondences inconsistent with the sought model manifold. &#10;Consequently, some form of robust estimation has to be applied to find a set of inliers and to estimate the parameters of the sought homography.&#10;In practice, either a randomized RANSAC-like~\cite{fischler1981random} robust estimator or an iteratively re-weighted least squares fitting~\cite{holland1977robust} is applied.&#10;&#10;The number of datasets on which recent homography and, in general, robust estimation papers evaluate their algorithms is severely limited. &#10;The Homogr dataset~\cite{fixingLORANSAC2012} consists only of a few image pairs with relatively small baselines and, thus, high inlier ratios.&#10;Given that recent robust estimators, \eg \cite{barath2018graph}, report lower than $0.5$ pixel average re-projection errors on the provided manually labeled correspondences, it is safe to say that this dataset is solved.&#10;The HPatches dataset~\cite{hpatches2017} consists of a few hundreds of image pairs, all looking at an almost completely planar scene, with either significant illumination or viewpoint (mostly in tilt angle) changes. &#10;While \cite{hpatches2017} is a useful tool for evaluating local feature detector and image matching methods, it is very easy for robust estimators~\cite{cvpr2020ransactutorial}.&#10;The ExtremeView (EVD) dataset~\cite{Mishkin2015MODS} poses a significantly more challenging problem for homography estimation than the previous two. &#10;The images undergo extreme view-point changes, therefore making both the feature matching and robust estimation tasks especially challenging. However, EVD consists only of 15 image pairs, severely limiting its benchmarking power. &#10;&#10;Besides the data part, a good benchmark has well-defined parameter tuning (training) and evaluation protocols and training-test set split. &#10;Otherwise, as it happens in other fields, the seemingly rapid progress might be an artifact of tuning the algorithms on the test data, or an artifact of the flawed evaluation procedure~\cite{musgrave2020metric,pGT2021,goyal2021revisiting}.&#10;&#10;In short, there are no available large-scale benchmarks with ground truth (GT) homographies that allow evaluating new algorithms on standard internet photos, \ie, ones not necessarily looking at completely planar scenes. &#10;&#10;As the first contribution, we create a large-scale dataset of \num{1046} large Planes in 3D (Pi3D) from a standard landmark dataset~\cite{wilson_eccv2014_1dsfm}. &#10;We use the scenes from the 1DSfM dataset as input and find 3D planes in the reconstructions. &#10;Second, we use the Pi3D dataset to find image pairs with estimatable homographies and create a large-scale homography benchmark (HEB) containing a total of \num{226260} homographies that can be considered GT when testing new algorithms (see Fig.~\ref{fig:example_images} for examples). &#10;A large proportion of the image pairs capture significant viewpoint and illumination changes.&#10;The homographies typically have low inlier ratio, thus making the robust estimation task challenging.&#10;Third, we compare a wide range of robust estimators, including recent ones based on neural networks, establishing the current state-of-the-art in robust homography estimation. &#10;As the forth contribution, we demonstrate that the dataset can be used to evaluate the uncertainty of partially or fully affine covariant features detectors~\cite{SIFT2004,AffNet2018}.&#10;While we show it on DoG features~\cite{lowe1999object}, the homographies can be leveraged similarly for the comparison with other detectors. &#10;&#10;Existing Datasets.&#10;The datasets traditionally used for evaluating homography estimators are the following. &#10;The Homogr dataset~\cite{fixingLORANSAC2012} consists of 16 image pairs with GT homographies. The GT comes from (also provided) hand-labeled correspondences, which later were optimized to improve the localization accuracy. There is no train-test split, nor a benchmark protocol.&#10;The ExtremeView dataset~\cite{Mishkin2015MODS} consists of 15 image pairs, taken under extreme viewpoint change, together with GT homographies and correspondences. The homographies are derived from hand-labeled correspondences that stem from multiple local feature detectors paired with an affine view synthesis procedure~\cite{Mishkin2015MODS} and RootSIFT descriptor~\cite{RootSIFT2012} matching.&#10;There is no train-test split, nor a benchmark protocol.&#10;The HPatches dataset~\cite{hpatches2017} was introduced in form of local patches for benchmarking descriptors and metric learning methods, later extended with images and homographies. &#10;It consists of 57 image sextuplets with significant illumination but negligible viewpoint changes and 59 ones with viewpoint, but no illumination changes. &#10;The viewpoint difference mostly consist of tilt (perspective change) in the horizontal direction and some shift -- no big rotation or scale changes.&#10;The GT was obtained from manually annotated correspondences for the initial model estimation and polished by minimizing MSE of image pixel re-projections.&#10;There is no official protocol, nor standard correspondences for homography evaluation -- every paper uses slightly different evaluations, but there is an official train-test split.&#10;&#10;To conclude, there is no difficult-enough, large-scale dataset with train-test split and benchmark protocol for evaluating robust homography estimation. &#10;Table~\ref{tab:datasets} summarizes the properties of each publicly available dataset and, also, that of the proposed one. &#10;Typical image pairs from the datasets are shown in the first three columns of Fig.~\ref{fig:example_images_from_datasets}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Data Benchmarking, Computer Science, Geometry and" />
-        </attvalues>
-      </node>
-      <node id="1607.00470" label="1607.00470">
-        <attvalues>
-          <attvalue for="0" value="Keyframe-based monocular SLAM: design, survey, and future directions" />
-          <attvalue for="1" value="  Extensive research in the field of monocular SLAM for the past fifteen years&#10;has yielded workable systems that found their way into various applications in&#10;robotics and augmented reality. Although filter-based monocular SLAM systems&#10;were common at some time, the more efficient keyframe-based solutions are&#10;becoming the de facto methodology for building a monocular SLAM system. The&#10;objective of this paper is threefold: first, the paper serves as a guideline&#10;for people seeking to design their own monocular SLAM according to specific&#10;environmental constraints. Second, it presents a survey that covers the various&#10;keyframe-based monocular SLAM systems in the literature, detailing the&#10;components of their implementation, and critically assessing the specific&#10;strategies made in each proposed solution. Third, the paper provides insight&#10;into the direction of future research in this field, to address the major&#10;limitations still facing monocular SLAM; namely, in the issues of illumination&#10;changes, initialization, highly dynamic motion, poorly textured scenes,&#10;repetitive textures, map maintenance, and failure recovery.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.09005" label="1801.09005">
-        <attvalues>
-          <attvalue for="0" value="A Two-point Method for PTZ Camera Calibration in Sports" />
-          <attvalue for="1" value="  Calibrating narrow field of view soccer cameras is challenging because there&#10;are very few field markings in the image. Unlike previous solutions, we propose&#10;a two-point method, which requires only two point correspondences given the&#10;prior knowledge of base location and orientation of a pan-tilt-zoom (PTZ)&#10;camera. We deploy this new calibration method to annotate pan-tilt-zoom data&#10;from soccer videos. The collected data are used as references for new images.&#10;We also propose a fast random forest method to predict pan-tilt angles without&#10;image-to-image feature matching, leading to an efficient calibration method for&#10;new images. We demonstrate our system on synthetic data and two real soccer&#10;datasets. Our two-point approach achieves superior performance over the&#10;state-of-the-art method.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.01587" label="2003.01587">
-        <attvalues>
-          <attvalue for="0" value="Image Matching across Wide Baselines: From Paper to Practice" />
-          <attvalue for="1" value="  We introduce a comprehensive benchmark for local features and robust&#10;estimation algorithms, focusing on the downstream task -- the accuracy of the&#10;reconstructed camera pose -- as our primary metric. Our pipeline's modular&#10;structure allows easy integration, configuration, and combination of different&#10;methods and heuristics. This is demonstrated by embedding dozens of popular&#10;algorithms and evaluating them, from seminal works to the cutting edge of&#10;machine learning research. We show that with proper settings, classical&#10;solutions may still outperform the perceived state of the art.&#10;  Besides establishing the actual state of the art, the conducted experiments&#10;reveal unexpected properties of Structure from Motion (SfM) pipelines that can&#10;help improve their performance, for both algorithmic and learned methods. Data&#10;and code are online https://github.com/vcg-uvic/image-matching-benchmark,&#10;providing an easy-to-use and flexible framework for the benchmarking of local&#10;features and robust estimation methods, both alongside and against&#10;top-performing methods. This work provides a basis for the Image Matching&#10;Challenge https://vision.uvic.ca/image-matching-challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.05304" label="2106.05304">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Point Cloud Shape Classification with a Simple and Effective&#10;  Baseline" />
-          <attvalue for="1" value="  Processing point cloud data is an important component of many real-world&#10;systems. As such, a wide variety of point-based approaches have been proposed,&#10;reporting steady benchmark improvements over time. We study the key ingredients&#10;of this progress and uncover two critical results. First, we find that&#10;auxiliary factors like different evaluation schemes, data augmentation&#10;strategies, and loss functions, which are independent of the model&#10;architecture, make a large difference in performance. The differences are large&#10;enough that they obscure the effect of architecture. When these factors are&#10;controlled for, PointNet++, a relatively older network, performs competitively&#10;with recent methods. Second, a very simple projection-based method, which we&#10;refer to as SimpleView, performs surprisingly well. It achieves on par or&#10;better results than sophisticated state-of-the-art methods on ModelNet40 while&#10;being half the size of PointNet++. It also outperforms state-of-the-art methods&#10;on ScanObjectNN, a real-world point cloud benchmark, and demonstrates better&#10;cross-dataset generalization. Code is available at&#10;https://github.com/princeton-vl/SimpleView.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.06704" label="1711.06704">
-        <attvalues>
-          <attvalue for="0" value="Repeatability Is Not Enough: Learning Affine Regions via&#10;  Discriminability" />
-          <attvalue for="1" value="  A method for learning local affine-covariant regions is presented. We show&#10;that maximizing geometric repeatability does not lead to local regions, a.k.a&#10;features,that are reliably matched and this necessitates descriptor-based&#10;learning. We explore factors that influence such learning and registration: the&#10;loss function, descriptor type, geometric parametrization and the trade-off&#10;between matchability and geometric accuracy and propose a novel hard&#10;negative-constant loss function for learning of affine regions. The affine&#10;shape estimator -- AffNet -- trained with the hard negative-constant loss&#10;outperforms the state-of-the-art in bag-of-words image retrieval and wide&#10;baseline stereo. The proposed training process does not require precisely&#10;geometrically aligned patches.The source codes and trained weights are&#10;available at https://github.com/ducha-aiki/affnet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.09564" label="2305.09564">
-        <attvalues>
-          <attvalue for="0" value="Image Reconstruction using Superpixel Clustering and Tensor Completion" />
-          <attvalue for="1" value="  This paper presents a pixel selection method for compact image representation&#10;based on superpixel segmentation and tensor completion. Our method divides the&#10;image into several regions that capture important textures or semantics and&#10;selects a representative pixel from each region to store. We experiment with&#10;different criteria for choosing the representative pixel and find that the&#10;centroid pixel performs the best. We also propose two smooth tensor completion&#10;algorithms that can effectively reconstruct different types of images from the&#10;selected pixels. Our experiments show that our superpixel-based method achieves&#10;better results than uniform sampling for various missing ratios.&#10;" />
-          <attvalue for="2" value="&#10;Dealing with incomplete data tensors is inevitable in real-world applications due to sensor malfunctioning, inaccurate data acquisition, communication problems or inappropriate handling. However, data elements can also be manually removed to optimise space requirements or to remove unwanted outliers. Estimating the unknown elements of an incomplete data tensor is known as {tensor completion} and has played important roles in many machine learning problems, e.g. image/video completion \cite{song2019tensor} and recommender systems \cite{frolov2017tensor}. Due to the importance of tensor completion in the above-mentioned applications, several efficient algorithms have been developed during the past few decades to solve it. Indeed, tensor completion has its basis from matrix completion \cite{candes2009exact}. Similar to the matrix completion where the main assumption is low-rank property of the underlying data matrix, for the tensor case, it is also assumed that the data tensor has low-rank structure. This assumption plays a key role in the formulation of the optimization problem and also deriving theoretical results. It should be noted that the notion of rank for tensors is not unique as it is for matrices, different types of tensor ranks have been defined \cite{cichocki2016low}. Minimizing the matrix rank is an NP hard problem and it has been proved that the nuclear norm is the convex envelope of the matrix rank which can be used to efficiently estimate the matrix rank \cite{fazel2003matrix}. Replacing the matrix rank function with the nuclear norm, converts a non-convex optimization problem into a convex one which is more favorable. Following the matrix case, the nuclear norm of the unfolding matrices has been used to efficiently recover different types of tensor ranks, see \cite{song2019tensor} for a comprehensive review on this topic.&#10;&#10;This paper is inspired by the idea of using tensor completion technology to compress and transmit images \cite{li2010tensor}. The idea is to sample a subset of pixels from an image at the source and send them to the destination through a network. Then, at the destination, the tensor completion algorithms are applied to the incomplete image to recover it. This can save memory and speed up data transmission. Most existing papers use uniform sampling to select pixels, but we wonder if other heuristic approaches can perform better. To the best of our knowledge, this question has not been investigated before. We propose to cluster a given image into several partitions that have similar characteristics and then select pixels from each partition. As an efficient clustering method, superpixel algorithms have been used in several applications such as image segmentation \cite{albayrak2019automatic, meinhold2018robust,chen2017linear,li2015superpixel} and object detection \cite{yan2015object}. Among several superpixel methods, Simple Linear Iterative&#10;Clustering (SLIC) algorithm \cite{achanta2010slic} is known for its high performance and fast execution time. Due to these key issues, we use it in our work as a preprocessing stage to find partitions with similar texture/semantic. Then, we select pixels from each superpixel separately. Since each superpixel has homogeneous features and share similar pixel information, the preprocessing stage can hep us to avoid sampling many redundant and similar pixels. Of course, there are several ways to select pixels from each superpixel such as the pixel located in the center or those are on the border. It is also possible to apply the clustering algorithm repeatedly to each superpixel to further explore important pixels. We have extensively investigated the performance of such different sampling strategies. Our simulation results confirmed that the pixel selection based on superpixel preprocessing provides better results than the uniform sampling for a variety of images. Besides, the superpixel preprocessing method with center pixel selection achieved the best results compared with selecting other pixels in the superpixels. &#10;&#10;For recovering the sampled/incomplete image, we also propose the Smooth Tensor nuclear norm (STNN) and Smooth Matrix Nuclear Norm (SMNN) algorithms for image completion. In particular, the smoothing processing enables the completion algorithms to provide better results and this is shown experimentally in our simulation results. The important pixel selection followed by applying efficient tensor completion algorithms totally improves the traditional uniform sampling approach. &#10;&#10;We summarize our main contributions as follows:&#10;\begin{itemize}&#10; \item We sample and capture important pixels in images using the superpixel technique.&#10;&#10;\item We develop efficient tensor completion algorithms with smoothing and filtering methods to enhance their performance.&#10;&#10;\item We conduct extensive simulations on various images with missing patterns, including {random } and {structured} ones.&#10;\end{itemize}&#10;&#10;The rest of this paper is organized as follows: Section \ref{Pre:Sec} introduces the notations and concepts that we use in our approach. Section \ref{Sec:Super} explains the superpixel technique and how it extracts important content or semantics from the images. Section \ref{Method:Sec} studies the importance of pixel selection and how to sample important pixels. Section \ref{Sec:Tencom} develops tensor completion algorithms with an efficient smoothing technique to improve their performance. Section \ref{Experiments:Sec} presents extensive experiments to demonstrate the applicability and feasibility of our methods. Section \ref{Conclude:Sec} concludes the paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Tensor Completion, Machine Learning, Engineering, Computer Vision, Superpixel Segmentation, Mathematics, Image Representation, Computer Science, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1711.10105" label="1711.10105">
-        <attvalues>
-          <attvalue for="0" value="Tensor Completion Algorithms in Big Data Analytics" />
-          <attvalue for="1" value="  Tensor completion is a problem of filling the missing or unobserved entries&#10;of partially observed tensors. Due to the multidimensional character of tensors&#10;in describing complex datasets, tensor completion algorithms and their&#10;applications have received wide attention and achievement in areas like data&#10;mining, computer vision, signal processing, and neuroscience. In this survey,&#10;we provide a modern overview of recent advances in tensor completion algorithms&#10;from the perspective of big data analytics characterized by diverse variety,&#10;large volume, and high velocity. We characterize these advances from four&#10;perspectives: general tensor completion algorithms, tensor completion with&#10;auxiliary information (variety), scalable tensor completion algorithms&#10;(volume), and dynamic tensor completion algorithms (velocity). Further, we&#10;identify several tensor completion applications on real-world data-driven&#10;problems and present some common experimental frameworks popularized in the&#10;literature. Our goal is to summarize these popular methods and introduce them&#10;to researchers and practitioners for promoting future research and&#10;applications. We conclude with a discussion of key challenges and promising&#10;research directions in this community for future exploration.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.06038" label="1603.06038">
-        <attvalues>
-          <attvalue for="0" value="Tensor Methods and Recommender Systems" />
-          <attvalue for="1" value="  A substantial progress in development of new and efficient tensor&#10;factorization techniques has led to an extensive research of their&#10;applicability in recommender systems field. Tensor-based recommender models&#10;push the boundaries of traditional collaborative filtering techniques by taking&#10;into account a multifaceted nature of real environments, which allows to&#10;produce more accurate, situational (e.g. context-aware, criteria-driven)&#10;recommendations. Despite the promising results, tensor-based methods are poorly&#10;covered in existing recommender systems surveys. This survey aims to complement&#10;previous works and provide a comprehensive overview on the subject. To the best&#10;of our knowledge, this is the first attempt to consolidate studies from various&#10;application domains in an easily readable, digestible format, which helps to&#10;get a notion of the current state of the field. We also provide a high level&#10;discussion of the future perspectives and directions for further improvement of&#10;tensor-based recommendation systems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="0805.4471" label="0805.4471">
-        <attvalues>
-          <attvalue for="0" value="Exact Matrix Completion via Convex Optimization" />
-          <attvalue for="1" value="  We consider a problem of considerable practical interest: the recovery of a&#10;data matrix from a sampling of its entries. Suppose that we observe m entries&#10;selected uniformly at random from a matrix M. Can we complete the matrix and&#10;recover the entries that we have not seen?&#10;  We show that one can perfectly recover most low-rank matrices from what&#10;appears to be an incomplete set of entries. We prove that if the number m of&#10;sampled entries obeys m &gt;= C n^{1.2} r log n for some positive numerical&#10;constant C, then with very high probability, most n by n matrices of rank r can&#10;be perfectly recovered by solving a simple convex optimization program. This&#10;program finds the matrix with minimum nuclear norm that fits the data. The&#10;condition above assumes that the rank is not too large. However, if one&#10;replaces the 1.2 exponent with 1.25, then the result holds for all values of&#10;the rank. Similar results hold for arbitrary rectangular matrices as well. Our&#10;results are connected with the recent literature on compressed sensing, and&#10;show that objects other than signals and images can be perfectly reconstructed&#10;from very limited information.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.11395" label="2203.11395">
-        <attvalues>
-          <attvalue for="0" value="A Binary Characterization Method for Shape Convexity and Applications" />
-          <attvalue for="1" value="  Convexity prior is one of the main cue for human vision and shape completion&#10;with important applications in image processing, computer vision. This paper&#10;focuses on characterization methods for convex objects and applications in&#10;image processing. We present a new method for convex objects representations&#10;using binary functions, that is, the convexity of a region is equivalent to a&#10;simple quadratic inequality constraint on its indicator function. Models are&#10;proposed firstly by incorporating this result for image segmentation with&#10;convexity prior and convex hull computation of a given set with and without&#10;noises. Then, these models are summarized to a general optimization problem on&#10;binary function(s) with the quadratic inequality. Numerical algorithm is&#10;proposed based on linearization technique, where the linearized problem is&#10;solved by a proximal alternating direction method of multipliers with&#10;guaranteed convergent. Numerical experiments demonstrate the efficiency and&#10;effectiveness of the proposed methods for image segmentation and convex hull&#10;computation in accuracy and computing time.&#10;" />
-          <attvalue for="2" value="&#10;Shape priors are widely used in image processing, computer vision and&#10;other fields. The commonly concerned generic shape priors contain&#10;star shape \cite{2019star-convex,2022star,Veksler2008StarSP}, convexity \cite{Gorelick2016convexity,luo2019convex}, connectivity \cite{2019connecvtivity}, constant width \cite{Parameteric2022}.&#10;The mathematical characterization methods for the concerned priors are very important in real applications.&#10;In this paper, we focus on the characterization method of convexity and its applications because the convexity prior is widely used in&#10;image segmentation and computer vision.&#10;Convexity prior is one of the main cue for human vision and shape completion.&#10;Therefore, many characterization methods for convexity have been proposed in the literature. These methods can be roughly categorized into&#10;two classes: boundary or curvature methods&#10;\cite{ukwatta2013efficient,bae2017ALMEuler,2022Luo} and region methods \cite{Gorelick2016convexity,royer2016convexity}.&#10;&#10;The boundary methods are based on the fact that the convexity of a region is equivalent to the nonnegativity of the curvature of&#10;its boundary. In order to compute the curvature $\kappa$ of an evolving boundary curve efficiently, the curve is usually embedded into a two dimensional function, named level set function $\phi$, and $\kappa=\nabla\cdot\frac{\nabla\phi}{|\nabla\phi|}$ \cite{goldman2005curvature} for each level set curve. In the pioneer work, the nonnegative curvatures of the boundary curve(s) are penalized only in the models to steer and preserve the convexity of the&#10;concerned objects \cite{ukwatta2013efficient,yang2017ventricle,bae2017ALMEuler}.&#10;Later, a special level set function, called signed distance function (SDF), is used to compute the curvature. Not only the computation formula is simplified as $\Delta\phi$ due to $|\nabla\phi|=1$ for SDF, but also it is proved that&#10;the convexity of the concerned region is equivalent to $\Delta\phi\geq0$ on the whole image domain rather than on the object boundary only \cite{yan2020convexity,luo2019convex}. This method is generalized to multiple convex objects \cite{luo2019convex}, convex ring \cite{2022Luo}, 3D convex object \cite{2021Luo} characterization, and applied to compute convex hull of given sets with and without outliers \cite{li2021new}.&#10;&#10;The region methods are based on the mathematical definition of convexity, i.e.,&#10;all the line segments between any two points in the region belong to the region as well.&#10;For these methods, binary function, which is 1 on the concerned region and 0 otherwise, i.e., the indicator function of the concerned region, is introduced.&#10;Based on the convexity definition and the introduced binary function,&#10;all $1$-$0$-$1$ configurations on all lines in the image domain should be excluded&#10;to obtain a convex result in \cite{Gorelick2016convexity}. Then&#10;this method is extended to multiple convex objects segmentation \cite{gorelick2017multi}.&#10;Another region based method employs the fact that&#10;the line segment of the two ends of any curve in a convex region is in the convex region \cite{royer2016convexity}.&#10;&#10;We lack efficient algorithms for the involved minimization problems of the two aforementioned approaches.&#10;The minimization problems for the region methods are NP-hard and only approximation solutions can be obtained by penalization and linearization techniques&#10;because the associated problems are combination optimization issues \cite{Gorelick2016convexity,gorelick2017multi,royer2016convexity}.&#10;The minimization problems for the&#10;boundary methods contain a nonsmooth and nonconvex term from curvature formula \cite{2022Luo,yang2017ventricle,bae2017ALMEuler}, which is very difficult to tackle.&#10;Therefore, it is deserved to develop a new characterization method for convexity shape prior.&#10;&#10;Very recently, we proposed a novel method for convex shape characterization in our short paper \cite{binary2021Luo}.&#10;Let $o$ be the indicator function associated with the concerned object $D$.&#10;The convexity $D$ is equivalently characterized by&#10;a quadratic constraint on $o$, that is&#10;$&#10;[b_r\ast o](x)[1-o(x)]\leq \frac{1}{2}[1-o(x)],~~ \forall r\geq0,&#10;$&#10;where '$\ast$' denotes the convolution operation of two functions, and $b_r$ is a positive radial function defined on the disc with radius $r$ and center $(0,0)$ such that $\int_{\mathbb{R}^2}b_r(x)dx=1$.&#10;&#10;This method combines the region method and boundary method.&#10;On one hand, this method characterizes convexity by the nonnegativity of curvature&#10;on the boundary essentially.&#10;On the other hand, binary function is adopted to denote the&#10;object of interest like the region based method.&#10;Thus, this method uses binary function rather than general level set function to compute the boundary curvature.&#10;&#10;The current paper extends this method in three aspects.&#10;Firstly, a general model&#10;is proposed, which can handle&#10;multiple objects segmentation with convexity prior and convex hull computation of a given set. A model for multiple objects segmentation with&#10;convexity prior is proposed by incorporating&#10;the characterization method into a probability model, and&#10;two variational models for convex hull computation of a given set with and without noises are designed, respectively. Then, the models are&#10;summarized as a general model mathematically.&#10;Secondly, an efficient algorithm is designed for the general model.&#10;Linearization technique for the quadratic terms is employed to&#10;alleviate the solving difficulties,&#10;and then a proximal alternating direction method of multipliers (ADMM) \cite{PADMM} is designed for the linearization problem.&#10;The proximal ADMM is very simple and efficient with closed form solutions to all sub-steps.&#10;Thirdly, an interactive technique is adopted&#10;to improve segmentation accuracy for complicated images, e.g. images with cluttered edges or illumination bias. Extensive experiments show the effectiveness and efficiency of the proposed models and algorithms.&#10;&#10;The rest of this paper is organized as follows. Section \ref{sec2} introduces some notations and preliminary results.&#10;The proposed models are presented in Section \ref{ourm}.&#10;Section \ref{sec3} is devoted to the&#10;algorithms for the general model.&#10;Numerical experiments and some performance comparisons are demonstrated in Section \ref{sec4}. Section \ref{sec5} concludes this paper and discusses some interesting topics for further research.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Image Processing, Physics, Convex Optimization, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1806.02279" label="1806.02279">
-        <attvalues>
-          <attvalue for="0" value="Deep Vessel Segmentation By Learning Graphical Connectivity" />
-          <attvalue for="1" value="  We propose a novel deep-learning-based system for vessel segmentation.&#10;Existing methods using CNNs have mostly relied on local appearances learned on&#10;the regular image grid, without considering the graphical structure of vessel&#10;shape. To address this, we incorporate a graph convolutional network into a&#10;unified CNN architecture, where the final segmentation is inferred by combining&#10;the different types of features. The proposed method can be applied to expand&#10;any type of CNN-based vessel segmentation method to enhance the performance.&#10;Experiments show that the proposed method outperforms the current&#10;state-of-the-art methods on two retinal image datasets as well as a coronary&#10;artery X-ray angiography dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.00254" label="1809.00254">
-        <attvalues>
-          <attvalue for="0" value="Parametric Shape Optimization using the Support Function" />
-          <attvalue for="1" value="  The optimization of shape functionals under convexity, diameter or constant&#10;width constraints shows numerical challenges. The support function can be used&#10;in order to approximate solutions to such problems by finite dimensional&#10;optimization problems under various constraints. We propose a numerical&#10;framework in dimensions two and three and we present applications from the&#10;field of convex geometry. We consider the optimization of functionals depending&#10;on the volume, perimeter and Dirichlet Laplace eigenvalues under the&#10;aforementioned constraints. In particular we confirm numerically Meissner's&#10;conjecture, regarding three dimensional bodies of constant width with minimal&#10;volume.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1509.02122" label="1509.02122">
-        <attvalues>
-          <attvalue for="0" value="Convexity Shape Constraints for Image Segmentation" />
-          <attvalue for="1" value="  Segmenting an image into multiple components is a central task in computer&#10;vision. In many practical scenarios, prior knowledge about plausible components&#10;is available. Incorporating such prior knowledge into models and algorithms for&#10;image segmentation is highly desirable, yet can be non-trivial. In this work,&#10;we introduce a new approach that allows, for the first time, to constrain some&#10;or all components of a segmentation to have convex shapes. Specifically, we&#10;extend the Minimum Cost Multicut Problem by a class of constraints that enforce&#10;convexity. To solve instances of this APX-hard integer linear program to&#10;optimality, we separate the proposed constraints in the branch-and-cut loop of&#10;a state-of-the-art ILP solver. Results on natural and biological images&#10;demonstrate the effectiveness of the approach as well as its advantage over the&#10;state-of-the-art heuristic.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.09600" label="2003.09600">
-        <attvalues>
-          <attvalue for="0" value="A level set representation method for N-dimensional convex shape and&#10;  applications" />
-          <attvalue for="1" value="  In this work, we present a new efficient method for convex shape&#10;representation, which is regardless of the dimension of the concerned objects,&#10;using level-set approaches. Convexity prior is very useful for object&#10;completion in computer vision. It is a very challenging task to design an&#10;efficient method for high dimensional convex objects representation. In this&#10;paper, we prove that the convexity of the considered object is equivalent to&#10;the convexity of the associated signed distance function. Then, the second&#10;order condition of convex functions is used to characterize the shape convexity&#10;equivalently. We apply this new method to two applications: object segmentation&#10;with convexity prior and convex hull problem (especially with outliers). For&#10;both applications, the involved problems can be written as a general&#10;optimization problem with three constraints. Efficient algorithm based on&#10;alternating direction method of multipliers is presented for the optimization&#10;problem. Numerical experiments are conducted to verify the effectiveness and&#10;efficiency of the proposed representation method and algorithm.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.08941" label="2009.08941">
-        <attvalues>
-          <attvalue for="0" value="Light Direction and Color Estimation from Single Image with Deep&#10;  Regression" />
-          <attvalue for="1" value="  We present a method to estimate the direction and color of the scene light&#10;source from a single image. Our method is based on two main ideas: (a) we use a&#10;new synthetic dataset with strong shadow effects with similar constraints to&#10;the SID dataset; (b) we define a deep architecture trained on the mentioned&#10;dataset to estimate the direction and color of the scene light source. Apart&#10;from showing good performance on synthetic images, we additionally propose a&#10;preliminary procedure to obtain light positions of the Multi-Illumination&#10;dataset, and, in this way, we also prove that our trained model achieves good&#10;performance when it is applied to real scenes.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Scene appearance is directly dependent on the light source properties, such as the spectral composition of emitted light, and the position and direction of the light source, whose interaction with the scene objects provoke shaded surfaces or dark cast shadows that become essential visual cues to understand image content. Esti\-ma\-ting the properties of the light conditions from a single image is an initial step to improve subsequent computer vision algorithms for image understanding. In this paper we perform a preliminary study to estimate color and position of light in a simple and unified approach, that is based on the shading properties of the image where we assume a single scene illuminant.&#10;&#10;Estimating the color of the light from a single image has been focus of attention in previous research. Computational color constancy (CC) has been studied in a large number of works \cite{FOSTER2011Color,Finlayson2006, gijsenij2011computational} where the problem was tackled from different points of views \cite{gijsenij2011computational}. A first approach was to extract statistics from RGB image values under different assumptions to estimate the canonical white. A second approach was to introduce spatio-statistical information like gradients or frequency content of the image. One last group of CC algorithms was to try to get the information from physical cues of the image (highlights, shadows, inter-reflections, etc). In the last years, new approaches have been based on deep learning frameworks where the solution is driven by the data with physical constrains in the loss functions. An updated comprehensive compendium and comparison of CC algorithm performances can be found in \cite{cheng2014illuminant,Lou2015DeepColor,das2018color,Yan2018MultipleIE}. In this work we propose one more approach based on a deep architecture, but color of the light source is jointly estimated with the light direction. &#10;&#10;Estimating the direction of the light has also been tackled from different areas like, computer graphics, computer vision or augmented reality. Single image light direction estimation can be divided in two different kinds of approaches. First, those in which light probes with known reflectance and geometric properties are used. A Specular sphere is commonly used to represent light position in different computer graphics applications \cite{Debevec1998RenderingSO,Agusanto2003PhotorealisticRF,Schnieders2011LightSE}. But, random shaped objects to detect light position were used by Mandl et-al in \cite{Mandl2017LearningLF}, jointly with a deep learning approach to get the light position with each of these random shaped object. Second, we find those works in which no probe is used, and where multiple image cues such as shading, shape and cast shadows are the basis to estimate light direction. Some examples of these works can be found in computer vision literature \cite{Lalonde2009EstimatingNI,samaras2003incorporating,Panagopoulos2011IlluminationEA,panagopoulos2009robust,arief2012realtime}.&#10;&#10;More recently, some deep learning methods have been proposed to estimate scene illumination and have been used for different computer graphics tasks. Gardner et al.\cite{gardner2017learning} introduced a method to convert low dynamic range (LDR) indoor images to high dynamic range (HDR) images, first they used a deep network to localize the light source in LDR image environmental map and then they used another network with these annoted LDR images to convert them to HDR image. Following a similar approach, Geoffroy et al. \cite{hold2017deep} introduced a method to convert outdoor LDR images to HDR images. They trained their network with a set of panorama images and predicted HDR environmental maps with sky and sun positions. Later on, Geoffroy et al \cite{gardner2019deep} extended their previous idea for indoor lighting but replaced the environmental maps with light geometric and photometric properties. Sun et al. \cite{Sun2019SingleIP} introduced an encoder-decoder based network to relight a single portrait image, the encoder predicts the input image environmental map and an embedding for the scene information, while the decoder builds a new version of the image scene with the target environmental map and obtains a relighted portrait. Very recently, Muramann et al. \cite{murmann2019dataset} have introduced the Multi-illuminant dataset of $1000$ scenes each one acquired under $25$ different light position conditions, and they used a deep network to predict a right sided illuminated image from its corresponding left sided illuminated image. In this work we will test our proposal on this new wild dataset after providing a procedure to compute the light direction from each sample.&#10;&#10;To sum up, we can state that a large range of works have tackled the problem of estimating color and direction of the scene light source from different points of views and focusing on specific applications. In this work we propose an easy end-to-end approach to jointly characterize the light source of a scene, both for color and direction. We pursuit to measure the level of accuracy we can achieve, in order it can be applied to a wide range of images, without using probes in the scene and becoming a robust preliminary stage to be subsequently combined with any task. To this end, the paper is organized as follows: first we introduce a new synthetic dataset, secondly we use it to train a deep architecture that estimates light properties from a single image, finally we show how our proposal performs on three different scenarios, synthetic, real indoor and natural images.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Computer Vision Applications, Mathematics, Physics, Light Source Estimation, Computer Science, Deep Learning Architecture, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="1812.03085" label="1812.03085">
-        <attvalues>
-          <attvalue for="0" value="Color Constancy by GANs: An Experimental Survey" />
-          <attvalue for="1" value="  In this paper, we formulate the color constancy task as an image-to-image&#10;translation problem using GANs. By conducting a large set of experiments on&#10;different datasets, an experimental survey is provided on the use of different&#10;types of GANs to solve for color constancy i.e. CC-GANs (Color Constancy GANs).&#10;Based on the experimental review, recommendations are given for the design of&#10;CC-GAN architectures based on different criteria, circumstances and datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.00090" label="1704.00090">
-        <attvalues>
-          <attvalue for="0" value="Learning to Predict Indoor Illumination from a Single Image" />
-          <attvalue for="1" value="  We propose an automatic method to infer high dynamic range illumination from&#10;a single, limited field-of-view, low dynamic range photograph of an indoor&#10;scene. In contrast to previous work that relies on specialized image capture,&#10;user input, and/or simple scene models, we train an end-to-end deep neural&#10;network that directly regresses a limited field-of-view photo to HDR&#10;illumination, without strong assumptions on scene geometry, material&#10;properties, or lighting. We show that this can be accomplished in a three step&#10;process: 1) we train a robust lighting classifier to automatically annotate the&#10;location of light sources in a large dataset of LDR environment maps, 2) we use&#10;these annotations to train a deep neural network that predicts the location of&#10;lights in a scene from a single limited field-of-view photo, and 3) we&#10;fine-tune this network using a small dataset of HDR environment maps to predict&#10;light intensities. This allows us to automatically recover high-quality HDR&#10;illumination estimates that significantly outperform previous state-of-the-art&#10;methods. Consequently, using our illumination estimates for applications like&#10;3D object insertion, we can achieve results that are photo-realistic, which is&#10;validated via a perceptual user study.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.06403" label="1611.06403">
-        <attvalues>
-          <attvalue for="0" value="Deep Outdoor Illumination Estimation" />
-          <attvalue for="1" value="  We present a CNN-based technique to estimate high-dynamic range outdoor&#10;illumination from a single low dynamic range image. To train the CNN, we&#10;leverage a large dataset of outdoor panoramas. We fit a low-dimensional&#10;physically-based outdoor illumination model to the skies in these panoramas&#10;giving us a compact set of parameters (including sun position, atmospheric&#10;conditions, and camera parameters). We extract limited field-of-view images&#10;from the panoramas, and train a CNN with this large set of input image--output&#10;lighting parameter pairs. Given a test image, this network can be used to infer&#10;illumination parameters that can, in turn, be used to reconstruct an outdoor&#10;illumination environment map. We demonstrate that our approach allows the&#10;recovery of plausible illumination conditions and enables photorealistic&#10;virtual object insertion from a single image. An extensive evaluation on both&#10;the panorama dataset and captured HDR environment maps shows that our technique&#10;significantly outperforms previous solutions to this problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.08812" label="1910.08812">
-        <attvalues>
-          <attvalue for="0" value="Deep Parametric Indoor Lighting Estimation" />
-          <attvalue for="1" value="  We present a method to estimate lighting from a single image of an indoor&#10;scene. Previous work has used an environment map representation that does not&#10;account for the localized nature of indoor lighting. Instead, we represent&#10;lighting as a set of discrete 3D lights with geometric and photometric&#10;parameters. We train a deep neural network to regress these parameters from a&#10;single image, on a dataset of environment maps annotated with depth. We propose&#10;a differentiable layer to convert these parameters to an environment map to&#10;compute our loss; this bypasses the challenge of establishing correspondences&#10;between estimated and ground truth lights. We demonstrate, via quantitative and&#10;qualitative evaluations, that our representation and training scheme lead to&#10;more accurate results compared to previous work, while allowing for more&#10;realistic 3D object compositing with spatially-varying lighting.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2302.07245" label="2302.07245">
-        <attvalues>
-          <attvalue for="0" value="WSD: Wild Selfie Dataset for Face Recognition in Selfie Images" />
-          <attvalue for="1" value="  With the rise of handy smart phones in the recent years, the trend of&#10;capturing selfie images is observed. Hence efficient approaches are required to&#10;be developed for recognising faces in selfie images. Due to the short distance&#10;between the camera and face in selfie images, and the different visual effects&#10;offered by the selfie apps, face recognition becomes more challenging with&#10;existing approaches. A dataset is needed to be developed to encourage the study&#10;to recognize faces in selfie images. In order to alleviate this problem and to&#10;facilitate the research on selfie face images, we develop a challenging Wild&#10;Selfie Dataset (WSD) where the images are captured from the selfie cameras of&#10;different smart phones, unlike existing datasets where most of the images are&#10;captured in controlled environment. The WSD dataset contains 45,424 images from&#10;42 individuals (i.e., 24 female and 18 male subjects), which are divided into&#10;40,862 training and 4,562 test images. The average number of images per subject&#10;is 1,082 with minimum and maximum number of images for any subject are 518 and&#10;2,634, respectively. The proposed dataset consists of several challenges,&#10;including but not limited to augmented reality filtering, mirrored images,&#10;occlusion, illumination, scale, expressions, view-point, aspect ratio, blur,&#10;partial faces, rotation, and alignment. We compare the proposed dataset with&#10;existing benchmark datasets in terms of different characteristics. The&#10;complexity of WSD dataset is also observed experimentally, where the&#10;performance of the existing state-of-the-art face recognition methods is poor&#10;on WSD dataset, compared to the existing datasets. Hence, the proposed WSD&#10;dataset opens up new challenges in the area of face recognition and can be&#10;beneficial to the community to study the specific challenges related to selfie&#10;images and develop improved methods for face recognition in selfie images.&#10;" />
-          <attvalue for="2" value="&#10;Face recognition has been a challenging problem for computer vision researchers during the past few years \cite{zhao2003face}, \cite{abate20072d}. Earlier hand-crafted feature representation and classical machine learning based methods were utilized for face recognition and analysis \cite{saber1998frontal}, \cite{wang2002facial}, \cite{deniz2003face}, \cite{ahonen2004face}, \cite{he2005face}, \cite{deniz2011face}, \cite{ldop}, \cite{chakraborty2018centre}, \cite{fdlbp}. Such hand-crafted feature-based methods often used to be influenced by the visual characteristics of the image, such as skin, texture, shape and facial parts features. These methods relied on traditional machine learning tools such as K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Decision Tree, Random Forest, etc. for recognition.&#10;&#10;After 2010, deep neural networks (DNNs) have emerged as a powerful learning mechanism in solving various tasks in computer vision, including face recognition. DNNs can learn the important features automatically from the image data in a hierarchical fashion at different levels of abstractness \cite{lecun2015deep}.&#10;&#10;The deep learning mechanism has shown a huge improvement in 2012 for image recognition using convolutional neural networks (CNNs) \cite{alexnet}. Since then several variants of CNNs have been proposed for different applications of computer vision and image processing \cite{vgg}, \cite{resnet}, \cite{fasterrcnn}, \cite{imageretrieval}, \cite{feng2022iris}. The key success of deep learning models was observed due to the availability of powerful computational machines, such as Graphical Processing Units (GPUs) which can perform heavy parallel computation, and availability of large-scale datasets \cite{imagenet}.&#10;&#10;The recent advancements in face recognition research, with the help of deep learning methods, can be observed from the outstanding performance of deep learning models \cite{srivastava2019performance}, such as VGGFace \cite{vggface}, \cite{vggface2} and FaceNet \cite{facenet}. So far, several deep learning models have been developed by the researchers for face recognition by introducing different CNNs, different objective functions, and different other strategies \cite{deepface}, \cite{sphereface}, \cite{arcface}, \cite{hmloss}, \cite{mishra2021multiscale}. A detailed discussion on the advancements of deep learning based face recogntion models can be found in a recent survey published by Wang and Deng in \cite{wang2021deep}.&#10;&#10;The recent progress in deep learning based face recognition technology is heavily governed by the large-scale face datasets. Hence, several benchmark face datasets have been developed to drive the research and development activities in face recognition. The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}. We present a detailed comparison of the different face datasets in terms of the size, annotations and different characteristics in Table \ref{tab:challengeComparison} and Table \ref{tab:datasetComparison}. Some other face datasets include multi-face dataset \cite{dubey2018multi}, YouTube Face Dataset \cite{wolf2011face}, CelebFaces Attributes Dataset (CelebA) \cite{liu2015faceattributes}, etc. These existing datasets were generally developed by mining the images of celebrities from the public domain. The major focus of these datasets is on more and more samples. Hence, some of these datasets are also noisy and does not contain the scenarios where users take their pictures by themselves using their smart phones, i.e., selfie images.&#10;&#10;Some reasearchers have tried to come across the datasets which include the selfie images, such as Selfie Dataset by Kalayeh et al. \cite{kalayeh2015take}, DocFace \cite{shi2018docface} \&amp; DocFace+ \cite{shi2019docface+} by Shi and Jan, and SelfieCity. The images in Selfie Dataset \cite{kalayeh2015take} is collected from Instagram and annotated with different attributes. Hence the Selfie Dataset \cite{kalayeh2015take} is used for analyzing ``How to Take a Good Selfie&quot; instead of face recognition. However, we aim to develop a wild selfie dataset, which poses a great challenge by avoiding any environmental constrains while taking the selfies. Further, the proposed dataset should be useful for face recognition in selfie images. The DocFace performs the matching of ID photos with live face photos/selfies \cite{shi2018docface}. The DocFace+ supports multiple ID/selfie per class for ID-Selfie matching \cite{shi2019docface+}.&#10;The SelfieCity dataset is collected from Instagram and labelled based on the Cities. This dataset is also not available publicly for the research purposes. &#10;&#10;The existing selfie datasets are either not available publicly or not suitable for face recognition in selfie images. Thus, through this paper, we propose a selfie dataset for face recognition. Following are the contributions of this paper:&#10;\begin{enumerate}&#10; \item A wild selfie dataset (WSD) containing more than 45K images from 42 individuals is proposed to facilitate face recognition in selfie images captured using smartphone cameras.&#10; \item The proposed dataset poses a great challenge in terms of severe variability in image characteristics, such as aspect ratio, partial faces, occlusion, illumination, scale, expression, viewpoint, blur, reflection and augmented reality.&#10; \item We test the performance of existing deep learning based face recognition models to show the complexity of the proposed WSD dataset.&#10;\end{enumerate}&#10;&#10;The rest of the paper is presented by detailing the proposed dataset in Section 2, experimental setup in Section 3, results in Section 4, and conclusion in Section 5.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Face Recognition, Information Science, Computer Vision, Engineering, Image Datasets, Artificial Intelligence, Human-Computer Interaction, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1803.07441" label="1803.07441">
-        <attvalues>
-          <attvalue for="0" value="LDOP: Local Directional Order Pattern for Robust Face Retrieval" />
-          <attvalue for="1" value="  The local descriptors have gained wide range of attention due to their&#10;enhanced discriminative abilities. It has been proved that the consideration of&#10;multi-scale local neighborhood improves the performance of the descriptor,&#10;though at the cost of increased dimension. This paper proposes a novel method&#10;to construct a local descriptor using multi-scale neighborhood by finding the&#10;local directional order among the intensity values at different scales in a&#10;particular direction. Local directional order is the multi-radius relationship&#10;factor in a particular direction. The proposed local directional order pattern&#10;(LDOP) for a particular pixel is computed by finding the relationship between&#10;the center pixel and local directional order indexes. It is required to&#10;transform the center value into the range of neighboring orders. Finally, the&#10;histogram of LDOP is computed over whole image to construct the descriptor. In&#10;contrast to the state-of-the-art descriptors, the dimension of the proposed&#10;descriptor does not depend upon the number of neighbors involved to compute the&#10;order; it only depends upon the number of directions. The introduced descriptor&#10;is evaluated over the image retrieval framework and compared with the&#10;state-of-the-art descriptors over challenging face databases such as PaSC, LFW,&#10;PubFig, FERET, AR, AT&amp;T, and ExtendedYale. The experimental results confirm the&#10;superiority and robustness of the LDOP descriptor.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.00511" label="2201.00511">
-        <attvalues>
-          <attvalue for="0" value="Centre Symmetric Quadruple Pattern: A Novel Descriptor for Facial Image&#10;  Recognition and Retrieval" />
-          <attvalue for="1" value="  Facial features are defined as the local relationships that exist amongst the&#10;pixels of a facial image. Hand-crafted descriptors identify the relationships&#10;of the pixels in the local neighbourhood defined by the kernel. Kernel is a two&#10;dimensional matrix which is moved across the facial image. Distinctive&#10;information captured by the kernel with limited number of pixel achieves&#10;satisfactory recognition and retrieval accuracies on facial images taken under&#10;constrained environment (controlled variations in light, pose, expressions, and&#10;background). To achieve similar accuracies under unconstrained environment&#10;local neighbourhood has to be increased, in order to encode more pixels.&#10;Increasing local neighbourhood also increases the feature length of the&#10;descriptor. In this paper we propose a hand-crafted descriptor namely Centre&#10;Symmetric Quadruple Pattern (CSQP), which is structurally symmetric and encodes&#10;the facial asymmetry in quadruple space. The proposed descriptor efficiently&#10;encodes larger neighbourhood with optimal number of binary bits. It has been&#10;shown using average entropy, computed over feature images encoded with the&#10;proposed descriptor, that the CSQP captures more meaningful information as&#10;compared to state of the art descriptors. The retrieval and recognition&#10;accuracies of the proposed descriptor has been compared with state of the art&#10;hand-crafted descriptors (CSLBP, CSLTP, LDP, LBP, SLBP and LDGP) on bench mark&#10;databases namely; LFW, Colour-FERET, and CASIA-face-v5. Result analysis shows&#10;that the proposed descriptor performs well under controlled as well as&#10;uncontrolled variations in pose, illumination, background and expressions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.06508" label="1709.06508">
-        <attvalues>
-          <attvalue for="0" value="Face Retrieval using Frequency Decoded Local Descriptor" />
-          <attvalue for="1" value="  The local descriptors have been the backbone of most of the computer vision&#10;problems. Most of the existing local descriptors are generated over the raw&#10;input images. In order to increase the discriminative power of the local&#10;descriptors, some researchers converted the raw image into multiple images with&#10;the help of some high and low pass frequency filters, then the local&#10;descriptors are computed over each filtered image and finally concatenated into&#10;a single descriptor. By doing so, these approaches do not utilize the inter&#10;frequency relationship which causes the less improvement in the discriminative&#10;power of the descriptor that could be achieved. In this paper, this problem is&#10;solved by utilizing the decoder concept of multi-channel decoded local binary&#10;pattern over the multi-frequency patterns. A frequency decoded local binary&#10;pattern (FDLBP) is proposed with two decoders. Each decoder works with one low&#10;frequency pattern and two high frequency patterns. Finally, the descriptors&#10;from both decoders are concatenated to form the single descriptor. The face&#10;retrieval experiments are conducted over four benchmarks and challenging&#10;databases such as PaSC, LFW, PubFig, and ESSEX. The experimental results&#10;confirm the superiority of the FDLBP descriptor as compared to the&#10;state-of-the-art descriptors such as LBP, SOBEL_LBP, BoF_LBP, SVD_S_LBP, mdLBP,&#10;etc.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.00641" label="2012.00641">
-        <attvalues>
-          <attvalue for="0" value="A Decade Survey of Content Based Image Retrieval using Deep Learning" />
-          <attvalue for="1" value="  The content based image retrieval aims to find the similar images from a&#10;large scale dataset against a query image. Generally, the similarity between&#10;the representative features of the query image and dataset images is used to&#10;rank the images for retrieval. In early days, various hand designed feature&#10;descriptors have been investigated based on the visual cues such as color,&#10;texture, shape, etc. that represent the images. However, the deep learning has&#10;emerged as a dominating alternative of hand-designed feature engineering from a&#10;decade. It learns the features automatically from the data. This paper presents&#10;a comprehensive survey of deep learning based developments in the past decade&#10;for content based image retrieval. The categorization of existing&#10;state-of-the-art methods from different perspectives is also performed for&#10;greater understanding of the progress. The taxonomy used in this survey covers&#10;different supervision, different networks, different descriptor type and&#10;different retrieval type. A performance analysis is also performed using the&#10;state-of-the-art methods. The insights are also presented for the benefit of&#10;the researchers to observe the progress and to make the best choices. The&#10;survey presented in this paper will help in further research progress in image&#10;retrieval using deep learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.08092" label="1710.08092">
-        <attvalues>
-          <attvalue for="0" value="VGGFace2: A dataset for recognising faces across pose and age" />
-          <attvalue for="1" value="  In this paper, we introduce a new large-scale face dataset named VGGFace2.&#10;The dataset contains 3.31 million images of 9131 subjects, with an average of&#10;362.6 images for each subject. Images are downloaded from Google Image Search&#10;and have large variations in pose, age, illumination, ethnicity and profession&#10;(e.g. actors, athletes, politicians). The dataset was collected with three&#10;goals in mind: (i) to have both a large number of identities and also a large&#10;number of images for each identity; (ii) to cover a large range of pose, age&#10;and ethnicity; and (iii) to minimize the label noise. We describe how the&#10;dataset was collected, in particular the automated and manual filtering stages&#10;to ensure a high accuracy for the images of each identity. To assess face&#10;recognition performance using the new dataset, we train ResNet-50 (with and&#10;without Squeeze-and-Excitation blocks) Convolutional Neural Networks on&#10;VGGFace2, on MS- Celeb-1M, and on their union, and show that training on&#10;VGGFace2 leads to improved recognition performance over pose and age. Finally,&#10;using the models trained on these datasets, we demonstrate state-of-the-art&#10;performance on all the IARPA Janus face recognition benchmarks, e.g. IJB-A,&#10;IJB-B and IJB-C, exceeding the previous state-of-the-art by a large margin.&#10;Datasets and models are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.08063" label="1704.08063">
-        <attvalues>
-          <attvalue for="0" value="SphereFace: Deep Hypersphere Embedding for Face Recognition" />
-          <attvalue for="1" value="  This paper addresses deep face recognition (FR) problem under open-set&#10;protocol, where ideal face features are expected to have smaller maximal&#10;intra-class distance than minimal inter-class distance under a suitably chosen&#10;metric space. However, few existing algorithms can effectively achieve this&#10;criterion. To this end, we propose the angular softmax (A-Softmax) loss that&#10;enables convolutional neural networks (CNNs) to learn angularly discriminative&#10;features. Geometrically, A-Softmax loss can be viewed as imposing&#10;discriminative constraints on a hypersphere manifold, which intrinsically&#10;matches the prior that faces also lie on a manifold. Moreover, the size of&#10;angular margin can be quantitatively adjusted by a parameter $m$. We further&#10;derive specific $m$ to approximate the ideal feature criterion. Extensive&#10;analysis and experiments on Labeled Face in the Wild (LFW), Youtube Faces (YTF)&#10;and MegaFace Challenge show the superiority of A-Softmax loss in FR tasks. The&#10;code has also been made publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09747" label="1908.09747">
-        <attvalues>
-          <attvalue for="0" value="Hard-Mining Loss based Convolutional Neural Network for Face Recognition" />
-          <attvalue for="1" value="  Face Recognition is one of the prominent problems in the computer vision&#10;domain. Witnessing advances in deep learning, significant work has been&#10;observed in face recognition, which touched upon various parts of the&#10;recognition framework like Convolutional Neural Network (CNN), Layers, Loss&#10;functions, etc. Various loss functions such as Cross-Entropy, Angular-Softmax&#10;and ArcFace have been introduced to learn the weights of network for face&#10;recognition. However, these loss functions do not give high priority to the&#10;hard samples as compared to the easy samples. Moreover, their learning process&#10;is biased due to a number of easy examples compared to hard examples. In this&#10;paper, we address this issue by considering hard examples with more priority.&#10;In order to do so, We propose a Hard-Mining loss by increasing the loss for&#10;harder examples and decreasing the loss for easy examples. The proposed concept&#10;is generic and can be used with any existing loss function. We test the&#10;Hard-Mining loss with different losses such as Cross-Entropy, Angular-Softmax&#10;and ArcFace. The proposed Hard-Mining loss is tested over widely used Labeled&#10;Faces in the Wild (LFW) and YouTube Faces (YTF) datasets. The training is&#10;performed over CASIA-WebFace and MS-Celeb-1M datasets. We use the residual&#10;network (i.e., ResNet18) for the experimental analysis. The experimental&#10;results suggest that the performance of existing loss functions is boosted when&#10;used in the framework of the proposed Hard-Mining loss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.06655" label="1804.06655">
-        <attvalues>
-          <attvalue for="0" value="Deep Face Recognition: A Survey" />
-          <attvalue for="1" value="  Deep learning applies multiple processing layers to learn representations of&#10;data with multiple levels of feature extraction. This emerging technique has&#10;reshaped the research landscape of face recognition (FR) since 2014, launched&#10;by the breakthroughs of DeepFace and DeepID. Since then, deep learning&#10;technique, characterized by the hierarchical architecture to stitch together&#10;pixels into invariant face representation, has dramatically improved the&#10;state-of-the-art performance and fostered successful real-world applications.&#10;In this survey, we provide a comprehensive review of the recent developments on&#10;deep FR, covering broad topics on algorithm designs, databases, protocols, and&#10;application scenes. First, we summarize different network architectures and&#10;loss functions proposed in the rapid evolution of the deep FR methods. Second,&#10;the related face processing methods are categorized into two classes:&#10;&quot;one-to-many augmentation&quot; and &quot;many-to-one normalization&quot;. Then, we summarize&#10;and compare the commonly used databases for both model training and evaluation.&#10;Third, we review miscellaneous scenes in deep FR, such as cross-factor,&#10;heterogenous, multiple-media and industrial scenes. Finally, the technical&#10;challenges and several promising directions are highlighted.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.7923" label="1411.7923">
-        <attvalues>
-          <attvalue for="0" value="Learning Face Representation from Scratch" />
-          <attvalue for="1" value="  Pushing by big data and deep convolutional neural network (CNN), the&#10;performance of face recognition is becoming comparable to human. Using private&#10;large scale training datasets, several groups achieve very high performance on&#10;LFW, i.e., 97% to 99%. While there are many open source implementations of CNN,&#10;none of large scale face dataset is publicly available. The current situation&#10;in the field of face recognition is that data is more important than algorithm.&#10;To solve this problem, this paper proposes a semi-automatical way to collect&#10;face images from Internet and builds a large scale dataset containing about&#10;10,000 subjects and 500,000 images, called CASIAWebFace. Based on the database,&#10;we use a 11-layer CNN to learn discriminative representation and obtain&#10;state-of-theart accuracy on LFW and YTF. The publication of CASIAWebFace will&#10;attract more research groups entering this field and accelerate the development&#10;of face recognition in the wild.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.01484" label="1611.01484">
-        <attvalues>
-          <attvalue for="0" value="UMDFaces: An Annotated Face Dataset for Training Deep Networks" />
-          <attvalue for="1" value="  Recent progress in face detection (including keypoint detection), and&#10;recognition is mainly being driven by (i) deeper convolutional neural network&#10;architectures, and (ii) larger datasets. However, most of the large datasets&#10;are maintained by private companies and are not publicly available. The&#10;academic computer vision community needs larger and more varied datasets to&#10;make further progress.&#10;  In this paper we introduce a new face dataset, called UMDFaces, which has&#10;367,888 annotated faces of 8,277 subjects. We also introduce a new face&#10;recognition evaluation protocol which will help advance the state-of-the-art in&#10;this area. We discuss how a large dataset can be collected and annotated using&#10;human annotators and deep networks. We provide human curated bounding boxes for&#10;faces. We also provide estimated pose (roll, pitch and yaw), locations of&#10;twenty-one key-points and gender information generated by a pre-trained neural&#10;network. In addition, the quality of keypoint annotations has been verified by&#10;humans for about 115,000 images. Finally, we compare the quality of the dataset&#10;with other publicly available face datasets at similar scales.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1607.08221" label="1607.08221">
-        <attvalues>
-          <attvalue for="0" value="MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition" />
-          <attvalue for="1" value="  In this paper, we design a benchmark task and provide the associated datasets&#10;for recognizing face images and link them to corresponding entity keys in a&#10;knowledge base. More specifically, we propose a benchmark task to recognize one&#10;million celebrities from their face images, by using all the possibly collected&#10;face images of this individual on the web as training data. The rich&#10;information provided by the knowledge base helps to conduct disambiguation and&#10;improve the recognition accuracy, and contributes to various real-world&#10;applications, such as image captioning and news video analysis. Associated with&#10;this task, we design and provide concrete measurement set, evaluation protocol,&#10;as well as training data. We also present in details our experiment setup and&#10;report promising baseline results. Our benchmark task could lead to one of the&#10;largest classification problems in computer vision. To the best of our&#10;knowledge, our training dataset, which contains 10M images in version 1, is the&#10;largest publicly available one in the world.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.01898" label="1810.01898">
-        <attvalues>
-          <attvalue for="0" value="A Multi-Face Challenging Dataset for Robust Face Recognition" />
-          <attvalue for="1" value="  Face recognition in images is an active area of interest among the computer&#10;vision researchers. However, recognizing human face in an unconstrained&#10;environment, is a relatively less-explored area of research. Multiple face&#10;recognition in unconstrained environment is a challenging task, due to the&#10;variation of view-point, scale, pose, illumination and expression of the face&#10;images. Partial occlusion of faces makes the recognition task even more&#10;challenging. The contribution of this paper is two-folds: introducing a&#10;challenging multiface dataset (i.e., IIITS MFace Dataset) for face recognition&#10;in unconstrained environment and evaluating the performance of state-of-the-art&#10;hand-designed and deep learning based face descriptors on the dataset. The&#10;proposed IIITS MFace dataset contains faces with challenges like pose&#10;variation, occlusion, mask, spectacle, expressions, change of illumination,&#10;etc. We experiment with several state-of-the-art face descriptors, including&#10;recent deep learning based face descriptors like VGGFace, and compare with the&#10;existing benchmark face datasets. Results of the experiments clearly show that&#10;the difficulty level of the proposed dataset is much higher compared to the&#10;benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.7766" label="1411.7766">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning Face Attributes in the Wild" />
-          <attvalue for="1" value="  Predicting face attributes in the wild is challenging due to complex face&#10;variations. We propose a novel deep learning framework for attribute prediction&#10;in the wild. It cascades two CNNs, LNet and ANet, which are fine-tuned jointly&#10;with attribute tags, but pre-trained differently. LNet is pre-trained by&#10;massive general object categories for face localization, while ANet is&#10;pre-trained by massive face identities for attribute prediction. This framework&#10;not only outperforms the state-of-the-art with a large margin, but also reveals&#10;valuable facts on learning face representation.&#10;  (1) It shows how the performances of face localization (LNet) and attribute&#10;prediction (ANet) can be improved by different pre-training strategies.&#10;  (2) It reveals that although the filters of LNet are fine-tuned only with&#10;image-level attribute tags, their response maps over entire images have strong&#10;indication of face locations. This fact enables training LNet for face&#10;localization with only image-level annotations, but without face bounding boxes&#10;or landmarks, which are required by all attribute recognition works.&#10;  (3) It also demonstrates that the high-level hidden neurons of ANet&#10;automatically discover semantic concepts after pre-training with massive face&#10;identities, and such concepts are significantly enriched after fine-tuning with&#10;attribute tags. Each attribute can be well explained with a sparse linear&#10;combination of these concepts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.02283" label="1805.02283">
-        <attvalues>
-          <attvalue for="0" value="DocFace: Matching ID Document Photos to Selfies" />
-          <attvalue for="1" value="  Numerous activities in our daily life, including transactions, access to&#10;services and transportation, require us to verify who we are by showing our ID&#10;documents containing face images, e.g. passports and driver licenses. An&#10;automatic system for matching ID document photos to live face images in real&#10;time with high accuracy would speedup the verification process and remove the&#10;burden on human operators. In this paper, by employing the transfer learning&#10;technique, we propose a new method, DocFace, to train a domain-specific network&#10;for ID document photo matching without a large dataset. Compared with the&#10;baseline of applying existing methods for general face recognition to this&#10;problem, our method achieves considerable improvement. A cross validation on an&#10;ID-Selfie dataset shows that DocFace improves the TAR from 61.14% to 92.77% at&#10;FAR=0.1%. Experimental results also indicate that given more training data, a&#10;viable system for automatic ID document photo matching can be developed and&#10;deployed.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.05620" label="1809.05620">
-        <attvalues>
-          <attvalue for="0" value="DocFace+: ID Document to Selfie Matching" />
-          <attvalue for="1" value="  Numerous activities in our daily life require us to verify who we are by&#10;showing our ID documents containing face images, such as passports and driver&#10;licenses, to human operators. However, this process is slow, labor intensive&#10;and unreliable. As such, an automated system for matching ID document photos to&#10;live face images (selfies) in real time and with high accuracy is required. In&#10;this paper, we propose DocFace+ to meet this objective. We first show that&#10;gradient-based optimization methods converge slowly (due to the underfitting of&#10;classifier weights) when many classes have very few samples, a characteristic&#10;of existing ID-selfie datasets. To overcome this shortcoming, we propose a&#10;method, called dynamic weight imprinting (DWI), to update the classifier&#10;weights, which allows faster convergence and more generalizable&#10;representations. Next, a pair of sibling networks with partially shared&#10;parameters are trained to learn a unified face representation with&#10;domain-specific parameters. Cross-validation on an ID-selfie dataset shows that&#10;while a publicly available general face matcher (SphereFace) only achieves a&#10;True Accept Rate (TAR) of 59.29+-1.55% at a False Accept Rate (FAR) of 0.1% on&#10;the problem, DocFace+ improves the TAR to 97.51+-0.40%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.16697" label="2211.16697">
-        <attvalues>
-          <attvalue for="0" value="SGDraw: Scene Graph Drawing Interface Using Object-Oriented&#10;  Representation" />
-          <attvalue for="1" value="  Scene understanding is an essential and challenging task in computer vision.&#10;To provide the visually fundamental graphical structure of an image, the scene&#10;graph has received increased attention due to its powerful semantic&#10;representation. However, it is difficult to draw a proper scene graph for image&#10;retrieval, image generation, and multi-modal applications. The conventional&#10;scene graph annotation interface is not easy to use in image annotations, and&#10;the automatic scene graph generation approaches using deep neural networks are&#10;prone to generate redundant content while disregarding details. In this work,&#10;we propose SGDraw, a scene graph drawing interface using object-oriented scene&#10;graph representation to help users draw and edit scene graphs interactively.&#10;For the proposed object-oriented representation, we consider the objects,&#10;attributes, and relationships of objects as a structural unit. SGDraw provides&#10;a web-based scene graph annotation and generation tool for scene understanding&#10;applications. To verify the effectiveness of the proposed interface, we&#10;conducted a comparison study with the conventional tool and the user experience&#10;study. The results show that SGDraw can help generate scene graphs with richer&#10;details and describe the images more accurately than traditional bounding box&#10;annotations. We believe the proposed SGDraw can be useful in various vision&#10;tasks, such as image retrieval and generation.&#10;" />
-          <attvalue for="2" value="&#10;Scene graph is a common and popular way to describe scene understanding, first proposed for image retrieval tasks to search for images with similar descriptions in image datasets~\cite{johnson2015image}. In addition, scene graphs are used for a wide range of vision applications, such as image retrieval~\cite{johnson2015image,qi2017online}, image captioning~\cite{chen2020say}, visual reasoning~\cite{shi2019explainable}, visual answering~\cite{hildebrandt2020scene}, and robotics~\cite{amiri2022reasoning}. The conventional approaches for scene graph creation have been mainly based on a manually annotated image detection dataset, and the annotation tasks are usually time-consuming and laborious. The visual genome dataset~\cite{krishna2017visual} is the most common and popular dataset for scene graphs, with around $10^5$ annotated images. However, the annotated scene graphs have challenging issues, such as sparse image annotation, repeated labels, and unreasonable annotation. In addition, these scene graphs are difficult to be modified and produced in conventional computer vision tasks. &#10;&#10;To solve these issues, one possible solution is the automatic generation of scene graph generation with deep learning approaches. The task of scene graph generation is to generate a corresponding graph-structure representation from an image, and abstract the objects and object relationships from the images. The automatic generation approaches can only generate limited scene graphs on commonly used object categories and predicates, and have difficulty for rare and previously unseen compositions~\cite{knyazev2021generative,knyazev2020graph}. In addition, these automatic approaches can complete the generation of objects and relationships but lack the description of attributes, which makes the generated scene graph incomplete. Therefore, we aim to provide a novel scene graph drawing interface with users in the loop.&#10;&#10;In this work, we propose an interactive drawing user interface, SGDraw for creating scene graphs using object-oriented representation, as shown in Figure \ref{fig:framework}. The proposed system can achieve more comprehensive and detailed results than the conventional annotation interfaces. SGDraw adopts object-oriented representation for scene graph annotation by considering objects, object relationships, and object attributes. SGDraw can take advantage of the user's cognitive ability in the drawing process from target images. We verified the proposed interface in the image dataset and conducted a comparison study between the conventional annotation method using bounding boxes and our proposed approach. The evaluation results demonstrated the superiority of the proposed approach in drawing comprehensive and detailed scene graphs.&#10;&#10;We list our main contributions as follows:&#10;&#10;\begin{itemize}&#10; \item We propose an object-oriented annotation approach of scene graphs, which adopts the structural unit of the object, its attributes, and the relationships among objects. In this way, we can describe the image in detail and create a comprehensive and detailed scene graph.&#10; \item We design a web-based lightweight annotation tool that is easy to deploy and operate so that the proposed SGDraw can draw scene graphs interactively.&#10; \item We conduct user experiments to verify the performance of our proposed object-oriented annotation approach for annotation, especially the effectiveness of annotation on complex images.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Artificial Intelligence, Human-Computer Interaction, Cognitive Science, Graphical Annotation, Computer Science, Scene Understanding" />
-        </attvalues>
-      </node>
-      <node id="2003.00387" label="2003.00387">
-        <attvalues>
-          <attvalue for="0" value="Say As You Wish: Fine-grained Control of Image Caption Generation with&#10;  Abstract Scene Graphs" />
-          <attvalue for="1" value="  Humans are able to describe image contents with coarse to fine details as&#10;they wish. However, most image captioning models are intention-agnostic which&#10;can not generate diverse descriptions according to different user intentions&#10;initiatively. In this work, we propose the Abstract Scene Graph (ASG) structure&#10;to represent user intention in fine-grained level and control what and how&#10;detailed the generated description should be. The ASG is a directed graph&#10;consisting of three types of \textbf{abstract nodes} (object, attribute,&#10;relationship) grounded in the image without any concrete semantic labels. Thus&#10;it is easy to obtain either manually or automatically. From the ASG, we propose&#10;a novel ASG2Caption model, which is able to recognise user intentions and&#10;semantics in the graph, and therefore generate desired captions according to&#10;the graph structure. Our model achieves better controllability conditioning on&#10;ASGs than carefully designed baselines on both VisualGenome and MSCOCO&#10;datasets. It also significantly improves the caption diversity via&#10;automatically sampling diverse ASGs as control signals.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.01855" label="1812.01855">
-        <attvalues>
-          <attvalue for="0" value="Explainable and Explicit Visual Reasoning over Scene Graphs" />
-          <attvalue for="1" value="  We aim to dismantle the prevalent black-box neural architectures used in&#10;complex visual reasoning tasks, into the proposed eXplainable and eXplicit&#10;Neural Modules (XNMs), which advance beyond existing neural module networks&#10;towards using scene graphs --- objects as nodes and the pairwise relationships&#10;as edges --- for explainable and explicit reasoning with structured knowledge.&#10;XNMs allow us to pay more attention to teach machines how to &quot;think&quot;,&#10;regardless of what they &quot;look&quot;. As we will show in the paper, by using scene&#10;graphs as an inductive bias, 1) we can design XNMs in a concise and flexible&#10;fashion, i.e., XNMs merely consist of 4 meta-types, which significantly reduce&#10;the number of parameters by 10 to 100 times, and 2) we can explicitly trace the&#10;reasoning-flow in terms of graph attentions. XNMs are so generic that they&#10;support a wide range of scene graph implementations with various qualities. For&#10;example, when the graphs are detected perfectly, XNMs achieve 100% accuracy on&#10;both CLEVR and CLEVR CoGenT, establishing an empirical performance upper-bound&#10;for visual reasoning; when the graphs are noisily detected from real-world&#10;images, XNMs are still robust to achieve a competitive 67.5% accuracy on&#10;VQAv2.0, surpassing the popular bag-of-objects attention models without graph&#10;structures.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.01072" label="2007.01072">
-        <attvalues>
-          <attvalue for="0" value="Scene Graph Reasoning for Visual Question Answering" />
-          <attvalue for="1" value="  Visual question answering is concerned with answering free-form questions&#10;about an image. Since it requires a deep linguistic understanding of the&#10;question and the ability to associate it with various objects that are present&#10;in the image, it is an ambitious task and requires techniques from both&#10;computer vision and natural language processing. We propose a novel method that&#10;approaches the task by performing context-driven, sequential reasoning based on&#10;the objects and their semantic and spatial relationships present in the scene.&#10;As a first step, we derive a scene graph which describes the objects in the&#10;image, as well as their attributes and their mutual relationships. A&#10;reinforcement agent then learns to autonomously navigate over the extracted&#10;scene graph to generate paths, which are then the basis for deriving answers.&#10;We conduct a first experimental study on the challenging GQA dataset with&#10;manually curated scene graphs, where our method almost reaches the level of&#10;human performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.10432" label="2202.10432">
-        <attvalues>
-          <attvalue for="0" value="Reasoning with Scene Graphs for Robot Planning under Partial&#10;  Observability" />
-          <attvalue for="1" value="  Robot planning in partially observable domains is difficult, because a robot&#10;needs to estimate the current state and plan actions at the same time. When the&#10;domain includes many objects, reasoning about the objects and their&#10;relationships makes robot planning even more difficult. In this paper, we&#10;develop an algorithm called scene analysis for robot planning (SARP) that&#10;enables robots to reason with visual contextual information toward achieving&#10;long-term goals under uncertainty. SARP constructs scene graphs, a factored&#10;representation of objects and their relations, using images captured from&#10;different positions, and reasons with them to enable context-aware robot&#10;planning under partial observability. Experiments have been conducted using&#10;multiple 3D environments in simulation, and a dataset collected by a real&#10;robot. In comparison to standard robot planning and scene analysis methods, in&#10;a target search domain, SARP improves both efficiency and accuracy in task&#10;completion. Supplementary material can be found at https://tinyurl.com/sarp22&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1602.07332" label="1602.07332">
-        <attvalues>
-          <attvalue for="0" value="Visual Genome: Connecting Language and Vision Using Crowdsourced Dense&#10;  Image Annotations" />
-          <attvalue for="1" value="  Despite progress in perceptual tasks such as image classification, computers&#10;still perform poorly on cognitive tasks such as image description and question&#10;answering. Cognition is core to tasks that involve not just recognizing, but&#10;reasoning about our visual world. However, models used to tackle the rich&#10;content in images for cognitive tasks are still being trained using the same&#10;datasets designed for perceptual tasks. To achieve success at cognitive tasks,&#10;models need to understand the interactions and relationships between objects in&#10;an image. When asked &quot;What vehicle is the person riding?&quot;, computers will need&#10;to identify the objects in an image as well as the relationships riding(man,&#10;carriage) and pulling(horse, carriage) in order to answer correctly that &quot;the&#10;person is riding a horse-drawn carriage&quot;.&#10;  In this paper, we present the Visual Genome dataset to enable the modeling of&#10;such relationships. We collect dense annotations of objects, attributes, and&#10;relationships within each image to learn these models. Specifically, our&#10;dataset contains over 100K images where each image has an average of 21&#10;objects, 18 attributes, and 18 pairwise relationships between objects. We&#10;canonicalize the objects, attributes, relationships, and noun phrases in region&#10;descriptions and questions answer pairs to WordNet synsets. Together, these&#10;annotations represent the densest and largest dataset of image descriptions,&#10;objects, attributes, relationships, and question answers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.05756" label="2007.05756">
-        <attvalues>
-          <attvalue for="0" value="Generative Compositional Augmentations for Scene Graph Prediction" />
-          <attvalue for="1" value="  Inferring objects and their relationships from an image in the form of a&#10;scene graph is useful in many applications at the intersection of vision and&#10;language. We consider a challenging problem of compositional generalization&#10;that emerges in this task due to a long tail data distribution. Current scene&#10;graph generation models are trained on a tiny fraction of the distribution&#10;corresponding to the most frequent compositions, e.g. &lt;cup, on, table&gt;.&#10;However, test images might contain zero- and few-shot compositions of objects&#10;and relationships, e.g. &lt;cup, on, surfboard&gt;. Despite each of the object&#10;categories and the predicate (e.g. 'on') being frequent in the training data,&#10;the models often fail to properly understand such unseen or rare compositions.&#10;To improve generalization, it is natural to attempt increasing the diversity of&#10;the training distribution. However, in the graph domain this is non-trivial. To&#10;that end, we propose a method to synthesize rare yet plausible scene graphs by&#10;perturbing real ones. We then propose and empirically study a model based on&#10;conditional generative adversarial networks (GANs) that allows us to generate&#10;visual features of perturbed scene graphs and learn from them in a joint&#10;fashion. When evaluated on the Visual Genome dataset, our approach yields&#10;marginal, but consistent improvements in zero- and few-shot metrics. We analyze&#10;the limitations of our approach indicating promising directions for future&#10;research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.08230" label="2005.08230">
-        <attvalues>
-          <attvalue for="0" value="Graph Density-Aware Losses for Novel Compositions in Scene Graph&#10;  Generation" />
-          <attvalue for="1" value="  Scene graph generation (SGG) aims to predict graph-structured descriptions of&#10;input images, in the form of objects and relationships between them. This task&#10;is becoming increasingly useful for progress at the interface of vision and&#10;language. Here, it is important - yet challenging - to perform well on novel&#10;(zero-shot) or rare (few-shot) compositions of objects and relationships. In&#10;this paper, we identify two key issues that limit such generalization. Firstly,&#10;we show that the standard loss used in this task is unintentionally a function&#10;of scene graph density. This leads to the neglect of individual edges in large&#10;sparse graphs during training, even though these contain diverse few-shot&#10;examples that are important for generalization. Secondly, the frequency of&#10;relationships can create a strong bias in this task, such that a blind model&#10;predicting the most frequent relationship achieves good performance.&#10;Consequently, some state-of-the-art models exploit this bias to improve&#10;results. We show that such models can suffer the most in their ability to&#10;generalize to rare compositions, evaluating two different models on the Visual&#10;Genome dataset and its more recent, improved version, GQA. To address these&#10;issues, we introduce a density-normalized edge loss, which provides more than a&#10;two-fold improvement in certain generalization metrics. Compared to other works&#10;in this direction, our enhancements require only a few lines of code and no&#10;added computational cost. We also highlight the difficulty of accurately&#10;evaluating models using existing metrics, especially on zero/few shots, and&#10;introduce a novel weighted metric.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.00050" label="2303.00050">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface" />
-          <attvalue for="1" value="  Reconstructing general dynamic scenes is important for many computer vision&#10;and graphics applications. Recent works represent the dynamic scene with neural&#10;radiance fields for photorealistic view synthesis, while their surface geometry&#10;is under-constrained and noisy. Other works introduce surface constraints to&#10;the implicit neural representation to disentangle the ambiguity of geometry and&#10;appearance field for static scene reconstruction. To bridge the gap between&#10;rendering dynamic scenes and recovering static surface geometry, we propose a&#10;template-free method to reconstruct surface geometry and appearance using&#10;neural implicit representations from multi-view videos. We leverage&#10;topology-aware deformation and the signed distance field to learn complex&#10;dynamic surfaces via differentiable volume rendering without scene-specific&#10;prior knowledge like template models. Furthermore, we propose a novel&#10;mask-based ray selection strategy to significantly boost the optimization on&#10;challenging time-varying regions. Experiments on different multi-view video&#10;datasets demonstrate that our method achieves high-fidelity surface&#10;reconstruction as well as photorealistic novel view synthesis.&#10;" />
-          <attvalue for="2" value="&#10;Recent success of deep learning techniques in 2D domain has sparked a surge of interest in higher dimensional problems, such as 3D computer vision tasks and their application fields in &#10;VR/AR, movie production, games etc. &#10;For instance, the neural radiance field (NeRF)~\cite{nerf} demonstrates that Multi-Layer Perceptron (MLP) neural networks can represent a scene by implicitly encoding the geometry and appearance into the parameter of networks. The learned model allows free-viewpoint photorealistic novel view synthesis through traditional volume rendering techniques. &#10;&#10;While NeRF was initially designed for static content, subsequent dynamic works &#10;either condition the neural field with addtional temporal input~\cite{nsff,xian2021space,nrf4}, or jointly optimize a deformation field and a neural radiance network~\cite {d-nerf, nerfies, NR-NeRF}. Nevertheless, compared to remarkable achievements in view synthesis, their performance&#10;in geometry representation is relatively unsatisfying.&#10;To address this problem, several works~\cite{volsdf,neus,NeuralWarp,dneus} employ the signed distance function(SDF) for implicit surface representation. Compared to density, SDF can be more efficiently regularized, relieving the entanglement between shape and appearance.&#10;However, these methods only reconstruct static scenes instead of more commonly seen dynamic ones.&#10;&#10;Another closely related research topic is recovering dynamic humans from videos using an articulated human body model such as SMPL~\cite{SMPL:2015}.&#10;Although these methods~\cite{anerf, peng2022animatable, chen2021accurate, neuralactor,peng2021neural,dd-nerf} have demonstrated impressive efficacy in view synthesis and reconstruction in terms of the human body, the requirement of human priors like the SMPL model hinders them from generalizing to other dynamic scenes. General dynamic scenes are challenging but widely used in applications, such as interaction with props and objects like balls, loose garments (skirts, cloaks), and movements of animals or robots.&#10;&#10;To overcome the above limitations, we propose DySurf, a neural multi-view dynamic surface reconstruction method without prior knowledge of the target shape. We map the spatial points along a camera ray from observation space to the canonical space&#10;, which is explicitly parameterized by &#10;the SE(3) field computed from a deformation network. This enables the model to share the coherence of geometry and appearance information across time. To handle challenging topology changes in dynamic scenes, we adopt the design of a hyperdimensional network~\cite {hypernerf}.&#10;For rendering, we employ the neural SDF and radiance field to represent geometry and appearance in the canonical space. To recover dynamic areas with fast motion, we propose a novel ray selection strategy that assigns a higher sampling probability to pixels of interest derived from &#10;dynamic masks.&#10;&#10;We show the performance of our method across different scenarios on a public dataset (GeneBody~\cite {cheng2022genebody}) as well as a multi-view dataset captured by ourselves. With the focus on dynamic surface reconstruction, we also demonstrate the capability of our work on view synthesis.&#10;In summary, the main contributions of this work are as follows: 1) An end-to-end framework called DySurf for topology-aware dynamic surface reconstruction from multi-view videos without templates; 2) A novel mask-based ray selection strategy to boost the optimization by focusing more on the time-varying foreground region; 3) Extensive evaluation of high-fidelity surface reconstruction on the public GeneBody dataset as well as our captured dataset. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Machine Learning, Computer Vision Applications, Neural Implicit Representations, Mathematics, Physics, Computer Science, Dynamic Scene Reconstruction, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2011.13084" label="2011.13084">
-        <attvalues>
-          <attvalue for="0" value="Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes" />
-          <attvalue for="1" value="  We present a method to perform novel view and time synthesis of dynamic&#10;scenes, requiring only a monocular video with known camera poses as input. To&#10;do this, we introduce Neural Scene Flow Fields, a new representation that&#10;models the dynamic scene as a time-variant continuous function of appearance,&#10;geometry, and 3D scene motion. Our representation is optimized through a neural&#10;network to fit the observed input views. We show that our representation can be&#10;used for complex dynamic scenes, including thin structures, view-dependent&#10;effects, and natural degrees of motion. We conduct a number of experiments that&#10;demonstrate our approach significantly outperforms recent monocular view&#10;synthesis methods, and show qualitative results of space-time view synthesis on&#10;a variety of real-world videos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.12950" label="2011.12950">
-        <attvalues>
-          <attvalue for="0" value="Space-time Neural Irradiance Fields for Free-Viewpoint Video" />
-          <attvalue for="1" value="  We present a method that learns a spatiotemporal neural irradiance field for&#10;dynamic scenes from a single video. Our learned representation enables&#10;free-viewpoint rendering of the input video. Our method builds upon recent&#10;advances in implicit representations. Learning a spatiotemporal irradiance&#10;field from a single video poses significant challenges because the video&#10;contains only one observation of the scene at any point in time. The 3D&#10;geometry of a scene can be legitimately represented in numerous ways since&#10;varying geometry (motion) can be explained with varying appearance and vice&#10;versa. We address this ambiguity by constraining the time-varying geometry of&#10;our dynamic scene representation using the scene depth estimated from video&#10;depth estimation methods, aggregating contents from individual frames into a&#10;single global representation. We provide an extensive quantitative evaluation&#10;and demonstrate compelling free-viewpoint rendering results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.09790" label="2012.09790">
-        <attvalues>
-          <attvalue for="0" value="Neural Radiance Flow for 4D View Synthesis and Video Processing" />
-          <attvalue for="1" value="  We present a method, Neural Radiance Flow (NeRFlow),to learn a 4D&#10;spatial-temporal representation of a dynamic scene from a set of RGB images.&#10;Key to our approach is the use of a neural implicit representation that learns&#10;to capture the 3D occupancy, radiance, and dynamics of the scene. By enforcing&#10;consistency across different modalities, our representation enables multi-view&#10;rendering in diverse dynamic scenes, including water pouring, robotic&#10;interaction, and real images, outperforming state-of-the-art methods for&#10;spatial-temporal view synthesis. Our approach works even when inputs images are&#10;captured with only one camera. We further demonstrate that the learned&#10;representation can serve as an implicit scene prior, enabling video processing&#10;tasks such as image super-resolution and de-noising without any additional&#10;supervision.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.09648" label="2112.09648">
-        <attvalues>
-          <attvalue for="0" value="Improving neural implicit surfaces geometry with patch warping" />
-          <attvalue for="1" value="  Neural implicit surfaces have become an important technique for multi-view 3D&#10;reconstruction but their accuracy remains limited. In this paper, we argue that&#10;this comes from the difficulty to learn and render high frequency textures with&#10;neural networks. We thus propose to add to the standard neural rendering&#10;optimization a direct photo-consistency term across the different views.&#10;Intuitively, we optimize the implicit geometry so that it warps views on each&#10;other in a consistent way. We demonstrate that two elements are key to the&#10;success of such an approach: (i) warping entire patches, using the predicted&#10;occupancy and normals of the 3D points along each ray, and measuring their&#10;similarity with a robust structural similarity (SSIM); (ii) handling visibility&#10;and occlusion in such a way that incorrect warps are not given too much&#10;importance while encouraging a reconstruction as complete as possible. We&#10;evaluate our approach, dubbed NeuralWarp, on the standard DTU and EPFL&#10;benchmarks and show it outperforms state of the art unsupervised implicit&#10;surfaces reconstructions by over 20% on both datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2211.11320" label="2211.11320">
-        <attvalues>
-          <attvalue for="0" value="Recovering Fine Details for Neural Implicit Surface Reconstruction" />
-          <attvalue for="1" value="  Recent works on implicit neural representations have made significant&#10;strides. Learning implicit neural surfaces using volume rendering has gained&#10;popularity in multi-view reconstruction without 3D supervision. However,&#10;accurately recovering fine details is still challenging, due to the underlying&#10;ambiguity of geometry and appearance representation. In this paper, we present&#10;D-NeuS, a volume rendering-base neural implicit surface reconstruction method&#10;capable to recover fine geometry details, which extends NeuS by two additional&#10;loss functions targeting enhanced reconstruction quality. First, we encourage&#10;the rendered surface points from alpha compositing to have zero signed distance&#10;values, alleviating the geometry bias arising from transforming SDF to density&#10;for volume rendering. Second, we impose multi-view feature consistency on the&#10;surface points, derived by interpolating SDF zero-crossings from sampled points&#10;along rays. Extensive quantitative and qualitative results demonstrate that our&#10;method reconstructs high-accuracy surfaces with details, and outperforms the&#10;state of the art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.06199" label="2102.06199">
-        <attvalues>
-          <attvalue for="0" value="A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape,&#10;  Appearance, and Pose" />
-          <attvalue for="1" value="  While deep learning reshaped the classical motion capture pipeline with&#10;feed-forward networks, generative models are required to recover fine alignment&#10;via iterative refinement. Unfortunately, the existing models are usually&#10;hand-crafted or learned in controlled conditions, only applicable to limited&#10;domains. We propose a method to learn a generative neural body model from&#10;unlabelled monocular videos by extending Neural Radiance Fields (NeRFs). We&#10;equip them with a skeleton to apply to time-varying and articulated motion. A&#10;key insight is that implicit models require the inverse of the forward&#10;kinematics used in explicit surface models. Our reparameterization defines&#10;spatial latent variables relative to the pose of body parts and thereby&#10;overcomes ill-posed inverse operations with an overparameterization. This&#10;enables learning volumetric body shape and appearance from scratch while&#10;jointly refining the articulated pose; all without ground truth labels for&#10;appearance, pose, or 3D shape on the input videos. When used for&#10;novel-view-synthesis and motion capture, our neural model improves accuracy on&#10;diverse datasets. Project website: https://lemonatsu.github.io/anerf/ .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2112.12390" label="2112.12390">
-        <attvalues>
-          <attvalue for="0" value="Learning Implicit Body Representations from Double Diffusion Based&#10;  Neural Radiance Fields" />
-          <attvalue for="1" value="  In this paper, we present a novel double diffusion based neural radiance&#10;field, dubbed DD-NeRF, to reconstruct human body geometry and render the human&#10;body appearance in novel views from a sparse set of images. We first propose a&#10;double diffusion mechanism to achieve expressive representations of input&#10;images by fully exploiting human body priors and image appearance details at&#10;two levels. At the coarse level, we first model the coarse human body poses and&#10;shapes via an unclothed 3D deformable vertex model as guidance. At the fine&#10;level, we present a multi-view sampling network to capture subtle geometric&#10;deformations and image detailed appearances, such as clothing and hair, from&#10;multiple input views. Considering the sparsity of the two level features, we&#10;diffuse them into feature volumes in the canonical space to construct neural&#10;radiance fields. Then, we present a signed distance function (SDF) regression&#10;network to construct body surfaces from the diffused features. Thanks to our&#10;double diffused representations, our method can even synthesize novel views of&#10;unseen subjects. Experiments on various datasets demonstrate that our approach&#10;outperforms the state-of-the-art in both geometric reconstruction and novel&#10;view synthesis.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.10353" label="2005.10353">
-        <attvalues>
-          <attvalue for="0" value="WHENet: Real-time Fine-Grained Estimation for Wide Range Head Pose" />
-          <attvalue for="1" value="  We present an end-to-end head-pose estimation network designed to predict&#10;Euler angles through the full range head yaws from a single RGB image. Existing&#10;methods perform well for frontal views but few target head pose from all&#10;viewpoints. This has applications in autonomous driving and retail. Our network&#10;builds on multi-loss approaches with changes to loss functions and training&#10;strategies adapted to wide range estimation. Additionally, we extract ground&#10;truth labelings of anterior views from a current panoptic dataset for the first&#10;time. The resulting Wide Headpose Estimation Network (WHENet) is the first&#10;fine-grained modern method applicable to the full-range of head yaws (hence&#10;wide) yet also meets or beats state-of-the-art methods for frontal head pose&#10;estimation. Our network is compact and efficient for mobile devices and&#10;applications.&#10;" />
-          <attvalue for="2" value="&#10;Head pose estimation (HPE) is the task of estimating the orientation of heads from images or video (see Figure~\ref{fig:full_range_image}) and has seen considerable research. Applications of HPE are wide-ranging and include (but are not limited to) virtual \&amp; augmented-reality~\cite{murphy2010head}, driver assistance, markerless motion capture~\cite{de2013robust} or as an integral component of gaze-estimation~\cite{murphy2008head} since gaze and head pose are tightly linked~\cite{langton2004influence}.&#10;&#10;The importance of HPE is well described in~\cite{murphy2008head}. They describe wide-ranging social interactions such as {mutual gaze}, {driver-pedestrian} \&amp; {driver-driver} interaction. It is also important in providing visual cues for the targets of conversation, to indicate appropriate times for speaker/listener role switches as well as to indicate agreement~\cite{murphy2008head}. For systems to interact with people naturally, it is important to be sensitive to head pose.&#10;&#10;Most HPE methods target frontal to profile poses since applications are plentiful, the face is feature-rich and training datasets are widely available. However, covering full range is useful in many areas including driver assistance~\cite{murphy2007head}, motion-capture and to generate attention maps for advertising and retail~\cite{siriteerakul2012advance}.&#10;&#10;Furthermore, though methods such as~\cite{ruiz2018fine,shao2019improving} perform well, they may be prohibitively large networks for mobile and embedded platforms. This is especially important in autonomous driving where many subsystems must operate concurrently. &#10;&#10;Based on these criteria, we developed the Wide Headpose Estimation Network (WHENet) which extends head-pose estimation to the full range of yaws (hence {wide}) using a mobile-friendly architecture. In doing so, we make the following contributions:&#10;&#10;\begin{itemize}&#10;\item We introduce a {wrapped loss} that significantly improves yaw accuracy for anterior views in full-range HPE.&#10;\item We detail an automated labeling process for the CMU Panoptic Dataset~\cite{joo2017panoptic} allowing it to be used for training and validation data in full-range HPE.&#10;\item We extend our modified network (WHENet) to the full range of yaws where it achieves state-of-the-art performance for full-range HPE {\em} and within 1.8\% of state-of-the-art for narrow range HPE despite being trained for another task.&#10;\item We demonstrate that simple modifications to HopeNet~\cite{ruiz2018fine} can achieve $29\%$ improvement over the original network and $5-13\%$ improvement over the current state-of-the-art FSANet~\cite{yang2019fsa} for narrow range HPE from RGB images in the AFLW2000~\cite{zhu2016face} and BIWI~\cite{fanelli_IJCV} datasets.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Head Pose Estimation, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1710.00925" label="1710.00925">
-        <attvalues>
-          <attvalue for="0" value="Fine-Grained Head Pose Estimation Without Keypoints" />
-          <attvalue for="1" value="  Estimating the head pose of a person is a crucial problem that has a large&#10;amount of applications such as aiding in gaze estimation, modeling attention,&#10;fitting 3D models to video and performing face alignment. Traditionally head&#10;pose is computed by estimating some keypoints from the target face and solving&#10;the 2D to 3D correspondence problem with a mean human head model. We argue that&#10;this is a fragile method because it relies entirely on landmark detection&#10;performance, the extraneous head model and an ad-hoc fitting step. We present&#10;an elegant and robust way to determine pose by training a multi-loss&#10;convolutional neural network on 300W-LP, a large synthetically expanded&#10;dataset, to predict intrinsic Euler angles (yaw, pitch and roll) directly from&#10;image intensities through joint binned pose classification and regression. We&#10;present empirical tests on common in-the-wild pose benchmark datasets which&#10;show state-of-the-art results. Additionally we test our method on a dataset&#10;usually used for pose estimation using depth and start to close the gap with&#10;state-of-the-art depth pose methods. We open-source our training and testing&#10;code as well as release our pre-trained models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.08609" label="1905.08609">
-        <attvalues>
-          <attvalue for="0" value="Improving Head Pose Estimation with a Combined Loss and Bounding Box&#10;  Margin Adjustment" />
-          <attvalue for="1" value="  We address a problem of estimating pose of a person's head from its RGB&#10;image. The employment of CNNs for the problem has contributed to significant&#10;improvement in accuracy in recent works. However, we show that the following&#10;two methods, despite their simplicity, can attain further improvement: (i)&#10;proper adjustment of the margin of bounding box of a detected face, and (ii)&#10;choice of loss functions. We show that the integration of these two methods&#10;achieve the new state-of-the-art on standard benchmark datasets for in-the-wild&#10;head pose estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.10123" label="2109.10123">
-        <attvalues>
-          <attvalue for="0" value="Survey on Semantic Stereo Matching / Semantic Depth Estimation" />
-          <attvalue for="1" value="  Stereo matching is one of the widely used techniques for inferring depth from&#10;stereo images owing to its robustness and speed. It has become one of the major&#10;topics of research since it finds its applications in autonomous driving,&#10;robotic navigation, 3D reconstruction, and many other fields. Finding pixel&#10;correspondences in non-textured, occluded and reflective areas is the major&#10;challenge in stereo matching. Recent developments have shown that semantic cues&#10;from image segmentation can be used to improve the results of stereo matching.&#10;Many deep neural network architectures have been proposed to leverage the&#10;advantages of semantic segmentation in stereo matching. This paper aims to give&#10;a comparison among the state of art networks both in terms of accuracy and in&#10;terms of speed which are of higher importance in real-time applications.&#10;" />
-          <attvalue for="2" value="&#10;&#10;1.1 Stereo Matching: Stereo Matching is a technique used to find point correspondences between two images of a scene acquired by known cameras. The traditional approaches such as &#10; Semi-Global Matching \cite{SGM} includes well-defined steps such as cost matching (calculating matching cost for each pixel in one of the stereo images depending on the corresponding pixel in the other image), cost aggregation (calculating matching cost for range of disparity values to form cost volumes), disparity estimation (calculating the disparity from cost volume by minimizing the cost function) and disparity post-processing (improving the disparity). These approaches make assumptions that the scene is uniformly illuminated and textured.\\&#10; \\&#10;1.2 Semantic Segmentation: Semantic Segmentation is an image &#10;segmentation methodology that approaches the problem by performing pixel-level classifications.&#10;Traditional approaches include superpixel segmentation \cite{Superpixel} (dividing an image into hundreds of non-overlapping superpixels), active contour methods \cite{ActiveContour} (methods which make use of the energy constraints and forces in the image for separation of region of interest) and watershed segmentation methods \cite{WaterShed} (algorithm which treats pixels values as a local topography).\\&#10;\\&#10;1.3 Synergies between Stereo Matching and Semantic Segmentation: Recent works in semantic matching \cite{SegStereo} have shown that the semantic labels obtained from semantic segmentation improve the accuracy of stereo matching in non-textured, occluded and reflective regions. Similarly, depth information obtained by stereo matching can be used to solve possible confusions between similar semantic categories \cite{SemForDipsEstimation}. This shows that segmentation information and depth maps complement each other and together they represent high-level information of the scene.\\&#10;\\&#10;1.4 State of the Art:&#10;Deep learning techniques have shown great success in both stereo matching and semantic segmentation during the past ten years. The state of the art uses deep neural networks that are trained for one of these tasks exclusively and they achieve very reliable accuracy. Recent developments \cite{SegStereo}, \cite{SemForDipsEstimation} in the field revealed that stereo matching and semantic segmentation are intrinsically related, and both pieces of&#10;information need to be considered in an integrated manner to succeed in challenging applications such as robotics and autonomous navigation. The interactive environment in such applications requires stereo matching to have a high-level understanding of the scene. Single deep neural networks jointly trained for both the tasks achieve better accuracy when compared to standalone models but are not efficient in terms of speed due to the network complexity. The work presented in this paper is aimed towards the possible exploration of various methods used in semantic&#10;stereo matching for real-time applications and compare them in terms of accuracy (network loss) and speed (fps) on benchmark datasets. We also try to compare, to what extent these methods are utilizing the synergies between depth information and semantic cues for the benefit of the applications.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Computer Vision Applications, Mathematics, Deep Neural Networks, Artificial Intelligence, Stereo Image Matching, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1807.11699" label="1807.11699">
-        <attvalues>
-          <attvalue for="0" value="SegStereo: Exploiting Semantic Information for Disparity Estimation" />
-          <attvalue for="1" value="  Disparity estimation for binocular stereo images finds a wide range of&#10;applications. Traditional algorithms may fail on featureless regions, which&#10;could be handled by high-level clues such as semantic segments. In this paper,&#10;we suggest that appropriate incorporation of semantic cues can greatly rectify&#10;prediction in commonly-used disparity estimation frameworks. Our method&#10;conducts semantic feature embedding and regularizes semantic cues as the loss&#10;term to improve learning disparity. Our unified model SegStereo employs&#10;semantic features from segmentation and introduces semantic softmax loss, which&#10;helps improve the prediction accuracy of disparity maps. The semantic cues work&#10;well in both unsupervised and supervised manners. SegStereo achieves&#10;state-of-the-art results on KITTI Stereo benchmark and produces decent&#10;prediction on both CityScapes and FlyingThings3D datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.10776" label="2011.10776">
-        <attvalues>
-          <attvalue for="0" value="DmifNet:3D Shape Reconstruction Based on Dynamic Multi-Branch&#10;  Information Fusion" />
-          <attvalue for="1" value="  3D object reconstruction from a single-view image is a long-standing&#10;challenging problem. Previous work was difficult to accurately reconstruct 3D&#10;shapes with a complex topology which has rich details at the edges and corners.&#10;Moreover, previous works used synthetic data to train their network, but domain&#10;adaptation problems occurred when tested on real data. In this paper, we&#10;propose a Dynamic Multi-branch Information Fusion Network (DmifNet) which can&#10;recover a high-fidelity 3D shape of arbitrary topology from a 2D image.&#10;Specifically, we design several side branches from the intermediate layers to&#10;make the network produce more diverse representations to improve the&#10;generalization ability of network. In addition, we utilize DoG (Difference of&#10;Gaussians) to extract edge geometry and corners information from input images.&#10;Then, we use a separate side branch network to process the extracted data to&#10;better capture edge geometry and corners feature information. Finally, we&#10;dynamically fuse the information of all branches to gain final predicted&#10;probability. Extensive qualitative and quantitative experiments on a&#10;large-scale publicly available dataset demonstrate the validity and efficiency&#10;of our method. Code and models are publicly available at&#10;https://github.com/leilimaster/DmifNet.&#10;" />
-          <attvalue for="2" value="&#10;3D reconstruction is an elementary problem of image processing and computer vision thanks to many potential useful real-world applications such as robotics, autonomous driving and augmented reality. Recently, 3D reconstruction based on deep learning has achieved remarkable results in many directions, such as 3D shape reconstruction from single view or multiple views \cite{wu2016learning,choy20163d,tatarchenko2019single} and shape completion \cite{xu2019disn}. Generally, most of the 3D reconstruction work is based on CNN encoder-decoder architecture\cite{choy20163d,richter2018matryoshka}. Specifically, single-view image reconstruction tasks usually take 2D CNN to encode 2D images and use different decoders to produce different final representations according to what representations the task needs. For example, if voxels\cite{wu2018learning,zhang2018learning}are expected to be the final representation, 3D CNN will be selected as the decoder.&#10;&#10;Historically, single-view image 3D reconstruction was achieved via shape-from-shading \cite{zhang1999shape,horn1970shape}. These work inferred the depth information of the visible surface through multiple clues (e.g. texture, defocus) and structural information from a single image. Recently, according to the output representation, the existing work on learning-based 3D reconstruction can be categorized into mesh-based, point-based and voxel-based. Since there is no clear way to generate a valid mesh, mesh-based representation is facing great challenges. Then, Wang ${et}$ ${al.}$ \cite{wang2018pixel2mesh} used graph convolutional neural network \cite{scarselli2008graph} to make an ellipsoid template gradually become the target object, but the result was usually limited to the spherical topology. Fan ${et}$ ${al.}$ \cite{fan2017point}introduced point clouds as an output representation for 3D reconstruction. However, the point-based representation requires many complex post-processing steps to generate a 3D mesh. Some work \cite{wu2016learning,choy20163d,wu20153d} have considered voxel-based representations and used 3D CNN to reconstruct the 3D shape from a single image. But, this is only possible with shallow architectures and small batch sizes due to memory limitation, which leads to slow training. To address the impact of memory limitation, Tatarchenko ${et}$ ${al.}$ \cite{tatarchenko2017octree} performed hierarchical partitioning of the output space for computational and storage efficiency, which helped predict higher resolution 3D shapes. Previous work chose to train their model on synthetic data with ground truth 3D information, but suffered from domain adaptation when tested on real data. To address the problem of domain adaptation, Wu ${et}$ ${al.}$ \cite{wu2017marrnet} proposed an end-to-end trainable model that sequentially estimated 2.5D sketches and 3D object shape. Groueix${et}$ ${al.}$ \cite{groueix2018papier} used small patches to splice the result of 3D shapes. However, it is easy to cause overlapping between the small patches. Recently, some work\cite{xu2019disn,mescheder2019occupancy,chen2019learning} implicitly represented the 3D surface as the continuous decision boundary of a deep neural network classifier. In other words, these work learned a classifier to predict whether a point is inside or outside of the boundary, using this classifier as the shape representation. However, since the shape is represented by the weights of the classifier or regression model, these methods ignore some low-level shape information.&#10;&#10;In this paper, we propose a multi-branch information fusion network for 3D reconstruction. We first strengthen the model's ability to capture the complex topology. To be more specific, as the difference between two different low-pass filtered images, the DoG is actually a band-pass filter, which removes high frequency components representing noise, and also some low frequency components representing the homogeneous areas in the image. The frequency components in the passing band are assumed to be associated to the edges in the images. So, we utilize DoG to extract edge geometry and corners information, and use a separate branch network to infer the 3D shape. In addition, previous work generally used synthetic data for training, but suffered from domain adaptation when tested on real data. So, inspired by Li ${et}$ ${al.}$ \cite{li2020dynamic} and Xu ${et}$ ${al.}$ \cite{xu20203d}, we design several side branches at different locations of the main branch network to improve the generalization ability. Finally, we dynamically fuse the multi-branch probability distribution information as the final prediction result. As shown in Fig.1, we show the reconstruction results of our method on real images. Our main contributions lie in the following aspects:&#10;\begin{itemize}&#10;\item We use DoG to process input images to extract edge geometry and corners information. Because we realize the object edge geometry and corners information are important for neural network to capture complex topology.&#10;\item &#09;We design side branches from the intermediate layers of our neural network, so each side branch produces more diverse representations along its own pathway.&#10;\item &#09;Unlike previous methods computing the average value or fixed weight of all branches predicted probability, we dynamically fuse the predicted probability of all branches to obtain the final predicted probability.&#10;\item &#09;Extensive evaluation on a large-scale publicly available dataset ShapeNet demonstrates our method can achieve higher evaluation results than the state-of-the-art methods.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Image Processing Techniques, Computer Graphics, Deep Learning Networks" />
-        </attvalues>
-      </node>
-      <node id="1610.07584" label="1610.07584">
-        <attvalues>
-          <attvalue for="0" value="Learning a Probabilistic Latent Space of Object Shapes via 3D&#10;  Generative-Adversarial Modeling" />
-          <attvalue for="1" value="  We study the problem of 3D object generation. We propose a novel framework,&#10;namely 3D Generative Adversarial Network (3D-GAN), which generates 3D objects&#10;from a probabilistic space by leveraging recent advances in volumetric&#10;convolutional networks and generative adversarial nets. The benefits of our&#10;model are three-fold: first, the use of an adversarial criterion, instead of&#10;traditional heuristic criteria, enables the generator to capture object&#10;structure implicitly and to synthesize high-quality 3D objects; second, the&#10;generator establishes a mapping from a low-dimensional probabilistic space to&#10;the space of 3D objects, so that we can sample objects without a reference&#10;image or CAD models, and explore the 3D object manifold; third, the adversarial&#10;discriminator provides a powerful 3D shape descriptor which, learned without&#10;supervision, has wide applications in 3D object recognition. Experiments&#10;demonstrate that our method generates high-quality 3D objects, and our&#10;unsupervisedly learned features achieve impressive performance on 3D object&#10;recognition, comparable with those of supervised learning methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.00449" label="1604.00449">
-        <attvalues>
-          <attvalue for="0" value="3D-R2N2: A Unified Approach for Single and Multi-view 3D Object&#10;  Reconstruction" />
-          <attvalue for="1" value="  Inspired by the recent success of methods that employ shape priors to achieve&#10;robust 3D reconstructions, we propose a novel recurrent neural network&#10;architecture that we call the 3D Recurrent Reconstruction Neural Network&#10;(3D-R2N2). The network learns a mapping from images of objects to their&#10;underlying 3D shapes from a large collection of synthetic data. Our network&#10;takes in one or more images of an object instance from arbitrary viewpoints and&#10;outputs a reconstruction of the object in the form of a 3D occupancy grid.&#10;Unlike most of the previous works, our network does not require any image&#10;annotations or object class labels for training or testing. Our extensive&#10;experimental analysis shows that our reconstruction framework i) outperforms&#10;the state-of-the-art methods for single view reconstruction, and ii) enables&#10;the 3D reconstruction of objects in situations when traditional SFM/SLAM&#10;methods fail (because of lack of texture and/or wide baseline).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.03678" label="1905.03678">
-        <attvalues>
-          <attvalue for="0" value="What Do Single-view 3D Reconstruction Networks Learn?" />
-          <attvalue for="1" value="  Convolutional networks for single-view object reconstruction have shown&#10;impressive performance and have become a popular subject of research. All&#10;existing techniques are united by the idea of having an encoder-decoder network&#10;that performs non-trivial reasoning about the 3D structure of the output space.&#10;In this work, we set up two alternative approaches that perform image&#10;classification and retrieval respectively. These simple baselines yield better&#10;results than state-of-the-art methods, both qualitatively and quantitatively.&#10;We show that encoder-decoder methods are statistically indistinguishable from&#10;these baselines, thus indicating that the current state of the art in&#10;single-view object reconstruction does not actually perform reconstruction but&#10;image classification. We identify aspects of popular experimental procedures&#10;that elicit this behavior and discuss ways to improve the current state of&#10;research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.10975" label="1804.10975">
-        <attvalues>
-          <attvalue for="0" value="Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers" />
-          <attvalue for="1" value="  In this paper, we develop novel, efficient 2D encodings for 3D geometry,&#10;which enable reconstructing full 3D shapes from a single image at high&#10;resolution. The key idea is to pose 3D shape reconstruction as a 2D prediction&#10;problem. To that end, we first develop a simple baseline network that predicts&#10;entire voxel tubes at each pixel of a reference view. By leveraging well-proven&#10;architectures for 2D pixel-prediction tasks, we attain state-of-the-art&#10;results, clearly outperforming purely voxel-based approaches. We scale this&#10;baseline to higher resolutions by proposing a memory-efficient shape encoding,&#10;which recursively decomposes a 3D shape into nested shape layers, similar to&#10;the pieces of a Matryoshka doll. This allows reconstructing highly detailed&#10;shapes with complex topology, as demonstrated in extensive experiments; we&#10;clearly outperform previous octree-based approaches despite having a much&#10;simpler architecture using standard network components. Our Matryoshka networks&#10;further enable reconstructing shapes from IDs or shape similarity, as well as&#10;shape sampling.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.05068" label="1809.05068">
-        <attvalues>
-          <attvalue for="0" value="Learning Shape Priors for Single-View 3D Completion and Reconstruction" />
-          <attvalue for="1" value="  The problem of single-view 3D shape completion or reconstruction is&#10;challenging, because among the many possible shapes that explain an&#10;observation, most are implausible and do not correspond to natural objects.&#10;Recent research in the field has tackled this problem by exploiting the&#10;expressiveness of deep convolutional networks. In fact, there is another level&#10;of ambiguity that is often overlooked: among plausible shapes, there are still&#10;multiple shapes that fit the 2D image equally well; i.e., the ground truth&#10;shape is non-deterministic given a single-view input. Existing fully supervised&#10;approaches fail to address this issue, and often produce blurry mean shapes&#10;with smooth surfaces but no fine details.&#10;  In this paper, we propose ShapeHD, pushing the limit of single-view shape&#10;completion and reconstruction by integrating deep generative models with&#10;adversarially learned shape priors. The learned priors serve as a regularizer,&#10;penalizing the model only if its output is unrealistic, not if it deviates from&#10;the ground truth. Our design thus overcomes both levels of ambiguity&#10;aforementioned. Experiments demonstrate that ShapeHD outperforms state of the&#10;art by a large margin in both shape completion and shape reconstruction on&#10;multiple real datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.11166" label="1812.11166">
-        <attvalues>
-          <attvalue for="0" value="Learning to Reconstruct Shapes from Unseen Classes" />
-          <attvalue for="1" value="  From a single image, humans are able to perceive the full 3D shape of an&#10;object by exploiting learned shape priors from everyday life. Contemporary&#10;single-image 3D reconstruction algorithms aim to solve this task in a similar&#10;fashion, but often end up with priors that are highly biased by training&#10;classes. Here we present an algorithm, Generalizable Reconstruction (GenRe),&#10;designed to capture more generic, class-agnostic shape priors. We achieve this&#10;with an inference network and training procedure that combine 2.5D&#10;representations of visible surfaces (depth and silhouette), spherical shape&#10;representations of both visible and non-visible surfaces, and 3D voxel-based&#10;representations, in a principled manner that exploits the causal structure of&#10;how 3D shapes give rise to 2D images. Experiments demonstrate that GenRe&#10;performs well on single-view shape reconstruction, and generalizes to diverse&#10;novel objects from categories not seen during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.00603" label="1612.00603">
-        <attvalues>
-          <attvalue for="0" value="A Point Set Generation Network for 3D Object Reconstruction from a&#10;  Single Image" />
-          <attvalue for="1" value="  Generation of 3D data by deep neural network has been attracting increasing&#10;attention in the research community. The majority of extant works resort to&#10;regular representations such as volumetric grids or collection of images;&#10;however, these representations obscure the natural invariance of 3D shapes&#10;under geometric transformations and also suffer from a number of other issues.&#10;In this paper we address the problem of 3D reconstruction from a single image,&#10;generating a straight-forward form of output -- point cloud coordinates. Along&#10;with this problem arises a unique and interesting issue, that the groundtruth&#10;shape for an input image may be ambiguous. Driven by this unorthodox output&#10;form and the inherent ambiguity in groundtruth, we design architecture, loss&#10;function and learning paradigm that are novel and effective. Our final solution&#10;is a conditional shape sampler, capable of predicting multiple plausible 3D&#10;point clouds from an input image. In experiments not only can our system&#10;outperform state-of-the-art methods on single image based 3d reconstruction&#10;benchmarks; but it also shows a strong performance for 3d shape completion and&#10;promising ability in making multiple plausible predictions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.09438" label="1703.09438">
-        <attvalues>
-          <attvalue for="0" value="Octree Generating Networks: Efficient Convolutional Architectures for&#10;  High-resolution 3D Outputs" />
-          <attvalue for="1" value="  We present a deep convolutional decoder architecture that can generate&#10;volumetric 3D outputs in a compute- and memory-efficient manner by using an&#10;octree representation. The network learns to predict both the structure of the&#10;octree, and the occupancy values of individual cells. This makes it a&#10;particularly valuable technique for generating 3D shapes. In contrast to&#10;standard decoders acting on regular voxel grids, the architecture does not have&#10;cubic complexity. This allows representing much higher resolution outputs with&#10;a limited memory budget. We demonstrate this in several application domains,&#10;including 3D convolutional autoencoders, generation of objects and whole scenes&#10;from high-level representations, and shape from a single image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.10739" label="2003.10739">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Hierarchical Mimicking Towards Consistent Optimization&#10;  Objectives" />
-          <attvalue for="1" value="  While the depth of modern Convolutional Neural Networks (CNNs) surpasses that&#10;of the pioneering networks with a significant margin, the traditional way of&#10;appending supervision only over the final classifier and progressively&#10;propagating gradient flow upstream remains the training mainstay. Seminal&#10;Deeply-Supervised Networks (DSN) were proposed to alleviate the difficulty of&#10;optimization arising from gradient flow through a long chain. However, it is&#10;still vulnerable to issues including interference to the hierarchical&#10;representation generation process and inconsistent optimization objectives, as&#10;illustrated theoretically and empirically in this paper. Complementary to&#10;previous training strategies, we propose Dynamic Hierarchical Mimicking, a&#10;generic feature learning mechanism, to advance CNN training with enhanced&#10;generalization ability. Partially inspired by DSN, we fork delicately designed&#10;side branches from the intermediate layers of a given neural network. Each&#10;branch can emerge from certain locations of the main branch dynamically, which&#10;not only retains representation rooted in the backbone network but also&#10;generates more diverse representations along its own pathway. We go one step&#10;further to promote multi-level interactions among different branches through an&#10;optimization formula with probabilistic prediction matching losses, thus&#10;guaranteeing a more robust optimization process and better representation&#10;ability. Experiments on both category and instance recognition tasks&#10;demonstrate the substantial improvements of our proposed method over its&#10;corresponding counterparts using diverse state-of-the-art CNN architectures.&#10;Code and models are publicly available at https://github.com/d-li14/DHM&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.03192" label="2409.03192">
-        <attvalues>
-          <attvalue for="0" value="PEPL: Precision-Enhanced Pseudo-Labeling for Fine-Grained Image&#10;  Classification in Semi-Supervised Learning" />
-          <attvalue for="1" value="  Fine-grained image classification has witnessed significant advancements with&#10;the advent of deep learning and computer vision technologies. However, the&#10;scarcity of detailed annotations remains a major challenge, especially in&#10;scenarios where obtaining high-quality labeled data is costly or&#10;time-consuming. To address this limitation, we introduce Precision-Enhanced&#10;Pseudo-Labeling(PEPL) approach specifically designed for fine-grained image&#10;classification within a semi-supervised learning framework. Our method&#10;leverages the abundance of unlabeled data by generating high-quality&#10;pseudo-labels that are progressively refined through two key phases: initial&#10;pseudo-label generation and semantic-mixed pseudo-label generation. These&#10;phases utilize Class Activation Maps (CAMs) to accurately estimate the semantic&#10;content and generate refined labels that capture the essential details&#10;necessary for fine-grained classification. By focusing on semantic-level&#10;information, our approach effectively addresses the limitations of standard&#10;data augmentation and image-mixing techniques in preserving critical&#10;fine-grained features. We achieve state-of-the-art performance on benchmark&#10;datasets, demonstrating significant improvements over existing semi-supervised&#10;strategies, with notable boosts in accuracy and robustness.Our code has been&#10;open sourced at https://github.com/TianSuya/SemiFG.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Fine-grained image classification \cite{wang2019survey,rong2021human,zhuang2020learning}, which involves distinguishing between visually similar classes, plays a crucial role in various applications such as species identification, product categorization, and medical diagnostics. Despite the remarkable success of deep learning in computer vision \cite{abdullahi2024systematic,rayed2024deep,lai2023multimodal}, achieving high accuracy in fine-grained classification remains challenging due to the scarcity of labeled data and the subtlety of distinguishing features \cite{guo2024fine}.&#10;&#10;The limited availability of labeled data, particularly in fine-grained domains, hinders the development of robust models. To mitigate this issue, semi-supervised learning (SSL) \cite{grandvalet2004semi, yang2022class,zeng2023self} techniques have been proposed to leverage large amounts of unlabeled data alongside a small labeled dataset. SSL methods, including pseudo-labeling \cite{lee2013pseudo} and consistency regularization \cite{laine2016temporal}, have shown promise in improving model performance with limited supervision. However, existing SSL approaches face significant challenges when applied to fine-grained image classification. Standard data augmentation techniques \cite{cubuk2020randaugment} \cite{cubuk2019autoaugmentlearningaugmentationpolicies} can disrupt critical visual cues, and fine-grained image features are destroyed. Image region mixing may also&#10;overlook the fine details essential for accurate classification \cite{su2021realistic}.&#10;&#10;To address these challenges, we present a novel \underline{P}recision-\underline{E}nhanced \underline{P}seudo-\underline{L}abeling (PEPL) approach tailored for fine-grained image classification. PEPL leverages CAMs \cite{jiang2021layercam} to generate high-quality pseudo-labels that capture the essential details necessary for fine-grained classification. Specifically, our method consists of two key phases: Initial Pseudo-Label Generation and Semantic-Mixed Pseudo-Label Generation. These phases utilize CAMs\cite{jiang2021layercam,muhammad2020eigen,zhang2024opti} to accurately estimate the semantic content\cite{chen2022class} and generate refined labels that capture the essential details necessary for fine-grained classification. By focusing on semantic-level information, our approach effectively addresses the limitations of standard data augmentation and image-mixing techniques in preserving critical fine-grained features, we have conducted extensive experiments on two commonly used datasets for fine-grained classification, and the results show that our method far exceeds the most advanced and representative semi-supervised methods\cite{yang2022survey,ouali2020overview} at present, with a 13\% improvement in accuracy over the fully supervised model on the CUB\_200\_2011 dataset using 20\% labeled data, and similar results to supervised learning using only 30\% labeled data. &#10;&#10;The key contributions of our work are as follows:&#10;&#10;(i) We propose the Precision-Enhanced Pseudo-Labeling (PEPL) approach specifically designed for fine-grained image classification.&#10;&#10;(ii) Our method generates high-quality pseudo-labels using CAMs, which are progressively refined to enhance the precision of the pseudo-labels.&#10;&#10;(iii) We demonstrate significant improvements in performance on benchmark datasets, outperforming existing semi-supervised strategies and achieving state-of-the-art accuracy.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Science, Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Fine-Grained Classification, Computer Science, Semi-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2111.01628" label="2111.01628">
-        <attvalues>
-          <attvalue for="0" value="Human Attention in Fine-grained Classification" />
-          <attvalue for="1" value="  The way humans attend to, process and classify a given image has the&#10;potential to vastly benefit the performance of deep learning models. Exploiting&#10;where humans are focusing can rectify models when they are deviating from&#10;essential features for correct decisions. To validate that human attention&#10;contains valuable information for decision-making processes such as&#10;fine-grained classification, we compare human attention and model explanations&#10;in discovering important features. Towards this goal, we collect human gaze&#10;data for the fine-grained classification dataset CUB and build a dataset named&#10;CUB-GHA (Gaze-based Human Attention). Furthermore, we propose the Gaze&#10;Augmentation Training (GAT) and Knowledge Fusion Network (KFN) to integrate&#10;human gaze knowledge into classification models. We implement our proposals in&#10;CUB-GHA and the recently released medical dataset CXR-Eye of chest X-ray&#10;images, which includes gaze data collected from a radiologist. Our result&#10;reveals that integrating human attention knowledge benefits classification&#10;effectively, e.g. improving the baseline by 4.38% on CXR. Hence, our work&#10;provides not only valuable insights into understanding human attention in&#10;fine-grained classification, but also contributes to future research in&#10;integrating human gaze with computer vision tasks. CUB-GHA and code are&#10;available at https://github.com/yaorong0921/CUB-GHA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.10191" label="2002.10191">
-        <attvalues>
-          <attvalue for="0" value="Learning Attentive Pairwise Interaction for Fine-Grained Classification" />
-          <attvalue for="1" value="  Fine-grained classification is a challenging problem, due to subtle&#10;differences among highly-confused categories. Most approaches address this&#10;difficulty by learning discriminative representation of individual input image.&#10;On the other hand, humans can effectively identify contrastive clues by&#10;comparing image pairs. Inspired by this fact, this paper proposes a simple but&#10;effective Attentive Pairwise Interaction Network (API-Net), which can&#10;progressively recognize a pair of fine-grained images by interaction.&#10;Specifically, API-Net first learns a mutual feature vector to capture semantic&#10;differences in the input pair. It then compares this mutual vector with&#10;individual vectors to generate gates for each input image. These distinct gate&#10;vectors inherit mutual context on semantic differences, which allow API-Net to&#10;attentively capture contrastive clues by pairwise interaction between two&#10;images. Additionally, we train API-Net in an end-to-end manner with a score&#10;ranking regularization, which can further generalize API-Net by taking feature&#10;priorities into account. We conduct extensive experiments on five popular&#10;benchmarks in fine-grained classification. API-Net outperforms the recent SOTA&#10;methods, i.e., CUB-200-2011 (90.0%), Aircraft(93.9%), Stanford Cars (95.3%),&#10;Stanford Dogs (90.3%), and NABirds (88.1%).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.07611" label="2305.07611">
-        <attvalues>
-          <attvalue for="0" value="Multimodal Sentiment Analysis: A Survey" />
-          <attvalue for="1" value="  Multimodal sentiment analysis has become an important research area in the&#10;field of artificial intelligence. With the latest advances in deep learning,&#10;this technology has reached new heights. It has great potential for both&#10;application and research, making it a popular research topic. This review&#10;provides an overview of the definition, background, and development of&#10;multimodal sentiment analysis. It also covers recent datasets and advanced&#10;models, emphasizing the challenges and future prospects of this technology.&#10;Finally, it looks ahead to future research directions. It should be noted that&#10;this review provides constructive suggestions for promising research directions&#10;and building better performing multimodal sentiment analysis models, which can&#10;help researchers in this field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.17766" label="2401.17766">
-        <attvalues>
-          <attvalue for="0" value="Fine-Grained Zero-Shot Learning: Advances, Challenges, and Prospects" />
-          <attvalue for="1" value="  Recent zero-shot learning (ZSL) approaches have integrated fine-grained&#10;analysis, i.e., fine-grained ZSL, to mitigate the commonly known seen/unseen&#10;domain bias and misaligned visual-semantics mapping problems, and have made&#10;profound progress. Notably, this paradigm differs from existing close-set&#10;fine-grained methods and, therefore, can pose unique and nontrivial challenges.&#10;However, to the best of our knowledge, there remains a lack of systematic&#10;summaries of this topic. To enrich the literature of this domain and provide a&#10;sound basis for its future development, in this paper, we present a broad&#10;review of recent advances for fine-grained analysis in ZSL. Concretely, we&#10;first provide a taxonomy of existing methods and techniques with a thorough&#10;analysis of each category. Then, we summarize the benchmark, covering publicly&#10;available datasets, models, implementations, and some more details as a&#10;library. Last, we sketch out some related applications. In addition, we discuss&#10;vital challenges and suggest potential future directions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.13719" label="1909.13719">
-        <attvalues>
-          <attvalue for="0" value="RandAugment: Practical automated data augmentation with a reduced search&#10;  space" />
-          <attvalue for="1" value="  Recent work has shown that data augmentation has the potential to&#10;significantly improve the generalization of deep learning models. Recently,&#10;automated augmentation strategies have led to state-of-the-art results in image&#10;classification and object detection. While these strategies were optimized for&#10;improving validation accuracy, they also led to state-of-the-art results in&#10;semi-supervised learning and improved robustness to common corruptions of&#10;images. An obstacle to a large-scale adoption of these methods is a separate&#10;search phase which increases the training complexity and may substantially&#10;increase the computational cost. Additionally, due to the separate search&#10;phase, these approaches are unable to adjust the regularization strength based&#10;on model or dataset size. Automated augmentation policies are often found by&#10;training small models on small datasets and subsequently applied to train&#10;larger models. In this work, we remove both of these obstacles. RandAugment has&#10;a significantly reduced search space which allows it to be trained on the&#10;target task with no need for a separate proxy task. Furthermore, due to the&#10;parameterization, the regularization strength may be tailored to different&#10;model and dataset sizes. RandAugment can be used uniformly across different&#10;tasks and datasets and works out of the box, matching or surpassing all&#10;previous automated augmentation approaches on CIFAR-10/100, SVHN, and ImageNet.&#10;On the ImageNet dataset we achieve 85.0% accuracy, a 0.6% increase over the&#10;previous state-of-the-art and 1.0% increase over baseline augmentation. On&#10;object detection, RandAugment leads to 1.0-1.3% improvement over baseline&#10;augmentation, and is within 0.3% mAP of AutoAugment on COCO. Finally, due to&#10;its interpretable hyperparameter, RandAugment may be used to investigate the&#10;role of data augmentation with varying model and dataset size. Code is&#10;available online.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.09501" label="1805.09501">
-        <attvalues>
-          <attvalue for="0" value="AutoAugment: Learning Augmentation Policies from Data" />
-          <attvalue for="1" value="  Data augmentation is an effective technique for improving the accuracy of&#10;modern image classifiers. However, current data augmentation implementations&#10;are manually designed. In this paper, we describe a simple procedure called&#10;AutoAugment to automatically search for improved data augmentation policies. In&#10;our implementation, we have designed a search space where a policy consists of&#10;many sub-policies, one of which is randomly chosen for each image in each&#10;mini-batch. A sub-policy consists of two operations, each operation being an&#10;image processing function such as translation, rotation, or shearing, and the&#10;probabilities and magnitudes with which the functions are applied. We use a&#10;search algorithm to find the best policy such that the neural network yields&#10;the highest validation accuracy on a target dataset. Our method achieves&#10;state-of-the-art accuracy on CIFAR-10, CIFAR-100, SVHN, and ImageNet (without&#10;additional data). On ImageNet, we attain a Top-1 accuracy of 83.5% which is&#10;0.4% better than the previous record of 83.1%. On CIFAR-10, we achieve an error&#10;rate of 1.5%, which is 0.6% better than the previous state-of-the-art.&#10;Augmentation policies we find are transferable between datasets. The policy&#10;learned on ImageNet transfers well to achieve significant improvements on other&#10;datasets, such as Oxford Flowers, Caltech-101, Oxford-IIT Pets, FGVC Aircraft,&#10;and Stanford Cars.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.00679" label="2104.00679">
-        <attvalues>
-          <attvalue for="0" value="A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained&#10;  Classification" />
-          <attvalue for="1" value="  We evaluate the effectiveness of semi-supervised learning (SSL) on a&#10;realistic benchmark where data exhibits considerable class imbalance and&#10;contains images from novel classes. Our benchmark consists of two fine-grained&#10;classification datasets obtained by sampling classes from the Aves and Fungi&#10;taxonomy. We find that recently proposed SSL methods provide significant&#10;benefits, and can effectively use out-of-class data to improve performance when&#10;deep networks are trained from scratch. Yet their performance pales in&#10;comparison to a transfer learning baseline, an alternative approach for&#10;learning from a few examples. Furthermore, in the transfer setting, while&#10;existing SSL methods provide improvements, the presence of out-of-class is&#10;often detrimental. In this setting, standard fine-tuning followed by&#10;distillation-based self-training is the most robust. Our work suggests that&#10;semi-supervised learning with experts on realistic datasets may require&#10;different strategies than those currently prevalent in the literature.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.00299" label="2008.00299">
-        <attvalues>
-          <attvalue for="0" value="Eigen-CAM: Class Activation Map using Principal Components" />
-          <attvalue for="1" value="  Deep neural networks are ubiquitous due to the ease of developing models and&#10;their influence on other domains. At the heart of this progress is&#10;convolutional neural networks (CNNs) that are capable of learning&#10;representations or features given a set of data. Making sense of such complex&#10;models (i.e., millions of parameters and hundreds of layers) remains&#10;challenging for developers as well as the end-users. This is partially due to&#10;the lack of tools or interfaces capable of providing interpretability and&#10;transparency. A growing body of literature, for example, class activation map&#10;(CAM), focuses on making sense of what a model learns from the data or why it&#10;behaves poorly in a given task. This paper builds on previous ideas to cope&#10;with the increasing demand for interpretable, robust, and transparent models.&#10;Our approach provides a simpler and intuitive (or familiar) way of generating&#10;CAM. The proposed Eigen-CAM computes and visualizes the principle components of&#10;the learned features/representations from the convolutional layers. Empirical&#10;studies were performed to compare the Eigen-CAM with the state-of-the-art&#10;methods (such as Grad-CAM, Grad-CAM++, CNN-fixations) by evaluating on&#10;benchmark datasets such as weakly-supervised localization and localizing&#10;objects in the presence of adversarial noise. Eigen-CAM was found to be robust&#10;against classification errors made by fully connected layers in CNNs, does not&#10;rely on the backpropagation of gradients, class relevance score, maximum&#10;activation locations, or any other form of weighting features. In addition, it&#10;works with all CNN models without the need to modify layers or retrain models.&#10;Empirical results show up to 12% improvement over the best method among the&#10;methods compared on weakly supervised object localization.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.07002" label="2301.07002">
-        <attvalues>
-          <attvalue for="0" value="Opti-CAM: Optimizing saliency maps for interpretability" />
-          <attvalue for="1" value="  Methods based on class activation maps (CAM) provide a simple mechanism to&#10;interpret predictions of convolutional neural networks by using linear&#10;combinations of feature maps as saliency maps. By contrast, masking-based&#10;methods optimize a saliency map directly in the image space or learn it by&#10;training another network on additional data.&#10;  In this work we introduce Opti-CAM, combining ideas from CAM-based and&#10;masking-based approaches. Our saliency map is a linear combination of feature&#10;maps, where weights are optimized per image such that the logit of the masked&#10;image for a given class is maximized. We also fix a fundamental flaw in two of&#10;the most common evaluation metrics of attribution methods. On several datasets,&#10;Opti-CAM largely outperforms other CAM-based approaches according to the most&#10;relevant classification metrics. We provide empirical evidence supporting that&#10;localization and classifier interpretability are not necessarily aligned.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.00962" label="2203.00962">
-        <attvalues>
-          <attvalue for="0" value="Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation" />
-          <attvalue for="1" value="  Extracting class activation maps (CAM) is arguably the most standard step of&#10;generating pseudo masks for weakly-supervised semantic segmentation (WSSS).&#10;Yet, we find that the crux of the unsatisfactory pseudo masks is the binary&#10;cross-entropy loss (BCE) widely used in CAM. Specifically, due to the&#10;sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to&#10;multiple classes co-occurring in the same receptive field. As a result, given a&#10;class, its hot CAM pixels may wrongly invade the area belonging to other&#10;classes, or the non-hot ones may be actually a part of the class. To this end,&#10;we introduce an embarrassingly simple yet surprisingly effective method:&#10;Reactivating the converged CAM with BCE by using softmax cross-entropy loss&#10;(SCE), dubbed \textbf{ReCAM}. Given an image, we use CAM to extract the feature&#10;pixels of each single class, and use them with the class label to learn another&#10;fully-connected layer (after the backbone) with SCE. Once converged, we extract&#10;ReCAM in the same way as in CAM. Thanks to the contrastive nature of SCE, the&#10;pixel response is disentangled into different classes and hence less mask&#10;ambiguity is expected. The evaluation on both PASCAL VOC and MS~COCO shows that&#10;ReCAM not only generates high-quality masks, but also supports plug-and-play in&#10;any CAM variant with little overhead.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.00550" label="2103.00550">
-        <attvalues>
-          <attvalue for="0" value="A Survey on Deep Semi-supervised Learning" />
-          <attvalue for="1" value="  Deep semi-supervised learning is a fast-growing field with a range of&#10;practical applications. This paper provides a comprehensive survey on both&#10;fundamentals and recent advances in deep semi-supervised learning methods from&#10;perspectives of model design and unsupervised loss functions. We first present&#10;a taxonomy for deep semi-supervised learning that categorizes existing methods,&#10;including deep generative methods, consistency regularization methods,&#10;graph-based methods, pseudo-labeling methods, and hybrid methods. Then we&#10;provide a comprehensive review of 52 representative methods and offer a&#10;detailed comparison of these methods in terms of the type of losses,&#10;contributions, and architecture differences. In addition to the progress in the&#10;past few years, we further discuss some shortcomings of existing methods and&#10;provide some tentative heuristic solutions for solving these open problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.05278" label="2006.05278">
-        <attvalues>
-          <attvalue for="0" value="An Overview of Deep Semi-Supervised Learning" />
-          <attvalue for="1" value="  Deep neural networks demonstrated their ability to provide remarkable&#10;performances on a wide range of supervised learning tasks (e.g., image&#10;classification) when trained on extensive collections of labeled data (e.g.,&#10;ImageNet). However, creating such large datasets requires a considerable amount&#10;of resources, time, and effort. Such resources may not be available in many&#10;practical cases, limiting the adoption and the application of many deep&#10;learning methods. In a search for more data-efficient deep learning methods to&#10;overcome the need for large annotated datasets, there is a rising research&#10;interest in semi-supervised learning and its applications to deep neural&#10;networks to reduce the amount of labeled data required, by either developing&#10;novel methods or adopting existing semi-supervised learning frameworks for a&#10;deep learning setting. In this paper, we provide a comprehensive overview of&#10;deep semi-supervised learning, starting with an introduction to the field,&#10;followed by a summarization of the dominant semi-supervised approaches in deep&#10;learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.11227" label="2006.11227">
-        <attvalues>
-          <attvalue for="0" value="Lookahead Adversarial Learning for Near Real-Time Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation is one of the most fundamental problems in computer&#10;vision with significant impact on a wide variety of applications. Adversarial&#10;learning is shown to be an effective approach for improving semantic&#10;segmentation quality by enforcing higher-level pixel correlations and&#10;structural information. However, state-of-the-art semantic segmentation models&#10;cannot be easily plugged into an adversarial setting because they are not&#10;designed to accommodate convergence and stability issues in adversarial&#10;networks. We bridge this gap by building a conditional adversarial network with&#10;a state-of-the-art segmentation model (DeepLabv3+) at its core. To battle the&#10;stability issues, we introduce a novel lookahead adversarial learning (LoAd)&#10;approach with an embedded label map aggregation module. We focus on semantic&#10;segmentation models that run fast at inference for near real-time field&#10;applications. Through extensive experimentation, we demonstrate that the&#10;proposed solution can alleviate divergence issues in an adversarial semantic&#10;segmentation setting and results in considerable performance improvements (+5%&#10;in some classes) on the baseline for three standard datasets.&#10;" />
-          <attvalue for="2" value="&#10;Semantic segmentation is a challenging task in computer vision. It is a pivotal step towards content-based image analysis and scene understanding as it empowers machines to distinguish between different regions of an image based on its semantic context. To this aim, semantic segmentation models are trained to assign semantic labels to each and every pixel of an image as well as to cluster them into groups. Semantic segmentation has received an upsurge of attention recently owing to its wide variety of applications in medical imaging \cite{ronneberger2015u, rezaei2017conditional}, autonomous driving \cite{menze2015object, cordts2016cityscapes}, satellite image processing \cite{volpi2015semantic, henry2018road}, and robotics \cite{geiger2013vision, shvets2018automatic}, to name a few. Early segmentation methodologies are mostly developed with clustering algorithms at their core and somehow trying to incorporate contour, edge, and structural information. Examples of such algorithms are active contours \cite{kass1988snakes}, region-growing \cite{nock2004statistical}, conditional random fields (CRFs) \cite{plath2009multi}, and sparse reconstruction based methods \cite{minaee2019admm}. Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge. &#10;&#10;Majority of these deep learning based methods formulate semantic segmentation as a classification problem where cross entropy (CE) with pixel independence assumption is employed as the optimization loss function. However, in practice, adjacent pixels of an image are highly correlated. These methods implicitly assume that correlation among pixels would be learned as receptive field of CNNs increases going deeper with convolutions. Recent studies challenge this assumption and argue that overlooking pixel correlations explicitly can lead to performance degradation especially with regards to capturing structural information embedded in target classes. These studies have proposed different approaches to capture pixel inter-dependencies. For instance, CRFs can be used to model pixel relationships and enforce label consistency between pixels \cite{liu2017deep, chen2017deeplab, shen2017semantic, liu2017deep}. However, CRFs are known to be time-consuming at inference and sensitive to variations in visual appearance. An alternative approach is extracting pixel affinity information from images and fusing them back to predicted label maps \cite{ke2018adaptive}; this comes at the cost of extra model branches and larger memory requirements. More recent studies have proposed changing the perspective and using different loss functions that encode the mutual information or structural similarity among nearby pixels in a regional fashion \cite{zhao2019region, zhao2019correlation} and have shown improvements. However, these losses can be derived in a sub-optimal manner by making major simplifying assumptions and by considering a small patch of pixels. &#10;&#10;Another avenue that has been explored to enforce structure in segmentation is employing adversarial learning \cite{luc2016semantic, souly2017semi, xue2018segan, hung2018adversarial}. In this setup, a segmentor-discriminator pair compete to outperform each other in creating realistic label maps and distinguishing them from ground truth ones. We think a conditional adversarial approach similar to \cite{luc2016semantic, souly2017semi} has the capacity to capture these pixel inter-dependencies and correlations in a more general (and not only local) fashion when compared to methods proposed in \cite{zhao2019region, zhao2019correlation}. In medical imaging, there are studies proposing (generative) adversarial networks for image segmentation \cite{imran2020progressive, chen2018semantic}; however, their aim is tackling domain shift in semi-supervised settings and they mostly employ U-Net type segmentors given their solid performance in this context. On the other hand, plugging state-of-the-art semantic segmentation models in an adversarial setting is prone to the well-known divergence and mode collapse issues \cite{arjovsky2017wasserstein, goodfellow2014explaining}. Specific architecture designs for generator and discriminator networks can help to stabilize the setup, but at the cost of limiting the application domain of adversarial networks. When it comes to semantic segmentation, we are bound to architectures specifically designed to excel in doing so. When we tried to establish an adversarial network with DeepLabv3+ \cite{chen2018encoder} as segmentor, we could not manage to stabilize the network regardless of remedies we pulled in. Bridging the gap between employing the state-of-the-art semantic segmentation models in adversarial settings and helping to stabilize them is the core idea of the proposed lookahead adversarial learning (LoAd) approach. Notably, we focus on models that run fast at inference time for near real-time field applications. That is why we opt for DeepLabv3+ base models that offer speed (no multi-scaling (MS), no CRFs) and performance at the same time. Nonetheless, in our ablation studies, we also demonstrate the impact of applying time-consuming post-processing steps such as MS on the performance of proposed solution. &#10;&#10;The proposed solution (LoAd) takes inspiration from the ``lookahead optimizer'' \cite{zhang2019lookahead} and allows the semantic segmentation adversarial network to go ahead and actually diverge to some extent, then inspired by DAGGER in imitation learning \cite{ross2011reduction} aggregates the degraded label maps, and steps backward to use these new sets of information and avoid further divergence. Notably, our label map aggregation strategy is different from collecting adversarial examples to retrain and stabilize generative adversarial networks (GANs) \cite{goodfellow2014explaining, liu2019rob}. We demonstrate that LoAd can alleviate divergence issues of adversarial training in a semantic segmentation setting leading to improvement in mean-intersection-over-union (mIoU) over the baseline DeepLabv3+ on Pascal VOC 2012, Cityscapes and Stanford Background datasets. We also show that in some classes $+5\%$ improvement in IoU beyond the baseline is achieved which is quite significant. We then qualitatively demonstrate that our solution is boosting the baseline in overall segmentation performance in tackling class swap/confusion, as well as in better understanding of structure and continuity of the target classes.&#10;&#10;The main contributions of this paper can be summarized as follows: a) we propose a lookahead adversarial learning method (LoAd) for semantic segmentation that helps alleviate stability issues in such settings, b) LoAd runs as fast as the baselines methods upon which it is applied, i.e., no extra delay at inference time, making it a great choice for near real-time field applications, c) besides avoiding class confusion, LoAd improves the performance of the baseline in creating structurally more consistent label maps with significant performance boost in some classes. Enhancing state-of-the-art segmentation models that can be trained on commodity GPUs and run fast at inference time (no CRFs, no multi-scaling) is the motive of this work.&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Adversarial Learning" />
-        </attvalues>
-      </node>
-      <node id="1704.07711" label="1704.07711">
-        <attvalues>
-          <attvalue for="0" value="An ADMM Approach to Masked Signal Decomposition Using Subspace&#10;  Representation" />
-          <attvalue for="1" value="  Signal decomposition is a classical problem in signal processing, which aims&#10;to separate an observed signal into two or more components each with its own&#10;property. Usually each component is described by its own subspace or&#10;dictionary. Extensive research has been done for the case where the components&#10;are additive, but in real world applications, the components are often&#10;non-additive. For example, an image may consist of a foreground object overlaid&#10;on a background, where each pixel either belongs to the foreground or the&#10;background. In such a situation, to separate signal components, we need to find&#10;a binary mask which shows the location of each component. Therefore it requires&#10;to solve a binary optimization problem. Since most of the binary optimization&#10;problems are intractable, we relax this problem to the approximated continuous&#10;problem, and solve it by alternating optimization technique. We show the&#10;application of the proposed algorithm for three applications: separation of&#10;text from background in images, separation of moving objects from a background&#10;undergoing global camera motion in videos, separation of sinusoidal and spike&#10;components in one dimensional signals. We demonstrate in each case that&#10;considering the non-additive nature of the problem can lead to significant&#10;improvement.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.08904" label="1803.08904">
-        <attvalues>
-          <attvalue for="0" value="Context Encoding for Semantic Segmentation" />
-          <attvalue for="1" value="  Recent work has made significant progress in improving spatial resolution for&#10;pixelwise labeling with Fully Convolutional Network (FCN) framework by&#10;employing Dilated/Atrous convolution, utilizing multi-scale features and&#10;refining boundaries. In this paper, we explore the impact of global contextual&#10;information in semantic segmentation by introducing the Context Encoding&#10;Module, which captures the semantic context of scenes and selectively&#10;highlights class-dependent featuremaps. The proposed Context Encoding Module&#10;significantly improves semantic segmentation results with only marginal extra&#10;computation cost over FCN. Our approach has achieved new state-of-the-art&#10;results 51.7% mIoU on PASCAL-Context, 85.9% mIoU on PASCAL VOC 2012. Our single&#10;model achieves a final score of 0.5567 on ADE20K test set, which surpass the&#10;winning entry of COCO-Place Challenge in 2017. In addition, we also explore how&#10;the Context Encoding Module can improve the feature representation of&#10;relatively shallow networks for the image classification on CIFAR-10 dataset.&#10;Our 14 layer network has achieved an error rate of 3.45%, which is comparable&#10;with state-of-the-art approaches with over 10 times more layers. The source&#10;code for the complete system are publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1606.07230" label="1606.07230">
-        <attvalues>
-          <attvalue for="0" value="Deep Learning Markov Random Field for Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation tasks can be well modeled by Markov Random Field (MRF).&#10;This paper addresses semantic segmentation by incorporating high-order&#10;relations and mixture of label contexts into MRF. Unlike previous works that&#10;optimized MRFs using iterative algorithm, we solve MRF by proposing a&#10;Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which&#10;enables deterministic end-to-end computation in a single forward pass.&#10;Specifically, DPN extends a contemporary CNN to model unary terms and&#10;additional layers are devised to approximate the mean field (MF) algorithm for&#10;pairwise terms. It has several appealing properties. First, different from the&#10;recent works that required many iterations of MF during back-propagation, DPN&#10;is able to achieve high performance by approximating one iteration of MF.&#10;Second, DPN represents various types of pairwise terms, making many existing&#10;models as its special cases. Furthermore, pairwise terms in DPN provide a&#10;unified framework to encode rich contextual information in high-dimensional&#10;data, such as images and videos. Third, DPN makes MF easier to be parallelized&#10;and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated&#10;on standard semantic image/video segmentation benchmarks, where a single DPN&#10;model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012,&#10;Cityscapes dataset and CamVid dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.10335" label="1803.10335">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Affinity Fields for Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation has made much progress with increasingly powerful&#10;pixel-wise classifiers and incorporating structural priors via Conditional&#10;Random Fields (CRF) or Generative Adversarial Networks (GAN). We propose a&#10;simpler alternative that learns to verify the spatial structure of segmentation&#10;during training only. Unlike existing approaches that enforce semantic labels&#10;on individual pixels and match labels between neighbouring pixels, we propose&#10;the concept of Adaptive Affinity Fields (AAF) to capture and match the semantic&#10;relations between neighbouring pixels in the label space. We use adversarial&#10;learning to select the optimal affinity field size for each semantic category.&#10;It is formulated as a minimax problem, optimizing our segmentation neural&#10;network in a best worst-case learning scenario. AAF is versatile for&#10;representing structures as a collection of pixel-centric relations, easier to&#10;train than GAN and more efficient than CRF without run-time inference. Our&#10;extensive evaluations on PASCAL VOC 2012, Cityscapes, and GTA5 datasets&#10;demonstrate its above-par segmentation performance and robust generalization&#10;across domains.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.12037" label="1910.12037">
-        <attvalues>
-          <attvalue for="0" value="Region Mutual Information Loss for Semantic Segmentation" />
-          <attvalue for="1" value="  Semantic segmentation is a fundamental problem in computer vision. It is&#10;considered as a pixel-wise classification problem in practice, and most&#10;segmentation models use a pixel-wise loss as their optimization riterion.&#10;However, the pixel-wise loss ignores the dependencies between pixels in an&#10;image. Several ways to exploit the relationship between pixels have been&#10;investigated, \eg, conditional random fields (CRF) and pixel affinity based&#10;methods. Nevertheless, these methods usually require additional model branches,&#10;large extra memories, or more inference time. In this paper, we develop a&#10;region mutual information (RMI) loss to model the dependencies among pixels&#10;more simply and efficiently. In contrast to the pixel-wise loss which treats&#10;the pixels as independent samples, RMI uses one pixel and its neighbour pixels&#10;to represent this pixel. Then for each pixel in an image, we get a&#10;multi-dimensional point that encodes the relationship between pixels, and the&#10;image is cast into a multi-dimensional distribution of these high-dimensional&#10;points. The prediction and ground truth thus can achieve high order consistency&#10;through maximizing the mutual information (MI) between their multi-dimensional&#10;distributions. Moreover, as the actual value of the MI is hard to calculate, we&#10;derive a lower bound of the MI and maximize the lower bound to maximize the&#10;real value of the MI. RMI only requires a few extra computational resources in&#10;the training stage, and there is no overhead during testing. Experimental&#10;results demonstrate that RMI can achieve substantial and consistent&#10;improvements in performance on PASCAL VOC 2012 and CamVid datasets. The code is&#10;available at https://github.com/ZJULearning/RMI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.08408" label="1611.08408">
-        <attvalues>
-          <attvalue for="0" value="Semantic Segmentation using Adversarial Networks" />
-          <attvalue for="1" value="  Adversarial training has been shown to produce state of the art results for&#10;generative image modeling. In this paper we propose an adversarial training&#10;approach to train semantic segmentation models. We train a convolutional&#10;semantic segmentation network along with an adversarial network that&#10;discriminates segmentation maps coming either from the ground truth or from the&#10;segmentation network. The motivation for our approach is that it can detect and&#10;correct higher-order inconsistencies between ground truth segmentation maps and&#10;the ones produced by the segmentation net. Our experiments show that our&#10;adversarial training approach leads to improved accuracy on the Stanford&#10;Background and PASCAL VOC 2012 datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2005.04311" label="2005.04311">
-        <attvalues>
-          <attvalue for="0" value="Progressive Adversarial Semantic Segmentation" />
-          <attvalue for="1" value="  Medical image computing has advanced rapidly with the advent of deep learning&#10;techniques such as convolutional neural networks. Deep convolutional neural&#10;networks can perform exceedingly well given full supervision. However, the&#10;success of such fully-supervised models for various image analysis tasks (e.g.,&#10;anatomy or lesion segmentation from medical images) is limited to the&#10;availability of massive amounts of labeled data. Given small sample sizes, such&#10;models are prohibitively data biased with large domain shift. To tackle this&#10;problem, we propose a novel end-to-end medical image segmentation model, namely&#10;Progressive Adversarial Semantic Segmentation (PASS), which can make improved&#10;segmentation predictions without requiring any domain-specific data during&#10;training time. Our extensive experimentation with 8 public diabetic retinopathy&#10;and chest X-ray datasets, confirms the effectiveness of PASS for accurate&#10;vascular and pulmonary segmentation, both for in-domain and cross-domain&#10;evaluations.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.07875" label="1701.07875">
-        <attvalues>
-          <attvalue for="0" value="Wasserstein GAN" />
-          <attvalue for="1" value="  We introduce a new algorithm named WGAN, an alternative to traditional GAN&#10;training. In this new model, we show that we can improve the stability of&#10;learning, get rid of problems like mode collapse, and provide meaningful&#10;learning curves useful for debugging and hyperparameter searches. Furthermore,&#10;we show that the corresponding optimization problem is sound, and provide&#10;extensive theoretical work highlighting the deep connections to other distances&#10;between distributions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.08610" label="1907.08610">
-        <attvalues>
-          <attvalue for="0" value="Lookahead Optimizer: k steps forward, 1 step back" />
-          <attvalue for="1" value="  The vast majority of successful deep neural networks are trained using&#10;variants of stochastic gradient descent (SGD) algorithms. Recent attempts to&#10;improve SGD can be broadly categorized into two approaches: (1) adaptive&#10;learning rate schemes, such as AdaGrad and Adam, and (2) accelerated schemes,&#10;such as heavy-ball and Nesterov momentum. In this paper, we propose a new&#10;optimization algorithm, Lookahead, that is orthogonal to these previous&#10;approaches and iteratively updates two sets of weights. Intuitively, the&#10;algorithm chooses a search direction by looking ahead at the sequence of fast&#10;weights generated by another optimizer. We show that Lookahead improves the&#10;learning stability and lowers the variance of its inner optimizer with&#10;negligible computation and memory cost. We empirically demonstrate Lookahead&#10;can significantly improve the performance of SGD and Adam, even with their&#10;default hyperparameter settings on ImageNet, CIFAR-10/100, neural machine&#10;translation, and Penn Treebank.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1011.0686" label="1011.0686">
-        <attvalues>
-          <attvalue for="0" value="A Reduction of Imitation Learning and Structured Prediction to No-Regret&#10;  Online Learning" />
-          <attvalue for="1" value="  Sequential prediction problems such as imitation learning, where future&#10;observations depend on previous predictions (actions), violate the common&#10;i.i.d. assumptions made in statistical learning. This leads to poor performance&#10;in theory and often in practice. Some recent approaches provide stronger&#10;guarantees in this setting, but remain somewhat unsatisfactory as they train&#10;either non-stationary or stochastic policies and require a large number of&#10;iterations. In this paper, we propose a new iterative algorithm, which trains a&#10;stationary deterministic policy, that can be seen as a no regret algorithm in&#10;an online learning setting. We show that any such no regret algorithm, combined&#10;with additional reduction assumptions, must find a policy with good performance&#10;under the distribution of observations it induces in such sequential settings.&#10;We demonstrate that this new approach outperforms previous approaches on two&#10;challenging imitation learning problems and a benchmark sequence labeling&#10;problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1412.6572" label="1412.6572">
-        <attvalues>
-          <attvalue for="0" value="Explaining and Harnessing Adversarial Examples" />
-          <attvalue for="1" value="  Several machine learning models, including neural networks, consistently&#10;misclassify adversarial examples---inputs formed by applying small but&#10;intentionally worst-case perturbations to examples from the dataset, such that&#10;the perturbed input results in the model outputting an incorrect answer with&#10;high confidence. Early attempts at explaining this phenomenon focused on&#10;nonlinearity and overfitting. We argue instead that the primary cause of neural&#10;networks' vulnerability to adversarial perturbation is their linear nature.&#10;This explanation is supported by new quantitative results while giving the&#10;first explanation of the most intriguing fact about them: their generalization&#10;across architectures and training sets. Moreover, this view yields a simple and&#10;fast method of generating adversarial examples. Using this approach to provide&#10;examples for adversarial training, we reduce the test set error of a maxout&#10;network on the MNIST dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2409.02567" label="2409.02567">
-        <attvalues>
-          <attvalue for="0" value="Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation" />
-          <attvalue for="1" value="  Segment Anything Model (SAM) has demonstrated powerful zero-shot segmentation&#10;performance in natural scenes. The recently released Segment Anything Model 2&#10;(SAM2) has further heightened researchers' expectations towards image&#10;segmentation capabilities. To evaluate the performance of SAM2 on&#10;class-agnostic instance-level segmentation tasks, we adopt different prompt&#10;strategies for SAM2 to cope with instance-level tasks for three relevant&#10;scenarios: Salient Instance Segmentation (SIS), Camouflaged Instance&#10;Segmentation (CIS), and Shadow Instance Detection (SID). In addition, to&#10;further explore the effectiveness of SAM2 in segmenting granular object&#10;structures, we also conduct detailed tests on the high-resolution Dichotomous&#10;Image Segmentation (DIS) benchmark to assess the fine-grained segmentation&#10;capability. Qualitative and quantitative experimental results indicate that the&#10;performance of SAM2 varies significantly across different scenarios. Besides,&#10;SAM2 is not particularly sensitive to segmenting high-resolution fine details.&#10;We hope this technique report can drive the emergence of SAM2-based adapters,&#10;aiming to enhance the performance ceiling of large vision models on&#10;class-agnostic instance segmentation tasks.&#10;" />
-          <attvalue for="2" value="&#10;&#10;T{h}e advent of large foundation models, including ChatGPT, GPT-4, and LLaMA, has revolutionized the artificial intelligence (AI) landscape. Powered by extensive datasets, these models excel in multi-modal processing, e.g., language, image, video, and audio, showcasing substantial progress in AI capabilities. Building on these developments, the Segment Anything Model (SAM)~\cite{kirillov2023segment} stands out breakthrough in scene segmentation with large vision models.&#10;The generality and Adaptability of SAM highlight its potential for understanding complex scenarios and targets, further expanding the frontiers of image segmentation tasks.&#10;&#10;SAM allows users to input custom prompts, such as points or bounding boxes, resulting in highly accurate segmentation masks. This adaptability enables SAM to perform a wide range of image segmentation tasks. More recently, the release of SAM2~\cite{ravi2024sam} further overcomes the limitation that SAM does not handle video content well. In the field of image segmentation, SAM2 has shown improvement in segmentation accuracy and inference efficiency. A variety of evaluations have recently emerged to examine the segmentation performance of SAM2 in different scenarios~\cite{lian2024evaluation, yan2024biomedical,zhu2024medical,ma2024segment,tang2024evaluating,lou2024zero,yu2024sam}. For instance, Lian {et al.}~\cite{lian2024evaluation} assessed its instance segmentation performance in underwater environments, while Yan {et al.}~\cite{yan2024biomedical} explored its effectiveness in endoscopic and microscopic images. Additionally, Ma {et al.}~\cite{ma2024segment} conducted a comprehensive benchmark of SAM2 across 11 medical image modalities and videos, highlighting its strengths and weaknesses compared to SAM and MedSAM. Furthermore, Tang {et al.}~\cite{tang2024evaluating} compared SAM2 and SAM on the camouflage object detection benchmark. This research found that SAM2 significantly degrades the performance of SAM2 compared to SAM for detecting camouflaged objects when no prompts are provided, and significantly improves its performance over SAM when segmentation prompts are available. These interesting findings raise curiosity about SAM2's performance in class-agnostic instance-level segmentation tasks.&#10;&#10;In this paper, we evaluate the performance of SAM2 in class-agnostic instance-level segmentation tasks, focusing on three distinct scenarios: Salient Instance Segmentation (SIS)~\cite{li2017instance}, Camouflaged Instance Segmentation (CIS)~\cite{pei2022osformer}, and Shadow Instance Detection (SID)~\cite{wang2022instance}. Moreover, We also thoroughly evaluate SAM2 on the high-resolution dichotomous image segmentation (DIS) benchmark~\cite{qin2022highly} to analyze its ability to segment granular target structures. We compare SAM2 with SAM and well-known specific models on multiple benchmarks. Based on extensive experimental results, we summarize the following conclusions:&#10;\begin{itemize}&#10; \item SAM2 outperforms task-specific methods for CIS and SIS when using bounding boxes as prompt inputs. However, the performance of SAM2 drops remarkably without box prompts, especially for camouflaged instances.&#10; \item SAM2 performs poorly on the DIS task, whether or not it uses bounding boxes as prompts. It indicates that SAM2 is not well suited for fine-grained segmentation of complex object structures.&#10; \item For the SID task, while SAM2 performs well in segmenting instances, it struggles with shadow matching.&#10; \item SAM2 with fewer parameters achieves superior results compared to SAM across four tasks when using bounding boxes as prompts. In contrast, SAM2 without box prompts performs inferior to SAM for SIS, CIS, and SID.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Artificial Intelligence, Deep Learning Models, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2408.02924" label="2408.02924">
-        <attvalues>
-          <attvalue for="0" value="Evaluation of Segment Anything Model 2: The Role of SAM2 in the&#10;  Underwater Environment" />
-          <attvalue for="1" value="  With breakthroughs in large-scale modeling, the Segment Anything Model (SAM)&#10;and its extensions have been attempted for applications in various underwater&#10;visualization tasks in marine sciences, and have had a significant impact on&#10;the academic community. Recently, Meta has further developed the Segment&#10;Anything Model 2 (SAM2), which significantly improves running speed and&#10;segmentation accuracy compared to its predecessor. This report aims to explore&#10;the potential of SAM2 in marine science by evaluating it on the underwater&#10;instance segmentation benchmark datasets UIIS and USIS10K. The experiments show&#10;that the performance of SAM2 is extremely dependent on the type of&#10;user-provided prompts. When using the ground truth bounding box as prompt, SAM2&#10;performed excellently in the underwater instance segmentation domain. However,&#10;when running in automatic mode, SAM2's ability with point prompts to sense and&#10;segment underwater instances is significantly degraded. It is hoped that this&#10;paper will inspire researchers to further explore the SAM model family in the&#10;underwater domain. The results and evaluation codes in this paper are available&#10;at https://github.com/LiamLian0727/UnderwaterSAM2Eval.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.00874" label="2408.00874">
-        <attvalues>
-          <attvalue for="0" value="Medical SAM 2: Segment medical images as video via Segment Anything&#10;  Model 2" />
-          <attvalue for="1" value="  In this paper, we introduce Medical SAM 2 (MedSAM-2), an advanced&#10;segmentation model that utilizes the SAM 2 framework to address both 2D and 3D&#10;medical image segmentation tasks. By adopting the philosophy of taking medical&#10;images as videos, MedSAM-2 not only applies to 3D medical images but also&#10;unlocks new One-prompt Segmentation capability. That allows users to provide a&#10;prompt for just one or a specific image targeting an object, after which the&#10;model can autonomously segment the same type of object in all subsequent&#10;images, regardless of temporal relationships between the images. We evaluated&#10;MedSAM-2 across a variety of medical imaging modalities, including abdominal&#10;organs, optic discs, brain tumors, thyroid nodules, and skin lesions, comparing&#10;it against state-of-the-art models in both traditional and interactive&#10;segmentation settings. Our findings show that MedSAM-2 not only surpasses&#10;existing models in performance but also exhibits superior generalization across&#10;a range of medical image segmentation tasks. Our code will be released at:&#10;https://github.com/MedicineToken/Medical-SAM2&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.03322" label="2408.03322">
-        <attvalues>
-          <attvalue for="0" value="Segment Anything in Medical Images and Videos: Benchmark and Deployment" />
-          <attvalue for="1" value="  Recent advances in segmentation foundation models have enabled accurate and&#10;efficient segmentation across a wide range of natural images and videos, but&#10;their utility to medical data remains unclear. In this work, we first present a&#10;comprehensive benchmarking of the Segment Anything Model 2 (SAM2) across 11&#10;medical image modalities and videos and point out its strengths and weaknesses&#10;by comparing it to SAM1 and MedSAM. Then, we develop a transfer learning&#10;pipeline and demonstrate SAM2 can be quickly adapted to medical domain by&#10;fine-tuning. Furthermore, we implement SAM2 as a 3D slicer plugin and Gradio&#10;API for efficient 3D image and video segmentation. The code has been made&#10;publicly available at \url{https://github.com/bowang-lab/MedSAM}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.01648" label="2408.01648">
-        <attvalues>
-          <attvalue for="0" value="Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment&#10;  Anything Model 2" />
-          <attvalue for="1" value="  The Segment Anything Model 2 (SAM 2) is the latest generation foundation&#10;model for image and video segmentation. Trained on the expansive Segment&#10;Anything Video (SA-V) dataset, which comprises 35.5 million masks across 50.9K&#10;videos, SAM 2 advances its predecessor's capabilities by supporting zero-shot&#10;segmentation through various prompts (e.g., points, boxes, and masks). Its&#10;robust zero-shot performance and efficient memory usage make SAM 2 particularly&#10;appealing for surgical tool segmentation in videos, especially given the&#10;scarcity of labeled data and the diversity of surgical procedures. In this&#10;study, we evaluate the zero-shot video segmentation performance of the SAM 2&#10;model across different types of surgeries, including endoscopy and microscopy.&#10;We also assess its performance on videos featuring single and multiple tools of&#10;varying lengths to demonstrate SAM 2's applicability and effectiveness in the&#10;surgical domain. We found that: 1) SAM 2 demonstrates a strong capability for&#10;segmenting various surgical videos; 2) When new tools enter the scene,&#10;additional prompts are necessary to maintain segmentation accuracy; and 3)&#10;Specific challenges inherent to surgical videos can impact the robustness of&#10;SAM 2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.04593" label="2408.04593">
-        <attvalues>
-          <attvalue for="0" value="SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and&#10;  Generalization in Surgical Video Segmentation" />
-          <attvalue for="1" value="  The recent Segment Anything Model (SAM) 2 has demonstrated remarkable&#10;foundational competence in semantic segmentation, with its memory mechanism and&#10;mask decoder further addressing challenges in video tracking and object&#10;occlusion, thereby achieving superior results in interactive segmentation for&#10;both images and videos. Building upon our previous empirical studies, we&#10;further explore the zero-shot segmentation performance of SAM 2 in&#10;robot-assisted surgery based on prompts, alongside its robustness against&#10;real-world corruption. For static images, we employ two forms of prompts:&#10;1-point and bounding box, while for video sequences, the 1-point prompt is&#10;applied to the initial frame. Through extensive experimentation on the MICCAI&#10;EndoVis 2017 and EndoVis 2018 benchmarks, SAM 2, when utilizing bounding box&#10;prompts, outperforms state-of-the-art (SOTA) methods in comparative&#10;evaluations. The results with point prompts also exhibit a substantial&#10;enhancement over SAM's capabilities, nearing or even surpassing existing&#10;unprompted SOTA methodologies. Besides, SAM 2 demonstrates improved inference&#10;speed and less performance degradation against various image corruption.&#10;Although slightly unsatisfactory results remain in specific edges or regions,&#10;SAM 2's robust adaptability to 1-point prompts underscores its potential for&#10;downstream surgical tasks with limited prompt requirements.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.03286" label="2408.03286">
-        <attvalues>
-          <attvalue for="0" value="Biomedical SAM 2: Segment Anything in Biomedical Images and Videos" />
-          <attvalue for="1" value="  Medical image segmentation and video object segmentation are essential for&#10;diagnosing and analyzing diseases by identifying and measuring biological&#10;structures. Recent advances in natural domain have been driven by foundation&#10;models like the Segment Anything Model 2 (SAM-2). To explore the performance of&#10;SAM-2 in biomedical applications, we designed three evaluation pipelines for&#10;single-frame 2D image segmentation, multi-frame 3D image segmentation and&#10;multi-frame video segmentation with varied prompt designs, revealing SAM-2's&#10;limitations in medical contexts. Consequently, we developed BioSAM-2, an&#10;enhanced foundation model optimized for biomedical data based on SAM-2. Our&#10;experiments show that BioSAM-2 not only surpasses the performance of existing&#10;state-of-the-art foundation models but also matches or even exceeds specialist&#10;models, demonstrating its efficacy and potential in the medical domain.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.03604" label="1704.03604">
-        <attvalues>
-          <attvalue for="0" value="Instance-Level Salient Object Segmentation" />
-          <attvalue for="1" value="  Image saliency detection has recently witnessed rapid progress due to deep&#10;convolutional neural networks. However, none of the existing methods is able to&#10;identify object instances in the detected salient regions. In this paper, we&#10;present a salient instance segmentation method that produces a saliency mask&#10;with distinct object instance labels for an input image. Our method consists of&#10;three steps, estimating saliency map, detecting salient object contours and&#10;identifying salient object instances. For the first two steps, we propose a&#10;multiscale saliency refinement network, which generates high-quality salient&#10;region masks and salient object contours. Once integrated with multiscale&#10;combinatorial grouping and a MAP-based subset optimization framework, our&#10;method can generate very promising salient object instance segmentation&#10;results. To promote further research and evaluation of salient instance&#10;segmentation, we also construct a new database of 1000 images and their&#10;pixelwise salient instance annotations. Experimental results demonstrate that&#10;our proposed method is capable of achieving state-of-the-art performance on all&#10;public benchmarks for salient region detection as well as on our new dataset&#10;for salient instance segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.02255" label="2207.02255">
-        <attvalues>
-          <attvalue for="0" value="OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers" />
-          <attvalue for="1" value="  We present OSFormer, the first one-stage transformer framework for&#10;camouflaged instance segmentation (CIS). OSFormer is based on two key designs.&#10;First, we design a location-sensing transformer (LST) to obtain the location&#10;label and instance-aware parameters by introducing the location-guided queries&#10;and the blend-convolution feedforward network. Second, we develop a&#10;coarse-to-fine fusion (CFF) to merge diverse context information from the LST&#10;encoder and CNN backbone. Coupling these two components enables OSFormer to&#10;efficiently blend local features and long-range context dependencies for&#10;predicting camouflaged instances. Compared with two-stage frameworks, our&#10;OSFormer reaches 41% AP and achieves good convergence efficiency without&#10;requiring enormous training data, i.e., only 3,040 samples under 60 epochs.&#10;Code link: https://github.com/PJLallen/OSFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.04614" label="2207.04614">
-        <attvalues>
-          <attvalue for="0" value="Instance Shadow Detection with A Single-Stage Detector" />
-          <attvalue for="1" value="  This paper formulates a new problem, instance shadow detection, which aims to&#10;detect shadow instance and the associated object instance that cast each shadow&#10;in the input image. To approach this task, we first compile a new dataset with&#10;the masks for shadow instances, object instances, and shadow-object&#10;associations. We then design an evaluation metric for quantitative evaluation&#10;of the performance of instance shadow detection. Further, we design a&#10;single-stage detector to perform instance shadow detection in an end-to-end&#10;manner, where the bidirectional relation learning module and the deformable&#10;maskIoU head are proposed in the detector to directly learn the relation&#10;between shadow instances and object instances and to improve the accuracy of&#10;the predicted masks. Finally, we quantitatively and qualitatively evaluate our&#10;method on the benchmark dataset of instance shadow detection and show the&#10;applicability of our method on light direction estimation and photo editing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.02604" label="1802.02604">
-        <attvalues>
-          <attvalue for="0" value="An Unsupervised Learning Model for Deformable Medical Image Registration" />
-          <attvalue for="1" value="  We present a fast learning-based algorithm for deformable, pairwise 3D&#10;medical image registration. Current registration methods optimize an objective&#10;function independently for each pair of images, which can be time-consuming for&#10;large data. We define registration as a parametric function, and optimize its&#10;parameters given a set of images from a collection of interest. Given a new&#10;pair of scans, we can quickly compute a registration field by directly&#10;evaluating the function using the learned parameters. We model this function&#10;using a convolutional neural network (CNN), and use a spatial transform layer&#10;to reconstruct one image from another while imposing smoothness constraints on&#10;the registration field. The proposed method does not require supervised&#10;information such as ground truth registration fields or anatomical landmarks.&#10;We demonstrate registration accuracy comparable to state-of-the-art 3D image&#10;registration, while operating orders of magnitude faster in practice. Our&#10;method promises to significantly speed up medical image analysis and processing&#10;pipelines, while facilitating novel directions in learning-based registration&#10;and its applications. Our code is available at&#10;https://github.com/balakg/voxelmorph .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.14557" label="2004.14557">
-        <attvalues>
-          <attvalue for="0" value="Learning Deformable Image Registration from Optimization: Perspective,&#10;  Modules, Bilevel Training and Beyond" />
-          <attvalue for="1" value="  Conventional deformable registration methods aim at solving an optimization&#10;model carefully designed on image pairs and their computational costs are&#10;exceptionally high. In contrast, recent deep learning based approaches can&#10;provide fast deformation estimation. These heuristic network architectures are&#10;fully data-driven and thus lack explicit geometric constraints, e.g.,&#10;topology-preserving, which are indispensable to generate plausible&#10;deformations. We design a new deep learning based framework to optimize a&#10;diffeomorphic model via multi-scale propagation in order to integrate&#10;advantages and avoid limitations of these two categories of approaches.&#10;Specifically, we introduce a generic optimization model to formulate&#10;diffeomorphic registration and develop a series of learnable architectures to&#10;obtain propagative updating in the coarse-to-fine feature space. Moreover, we&#10;propose a novel bilevel self-tuned training strategy, allowing efficient search&#10;of task-specific hyper-parameters. This training strategy increases the&#10;flexibility to various types of data while reduces computational and human&#10;burdens. We conduct two groups of image registration experiments on 3D volume&#10;datasets including image-to-atlas registration on brain MRI data and&#10;image-to-image registration on liver CT data. Extensive results demonstrate the&#10;state-of-the-art performance of the proposed method with diffeomorphic&#10;guarantee and extreme efficiency. We also apply our framework to challenging&#10;multi-modal image registration, and investigate how our registration to support&#10;the down-streaming tasks for medical image analysis including multi-modal&#10;fusion and image segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.14164" label="2206.14164">
-        <attvalues>
-          <attvalue for="0" value="Visualizing and Alleviating the Effect of Radial Distortion on Camera&#10;  Calibration Using Principal Lines" />
-          <attvalue for="1" value="  Preparing appropriate images for camera calibration is crucial to obtain&#10;accurate results. In this paper, new suggestions for preparing such data to&#10;alleviate the adverse effect of radial distortion for a calibration procedure&#10;using principal lines are developed through the investigations of: (i)&#10;identifying directions of checkerboard movements in an image which will result&#10;in maximum (and minimum) influence on the calibration results, and (ii)&#10;inspecting symmetry and monotonicity of such effect in (i) using the above&#10;principal lines. Accordingly, it is suggested that the estimation of principal&#10;point should based on linearly independent pairs of nearly parallel principal&#10;lines, with a member in each pair corresponds to a near 180-degree rotation (in&#10;the image plane) of the other. Experimental results show that more robust and&#10;consistent calibration results for the foregoing estimation can actually be&#10;obtained, compared with the renowned algebraic methods which estimate&#10;distortion parameters explicitly.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Camera calibration estimates intrinsic and extrinsic camera parameters to associate an image pixel to the corresponding 3D point in the real world. In general, accurate calibration is a necessity for accurate representation of the real world using information extracted from the captured images and thus play a crucial role in computer vision applications such as intelligent video surveillance \cite{Appli_video_surveillance}, autonomous driving systems \cite{Appli_auto_driving_01, Appli_auto_driving_02} and accurate measurements \cite{Appli_accurate_measurement}.\par&#10;&#10;For more than two decades, Zhang’s method \cite{zhang2000,zhang1998} has been the most popular solution to camera calibration due to its simplicity of only using images of a checkerboard (CB), in various poses, to estimate all camera parameters. The method is based on an algebraic closed-form solution in getting an initial estimation, before a nonlinear optimization, which minimizes the re-projection error of corner points of the CB on the images, is performed to finalize the estimation of camera parameters including distortion coefficients. Incidentally, it is suggested in \cite{zhang2000} that the CB patterns should cover whole image, rather than just occupying a small part of the image, for more accurate estimate of distortion coefficients.\par&#10;&#10;Under the foregoing condition of whole image coverage, corners of the CB pattern may still concentrate on the less distorted center part of the image plane. Accordingly, the idea is extended in \cite{distribute02} to suggest that each image should be completely occupied by (part of) the pattern, with essentially no background area left, for better estimation of camera distortion. As the CB is partially visible, special patterns are included in the new CB pattern to simplify the identification of its visible corner points. In addition, somehow similar suggestion is proposed in \cite{distribute01}, i.e., corner points should be distributed uniformly, on the average, for a set of CB images without considering the distortion explicitly.\par&#10;&#10;Given a set of CB patterns, there are calibration methods which suggest additional CB poses for better calibration results. Assuming Gaussian noises, the next best pose is directly estimated in \cite{estimate_next_01} by minimizing both the calibration and corner uncertainty. However, system errors from the distortion may cause the method to fail. Such errors are further decreased in \cite{estimate_next_02} by increasing parameters of the pinhole camera model but without guaranteeing low system errors. In \cite{estimate_next_03}, a score function is proposed to estimate an optimal pose set in offline mode. However, such scheme is unrealistic as some camera parameters, which are yet to be determined, are used in the score function. Besides, the method does not consider distortion explicitly either.\par&#10;&#10;Unlike the above methods which are based on Zhang's algebraic formulation to estimate all camera parameters simultaneously. A novel geometry-based camera calibration is proposed in \cite{chuang2021} which estimated camera geometry sequentially by first finding principal lines (PLs). A PL of a CB, e.g., the $y$-axis shown in Fig. \ref{fig:relationship_of_pimg_and_pchkb} (a), corresponds to the intersectionof image plane and the plane containing camera optical axis and perpendicular to both the CB and the image. As a PL thus defined should pass through the principal point (PP), a set of linearly independent PLs are used in \cite{chuang2021} to find the PP as their intersection. Thus, PLs with problematic intersections can simply be identified as outliers without resorting to probabilistic RANSAC \cite{RANSAC_01, RANSAC_02}.\par&#10;&#10;In this paper, variation of PP location resulted from different sets of PLs, each having the above property of symmetry, are used to visualize of the adverse&#10;effect of radial distortion on camera calibration, while novel and systematic arrangements of CB poses are suggested accordingly for alleviating such effect. The main contributions of the paper include:&#10;\begin{enumerate}[label=(\roman*)]&#10; \itemsep=-4pt&#10; \item (Directionality analysis) Identifying directions of CB shift in the image which will result in maximum deflection of the PL due to radial distortion. &#10; \item (Monotonicity analysis) Demonstrating that the deflection in \ref{item:first}, which can affect PP estimate, will increase monotonically with the amount of CB shift. &#10; \item Providing suggestions of CB poses, which are easy to set up, for minimizing the effect of distortion. &#10;\end{enumerate}&#10;&#10;In Sec. 2, the setup of a reference set of CB poses used in \cite{chuang2021} are reviewed, wherein near perfect PLs (and the PP) can be obtained without considering image distortion. In Sec. 3, variation of this reference is employed to visualize effect of the distortion, before \ref{item:third} is accomplished. Finally, some experimental results and concluding remarks are provided.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Principal Line Estimation, Mathematics, Image Distortion Analysis, Image Processing, Physics, Camera Calibration, Computer Science, Geometric Optics" />
-        </attvalues>
-      </node>
-      <node id="1811.03264" label="1811.03264">
-        <attvalues>
-          <attvalue for="0" value="Calibration Wizard: A Guidance System for Camera Calibration Based on&#10;  Modelling Geometric and Corner Uncertainty" />
-          <attvalue for="1" value="  It is well known that the accuracy of a calibration depends strongly on the&#10;choice of camera poses from which images of a calibration object are acquired.&#10;We present a system -- Calibration Wizard -- that interactively guides a user&#10;towards taking optimal calibration images. For each new image to be taken, the&#10;system computes, from all previously acquired images, the pose that leads to&#10;the globally maximum reduction of expected uncertainty on intrinsic parameters&#10;and then guides the user towards that pose. We also show how to incorporate&#10;uncertainty in corner point position in a novel principled manner, for both,&#10;calibration and computation of the next best pose. Synthetic and real-world&#10;experiments are performed to demonstrate the effectiveness of Calibration&#10;Wizard.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.02392" label="2408.02392">
-        <attvalues>
-          <attvalue for="0" value="MaFreeI2P: A Matching-Free Image-to-Point Cloud Registration Paradigm&#10;  with Active Camera Pose Retrieval" />
-          <attvalue for="1" value="  Image-to-point cloud registration seeks to estimate their relative camera&#10;pose, which remains an open question due to the data modality gaps. The recent&#10;matching-based methods tend to tackle this by building 2D-3D correspondences.&#10;In this paper, we reveal the information loss inherent in these methods and&#10;propose a matching-free paradigm, named MaFreeI2P. Our key insight is to&#10;actively retrieve the camera pose in SE(3) space by contrasting the geometric&#10;features between the point cloud and the query image. To achieve this, we first&#10;sample a set of candidate camera poses and construct their cost volume using&#10;the cross-modal features. Superior to matching, cost volume can preserve more&#10;information and its feature similarity implicitly reflects the confidence level&#10;of the sampled poses. Afterwards, we employ a convolutional network to&#10;adaptively formulate a similarity assessment function, where the input cost&#10;volume is further improved by filtering and pose-based weighting. Finally, we&#10;update the camera pose based on the similarity scores, and adopt a heuristic&#10;strategy to iteratively shrink the pose sampling space for convergence. Our&#10;MaFreeI2P achieves a very competitive registration accuracy and recall on the&#10;KITTI-Odometry and Apollo-DaoxiangLake datasets.&#10;" />
-          <attvalue for="2" value="&#10;Given a 3D scene point cloud and an RGB image capturing part of the same scene, image-to-point cloud registration aims at estimating the camera pose with respect to the coordinate frame of the 3D scene, as illustrated in Fig. \ref{fig1} (a). It has various computer vision applications, including but not limited to camera relocalization in a pre-built 3D map, 3D reconstruction, and camera-LiDAR calibration. However, RGB images capture colors and textures with 2D grid data, whereas point clouds represent 3D geometry in an unordered format. This disparity poses significant challenges to the task.&#10;&#10;Following the matching-based approaches used for same-modal registration in images \cite{sarlin2020superglue, 2021LoFTR, Xu_2022_CVPR, xie2024deepmatcher} or point clouds \cite{lu2021hregnet, huang2021predator, qin2022geometric, wu2023sacf}, the mainstream pipeline (as illustrated in Fig. \ref{fig1} (b)) for image-to-point cloud registration is to first extract the 2D-3D correspondences with pseudo-siamese neural networks \cite{xia2021pseudo}. Subsequently, the camera pose is estimated using a RANSAC-based PnP solver \cite{fischler1981random, lepetit2009ep}. Recent algorithms such as 2D3D-MatchNet \cite{feng20192d3d}, CorrI2P \cite{ren2022corri2p} and CFI2P \cite{yao2023cfi2p} fall into this category. Although they solve the task by combining deep learning with traditional geometric vision model \cite{zheng2013revisiting}, they are still plagued by at least two issues. 1) Analogous to the concept of hard label in knowledge distillation \cite{gou2021knowledge}, a correspondence can be seen as a hard result that only retains the information of the matched point-pixel pair and discards the beneficial information between other points or pixels. 2) The RANSAC-based PnP solver eliminates correspondence outliers by iterative sampling, so the pose estimation is derived from the local perspectives on the sampled subsets.&#10;&#10;There are also some matching-free algorithms \cite{li2021deepi2p, 9799751} that bypass building 2D-3D correspondences. DeepI2P \cite{li2021deepi2p} casts the registration process into a two-stage point classification and pose optimization framework. However, it suffers from low registration accuracy because the classification merely determines whether a 3D point is inside the image frustum, which is insufficient for precise pose fine-tuning. EFGHNet \cite{9799751} adopts a divide-and-conquer strategy to estimate rotation and translation sequentially. It projects the point cloud onto a 360° panoramic depth map and then seeks out the shared geometric features between this map and the image. Thus, EFGHNet is confined to scenarios with small translation, since significant movement would destroy the shared geometric features. So far, these matching-free methods significantly lagged behind their matching-based counterparts.&#10;&#10;In this paper, we introduce MaFreeI2P, a novel Matching-Free Image-to-Point cloud registration paradigm that overcomes the above limitations. We still adopt a pseudo-siamese network as the backbone to learn 2D and 3D features. Inspired by the success of cost volume for depth estimation \cite{tian2023dps} and stucture-from-motion \cite{wei2023deepsfm}, we fuse the cross-modal features to construct a pose-based cost volume, rather than establishing 2D-3D correspondences that lead to substantial information loss. Specifically, we first uniformly sample a set of discrete candidate camera poses around the current or initial pose, and use them to project the point cloud into various 2D views. For each sampled camera pose, the 3D features are aggregated according to its corresponding 2D view, and then concatenated with 2D features to form an individual cost volume unit. All the units are fed into a convolutional network to calculate the similarity scores, and we update the current camera pose as the one with highest score. The above process of pose sampling and similarity assessment is repeated iteratively, as shown in Fig. \ref{fig1} (c). Such intuitive pipeline has at least three advantages. 1) It has excellent interpretability, reflecting the instinctive process of humans in adjusting and observing to align two objects. 2) By building cost volume, it can retain and exploit more information than the matching-based pipeline as illustrated in Fig. \ref{fig2}. 3) Since no RANSAC strategy is applied to the points and pixels, they are all taken into global consideration within each cost volume unit.&#10;&#10;Beyond the basic paradigm, to guarantee convergence and achieve a global optimum, we emulate classic gradient descent algorithms for sampling camera poses. It begins with an initially large sampling space and step size, which are then reduced iteratively. To improve the feature quality of the basic cost volume, we opt to identify the 3D points outside the image frustum and zero out the corresponding features. Furthermore, to focus on the 2D and 3D features that are more critical within the cost volume, we assign pose-based weights to different regions, thereby differentiating their importance in the computation of similarity scores.&#10;&#10;In summary, our main contributions include: 1) We propose a novel matching-free image-to-point cloud registration paradigm, retrieving the camera pose in SE(3) space via active sampling and similarity assessment. 2) We build camera pose-based cost volume with the cross-modal features, preserving richer information than 2D-3D matching, where each cost unit implicitly reflects the confidence level of a sampled pose. 3) We design a convolutional network to adaptively formulate the similarity assessment function, where the input cost volume is further improved by filtering and pose-based weighting. 4) We achieve state-of-the-art performance in registration accuracy and recall on the KITTI-Odometry \cite{geiger2013vision} and Apollo-DaoxiangLake \cite{zhou2020da4ad} datasets.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Point Cloud Processing, Image Registration, Computer Vision, Engineering, Machine Learning, Mathematics, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2107.11992" label="2107.11992">
-        <attvalues>
-          <attvalue for="0" value="HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point&#10;  Cloud Registration" />
-          <attvalue for="1" value="  Point cloud registration is a fundamental problem in 3D computer vision.&#10;Outdoor LiDAR point clouds are typically large-scale and complexly distributed,&#10;which makes the registration challenging. In this paper, we propose an&#10;efficient hierarchical network named HRegNet for large-scale outdoor LiDAR&#10;point cloud registration. Instead of using all points in the point clouds,&#10;HRegNet performs registration on hierarchically extracted keypoints and&#10;descriptors. The overall framework combines the reliable features in deeper&#10;layer and the precise position information in shallower layers to achieve&#10;robust and precise registration. We present a correspondence network to&#10;generate correct and accurate keypoints correspondences. Moreover, bilateral&#10;consensus and neighborhood consensus are introduced for keypoints matching and&#10;novel similarity features are designed to incorporate them into the&#10;correspondence network, which significantly improves the registration&#10;performance. Besides, the whole network is also highly efficient since only a&#10;small number of keypoints are used for registration. Extensive experiments are&#10;conducted on two large-scale outdoor LiDAR point cloud datasets to demonstrate&#10;the high accuracy and efficiency of the proposed HRegNet. The project website&#10;is https://ispc-group.github.io/hregnet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.00896" label="2105.00896">
-        <attvalues>
-          <attvalue for="0" value="Pseudo Siamese Network for Few-shot Intent Generation" />
-          <attvalue for="1" value="  Few-shot intent detection is a challenging task due to the scare annotation&#10;problem. In this paper, we propose a Pseudo Siamese Network (PSN) to generate&#10;labeled data for few-shot intents and alleviate this problem. PSN consists of&#10;two identical subnetworks with the same structure but different weights: an&#10;action network and an object network. Each subnetwork is a transformer-based&#10;variational autoencoder that tries to model the latent distribution of&#10;different components in the sentence. The action network is learned to&#10;understand action tokens and the object network focuses on object-related&#10;expressions. It provides an interpretable framework for generating an utterance&#10;with an action and an object existing in a given intent. Experiments on two&#10;real-world datasets show that PSN achieves state-of-the-art performance for the&#10;generalized few shot intent detection task.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.09742" label="1904.09742">
-        <attvalues>
-          <attvalue for="0" value="2D3D-MatchNet: Learning to Match Keypoints Across 2D Image and 3D Point&#10;  Cloud" />
-          <attvalue for="1" value="  Large-scale point cloud generated from 3D sensors is more accurate than its&#10;image-based counterpart. However, it is seldom used in visual pose estimation&#10;due to the difficulty in obtaining 2D-3D image to point cloud correspondences.&#10;In this paper, we propose the 2D3D-MatchNet - an end-to-end deep network&#10;architecture to jointly learn the descriptors for 2D and 3D keypoint from image&#10;and point cloud, respectively. As a result, we are able to directly match and&#10;establish 2D-3D correspondences from the query image and 3D point cloud&#10;reference map for visual pose estimation. We create our Oxford 2D-3D Patches&#10;dataset from the Oxford Robotcar dataset with the ground truth camera poses and&#10;2D-3D image to point cloud correspondences for training and testing the deep&#10;network. Experimental results verify the feasibility of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.05483" label="2207.05483">
-        <attvalues>
-          <attvalue for="0" value="CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence" />
-          <attvalue for="1" value="  Motivated by the intuition that the critical step of localizing a 2D image in&#10;the corresponding 3D point cloud is establishing 2D-3D correspondence between&#10;them, we propose the first feature-based dense correspondence framework for&#10;addressing the image-to-point cloud registration problem, dubbed CorrI2P, which&#10;consists of three modules, i.e., feature embedding, symmetric overlapping&#10;region detection, and pose estimation through the established correspondence.&#10;Specifically, given a pair of a 2D image and a 3D point cloud, we first&#10;transform them into high-dimensional feature space and feed the resulting&#10;features into a symmetric overlapping region detector to determine the region&#10;where the image and point cloud overlap each other. Then we use the features of&#10;the overlapping regions to establish the 2D-3D correspondence before running&#10;EPnP within RANSAC to estimate the camera's pose. Experimental results on KITTI&#10;and NuScenes datasets show that our CorrI2P outperforms state-of-the-art&#10;image-to-point cloud registration methods significantly. We will make the code&#10;publicly available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.05525" label="2006.05525">
-        <attvalues>
-          <attvalue for="0" value="Knowledge Distillation: A Survey" />
-          <attvalue for="1" value="  In recent years, deep neural networks have been successful in both industry&#10;and academia, especially for computer vision tasks. The great success of deep&#10;learning is mainly due to its scalability to encode large-scale data and to&#10;maneuver billions of model parameters. However, it is a challenge to deploy&#10;these cumbersome deep models on devices with limited resources, e.g., mobile&#10;phones and embedded devices, not only because of the high computational&#10;complexity but also the large storage requirements. To this end, a variety of&#10;model compression and acceleration techniques have been developed. As a&#10;representative type of model compression and acceleration, knowledge&#10;distillation effectively learns a small student model from a large teacher&#10;model. It has received rapid increasing attention from the community. This&#10;paper provides a comprehensive survey of knowledge distillation from the&#10;perspectives of knowledge categories, training schemes, teacher-student&#10;architecture, distillation algorithms, performance comparison and applications.&#10;Furthermore, challenges in knowledge distillation are briefly reviewed and&#10;comments on future research are discussed and forwarded.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.03026" label="2003.03026">
-        <attvalues>
-          <attvalue for="0" value="DA4AD: End-to-End Deep Attention-based Visual Localization for&#10;  Autonomous Driving" />
-          <attvalue for="1" value="  We present a visual localization framework based on novel deep attention&#10;aware features for autonomous driving that achieves centimeter level&#10;localization accuracy. Conventional approaches to the visual localization&#10;problem rely on handcrafted features or human-made objects on the road. They&#10;are known to be either prone to unstable matching caused by severe appearance&#10;or lighting changes, or too scarce to deliver constant and robust localization&#10;results in challenging scenarios. In this work, we seek to exploit the deep&#10;attention mechanism to search for salient, distinctive and stable features that&#10;are good for long-term matching in the scene through a novel end-to-end deep&#10;neural network. Furthermore, our learned feature descriptors are demonstrated&#10;to be competent to establish robust matches and therefore successfully estimate&#10;the optimal camera poses with high precision. We comprehensively validate the&#10;effectiveness of our method using a freshly collected dataset with high-quality&#10;ground truth trajectories and hardware synchronization between sensors. Results&#10;demonstrate that our method achieves a competitive localization accuracy when&#10;compared to the LiDAR-based localization solutions under various challenging&#10;circumstances, leading to a potential low-cost localization solution for&#10;autonomous driving.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.18175" label="2402.18175">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Spatially Variant PSF Estimation for Aberration-Aware&#10;  Depth-from-Defocus" />
-          <attvalue for="1" value="  In this paper, we address the task of aberration-aware depth-from-defocus&#10;(DfD), which takes account of spatially variant point spread functions (PSFs)&#10;of a real camera. To effectively obtain the spatially variant PSFs of a real&#10;camera without requiring any ground-truth PSFs, we propose a novel&#10;self-supervised learning method that leverages the pair of real sharp and&#10;blurred images, which can be easily captured by changing the aperture setting&#10;of the camera. In our PSF estimation, we assume rotationally symmetric PSFs and&#10;introduce the polar coordinate system to more accurately learn the PSF&#10;estimation network. We also handle the focus breathing phenomenon that occurs&#10;in real DfD situations. Experimental results on synthetic and real data&#10;demonstrate the effectiveness of our method regarding both the PSF estimation&#10;and the depth estimation.&#10;" />
-          <attvalue for="2" value="&#10;Depth-from-defocus (DfD) is one of the well-adopted depth sensing techniques and estimates the scene depth from focal stack images based on the relationship between the depth and the defocus amount. &#10;Existing DfD methods~\cite{hiura1998depth, suwajanakorn2015depth, subbarao1994depth, song2018depth, maximov2020focus, wang2021bridging, yang2022deep} often assume a simplistic spatially invariant point spread function~(PSF) due to the difficulty in acquiring spatially variant PSFs of a real camera with non-ideal optics including lens distortions and optical aberrations. However, the assumption of a spatially invariant PSF is detrimental to deep-learning-based DfD methods, since they are typically trained using synthetic focal stack images assuming accurate PSFs of a real camera.&#10;&#10;Existing spatially variant PSF estimation methods are classified as optimization-based methods~\cite{mannan2016blur, mannan2016good, joshi2008psf, mosleh2015camera, delbracio2012non} and learning-based methods~\cite{yang2023aberration, tseng2021differentiable}. The optimization-based methods use a real blurred image of a known calibration pattern to derive the PSF of each pixel that minimizes the difference between the real blurred pattern and the synthetically blurred pattern by the PSF. The learning-based methods train a neural network to infer PSFs of arbitrary pixels assuming that ground-truth PSFs are available for some of the image pixels, which are typically obtained using detailed lens parameters. These existing methods still have some limitations in practicability, such as the necessity of precise pattern alignment and ground-truth PSFs. &#10;&#10;In this paper, we propose a novel deep-learning-based spatially variant PSF estimation method. For training a PSF estimation network~(PSF-Net), we leverage the pair of real sharp and real blurred images, as shown in Fig.~\ref{fig:pipeline}(a). This brings us two main benefits: (i)~Our method is self-supervised without requiring ground-truth PSFs because the pair of sharp and blurred images can be conveniently captured using a real camera by adjusting the aperture size. (ii)~Our method is image alignment-free and naturally handles the effect of focus breathing, which is the phenomenon that the scale of image magnification (i.e., field-of-view) slightly differs according to the focus distance (details are in Sec.~\ref{sec:psfnet}).&#10;In our PSF-Net training, we introduce the polar coordinate PSF model~\cite{maeda2005integrating} to incorporate the rotation symmetry of the PSFs, which improves the accuracy of the PSF estimation. Using the estimated spatially variant PSF maps for each focus distance of the focal stack, we generate synthetic focal stack images to train a DfD network, where we additionally provide the image height~(IH) information as a positional input to make the DfD network learn the depth considering the spatial variations of the PSFs, as illustrated in Fig.~\ref{fig:pipeline}(b). We experimentally demonstrate the effectiveness of our method in terms of both the PSF estimation and the depth estimation.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Optics, Physics, Image Processing, Depth Estimation, Camera Calibration, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2303.04654" label="2303.04654">
-        <attvalues>
-          <attvalue for="0" value="Aberration-Aware Depth-from-Focus" />
-          <attvalue for="1" value="  Computer vision methods for depth estimation usually use simple camera models&#10;with idealized optics. For modern machine learning approaches, this creates an&#10;issue when attempting to train deep networks with simulated data, especially&#10;for focus-sensitive tasks like Depth-from-Focus. In this work, we investigate&#10;the domain gap caused by off-axis aberrations that will affect the decision of&#10;the best-focused frame in a focal stack. We then explore bridging this domain&#10;gap through aberration-aware training (AAT). Our approach involves a&#10;lightweight network that models lens aberrations at different positions and&#10;focus distances, which is then integrated into the conventional network&#10;training pipeline. We evaluate the generality of pretrained models on both&#10;synthetic and real-world data. Our experimental results demonstrate that the&#10;proposed AAT scheme can improve depth estimation accuracy without fine-tuning&#10;the model or modifying the network architecture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.02861" label="2309.02861">
-        <attvalues>
-          <attvalue for="0" value="Image Aesthetics Assessment via Learnable Queries" />
-          <attvalue for="1" value="  Image aesthetics assessment (IAA) aims to estimate the aesthetics of images.&#10;Depending on the content of an image, diverse criteria need to be selected to&#10;assess its aesthetics. Existing works utilize pre-trained vision backbones&#10;based on content knowledge to learn image aesthetics. However, training those&#10;backbones is time-consuming and suffers from attention dispersion. Inspired by&#10;learnable queries in vision-language alignment, we propose the Image Aesthetics&#10;Assessment via Learnable Queries (IAA-LQ) approach. It adapts learnable queries&#10;to extract aesthetic features from pre-trained image features obtained from a&#10;frozen image encoder. Extensive experiments on real-world data demonstrate the&#10;advantages of IAA-LQ, beating the best state-of-the-art method by 2.2% and 2.1%&#10;in terms of SRCC and PLCC, respectively.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image aesthetics assessment (IAA) is a computer vision task that aims to evaluate the aesthetic quality of images. Such a capability is beneficial for downstream applications including image recommendation, enhancement, retrieval, and generation~\cite{zhang2021comprehensive}. Due to the inherent subjectivity and ambiguity associated with image aesthetics, the ground truth of image aesthetics is usually determined by the opinions of different reviewers in the form of the mean opinion score (MOS) or the distribution of opinion scores (DOS).&#10;&#10;Depending on the content of the image, there can be different emphases involved in aesthetics assessment. Early works proposed to split the images into different semantic groups and extract different sets of aesthetic features~\cite{tang2013content, kao2016hierarchical, wang2016multi}. The features can be handcrafted under the guidance of photography rules~\cite{tang2013content}, or extracted by deep neural networks~\cite{kao2016hierarchical, wang2016multi}. However, there may be special cases where an image does not belong to any predefined semantic group. Explicitly splitting these images based on semantics can result in the relationships among different semantic groups being overlooked.&#10;&#10;Therefore, later works~\cite{hosu2019effective, he2022rethinking, xiong2023federated} attempted to implicitly extract aesthetic features from pre-trained semantic backbones. Since fine-tuning the entire backbone is computationally expensive~\cite{hosu2019effective, xiong2023federated} and might lead to attention dispersion~\cite{he2022rethinking}, the backbone is only used as a feature extractor. However, since the vision backbones are pre-trained primarily for image classification~\cite{hosu2019effective, xiong2023federated} or scene recognition~\cite{he2022rethinking}, they lack knowledge regarding the aesthetic attributes (e.g., composition) that are less related to semantics. Moreover, some of these works~\cite{hosu2019effective, xiong2023federated} require input images in full resolution and additional feature extraction stages in advance, which lack efficiency and practical applicability.&#10;&#10;More recently, with the prevalence of pre-trained large vision-language models, works that utilize such models together with specially designed prompts for IAA are starting to emerge~\cite{wang2023exploring, ke2023vila}. The vision-language models used are either pre-trained solely on general image-text pairs~\cite{wang2023exploring}, or further pre-trained on aesthetic image-text pairs~\cite{ke2023vila}. Either way, their pre-trained models are not limited to aesthetic-related semantic patterns but also cover relatively abstract knowledge related to image aesthetics. However, they use relatively simple single prompts (e.g., ``good image&quot;) or prompt pairs (e.g., ``good photo&quot; and ``bad photo&quot;) to extract aesthetic-related knowledge, which cannot deal with complex IAA tasks.&#10;&#10;Inspired by BLIP-2~\cite{li2023blip}, a vision-language pre-training model that uses learnable queries to align vision and language pre-trained features extracted from frozen unimodal models, we propose the \underline{I}mage \underline{A}esthetics \underline{A}ssessment via \underline{L}earnable \underline{Q}ueries ({\tt{IAA-LQ}}) approach.&#10;It trains learnable queries to extract aesthetic features from the pre-trained vision backbones. With a flexible quantity, the learnable queries can extract the most aesthetics-related image patterns from the frozen vision backbone. Extensive experiments on real-world data demonstrate the advantages of {\tt{IAA-LQ}}, beating the best state-of-the-art method by 2.2\% and 2.1\% in terms of Spearman's rank correlation coefficient (SRCC) and Pearson linear correlation coefficient (PLCC), respectively.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Computer Vision, Image Aesthetics Assessment, Mathematics, Artificial Intelligence, Cognitive Science, Deep Learning Methods, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1904.01382" label="1904.01382">
-        <attvalues>
-          <attvalue for="0" value="Effective Aesthetics Prediction with Multi-level Spatially Pooled&#10;  Features" />
-          <attvalue for="1" value="  We propose an effective deep learning approach to aesthetics quality&#10;assessment that relies on a new type of pre-trained features, and apply it to&#10;the AVA data set, the currently largest aesthetics database. While previous&#10;approaches miss some of the information in the original images, due to taking&#10;small crops, down-scaling or warping the originals during training, we propose&#10;the first method that efficiently supports full resolution images as an input,&#10;and can be trained on variable input sizes. This allows us to significantly&#10;improve upon the state of the art, increasing the Spearman rank-order&#10;correlation coefficient (SRCC) of ground-truth mean opinion scores (MOS) from&#10;the existing best reported of 0.612 to 0.756. To achieve this performance, we&#10;extract multi-level spatially pooled (MLSP) features from all convolutional&#10;blocks of a pre-trained InceptionResNet-v2 network, and train a custom shallow&#10;Convolutional Neural Network (CNN) architecture on these new features.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.12396" label="2207.12396">
-        <attvalues>
-          <attvalue for="0" value="Exploring CLIP for Assessing the Look and Feel of Images" />
-          <attvalue for="1" value="  Measuring the perception of visual content is a long-standing problem in&#10;computer vision. Many mathematical models have been developed to evaluate the&#10;look or quality of an image. Despite the effectiveness of such tools in&#10;quantifying degradations such as noise and blurriness levels, such&#10;quantification is loosely coupled with human language. When it comes to more&#10;abstract perception about the feel of visual content, existing methods can only&#10;rely on supervised models that are explicitly trained with labeled data&#10;collected via laborious user study. In this paper, we go beyond the&#10;conventional paradigms by exploring the rich visual language prior encapsulated&#10;in Contrastive Language-Image Pre-training (CLIP) models for assessing both the&#10;quality perception (look) and abstract perception (feel) of images in a&#10;zero-shot manner. In particular, we discuss effective prompt designs and show&#10;an effective prompt pairing strategy to harness the prior. We also provide&#10;extensive experiments on controlled datasets and Image Quality Assessment (IQA)&#10;benchmarks. Our results show that CLIP captures meaningful priors that&#10;generalize well to different perceptual assessments. Code is avaliable at&#10;https://github.com/IceClear/CLIP-IQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.14302" label="2303.14302">
-        <attvalues>
-          <attvalue for="0" value="VILA: Learning Image Aesthetics from User Comments with Vision-Language&#10;  Pretraining" />
-          <attvalue for="1" value="  Assessing the aesthetics of an image is challenging, as it is influenced by&#10;multiple factors including composition, color, style, and high-level semantics.&#10;Existing image aesthetic assessment (IAA) methods primarily rely on&#10;human-labeled rating scores, which oversimplify the visual aesthetic&#10;information that humans perceive. Conversely, user comments offer more&#10;comprehensive information and are a more natural way to express human opinions&#10;and preferences regarding image aesthetics. In light of this, we propose&#10;learning image aesthetics from user comments, and exploring vision-language&#10;pretraining methods to learn multimodal aesthetic representations.&#10;Specifically, we pretrain an image-text encoder-decoder model with&#10;image-comment pairs, using contrastive and generative objectives to learn rich&#10;and generic aesthetic semantics without human labels. To efficiently adapt the&#10;pretrained model for downstream IAA tasks, we further propose a lightweight&#10;rank-based adapter that employs text as an anchor to learn the aesthetic&#10;ranking concept. Our results show that our pretrained aesthetic vision-language&#10;model outperforms prior works on image aesthetic captioning over the&#10;AVA-Captions dataset, and it has powerful zero-shot capability for aesthetic&#10;tasks such as zero-shot style classification and zero-shot IAA, surpassing many&#10;supervised baselines. With only minimal finetuning parameters using the&#10;proposed adapter module, our model achieves state-of-the-art IAA performance&#10;over the AVA dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2301.12597" label="2301.12597">
-        <attvalues>
-          <attvalue for="0" value="BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image&#10;  Encoders and Large Language Models" />
-          <attvalue for="1" value="  The cost of vision-and-language pre-training has become increasingly&#10;prohibitive due to end-to-end training of large-scale models. This paper&#10;proposes BLIP-2, a generic and efficient pre-training strategy that bootstraps&#10;vision-language pre-training from off-the-shelf frozen pre-trained image&#10;encoders and frozen large language models. BLIP-2 bridges the modality gap with&#10;a lightweight Querying Transformer, which is pre-trained in two stages. The&#10;first stage bootstraps vision-language representation learning from a frozen&#10;image encoder. The second stage bootstraps vision-to-language generative&#10;learning from a frozen language model. BLIP-2 achieves state-of-the-art&#10;performance on various vision-language tasks, despite having significantly&#10;fewer trainable parameters than existing methods. For example, our model&#10;outperforms Flamingo80B by 8.7% on zero-shot VQAv2 with 54x fewer trainable&#10;parameters. We also demonstrate the model's emerging capabilities of zero-shot&#10;image-to-text generation that can follow natural language instructions.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2408.01998" label="2408.01998">
-        <attvalues>
-          <attvalue for="0" value="What Happens Without Background? Constructing Foreground-Only Data for&#10;  Fine-Grained Tasks" />
-          <attvalue for="1" value="  Fine-grained recognition, a pivotal task in visual signal processing, aims to&#10;distinguish between similar subclasses based on discriminative information&#10;present in samples. However, prevailing methods often erroneously focus on&#10;background areas, neglecting the capture of genuinely effective discriminative&#10;information from the subject, thus impeding practical application. To&#10;facilitate research into the impact of background noise on models and enhance&#10;their ability to concentrate on the subject's discriminative features, we&#10;propose an engineered pipeline that leverages the capabilities of SAM and Detic&#10;to create fine-grained datasets with only foreground subjects, devoid of&#10;background. Extensive cross-experiments validate this approach as a&#10;preprocessing step prior to training, enhancing algorithmic performance and&#10;holding potential for further modal expansion of the data.&#10;" />
-          <attvalue for="2" value="&#10;Fine-grained recognition stands as a cornerstone in the field of computer vision\cite{xu2023mmcosine}, tasked with the discernment of subtle differences between closely related subclasses within a category. This capability is paramount for applications ranging from biodiversity conservation to medical diagnostics\cite{he2022transfg}, where the accurate identification of species or conditions can have profound implications.&#10;&#10;Despite certain progress in related research, existing methods are often hampered by an erroneous focus on background elements, which detracts from the extraction of key discriminative cues from the subject itself\cite{zha2023boosting}. &#10;As illustrated by the Grad-CAM\cite{selvaraju2017grad} visualisation in Figure \ref{fig_cam}, there is an undesired focus on non-subject content, regardless of whether CNN-based or Transformer-based backbones are used. This misfocus limits model accuracy and real-world applicability, as the discriminative information on the subject is the truly effective element.&#10;&#10;In light of these issues, this study introduces an innovative engineered pipeline to facilitate further research into the impact of background noise on model performance and enhance the model's ability to focus on the subject's genuine discriminative features. By leveraging the capabilities of SAM\cite{kirillov2023segment} and Detic\cite{zhou2022detecting}, we have constructed a dataset that isolates the foreground subjects, thereby providing a more refined and accurate environment for model training and evaluation. Further, we combine a variety of different commonly used Backbone explorations done on the new data obtained.&#10;&#10;The contributions of this paper can be summarized as follows:&#10;\begin{itemize}[leftmargin=15pt,topsep=3pt]&#10; \item Firstly, we demonstrate through visualization that existing fine-grained models have a high probability of being influenced by the background, incorrectly focusing on non-subject regions at the expense of learning truly valid discriminative information, thereby hindering practical applicability.&#10; \item To address this issue, we propose an engineered pipeline utilizing Visual Foundation Models to generate foreground-only data, facilitating research into the impact of background noise and improving the model's capacity to capture discriminative subject features.&#10; \item Furthermore, this pipeline can be used as a general data pre-processing procedure in fine-grained tasks, producing more densely distributed and higher quality training data. Comprehensive cross-validation experiments confirm the performance boost from the preprocessing process.&#10; \item Additionally, we propose that the generated foreground data can also serve as the foundation for expanding the corresponding datasets into more modalities.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Deep Learning Algorithms, Machine Learning, Engineering, Computer Vision, Visual Signal Processing, Mathematics, Computer Science, Fine-Grained Recognition, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2303.05338" label="2303.05338">
-        <attvalues>
-          <attvalue for="0" value="MMCosine: Multi-Modal Cosine Loss Towards Balanced Audio-Visual&#10;  Fine-Grained Learning" />
-          <attvalue for="1" value="  Audio-visual learning helps to comprehensively understand the world by fusing&#10;practical information from multiple modalities. However, recent studies show&#10;that the imbalanced optimization of uni-modal encoders in a joint-learning&#10;model is a bottleneck to enhancing the model's performance. We further find&#10;that the up-to-date imbalance-mitigating methods fail on some audio-visual&#10;fine-grained tasks, which have a higher demand for distinguishable feature&#10;distribution. Fueled by the success of cosine loss that builds hyperspherical&#10;feature spaces and achieves lower intra-class angular variability, this paper&#10;proposes Multi-Modal Cosine loss, MMCosine. It performs a modality-wise $L_2$&#10;normalization to features and weights towards balanced and better multi-modal&#10;fine-grained learning. We demonstrate that our method can alleviate the&#10;imbalanced optimization from the perspective of weight norm and fully exploit&#10;the discriminability of the cosine metric. Extensive experiments prove the&#10;effectiveness of our method and the versatility with advanced multi-modal&#10;fusion strategies and up-to-date imbalance-mitigating methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.07976" label="2103.07976">
-        <attvalues>
-          <attvalue for="0" value="TransFG: A Transformer Architecture for Fine-grained Recognition" />
-          <attvalue for="1" value="  Fine-grained visual classification (FGVC) which aims at recognizing objects&#10;from subcategories is a very challenging task due to the inherently subtle&#10;inter-class differences. Most existing works mainly tackle this problem by&#10;reusing the backbone network to extract features of detected discriminative&#10;regions. However, this strategy inevitably complicates the pipeline and pushes&#10;the proposed regions to contain most parts of the objects thus fails to locate&#10;the really important parts. Recently, vision transformer (ViT) shows its strong&#10;performance in the traditional classification task. The self-attention&#10;mechanism of the transformer links every patch token to the classification&#10;token. In this work, we first evaluate the effectiveness of the ViT framework&#10;in the fine-grained recognition setting. Then motivated by the strength of the&#10;attention link can be intuitively considered as an indicator of the importance&#10;of tokens, we further propose a novel Part Selection Module that can be applied&#10;to most of the transformer architectures where we integrate all raw attention&#10;weights of the transformer into an attention map for guiding the network to&#10;effectively and accurately select discriminative image patches and compute&#10;their relations. A contrastive loss is applied to enlarge the distance between&#10;feature representations of confusing classes. We name the augmented&#10;transformer-based model TransFG and demonstrate the value of it by conducting&#10;experiments on five popular fine-grained benchmarks where we achieve&#10;state-of-the-art performance. Qualitative results are presented for better&#10;understanding of our model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.01439" label="2210.01439">
-        <attvalues>
-          <attvalue for="0" value="Boosting Few-shot Fine-grained Recognition with Background Suppression&#10;  and Foreground Alignment" />
-          <attvalue for="1" value="  Few-shot fine-grained recognition (FS-FGR) aims to recognize novel&#10;fine-grained categories with the help of limited available samples.&#10;Undoubtedly, this task inherits the main challenges from both few-shot learning&#10;and fine-grained recognition. First, the lack of labeled samples makes the&#10;learned model easy to overfit. Second, it also suffers from high intra-class&#10;variance and low inter-class differences in the datasets. To address this&#10;challenging task, we propose a two-stage background suppression and foreground&#10;alignment framework, which is composed of a background activation suppression&#10;(BAS) module, a foreground object alignment (FOA) module, and a local-to-local&#10;(L2L) similarity metric. Specifically, the BAS is introduced to generate a&#10;foreground mask for localization to weaken background disturbance and enhance&#10;dominative foreground objects. The FOA then reconstructs the feature map of&#10;each support sample according to its correction to the query ones, which&#10;addresses the problem of misalignment between support-query image pairs. To&#10;enable the proposed method to have the ability to capture subtle differences in&#10;confused samples, we present a novel L2L similarity metric to further measure&#10;the local similarity between a pair of aligned spatial features in the&#10;embedding space. What's more, considering that background interference brings&#10;poor robustness, we infer the pairwise similarity of feature maps using both&#10;the raw image and the refined image. Extensive experiments conducted on&#10;multiple popular fine-grained benchmarks demonstrate that our method&#10;outperforms the existing state of the art by a large margin. The source codes&#10;are available at: https://github.com/CSer-Tang-hao/BSFA-FSFG.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2201.02605" label="2201.02605">
-        <attvalues>
-          <attvalue for="0" value="Detecting Twenty-thousand Classes using Image-level Supervision" />
-          <attvalue for="1" value="  Current object detectors are limited in vocabulary size due to the small&#10;scale of detection datasets. Image classifiers, on the other hand, reason about&#10;much larger vocabularies, as their datasets are larger and easier to collect.&#10;We propose Detic, which simply trains the classifiers of a detector on image&#10;classification data and thus expands the vocabulary of detectors to tens of&#10;thousands of concepts. Unlike prior work, Detic does not need complex&#10;assignment schemes to assign image labels to boxes based on model predictions,&#10;making it much easier to implement and compatible with a range of detection&#10;architectures and backbones. Our results show that Detic yields excellent&#10;detectors even for classes without box annotations. It outperforms prior work&#10;on both open-vocabulary and long-tail detection benchmarks. Detic provides a&#10;gain of 2.4 mAP for all classes and 8.3 mAP for novel classes on the&#10;open-vocabulary LVIS benchmark. On the standard LVIS benchmark, Detic obtains&#10;41.7 mAP when evaluated on all classes, or only rare classes, hence closing the&#10;gap in performance for object categories with few samples. For the first time,&#10;we train a detector with all the twenty-one-thousand classes of the ImageNet&#10;dataset and show that it generalizes to new datasets without finetuning. Code&#10;is available at \url{https://github.com/facebookresearch/Detic}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09300" label="1908.09300">
-        <attvalues>
-          <attvalue for="0" value="A Comparison of CNN and Classic Features for Image Retrieval" />
-          <attvalue for="1" value="  Feature detectors and descriptors have been successfully used for various&#10;computer vision tasks, such as video object tracking and content-based image&#10;retrieval. Many methods use image gradients in different stages of the&#10;detection-description pipeline to describe local image structures. Recently,&#10;some, or all, of these stages have been replaced by convolutional neural&#10;networks (CNNs), in order to increase their performance. A detector is defined&#10;as a selection problem, which makes it more challenging to implement as a CNN.&#10;They are therefore generally defined as regressors, converting input images to&#10;score maps and keypoints can be selected with non-maximum suppression. This&#10;paper discusses and compares several recent methods that use CNNs for keypoint&#10;detection. Experiments are performed both on the CNN based approaches, as well&#10;as a selection of conventional methods. In addition to qualitative measures&#10;defined on keypoints and descriptors, the bag-of-words (BoW) model is used to&#10;implement an image retrieval application, in order to determine how the methods&#10;perform in practice. The results show that each type of features are best in&#10;different contexts.&#10;" />
-          <attvalue for="2" value="&#10;As billions of images and millions of hours of video are uploaded on the Internet every year, the ability to decode their contents is a challenge for computers that has great value. However, extracting high-level information from raw-pixel data has been the topic of a lot of research. Most traditional methods rely on detection and extraction of salient features. A plethora of methods that detect and describe features have been proposed \cite{salahat2017recent,li2015survey}. One of the most popular methods is SIFT \cite{lowe2004distinctive}. Some methods approximate image gradients, as directly calculating them is an expensive operation and the added performance of calculating exact gradients is usually not worth the extra processing time \cite{bay2008speeded}. Other methods work by directly comparing pixel intensities and do not approximate gradients \cite{Rublee:2011:OEA:2355573.2356268,Leutenegger:2011:BBR:2355573.2356277,alahi2012freak}. Such methods may be preferable in on-line applications computational efficiency is critical. &#10;In recent years, researchers have proposed CNN based methods for detecting and extracting features \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}. CNNs have been successfully used to solve complex problems, such as image classification or speech recognition, but their application to feature detection and description is relatively new. The learning process of these methods can generally be divided into two classes, supervised learning and unsupervised learning. They key difference between the two is existence of annotated data, which help the supervised methods have a very efficient learning process. The result though might be biased according to biases in the data labels. On the other hand unsupervised methods rely on the method's intrinsic ability to find clusters of similar data. This, however, makes the task more challenging and requires a specifically designed algorithm for learning. The goal of this paper is to review how CNNs can be used for feature detection and extraction and compare the performance of the resulting CNNs to the traditional, hand crafted, detectors and extractors.\par&#10;Many works have compared different local feature detectors, either by comparing literature to their own method or making a generic comparison \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning,li2015survey,salahat2017recent}. To the best of our knowledge, there is minimal work that compared CNN based detectors. Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}. This paper performs an in depth comparison between CNN based and traditional detectors. We evaluate the methods based on robustness in the presence of common distortions (such as rotation, scaling, and noise) and also directly on the domain of image retrieval.\par&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Convolutional Neural Networks, Mathematics, Artificial Intelligence, Image Processing, Image Feature Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1703.06376" label="1703.06376">
-        <attvalues>
-          <attvalue for="0" value="Recent Advances in Features Extraction and Description Algorithms: A&#10;  Comprehensive Survey" />
-          <attvalue for="1" value="  Computer vision is one of the most active research fields in information&#10;technology today. Giving machines and robots the ability to see and comprehend&#10;the surrounding world at the speed of sight creates endless potential&#10;applications and opportunities. Feature detection and description algorithms&#10;can be indeed considered as the retina of the eyes of such machines and robots.&#10;However, these algorithms are typically computationally intensive, which&#10;prevents them from achieving the speed of sight real-time performance. In&#10;addition, they differ in their capabilities and some may favor and work better&#10;given a specific type of input compared to others. As such, it is essential to&#10;compactly report their pros and cons as well as their performances and recent&#10;advances. This paper is dedicated to provide a comprehensive overview on the&#10;state-of-the-art and recent advances in feature detection and description&#10;algorithms. Specifically, it starts by overviewing fundamental concepts. It&#10;then compares, reports and discusses their performance and capabilities. The&#10;Maximally Stable Extremal Regions algorithm and the Scale Invariant Feature&#10;Transform algorithms, being two of the best of their type, are selected to&#10;report their recent algorithmic derivatives.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.01224" label="1605.01224">
-        <attvalues>
-          <attvalue for="0" value="Learning Covariant Feature Detectors" />
-          <attvalue for="1" value="  Local covariant feature detection, namely the problem of extracting viewpoint&#10;invariant features from images, has so far largely resisted the application of&#10;machine learning techniques. In this paper, we propose the first fully general&#10;formulation for learning local covariant feature detectors. We propose to cast&#10;detection as a regression problem, enabling the use of powerful regressors such&#10;as deep neural networks. We then derive a covariance constraint that can be&#10;used to automatically learn which visual structures provide stable anchors for&#10;local feature detection. We support these ideas theoretically, proposing a&#10;novel analysis of local features in term of geometric transformations, and we&#10;show that all common and many uncommon detectors can be derived in this&#10;framework. Finally, we present empirical results on translation and rotation&#10;covariant detectors on standard feature benchmarks, showing the power and&#10;flexibility of the framework.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1411.4568" label="1411.4568">
-        <attvalues>
-          <attvalue for="0" value="TILDE: A Temporally Invariant Learned DEtector" />
-          <attvalue for="1" value="  We introduce a learning-based approach to detect repeatable keypoints under&#10;drastic imaging changes of weather and lighting conditions to which&#10;state-of-the-art keypoint detectors are surprisingly sensitive. We first&#10;identify good keypoint candidates in multiple training images taken from the&#10;same viewpoint. We then train a regressor to predict a score map whose maxima&#10;are those points so that they can be found by simple non-maximum suppression.&#10;As there are no standard datasets to test the influence of these kinds of&#10;changes, we created our own, which we will make publicly available. We will&#10;show that our method significantly outperforms the state-of-the-art methods in&#10;such challenging conditions, while still achieving state-of-the-art performance&#10;on the untrained standard Oxford dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1611.07571" label="1611.07571">
-        <attvalues>
-          <attvalue for="0" value="Quad-networks: unsupervised learning to rank for interest point&#10;  detection" />
-          <attvalue for="1" value="  Several machine learning tasks require to represent the data using only a&#10;sparse set of interest points. An ideal detector is able to find the&#10;corresponding interest points even if the data undergo a transformation typical&#10;for a given domain. Since the task is of high practical interest in computer&#10;vision, many hand-crafted solutions were proposed. In this paper, we ask a&#10;fundamental question: can we learn such detectors from scratch? Since it is&#10;often unclear what points are &quot;interesting&quot;, human labelling cannot be used to&#10;find a truly unbiased solution. Therefore, the task requires an unsupervised&#10;formulation. We are the first to propose such a formulation: training a neural&#10;network to rank points in a transformation-invariant manner. Interest points&#10;are then extracted from the top/bottom quantiles of this ranking. We validate&#10;our approach on two tasks: standard RGB image interest point detection and&#10;challenging cross-modal interest point detection between RGB and depth images.&#10;We quantitatively show that our unsupervised method performs better or on-par&#10;with baselines.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1511.04273" label="1511.04273">
-        <attvalues>
-          <attvalue for="0" value="Learning to Assign Orientations to Feature Points" />
-          <attvalue for="1" value="  We show how to train a Convolutional Neural Network to assign a canonical&#10;orientation to feature points given an image patch centered on the feature&#10;point. Our method improves feature point matching upon the state-of-the art and&#10;can be used in conjunction with any existing rotation sensitive descriptors. To&#10;avoid the tedious and almost impossible task of finding a target orientation to&#10;learn, we propose to use Siamese networks which implicitly find the optimal&#10;orientations during training. We also propose a new type of activation function&#10;for Neural Networks that generalizes the popular ReLU, maxout, and PReLU&#10;activation functions. This novel activation performs better for our task. We&#10;validate the effectiveness of our method extensively with four existing&#10;datasets, including two non-planar datasets, as well as our own dataset. We&#10;show that we outperform the state-of-the-art without the need of retraining for&#10;each dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2404.19489" label="2404.19489">
-        <attvalues>
-          <attvalue for="0" value="EvGNN: An Event-driven Graph Neural Network Accelerator for Edge Vision" />
-          <attvalue for="1" value="  Edge vision systems combining sensing and embedded processing promise&#10;low-latency, decentralized, and energy-efficient solutions that forgo reliance&#10;on the cloud. As opposed to conventional frame-based vision sensors,&#10;event-based cameras deliver a microsecond-scale temporal resolution with sparse&#10;information encoding, thereby outlining new opportunities for edge vision&#10;systems. However, mainstream algorithms for frame-based vision, which mostly&#10;rely on convolutional neural networks (CNNs), can hardly exploit the advantages&#10;of event-based vision as they are typically optimized for dense matrix-vector&#10;multiplications. While event-driven graph neural networks (GNNs) have recently&#10;emerged as a promising solution for sparse event-based vision, their irregular&#10;structure is a challenge that currently hinders the design of efficient&#10;hardware accelerators. In this paper, we propose EvGNN, the first event-driven&#10;GNN accelerator for low-footprint, ultra-low-latency, and high-accuracy edge&#10;vision with event-based cameras. It relies on three central ideas: (i) directed&#10;dynamic graphs exploiting single-hop nodes with edge-free storage, (ii) event&#10;queues for the efficient identification of local neighbors within a&#10;spatiotemporally decoupled search range, and (iii) a novel layer-parallel&#10;processing scheme enabling the low-latency execution of multi-layer GNNs. We&#10;deployed EvGNN on a Xilinx KV260 Ultrascale+ MPSoC platform and benchmarked it&#10;on the N-CARS dataset for car recognition, demonstrating a classification&#10;accuracy of 87.8% and an average latency per event of 16$\mu$s, thereby&#10;enabling real-time, microsecond-resolution event-based vision at the edge.&#10;" />
-          <attvalue for="2" value="&#10;E{dge} vision systems combine digital cameras and embedded computing to capture and process visual information within a limited power budget, typically ranging from milliwatts to a few watts \cite{sparseyolo, tinytracker, pulp, lopecs}. While successful in a wide range of applications, from the Internet-of-Things to robotics \cite{edgeeye, flyingiot, robot_sense_1}, edge vision systems relying on conventional frame-based cameras are ill-suited for latency-critical scenarios requiring decisions within microseconds, such as autonomous drone navigation \cite{navion}. Indeed, standard frame-based cameras fail to meet these latency requirements as they typically capture 30-60 frames per second (FPS), while the power dissipation of 1000-FPS high-speed cameras can reach tens of watts \cite{1000fps}, far exceeding the power budget of common edge vision systems.&#10;&#10;Event-based cameras, also called silicon retinas~\cite{sretina} or dynamic vision sensors (DVSes) \cite{dvs}, have attracted growing interest in low-power high-speed edge vision applications. As opposed to their frame-based counterparts, which record absolute light intensity for every pixel in the field of view at a fixed sampling rate, event-based cameras are locally sensitive: their pixels are individually activated only when the received light intensity changes beyond a preset threshold~\cite{event_survey}. When this threshold is exceeded, event packets containing the pixel address are generated. This event-driven scheme operating at the pixel level leads to (i)~a temporal resolution on the order of microseconds, which helps alleviate motion blur, and (ii)~a power consumption on the order of milliwatts thanks to sparsity~\cite{event_survey}. Indeed, as no event is generated in the absence of light intensity changes, static background information is filtered out: only the moving objects are contained in the event stream.&#10;&#10;As the standard neural network model for computer vision tasks, convolutional neural networks (CNNs) have demonstrated excellent performance for scenarios such as object recognition or detection \cite{alex,resnet,yolotiny,mobilenetv3,dualconv,cnn_survey}. However, since they are optimized for the mainstream frame-based vision paradigm, techniques for sparse event-based data are still at an early stage. Indeed, current event-based vision systems usually rely on a straightforward conversion of event streams into images by accumulating events from each pixel within a given time window, which is also known as the dense-frame approach \cite{hugnet_ref_13,time_slice,dense_frame}. While this conversion allows leveraging mature frame-based algorithms to process event-based data, accumulation windows on the order of milliseconds are necessary to obtain reasonable performance \cite{time_slice}, thus discarding the original microsecond-level resolution of the event-based data. &#10;&#10;As solving this challenge requires departing from frame-based vision, graph representations of event-based data have recently started being explored \cite{evs, aegnn, hugnet}. By regarding individual events as nodes and their spatiotemporal relationship as edges, a stream of events can be converted into a fully equivalent graph (\ieevent graph) without degrading the temporal resolution nor losing the inherent sparsity of the data. While this sparse event graph is well suited for processing by graph neural networks (GNNs) \cite{gcn, spline}, the standard approach consisting of first constructing the full event graph, and then processing it with a GNN, fails to exploit the event-driven nature of the data and is bound to millisecond-level latencies in dedicated hardware~\cite{flowgnn, blockgnn}. To solve this challenge, techniques to construct and process the event graph in a dynamic fashion (\ie on a per-event basis) have recently been proposed~\cite{aegnn, hugnet}, thereby enabling both sparsity-aware and event-driven processing. However, designing efficient hardware accelerators for these sparse, irregular, and fully data-driven workloads, is currently an open challenge.&#10;&#10;In this work, we propose \prjname, the first event-driven GNN accelerator for edge vision at microsecond-level latencies that supports the end-to-end hardware acceleration of an event graph, from event-based input acquisition to dynamic event graph construction and real-time GNN inference. Our main contributions are as follows:&#10;&#10;\begin{itemize}&#10;&#10;\item We exploit the causality of event graphs through directed edges to achieve ultra-low-latency decisions by only processing the local subgraph of direct neighbors around a new event, preserving accuracy while enabling an edge-free storage that drastically reduces memory footprint.&#10;&#10;\item We adopt a hardware-friendly spatiotemporally decoupled prism neighbor search scheme during the event-graph construction. A dedicated search engine based on cascaded event queues is implemented in hardware to quickly identify valid neighbors in the spatial then temporal dimension, ensuring the efficient construction of event-wise local subgraphs.&#10;&#10;\item We introduce the concept of layer parallelism to speed up the processing of event-based GNNs with directed edges, reusing past information on a new event's neighborhood to parallelize the computation of every layer's new features, thereby reducing the end-to-end latency per event update.&#10;&#10;\item We finally deploy \prjname~on a Xilinx KV260 Ultrascale$+$ MPSoC platform and evaluate our design with the real-world dataset N-CARS \cite{ncars} for car recognition. Our design achieves a prediction accuracy of 87.8\%, and reaches an average $16$-$\mu$s latency per event, while limiting the overall on-chip memory footprint to $1.76$MB, thereby enabling low-footprint low-latency edge vision.&#10;\end{itemize}&#10;&#10;This paper is organized as follows. Section II introduces key concepts and algorithms of event-based data, event graphs, and GNNs. Our hardware-algorithm co-design approach is described in Section III, covering the two key steps of event graph construction and GNN processing, with the aim to reduce the computational and memory footprints toward deployment on edge hardware. Section IV covers the proposed \prjname~architecture and implementation. Finally, benchmarking results are provided in Section V, while Section VI provides concluding remarks.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Edge Vision Systems, Computer Vision, Computer Architecture, Artificial Intelligence, Physics, Electrical Engineering, Event-Based Cameras, Computer Science, Graph Neural Networks" />
-        </attvalues>
-      </node>
-      <node id="1904.08405" label="1904.08405">
-        <attvalues>
-          <attvalue for="0" value="Event-based Vision: A Survey" />
-          <attvalue for="1" value="  Event cameras are bio-inspired sensors that differ from conventional frame&#10;cameras: Instead of capturing images at a fixed rate, they asynchronously&#10;measure per-pixel brightness changes, and output a stream of events that encode&#10;the time, location and sign of the brightness changes. Event cameras offer&#10;attractive properties compared to traditional cameras: high temporal resolution&#10;(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low&#10;power consumption, and high pixel bandwidth (on the order of kHz) resulting in&#10;reduced motion blur. Hence, event cameras have a large potential for robotics&#10;and computer vision in challenging scenarios for traditional cameras, such as&#10;low-latency, high speed, and high dynamic range. However, novel methods are&#10;required to process the unconventional output of these sensors in order to&#10;unlock their potential. This paper provides a comprehensive overview of the&#10;emerging field of event-based vision, with a focus on the applications and the&#10;algorithms developed to unlock the outstanding properties of event cameras. We&#10;present event cameras from their working principle, the actual sensors that are&#10;available and the tasks that they have been used for, from low-level vision&#10;(feature detection and tracking, optic flow, etc.) to high-level vision&#10;(reconstruction, segmentation, recognition). We also discuss the techniques&#10;developed to process events, including learning-based techniques, as well as&#10;specialized processors for these novel sensors, such as spiking neural&#10;networks. Additionally, we highlight the challenges that remain to be tackled&#10;and the opportunities that lie ahead in the search for a more efficient,&#10;bio-inspired way for machines to perceive and interact with the world.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.02244" label="1905.02244">
-        <attvalues>
-          <attvalue for="0" value="Searching for MobileNetV3" />
-          <attvalue for="1" value="  We present the next generation of MobileNets based on a combination of&#10;complementary search techniques as well as a novel architecture design.&#10;MobileNetV3 is tuned to mobile phone CPUs through a combination of&#10;hardware-aware network architecture search (NAS) complemented by the NetAdapt&#10;algorithm and then subsequently improved through novel architecture advances.&#10;This paper starts the exploration of how automated search algorithms and&#10;network design can work together to harness complementary approaches improving&#10;the overall state of the art. Through this process we create two new MobileNet&#10;models for release: MobileNetV3-Large and MobileNetV3-Small which are targeted&#10;for high and low resource use cases. These models are then adapted and applied&#10;to the tasks of object detection and semantic segmentation. For the task of&#10;semantic segmentation (or any dense pixel prediction), we propose a new&#10;efficient segmentation decoder Lite Reduced Atrous Spatial Pyramid Pooling&#10;(LR-ASPP). We achieve new state of the art results for mobile classification,&#10;detection and segmentation. MobileNetV3-Large is 3.2\% more accurate on&#10;ImageNet classification while reducing latency by 15\% compared to MobileNetV2.&#10;MobileNetV3-Small is 4.6\% more accurate while reducing latency by 5\% compared&#10;to MobileNetV2. MobileNetV3-Large detection is 25\% faster at roughly the same&#10;accuracy as MobileNetV2 on COCO detection. MobileNetV3-Large LR-ASPP is 30\%&#10;faster than MobileNetV2 R-ASPP at similar accuracy for Cityscapes segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2202.07481" label="2202.07481">
-        <attvalues>
-          <attvalue for="0" value="DualConv: Dual Convolutional Kernels for Lightweight Deep Neural&#10;  Networks" />
-          <attvalue for="1" value="  CNN architectures are generally heavy on memory and computational&#10;requirements which makes them infeasible for embedded systems with limited&#10;hardware resources. We propose dual convolutional kernels (DualConv) for&#10;constructing lightweight deep neural networks. DualConv combines 3$\times$3 and&#10;1$\times$1 convolutional kernels to process the same input feature map channels&#10;simultaneously and exploits the group convolution technique to efficiently&#10;arrange convolutional filters. DualConv can be employed in any CNN model such&#10;as VGG-16 and ResNet-50 for image classification, YOLO and R-CNN for object&#10;detection, or FCN for semantic segmentation. In this paper, we extensively test&#10;DualConv for classification since these network architectures form the&#10;backbones for many other tasks. We also test DualConv for image detection on&#10;YOLO-V3. Experimental results show that, combined with our structural&#10;innovations, DualConv significantly reduces the computational cost and number&#10;of parameters of deep neural networks while surprisingly achieving slightly&#10;higher accuracy than the original models in some cases. We use DualConv to&#10;further reduce the number of parameters of the lightweight MobileNetV2 by 54%&#10;with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of&#10;parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by&#10;4.11% on the same dataset. Furthermore, DualConv significantly improves the&#10;YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL VOC&#10;dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.02806" label="2004.02806">
-        <attvalues>
-          <attvalue for="0" value="A Survey of Convolutional Neural Networks: Analysis, Applications, and&#10;  Prospects" />
-          <attvalue for="1" value="  Convolutional Neural Network (CNN) is one of the most significant networks in&#10;the deep learning field. Since CNN made impressive achievements in many areas,&#10;including but not limited to computer vision and natural language processing,&#10;it attracted much attention both of industry and academia in the past few&#10;years. The existing reviews mainly focus on the applications of CNN in&#10;different scenarios without considering CNN from a general perspective, and&#10;some novel ideas proposed recently are not covered. In this review, we aim to&#10;provide novel ideas and prospects in this fast-growing field as much as&#10;possible. Besides, not only two-dimensional convolution but also&#10;one-dimensional and multi-dimensional ones are involved. First, this review&#10;starts with a brief introduction to the history of CNN. Second, we provide an&#10;overview of CNN. Third, classic and advanced CNN models are introduced,&#10;especially those key points making them reach state-of-the-art results. Fourth,&#10;through experimental analysis, we draw some conclusions and provide several&#10;rules of thumb for function selection. Fifth, the applications of&#10;one-dimensional, two-dimensional, and multi-dimensional convolution are&#10;covered. Finally, some open issues and promising directions for CNN are&#10;discussed to serve as guidelines for future work.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.08245" label="1904.08245">
-        <attvalues>
-          <attvalue for="0" value="End-to-End Learning of Representations for Asynchronous Event-Based Data" />
-          <attvalue for="1" value="  Event cameras are vision sensors that record asynchronous streams of&#10;per-pixel brightness changes, referred to as &quot;events&quot;. They have appealing&#10;advantages over frame-based cameras for computer vision, including high&#10;temporal resolution, high dynamic range, and no motion blur. Due to the sparse,&#10;non-uniform spatiotemporal layout of the event signal, pattern recognition&#10;algorithms typically aggregate events into a grid-based representation and&#10;subsequently process it by a standard vision pipeline, e.g., Convolutional&#10;Neural Network (CNN). In this work, we introduce a general framework to convert&#10;event streams into grid-based representations through a sequence of&#10;differentiable operations. Our framework comes with two main advantages: (i)&#10;allows learning the input event representation together with the task dedicated&#10;network in an end to end manner, and (ii) lays out a taxonomy that unifies the&#10;majority of extant event representations in the literature and identifies novel&#10;ones. Empirically, we show that our approach to learning the event&#10;representation end-to-end yields an improvement of approximately 12% on optical&#10;flow estimation and object recognition over state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.09011" label="1708.09011">
-        <attvalues>
-          <attvalue for="0" value="Real-Time 6DOF Pose Relocalization for Event Cameras with Stacked&#10;  Spatial LSTM Networks" />
-          <attvalue for="1" value="  We present a new method to relocalize the 6DOF pose of an event camera solely&#10;based on the event stream. Our method first creates the event image from a list&#10;of events that occurs in a very short time interval, then a Stacked Spatial&#10;LSTM Network (SP-LSTM) is used to learn the camera pose. Our SP-LSTM is&#10;composed of a CNN to learn deep features from the event images and a stack of&#10;LSTM to learn spatial dependencies in the image feature space. We show that the&#10;spatial dependency plays an important role in the relocalization task and the&#10;SP-LSTM can effectively learn this information. The experimental results on a&#10;publicly available dataset show that our approach generalizes well and&#10;outperforms recent methods by a substantial margin. Overall, our proposed&#10;method reduces by approx. 6 times the position error and 3 times the&#10;orientation error compared to the current state of the art. The source code and&#10;trained models will be released.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.14419" label="2308.14419">
-        <attvalues>
-          <attvalue for="0" value="Graph-based Asynchronous Event Processing for Rapid Object Recognition" />
-          <attvalue for="1" value="  Different from traditional video cameras, event cameras capture asynchronous&#10;events stream in which each event encodes pixel location, trigger time, and the&#10;polarity of the brightness changes. In this paper, we introduce a novel&#10;graph-based framework for event cameras, namely SlideGCN. Unlike some recent&#10;graph-based methods that use groups of events as input, our approach can&#10;efficiently process data event-by-event, unlock the low latency nature of&#10;events data while still maintaining the graph's structure internally. For fast&#10;graph construction, we develop a radius search algorithm, which better exploits&#10;the partial regular structure of event cloud against k-d tree based generic&#10;methods. Experiments show that our method reduces the computational complexity&#10;up to 100 times with respect to current graph-based methods while keeping&#10;state-of-the-art performance on object recognition. Moreover, we verify the&#10;superiority of event-wise processing with our method. When the state becomes&#10;stable, we can give a prediction with high confidence, thus making an early&#10;recognition. Project page: \url{https://zju3dv.github.io/slide_gcn/}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.13103" label="2204.13103">
-        <attvalues>
-          <attvalue for="0" value="FlowGNN: A Dataflow Architecture for Real-Time Workload-Agnostic Graph&#10;  Neural Network Inference" />
-          <attvalue for="1" value="  Graph neural networks (GNNs) have recently exploded in popularity thanks to&#10;their broad applicability to graph-related problems such as quantum chemistry,&#10;drug discovery, and high energy physics. However, meeting demand for novel GNN&#10;models and fast inference simultaneously is challenging due to the gap between&#10;developing efficient accelerators and the rapid creation of new GNN models.&#10;Prior art focuses on accelerating specific classes of GNNs, such as Graph&#10;Convolutional Networks (GCN), but lacks generality to support a wide range of&#10;existing or new GNN models. Furthermore, most works rely on graph&#10;pre-processing to exploit data locality, making them unsuitable for real-time&#10;applications. To address these limitations, in this work, we propose a generic&#10;dataflow architecture for GNN acceleration, named FlowGNN, which is&#10;generalizable to the majority of message-passing GNNs. The contributions are&#10;three-fold. First, we propose a novel and scalable dataflow architecture, which&#10;generally supports a wide range of GNN models with message-passing mechanism.&#10;The architecture features a configurable dataflow optimized for simultaneous&#10;computation of node embedding, edge embedding, and message passing, which is&#10;generally applicable to all models. We also propose a rich library of&#10;model-specific components. Second, we deliver ultra-fast real-time GNN&#10;inference without any graph pre-processing, making it agnostic to dynamically&#10;changing graph structures. Third, we verify our architecture on the Xilinx&#10;Alveo U50 FPGA board and measure the on-board end-to-end performance. We&#10;achieve a speed-up of up to 24-254x against CPU (6226R) and 1.3-477x against&#10;GPU (A6000) (with batch sizes 1 through 1024); we also outperform the SOTA GNN&#10;accelerator I-GCN by 1.26x speedup and 1.55x energy efficiency over four&#10;datasets. Our implementation code and on-board measurement are publicly&#10;available on GitHub.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.17149" label="2203.17149">
-        <attvalues>
-          <attvalue for="0" value="AEGNN: Asynchronous Event-based Graph Neural Networks" />
-          <attvalue for="1" value="  The best performing learning algorithms devised for event cameras work by&#10;first converting events into dense representations that are then processed&#10;using standard CNNs. However, these steps discard both the sparsity and high&#10;temporal resolution of events, leading to high computational burden and&#10;latency. For this reason, recent works have adopted Graph Neural Networks&#10;(GNNs), which process events as ``static&quot; spatio-temporal graphs, which are&#10;inherently &quot;sparse&quot;. We take this trend one step further by introducing&#10;Asynchronous, Event-based Graph Neural Networks (AEGNNs), a novel&#10;event-processing paradigm that generalizes standard GNNs to process events as&#10;``evolving&quot; spatio-temporal graphs. AEGNNs follow efficient update rules that&#10;restrict recomputation of network activations only to the nodes affected by&#10;each new event, thereby significantly reducing both computation and latency for&#10;event-by-event processing. AEGNNs are easily trained on synchronous inputs and&#10;can be converted to efficient, &quot;asynchronous&quot; networks at test time. We&#10;thoroughly validate our method on object classification and detection tasks,&#10;where we show an up to a 11-fold reduction in computational complexity (FLOPs),&#10;with similar or even better performance than state-of-the-art asynchronous&#10;methods. This reduction in computation directly translates to an 8-fold&#10;reduction in computational latency when compared to standard GNNs, which opens&#10;the door to low-latency event-based processing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07913" label="1803.07913">
-        <attvalues>
-          <attvalue for="0" value="HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object&#10;  Classification" />
-          <attvalue for="1" value="  Event-based cameras have recently drawn the attention of the Computer Vision&#10;community thanks to their advantages in terms of high temporal resolution, low&#10;power consumption and high dynamic range, compared to traditional frame-based&#10;cameras. These properties make event-based cameras an ideal choice for&#10;autonomous vehicles, robot navigation or UAV vision, among others. However, the&#10;accuracy of event-based object classification algorithms, which is of crucial&#10;importance for any reliable system working in real-world conditions, is still&#10;far behind their frame-based counterparts. Two main reasons for this&#10;performance gap are: 1. The lack of effective low-level representations and&#10;architectures for event-based object classification and 2. The absence of large&#10;real-world event-based datasets. In this paper we address both problems. First,&#10;we introduce a novel event-based feature representation together with a new&#10;machine learning architecture. Compared to previous approaches, we use local&#10;memory units to efficiently leverage past temporal information and build a&#10;robust event-based representation. Second, we release the first large&#10;real-world event-based dataset for object classification. We compare our method&#10;to the state-of-the-art with extensive experiments, showing better&#10;classification performance and real-time computation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2212.02319" label="2212.02319">
-        <attvalues>
-          <attvalue for="0" value="Robust and Accurate Cylinder Triangulation" />
-          <attvalue for="1" value="  In this paper we present methods for triangulation of infinite cylinders from&#10;image line silhouettes. We show numerically that linear estimation of a general&#10;quadric surface is inherently a badly posed problem. Instead we propose to&#10;constrain the conic section to a circle, and give algebraic constraints on the&#10;dual conic, that models this manifold. Using these constraints we derive a fast&#10;minimal solver based on three image silhouette lines, that can be used to&#10;bootstrap robust estimation schemes such as RANSAC. We also present a&#10;constrained least squares solver that can incorporate all available image lines&#10;for accurate estimation. The algorithms are tested on both synthetic and real&#10;data, where they are shown to give accurate results, compared to previous&#10;methods.&#10;" />
-          <attvalue for="2" value="&#10;The world is a complex place. In order to make sense of it, we as humans typically make simplifications in order to understand and interpret our surroundings. The use of geometric primitives, to represent and build up the world, has been advocated both from a computer vision perspective, via the influential ideas of Marr \cite{marr1978representation}, and from a psychology perspective, via the recognition-by-components (RBC) theory of Biederman \cite{biederman1985human}, for a long time. These ideas were put forward mostly in the context of object recognition. Today, applications for extraction of semantic information from images and scenes are typically highly data driven, with representations largely learned and coded implicitly in neural network architectures. Geometric estimation in Structure from Motion (SfM) and Simultaneous Localization and Mapping (SLAM) application are, on the other hand, often based on explicit representations (sparse 3D-points and camera matrices) even if the image feature representations are learned. There are today mature methods that do SfM and SLAM efficiently \cite{mur2015orb,schoenberger2016sfm,schoenberger2016mvs}. Sparse point clouds are well suited for matching, camera geometry estimation and optimization. However, they do not scale well for very large scenes, they are often not stable over time \cite{sattler2018benchmarking}, and are bad for downstream tasks such as interpretation and recognition. For this reason we are interested in investigating the use of mid-level representations that carry more semantic meaning than simple points do. One such possible primitive shape is a cylinder. In this paper we will specifically study the problem of how one can efficiently estimate the 3D structure of a cylinder given views of it, in a number of images. We will show how one can triangulate the shape from silhouette lines in two or more images, in both a robust and an accurate way. Note that we in this paper restrict image information to the silhouettes of the sides of cylinders, and not the apparent contours of the cross sections, i.e.\ we consider infinitely long cylinders (analogous to using lines or line segments for line geoemetry). &#10;Working with lines in images is quite difficult, but line segment detectors have matured over the last years, \cite{xu2021line,dai2021fully,von2012lsd,pautrat2021sold2} and we foresee that they will soon be applicable for many general computer vision tasks. &#10;&#10;A cylinder is an example of a quadric surface, and can basically be estimated linearly, given silhouette lines in a number of views with known camera geometry. However, as we will show in a number of real examples, this problem is inherently ill-posed, and in order to find realistic solutions we need to constrain the estimation process. Our contributions in the paper are that we &#10;(i) formulate polynomial constraints on a quadric being a cylinder, &#10;(ii) derive a minimal solver based on these constraints that can be used for robust estimation, and&#10;(iii) derive a non-iterative constrained least squares solver that can be used for accurate estimation. Code for solvers will be made publicly available upon acceptance.&#10;&#10;There exists previous work on a number of geometric problems involving quadrics and conics, but most work is concerned with general projective models, e.g. \cite{de1993conics,ma1996ellipsoid}.&#10;For planar conics and circles there are several results on pose, homography estimation and relative pose, e.g. in \cite{kaminski2003multiple,mudigonda2004geometric,frosio2012linear,mei2017monocular,frosio2016camera}. The two view relative pose problem was considered in \cite{kahl1998using,quan1996conic} which is closely related to the theory for general silhouettes, \cite{astrom-cipolla-etal-ijcv-99,astrom-kahl-itpam-99}.&#10;In \cite{navab2006canonical} the authors present algorithms for pose, 3D reconstruction and structure from motion from cylinders. These methods do not, however, consider robust or least square solutions for triangulation.&#10;Work from \cite{winkler2013curve} and \cite{sun2019camera} are addressing camera calibration using cylinders, but using the apparent contours of the ends of finite cylinders instead. &#10;The inverse problem, camera position from known cylinders, is covered in \cite{gummesonengman2022fast}.&#10; The triangulation problem has also been considered for other geometric shapes, such as lines \cite{bartoli2005structure}, planar conics \cite{josephson2008triangulation} and more complex surfaces based on line representations \cite{keppel1975approximating}.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Geometric Estimation, Algebraic Constraints, Physics, Computer Science, Geometry, Numerical Analysis" />
-        </attvalues>
-      </node>
-      <node id="2101.01909" label="2101.01909">
-        <attvalues>
-          <attvalue for="0" value="Line Segment Detection Using Transformers without Edges" />
-          <attvalue for="1" value="  In this paper, we present a joint end-to-end line segment detection algorithm&#10;using Transformers that is post-processing and heuristics-guided intermediate&#10;processing (edge/junction/region detection) free. Our method, named LinE&#10;segment TRansformers (LETR), takes advantages of having integrated tokenized&#10;queries, a self-attention mechanism, and an encoding-decoding strategy within&#10;Transformers by skipping standard heuristic designs for the edge element&#10;detection and perceptual grouping processes. We equip Transformers with a&#10;multi-scale encoder/decoder strategy to perform fine-grained line segment&#10;detection under a direct endpoint distance loss. This loss term is particularly&#10;suitable for detecting geometric structures such as line segments that are not&#10;conveniently represented by the standard bounding box representations. The&#10;Transformers learn to gradually refine line segments through layers of&#10;self-attention. In our experiments, we show state-of-the-art results on&#10;Wireframe and YorkUrban benchmarks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.11207" label="2104.11207">
-        <attvalues>
-          <attvalue for="0" value="Fully Convolutional Line Parsing" />
-          <attvalue for="1" value="  We present a one-stage Fully Convolutional Line Parsing network (F-Clip) that&#10;detects line segments from images. The proposed network is very simple and&#10;flexible with variations that gracefully trade off between speed and accuracy&#10;for different applications. F-Clip detects line segments in an end-to-end&#10;fashion by predicting each line's center position, length, and angle. We&#10;further customize the design of convolution kernels of our fully convolutional&#10;network to effectively exploit the statistical priors of the distribution of&#10;line angles in real image datasets. We conduct extensive experiments and show&#10;that our method achieves a significantly better trade-off between efficiency&#10;and accuracy, resulting in a real-time line detector at up to 73 FPS on a&#10;single GPU. Such inference speed makes our method readily applicable to&#10;real-time tasks without compromising any accuracy of previous methods.&#10;Moreover, when equipped with a performance-improving backbone network, F-Clip&#10;is able to significantly outperform all state-of-the-art line detectors on&#10;accuracy at a similar or even higher frame rate. In other word, under same&#10;inference speed, F-Clip always achieving best accuracy compare with other&#10;methods. Source code https://github.com/Delay-Xili/F-Clip.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.03362" label="2104.03362">
-        <attvalues>
-          <attvalue for="0" value="SOLD2: Self-supervised Occlusion-aware Line Description and Detection" />
-          <attvalue for="1" value="  Compared to feature point detection and description, detecting and matching&#10;line segments offer additional challenges. Yet, line features represent a&#10;promising complement to points for multi-view tasks. Lines are indeed&#10;well-defined by the image gradient, frequently appear even in poorly textured&#10;areas and offer robust structural cues. We thus hereby introduce the first&#10;joint detection and description of line segments in a single deep network.&#10;Thanks to a self-supervised training, our method does not require any annotated&#10;line labels and can therefore generalize to any dataset. Our detector offers&#10;repeatable and accurate localization of line segments in images, departing from&#10;the wireframe parsing approach. Leveraging the recent progresses in descriptor&#10;learning, our proposed line descriptor is highly discriminative, while&#10;remaining robust to viewpoint changes and occlusions. We evaluate our approach&#10;against previous line detection and description methods on several multi-view&#10;datasets created with homographic warps as well as real-world viewpoint&#10;changes. Our full pipeline yields higher repeatability, localization accuracy&#10;and matching metrics, and thus represents a first step to bridge the gap with&#10;learned feature points methods. Code and trained weights are available at&#10;https://github.com/cvg/SOLD2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.11588" label="2008.11588">
-        <attvalues>
-          <attvalue for="0" value="A Prospective Study on Sequence-Driven Temporal Sampling and Ego-Motion&#10;  Compensation for Action Recognition in the EPIC-Kitchens Dataset" />
-          <attvalue for="1" value="  Action recognition is currently one of the top-challenging research fields in&#10;computer vision. Convolutional Neural Networks (CNNs) have significantly&#10;boosted its performance but rely on fixed-size spatio-temporal windows of&#10;analysis, reducing CNNs temporal receptive fields. Among action recognition&#10;datasets, egocentric recorded sequences have become of important relevance&#10;while entailing an additional challenge: ego-motion is unavoidably transferred&#10;to these sequences. The proposed method aims to cope with it by estimating this&#10;ego-motion or camera motion. The estimation is used to temporally partition&#10;video sequences into motion-compensated temporal \textit{chunks} showing the&#10;action under stable backgrounds and allowing for a content-driven temporal&#10;sampling. A CNN trained in an end-to-end fashion is used to extract temporal&#10;features from each \textit{chunk}, which are late fused. This process leads to&#10;the extraction of features from the whole temporal range of an action,&#10;increasing the temporal receptive field of the network.&#10;" />
-          <attvalue for="2" value="&#10;Video action classification is a highly emerging research topic in computer vision \cite{kay2017kinetics, carreira2018short, sigurdsson2016hollywood} due to its potential wide range of applications.&#10;&#10;Among reported approaches, those relying on the use of Convolutional Neural Networks (CNNs) have reported the highest performances. These methods are based on the extraction of spatio-temporal features on the video, which are then used to classify the action recorded. Due to processing constraints, these features are instead usually extracted on video segments—i.e. fixed-size spatio-temporal windows obtained by sampling and cutting the video, hence discarding the rest of the video sequence. CNNs with reduced temporal receptive fields emerge from this design criteria.&#10;&#10;There are several video action classification datasets in the literature, among them, egocentric ones, i.e. those recorded from a first person point of view, have become of crucial importance \cite{damen2018scaling}. The egocentric domain presents a set of relevant advantages with respect to third-person recordings. The point of view in egocentric videos, closely related to the position of humans eyes, provides a view similar to what of what we humans actually see. Besides, the close proximity of the wearable camera to the undergoing action provides closer and more detailed objects representations compared to third-person view recordings.&#10;&#10;However, egocentric recordings entail an additional challenge: human body or ego-motion is inevitably transferred to video sequences (see left column in Figure \ref{fig:Compensated Video}), creating motion and temporal patterns that may occlude or befoul the action's ones. This ego-motion might hinder the performance of modern CNNs trained to recognize actions in videos \cite{price2019evaluation,feichtenhofer2019slowfast, wang2018non,wang2019baidu} as these are focused on extracting the representative temporal features defining an action. &#10;&#10;Our proposal aims to cope with ego-motion problems while providing a sequence-driven adaptive temporal sampling scheme. This process aims to filter out the majority of the ego-motion, leading to an action segment with a quasi-static background. Camera motion compensation accentuates the representative motion in a specific action, such as the movement of objects or hands (Figure \ref{fig:Compensated Video}), which might lead to more representative and distinguishable action features. In addition, we use the camera motion estimation to temporally divide the sequence into representative context temporal chunks. This temporal partitioning benefits the feature extraction process by: 1) Easing the process of camera motion compensation by having chunks that represent contextually similar backgrounds with limited camera motion. 2) Enabling the use of an irregular temporal sampling policy hence expanding the CNN temporal receptive field. We adopt a shared end-to-end fashion CNN to independently analyze each of the camera motion compensated chunks. Finally, features from each chunk are late fused to obtain final action predictions.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Action Recognition, Artificial Intelligence, Computer Science, Neural Network Optimization, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1705.06950" label="1705.06950">
-        <attvalues>
-          <attvalue for="0" value="The Kinetics Human Action Video Dataset" />
-          <attvalue for="1" value="  We describe the DeepMind Kinetics human action video dataset. The dataset&#10;contains 400 human action classes, with at least 400 video clips for each&#10;action. Each clip lasts around 10s and is taken from a different YouTube video.&#10;The actions are human focussed and cover a broad range of classes including&#10;human-object interactions such as playing instruments, as well as human-human&#10;interactions such as shaking hands. We describe the statistics of the dataset,&#10;how it was collected, and give some baseline performance figures for neural&#10;network architectures trained and tested for human action classification on&#10;this dataset. We also carry out a preliminary analysis of whether imbalance in&#10;the dataset leads to bias in the classifiers.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.02748" label="1804.02748">
-        <attvalues>
-          <attvalue for="0" value="Scaling Egocentric Vision: The EPIC-KITCHENS Dataset" />
-          <attvalue for="1" value="  First-person vision is gaining interest as it offers a unique viewpoint on&#10;people's interaction with objects, their attention, and even intention.&#10;However, progress in this challenging domain has been relatively slow due to&#10;the lack of sufficiently large datasets. In this paper, we introduce&#10;EPIC-KITCHENS, a large-scale egocentric video benchmark recorded by 32&#10;participants in their native kitchen environments. Our videos depict&#10;nonscripted daily activities: we simply asked each participant to start&#10;recording every time they entered their kitchen. Recording took place in 4&#10;cities (in North America and Europe) by participants belonging to 10 different&#10;nationalities, resulting in highly diverse cooking styles. Our dataset features&#10;55 hours of video consisting of 11.5M frames, which we densely labeled for a&#10;total of 39.6K action segments and 454.3K object bounding boxes. Our annotation&#10;is unique in that we had the participants narrate their own videos (after&#10;recording), thus reflecting true intention, and we crowd-sourced ground-truths&#10;based on these. We describe our object, action and anticipation challenges, and&#10;evaluate several baselines over two test splits, seen and unseen kitchens.&#10;Dataset and Project page: http://epic-kitchens.github.io&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00867" label="1908.00867">
-        <attvalues>
-          <attvalue for="0" value="An Evaluation of Action Recognition Models on EPIC-Kitchens" />
-          <attvalue for="1" value="  We benchmark contemporary action recognition models (TSN, TRN, and TSM) on&#10;the recently introduced EPIC-Kitchens dataset and release pretrained models on&#10;GitHub (https://github.com/epic-kitchens/action-models) for others to build&#10;upon. In contrast to popular action recognition datasets like Kinetics,&#10;Something-Something, UCF101, and HMDB51, EPIC-Kitchens is shot from an&#10;egocentric perspective and captures daily actions in-situ. In this report, we&#10;aim to understand how well these models can tackle the challenges present in&#10;this dataset, such as its long tail class distribution, unseen environment test&#10;set, and multiple tasks (verb, noun and, action classification). We discuss the&#10;models' shortcomings and avenues for future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1906.09383" label="1906.09383">
-        <attvalues>
-          <attvalue for="0" value="Baidu-UTS Submission to the EPIC-Kitchens Action Recognition Challenge&#10;  2019" />
-          <attvalue for="1" value="  In this report, we present the Baidu-UTS submission to the EPIC-Kitchens&#10;Action Recognition Challenge in CVPR 2019. This is the winning solution to this&#10;challenge. In this task, the goal is to predict verbs, nouns, and actions from&#10;the vocabulary for each video segment. The EPIC-Kitchens dataset contains&#10;various small objects, intense motion blur, and occlusions. It is challenging&#10;to locate and recognize the object that an actor interacts with. To address&#10;these problems, we utilize object detection features to guide the training of&#10;3D Convolutional Neural Networks (CNN), which can significantly improve the&#10;accuracy of noun prediction. Specifically, we introduce a Gated Feature&#10;Aggregator module to learn from the clip feature and the object feature. This&#10;module can strengthen the interaction between the two kinds of activations and&#10;avoid gradient exploding. Experimental results demonstrate our approach&#10;outperforms other methods on both seen and unseen test set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.00561" label="2403.00561">
-        <attvalues>
-          <attvalue for="0" value="Multi-Task Learning Using Uncertainty to Weigh Losses for Heterogeneous&#10;  Face Attribute Estimation" />
-          <attvalue for="1" value="  Face images contain a wide variety of attribute information. In this paper,&#10;we propose a generalized framework for joint estimation of ordinal and nominal&#10;attributes based on information sharing. We tackle the correlation problem&#10;between heterogeneous attributes using hard parameter sharing of shallow&#10;features, and trade-off multiple loss functions by considering homoskedastic&#10;uncertainty for each attribute estimation task. This leads to optimal&#10;estimation of multiple attributes of the face and reduces the training cost of&#10;multitask learning. Experimental results on benchmarks with multiple face&#10;attributes show that the proposed approach has superior performance compared to&#10;state of the art. Finally, we discuss the bias issues arising from the proposed&#10;approach in face attribute estimation and validate its feasibility on edge&#10;systems.&#10;" />
-          <attvalue for="2" value="&#10;A{s} an important cue for social interaction, face images provide a variety of salient information. Over the past 20 years, significant advances have occurred in the field of extracting attribute information from face images to determine subject identity. And it is applied to video security surveillance, personality recommendation, social media \cite{1-7proencca2021uu,yin2020fan}, etc.&#10;&#10;However, most previous studies \cite{cao2020rank,icmeage} are limited to estimating a single attribute, or learning separate model for each face attribute. Considering the complexity and computational effort, this is not suitable for realistic applications anymore. &#10;&#10;To address these limitations, new approaches have been developed to explore attribute correlations for joint estimation of multiple face attributes. Even these approaches have some serious limitations. For example, Sun et al.\cite{sun2017demographic} used the same features to estimate all attributes of a face, ignoring the heterogeneity between attributes. Han et al.\cite{huhan2017} abandoned lightweighting to explore correlations between heterogeneous attributes.&#10;&#10;As depicted in Figure~\ref{face}, we observe rich correlations and heterogeneities among multiple attributes of the face. For instance, individuals with a mustache and short hair are more likely to be male, and less likely to wear lipstick. Attributes like age follow an ordered pattern, while gender and race are nominal. While attribute correlation can be utilized to improve the robustness of attribute estimation, attribute heterogeneity should also be tackled by designing appropriate prediction models.&#10;&#10;The hierarchical nature that CNNs possess provides a medium for expressing the correlation of heterogeneous face attributes\cite{zeiler2014visualizing}. The lower layers of CNNs are inclined to capture local common features within the image, such as textures and edges. These features contain relatively less semantic information but exhibit precise target localization. In contrast, the upper layers of CNNs gradually learn category-related features with rich semantic. Therefore, we propose a multi-task learning approach based on deep CNNs to take into account the correlation and heterogeneity of face attributes.&#10;&#10;In this work, we shared the low-level parameters across all tasks and designed separate classifiers for each task. We focus on ordinal and nominal face attributes, transforming the ordinal regression problem into a linear combination of several binary classification subproblems, which simplifies the task of ordinal attribute estimation.&#10;&#10;Specifically, we eschew the conventional approach of manually adjusting multi-task loss weights as observed in previously studies\cite{han2017heterogeneous}. Instead, we construe the homoscedastic uncertainty as a task-dependent weight. This effectively quantifies the noise and scale inherent in each task, facilitating the exploration for optimal weights.&#10;&#10;In summary, the key contributions of this paper are:&#10;\begin{itemize}&#10;\item We propose a Deep Multi-Task Learning (DMTL) approach for estimating ordinal and nominal attributes of faces and verify that it is feasible in edge systems;&#10;&#10;\item In the estimation of ordinal attributes, we treat the regression problem as a linear combination of a series of binary classification problems, thereby mitigating estimation bias;&#10;&#10;\item Based on the homoskedastic uncertainty, we realize the optimal loss-weight search for multiple heterogeneous attribute estimation tasks.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Mathematics, Face Attribute Estimation, Artificial Intelligence, Multitask Learning, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2007.04316" label="2007.04316">
-        <attvalues>
-          <attvalue for="0" value="The UU-Net: Reversible Face De-Identification for Visual Surveillance&#10;  Video Footage" />
-          <attvalue for="1" value="  We propose a reversible face de-identification method for low resolution&#10;video data, where landmark-based techniques cannot be reliably used. Our&#10;solution is able to generate a photo realistic de-identified stream that meets&#10;the data protection regulations and can be publicly released under minimal&#10;privacy constraints. Notably, such stream encapsulates all the information&#10;required to later reconstruct the original scene, which is useful for&#10;scenarios, such as crime investigation, where the identification of the&#10;subjects is of most importance. We describe a learning process that jointly&#10;optimizes two main components: 1) a public module, that receives the raw data&#10;and generates the de-identified stream, where the ID information is surrogated&#10;in a photo-realistic and seamless way; and 2) a private module, designed for&#10;legal/security authorities, that analyses the public stream and reconstructs&#10;the original scene, disclosing the actual IDs of all the subjects in the scene.&#10;The proposed solution is landmarks-free and uses a conditional generative&#10;adversarial network to generate synthetic faces that preserve pose, lighting,&#10;background information and even facial expressions. Also, we enable full&#10;control over the set of soft facial attributes that should be preserved between&#10;the raw and de-identified data, which broads the range of applications for this&#10;solution. Our experiments were conducted in three different visual surveillance&#10;datasets (BIODI, MARS and P-DESTRE) and showed highly encouraging results. The&#10;source code is available at https://github.com/hugomcp/uu-net.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.11680" label="1911.11680">
-        <attvalues>
-          <attvalue for="0" value="FAN: Feature Adaptation Network for Surveillance Face Recognition and&#10;  Normalization" />
-          <attvalue for="1" value="  This paper studies face recognition (FR) and normalization in surveillance&#10;imagery. Surveillance FR is a challenging problem that has great values in law&#10;enforcement. Despite recent progress in conventional FR, less effort has been&#10;devoted to surveillance FR. To bridge this gap, we propose a Feature Adaptation&#10;Network (FAN) to jointly perform surveillance FR and normalization. Our face&#10;normalization mainly acts on the aspect of image resolution, closely related to&#10;face super-resolution. However, previous face super-resolution methods require&#10;paired training data with pixel-to-pixel correspondence, which is typically&#10;unavailable between real-world low-resolution and high-resolution faces. FAN&#10;can leverage both paired and unpaired data as we disentangle the features into&#10;identity and non-identity components and adapt the distribution of the identity&#10;features, which breaks the limit of current face super-resolution methods. We&#10;further propose a random scale augmentation scheme to learn resolution robust&#10;identity features, with advantages over previous fixed scale augmentation.&#10;Extensive experiments on LFW, WIDER FACE, QUML-SurvFace and SCface datasets&#10;have shown the effectiveness of our method on surveillance FR and&#10;normalization.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.07884" label="1901.07884">
-        <attvalues>
-          <attvalue for="0" value="Rank consistent ordinal regression for neural networks with application&#10;  to age estimation" />
-          <attvalue for="1" value="  In many real-world prediction tasks, class labels include information about&#10;the relative ordering between labels, which is not captured by commonly-used&#10;loss functions such as multi-category cross-entropy. Recently, the deep&#10;learning community adopted ordinal regression frameworks to take such ordering&#10;information into account. Neural networks were equipped with ordinal regression&#10;capabilities by transforming ordinal targets into binary classification&#10;subtasks. However, this method suffers from inconsistencies among the different&#10;binary classifiers. To resolve these inconsistencies, we propose the COnsistent&#10;RAnk Logits (CORAL) framework with strong theoretical guarantees for&#10;rank-monotonicity and consistent confidence scores. Moreover, the proposed&#10;method is architecture-agnostic and can extend arbitrary state-of-the-art deep&#10;neural network classifiers for ordinal regression tasks. The empirical&#10;evaluation of the proposed rank-consistent method on a range of face-image&#10;datasets for age prediction shows a substantial reduction of the prediction&#10;error compared to the reference ordinal regression network.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.17156" label="2203.17156">
-        <attvalues>
-          <attvalue for="0" value="Adaptive Mean-Residue Loss for Robust Facial Age Estimation" />
-          <attvalue for="1" value="  Automated facial age estimation has diverse real-world applications in&#10;multimedia analysis, e.g., video surveillance, and human-computer interaction.&#10;However, due to the randomness and ambiguity of the aging process, age&#10;assessment is challenging. Most research work over the topic regards the task&#10;as one of age regression, classification, and ranking problems, and cannot well&#10;leverage age distribution in representing labels with age ambiguity. In this&#10;work, we propose a simple yet effective loss function for robust facial age&#10;estimation via distribution learning, i.e., adaptive mean-residue loss, in&#10;which, the mean loss penalizes the difference between the estimated age&#10;distribution's mean and the ground-truth age, whereas the residue loss&#10;penalizes the entropy of age probability out of dynamic top-K in the&#10;distribution. Experimental results in the datasets FG-NET and CLAP2016 have&#10;validated the effectiveness of the proposed loss. Our code is available at&#10;https://github.com/jacobzhaoziyuan/AMR-Loss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1311.2901" label="1311.2901">
-        <attvalues>
-          <attvalue for="0" value="Visualizing and Understanding Convolutional Networks" />
-          <attvalue for="1" value="  Large Convolutional Network models have recently demonstrated impressive&#10;classification performance on the ImageNet benchmark. However there is no clear&#10;understanding of why they perform so well, or how they might be improved. In&#10;this paper we address both issues. We introduce a novel visualization technique&#10;that gives insight into the function of intermediate feature layers and the&#10;operation of the classifier. We also perform an ablation study to discover the&#10;performance contribution from different model layers. This enables us to find&#10;model architectures that outperform Krizhevsky \etal on the ImageNet&#10;classification benchmark. We show our ImageNet model generalizes well to other&#10;datasets: when the softmax classifier is retrained, it convincingly beats the&#10;current state-of-the-art results on Caltech-101 and Caltech-256 datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.00906" label="1706.00906">
-        <attvalues>
-          <attvalue for="0" value="Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning&#10;  Approach" />
-          <attvalue for="1" value="  Face attribute estimation has many potential applications in video&#10;surveillance, face retrieval, and social media. While a number of methods have&#10;been proposed for face attribute estimation, most of them did not explicitly&#10;consider the attribute correlation and heterogeneity (e.g., ordinal vs. nominal&#10;and holistic vs. local) during feature representation learning. In this paper,&#10;we present a Deep Multi-Task Learning (DMTL) approach to jointly estimate&#10;multiple heterogeneous attributes from a single face image. In DMTL, we tackle&#10;attribute correlation and heterogeneity with convolutional neural networks&#10;(CNNs) consisting of shared feature learning for all the attributes, and&#10;category-specific feature learning for heterogeneous attributes. We also&#10;introduce an unconstrained face database (LFW+), an extension of public-domain&#10;LFW, with heterogeneous demographic attributes (age, gender, and race) obtained&#10;via crowdsourcing. Experimental results on benchmarks with multiple face&#10;attributes (MORPH II, LFW+, CelebA, LFWA, and FotW) show that the proposed&#10;approach has superior performance compared to state of the art. Finally,&#10;evaluations on a public-domain face database (LAP) with a single attribute show&#10;that the proposed approach has excellent generalization ability.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.11900" label="2304.11900">
-        <attvalues>
-          <attvalue for="0" value="Learning Visibility Field for Detailed 3D Human Reconstruction and&#10;  Relighting" />
-          <attvalue for="1" value="  Detailed 3D reconstruction and photo-realistic relighting of digital humans&#10;are essential for various applications. To this end, we propose a novel&#10;sparse-view 3d human reconstruction framework that closely incorporates the&#10;occupancy field and albedo field with an additional visibility field--it not&#10;only resolves occlusion ambiguity in multiview feature aggregation, but can&#10;also be used to evaluate light attenuation for self-shadowed relighting. To&#10;enhance its training viability and efficiency, we discretize visibility onto a&#10;fixed set of sample directions and supply it with coupled geometric 3D depth&#10;feature and local 2D image feature. We further propose a novel&#10;rendering-inspired loss, namely TransferLoss, to implicitly enforce the&#10;alignment between visibility and occupancy field, enabling end-to-end joint&#10;training. Results and extensive experiments demonstrate the effectiveness of&#10;the proposed method, as it surpasses state-of-the-art in terms of&#10;reconstruction accuracy while achieving comparably accurate relighting to&#10;ray-traced ground truth.&#10;" />
-          <attvalue for="2" value="&#10;&#10;3D reconstruction and relighting are of great importance in human digitization, especially in supporting realistic rendering in varying virtual environments, that can be widely applied in AR/VR \cite{ma2021pixel, orts2016holoportation}, holographic communication \cite{lawrence2021project, zhang2022virtualcube}, movie and gaming industry \cite{debevec2000pursuing}. &#10;&#10;Traditional methods often require dense camera setups using multi-view stereo, non-rigid registration and texture mapping \cite{fanello2014learning, guo2015robust}. To enhance capture realism, researchers have extended them with additional synchronous variable illumination systems, which aid photometric stereo for detail reconstruction and material acquisition \cite{guo2019relightables}. However, these systems are often too complex, expensive and difficult to maintain, thus preventing widespread applications.&#10;&#10;By leveraging deep prior and neural representation, sophisticated dense camera setups can be reduced to a single camera, leading to blossoms in learning-based human reconstruction. In particular, encoding human geometry and appearance as continuous fields using Multi-Layer Perceptron (MLP) has emerged as a promising lead. Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}. However, single-view reconstruction quality is restricted by its inherent depth ambiguity, thus limiting its application under view-consistent high-quality requirements.&#10;&#10;Therefore, as the trade-off between view coverage and system accessibility, sparse-view reconstruction has become a research hotpot. The predominant practice is to project the query point onto each view to interpolate local features, which are then aggregated and fed to MLP for inference \cite{saito2019pifu, yu2021function4d, suo2021neuralhumanfvv, shao2022diffustereo, chibane2020implicit, huang2021dynamic}. This method suffers from occlusion ambiguity, where some views may well be occluded, and mixing their features with visible ones causes inefficient feature utilization, thus penalizing the reconstruction quality \cite{saito2019pifu}. A natural solution is to filter features based on view visibility. Human templates such as SMPL \cite{loper2015smpl} can serve as effective guidance \cite{bhatnagar2020ipnet, zheng2021deepmulticap, peng2021neural, verbin2022ref}, but introduce additional template alignment errors and therefore cannot guarantee complete occlusion awareness. Function4D \cite{yu2021function4d} leverages the truncated Projective Signed Distance Function (PSDF) for visibility indication, but its level of details is susceptible to depth noise.&#10;&#10;To this end, we directly model a continuous visibility field, which can be efficiently learned with our proposed framework and discretization technique using sparse-view RGB-D input. The visibility field enables efficient visibility query, which effectively guides multi-view feature aggregation for more accurate occupancy and albedo inference. Moreover, visibility can also be directly used for light attenuation evaluation--the key ingredient in achieving realistic self-shadowing. When supervising jointly with our novel TransferLoss, the alignment between the visibility field and occupancy field can be implicitly enforced without between-field constraints, such as matching visibility with occupancy ray integral. We train our framework end-to-end and demonstrate its effectiveness in detailed 3D human reconstruction by quantity and quality comparison with the state-of-the-art. We directly relight the reconstructed geometry with inferred visibility using diffuse Bidirectional Reflectance Distribution Function (BRDF) as in \ref{fig_teaser}, which achieves photo-realistic self-shadowing without any post ray-tracing steps. To conclude, our contributions include:&#10;\begin{itemize}&#10; \item An end-to-end framework for sparse-view detailed 3D human reconstruction that also supports direct self-shadowed relighting. &#10; \item A novel method of visibility field learning, with the specifically designed TransferLoss significantly improves field alignment.&#10; \item A visibility-guided multi-view feature aggregation strategy that guarantees occlusion awareness.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Computer Vision, Machine Learning, Digital Human Relighting, Mathematics, Physics, Computer Science, Computer Graphics" />
-        </attvalues>
-      </node>
-      <node id="2104.04638" label="2104.04638">
-        <attvalues>
-          <attvalue for="0" value="Pixel Codec Avatars" />
-          <attvalue for="1" value="  Telecommunication with photorealistic avatars in virtual or augmented reality&#10;is a promising path for achieving authentic face-to-face communication in 3D&#10;over remote physical distances. In this work, we present the Pixel Codec&#10;Avatars (PiCA): a deep generative model of 3D human faces that achieves state&#10;of the art reconstruction performance while being computationally efficient and&#10;adaptive to the rendering conditions during execution. Our model combines two&#10;core ideas: (1) a fully convolutional architecture for decoding spatially&#10;varying features, and (2) a rendering-adaptive per-pixel decoder. Both&#10;techniques are integrated via a dense surface representation that is learned in&#10;a weakly-supervised manner from low-topology mesh tracking over training&#10;images. We demonstrate that PiCA improves reconstruction over existing&#10;techniques across testing expressions and views on persons of different gender&#10;and skin tone. Importantly, we show that the PiCA model is much smaller than&#10;the state-of-art baseline model, and makes multi-person telecommunicaiton&#10;possible: on a single Oculus Quest 2 mobile VR headset, 5 avatars are rendered&#10;in realtime in the same scene.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.00049" label="1901.00049">
-        <attvalues>
-          <attvalue for="0" value="SiCloPe: Silhouette-Based Clothed People" />
-          <attvalue for="1" value="  We introduce a new silhouette-based representation for modeling clothed human&#10;bodies using deep generative models. Our method can reconstruct a complete and&#10;textured 3D model of a person wearing clothes from a single input picture.&#10;Inspired by the visual hull algorithm, our implicit representation uses 2D&#10;silhouettes and 3D joints of a body pose to describe the immense shape&#10;complexity and variations of clothed people. Given a segmented 2D silhouette of&#10;a person and its inferred 3D joints from the input picture, we first synthesize&#10;consistent silhouettes from novel view points around the subject. The&#10;synthesized silhouettes which are the most consistent with the input&#10;segmentation are fed into a deep visual hull algorithm for robust 3D shape&#10;prediction. We then infer the texture of the subject's back view using the&#10;frontal image and segmentation mask as input to a conditional generative&#10;adversarial network. Our experiments demonstrate that our silhouette-based&#10;model is an effective representation and the appearance of the back view can be&#10;predicted reliably using an image-to-image translation network. While classic&#10;methods based on parametric models often fail for single-view images of&#10;subjects with challenging clothing, our approach can still produce successful&#10;results, which are comparable to those obtained from multi-view input.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.05172" label="1905.05172">
-        <attvalues>
-          <attvalue for="0" value="PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human&#10;  Digitization" />
-          <attvalue for="1" value="  We introduce Pixel-aligned Implicit Function (PIFu), a highly effective&#10;implicit representation that locally aligns pixels of 2D images with the global&#10;context of their corresponding 3D object. Using PIFu, we propose an end-to-end&#10;deep learning method for digitizing highly detailed clothed humans that can&#10;infer both 3D surface and texture from a single image, and optionally, multiple&#10;input images. Highly intricate shapes, such as hairstyles, clothing, as well as&#10;their variations and deformations can be digitized in a unified way. Compared&#10;to existing representations used for 3D deep learning, PIFu can produce&#10;high-resolution surfaces including largely unseen regions such as the back of a&#10;person. In particular, it is memory efficient unlike the voxel representation,&#10;can handle arbitrary topology, and the resulting surface is spatially aligned&#10;with the input image. Furthermore, while previous techniques are designed to&#10;process either a single image or multiple views, PIFu extends naturally to&#10;arbitrary number of views. We demonstrate high-resolution and robust&#10;reconstructions on real world images from the DeepFashion dataset, which&#10;contains a variety of challenging clothing types. Our method achieves&#10;state-of-the-art performance on a public benchmark and outperforms the prior&#10;work for clothed human digitization from a single image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.13988" label="2007.13988">
-        <attvalues>
-          <attvalue for="0" value="Monocular Real-Time Volumetric Performance Capture" />
-          <attvalue for="1" value="  We present the first approach to volumetric performance capture and&#10;novel-view rendering at real-time speed from monocular video, eliminating the&#10;need for expensive multi-view systems or cumbersome pre-acquisition of a&#10;personalized template model. Our system reconstructs a fully textured 3D human&#10;from each frame by leveraging Pixel-Aligned Implicit Function (PIFu). While&#10;PIFu achieves high-resolution reconstruction in a memory-efficient manner, its&#10;computationally expensive inference prevents us from deploying such a system&#10;for real-time applications. To this end, we propose a novel hierarchical&#10;surface localization algorithm and a direct rendering method without explicitly&#10;extracting surface meshes. By culling unnecessary regions for evaluation in a&#10;coarse-to-fine manner, we successfully accelerate the reconstruction by two&#10;orders of magnitude from the baseline without compromising the quality.&#10;Furthermore, we introduce an Online Hard Example Mining (OHEM) technique that&#10;effectively suppresses failure modes due to the rare occurrence of challenging&#10;examples. We adaptively update the sampling probability of the training data&#10;based on the current reconstruction accuracy, which effectively alleviates&#10;reconstruction artifacts. Our experiments and evaluations demonstrate the&#10;robustness of our system to various challenging angles, illuminations, poses,&#10;and clothing styles. We also show that our approach compares favorably with the&#10;state-of-the-art monocular performance capture. Our proposed approach removes&#10;the need for multi-view studio settings and enables a consumer-accessible&#10;solution for volumetric capture.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00452" label="2004.00452">
-        <attvalues>
-          <attvalue for="0" value="PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution&#10;  3D Human Digitization" />
-          <attvalue for="1" value="  Recent advances in image-based 3D human shape estimation have been driven by&#10;the significant improvement in representation power afforded by deep neural&#10;networks. Although current approaches have demonstrated the potential in real&#10;world settings, they still fail to produce reconstructions with the level of&#10;detail often present in the input images. We argue that this limitation stems&#10;primarily form two conflicting requirements; accurate predictions require large&#10;context, but precise predictions require high resolution. Due to memory&#10;limitations in current hardware, previous approaches tend to take low&#10;resolution images as input to cover large spatial context, and produce less&#10;precise (or low resolution) 3D estimates as a result. We address this&#10;limitation by formulating a multi-level architecture that is end-to-end&#10;trainable. A coarse level observes the whole image at lower resolution and&#10;focuses on holistic reasoning. This provides context to an fine level which&#10;estimates highly detailed geometry by observing higher-resolution images. We&#10;demonstrate that our approach significantly outperforms existing&#10;state-of-the-art techniques on single image human shape reconstruction by fully&#10;leveraging 1k-resolution input images.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.08906" label="2204.08906">
-        <attvalues>
-          <attvalue for="0" value="Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing" />
-          <attvalue for="1" value="  We present PHORHUM, a novel, end-to-end trainable, deep neural network&#10;methodology for photorealistic 3D human reconstruction given just a monocular&#10;RGB image. Our pixel-aligned method estimates detailed 3D geometry and, for the&#10;first time, the unshaded surface color together with the scene illumination.&#10;Observing that 3D supervision alone is not sufficient for high fidelity color&#10;reconstruction, we introduce patch-based rendering losses that enable reliable&#10;color reconstruction on visible parts of the human, and detailed and plausible&#10;color estimation for the non-visible parts. Moreover, our method specifically&#10;addresses methodological and practical limitations of prior work in terms of&#10;representing geometry, albedo, and illumination effects, in an end-to-end model&#10;where factors can be effectively disentangled. In extensive experiments, we&#10;demonstrate the versatility and robustness of our approach. Our&#10;state-of-the-art results validate the method qualitatively and for different&#10;metrics, for both geometric and color reconstruction.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11432" label="2007.11432">
-        <attvalues>
-          <attvalue for="0" value="Combining Implicit Function Learning and Parametric Models for 3D Human&#10;  Reconstruction" />
-          <attvalue for="1" value="  Implicit functions represented as deep learning approximations are powerful&#10;for reconstructing 3D surfaces. However, they can only produce static surfaces&#10;that are not controllable, which provides limited ability to modify the&#10;resulting model by editing its pose or shape parameters. Nevertheless, such&#10;features are essential in building flexible models for both computer graphics&#10;and computer vision. In this work, we present methodology that combines&#10;detail-rich implicit functions and parametric representations in order to&#10;reconstruct 3D models of people that remain controllable and accurate even in&#10;the presence of clothing. Given sparse 3D point clouds sampled on the surface&#10;of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict&#10;the outer 3D surface of the dressed person, the and inner body surface, and the&#10;semantic correspondences to a parametric body model. We subsequently use&#10;correspondences to fit the body model to our inner surface and then non-rigidly&#10;deform it (under a parametric body + displacement model) to the outer surface&#10;in order to capture garment, face and hair detail. In quantitative and&#10;qualitative experiments with both full body data and hand scans we show that&#10;the proposed methodology generalizes, and is effective even given incomplete&#10;point clouds collected from single-view depth images. Our models and code can&#10;be downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2406.04002" label="2406.04002">
-        <attvalues>
-          <attvalue for="0" value="3rd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation" />
-          <attvalue for="1" value="  Video panoptic segmentation is an advanced task that extends panoptic&#10;segmentation by applying its concept to video sequences. In the hope of&#10;addressing the challenge of video panoptic segmentation in diverse conditions,&#10;We utilize DVIS++ as our baseline model and enhance it by introducing a&#10;comprehensive approach centered on the query-wise ensemble, supplemented by&#10;additional techniques. Our proposed approach achieved a VPQ score of 57.01 on&#10;the VIPSeg test set, and ranked 3rd in the VPS track of the 3rd Pixel-level&#10;Video Understanding in the Wild Challenge.&#10;" />
-          <attvalue for="2" value="&#10;In order to unify the video tasks including recognition, detection, tracking and segmentation, Kim et al. propose Video Panoptic Segmentation(VPS) by extending the concept of panoptic segmentation from image domain to video domain~\cite{vps}. &#10;Video panoptic segmentation can be seen as a fusion of video semantic segmentation and video instance segmentation \cite{vis}. More specifically, in a VPS task, each pixel in an image is required to be assigned both a semantic label and an instance ID. &#10;The VPS task has received a lot of attention due to its wide application in downstream tesks including video editing, video understanding and autonomous driving.&#10;&#10;Early studies in VPS domain focused on adapting image-based panoramic segmentation methods to the video domain by leveraging temporal consistency across video frames. VPSNet\cite{vps} integrated a temporal pixel-level fusion module and an object tracking head into a image panoptic segmentation network UPSNet\cite{upsnet} to achieve panoptic video results. Video K-Net\cite{video-knet} utilizes cross-temporal kernel interaction based on K-Net \cite{knet} to simultaneously segment and track both &quot;things&quot; and &quot;stuff&quot; in videos. ViP-DeepLab\cite{vip-deeplab}, adapted from Panoptic-DeepLab\cite{panoptic-deeplab} achieves temporal alignment by utilizing offset predictions for consecutive frames, ensuring consistent instance grouping across the video sequence. Considering that earlier methods necessitated the use of several distinct networks and sophisticated post-processing, Max-DeepLab\cite{max-deeplab} proposed the first end-to-end model for panoptic segmentation that applies a pixel and memory dual-path modeling. MaXTron \cite{maxtron} integrates a mask transformer with trajectory attention to conduct VPS, enhancing temporal coherence with its within-clip and cross-clip tracking modules. &#10;&#10;Recently, DVIS \cite{dvis} introduces a decoupled framework that divides video segmentation into three independent sub-tasks: image segmentation, referring tracking and temporal refinement. Due to its outstanding performances, DVIS has achieved first place in the VPS track of 2nd PVUW Challenge in CVPR 2023. DVIS++\cite{dvis++} introduced denoising training and contrastive learning strategies into DVIS, resulting in a significant improvement and achieving state-of-the-art performance on VIPSeg\cite{vipseg} dataset. &#10;&#10;In this paper, we utilized DVIS++ as our baseline, and tried several techniques to deal with the inconsistency in the predicted results. Note that DVIS++ follows the set prediction paradigm, which generates superfluous queries, each representing a binary segmentation mask for a single class, usual practices of test time augmentation and model ensemble cannot be applied here directly. Therefore, we proposes an query-wise ensemble method to solve this problem.    &#10;&#10;With the help of the query-wise ensemble method mentioned above and other additional techniques, our approach achieved 57.01 VPQ on the test set of VIPSeg dataset and ranked 3rd in the VPS track of the 3nd PVUW Challenge in CVPR 2024.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Artificial Intelligence, Deep Learning, Computer Science, Video Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1901.03784" label="1901.03784">
-        <attvalues>
-          <attvalue for="0" value="UPSNet: A Unified Panoptic Segmentation Network" />
-          <attvalue for="1" value="  In this paper, we propose a unified panoptic segmentation network (UPSNet)&#10;for tackling the newly proposed panoptic segmentation task. On top of a single&#10;backbone residual network, we first design a deformable convolution based&#10;semantic segmentation head and a Mask R-CNN style instance segmentation head&#10;which solve these two subtasks simultaneously. More importantly, we introduce a&#10;parameter-free panoptic head which solves the panoptic segmentation via&#10;pixel-wise classification. It first leverages the logits from the previous two&#10;heads and then innovatively expands the representation for enabling prediction&#10;of an extra unknown class which helps better resolve the conflicts between&#10;semantic and instance segmentation. Additionally, it handles the challenge&#10;caused by the varying number of instances and permits back propagation to the&#10;bottom modules in an end-to-end manner. Extensive experimental results on&#10;Cityscapes, COCO and our internal dataset demonstrate that our UPSNet achieves&#10;state-of-the-art performance with much faster inference. Code has been made&#10;available at: https://github.com/uber-research/UPSNet&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.14855" label="2106.14855">
-        <attvalues>
-          <attvalue for="0" value="K-Net: Towards Unified Image Segmentation" />
-          <attvalue for="1" value="  Semantic, instance, and panoptic segmentations have been addressed using&#10;different and specialized frameworks despite their underlying connections. This&#10;paper presents a unified, simple, and effective framework for these essentially&#10;similar tasks. The framework, named K-Net, segments both instances and semantic&#10;categories consistently by a group of learnable kernels, where each kernel is&#10;responsible for generating a mask for either a potential instance or a stuff&#10;class. To remedy the difficulties of distinguishing various instances, we&#10;propose a kernel update strategy that enables each kernel dynamic and&#10;conditional on its meaningful group in the input image. K-Net can be trained in&#10;an end-to-end manner with bipartite matching, and its training and inference&#10;are naturally NMS-free and box-free. Without bells and whistles, K-Net&#10;surpasses all previous published state-of-the-art single-model results of&#10;panoptic segmentation on MS COCO test-dev split and semantic segmentation on&#10;ADE20K val split with 55.2% PQ and 54.3% mIoU, respectively. Its instance&#10;segmentation performance is also on par with Cascade Mask R-CNN on MS COCO with&#10;60%-90% faster inference speeds. Code and models will be released at&#10;https://github.com/ZwwWayne/K-Net/.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1911.10194" label="1911.10194">
-        <attvalues>
-          <attvalue for="0" value="Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up&#10;  Panoptic Segmentation" />
-          <attvalue for="1" value="  In this work, we introduce Panoptic-DeepLab, a simple, strong, and fast&#10;system for panoptic segmentation, aiming to establish a solid baseline for&#10;bottom-up methods that can achieve comparable performance of two-stage methods&#10;while yielding fast inference speed. In particular, Panoptic-DeepLab adopts the&#10;dual-ASPP and dual-decoder structures specific to semantic, and instance&#10;segmentation, respectively. The semantic segmentation branch is the same as the&#10;typical design of any semantic segmentation model (e.g., DeepLab), while the&#10;instance segmentation branch is class-agnostic, involving a simple instance&#10;center regression. As a result, our single Panoptic-DeepLab simultaneously&#10;ranks first at all three Cityscapes benchmarks, setting the new state-of-art of&#10;84.2% mIoU, 39.0% AP, and 65.5% PQ on test set. Additionally, equipped with&#10;MobileNetV3, Panoptic-DeepLab runs nearly in real-time with a single 1025x2049&#10;image (15.8 frames per second), while achieving a competitive performance on&#10;Cityscapes (54.1 PQ% on test set). On Mapillary Vistas test set, our ensemble&#10;of six models attains 42.7% PQ, outperforming the challenge winner in 2018 by a&#10;healthy margin of 1.5%. Finally, our Panoptic-DeepLab also performs on par with&#10;several top-down approaches on the challenging COCO dataset. For the first&#10;time, we demonstrate a bottom-up approach could deliver state-of-the-art&#10;results on panoptic segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.00759" label="2012.00759">
-        <attvalues>
-          <attvalue for="0" value="MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers" />
-          <attvalue for="1" value="  We present MaX-DeepLab, the first end-to-end model for panoptic segmentation.&#10;Our approach simplifies the current pipeline that depends heavily on surrogate&#10;sub-tasks and hand-designed components, such as box detection, non-maximum&#10;suppression, thing-stuff merging, etc. Although these sub-tasks are tackled by&#10;area experts, they fail to comprehensively solve the target task. By contrast,&#10;our MaX-DeepLab directly predicts class-labeled masks with a mask transformer,&#10;and is trained with a panoptic quality inspired loss via bipartite matching.&#10;Our mask transformer employs a dual-path architecture that introduces a global&#10;memory path in addition to a CNN path, allowing direct communication with any&#10;CNN layers. As a result, MaX-DeepLab shows a significant 7.1% PQ gain in the&#10;box-free regime on the challenging COCO dataset, closing the gap between&#10;box-based and box-free methods for the first time. A small variant of&#10;MaX-DeepLab improves 3.0% PQ over DETR with similar parameters and M-Adds.&#10;Furthermore, MaX-DeepLab, without test time augmentation, achieves new&#10;state-of-the-art 51.3% PQ on COCO test-dev set. Code is available at&#10;https://github.com/google-research/deeplab2.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.13305" label="2312.13305">
-        <attvalues>
-          <attvalue for="0" value="DVIS++: Improved Decoupled Framework for Universal Video Segmentation" />
-          <attvalue for="1" value="  We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS)&#10;framework, a novel approach for the challenging task of universal video&#10;segmentation, including video instance segmentation (VIS), video semantic&#10;segmentation (VSS), and video panoptic segmentation (VPS). Unlike previous&#10;methods that model video segmentation in an end-to-end manner, our approach&#10;decouples video segmentation into three cascaded sub-tasks: segmentation,&#10;tracking, and refinement. This decoupling design allows for simpler and more&#10;effective modeling of the spatio-temporal representations of objects,&#10;especially in complex scenes and long videos. Accordingly, we introduce two&#10;novel components: the referring tracker and the temporal refiner. These&#10;components track objects frame by frame and model spatio-temporal&#10;representations based on pre-aligned features. To improve the tracking&#10;capability of DVIS, we propose a denoising training strategy and introduce&#10;contrastive learning, resulting in a more robust framework named DVIS++.&#10;Furthermore, we evaluate DVIS++ in various settings, including open vocabulary&#10;and using a frozen pre-trained backbone. By integrating CLIP with DVIS++, we&#10;present OV-DVIS++, the first open-vocabulary universal video segmentation&#10;framework. We conduct extensive experiments on six mainstream benchmarks,&#10;including the VIS, VSS, and VPS datasets. Using a unified architecture, DVIS++&#10;significantly outperforms state-of-the-art specialized methods on these&#10;benchmarks in both close- and open-vocabulary settings.&#10;Code:~\url{https://github.com/zhang-tao-whu/DVIS_Plus}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2009.09960" label="2009.09960">
-        <attvalues>
-          <attvalue for="0" value="Towards Fast, Accurate and Stable 3D Dense Face Alignment" />
-          <attvalue for="1" value="  Existing methods of 3D dense face alignment mainly concentrate on accuracy,&#10;thus limiting the scope of their practical applications. In this paper, we&#10;propose a novel regression framework named 3DDFA-V2 which makes a balance among&#10;speed, accuracy and stability. Firstly, on the basis of a lightweight backbone,&#10;we propose a meta-joint optimization strategy to dynamically regress a small&#10;set of 3DMM parameters, which greatly enhances speed and accuracy&#10;simultaneously. To further improve the stability on videos, we present a&#10;virtual synthesis method to transform one still image to a short-video which&#10;incorporates in-plane and out-of-plane face moving. On the premise of high&#10;accuracy and stability, 3DDFA-V2 runs at over 50fps on a single CPU core and&#10;outperforms other state-of-the-art heavy models simultaneously. Experiments on&#10;several challenging datasets validate the efficiency of our method. Pre-trained&#10;models and code are available at https://github.com/cleardusk/3DDFA_V2.&#10;" />
-          <attvalue for="2" value="&#10;3D dense face alignment is essential for many face related tasks, e.g., recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}. Recent studies are mainly divided into two categories: 3D Morphable Model (3DMM) parameters regression~\cite{jourabloo2016large,zhu2016face,liu2016joint,liu2017dense,tuan2017regressing,zhu2019face,3ddfa_cleardusk} and dense vertices regression~\cite{jackson2017large,feng2018joint}.&#10;Dense vertices regression methods directly regress the coordinates of all the 3D points (usually more than 20,000) through a fully convolutional network~\cite{jackson2017large,feng2018joint}, achieving the state-of-the-art performance.&#10;However, the resolution of reconstructed faces relies on the size of the feature map and these methods rely on heavy networks like hourglass~\cite{newell2016stacked} or its variants, which are slow and memory-consuming in inference.&#10;The natural way of speeding it up is to prune channels. We try to prune 77.5\% channels on the state-of-the-art PRNet~\cite{feng2018joint} to achieve real-time speed on CPU, but find the error greatly increases 44.8\% (3.62\% vs. 5.24\%).&#10;Besides, an obvious disadvantage is the presence of checkerboard artifacts due to the deconvolution operators, which is present in the supplementary material.&#10;&#10;Another strategy is to regress a small set of 3DMM parameters (usually less than 200). Compared with dense vertices, 3DMM parameters have low dimensionality and low redundancy, which are appropriate to regress by a lightweight network.&#10;However, different 3DMM parameters influence the reconstructed 3D face~\cite{zhu2016face} differently, making the regression challenging since we have to dynamically re-weight each parameter according to their importance during training. Cascaded structures~\cite{zhu2016face,liu2017dense,zhu2019face} are always adopted to progressively update the parameters but the computation cost is increased linearly with the number of cascaded stages.&#10;&#10;In this paper, we aim to accelerate the speed to CPU real time and achieve the state-of-the-art performance simultaneously. To this end, we choose to regress 3DMM parameters with a fast backbone, e.g. MobileNet. To handle the optimization problem of the parameters regression framework, we exploit two different loss terms WPDC and VDC~\cite{zhu2016face} (see Sec.~\ref{sec_meta_joint}) and propose our meta-joint optimization to combine the advantages of them.&#10;The meta-joint optimization looks ahead by $k$-steps with WPDC and VDC on the meta-train batches, then dynamically selects the better one according to the error on the meta-test batch. By doing so, the whole optimization converges faster and achieves better performance than the vanilla-joint optimization.&#10;Besides, a landmark-regression regularization is introduced to further alleviate the optimization problem to achieve higher accuracy.&#10;In addition to single image, 3D face applications on videos are becoming more and more popular~\cite{cao20133d,cao2018stabilized,kim2018deep,kim2019neural}, where reconstructing stable results across consecutive frames is important, but it is often ignored by recent methods~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face}.&#10;Video-based training~\cite{peng2016recurrent,liu2018two,dong2018supervision,tai2018towards} is always adopted to improve the stability in 2D face alignment.&#10;However, no video databases are publicly available for 3D dense face alignment.&#10;To address it, we propose a 3D aided short-video-synthesis method, which simulates both in-plane and out-of-plane face moving to transform one still image to a short video, so that our network can adjust results of consecutive frames.&#10;Experiments show our short-video-synthesis method significantly improves the stability on videos.&#10;&#10;In general, our proposed framework 3DDFA-V2 are (i) fast: It takes about 7.2ms with an single image as input (almost 24x faster than PRNet) and runs at over 50fps (19.2ms) on a single CPU core or over 130fps (7.2ms) on multiple CPU cores (i5-8259U processor), (ii) accurate: By dynamically optimizing 3DMM parameters through a novel meta-optimization strategy combining the fast WPDC and VDC, we surpass the state-of-the-art results~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face} under a strict computation burden in inference, and (iii) stable: In a mini-batch, one still image is transformed slightly and smoothly into a short synthetic video, involving both in-plane and out-of-plane rotations, which provides temporal information of adjacent frames for training.&#10;Extensive experimental results on four datasets show that the overall performance of our method is the best.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Computer Science, Computer Graphics, Face Alignment" />
-        </attvalues>
-      </node>
-      <node id="2003.07733" label="2003.07733">
-        <attvalues>
-          <attvalue for="0" value="Learning Meta Face Recognition in Unseen Domains" />
-          <attvalue for="1" value="  Face recognition systems are usually faced with unseen domains in real-world&#10;applications and show unsatisfactory performance due to their poor&#10;generalization. For example, a well-trained model on webface data cannot deal&#10;with the ID vs. Spot task in surveillance scenario. In this paper, we aim to&#10;learn a generalized model that can directly handle new unseen domains without&#10;any model updating. To this end, we propose a novel face recognition method via&#10;meta-learning named Meta Face Recognition (MFR). MFR synthesizes the&#10;source/target domain shift with a meta-optimization objective, which requires&#10;the model to learn effective representations not only on synthesized source&#10;domains but also on synthesized target domains. Specifically, we build&#10;domain-shift batches through a domain-level sampling strategy and get&#10;back-propagated gradients/meta-gradients on synthesized source/target domains&#10;by optimizing multi-domain distributions. The gradients and meta-gradients are&#10;further combined to update the model to improve generalization. Besides, we&#10;propose two benchmarks for generalized face recognition evaluation. Experiments&#10;on our benchmarks validate the generalization of our method compared to several&#10;baselines and other state-of-the-arts. The proposed benchmarks will be&#10;available at https://github.com/cleardusk/MFR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.01196" label="1806.01196">
-        <attvalues>
-          <attvalue for="0" value="Face Synthesis for Eyeglass-Robust Face Recognition" />
-          <attvalue for="1" value="  In the application of face recognition, eyeglasses could significantly&#10;degrade the recognition accuracy. A feasible method is to collect large-scale&#10;face images with eyeglasses for training deep learning methods. However, it is&#10;difficult to collect the images with and without glasses of the same identity,&#10;so that it is difficult to optimize the intra-variations caused by eyeglasses.&#10;In this paper, we propose to address this problem in a virtual synthesis&#10;manner. The high-fidelity face images with eyeglasses are synthesized based on&#10;3D face model and 3D eyeglasses. Models based on deep learning methods are then&#10;trained on the synthesized eyeglass face dataset, achieving better performance&#10;than previous ones. Experiments on the real face database validate the&#10;effectiveness of our synthesized data for improving eyeglass face recognition&#10;performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.01173" label="2001.01173">
-        <attvalues>
-          <attvalue for="0" value="Informative Sample Mining Network for Multi-Domain Image-to-Image&#10;  Translation" />
-          <attvalue for="1" value="  The performance of multi-domain image-to-image translation has been&#10;significantly improved by recent progress in deep generative models. Existing&#10;approaches can use a unified model to achieve translations between all the&#10;visual domains. However, their outcomes are far from satisfying when there are&#10;large domain variations. In this paper, we reveal that improving the sample&#10;selection strategy is an effective solution. To select informative samples, we&#10;dynamically estimate sample importance during the training of Generative&#10;Adversarial Networks, presenting Informative Sample Mining Network. We&#10;theoretically analyze the relationship between the sample importance and the&#10;prediction of the global optimal discriminator. Then a practical importance&#10;estimation function for general conditions is derived. Furthermore, we propose&#10;a novel multi-stage sample training scheme to reduce sample hardness while&#10;preserving sample informativeness. Extensive experiments on a wide range of&#10;specific image-to-image translation tasks are conducted, and the results&#10;demonstrate our superiority over current state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.05447" label="2102.05447">
-        <attvalues>
-          <attvalue for="0" value="Searching for Alignment in Face Recognition" />
-          <attvalue for="1" value="  A standard pipeline of current face recognition frameworks consists of four&#10;individual steps: locating a face with a rough bounding box and several&#10;fiducial landmarks, aligning the face image using a pre-defined template,&#10;extracting representations and comparing. Among them, face detection, landmark&#10;detection and representation learning have long been studied and a lot of works&#10;have been proposed. As an essential step with a significant impact on&#10;recognition performance, the alignment step has attracted little attention. In&#10;this paper, we first explore and highlight the effects of different alignment&#10;templates on face recognition. Then, for the first time, we try to search for&#10;the optimal template automatically. We construct a well-defined searching space&#10;by decomposing the template searching into the crop size and vertical shift,&#10;and propose an efficient method Face Alignment Policy Search (FAPS). Besides, a&#10;well-designed benchmark is proposed to evaluate the searched policy.&#10;Experiments on our proposed benchmark validate the effectiveness of our method&#10;to improve face recognition performance.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1203.6722" label="1203.6722">
-        <attvalues>
-          <attvalue for="0" value="Face Expression Recognition and Analysis: The State of the Art" />
-          <attvalue for="1" value="  The automatic recognition of facial expressions has been an active research&#10;topic since the early nineties. There have been several advances in the past&#10;few years in terms of face detection and tracking, feature extraction&#10;mechanisms and the techniques used for expression classification. This paper&#10;surveys some of the published work since 2001 till date. The paper presents a&#10;time-line view of the advances made in this field, the applications of&#10;automatic face expression recognizers, the characteristics of an ideal system,&#10;the databases that have been used and the advances made in terms of their&#10;standardization and a detailed summary of the state of the art. The paper also&#10;discusses facial parameterization using FACS Action Units (AUs) and MPEG-4&#10;Facial Animation Parameters (FAPs) and the recent advances in face detection,&#10;tracking and feature extraction methods. Notes have also been presented on&#10;emotions, expressions and facial features, discussion on the six prototypic&#10;expressions and the recent studies on expression classifiers. The paper ends&#10;with a note on the challenges and the future work. This paper has been written&#10;in a tutorial style with the intention of helping students and researchers who&#10;are new to this field.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.08472" label="1806.08472">
-        <attvalues>
-          <attvalue for="0" value="Learning a High Fidelity Pose Invariant Model for High-resolution Face&#10;  Frontalization" />
-          <attvalue for="1" value="  Face frontalization refers to the process of synthesizing the frontal view of&#10;a face from a given profile. Due to self-occlusion and appearance distortion in&#10;the wild, it is extremely challenging to recover faithful results and preserve&#10;texture details in a high-resolution. This paper proposes a High Fidelity Pose&#10;Invariant Model (HF-PIM) to produce photographic and identity-preserving&#10;results. HF-PIM frontalizes the profiles through a novel texture warping&#10;procedure and leverages a dense correspondence field to bind the 2D and 3D&#10;surface spaces. We decompose the prerequisite of warping into dense&#10;correspondence field estimation and facial texture map recovering, which are&#10;both well addressed by deep networks. Different from those reconstruction&#10;methods relying on 3D data, we also propose Adversarial Residual Dictionary&#10;Learning (ARDL) to supervise facial texture map recovering with only monocular&#10;images. Exhaustive experiments on both controlled and uncontrolled environments&#10;demonstrate that the proposed method not only boosts the performance of&#10;pose-invariant face recognition but also dramatically improves high-resolution&#10;frontalization appearances.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.08061" label="2003.08061">
-        <attvalues>
-          <attvalue for="0" value="Deep Spatial Gradient and Temporal Depth Learning for Face Anti-spoofing" />
-          <attvalue for="1" value="  Face anti-spoofing is critical to the security of face recognition systems.&#10;Depth supervised learning has been proven as one of the most effective methods&#10;for face anti-spoofing. Despite the great success, most previous works still&#10;formulate the problem as a single-frame multi-task one by simply augmenting the&#10;loss with depth, while neglecting the detailed fine-grained information and the&#10;interplay between facial depths and moving patterns. In contrast, we design a&#10;new approach to detect presentation attacks from multiple frames based on two&#10;insights: 1) detailed discriminative clues (e.g., spatial gradient magnitude)&#10;between living and spoofing face may be discarded through stacked vanilla&#10;convolutions, and 2) the dynamics of 3D moving faces provide important clues in&#10;detecting the spoofing faces. The proposed method is able to capture&#10;discriminative details via Residual Spatial Gradient Block (RSGB) and encode&#10;spatio-temporal information from Spatio-Temporal Propagation Module (STPM)&#10;efficiently. Moreover, a novel Contrastive Depth Loss is presented for more&#10;accurate depth supervision. To assess the efficacy of our method, we also&#10;collect a Double-modal Anti-spoofing Dataset (DMAD) which provides actual depth&#10;for each sample. The experiments demonstrate that the proposed approach&#10;achieves state-of-the-art results on five benchmark datasets including&#10;OULU-NPU, SiW, CASIA-MFSD, Replay-Attack, and the new DMAD. Codes will be&#10;available at https://github.com/clks-wzz/FAS-SGTD.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.04092" label="2003.04092">
-        <attvalues>
-          <attvalue for="0" value="Searching Central Difference Convolutional Networks for Face&#10;  Anti-Spoofing" />
-          <attvalue for="1" value="  Face anti-spoofing (FAS) plays a vital role in face recognition systems. Most&#10;state-of-the-art FAS methods 1) rely on stacked convolutions and&#10;expert-designed network, which is weak in describing detailed fine-grained&#10;information and easily being ineffective when the environment varies (e.g.,&#10;different illumination), and 2) prefer to use long sequence as input to extract&#10;dynamic features, making them difficult to deploy into scenarios which need&#10;quick response. Here we propose a novel frame level FAS method based on Central&#10;Difference Convolution (CDC), which is able to capture intrinsic detailed&#10;patterns via aggregating both intensity and gradient information. A network&#10;built with CDC, called the Central Difference Convolutional Network (CDCN), is&#10;able to provide more robust modeling capacity than its counterpart built with&#10;vanilla convolution. Furthermore, over a specifically designed CDC search&#10;space, Neural Architecture Search (NAS) is utilized to discover a more powerful&#10;network structure (CDCN++), which can be assembled with Multiscale Attention&#10;Fusion Module (MAFM) for further boosting performance. Comprehensive&#10;experiments are performed on six benchmark datasets to show that 1) the&#10;proposed method not only achieves superior performance on intra-dataset testing&#10;(especially 0.2% ACER in Protocol-1 of OULU-NPU dataset), 2) it also&#10;generalizes well on cross-dataset testing (particularly 6.5% HTER from&#10;CASIA-MFSD to Replay-Attack datasets). The codes are available at&#10;\href{https://github.com/ZitongYu/CDCN}{https://github.com/ZitongYu/CDCN}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.02157" label="2007.02157">
-        <attvalues>
-          <attvalue for="0" value="Face Anti-Spoofing with Human Material Perception" />
-          <attvalue for="1" value="  Face anti-spoofing (FAS) plays a vital role in securing the face recognition&#10;systems from presentation attacks. Most existing FAS methods capture various&#10;cues (e.g., texture, depth and reflection) to distinguish the live faces from&#10;the spoofing faces. All these cues are based on the discrepancy among physical&#10;materials (e.g., skin, glass, paper and silicone). In this paper we rephrase&#10;face anti-spoofing as a material recognition problem and combine it with&#10;classical human material perception [1], intending to extract discriminative&#10;and robust features for FAS. To this end, we propose the Bilateral&#10;Convolutional Networks (BCN), which is able to capture intrinsic material-based&#10;patterns via aggregating multi-level bilateral macro- and micro- information.&#10;Furthermore, Multi-level Feature Refinement Module (MFRM) and multi-head&#10;supervision are utilized to learn more robust features. Comprehensive&#10;experiments are performed on six benchmark datasets, and the proposed method&#10;achieves superior performance on both intra- and cross-dataset testings. One&#10;highlight is that we achieve overall 11.3$\pm$9.5\% EER for cross-type testing&#10;in SiW-M dataset, which significantly outperforms previous results. We hope&#10;this work will facilitate future cooperation between FAS and material&#10;communities.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.00488" label="1901.00488">
-        <attvalues>
-          <attvalue for="0" value="Improving Face Anti-Spoofing by 3D Virtual Synthesis" />
-          <attvalue for="1" value="  Face anti-spoofing is crucial for the security of face recognition systems.&#10;Learning based methods especially deep learning based methods need large-scale&#10;training samples to reduce overfitting. However, acquiring spoof data is very&#10;expensive since the live faces should be re-printed and re-captured in many&#10;views. In this paper, we present a method to synthesize virtual spoof data in&#10;3D space to alleviate this problem. Specifically, we consider a printed photo&#10;as a flat surface and mesh it into a 3D object, which is then randomly bent and&#10;rotated in 3D space. Afterward, the transformed 3D photo is rendered through&#10;perspective projection as a virtual sample. The synthetic virtual samples can&#10;significantly boost the anti-spoofing performance when combined with a proposed&#10;data balancing strategy. Our promising results open up new possibilities for&#10;advancing face anti-spoofing using cheap and large-scale synthetic data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1709.01442" label="1709.01442">
-        <attvalues>
-          <attvalue for="0" value="Dense Face Alignment" />
-          <attvalue for="1" value="  Face alignment is a classic problem in the computer vision field. Previous&#10;works mostly focus on sparse alignment with a limited number of facial landmark&#10;points, i.e., facial landmark detection. In this paper, for the first time, we&#10;aim at providing a very dense 3D alignment for large-pose face images. To&#10;achieve this, we train a CNN to estimate the 3D face shape, which not only&#10;aligns limited facial landmarks but also fits face contours and SIFT feature&#10;points. Moreover, we also address the bottleneck of training CNN with multiple&#10;datasets, due to different landmark markups on different datasets, such as 5,&#10;34, 68. Experimental results show our method not only provides high-quality,&#10;dense 3D face fitting but also outperforms the state-of-the-art facial landmark&#10;detection methods on the challenging datasets. Our model can run at real time&#10;during testing.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.01005" label="1804.01005">
-        <attvalues>
-          <attvalue for="0" value="Face Alignment in Full Pose Range: A 3D Total Solution" />
-          <attvalue for="1" value="  Face alignment, which fits a face model to an image and extracts the semantic&#10;meanings of facial pixels, has been an important topic in the computer vision&#10;community. However, most algorithms are designed for faces in small to medium&#10;poses (yaw angle is smaller than 45 degrees), which lack the ability to align&#10;faces in large poses up to 90 degrees. The challenges are three-fold. Firstly,&#10;the commonly used landmark face model assumes that all the landmarks are&#10;visible and is therefore not suitable for large poses. Secondly, the face&#10;appearance varies more drastically across large poses, from the frontal view to&#10;the profile view. Thirdly, labelling landmarks in large poses is extremely&#10;challenging since the invisible landmarks have to be guessed. In this paper, we&#10;propose to tackle these three challenges in an new alignment framework termed&#10;3D Dense Face Alignment (3DDFA), in which a dense 3D Morphable Model (3DMM) is&#10;fitted to the image via Cascaded Convolutional Neural Networks. We also utilize&#10;3D information to synthesize face images in profile views to provide abundant&#10;samples for training. Experiments on the challenging AFLW database show that&#10;the proposed approach achieves significant improvements over the&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1805.11714" label="1805.11714">
-        <attvalues>
-          <attvalue for="0" value="Deep Video Portraits" />
-          <attvalue for="1" value="  We present a novel approach that enables photo-realistic re-animation of&#10;portrait videos using only an input video. In contrast to existing approaches&#10;that are restricted to manipulations of facial expressions only, we are the&#10;first to transfer the full 3D head position, head rotation, face expression,&#10;eye gaze, and eye blinking from a source actor to a portrait video of a target&#10;actor. The core of our approach is a generative neural network with a novel&#10;space-time architecture. The network takes as input synthetic renderings of a&#10;parametric face model, based on which it predicts photo-realistic video frames&#10;for a given target actor. The realism in this rendering-to-video transfer is&#10;achieved by careful adversarial training, and as a result, we can create&#10;modified target videos that mimic the behavior of the synthetically-created&#10;input. In order to enable source-to-target video re-animation, we render a&#10;synthetic target video with the reconstructed head animation parameters from a&#10;source video, and feed it into the trained network -- thus taking full control&#10;of the target. With the ability to freely recombine source and target&#10;parameters, we are able to demonstrate a large variety of video rewrite&#10;applications without explicitly modeling hair, body or background. For&#10;instance, we can reenact the full head using interactive user-controlled&#10;editing, and realize high-fidelity visual dubbing. To demonstrate the high&#10;quality of our output, we conduct an extensive series of experiments and&#10;evaluations, where for instance a user study shows that our video edits are&#10;hard to detect.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.02518" label="1909.02518">
-        <attvalues>
-          <attvalue for="0" value="Neural Style-Preserving Visual Dubbing" />
-          <attvalue for="1" value="  Dubbing is a technique for translating video content from one language to&#10;another. However, state-of-the-art visual dubbing techniques directly copy&#10;facial expressions from source to target actors without considering&#10;identity-specific idiosyncrasies such as a unique type of smile. We present a&#10;style-preserving visual dubbing approach from single video inputs, which&#10;maintains the signature style of target actors when modifying facial&#10;expressions, including mouth motions, to match foreign languages. At the heart&#10;of our approach is the concept of motion style, in particular for facial&#10;expressions, i.e., the person-specific expression change that is yet another&#10;essential factor beyond visual accuracy in face editing applications. Our&#10;method is based on a recurrent generative adversarial network that captures the&#10;spatiotemporal co-activation of facial expressions, and enables generating and&#10;modifying the facial expressions of the target actor while preserving their&#10;style. We train our model with unsynchronized source and target videos in an&#10;unsupervised manner using cycle-consistency and mouth expression losses, and&#10;synthesize photorealistic video frames using a layered neural face renderer.&#10;Our approach generates temporally coherent results, and handles dynamic&#10;backgrounds. Our results show that our dubbing approach maintains the&#10;idiosyncratic style of the target actor better than previous approaches, even&#10;for widely differing source and target actors.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1608.05477" label="1608.05477">
-        <attvalues>
-          <attvalue for="0" value="A Recurrent Encoder-Decoder Network for Sequential Face Alignment" />
-          <attvalue for="1" value="  We propose a novel recurrent encoder-decoder network model for real-time&#10;video-based face alignment. Our proposed model predicts 2D facial point maps&#10;regularized by a regression loss, while uniquely exploiting recurrent learning&#10;at both spatial and temporal dimensions. At the spatial level, we add a&#10;feedback loop connection between the combined output response map and the&#10;input, in order to enable iterative coarse-to-fine face alignment using a&#10;single network model. At the temporal level, we first decouple the features in&#10;the bottleneck of the network into temporal-variant factors, such as pose and&#10;expression, and temporal-invariant factors, such as identity information.&#10;Temporal recurrent learning is then applied to the decoupled temporal-variant&#10;features, yielding better generalization and significantly more accurate&#10;results at test time. We perform a comprehensive experimental analysis, showing&#10;the importance of each component of our proposed model, as well as superior&#10;results over the state-of-the-art in standard datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.00342" label="1811.00342">
-        <attvalues>
-          <attvalue for="0" value="Towards Highly Accurate and Stable Face Alignment for High-Resolution&#10;  Videos" />
-          <attvalue for="1" value="  In recent years, heatmap regression based models have shown their&#10;effectiveness in face alignment and pose estimation. However, Conventional&#10;Heatmap Regression (CHR) is not accurate nor stable when dealing with&#10;high-resolution facial videos, since it finds the maximum activated location in&#10;heatmaps which are generated from rounding coordinates, and thus leads to&#10;quantization errors when scaling back to the original high-resolution space. In&#10;this paper, we propose a Fractional Heatmap Regression (FHR) for&#10;high-resolution video-based face alignment. The proposed FHR can accurately&#10;estimate the fractional part according to the 2D Gaussian function by sampling&#10;three points in heatmaps. To further stabilize the landmarks among continuous&#10;video frames while maintaining the precise at the same time, we propose a novel&#10;stabilization loss that contains two terms to address time delay and non-smooth&#10;issues, respectively. Experiments on 300W, 300-VW and Talking Face datasets&#10;clearly demonstrate that the proposed method is more accurate and stable than&#10;the state-of-the-art models.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.06185" label="1912.06185">
-        <attvalues>
-          <attvalue for="0" value="Learning Effective Visual Relationship Detector on 1 GPU" />
-          <attvalue for="1" value="  We present our winning solution to the Open Images 2019 Visual Relationship&#10;challenge. This is the largest challenge of its kind to date with nearly 9&#10;million training images. Challenge task consists of detecting objects and&#10;identifying relationships between them in complex scenes. Our solution has&#10;three stages, first object detection model is fine-tuned for the challenge&#10;classes using a novel weight transfer approach. Then, spatio-semantic and&#10;visual relationship models are trained on candidate object pairs. Finally,&#10;features and model predictions are combined to generate the final relationship&#10;prediction. Throughout the challenge we focused on minimizing the hardware&#10;requirements of our architecture. Specifically, our weight transfer approach&#10;enables much faster optimization, allowing the entire architecture to be&#10;trained on a single GPU in under two days. In addition to efficient&#10;optimization, our approach also achieves superior accuracy winning first place&#10;out of over 200 teams, and outperforming the second place team by over $5\%$ on&#10;the held-out private leaderboard.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Visual relationship detection is a core computer vision task that has gained a&#10;lot of attention recently~\cite{chen2019scene,yang2018graph,zellers2018scenegraphs,zhang2019vrd}. &#10;The task comprises of object detection followed by&#10;visual relationship prediction to identify relationships between pairs of objects.&#10;Relationship identification involves inferring complex spatial, semantic and visual &#10;information between objects in a given scene, which is a challenging task. &#10;Successfully solving this task is a natural first step towards scene understanding &#10;and reasoning. The Open Images 2019 Visual Relationship challenge introduces a uniquely &#10;large and diverse dataset of annotated images designed to benchmark visual relationship&#10;models in a standardised setting. The challenge dataset is based on the Open Images&#10;V5 dataset~\cite{kuznetsova2018open}, which contains 9 million images annotated with &#10;class labels, bounding boxes, segmentation masks and visual relationships.&#10;&#10;The challenge task is to detect objects and their associated&#10;relationships. The relationships include human-object relationships (e.g. ``man&#10;holding camera''), object-object relationships (e.g. ``spoon on table''), and&#10;object-attribute relationships (e.g. ``handbag is made of leather'').&#10;Each of the relationships can be expressed as a triplet, written as a pair of&#10;objects connected by a relationship predicate e.g. (``beer&quot;, ``on&quot;, ``table&quot;). Visual&#10;attributes are also triplets where object is connected with an attribute&#10;using the ``is'' relationship e.g. (``table&quot;, ``is&quot;, ``wooden''). The challenge &#10;contains 329 unique triplets, which span 57 different object classes, 5&#10;attributes, and 10 predicates. In this paper we present our solution which ranked &#10;first out of over 200 teams, and outperformed the second place team by over $5\%$ on &#10;the held-out private leaderboard. To make our approach more practical, we focus&#10;on minimizing the hardware requirements during training. Specifically, we show that &#10;through transfer learning we can significantly speed up optimization, and train the&#10;entire model on a {single} GPU in under two days.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Deep Learning Optimization, Visual Relationship, Mathematics, Artificial Intelligence, Object Detection, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1904.11622" label="1904.11622">
-        <attvalues>
-          <attvalue for="0" value="Scene Graph Prediction with Limited Labels" />
-          <attvalue for="1" value="  Visual knowledge bases such as Visual Genome power numerous applications in&#10;computer vision, including visual question answering and captioning, but suffer&#10;from sparse, incomplete relationships. All scene graph models to date are&#10;limited to training on a small set of visual relationships that have thousands&#10;of training labels each. Hiring human annotators is expensive, and using&#10;textual knowledge base completion methods are incompatible with visual data. In&#10;this paper, we introduce a semi-supervised method that assigns probabilistic&#10;relationship labels to a large number of unlabeled images using few labeled&#10;examples. We analyze visual relationships to suggest two types of&#10;image-agnostic features that are used to generate noisy heuristics, whose&#10;outputs are aggregated using a factor graph-based generative model. With as few&#10;as 10 labeled examples per relationship, the generative model creates enough&#10;training data to train any existing state-of-the-art scene graph model. We&#10;demonstrate that our method outperforms all baseline approaches on scene graph&#10;prediction by 5.16 recall@100 for PREDCLS. In our limited label setting, we&#10;define a complexity metric for relationships that serves as an indicator (R^2 =&#10;0.778) for conditions under which our method succeeds over transfer learning,&#10;the de-facto approach for training with limited labels.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.00191" label="1808.00191">
-        <attvalues>
-          <attvalue for="0" value="Graph R-CNN for Scene Graph Generation" />
-          <attvalue for="1" value="  We propose a novel scene graph generation model called Graph R-CNN, that is&#10;both effective and efficient at detecting objects and their relations in&#10;images. Our model contains a Relation Proposal Network (RePN) that efficiently&#10;deals with the quadratic number of potential relations between objects in an&#10;image. We also propose an attentional Graph Convolutional Network (aGCN) that&#10;effectively captures contextual information between objects and relations.&#10;Finally, we introduce a new evaluation metric that is more holistic and&#10;realistic than existing metrics. We report state-of-the-art performance on&#10;scene graph generation as evaluated using both existing and our proposed&#10;metrics.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1711.06640" label="1711.06640">
-        <attvalues>
-          <attvalue for="0" value="Neural Motifs: Scene Graph Parsing with Global Context" />
-          <attvalue for="1" value="  We investigate the problem of producing structured graph representations of&#10;visual scenes. Our work analyzes the role of motifs: regularly appearing&#10;substructures in scene graphs. We present new quantitative insights on such&#10;repeated structures in the Visual Genome dataset. Our analysis shows that&#10;object labels are highly predictive of relation labels but not vice-versa. We&#10;also find that there are recurring patterns even in larger subgraphs: more than&#10;50% of graphs contain motifs involving at least two relations. Our analysis&#10;motivates a new baseline: given object detections, predict the most frequent&#10;relation between object pairs with the given labels, as seen in the training&#10;set. This baseline improves on the previous state-of-the-art by an average of&#10;3.6% relative improvement across evaluation settings. We then introduce Stacked&#10;Motif Networks, a new architecture designed to capture higher order motifs in&#10;scene graphs that further improves over our strong baseline by an average 7.1%&#10;relative gain. Our code is available at github.com/rowanz/neural-motifs.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2403.17994" label="2403.17994">
-        <attvalues>
-          <attvalue for="0" value="Solution for Point Tracking Task of ICCV 1st Perception Test Challenge&#10;  2023" />
-          <attvalue for="1" value="  This report proposes an improved method for the Tracking Any Point (TAP)&#10;task, which tracks any physical surface through a video. Several existing&#10;approaches have explored the TAP by considering the temporal relationships to&#10;obtain smooth point motion trajectories, however, they still suffer from the&#10;cumulative error caused by temporal prediction. To address this issue, we&#10;propose a simple yet effective approach called TAP with confident static points&#10;(TAPIR+), which focuses on rectifying the tracking of the static point in the&#10;videos shot by a static camera. To clarify, our approach contains two key&#10;components: (1) Multi-granularity Camera Motion Detection, which could identify&#10;the video sequence by the static camera shot. (2) CMR-based point trajectory&#10;prediction with one moving object segmentation approach to isolate the static&#10;point from the moving object. Our approach ranked first in the final test with&#10;a score of 0.46.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Deep learning techniques have attracted widespread attention in multiple research fields~\cite{0074ZGGZ22,YangHGXX23,YangWZX019,YangZZXJY23}. Integrated into single-point tracking, which focuses on locating and following a specific object or feature across consecutive frames, they play a critical role in computer vision for tasks such as scene understanding and action recognition. As shown in Figure \ref{fig: problem} (a), it entails identifying whether two pixels from different images of the same video correspond to the same point on a physical surface. In this competition, we adopted a zero-shot strategy and explored three approaches: OmniMotion\cite{abs-2306-05422}, TAPIR\cite{abs-2306-08637}, and Cotraker\cite{abs-2307-07635}. The experimental results show that OmniMotion and Cotraker had poor generalization on the Perception Test, while TAPIR performed relatively better, and thus it was used as the baseline.&#10;&#10;By analyzing TAPIR's motion trajectory predictions, we have identified two issues, as illustrated in Figure \ref{fig: problem} (b): i) Static Point Jitter: In static camera videos, points occasionally showed slight positional jitter despite being static in the Ground Truth Tracker. ii) Static Point Pseudo-Following: While predicting point trajectories, originally static points often shifted due to the coverage of moving objects, either following object movement or drifting in subsequent predictions.&#10;&#10;To deal with these challenges, we propose a two-stage approach, which is called TAPIR+. Firstly, a multi-granularity temporal shot motion detection algorithm was designed to distinguish between moving and static camera shots by assessing frame displacement at global and local scales. Secondly, we employed differential processing for videos shot by moving and static cameras. For moving camera videos, we used TAPIR for point trajectory prediction. For static camera videos, we applied a moving object segmentation algorithm to detect motion areas, considering them reliable for point trajectory prediction, while the remaining points were treated as static based on a static baseline.&#10;&#10;We introduce a multi-level error mitigation strategy, and its contributions can be summarized as follows:&#10;\begin{itemize}&#10;[itemsep=0pt,parsep=0pt,topsep=0pt,partopsep=0pt,leftmargin=*]&#10;&#10;\item Multi-granularity temporal shot motion detection algorithm to assess camera shot motion.&#10; \item Partition-based point trajectory prediction algorithm based on confident moving regions to alleviate jitter in static point predictions and misjudgments caused by occlusion.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Camera Motion Detection, Object Segmentation, Computer Science, Video Tracking, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="2306.05422" label="2306.05422">
-        <attvalues>
-          <attvalue for="0" value="Tracking Everything Everywhere All at Once" />
-          <attvalue for="1" value="  We present a new test-time optimization method for estimating dense and&#10;long-range motion from a video sequence. Prior optical flow or particle video&#10;tracking algorithms typically operate within limited temporal windows,&#10;struggling to track through occlusions and maintain global consistency of&#10;estimated motion trajectories. We propose a complete and globally consistent&#10;motion representation, dubbed OmniMotion, that allows for accurate, full-length&#10;motion estimation of every pixel in a video. OmniMotion represents a video&#10;using a quasi-3D canonical volume and performs pixel-wise tracking via&#10;bijections between local and canonical space. This representation allows us to&#10;ensure global consistency, track through occlusions, and model any combination&#10;of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and&#10;real-world footage show that our approach outperforms prior state-of-the-art&#10;methods by a large margin both quantitatively and qualitatively. See our&#10;project page for more results: http://omnimotion.github.io/&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2306.08637" label="2306.08637">
-        <attvalues>
-          <attvalue for="0" value="TAPIR: Tracking Any Point with per-frame Initialization and temporal&#10;  Refinement" />
-          <attvalue for="1" value="  We present a novel model for Tracking Any Point (TAP) that effectively tracks&#10;any queried point on any physical surface throughout a video sequence. Our&#10;approach employs two stages: (1) a matching stage, which independently locates&#10;a suitable candidate point match for the query point on every other frame, and&#10;(2) a refinement stage, which updates both the trajectory and query features&#10;based on local correlations. The resulting model surpasses all baseline methods&#10;by a significant margin on the TAP-Vid benchmark, as demonstrated by an&#10;approximate 20% absolute average Jaccard (AJ) improvement on DAVIS. Our model&#10;facilitates fast inference on long and high-resolution video sequences. On a&#10;modern GPU, our implementation has the capacity to track points faster than&#10;real-time, and can be flexibly extended to higher-resolution videos. Given the&#10;high-quality trajectories extracted from a large dataset, we demonstrate a&#10;proof-of-concept diffusion model which generates trajectories from static&#10;images, enabling plausible animations. Visualizations, source code, and&#10;pretrained models can be found on our project webpage.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.07635" label="2307.07635">
-        <attvalues>
-          <attvalue for="0" value="CoTracker: It is Better to Track Together" />
-          <attvalue for="1" value="  We introduce CoTracker, a transformer-based model that tracks dense points in&#10;a frame jointly across a video sequence. This differs from most existing&#10;state-of-the-art approaches that track points independently, ignoring their&#10;correlation. We show that joint tracking results in a significantly higher&#10;tracking accuracy and robustness. We also provide several technical&#10;innovations, including the concept of virtual tracks, which allows CoTracker to&#10;track 70k points jointly and simultaneously. Furthermore, CoTracker operates&#10;causally on short windows (hence, it is suitable for online tasks), but is&#10;trained by unrolling the windows across longer video sequences, which enables&#10;and significantly improves long-term tracking. We demonstrate qualitatively&#10;impressive tracking results, where points can be tracked for a long time even&#10;when they are occluded or leave the field of view. Quantitatively, CoTracker&#10;outperforms all recent trackers on standard benchmarks, often by a substantial&#10;margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2008.04378" label="2008.04378">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Deep Metric Learning with Transformed Attention Consistency&#10;  and Contrastive Clustering Loss" />
-          <attvalue for="1" value="  Existing approaches for unsupervised metric learning focus on exploring&#10;self-supervision information within the input image itself. We observe that,&#10;when analyzing images, human eyes often compare images against each other&#10;instead of examining images individually. In addition, they often pay attention&#10;to certain keypoints, image regions, or objects which are discriminative&#10;between image classes but highly consistent within classes. Even if the image&#10;is being transformed, the attention pattern will be consistent. Motivated by&#10;this observation, we develop a new approach to unsupervised deep metric&#10;learning where the network is learned based on self-supervision information&#10;across images instead of within one single image. To characterize the&#10;consistent pattern of human attention during image comparisons, we introduce&#10;the idea of transformed attention consistency. It assumes that visually similar&#10;images, even undergoing different image transforms, should share the same&#10;consistent visual attention map. This consistency leads to a pairwise&#10;self-supervision loss, allowing us to learn a Siamese deep neural network to&#10;encode and compare images against their transformed or matched pairs. To&#10;further enhance the inter-class discriminative power of the feature generated&#10;by this network, we adapt the concept of triplet loss from supervised metric&#10;learning to our unsupervised case and introduce the contrastive clustering&#10;loss. Our extensive experimental results on benchmark datasets demonstrate that&#10;our proposed method outperforms current state-of-the-art methods for&#10;unsupervised metric learning by a large margin.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Deep metric learning aims to learn discriminative features that can aggregate visually similar images into compact clusters in the high-dimensional feature space while separating images of different classes from each other.&#10;It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}.&#10;In supervised deep metric learning, we assume that the labels for training data are available. &#10;In this paper, we consider unsupervised deep metric learning where the image labels are not available.&#10;Learning directly and automatically from images in an unsupervised manner without human supervision represents &#10;a very important yet challenging task in computer vision and machine learning.&#10;&#10;Clustering is one of the earliest methods developed for unsupervised learning. &#10;Recently, motivated by the remarkable success of deep learning, researchers have started to develop unsupervised learning methods using deep neural networks \cite{caron2018deep}. Auto-encoder trains an encoder deep neural network to output feature representations with sufficient information to reconstruct input images by a paired decoder \cite{zhang2019aet}. &#10;As we know, during deep neural network training, the network model is updated and learned in an iterative and progressive manner so that the network output can match the target. In other works, deep neural networks need human supervision to provide ground-truth labels. However, in unsupervised learning, there are no labels available. To address this issue, researchers have exploited the unique characteristics of images and videos to create various self-supervised labels, objective functions, or loss functions, which essentially convert the unsupervised learning into a supervised one so that the deep neural networks can be successfully trained. &#10;For example, in DeepCluster \cite{caron2018deep}, clustering is used to generate pseudo labels for images. &#10;Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, &#10;solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms &#10;\cite{zhang2019aet}, etc. &#10;Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}.&#10;&#10;In this work, we propose to explore a new approach to unsupervised deep metric learning. We observe that existing methods for unsupervised metric learning focus on learning a network to analyze the input image itself. As we know, when examining and classifying images, human eyes compare images back and forth in order to identify discriminative features \cite{gazzaniga2009cognitive}. In other words, comparison plays an important role in human visual learning.&#10;When comparing images, they often pay attention to certain keypoints, image regions, or objects which are discriminative between image classes but highly consistent across image within classes. &#10;Even when the image is being transformed, the attention areas will be consistent. To further illustrate this, we provide three examples in Fig. \ref{fig:transformed-attention}. In (a), human eyes can easily tell the top image $A$ of the first column and the bottom image $B$ are the same bird since they have the same visual characteristics. The attention will be on the feather texture and head shape. In the pixel domain, $A$ and $B$ are up to a spatial transform, specifically, cropping plus resizing.&#10;When the human eyes moves from image $A$ to its transformed version $B$, the attention will be also transformed so that it can be still focused on the head and feather. If we represent this attention using the attention map in deep neural networks, the attention map ${\cal M}(A)$ for image $A$ and the attention map ${\cal M}(B)$ for image $B$ should also follow the same transform, as shown in the second column of Fig. \ref{fig:transformed-attention}(a). We can also see this consistency of attention across image under different transforms in other examples in Figs. \ref{fig:transformed-attention} (b) and (c). &#10;&#10;This lead to our idea of transformed attention consistency. Based on this idea, we develop a new approach to unsupervised deep metric learning based on image comparison. Specifically, using this consistency, we can define a pairwise self-supervision loss, allowing us to learn a Siamese deep neural network to encode and compare images against their transformed or matched pairs. To further enhance the inter-class discriminative power of the feature generated by this network, we adapt the concept of triplet loss from supervised metric learning to our unsupervised case and introduce the contrastive clustering loss. Our extensive experimental results on benchmark datasets demonstrate that our proposed method outperforms current state-of-the-art methods by a large margin. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Image Comparison Analysis, Machine Learning, Computer Vision, Deep Metric Learning, Self-Supervision Techniques, Mathematics, Artificial Intelligence, Cognitive Science, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1502.05908" label="1502.05908">
-        <attvalues>
-          <attvalue for="0" value="Learning Descriptors for Object Recognition and 3D Pose Estimation" />
-          <attvalue for="1" value="  Detecting poorly textured objects and estimating their 3D pose reliably is&#10;still a very challenging problem. We introduce a simple but powerful approach&#10;to computing descriptors for object views that efficiently capture both the&#10;object identity and 3D pose. By contrast with previous manifold-based&#10;approaches, we can rely on the Euclidean distance to evaluate the similarity&#10;between descriptors, and therefore use scalable Nearest Neighbor search methods&#10;to efficiently handle a large number of objects under a large range of poses.&#10;To achieve this, we train a Convolutional Neural Network to compute these&#10;descriptors by enforcing simple similarity and dissimilarity constraints&#10;between the descriptors. We show that our constraints nicely untangle the&#10;images from different objects and different views into clusters that are not&#10;only well-separated but also structured as the corresponding sets of poses: The&#10;Euclidean distance between descriptors is large when the descriptors are from&#10;different objects, and directly related to the distance between the poses when&#10;the descriptors are from the same object. These important properties allow us&#10;to outperform state-of-the-art object views representations on challenging RGB&#10;and RGB-D data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.06189" label="1803.06189">
-        <attvalues>
-          <attvalue for="0" value="Triplet-Center Loss for Multi-View 3D Object Retrieval" />
-          <attvalue for="1" value="  Most existing 3D object recognition algorithms focus on leveraging the strong&#10;discriminative power of deep learning models with softmax loss for the&#10;classification of 3D data, while learning discriminative features with deep&#10;metric learning for 3D object retrieval is more or less neglected. In the&#10;paper, we study variants of deep metric learning losses for 3D object&#10;retrieval, which did not receive enough attention from this area. First , two&#10;kinds of representative losses, triplet loss and center loss, are introduced&#10;which could learn more discriminative features than traditional classification&#10;loss. Then, we propose a novel loss named triplet-center loss, which can&#10;further enhance the discriminative power of the features. The proposed&#10;triplet-center loss learns a center for each class and requires that the&#10;distances between samples and centers from the same class are closer than those&#10;from different classes. Extensive experimental results on two popular 3D object&#10;retrieval benchmarks and two widely-adopted sketch-based 3D shape retrieval&#10;benchmarks consistently demonstrate the effectiveness of our proposed loss, and&#10;significant improvements have been achieved compared with the&#10;state-of-the-arts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1605.05863" label="1605.05863">
-        <attvalues>
-          <attvalue for="0" value="Siamese Instance Search for Tracking" />
-          <attvalue for="1" value="  In this paper we present a tracker, which is radically different from&#10;state-of-the-art trackers: we apply no model updating, no occlusion detection,&#10;no combination of trackers, no geometric matching, and still deliver&#10;state-of-the-art tracking performance, as demonstrated on the popular online&#10;tracking benchmark (OTB) and six very challenging YouTube videos. The presented&#10;tracker simply matches the initial patch of the target in the first frame with&#10;candidates in a new frame and returns the most similar patch by a learned&#10;matching function. The strength of the matching function comes from being&#10;extensively trained generically, i.e., without any data of the target, using a&#10;Siamese deep neural network, which we design for tracking. Once learned, the&#10;matching function is used as is, without any adapting, to track previously&#10;unseen targets. It turns out that the learned matching function is so powerful&#10;that a simple tracker built upon it, coined Siamese INstance search Tracker,&#10;SINT, which only uses the original observation of the target from the first&#10;frame, suffices to reach state-of-the-art performance. Further, we show the&#10;proposed tracker even allows for target re-identification after the target was&#10;absent for a complete video shot.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.11206" label="1807.11206">
-        <attvalues>
-          <attvalue for="0" value="Hard-Aware Point-to-Set Deep Metric for Person Re-identification" />
-          <attvalue for="1" value="  Person re-identification (re-ID) is a highly challenging task due to large&#10;variations of pose, viewpoint, illumination, and occlusion. Deep metric&#10;learning provides a satisfactory solution to person re-ID by training a deep&#10;network under supervision of metric loss, e.g., triplet loss. However, the&#10;performance of deep metric learning is greatly limited by traditional sampling&#10;methods. To solve this problem, we propose a Hard-Aware Point-to-Set (HAP2S)&#10;loss with a soft hard-mining scheme. Based on the point-to-set triplet loss&#10;framework, the HAP2S loss adaptively assigns greater weights to harder samples.&#10;Several advantageous properties are observed when compared with other&#10;state-of-the-art loss functions: 1) Accuracy: HAP2S loss consistently achieves&#10;higher re-ID accuracies than other alternatives on three large-scale benchmark&#10;datasets; 2) Robustness: HAP2S loss is more robust to outliers than other&#10;losses; 3) Flexibility: HAP2S loss does not rely on a specific weight function,&#10;i.e., different instantiations of HAP2S loss are equally effective. 4)&#10;Generality: In addition to person re-ID, we apply the proposed method to&#10;generic deep metric learning benchmarks including CUB-200-2011 and Cars196, and&#10;also achieve state-of-the-art results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.07737" label="1703.07737">
-        <attvalues>
-          <attvalue for="0" value="In Defense of the Triplet Loss for Person Re-Identification" />
-          <attvalue for="1" value="  In the past few years, the field of computer vision has gone through a&#10;revolution fueled mainly by the advent of large datasets and the adoption of&#10;deep convolutional neural networks for end-to-end learning. The person&#10;re-identification subfield is no exception to this. Unfortunately, a prevailing&#10;belief in the community seems to be that the triplet loss is inferior to using&#10;surrogate losses (classification, verification) followed by a separate metric&#10;learning step. We show that, for models trained from scratch as well as&#10;pretrained ones, using a variant of the triplet loss to perform end-to-end deep&#10;metric learning outperforms most other published methods by a large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.04596" label="1901.04596">
-        <attvalues>
-          <attvalue for="0" value="AET vs. AED: Unsupervised Representation Learning by Auto-Encoding&#10;  Transformations rather than Data" />
-          <attvalue for="1" value="  The success of deep neural networks often relies on a large amount of labeled&#10;examples, which can be difficult to obtain in many real scenarios. To address&#10;this challenge, unsupervised methods are strongly preferred for training neural&#10;networks without using any labeled data. In this paper, we present a novel&#10;paradigm of unsupervised representation learning by Auto-Encoding&#10;Transformation (AET) in contrast to the conventional Auto-Encoding Data (AED)&#10;approach. Given a randomly sampled transformation, AET seeks to predict it&#10;merely from the encoded features as accurately as possible at the output end.&#10;The idea is the following: as long as the unsupervised features successfully&#10;encode the essential information about the visual structures of original and&#10;transformed images, the transformation can be well predicted. We will show that&#10;this AET paradigm allows us to instantiate a large variety of transformations,&#10;from parameterized, to non-parameterized and GAN-induced ones. Our experiments&#10;show that AET greatly improves over existing unsupervised approaches, setting&#10;new state-of-the-art performances being greatly closer to the upper bounds by&#10;their fully supervised counterparts on CIFAR-10, ImageNet and Places datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.07728" label="1803.07728">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Representation Learning by Predicting Image Rotations" />
-          <attvalue for="1" value="  Over the last years, deep convolutional neural networks (ConvNets) have&#10;transformed the field of computer vision thanks to their unparalleled capacity&#10;to learn high level semantic image features. However, in order to successfully&#10;learn those features, they usually require massive amounts of manually labeled&#10;data, which is both expensive and impractical to scale. Therefore, unsupervised&#10;semantic feature learning, i.e., learning without requiring manual annotation&#10;effort, is of crucial importance in order to successfully harvest the vast&#10;amount of visual data that are available today. In our work we propose to learn&#10;image features by training ConvNets to recognize the 2d rotation that is&#10;applied to the image that it gets as input. We demonstrate both qualitatively&#10;and quantitatively that this apparently simple task actually provides a very&#10;powerful supervisory signal for semantic feature learning. We exhaustively&#10;evaluate our method in various unsupervised feature learning benchmarks and we&#10;exhibit in all of them state-of-the-art performance. Specifically, our results&#10;on those benchmarks demonstrate dramatic improvements w.r.t. prior&#10;state-of-the-art approaches in unsupervised representation learning and thus&#10;significantly close the gap with supervised feature learning. For instance, in&#10;PASCAL VOC 2007 detection task our unsupervised pre-trained AlexNet model&#10;achieves the state-of-the-art (among unsupervised methods) mAP of 54.4% that is&#10;only 2.4 points lower from the supervised case. We get similarly striking&#10;results when we transfer our unsupervised learned features on various other&#10;tasks, such as ImageNet classification, PASCAL classification, PASCAL&#10;segmentation, and CIFAR-10 classification. The code and models of our paper&#10;will be published on: https://github.com/gidariss/FeatureLearningRotNet .&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.13264" label="2401.13264">
-        <attvalues>
-          <attvalue for="0" value="Enhancing cross-domain detection: adaptive class-aware contrastive&#10;  transformer" />
-          <attvalue for="1" value="  Recently,the detection transformer has gained substantial attention for its&#10;inherent minimal post-processing requirement.However,this paradigm relies on&#10;abundant training data,yet in the context of the cross-domain&#10;adaptation,insufficient labels in the target domain exacerbate issues of class&#10;imbalance and model performance degradation.To address these challenges, we&#10;propose a novel class-aware cross domain detection transformer based on the&#10;adversarial learning and mean-teacher framework.First,considering the&#10;inconsistencies between the classification and regression tasks,we introduce an&#10;IoU-aware prediction branch and exploit the consistency of classification and&#10;location scores to filter and reweight pseudo labels.Second, we devise a&#10;dynamic category threshold refinement to adaptively manage model&#10;confidence.Third,to alleviate the class imbalance,an instance-level class-aware&#10;contrastive learning module is presented to encourage the generation of&#10;discriminative features for each class,particularly benefiting minority&#10;classes.Experimental results across diverse domain-adaptive scenarios validate&#10;our method's effectiveness in improving performance and alleviating class&#10;imbalance issues,which outperforms the state-of-the-art transformer based&#10;methods.&#10;" />
-          <attvalue for="2" value="&#10;In recent years, the object detection model based on transformer architecture has become a very potential detector with its complete end-to-end characteristics, no post-processing and better generalization performance \cite{tvt,daformer}. However, when testing in a scenario with domain gaps, its performance will still be significantly degraded due to different data distributions and domain shifts, such as weather changes and stylistic variations. This motivates us to explore the unsupervised domain adaptation based on transformer detector.&#10;&#10;Unsupervised domain adaptation aims to align a labeled source domain’s detector with an unlabeled target domain, bridging the domain gap \cite{SWDA,cross}.In cross-domain detection, we have the whole training images and complete labels of the source domain, while only have the image of the target domain. Almost all existing transformer-based detectors for domain adaptation apply adversarial learning to extract common features between different domains \cite{SFA,AQT}. Nevertheless, owing to the absence of instance labels for target domain, method based adversarial learning fails to exploit the latent category information in the target domain, still manifesting an obvious performance gaps compared to the fully supervised models. Inspired by the success of recent study \cite{MTTrans,cross-modal,enhanced} using self-training method, we link mean-teacher mechanism \cite{mean} with adversarial learning to generate pseudo labels for target domain, thus mining the information inside the data of target domain. Despite this effort, there still exists some obstacles in mean-teacher framework hindering the enhancement of detection performance.&#10;&#10;Existing work \cite{MTTrans} based on vanilla mean-teacher architecture filters the pseudo labels only using their classification scores with a unified static threshold. However, it ignores the inconsistency \cite{consistent} between classification and regression task in object detection, which means high confidence score do not always guarantee the localization accuracy \cite{soft}. Simply using classification scores to filter pseudo labels may leave the inaccurate pseudo boxes, which inevitably introduces bias into the detection model and leads to performance degradation. Besides, the prior study overlooked the bias issue introduced by class imbalance. Owing to the imbalance data distribution, the model tends to predict the majority classes while assigning lower confidence scores to minority classes. As shown in Fig \ref{fig:imbalance}, taking foggy cityscapes \cite{foggy} dataset as an example, there is a serious category imbalance issue in data distribution. Using uniform threshold across all categories will increase the false positives as the number of pseudo labels in majority classes, like car and person, far surpasses the count of ground truth. As a result, it will lead to inefficient training and biased supervision, amplifying the class imbalance issue.&#10;&#10;To overcome these obstacles, we propose the adaptive class-aware contrastive transformer model. To deal with inaccurate localization for pseudo boxes, we introduce an IoU prediction branch to refine and reweight pseudo-labels by leveraging both classification and localization scores. Furthermore, to address the issue of class imbalance, a Gaussian Mixture Model (GMM) \cite{semidetr, consistent} is implemented to generate adaptive threshold for each category, utilizing the scores of each category pseudo boxes. This module not only enable to stabilize the number of true positive samples for all classes pseudo labels at different training steps but also serve the parameters for subsequent category reweighting. Moreover, to further mitigate class imbalance issue, we introduce a class-aware contrastive learning module to enhance the similarity of the same class instances in terms of the feature spatial distribution and ensures the considerable spatial separation between different class instances, particularly for minority classes by reweighting.&#10;&#10;The main contributions of this paper are as follows: (1) We apply the consistency of classification and localization to filter and reweight pseudo labels, thereby acquiring higher quality pseudo boxes for robust training. (2) We introduce the GMM module derived adaptive category thresholds, which can reduce the false negatives for minority classes to mitigate biased supervision due to data class imbalance. (3) Design of an object-level class-aware contrastive learning module, which enhances feature separability specially for rare categories by reweighting and avoid overfitting of majority classes. (4) The proposed method shows exceptional performance across three representative domain-adaptive benchmarks, surpassing prior transformer-based methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Domain Adaptation, Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2108.05988" label="2108.05988">
-        <attvalues>
-          <attvalue for="0" value="TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation" />
-          <attvalue for="1" value="  Unsupervised domain adaptation (UDA) aims to transfer the knowledge learnt&#10;from a labeled source domain to an unlabeled target domain. Previous work is&#10;mainly built upon convolutional neural networks (CNNs) to learn&#10;domain-invariant representations. With the recent exponential increase in&#10;applying Vision Transformer (ViT) to vision tasks, the capability of ViT in&#10;adapting cross-domain knowledge, however, remains unexplored in the literature.&#10;To fill this gap, this paper first comprehensively investigates the&#10;transferability of ViT on a variety of domain adaptation tasks. Surprisingly,&#10;ViT demonstrates superior transferability over its CNNs-based counterparts with&#10;a large margin, while the performance can be further improved by incorporating&#10;adversarial adaptation. Notwithstanding, directly using CNNs-based adaptation&#10;strategies fails to take the advantage of ViT's intrinsic merits (e.g.,&#10;attention mechanism and sequential image representation) which play an&#10;important role in knowledge transfer. To remedy this, we propose an unified&#10;framework, namely Transferable Vision Transformer (TVT), to fully exploit the&#10;transferability of ViT for domain adaptation. Specifically, we delicately&#10;devise a novel and effective unit, which we term Transferability Adaption&#10;Module (TAM). By injecting learned transferabilities into attention blocks, TAM&#10;compels ViT focus on both transferable and discriminative features. Besides, we&#10;leverage discriminative clustering to enhance feature diversity and separation&#10;which are undermined during adversarial domain alignment. To verify its&#10;versatility, we perform extensive studies of TVT on four benchmarks and the&#10;experimental results demonstrate that TVT attains significant improvements&#10;compared to existing state-of-the-art UDA methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.14887" label="2111.14887">
-        <attvalues>
-          <attvalue for="0" value="DAFormer: Improving Network Architectures and Training Strategies for&#10;  Domain-Adaptive Semantic Segmentation" />
-          <attvalue for="1" value="  As acquiring pixel-wise annotations of real-world images for semantic&#10;segmentation is a costly process, a model can instead be trained with more&#10;accessible synthetic data and adapted to real images without requiring their&#10;annotations. This process is studied in unsupervised domain adaptation (UDA).&#10;Even though a large number of methods propose new adaptation strategies, they&#10;are mostly based on outdated network architectures. As the influence of recent&#10;network architectures has not been systematically studied, we first benchmark&#10;different network architectures for UDA and newly reveal the potential of&#10;Transformers for UDA semantic segmentation. Based on the findings, we propose a&#10;novel UDA method, DAFormer. The network architecture of DAFormer consists of a&#10;Transformer encoder and a multi-level context-aware feature fusion decoder. It&#10;is enabled by three simple but crucial training strategies to stabilize the&#10;training and to avoid overfitting to the source domain: While (1) Rare Class&#10;Sampling on the source domain improves the quality of the pseudo-labels by&#10;mitigating the confirmation bias of self-training toward common classes, (2) a&#10;Thing-Class ImageNet Feature Distance and (3) a learning rate warmup promote&#10;feature transfer from ImageNet pretraining. DAFormer represents a major advance&#10;in UDA. It improves the state of the art by 10.8 mIoU for GTA-to-Cityscapes and&#10;5.4 mIoU for Synthia-to-Cityscapes and enables learning even difficult classes&#10;such as train, bus, and truck well. The implementation is available at&#10;https://github.com/lhoyer/DAFormer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.04798" label="1812.04798">
-        <attvalues>
-          <attvalue for="0" value="Strong-Weak Distribution Alignment for Adaptive Object Detection" />
-          <attvalue for="1" value="  We propose an approach for unsupervised adaptation of object detectors from&#10;label-rich to label-poor domains which can significantly reduce annotation&#10;costs associated with detection. Recently, approaches that align distributions&#10;of source and target images using an adversarial loss have been proven&#10;effective for adapting object classifiers. However, for object detection, fully&#10;matching the entire distributions of source and target images to each other at&#10;the global image level may fail, as domains could have distinct scene layouts&#10;and different combinations of objects. On the other hand, strong matching of&#10;local features such as texture and color makes sense, as it does not change&#10;category level semantics. This motivates us to propose a novel method for&#10;detector adaptation based on strong local alignment and weak global alignment.&#10;Our key contribution is the weak alignment model, which focuses the adversarial&#10;alignment loss on images that are globally similar and puts less emphasis on&#10;aligning images that are globally dissimilar. Additionally, we design the&#10;strong domain alignment model to only look at local receptive fields of the&#10;feature map. We empirically verify the effectiveness of our method on four&#10;datasets comprising both large and small domain shifts. Our code is available&#10;at \url{https://github.com/VisionLearningGroup/DA_Detection}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.13216" label="2111.13216">
-        <attvalues>
-          <attvalue for="0" value="Cross-Domain Adaptive Teacher for Object Detection" />
-          <attvalue for="1" value="  We address the task of domain adaptation in object detection, where there is&#10;a domain gap between a domain with annotations (source) and a domain of&#10;interest without annotations (target). As an effective semi-supervised learning&#10;method, the teacher-student framework (a student model is supervised by the&#10;pseudo labels from a teacher model) has also yielded a large accuracy gain in&#10;cross-domain object detection. However, it suffers from the domain shift and&#10;generates many low-quality pseudo labels (\textit{e.g.,} false positives),&#10;which leads to sub-optimal performance. To mitigate this problem, we propose a&#10;teacher-student framework named Adaptive Teacher (AT) which leverages domain&#10;adversarial learning and weak-strong data augmentation to address the domain&#10;gap. Specifically, we employ feature-level adversarial training in the student&#10;model, allowing features derived from the source and target domains to share&#10;similar distributions. This process ensures the student model produces&#10;domain-invariant features. Furthermore, we apply weak-strong augmentation and&#10;mutual learning between the teacher model (taking data from the target domain)&#10;and the student model (taking data from both domains). This enables the teacher&#10;model to learn the knowledge from the student model without being biased to the&#10;source domain. We show that AT demonstrates superiority over existing&#10;approaches and even Oracle (fully-supervised) models by a large margin. For&#10;example, we achieve 50.9% (49.3%) mAP on Foggy Cityscape (Clipart1K), which is&#10;9.2% (5.2%) and 8.2% (11.0%) higher than previous state-of-the-art and Oracle,&#10;respectively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2107.12636" label="2107.12636">
-        <attvalues>
-          <attvalue for="0" value="Exploring Sequence Feature Alignment for Domain Adaptive Detection&#10;  Transformers" />
-          <attvalue for="1" value="  Detection transformers have recently shown promising object detection results&#10;and attracted increasing attention. However, how to develop effective domain&#10;adaptation techniques to improve its cross-domain performance remains&#10;unexplored and unclear. In this paper, we delve into this topic and empirically&#10;find that direct feature distribution alignment on the CNN backbone only brings&#10;limited improvements, as it does not guarantee domain-invariant sequence&#10;features in the transformer for prediction. To address this issue, we propose a&#10;novel Sequence Feature Alignment (SFA) method that is specially designed for&#10;the adaptation of detection transformers. Technically, SFA consists of a domain&#10;query-based feature alignment (DQFA) module and a token-wise feature alignment&#10;(TDA) module. In DQFA, a novel domain query is used to aggregate and align&#10;global context from the token sequence of both domains. DQFA reduces the domain&#10;discrepancy in global feature representations and object relations when&#10;deploying in the transformer encoder and decoder, respectively. Meanwhile, TDA&#10;aligns token features in the sequence from both domains, which reduces the&#10;domain gaps in local and instance-level feature representations in the&#10;transformer encoder and decoder, respectively. Besides, a novel bipartite&#10;matching consistency loss is proposed to enhance the feature discriminability&#10;for robust object detection. Experiments on three challenging benchmarks show&#10;that SFA outperforms state-of-the-art domain adaptive object detection methods.&#10;Code has been made available at: https://github.com/encounter1997/SFA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.12647" label="2207.12647">
-        <attvalues>
-          <attvalue for="0" value="Cross-Modal Causal Relational Reasoning for Event-Level Visual Question&#10;  Answering" />
-          <attvalue for="1" value="  Existing visual question answering methods often suffer from cross-modal&#10;spurious correlations and oversimplified event-level reasoning processes that&#10;fail to capture event temporality, causality, and dynamics spanning over the&#10;video. In this work, to address the task of event-level visual question&#10;answering, we propose a framework for cross-modal causal relational reasoning.&#10;In particular, a set of causal intervention operations is introduced to&#10;discover the underlying causal structures across visual and linguistic&#10;modalities. Our framework, named Cross-Modal Causal RelatIonal Reasoning&#10;(CMCIR), involves three modules: i) Causality-aware Visual-Linguistic Reasoning&#10;(CVLR) module for collaboratively disentangling the visual and linguistic&#10;spurious correlations via front-door and back-door causal interventions; ii)&#10;Spatial-Temporal Transformer (STT) module for capturing the fine-grained&#10;interactions between visual and linguistic semantics; iii) Visual-Linguistic&#10;Feature Fusion (VLFF) module for learning the global semantic-aware&#10;visual-linguistic representations adaptively. Extensive experiments on four&#10;event-level datasets demonstrate the superiority of our CMCIR in discovering&#10;visual-linguistic causal structures and achieving robust event-level visual&#10;question answering. The datasets, code, and models are available at&#10;https://github.com/HCPLab-SYSU/CMCIR.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.01780" label="1703.01780">
-        <attvalues>
-          <attvalue for="0" value="Mean teachers are better role models: Weight-averaged consistency&#10;  targets improve semi-supervised deep learning results" />
-          <attvalue for="1" value="  The recently proposed Temporal Ensembling has achieved state-of-the-art&#10;results in several semi-supervised learning benchmarks. It maintains an&#10;exponential moving average of label predictions on each training example, and&#10;penalizes predictions that are inconsistent with this target. However, because&#10;the targets change only once per epoch, Temporal Ensembling becomes unwieldy&#10;when learning large datasets. To overcome this problem, we propose Mean&#10;Teacher, a method that averages model weights instead of label predictions. As&#10;an additional benefit, Mean Teacher improves test accuracy and enables training&#10;with fewer labels than Temporal Ensembling. Without changing the network&#10;architecture, Mean Teacher achieves an error rate of 4.35% on SVHN with 250&#10;labels, outperforming Temporal Ensembling trained with 1000 labels. We also&#10;show that a good network architecture is crucial to performance. Combining Mean&#10;Teacher and Residual Networks, we improve the state of the art on CIFAR-10 with&#10;4000 labels from 10.55% to 6.28%, and on ImageNet 2012 with 10% of the labels&#10;from 35.24% to 9.11%.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2209.01589" label="2209.01589">
-        <attvalues>
-          <attvalue for="0" value="Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in&#10;  Semi-supervised Object Detection" />
-          <attvalue for="1" value="  In this study, we dive deep into the inconsistency of pseudo targets in&#10;semi-supervised object detection (SSOD). Our core observation is that the&#10;oscillating pseudo-targets undermine the training of an accurate detector. It&#10;injects noise into the student's training, leading to severe overfitting&#10;problems. Therefore, we propose a systematic solution, termed&#10;ConsistentTeacher, to reduce the inconsistency. First, adaptive anchor&#10;assignment~(ASA) substitutes the static IoU-based strategy, which enables the&#10;student network to be resistant to noisy pseudo-bounding boxes. Then we&#10;calibrate the subtask predictions by designing a 3D feature alignment&#10;module~(FAM-3D). It allows each classification feature to adaptively query the&#10;optimal feature vector for the regression task at arbitrary scales and&#10;locations. Lastly, a Gaussian Mixture Model (GMM) dynamically revises the score&#10;threshold of pseudo-bboxes, which stabilizes the number of ground truths at an&#10;early stage and remedies the unreliable supervision signal during training.&#10;ConsistentTeacher provides strong results on a large range of SSOD evaluations.&#10;It achieves 40.0 mAP with ResNet-50 backbone given only 10% of annotated&#10;MS-COCO data, which surpasses previous baselines using pseudo labels by around&#10;3 mAP. When trained on fully annotated MS-COCO with additional unlabeled data,&#10;the performance further increases to 47.7 mAP. Our code is available at&#10;\url{https://github.com/Adamdad/ConsistentTeacher}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.08095" label="2307.08095">
-        <attvalues>
-          <attvalue for="0" value="Semi-DETR: Semi-Supervised Object Detection with Detection Transformers" />
-          <attvalue for="1" value="  We analyze the DETR-based framework on semi-supervised object detection&#10;(SSOD) and observe that (1) the one-to-one assignment strategy generates&#10;incorrect matching when the pseudo ground-truth bounding box is inaccurate,&#10;leading to training inefficiency; (2) DETR-based detectors lack deterministic&#10;correspondence between the input query and its prediction output, which hinders&#10;the applicability of the consistency-based regularization widely used in&#10;current SSOD methods. We present Semi-DETR, the first transformer-based&#10;end-to-end semi-supervised object detector, to tackle these problems.&#10;Specifically, we propose a Stage-wise Hybrid Matching strategy that combines&#10;the one-to-many assignment and one-to-one assignment strategies to improve the&#10;training efficiency of the first stage and thus provide high-quality pseudo&#10;labels for the training of the second stage. Besides, we introduce a Crossview&#10;Query Consistency method to learn the semantic feature invariance of object&#10;queries from different views while avoiding the need to find deterministic&#10;query correspondence. Furthermore, we propose a Cost-based Pseudo Label Mining&#10;module to dynamically mine more pseudo boxes based on the matching cost of&#10;pseudo ground truth bounding boxes for consistency training. Extensive&#10;experiments on all SSOD settings of both COCO and Pascal VOC benchmark datasets&#10;show that our Semi-DETR method outperforms all state-of-the-art methods by&#10;clear margins. The PaddlePaddle version code1 is at&#10;https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/semi_det/semi_detr.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.07736" label="2108.07736">
-        <attvalues>
-          <attvalue for="0" value="A Hybrid Sparse-Dense Monocular SLAM System for Autonomous Driving" />
-          <attvalue for="1" value="  In this paper, we present a system for incrementally reconstructing a dense&#10;3D model of the geometry of an outdoor environment using a single monocular&#10;camera attached to a moving vehicle. Dense models provide a rich representation&#10;of the environment facilitating higher-level scene understanding, perception,&#10;and planning. Our system employs dense depth prediction with a hybrid mapping&#10;architecture combining state-of-the-art sparse features and dense fusion-based&#10;visual SLAM algorithms within an integrated framework. Our novel contributions&#10;include design of hybrid sparse-dense camera tracking and loop closure, and&#10;scale estimation improvements in dense depth prediction. We use the motion&#10;estimates from the sparse method to overcome the large and variable inter-frame&#10;displacement typical of outdoor vehicle scenarios. Our system then registers&#10;the live image with the dense model using whole-image alignment. This enables&#10;the fusion of the live frame and dense depth prediction into the model. Global&#10;consistency and alignment between the sparse and dense models are achieved by&#10;applying pose constraints from the sparse method directly within the&#10;deformation of the dense model. We provide qualitative and quantitative results&#10;for both trajectory estimation and surface reconstruction accuracy,&#10;demonstrating competitive performance on the KITTI dataset. Qualitative results&#10;of the proposed approach are illustrated in https://youtu.be/Pn2uaVqjskY.&#10;Source code for the project is publicly available at the following repository&#10;https://github.com/robotvisionmu/DenseMonoSLAM.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Over the past decade, approaches to fusion-based dense visual SLAM have demonstrated the ability to build high fidelity dense $3D$ models of an environment facilitating higher-level scene understanding, perception and planning \cite{kinectfusion}, \cite{kintinuous}, \cite{elasticfusion}, \cite{bundlefusion}. However, given that these techniques typically require active RGB-D sensors, their applicability in outdoor scenarios has been limited. On the other hand, sparse and semi-dense monocular systems have found a large degree of success in these environments but are limited in their resulting sparse or partially dense representations \cite{orbslam}, \cite{orbslam}, \cite{orbslam3}, \cite{orb_vi}, \cite{lsd_slam}, \cite{svo}, \cite{svo2}. Here, we investigate the potential of recent results in monocular dense depth prediction in combining these approaches' strengths. In particular, we employ dense depth prediction in developing a monocular SLAM system that achieves comparable accuracy to sparse camera tracking algorithms while allowing dense fusion-based modeling of the environment. We demonstrate the effectiveness of this approach by tracking and mapping an environment from a single monocular camera attached to a moving vehicle. \par&#10;&#10;Using passive vision sensors over active sensors, such as LiDAR, in autonomous driving SLAM systems has many advantages. A calibrated monocular camera is inexpensive, lightweight, and can be used as a direction sensor providing rich photometric and geometric measurements of the scene at every frame. In contrast, LiDAR is expensive, heavy, and produces a relatively sparse signal compared to the dense measurements provided by a vision sensor. Indeed, recent deep learning-based depth estimation and object detection indicates that the accuracy advantage of LiDAR over monocular vision-based systems is closing \cite{lidar}.\par&#10;&#10;On the other hand, monocular vision systems suffer from ambiguity and drift in the estimated scale of the reconstruction \cite{scaledrift-aware} and are not as robust as stereo and multi-sensor systems in texture-less regions and during fast camera motion \cite{orbslam3, orb_vi}. Furthermore, with the exception of a few techniques for dense and semi-dense map representations \cite{dtam, lsd_slam, semi_dense_odom}, these systems produce sparse reconstructions which only offer a coarse-grained description of the scene's geometry \cite{monoslam, ptam, graphslam, orbslam, orbslam2}.&#10;&#10;Multi-sensor platforms attempt to leverage the combined advantages of monocular, inertial, stereo, LiDAR, and GPS sensors. However, such platforms need careful internal and external calibration, which can be done online to maintain synchronization and geometric alignment between the sensors and, ultimately, achieve accurate performance. In the autonomous driving domain, even the most carefully calibrated rig will move and warp in real-time due to the effects of heat, wear and tear of mechanical parts like tires, variations in loading of the vehicle, and the knocks and bumps inherent in driving on roads \cite{stereo_continuous_calibration}. GPS comes with its complexities, requiring line-of-sight with at least four satellites to infer position, which can prove challenging in urban environments \cite{gps_solutions}.\par&#10;&#10;More recently, several systems have emerged that represent the state-of-the-art in traditional monocular SLAM, and visual-odometry performance \cite{orbslam3, orbslam2, dso, ldso, svo, svo2, lsd_slam}. The ORB-SLAM open-source SLAM frameworks \cite{orbslam,orbslam2,orbslam3} in particular represent the state-of-the-art in monocular SLAM. Similar to PTAM, ORB-SLAM \cite{orbslam2} splits the SLAM problem into sub-problems that can be solved in different threads; (i) a feature-based camera tracking thread that runs in real-time; (ii) a local mapping thread that takes keyframes extracted by the first thread and optimizes a local area of the map around this keyframe using bundle-adjustment (BA); and (iii) a global mapping thread that integrates loop closure constraints and maintains global consistency.\par&#10;&#10;\del{A class of v}\add{V}isual SLAM systems based around the concepts of dense alternation, every pixel fusion, and direct whole-image alignment-based camera tracking, have shown that it is possible to build dense, high-fidelity models of indoor scenes in real-time with active depth sensors such as the Microsoft Kinect\del{\cite{kinectfusion, elasticfusion, kintinuous, bundlefusion, badslam}}. \add{These systems and the models they produce have many benefits in downstream machine perceptions and spatial reasoning tasks.}\del{When it comes to downstream machine perception, and spatial reasoning tasks, these systems and the models they produce have many benefits.} They can produce `watertight' maps that model the continuous surface that underlies the scene. For example, this allows them to synthesize accurate novel views and realistically fuse augmentations into the scene for AR effects \cite{kinectfusion, elasticfusion, kintinuous, bundlefusion, badslam}. High-resolution dense models may become more necessary for robot planning in the future to accommodate a broader range of scenarios (e.g., analyzing surface geometry in unevenly paved roads \cite{dense_path_planning}, obstacle detection and avoidance \cite{lidar}, etc.).\par&#10;&#10;More and more deep learning is being used to improve the performance of SLAM systems. Broadly speaking, neural networks have been applied in two critical areas within \del{SLAM}\add{the SLAM problem}; in the representation of geometric and visual information \cite{codeslam, imap, representation_learning}, and in overcoming the ill-posedness of many of SLAM's subproblems, for example, depth estimation from monocular images \cite{unrectdepthnet, depth_ego_unsup, depth_eigen, left_right_consistency}. \del{Several systems have been presented that demonstrate hybrid approaches to monocular visual SLAM and VO \cite{codeslam, cnn_slam, d3vo}, while other approaches implement a fully learned SLAM pipeline \cite{deep_tam, deep_vo}. In comparison to this work, these systems are either not suited to fast, outdoor camera motion \cite{codeslam, deep_tam, cnn_slam} or do not produce dense models \cite{d3vo, deep_vo}}\add{Several systems have been presented that demonstrate hybrid approaches to monocular visual SLAM and VO \cite{deepfactors, codeslam, cnn_slam, d3vo}, while other approaches implement a fully learned SLAM pipeline \cite{deep_tam, deep_vo}. In comparison to this work, these systems have either not been designed with large-scale fast, outdoor camera motion in mind \cite{deepfactors, codeslam, deep_tam, cnn_slam}, or do not produce dense models \cite{d3vo, deep_vo}.}\par&#10;&#10;\add{The work most similar to \add{our aproach}\del{the current work} is the dense surfel-based fusion system of \cite{scalable_dense_surfel}, \del{in which a hybrid approach is proposed that combines the}\add{combining} camera tracking from a sparse SLAM system (ORB-SLAM2 \cite{orbslam2}) with a dense fusion-based back-end. The substantial difference with the current work is \del{in} the degree to which they couple the sparse and dense maps. In \cite{scalable_dense_surfel}, \add{a sub-mapping approach is employed where} each surfel is anchored to a reference keyframe from the sparse system. Local and global optimisations of the keyframe poses by the sparse system are reflected in the dense map by ensuring the pose of a surfel relative to its anchor keyframe remains constant. In contrast, our system maintains a much looser coupling between the sparse and dense representations. The sparse pose is used to initialise a dense alignment that registers the camera to the dense map allowing for fusion of the current frame. Loop closures from the sparse system are used to constrain a deformation graph that non-rigidly deforms the surfels themselves. In this way, the benefits of each system are leveraged, while the systems themselves are only indirectly connected through $6$-DOF transformations and $3D$ constraints. \add{Furthermore, w}e quantitatively demonstrate the effectiveness of our approach in outdoor automotive environments using deep-learned depth estimation.}\par&#10;&#10;\del{The contribution of this paper is the development of a system for live dense metric $3D$ reconstruction in outdoor environments using a single RGB camera attached to a moving vehicle. Although other researchers have reported semi-dense SLAM or dense VO approaches, this is the first fully dense SLAM system evaluated on automotive scenarios to the best of our knowledge. The novelty of our approach lies in the use of a dense depth prediction network \cite{unrectdepthnet} within a hybrid architecture combining state-of-the-art sparse feature tracking and dense fusion-based visual mapping algorithms. We improve our previous approach to dense depth prediction \cite{unrectdepthnet} for the SLAM application by the addition of new regularization losses and better scale estimation.}&#10;\add{The contribution of this paper is the development of a system for live dense metric $3D$ reconstruction in outdoor environments using a single RGB camera attached to a moving vehicle. Although other researchers have reported dense and semi-dense SLAM or dense VO approaches, to the best of our knowledge, this is the first fully dense SLAM system quantitatively evaluated on automotive scenarios. The novelty of our approach lies in the use of a dense depth prediction network \cite{unrectdepthnet} within a hybrid architecture, combining state-of-the-art sparse feature tracking and dense fusion-based visual mapping algorithms in a loosely coupled fashion. We improve our previous approach to dense depth prediction \cite{unrectdepthnet} for the SLAM application by the addition of new regularization losses and better scale estimation.}&#10;&#10;Using the motion estimates from the sparse method to overcome the large and variable inter-frame displacement typical of outdoor vehicle scenarios, our system then registers the live image with the dense model using whole-image alignment. This allows dense depth prediction and fusion of the live frame into the model. Global consistency and alignment between the sparse and dense models are achieved by applying pose constraints from the sparse method directly within the deformation of the dense model. We evaluate the method over the KITTI benchmark dataset, providing qualitative and quantitative results for both the trajectory and surface reconstruction accuracy.\par&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Dense Depth Prediction, Computer Vision, Engineering, Mathematics, Geometric Computing, Computer Science, Visual SLAM Algorithms, Robotics" />
-        </attvalues>
-      </node>
-      <node id="1604.01093" label="1604.01093">
-        <attvalues>
-          <attvalue for="0" value="BundleFusion: Real-time Globally Consistent 3D Reconstruction using&#10;  On-the-fly Surface Re-integration" />
-          <attvalue for="1" value="  Real-time, high-quality, 3D scanning of large-scale scenes is key to mixed&#10;reality and robotic applications. However, scalability brings challenges of&#10;drift in pose estimation, introducing significant errors in the accumulated&#10;model. Approaches often require hours of offline processing to globally correct&#10;model errors. Recent online methods demonstrate compelling results, but suffer&#10;from: (1) needing minutes to perform online correction preventing true&#10;real-time use; (2) brittle frame-to-frame (or frame-to-model) pose estimation&#10;resulting in many tracking failures; or (3) supporting only unstructured&#10;point-based representations, which limit scan quality and applicability. We&#10;systematically address these issues with a novel, real-time, end-to-end&#10;reconstruction framework. At its core is a robust pose estimation strategy,&#10;optimizing per frame for a global set of camera poses by considering the&#10;complete history of RGB-D input with an efficient hierarchical approach. We&#10;remove the heavy reliance on temporal tracking, and continually localize to the&#10;globally optimized frames instead. We contribute a parallelizable optimization&#10;framework, which employs correspondences based on sparse features and dense&#10;geometric and photometric matching. Our approach estimates globally optimized&#10;(i.e., bundle adjusted) poses in real-time, supports robust tracking with&#10;recovery from gross tracking failures (i.e., relocalization), and re-estimates&#10;the 3D model in real-time to ensure global consistency; all within a single&#10;framework. Our approach outperforms state-of-the-art online systems with&#10;quality on par to offline methods, but with unprecedented speed and scan&#10;completeness. Our framework leads to a comprehensive online scanning solution&#10;for large indoor environments, enabling ease of use and high-quality results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.11898" label="2007.11898">
-        <attvalues>
-          <attvalue for="0" value="ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial&#10;  and Multi-Map SLAM" />
-          <attvalue for="1" value="  This paper presents ORB-SLAM3, the first system able to perform visual,&#10;visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras,&#10;using pin-hole and fisheye lens models. The first main novelty is a&#10;feature-based tightly-integrated visual-inertial SLAM system that fully relies&#10;on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization&#10;phase. The result is a system that operates robustly in real-time, in small and&#10;large, indoor and outdoor environments, and is 2 to 5 times more accurate than&#10;previous approaches. The second main novelty is a multiple map system that&#10;relies on a new place recognition method with improved recall. Thanks to it,&#10;ORB-SLAM3 is able to survive to long periods of poor visual information: when&#10;it gets lost, it starts a new map that will be seamlessly merged with previous&#10;maps when revisiting mapped areas. Compared with visual odometry systems that&#10;only use information from the last few seconds, ORB-SLAM3 is the first system&#10;able to reuse in all the algorithm stages all previous information. This allows&#10;to include in bundle adjustment co-visible keyframes, that provide high&#10;parallax observations boosting accuracy, even if they are widely separated in&#10;time or if they come from a previous mapping session. Our experiments show&#10;that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems&#10;available in the literature, and significantly more accurate. Notably, our&#10;stereo-inertial SLAM achieves an average accuracy of 3.6 cm on the EuRoC drone&#10;and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting&#10;representative of AR/VR scenarios. For the benefit of the community we make&#10;public the source code.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.05949" label="1610.05949">
-        <attvalues>
-          <attvalue for="0" value="Visual-Inertial Monocular SLAM with Map Reuse" />
-          <attvalue for="1" value="  In recent years there have been excellent results in Visual-Inertial Odometry&#10;techniques, which aim to compute the incremental motion of the sensor with high&#10;accuracy and robustness. However these approaches lack the capability to close&#10;loops, and trajectory estimation accumulates drift even if the sensor is&#10;continually revisiting the same place. In this work we present a novel&#10;tightly-coupled Visual-Inertial Simultaneous Localization and Mapping system&#10;that is able to close loops and reuse its map to achieve zero-drift&#10;localization in already mapped areas. While our approach can be applied to any&#10;camera configuration, we address here the most general problem of a monocular&#10;camera, with its well-known scale ambiguity. We also propose a novel IMU&#10;initialization method, which computes the scale, the gravity direction, the&#10;velocity, and gyroscope and accelerometer biases, in a few seconds with high&#10;accuracy. We test our system in the 11 sequences of a recent micro-aerial&#10;vehicle public dataset achieving a typical scale factor error of 1% and&#10;centimeter precision. We compare to the state-of-the-art in visual-inertial&#10;odometry in sequences with revisiting, proving the better accuracy of our&#10;method due to map reuse and no drift accumulation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.07179" label="1812.07179">
-        <attvalues>
-          <attvalue for="0" value="Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object&#10;  Detection for Autonomous Driving" />
-          <attvalue for="1" value="  3D object detection is an essential task in autonomous driving. Recent&#10;techniques excel with highly accurate detection rates, provided the 3D input&#10;data is obtained from precise but expensive LiDAR technology. Approaches based&#10;on cheaper monocular or stereo imagery data have, until now, resulted in&#10;drastically lower accuracies --- a gap that is commonly attributed to poor&#10;image-based depth estimation. However, in this paper we argue that it is not&#10;the quality of the data but its representation that accounts for the majority&#10;of the difference. Taking the inner workings of convolutional neural networks&#10;into consideration, we propose to convert image-based depth maps to&#10;pseudo-LiDAR representations --- essentially mimicking the LiDAR signal. With&#10;this representation we can apply different existing LiDAR-based detection&#10;algorithms. On the popular KITTI benchmark, our approach achieves impressive&#10;improvements over the existing state-of-the-art in image-based performance ---&#10;raising the detection accuracy of objects within the 30m range from the&#10;previous state-of-the-art of 22% to an unprecedented 74%. At the time of&#10;submission our algorithm holds the highest entry on the KITTI 3D object&#10;detection leaderboard for stereo-image-based approaches. Our code is publicly&#10;available at https://github.com/mileyan/pseudo_lidar.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1610.06475" label="1610.06475">
-        <attvalues>
-          <attvalue for="0" value="ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D&#10;  Cameras" />
-          <attvalue for="1" value="  We present ORB-SLAM2 a complete SLAM system for monocular, stereo and RGB-D&#10;cameras, including map reuse, loop closing and relocalization capabilities. The&#10;system works in real-time on standard CPUs in a wide variety of environments&#10;from small hand-held indoors sequences, to drones flying in industrial&#10;environments and cars driving around a city. Our back-end based on bundle&#10;adjustment with monocular and stereo observations allows for accurate&#10;trajectory estimation with metric scale. Our system includes a lightweight&#10;localization mode that leverages visual odometry tracks for unmapped regions&#10;and matches to map points that allow for zero-drift localization. The&#10;evaluation on 29 popular public sequences shows that our method achieves&#10;state-of-the-art accuracy, being in most cases the most accurate SLAM solution.&#10;We publish the source code, not only for the benefit of the SLAM community, but&#10;with the aim of being an out-of-the-box SLAM solution for researchers in other&#10;fields.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.06676" label="2007.06676">
-        <attvalues>
-          <attvalue for="0" value="UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a&#10;  Generic Framework for Handling Common Camera Distortion Models" />
-          <attvalue for="1" value="  In classical computer vision, rectification is an integral part of multi-view&#10;depth estimation. It typically includes epipolar rectification and lens&#10;distortion correction. This process simplifies the depth estimation&#10;significantly, and thus it has been adopted in CNN approaches. However,&#10;rectification has several side effects, including a reduced field of view&#10;(FOV), resampling distortion, and sensitivity to calibration errors. The&#10;effects are particularly pronounced in case of significant distortion (e.g.,&#10;wide-angle fisheye cameras). In this paper, we propose a generic scale-aware&#10;self-supervised pipeline for estimating depth, euclidean distance, and visual&#10;odometry from unrectified monocular videos. We demonstrate a similar level of&#10;precision on the unrectified KITTI dataset with barrel distortion comparable to&#10;the rectified KITTI dataset. The intuition being that the rectification step&#10;can be implicitly absorbed within the CNN model, which learns the distortion&#10;model without increasing complexity. Our approach does not suffer from a&#10;reduced field of view and avoids computational costs for rectification at&#10;inference time. To further illustrate the general applicability of the proposed&#10;framework, we apply it to wide-angle fisheye cameras with 190$^\circ$&#10;horizontal field of view. The training framework UnRectDepthNet takes in the&#10;camera distortion model as an argument and adapts projection and unprojection&#10;functions accordingly. The proposed algorithm is evaluated further on the KITTI&#10;rectified dataset, and we achieve state-of-the-art results that improve upon&#10;our previous work FisheyeDistanceNet. Qualitative results on a distorted test&#10;scene video sequence indicate excellent performance&#10;https://youtu.be/K6pbx3bU4Ss.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.01900" label="1808.01900">
-        <attvalues>
-          <attvalue for="0" value="DeepTAM: Deep Tracking and Mapping" />
-          <attvalue for="1" value="  We present a system for keyframe-based dense camera tracking and depth map&#10;estimation that is entirely learned. For tracking, we estimate small pose&#10;increments between the current camera image and a synthetic viewpoint. This&#10;significantly simplifies the learning problem and alleviates the dataset bias&#10;for camera motions. Further, we show that generating a large number of pose&#10;hypotheses leads to more accurate predictions. For mapping, we accumulate&#10;information in a cost volume centered at the current depth estimate. The&#10;mapping network then combines the cost volume and the keyframe image to update&#10;the depth prediction, thereby effectively making use of depth measurements and&#10;image-based priors. Our approach yields state-of-the-art results with few&#10;images and is robust with respect to noisy camera poses. We demonstrate that&#10;the performance of our 6 DOF tracking competes with RGB-D tracking algorithms.&#10;We compare favorably against strong classic and deep learning powered dense&#10;depth algorithms.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.05049" label="2001.05049">
-        <attvalues>
-          <attvalue for="0" value="DeepFactors: Real-Time Probabilistic Dense Monocular SLAM" />
-          <attvalue for="1" value="  The ability to estimate rich geometry and camera motion from monocular&#10;imagery is fundamental to future interactive robotics and augmented reality&#10;applications. Different approaches have been proposed that vary in scene&#10;geometry representation (sparse landmarks, dense maps), the consistency metric&#10;used for optimising the multi-view problem, and the use of learned priors. We&#10;present a SLAM system that unifies these methods in a probabilistic framework&#10;while still maintaining real-time performance. This is achieved through the use&#10;of a learned compact depth map representation and reformulating three different&#10;types of errors: photometric, reprojection and geometric, which we make use of&#10;within standard factor graph software. We evaluate our system on trajectory&#10;estimation and depth reconstruction on real-world sequences and present various&#10;examples of estimated dense geometry.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.04250" label="1909.04250">
-        <attvalues>
-          <attvalue for="0" value="Real-time Scalable Dense Surfel Mapping" />
-          <attvalue for="1" value="  In this paper, we propose a novel dense surfel mapping system that scales&#10;well in different environments with only CPU computation. Using a sparse SLAM&#10;system to estimate camera poses, the proposed mapping system can fuse intensity&#10;images and depth images into a globally consistent model. The system is&#10;carefully designed so that it can build from room-scale environments to&#10;urban-scale environments using depth images from RGB-D cameras, stereo cameras&#10;or even a monocular camera. First, superpixels extracted from both intensity&#10;and depth images are used to model surfels in the system. superpixel-based&#10;surfels make our method both run-time efficient and memory efficient. Second,&#10;surfels are further organized according to the pose graph of the SLAM system to&#10;achieve $O(1)$ fusion time regardless of the scale of reconstructed models.&#10;Third, a fast map deformation using the optimized pose graph enables the map to&#10;achieve global consistency in real-time. The proposed surfel mapping system is&#10;compared with other state-of-the-art methods on synthetic datasets. The&#10;performances of urban-scale and room-scale reconstruction are demonstrated&#10;using the KITTI dataset and autonomous aggressive flights, respectively. The&#10;code is available for the benefit of the community.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.13070" label="2208.13070">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Face Presentation Attack Detection with Dynamic&#10;  Grayscale Snippets" />
-          <attvalue for="1" value="  Face presentation attack detection (PAD) plays an important role in defending&#10;face recognition systems against presentation attacks. The success of PAD&#10;largely relies on supervised learning that requires a huge number of labeled&#10;data, which is especially challenging for videos and often requires expert&#10;knowledge. To avoid the costly collection of labeled data, this paper presents&#10;a novel method for self-supervised video representation learning via motion&#10;prediction. To achieve this, we exploit the temporal consistency based on three&#10;RGB frames which are acquired at three different times in the video sequence.&#10;The obtained frames are then transformed into grayscale images where each image&#10;is specified to three different channels such as R(red), G(green), and B(blue)&#10;to form a dynamic grayscale snippet (DGS). Motivated by this, the labels are&#10;automatically generated to increase the temporal diversity based on DGS by&#10;using the different temporal lengths of the videos, which prove to be very&#10;helpful for the downstream task. Benefiting from the self-supervised nature of&#10;our method, we report the results that outperform existing methods on four&#10;public benchmarks, namely, Replay-Attack, MSU-MFSD, CASIA-FASD, and OULU-NPU.&#10;Explainability analysis has been carried out through LIME and Grad-CAM&#10;techniques to visualize the most important features used in the DGS.&#10;" />
-          <attvalue for="2" value="&#10;Facial recognition technology has been employed for numerous real-world applications, such as airport passenger screening, mobile phones, banking, and law enforcement surveillance. However, recent studies show that face recognition systems are vulnerable to presentation attacks, such as silicone masks, replay of video, and photo attacks. In particular, these spoofing attacks pose a great threat to face recognition systems and can throw off the technology. To protect the face recognition systems against spoofing attacks, many software-based or hardware-based methods have been introduced \cite{li2022face, zhang2020casia, wang2022domain, liu2021face}. These methods provide an improved detection but their performances degrade drastically under real-world variations (e.g., the quality of video recording devices and illuminations). Thus, facial recognition technology needs to be reinforced with presentation attack detection (PAD) techniques to secure the existing face biometric systems.&#10;&#10;One way to address the problem of spoofing consists of using temporal feature learning. The existing countermeasures can be roughly divided into three main categories: (i) extracting dynamic features through CNN network, e.g., based on optical flow \cite{li2022face}, (ii) using Spatio-temporal features based on two-stream CNN \cite{kim2020action}, and (iii) learning long-range (sequential) data e.g., through Recurrent Neural Networks (RNN) \cite{muhammad2022self}. Indeed, the widespread adoption of temporal feature learning for face PAD in supervised settings has achieved excellent results. However, the success of supervised feature learning relies on massive amounts of manually labeled data, which is expensive and requires great effort. Alternatively, a prominent paradigm is the so-called self-supervised learning (SSL) which aims to empower machines without explicit annotations and has shown potential in both image and video domains \cite{schiappa2022self}. SSL can be separated into two main categories: exploiting context and videos. &#10;&#10;Various methods have been proposed in context-based self-supervised learning by designing image-specific pretext tasks such as colorizing gray images, image in-painting, jigsaw puzzles of image patches, and rotations of images. Compared with static images, video-specific pretext tasks (e.g., predicting clip orders, time arrows, and paces) provide richer sources of “supervision” \cite{jaiswal2020survey}. Therefore, it is not trivial to expand the image-based approaches directly to the video domain due to dynamic and fine-grained movements or actions \cite{schiappa2022self}. &#10;&#10;In order to avoid human labeling, Walker {et al} \cite{walker2015dense} proposed to estimate the motion information based on dense optical flow. Their approach avoids data annotation by employing a convolutional neural network (CNN) for motion prediction. However, the optical flow algorithm increases computational complexity and the memory space in dealing with video data. Usman {et al} \cite{muhammad2022self} proposed a self-supervised learning for face PAD based on a temporal sequence sampling. In their approach, the expensive estimation of the global motion is needed, which causes to increase in the computational complexity of the method. Recently, the authors in \cite{muhammad2022adaptive} claimed that the most important patterns for determining the decision on PAD are consistent with motion cues associated with the artifacts, i.e., eye blinking, hand trembling, and head rotation. These motion cues are valuable and essential to examining real and spoofing attacks where the relative motion between the background and the face region can be vital. Although previous video-based PAD methods provide an improved performance \cite{muhammad2022self,muhammad2022adaptive}, such methods require complex feature engineering skills and increase the computational cost of the model. &#10;&#10;Inspired by the above discussion, our goal is to develop not only a computationally efficient video representation learning method for summarizing the motion information, but also extend to define the pretext task as a motion prediction problem for self-supervised learning. Specifically, we achieve this by dividing the video into a set of non-overlapping segments. Then, for each segment, three reference frames are chosen and transformed into three single-channel (i.e., grayscale) images. The final 3-channel image which we called dynamic grayscale snippet (DGS) is formed by assigning each into three different channels such as R(red), G(green), and B(blue), separately. Fig.1 displays some example images of the DGS method. One can see that the background has grayscales due to the static background while the motion is well separable by representing colors. &#10;&#10;Motivated by this, we present a self-supervised video representation learning where the pretext task is to predict the lengths of the videos based on the DGS method. In particular, the proposed DGS with different temporal lengths provides supervision to predict future motion that can be used for the downstream task. It is worth mentioning that the importance of DGS is not limited to SSL and has at least three main advantages. First, DGS can be input to any 2D CNN for a still image, where “still” captures the long-term motion dynamics in the video. Second, DGS decreases the risk of over-fitting of human faces, since the PAD videos consist of hundreds of frames repeating similar face patterns. Third, the representation provides a fast approximation in comparison to features obtained by complex processing such as optical flow \cite{horn1981determining}, or global motion estimation \cite{muhammad2022self} (we will further discuss this aspect in sectionIII). &#10;&#10;In summary, our main contribution is three-fold. (i) We propose a 3-channel image, termed a dynamic grayscale snippet (DGS) for motion information that enables the use of a 2D CNN for the temporal stream and reduces the computational complexity simultaneously. (ii) To mitigate the cost of collecting a large amount of labeled data, different temporal lengths based on DGS are used in self-supervised learning to complement pretext task that offers a powerful supervisory signal for video-based face PAD. (iii) The effectiveness of the proposed approach is evaluated on four standard databases and promising generalization ability is achieved especially using the official cross-test and intra-dataset evaluation protocols.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Video Representation Analysis, Mathematics, Face Recognition Security, Artificial Intelligence, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2207.00419" label="2207.00419">
-        <attvalues>
-          <attvalue for="0" value="Self-Supervised Learning for Videos: A Survey" />
-          <attvalue for="1" value="  The remarkable success of deep learning in various domains relies on the&#10;availability of large-scale annotated datasets. However, obtaining annotations&#10;is expensive and requires great effort, which is especially challenging for&#10;videos. Moreover, the use of human-generated annotations leads to models with&#10;biased learning and poor domain generalization and robustness. As an&#10;alternative, self-supervised learning provides a way for representation&#10;learning which does not require annotations and has shown promise in both image&#10;and video domains. Different from the image domain, learning video&#10;representations are more challenging due to the temporal dimension, bringing in&#10;motion and other environmental dynamics. This also provides opportunities for&#10;video-exclusive ideas that advance self-supervised learning in the video and&#10;multimodal domain. In this survey, we provide a review of existing approaches&#10;on self-supervised learning focusing on the video domain. We summarize these&#10;methods into four different categories based on their learning objectives: 1)&#10;pretext tasks, 2) generative learning, 3) contrastive learning, and 4)&#10;cross-modal agreement. We further introduce the commonly used datasets,&#10;downstream evaluation tasks, insights into the limitations of existing works,&#10;and the potential future directions in this area.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.00362" label="2011.00362">
-        <attvalues>
-          <attvalue for="0" value="A Survey on Contrastive Self-supervised Learning" />
-          <attvalue for="1" value="  Self-supervised learning has gained popularity because of its ability to&#10;avoid the cost of annotating large-scale datasets. It is capable of adopting&#10;self-defined pseudo labels as supervision and use the learned representations&#10;for several downstream tasks. Specifically, contrastive learning has recently&#10;become a dominant component in self-supervised learning methods for computer&#10;vision, natural language processing (NLP), and other domains. It aims at&#10;embedding augmented versions of the same sample close to each other while&#10;trying to push away embeddings from different samples. This paper provides an&#10;extensive review of self-supervised methods that follow the contrastive&#10;approach. The work explains commonly used pretext tasks in a contrastive&#10;learning setup, followed by different architectures that have been proposed so&#10;far. Next, we have a performance comparison of different methods for multiple&#10;downstream tasks such as image classification, object detection, and action&#10;recognition. Finally, we conclude with the limitations of the current methods&#10;and the need for further techniques and future directions to make substantial&#10;progress.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1505.00295" label="1505.00295">
-        <attvalues>
-          <attvalue for="0" value="Dense Optical Flow Prediction from a Static Image" />
-          <attvalue for="1" value="  Given a scene, what is going to move, and in what direction will it move?&#10;Such a question could be considered a non-semantic form of action prediction.&#10;In this work, we present a convolutional neural network (CNN) based approach&#10;for motion prediction. Given a static image, this CNN predicts the future&#10;motion of each and every pixel in the image in terms of optical flow. Our CNN&#10;model leverages the data in tens of thousands of realistic videos to train our&#10;model. Our method relies on absolutely no human labeling and is able to predict&#10;motion based on the context of the scene. Because our CNN model makes no&#10;assumptions about the underlying scene, it can predict future optical flow on a&#10;diverse set of scenarios. We outperform all previous approaches by large&#10;margins.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.11394" label="1903.11394">
-        <attvalues>
-          <attvalue for="0" value="Motion Deblurring with an Adaptive Network" />
-          <attvalue for="1" value="  In this paper, we address the problem of dynamic scene deblurring in the&#10;presence of motion blur. Restoration of images affected by severe blur&#10;necessitates a network design with a large receptive field, which existing&#10;networks attempt to achieve through simple increment in the number of generic&#10;convolution layers, kernel-size, or the scales at which the image is processed.&#10;However, increasing the network capacity in this manner comes at the expense of&#10;increase in model size and inference speed, and ignoring the non-uniform nature&#10;of blur. We present a new architecture composed of spatially adaptive residual&#10;learning modules that implicitly discover the spatially varying shifts&#10;responsible for non-uniform blur in the input image and learn to modulate the&#10;filters. This capability is complemented by a self-attentive module which&#10;captures non-local relationships among the intermediate features and enhances&#10;the receptive field. We then incorporate a spatiotemporal recurrent module in&#10;the design to also facilitate efficient video deblurring. Our networks can&#10;implicitly model the spatially-varying deblurring process, while dispensing&#10;with multi-scale processing and large filters entirely. Extensive qualitative&#10;and quantitative comparisons with prior art on benchmark dynamic scene&#10;deblurring datasets clearly demonstrate the superiority of the proposed&#10;networks via reduction in model-size and significant improvements in accuracy&#10;and speed, enabling almost real-time deblurring.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Motion deblurring is a challenging problem in computer vision due to its ill-posed nature. The past decade has witnessed significant advances in deblurring, wherein major efforts have gone into designing priors that are apt for recovering the underlying undistorted image and the camera trajectory \cite{xu2010two,pan2016robust,fergus2006removing,shan2008high,&#10; cho2009fast,joshi2008psf,krishnan2009fast,krishnan2011blind,xu2010two,&#10; pan2014deblurring,pan2016blind,vasu2017local,yan2017image, chandramouli2010inferring, paramanand2011depth, paramanand2013non, vijay2013non, rao2014inferring,nimisha2018unsupervised,mohan2019unconstrained,purohit2020region,mohan2021deep,paramanand2014shape,rao2014harnessing, nimisha2017blur,vasu2017local,nimisha2018generating,vasu2018non,purohit2018learning,purohit2019bringing}. An exhaustive survey of uniform blind deblurring algorithms can be found in \cite{lai2016comparative}. Few approaches~\cite{chakrabarti2016neural,schuler2013machine,schuler2016learning} have proposed hybrid algorithms where a Convolutional Neural Network (CNN) estimates the blur kernel, which is then used in an alternative optimization framework for recovering the latent image. &#10;&#10;However, these methods have been developed based on a rather strong constraint that the scene is planar and that the blur is governed by only camera motion. This precludes commonly occurring blur in most practical settings. Real-world blur arises from various sources including moving objects, camera shake and depth variations, causing different pixels to acquire different motion trajectories. A class of algorithms involve segmentation methods to relax the static and fronto-parallel scene assumption by independently restoring different blurred regions in the scene~\cite{hyun2013dynamic}. However, these methods depend heavily on an accurate segmentation-map. Few methods \cite{sun2015learning,gong2017motion} circumvent the segmentation stage by training CNNs to estimate locally linear blur kernels and feeding them to a non-uniform deblurring algorithm based on patch-level prior. However, they are limited in their capability when it comes to general dynamic scenes. &#10;&#10;Conventional approaches for video deblurring are based&#10;on image deblurring techniques (using priors on the latent sharp frames and the blur kernels) which remove uniform blurs \cite{cai2009blind,zhang2013multi} and non-uniform blur caused by rotational camera motion \cite{li2010generating,cho2012registration,zhang2014multi,zhang2015intra}. However, these approaches are applicable only under the strong assumption of static scenes and absence of depth-dependent distortions. The work in \cite{wulff2014modeling} proposed a segmentation-based approach to address different blurs in foreground and background regions. Kim et al. \cite{hyun2015generalized} further relaxed the constraint on the scene motion by parameterizing spatially varying blur kernel using optical flow.&#10;&#10;With the introduction of labeled realistic motion blur datasets \cite{su2017deep}, deep learning based approaches have been proposed to estimate sharp video frames in an end-to-end manner. Deep Video Deblurring (DVD)~\cite{su2017deep} is the first such work to address generalized video deblurring wherein a neural network accepts a stack of neighboring blurry frames for deblurring. They perform off-line stabilization of the blurred frames before feeding them to the network, which learns to exploit the information from multiple frames to deblur the central frame. Nevertheless, when images are heavily blurred, this method introduces temporal artifacts that become more visible after stabilization. Few methods have also been proposed for burst image deblurring \cite{wieschollek2017learning,aittala2018burst} which utilize number of observations with independent blurs to restore a scene, but are not trained for general video deblurring. Online Video Deblurring (OVD)~\cite{hyun2017online} presents a faster design for video deblurring which does not require frame alignment. It utilizes temporal connections to increase the receptive field of the network. Although OVD can handle large motion blur without adding a computational overhead, it lacks in accuracy and is not real-time.&#10;&#10;There are two major limitations shared by prior deblurring works. Firstly, the filters of a generic CNN are spatially invariant (with spatially-uniform receptive field), which is suboptimal to model the process of dynamic scene deblurring and limits their accuracy. Secondly, existing methods achieve high receptive field through networks with a large number of parameters and high computational footprint, making them unsuitable for real-time applications. As the only other work of this kind, \cite{zhang2018dynamic} recently proposed a design composed of multiple CNNs and Recurrent Neural Networks (RNN) to learn spatially varying weights for deblurring. However, their performance is inferior to the state-of-the-art~\cite{tao2018scale} in several aspects. Reaching a trade-off among inference time, accuracy of restoration, and receptive field is a non-trivial task which we address in this paper. We investigate position and motion-aware CNN architecture, which can efficiently handle multiple image segments undergoing motion with different magnitude and direction. &#10;&#10;Following recent developments, we adopt an end-to-end learning based approach to directly estimate the restored sharp image. For single image deblurring, we build a fully convolutional architecture equipped with filter-transformation and feature modulation capability suited for the task of motion deblurring. Our design hinges on the fact that motion blur is essentially an aggregation of various spatially varying transformations of the image, and a network that implicitly adapts to the location and direction of such motion, is a better candidate for the restoration task. Next, we address the problem of video deblurring, wherein we extend our single image deblurring network to exploit the redundancy across consecutive frames of a video to guide the process. To this end, we introduce spatio-temporal recurrence at frame and feature-level to efficiently restore sequences of blurred frames.&#10;&#10;Our network contains various layers to spatially transform intermediate filters as well as the feature maps. Its advantages over prior art are three-fold: 1. It is fully convolutional and parametrically efficient: deblurring can be achieved with just a single forward pass through a compact network. 2. Its components can be easily introduced into other architectures and trained in an end-to-end manner using conventional loss functions. 3. The transformations estimated by the network are dynamic and hence can be meaningfully interpreted for any test image. &#10;&#10;The efficiency of our architecture is demonstrated through comprehensive comparisons with the state-of-the-art on image and video deblurring benchmarks. While a majority of image and video deblurring networks contain $&gt;7$ million parameters, our model achieves superior performance at only a fraction of this size, while being computationally more efficient, resulting in real-time deblurring of images on a single GPU.\\&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Neural Network Design, Mathematics, Computer Science, Image Deblurring, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1804.02913" label="1804.02913">
-        <attvalues>
-          <attvalue for="0" value="Bringing Alive Blurred Moments" />
-          <attvalue for="1" value="  We present a solution for the goal of extracting a video from a single motion&#10;blurred image to sequentially reconstruct the clear views of a scene as beheld&#10;by the camera during the time of exposure. We first learn motion representation&#10;from sharp videos in an unsupervised manner through training of a convolutional&#10;recurrent video autoencoder network that performs a surrogate task of video&#10;reconstruction. Once trained, it is employed for guided training of a motion&#10;encoder for blurred images. This network extracts embedded motion information&#10;from the blurred image to generate a sharp video in conjunction with the&#10;trained recurrent video decoder. As an intermediate step, we also design an&#10;efficient architecture that enables real-time single image deblurring and&#10;outperforms competing methods across all factors: accuracy, speed, and&#10;compactness. Experiments on real scenes and standard datasets demonstrate the&#10;superiority of our framework over the state-of-the-art and its ability to&#10;generate a plausible sequence of temporally consistent sharp frames.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1603.04771" label="1603.04771">
-        <attvalues>
-          <attvalue for="0" value="A Neural Approach to Blind Motion Deblurring" />
-          <attvalue for="1" value="  We present a new method for blind motion deblurring that uses a neural&#10;network trained to compute estimates of sharp image patches from observations&#10;that are blurred by an unknown motion kernel. Instead of regressing directly to&#10;patch intensities, this network learns to predict the complex Fourier&#10;coefficients of a deconvolution filter to be applied to the input patch for&#10;restoration. For inference, we apply the network independently to all&#10;overlapping patches in the observed image, and average its outputs to form an&#10;initial estimate of the sharp image. We then explicitly estimate a single&#10;global blur kernel by relating this estimate to the observed image, and finally&#10;perform non-blind deconvolution with this kernel. Our method exhibits accuracy&#10;and robustness close to state-of-the-art iterative methods, while being much&#10;faster when parallelized on GPU hardware.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1406.7444" label="1406.7444">
-        <attvalues>
-          <attvalue for="0" value="Learning to Deblur" />
-          <attvalue for="1" value="  We describe a learning-based approach to blind image deconvolution. It uses a&#10;deep layered architecture, parts of which are borrowed from recent work on&#10;neural network learning, and parts of which incorporate computations that are&#10;specific to image deconvolution. The system is trained end-to-end on a set of&#10;artificially generated training examples, enabling competitive performance in&#10;blind deconvolution, both with respect to quality and runtime.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1503.00593" label="1503.00593">
-        <attvalues>
-          <attvalue for="0" value="Learning a Convolutional Neural Network for Non-uniform Motion Blur&#10;  Removal" />
-          <attvalue for="1" value="  In this paper, we address the problem of estimating and removing non-uniform&#10;motion blur from a single blurry image. We propose a deep learning approach to&#10;predicting the probabilistic distribution of motion blur at the patch level&#10;using a convolutional neural network (CNN). We further extend the candidate set&#10;of motion kernels predicted by the CNN using carefully designed image&#10;rotations. A Markov random field model is then used to infer a dense&#10;non-uniform motion blur field enforcing motion smoothness. Finally, motion blur&#10;is removed by a non-uniform deblurring model using patch-level image prior.&#10;Experimental evaluations show that our approach can effectively estimate and&#10;remove complex non-uniform motion blur that is not handled well by previous&#10;approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.02583" label="1612.02583">
-        <attvalues>
-          <attvalue for="0" value="From Motion Blur to Motion Flow: a Deep Learning Solution for Removing&#10;  Heterogeneous Motion Blur" />
-          <attvalue for="1" value="  Removing pixel-wise heterogeneous motion blur is challenging due to the&#10;ill-posed nature of the problem. The predominant solution is to estimate the&#10;blur kernel by adding a prior, but the extensive literature on the subject&#10;indicates the difficulty in identifying a prior which is suitably informative,&#10;and general. Rather than imposing a prior based on theory, we propose instead&#10;to learn one from the data. Learning a prior over the latent image would&#10;require modeling all possible image content. The critical observation&#10;underpinning our approach is thus that learning the motion flow instead allows&#10;the model to focus on the cause of the blur, irrespective of the image content.&#10;This is a much easier learning task, but it also avoids the iterative process&#10;through which latent image priors are typically applied. Our approach directly&#10;estimates the motion flow from the blurred image through a fully-convolutional&#10;deep neural network (FCN) and recovers the unblurred image from the estimated&#10;motion flow. Our FCN is the first universal end-to-end mapping from the blurred&#10;image to the dense motion flow. To train the FCN, we simulate motion flows to&#10;generate synthetic blurred-image-motion-flow pairs thus avoiding the need for&#10;human labeling. Extensive experiments on challenging realistic blurred images&#10;demonstrate that the proposed method outperforms the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1507.02438" label="1507.02438">
-        <attvalues>
-          <attvalue for="0" value="Generalized Video Deblurring for Dynamic Scenes" />
-          <attvalue for="1" value="  Several state-of-the-art video deblurring methods are based on a strong&#10;assumption that the captured scenes are static. These methods fail to deblur&#10;blurry videos in dynamic scenes. We propose a video deblurring method to deal&#10;with general blurs inherent in dynamic scenes, contrary to other methods. To&#10;handle locally varying and general blurs caused by various sources, such as&#10;camera shake, moving objects, and depth variation in a scene, we approximate&#10;pixel-wise kernel with bidirectional optical flows. Therefore, we propose a&#10;single energy model that simultaneously estimates optical flows and latent&#10;frames to solve our deblurring problem. We also provide a framework and&#10;efficient solvers to optimize the energy model. By minimizing the proposed&#10;energy function, we achieve significant improvements in removing blurs and&#10;estimating accurate optical flows in blurry frames. Extensive experimental&#10;results demonstrate the superiority of the proposed method in real and&#10;challenging videos that state-of-the-art methods fail in either deblurring or&#10;optical flow estimation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1708.04208" label="1708.04208">
-        <attvalues>
-          <attvalue for="0" value="Learning Blind Motion Deblurring" />
-          <attvalue for="1" value="  As handheld video cameras are now commonplace and available in every&#10;smartphone, images and videos can be recorded almost everywhere at anytime.&#10;However, taking a quick shot frequently yields a blurry result due to unwanted&#10;camera shake during recording or moving objects in the scene. Removing these&#10;artifacts from the blurry recordings is a highly ill-posed problem as neither&#10;the sharp image nor the motion blur kernel is known. Propagating information&#10;between multiple consecutive blurry observations can help restore the desired&#10;sharp image or video. Solutions for blind deconvolution based on neural&#10;networks rely on a massive amount of ground-truth data which is hard to&#10;acquire. In this work, we propose an efficient approach to produce a&#10;significant amount of realistic training data and introduce a novel recurrent&#10;network architecture to deblur frames taking temporal information into account,&#10;which can efficiently handle arbitrary spatial and temporal input sizes. We&#10;demonstrate the versatility of our approach in a comprehensive comparison on a&#10;number of challening real-world examples.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1704.03285" label="1704.03285">
-        <attvalues>
-          <attvalue for="0" value="Online Video Deblurring via Dynamic Temporal Blending Network" />
-          <attvalue for="1" value="  State-of-the-art video deblurring methods are capable of removing non-uniform&#10;blur caused by unwanted camera shake and/or object motion in dynamic scenes.&#10;However, most existing methods are based on batch processing and thus need&#10;access to all recorded frames, rendering them computationally demanding and&#10;time consuming and thus limiting their practical use. In contrast, we propose&#10;an online (sequential) video deblurring method based on a spatio-temporal&#10;recurrent network that allows for real-time performance. In particular, we&#10;introduce a novel architecture which extends the receptive field while keeping&#10;the overall size of the network small to enable fast execution. In doing so,&#10;our network is able to remove even large blur caused by strong camera shake&#10;and/or fast moving objects. Furthermore, we propose a novel network layer that&#10;enforces temporal consistency between consecutive frames by dynamic temporal&#10;blending which compares and adaptively (at test time) shares features obtained&#10;at different time steps. We show the superiority of the proposed method in an&#10;extensive experimental evaluation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.01770" label="1802.01770">
-        <attvalues>
-          <attvalue for="0" value="Scale-recurrent Network for Deep Image Deblurring" />
-          <attvalue for="1" value="  In single image deblurring, the &quot;coarse-to-fine&quot; scheme, i.e. gradually&#10;restoring the sharp image on different resolutions in a pyramid, is very&#10;successful in both traditional optimization-based methods and recent&#10;neural-network-based approaches. In this paper, we investigate this strategy&#10;and propose a Scale-recurrent Network (SRN-DeblurNet) for this deblurring task.&#10;Compared with the many recent learning-based approaches in [25], it has a&#10;simpler network structure, a smaller number of parameters and is easier to&#10;train. We evaluate our method on large-scale deblurring datasets with complex&#10;motion. Results show that our method can produce better quality results than&#10;state-of-the-arts, both quantitatively and qualitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2402.07220" label="2402.07220">
-        <attvalues>
-          <attvalue for="0" value="KVQ: Kwai Video Quality Assessment for Short-form Videos" />
-          <attvalue for="1" value="  Short-form UGC video platforms, like Kwai and TikTok, have been an emerging&#10;and irreplaceable mainstream media form, thriving on user-friendly engagement,&#10;and kaleidoscope creation, etc. However, the advancing content-generation&#10;modes, e.g., special effects, and sophisticated processing workflows, e.g.,&#10;de-artifacts, have introduced significant challenges to recent UGC video&#10;quality assessment: (i) the ambiguous contents hinder the identification of&#10;quality-determined regions. (ii) the diverse and complicated hybrid distortions&#10;are hard to distinguish. To tackle the above challenges and assist in the&#10;development of short-form videos, we establish the first large-scale&#10;Kaleidoscope short Video database for Quality assessment, termed KVQ, which&#10;comprises 600 user-uploaded short videos and 3600 processed videos through the&#10;diverse practical processing workflows, including pre-processing, transcoding,&#10;and enhancement. Among them, the absolute quality score of each video and&#10;partial ranking score among indistinguishable samples are provided by a team of&#10;professional researchers specializing in image processing. Based on this&#10;database, we propose the first short-form video quality evaluator, i.e., KSVQE,&#10;which enables the quality evaluator to identify the quality-determined&#10;semantics with the content understanding of large vision language models (i.e.,&#10;CLIP) and distinguish the distortions with the distortion understanding module.&#10;Experimental results have shown the effectiveness of KSVQE on our KVQ database&#10;and popular VQA databases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2308.00729" label="2308.00729">
-        <attvalues>
-          <attvalue for="0" value="Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video&#10;  Quality Assessment" />
-          <attvalue for="1" value="  Video quality assessment (VQA) has attracted growing attention in recent&#10;years. While the great expense of annotating large-scale VQA datasets has&#10;become the main obstacle for current deep-learning methods. To surmount the&#10;constraint of insufficient training data, in this paper, we first consider the&#10;complete range of video distribution diversity (\ie content, distortion,&#10;motion) and employ diverse pretrained models (\eg architecture, pretext task,&#10;pre-training dataset) to benefit quality representation. An Adaptive Diverse&#10;Quality-aware feature Acquisition (Ada-DQA) framework is proposed to capture&#10;desired quality-related features generated by these frozen pretrained models.&#10;By leveraging the Quality-aware Acquisition Module (QAM), the framework is able&#10;to extract more essential and relevant features to represent quality. Finally,&#10;the learned quality representation is utilized as supplementary supervisory&#10;information, along with the supervision of the labeled quality score, to guide&#10;the training of a relatively lightweight VQA model in a knowledge distillation&#10;manner, which largely reduces the computational cost during inference.&#10;Experimental results on three mainstream no-reference VQA benchmarks clearly&#10;show the superior performance of Ada-DQA in comparison with current&#10;state-of-the-art approaches without using extra training data of VQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.09853" label="2206.09853">
-        <attvalues>
-          <attvalue for="0" value="DisCoVQA: Temporal Distortion-Content Transformers for Video Quality&#10;  Assessment" />
-          <attvalue for="1" value="  The temporal relationships between frames and their influences on video&#10;quality assessment (VQA) are still under-studied in existing works. These&#10;relationships lead to two important types of effects for video quality.&#10;Firstly, some temporal variations (such as shaking, flicker, and abrupt scene&#10;transitions) are causing temporal distortions and lead to extra quality&#10;degradations, while other variations (e.g. those related to meaningful&#10;happenings) do not. Secondly, the human visual system often has different&#10;attention to frames with different contents, resulting in their different&#10;importance to the overall video quality. Based on prominent time-series&#10;modeling ability of transformers, we propose a novel and effective&#10;transformer-based VQA method to tackle these two issues. To better&#10;differentiate temporal variations and thus capture the temporal distortions, we&#10;design a transformer-based Spatial-Temporal Distortion Extraction (STDE)&#10;module. To tackle with temporal quality attention, we propose the&#10;encoder-decoder-like temporal content transformer (TCT). We also introduce the&#10;temporal sampling on features to reduce the input length for the TCT, so as to&#10;improve the learning effectiveness and efficiency of this module. Consisting of&#10;the STDE and the TCT, the proposed Temporal Distortion-Content Transformers for&#10;Video Quality Assessment (DisCoVQA) reaches state-of-the-art performance on&#10;several VQA benchmarks without any extra pre-training datasets and up to 10%&#10;better generalization ability than existing methods. We also conduct extensive&#10;ablation experiments to prove the effectiveness of each part in our proposed&#10;model, and provide visualizations to prove that the proposed modules achieve&#10;our intention on modeling these temporal issues. We will publish our codes and&#10;pretrained weights later.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.08522" label="2401.08522">
-        <attvalues>
-          <attvalue for="0" value="Video Quality Assessment Based on Swin TransformerV2 and Coarse to Fine&#10;  Strategy" />
-          <attvalue for="1" value="  The objective of non-reference video quality assessment is to evaluate the&#10;quality of distorted video without access to reference high-definition&#10;references. In this study, we introduce an enhanced spatial perception module,&#10;pre-trained on multiple image quality assessment datasets, and a lightweight&#10;temporal fusion module to address the no-reference visual quality assessment&#10;(NR-VQA) task. This model implements Swin Transformer V2 as a local-level&#10;spatial feature extractor and fuses these multi-stage representations through a&#10;series of transformer layers. Furthermore, a temporal transformer is utilized&#10;for spatiotemporal feature fusion across the video. To accommodate compressed&#10;videos of varying bitrates, we incorporate a coarse-to-fine contrastive&#10;strategy to enrich the model's capability to discriminate features from videos&#10;of different bitrates. This is an expanded version of the one-page abstract.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.17014" label="2309.17014">
-        <attvalues>
-          <attvalue for="0" value="FreqAlign: Excavating Perception-oriented Transferability for Blind&#10;  Image Quality Assessment from A Frequency Perspective" />
-          <attvalue for="1" value="  Blind Image Quality Assessment (BIQA) is susceptible to poor transferability&#10;when the distribution shift occurs, e.g., from synthesis degradation to&#10;authentic degradation. To mitigate this, some studies have attempted to design&#10;unsupervised domain adaptation (UDA) based schemes for BIQA, which intends to&#10;eliminate the domain shift through adversarial-based feature alignment.&#10;However, the feature alignment is usually taken at the low-frequency space of&#10;features since the global average pooling operation. This ignores the&#10;transferable perception knowledge in other frequency components and causes the&#10;sub-optimal solution for the UDA of BIQA. To overcome this, from a novel&#10;frequency perspective, we propose an effective alignment strategy, i.e.,&#10;Frequency Alignment (dubbed FreqAlign), to excavate the perception-oriented&#10;transferability of BIQA in the frequency space.&#10;  Concretely, we study what frequency components of features are more proper&#10;for perception-oriented alignment. Based on this, we propose to improve the&#10;perception-oriented transferability of BIQA by performing feature frequency&#10;decomposition and selecting the frequency components that contained the most&#10;transferable perception knowledge for alignment. To achieve a stable and&#10;effective frequency selection, we further propose the frequency movement with a&#10;sliding window to find the optimal frequencies for alignment, which is composed&#10;of three strategies, i.e., warm up with pre-training, frequency movement-based&#10;selection, and perturbation-based finetuning. Extensive experiments under&#10;different domain adaptation settings of BIQA have validated the effectiveness&#10;of our proposed method. The code will be released at&#10;https://github.com/lixinustc/Openworld-IQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14115" label="2104.14115">
-        <attvalues>
-          <attvalue for="0" value="LIQA: Lifelong Blind Image Quality Assessment" />
-          <attvalue for="1" value="  Existing blind image quality assessment (BIQA) methods are mostly designed in&#10;a disposable way and cannot evolve with unseen distortions adaptively, which&#10;greatly limits the deployment and application of BIQA models in real-world&#10;scenarios. To address this problem, we propose a novel Lifelong blind Image&#10;Quality Assessment (LIQA) approach, targeting to achieve the lifelong learning&#10;of BIQA. Without accessing to previous training data, our proposed LIQA can not&#10;only learn new distortions, but also mitigate the catastrophic forgetting of&#10;seen distortions. Specifically, we adopt the Split-and-Merge distillation&#10;strategy to train a single-head network that makes task-agnostic predictions.&#10;In the split stage, we first employ a distortion-specific generator to obtain&#10;the pseudo features of each seen distortion. Then, we use an auxiliary&#10;multi-head regression network to generate the predicted quality of each seen&#10;distortion. In the merge stage, we replay the pseudo features paired with&#10;pseudo labels to distill the knowledge of multiple heads, which can build the&#10;final regressed single head. Experimental results demonstrate that the proposed&#10;LIQA method can handle the continuous shifts of different distortion types and&#10;even datasets. More importantly, our LIQA model can achieve stable performance&#10;even if the task sequence is long.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.14489" label="2207.14489">
-        <attvalues>
-          <attvalue for="0" value="StyleAM: Perception-Oriented Unsupervised Domain Adaption for&#10;  Non-reference Image Quality Assessment" />
-          <attvalue for="1" value="  Deep neural networks (DNNs) have shown great potential in non-reference image&#10;quality assessment (NR-IQA). However, the annotation of NR-IQA is&#10;labor-intensive and time-consuming, which severely limits their application&#10;especially for authentic images. To relieve the dependence on quality&#10;annotation, some works have applied unsupervised domain adaptation (UDA) to&#10;NR-IQA. However, the above methods ignore that the alignment space used in&#10;classification is sub-optimal, since the space is not elaborately designed for&#10;perception. To solve this challenge, we propose an effective&#10;perception-oriented unsupervised domain adaptation method StyleAM for NR-IQA,&#10;which transfers sufficient knowledge from label-rich source domain data to&#10;label-free target domain images via Style Alignment and Mixup. Specifically, we&#10;find a more compact and reliable space i.e., feature style space for&#10;perception-oriented UDA based on an interesting/amazing observation, that the&#10;feature style (i.e., the mean and variance) of the deep layer in DNNs is&#10;exactly associated with the quality score in NR-IQA. Therefore, we propose to&#10;align the source and target domains in a more perceptual-oriented space i.e.,&#10;the feature style space, to reduce the intervention from other&#10;quality-irrelevant feature factors. Furthermore, to increase the consistency&#10;between quality score and its feature style, we also propose a novel feature&#10;augmentation strategy Style Mixup, which mixes the feature styles (i.e., the&#10;mean and variance) before the last layer of DNNs together with mixing their&#10;labels. Extensive experimental results on two typical cross-domain settings&#10;(i.e., synthetic to authentic, and multiple distortions to one distortion) have&#10;demonstrated the effectiveness of our proposed StyleAM on NR-IQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2307.16813" label="2307.16813">
-        <attvalues>
-          <attvalue for="0" value="Capturing Co-existing Distortions in User-Generated Content for&#10;  No-reference Video Quality Assessment" />
-          <attvalue for="1" value="  Video Quality Assessment (VQA), which aims to predict the perceptual quality&#10;of a video, has attracted raising attention with the rapid development of&#10;streaming media technology, such as Facebook, TikTok, Kwai, and so on. Compared&#10;with other sequence-based visual tasks (\textit{e.g.,} action recognition), VQA&#10;faces two under-estimated challenges unresolved in User Generated Content (UGC)&#10;videos. \textit{First}, it is not rare that several frames containing serious&#10;distortions (\textit{e.g.,}blocking, blurriness), can determine the perceptual&#10;quality of the whole video, while other sequence-based tasks require more&#10;frames of equal importance for representations. \textit{Second}, the perceptual&#10;quality of a video exhibits a multi-distortion distribution, due to the&#10;differences in the duration and probability of occurrence for various&#10;distortions. In order to solve the above challenges, we propose \textit{Visual&#10;Quality Transformer (VQT)} to extract quality-related sparse features more&#10;efficiently. Methodologically, a Sparse Temporal Attention (STA) is proposed to&#10;sample keyframes by analyzing the temporal correlation between frames, which&#10;reduces the computational complexity from $O(T^2)$ to $O(T \log T)$.&#10;Structurally, a Multi-Pathway Temporal Network (MPTN) utilizes multiple STA&#10;modules with different degrees of sparsity in parallel, capturing co-existing&#10;distortions in a video. Experimentally, VQT demonstrates superior performance&#10;than many \textit{state-of-the-art} methods in three public no-reference VQA&#10;datasets. Furthermore, VQT shows better performance in four full-reference VQA&#10;datasets against widely-adopted industrial algorithms (\textit{i.e.,} VMAF and&#10;AVQT).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.06440" label="2304.06440">
-        <attvalues>
-          <attvalue for="0" value="Zoom-VQA: Patches, Frames and Clips Integration for Video Quality&#10;  Assessment" />
-          <attvalue for="1" value="  Video quality assessment (VQA) aims to simulate the human perception of video&#10;quality, which is influenced by factors ranging from low-level color and&#10;texture details to high-level semantic content. To effectively model these&#10;complicated quality-related factors, in this paper, we decompose video into&#10;three levels (\ie, patch level, frame level, and clip level), and propose a&#10;novel Zoom-VQA architecture to perceive spatio-temporal features at different&#10;levels. It integrates three components: patch attention module, frame pyramid&#10;alignment, and clip ensemble strategy, respectively for capturing&#10;region-of-interest in the spatial dimension, multi-level information at&#10;different feature levels, and distortions distributed over the temporal&#10;dimension. Owing to the comprehensive design, Zoom-VQA obtains state-of-the-art&#10;results on four VQA benchmarks and achieves 2nd place in the NTIRE 2023 VQA&#10;challenge. Notably, Zoom-VQA has outperformed the previous best results on two&#10;subsets of LSVQ, achieving 0.8860 (+1.0%) and 0.7985 (+1.9%) of SRCC on the&#10;respective subsets. Adequate ablation studies further verify the effectiveness&#10;of each component. Codes and models are released in&#10;https://github.com/k-zha14/Zoom-VQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.08124" label="2207.08124">
-        <attvalues>
-          <attvalue for="0" value="Source-free Unsupervised Domain Adaptation for Blind Image Quality&#10;  Assessment" />
-          <attvalue for="1" value="  Existing learning-based methods for blind image quality assessment (BIQA) are&#10;heavily dependent on large amounts of annotated training data, and usually&#10;suffer from a severe performance degradation when encountering the&#10;domain/distribution shift problem. Thanks to the development of unsupervised&#10;domain adaptation (UDA), some works attempt to transfer the knowledge from a&#10;label-sufficient source domain to a label-free target domain under domain shift&#10;with UDA. However, it requires the coexistence of source and target data, which&#10;might be impractical for source data due to the privacy or storage issues. In&#10;this paper, we take the first step towards the source-free unsupervised domain&#10;adaptation (SFUDA) in a simple yet efficient manner for BIQA to tackle the&#10;domain shift without access to the source data. Specifically, we cast the&#10;quality assessment task as a rating distribution prediction problem. Based on&#10;the intrinsic properties of BIQA, we present a group of well-designed&#10;self-supervised objectives to guide the adaptation of the BN affine parameters&#10;towards the target domain. Among them, minimizing the prediction entropy and&#10;maximizing the batch prediction diversity aim to encourage more confident&#10;results while avoiding the trivial solution. Besides, based on the observation&#10;that the IQA rating distribution of single image follows the Gaussian&#10;distribution, we apply Gaussian regularization to the predicted rating&#10;distribution to make it more consistent with the nature of human scoring.&#10;Extensive experimental results under cross-domain scenarios demonstrated the&#10;effectiveness of our proposed method to mitigate the domain shift.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00375" label="1908.00375">
-        <attvalues>
-          <attvalue for="0" value="Quality Assessment of In-the-Wild Videos" />
-          <attvalue for="1" value="  Quality assessment of in-the-wild videos is a challenging problem because of&#10;the absence of reference videos and shooting distortions. Knowledge of the&#10;human visual system can help establish methods for objective quality assessment&#10;of in-the-wild videos. In this work, we show two eminent effects of the human&#10;visual system, namely, content-dependency and temporal-memory effects, could be&#10;used for this purpose. We propose an objective no-reference video quality&#10;assessment method by integrating both effects into a deep neural network. For&#10;content-dependency, we extract features from a pre-trained image classification&#10;neural network for its inherent content-aware property. For temporal-memory&#10;effects, long-term dependencies, especially the temporal hysteresis, are&#10;integrated into the network with a gated recurrent unit and a&#10;subjectively-inspired temporal pooling layer. To validate the performance of&#10;our method, experiments are conducted on three publicly available in-the-wild&#10;video quality assessment databases: KoNViD-1k, CVD2014, and LIVE-Qualcomm,&#10;respectively. Experimental results demonstrate that our proposed method&#10;outperforms five state-of-the-art methods by a large margin, specifically,&#10;12.39%, 15.71%, 15.45%, and 18.09% overall performance improvements over the&#10;second-best method VBLIINDS, in terms of SROCC, KROCC, PLCC and RMSE,&#10;respectively. Moreover, the ablation study verifies the crucial role of both&#10;the content-aware features and the modeling of temporal-memory effects. The&#10;PyTorch implementation of our method is released at&#10;https://github.com/lidq92/VSFA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.13230" label="2106.13230">
-        <attvalues>
-          <attvalue for="0" value="Video Swin Transformer" />
-          <attvalue for="1" value="  The vision community is witnessing a modeling shift from CNNs to&#10;Transformers, where pure Transformer architectures have attained top accuracy&#10;on the major video recognition benchmarks. These video models are all built on&#10;Transformer layers that globally connect patches across the spatial and&#10;temporal dimensions. In this paper, we instead advocate an inductive bias of&#10;locality in video Transformers, which leads to a better speed-accuracy&#10;trade-off compared to previous approaches which compute self-attention globally&#10;even with spatial-temporal factorization. The locality of the proposed video&#10;architecture is realized by adapting the Swin Transformer designed for the&#10;image domain, while continuing to leverage the power of pre-trained image&#10;models. Our approach achieves state-of-the-art accuracy on a broad range of&#10;video recognition benchmarks, including on action recognition (84.9 top-1&#10;accuracy on Kinetics-400 and 86.1 top-1 accuracy on Kinetics-600 with ~20x less&#10;pre-training data and ~3x smaller model size) and temporal modeling (69.6 top-1&#10;accuracy on Something-Something v2). The code and models will be made publicly&#10;available at https://github.com/SwinTransformer/Video-Swin-Transformer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2205.04264" label="2205.04264">
-        <attvalues>
-          <attvalue for="0" value="SwinIQA: Learned Swin Distance for Compressed Image Quality Assessment" />
-          <attvalue for="1" value="  Image compression has raised widespread interest recently due to its&#10;significant importance for multimedia storage and transmission. Meanwhile, a&#10;reliable image quality assessment (IQA) for compressed images can not only help&#10;to verify the performance of various compression algorithms but also help to&#10;guide the compression optimization in turn. In this paper, we design a&#10;full-reference image quality assessment metric SwinIQA to measure the&#10;perceptual quality of compressed images in a learned Swin distance space. It is&#10;known that the compression artifacts are usually non-uniformly distributed with&#10;diverse distortion types and degrees. To warp the compressed images into the&#10;shared representation space while maintaining the complex distortion&#10;information, we extract the hierarchical feature representations from each&#10;stage of the Swin Transformer. Besides, we utilize cross attention operation to&#10;map the extracted feature representations into a learned Swin distance space.&#10;Experimental results show that the proposed metric achieves higher consistency&#10;with human's perceptual judgment compared with both traditional methods and&#10;learning-based methods on CLIC datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.06177" label="2207.06177">
-        <attvalues>
-          <attvalue for="0" value="RTN: Reinforced Transformer Network for Coronary CT Angiography&#10;  Vessel-level Image Quality Assessment" />
-          <attvalue for="1" value="  Coronary CT Angiography (CCTA) is susceptible to various distortions (e.g.,&#10;artifacts and noise), which severely compromise the exact diagnosis of&#10;cardiovascular diseases. The appropriate CCTA Vessel-level Image Quality&#10;Assessment (CCTA VIQA) algorithm can be used to reduce the risk of error&#10;diagnosis. The primary challenges of CCTA VIQA are that the local part of&#10;coronary that determines final quality is hard to locate. To tackle the&#10;challenge, we formulate CCTA VIQA as a multiple-instance learning (MIL)&#10;problem, and exploit Transformer-based MIL backbone (termed as T-MIL) to&#10;aggregate the multiple instances along the coronary centerline into the final&#10;quality. However, not all instances are informative for final quality. There&#10;are some quality-irrelevant/negative instances intervening the exact quality&#10;assessment(e.g., instances covering only background or the coronary in&#10;instances is not identifiable). Therefore, we propose a Progressive&#10;Reinforcement learning based Instance Discarding module (termed as PRID) to&#10;progressively remove quality-irrelevant/negative instances for CCTA VIQA. Based&#10;on the above two modules, we propose a Reinforced Transformer Network (RTN) for&#10;automatic CCTA VIQA based on end-to-end optimization. Extensive experimental&#10;results demonstrate that our proposed method achieves the state-of-the-art&#10;performance on the real-world CCTA dataset, exceeding previous MIL methods by a&#10;large margin.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.14047" label="2204.14047">
-        <attvalues>
-          <attvalue for="0" value="A Deep Learning based No-reference Quality Assessment Model for UGC&#10;  Videos" />
-          <attvalue for="1" value="  Quality assessment for User Generated Content (UGC) videos plays an important&#10;role in ensuring the viewing experience of end-users. Previous UGC video&#10;quality assessment (VQA) studies either use the image recognition model or the&#10;image quality assessment (IQA) models to extract frame-level features of UGC&#10;videos for quality regression, which are regarded as the sub-optimal solutions&#10;because of the domain shifts between these tasks and the UGC VQA task. In this&#10;paper, we propose a very simple but effective UGC VQA model, which tries to&#10;address this problem by training an end-to-end spatial feature extraction&#10;network to directly learn the quality-aware spatial feature representation from&#10;raw pixels of the video frames. We also extract the motion features to measure&#10;the temporal-related distortions that the spatial features cannot model. The&#10;proposed model utilizes very sparse frames to extract spatial features and&#10;dense frames (i.e. the video chunk) with a very low spatial resolution to&#10;extract motion features, which thereby has low computational complexity. With&#10;the better quality-aware features, we only use the simple multilayer perception&#10;layer (MLP) network to regress them into the chunk-level quality scores, and&#10;then the temporal average pooling strategy is adopted to obtain the video-level&#10;quality score. We further introduce a multi-scale quality fusion strategy to&#10;solve the problem of VQA across different spatial resolutions, where the&#10;multi-scale weights are obtained from the contrast sensitivity function of the&#10;human visual system. The experimental results show that the proposed model&#10;achieves the best performance on five popular UGC VQA databases, which&#10;demonstrates the effectiveness of the proposed model. The code will be publicly&#10;available.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.12726" label="2305.12726">
-        <attvalues>
-          <attvalue for="0" value="Towards Explainable In-the-Wild Video Quality Assessment: A Database and&#10;  a Language-Prompted Approach" />
-          <attvalue for="1" value="  The proliferation of in-the-wild videos has greatly expanded the Video&#10;Quality Assessment (VQA) problem. Unlike early definitions that usually focus&#10;on limited distortion types, VQA on in-the-wild videos is especially&#10;challenging as it could be affected by complicated factors, including various&#10;distortions and diverse contents. Though subjective studies have collected&#10;overall quality scores for these videos, how the abstract quality scores relate&#10;with specific factors is still obscure, hindering VQA methods from more&#10;concrete quality evaluations (e.g. sharpness of a video). To solve this&#10;problem, we collect over two million opinions on 4,543 in-the-wild videos on 13&#10;dimensions of quality-related factors, including in-capture authentic&#10;distortions (e.g. motion blur, noise, flicker), errors introduced by&#10;compression and transmission, and higher-level experiences on semantic contents&#10;and aesthetic issues (e.g. composition, camera trajectory), to establish the&#10;multi-dimensional Maxwell database. Specifically, we ask the subjects to label&#10;among a positive, a negative, and a neutral choice for each dimension. These&#10;explanation-level opinions allow us to measure the relationships between&#10;specific quality factors and abstract subjective quality ratings, and to&#10;benchmark different categories of VQA algorithms on each dimension, so as to&#10;more comprehensively analyze their strengths and weaknesses. Furthermore, we&#10;propose the MaxVQA, a language-prompted VQA approach that modifies&#10;vision-language foundation model CLIP to better capture important quality&#10;issues as observed in our analyses. The MaxVQA can jointly evaluate various&#10;specific quality factors and final quality scores with state-of-the-art&#10;accuracy on all dimensions, and superb generalization ability on existing&#10;datasets. Code and data available at https://github.com/VQAssessment/MaxVQA.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2312.17090" label="2312.17090">
-        <attvalues>
-          <attvalue for="0" value="Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined&#10;  Levels" />
-          <attvalue for="1" value="  The explosion of visual content available online underscores the requirement&#10;for an accurate machine assessor to robustly evaluate scores across diverse&#10;types of visual contents. While recent studies have demonstrated the&#10;exceptional potentials of large multi-modality models (LMMs) on a wide range of&#10;related fields, in this work, we explore how to teach them for visual rating&#10;aligned with human opinions. Observing that human raters only learn and judge&#10;discrete text-defined levels in subjective studies, we propose to emulate this&#10;subjective process and teach LMMs with text-defined rating levels instead of&#10;scores. The proposed Q-Align achieves state-of-the-art performance on image&#10;quality assessment (IQA), image aesthetic assessment (IAA), as well as video&#10;quality assessment (VQA) tasks under the original LMM structure. With the&#10;syllabus, we further unify the three tasks into one model, termed the OneAlign.&#10;In our experiments, we demonstrate the advantage of the discrete-level-based&#10;syllabus over direct-score-based variants for LMMs. Our code and the&#10;pre-trained weights are released at https://github.com/Q-Future/Q-Align.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.06457" label="1904.06457">
-        <attvalues>
-          <attvalue for="0" value="YouTube UGC Dataset for Video Compression Research" />
-          <attvalue for="1" value="  Non-professional video, commonly known as User Generated Content (UGC) has&#10;become very popular in today's video sharing applications. However, traditional&#10;metrics used in compression and quality assessment, like BD-Rate and PSNR, are&#10;designed for pristine originals. Thus, their accuracy drops significantly when&#10;being applied on non-pristine originals (the majority of UGC). Understanding&#10;difficulties for compression and quality assessment in the scenario of UGC is&#10;important, but there are few public UGC datasets available for research. This&#10;paper introduces a large scale UGC dataset (1500 20 sec video clips) sampled&#10;from millions of YouTube videos. The dataset covers popular categories like&#10;Gaming, Sports, and new features like High Dynamic Range (HDR). Besides a novel&#10;sampling method based on features extracted from encoding, challenges for UGC&#10;compression and quality evaluation are also discussed. Shortcomings of&#10;traditional reference-based metrics on UGC are addressed. We demonstrate a&#10;promising way to evaluate UGC quality by no-reference objective quality&#10;metrics, and evaluate the current dataset with three no-reference metrics&#10;(Noise, Banding, and SLEEQ).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2401.14838" label="2401.14838">
-        <attvalues>
-          <attvalue for="0" value="Multi-modality action recognition based on dual feature shift in vehicle&#10;  cabin monitoring" />
-          <attvalue for="1" value="  Driver Action Recognition (DAR) is crucial in vehicle cabin monitoring&#10;systems. In real-world applications, it is common for vehicle cabins to be&#10;equipped with cameras featuring different modalities. However, multi-modality&#10;fusion strategies for the DAR task within car cabins have rarely been studied.&#10;In this paper, we propose a novel yet efficient multi-modality driver action&#10;recognition method based on dual feature shift, named DFS. DFS first integrates&#10;complementary features across modalities by performing modality feature&#10;interaction. Meanwhile, DFS achieves the neighbour feature propagation within&#10;single modalities, by feature shifting among temporal frames. To learn common&#10;patterns and improve model efficiency, DFS shares feature extracting stages&#10;among multiple modalities. Extensive experiments have been carried out to&#10;verify the effectiveness of the proposed DFS model on the Drive\&amp;Act dataset.&#10;The results demonstrate that DFS achieves good performance and improves the&#10;efficiency of multi-modality driver action recognition.&#10;" />
-          <attvalue for="2" value="&#10;The Driver Action Recognition (DAR) task involves automatically identifying drivers' secondary activities within the vehicle cabin during driving \cite{DBLP:journals/sensors/TrabelsiKDEB22}. DAR is crucial for enhancing driving safety and promoting efficient interactions between humans and vehicles.&#10;Recently, significant progress on the DAR task has been achieved due to the advances in automation technologies and the application of deep learning methods \cite{DBLP:journals/itsm/RongHHLK22}. &#10;Several methods have been proposed for DAR, often building upon general human action recognition models that utilize 3D convolutional neural networks (CNNs) \cite{chen2023skateboardai} and vision transformers \cite{wang2023videomaev2}. &#10;Among them, the temporal shift module (TSM) provides an efficient solution by shifting features from neighbour frames \cite{DBLP:conf/iccv/LinGH19}.&#10;However, existing research primarily concentrates on extracting spatial-temporal features to enhance DAR performance within single-modality input, such as&#10;Infra-red (IR) video frames \cite{Wharton2021CoarseTA}.&#10;&#10;Features from a single-modality input may be insufficient to accurately support long-term action recognition. First, the car cabin environment is complex, with limited available features. As shown in Fig.~\ref{sample}, drivers' actions are performed by the same individual, with only a portion of the body (such as the upper body) visible. They often exhibit highly similar movements of body parts (such as eating and drinking). Second, the lighting conditions within the cabin are unstable, depending on factors such as weather and transportation infrastructure, and can be affected by sub-optimal factors like variations in sunlight. For example, in Fig~\ref{sample}, the RGB and IR frames can have low brightness or be exposed to weather-related conditions. Given these challenges, incorporating features from multiple modalities becomes crucial for effectively addressing the complex and demanding DAR task.&#10;&#10;In real-world scenarios, it is common for vehicle cabins to be equipped with cameras that offer different modalities (such as RGB, IR, and depth) and views (such as front and top-right). Consequently, the DAR task is inherently multi-modal, with each data modality potentially providing valuable information. Therefore, investigating effective ways to utilize multimodal inputs and extract temporal features is essential for enhancing DAR in car cabin monitoring.&#10;&#10;Several methods conducted multi-modality DAR by utilizing score or late fusion strategies.&#10;Khan et al. utilized average late fusion to detect driving behaviours with depth and IR modalities \cite{DBLP:conf/crv/KhanSSPA22}.&#10;Ma et al. employed a multi-scale channel attention module for the score fusion of depth and IR inputs \cite{DBLP:journals/corr/abs-2210-09441}. &#10;Alina et al. compared several late fusion strategies for DAR task \cite{DBLP:conf/ivs/RoitbergPMSSS22}.&#10;Jiang et al. built a multi-camera DAR model by training single-camera feature extractors \cite{10226521}.&#10;Current methods typically train separate encoders for each modality, which leads to inefficiency in computational complexity. Additionally, these approaches do not adequately consider the temporal correlations among frames.&#10;&#10;In this paper, we propose a novel and efficient multi-modality driver action recognition model based on dual feature shift, named DFS. DFS first integrates complementary features across modalities by performing feature interaction along the modality dimension. Then, DFS shifts the features along the temporal dimension within a single modality, thereby facilitating feature propagation between frames. By utilizing both modality and temporal shift operations, DFS can improve inter- and intra-modality feature interactions without additional computational costs. To further learn common patterns and improve model efficiency, DFS shares certain feature encoders among modalities in the framework. To verify the effectiveness of DFS, extensive experiments have been conducted on the Drive\&amp;Act dataset.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Driver Action Recognition, Mathematics, Artificial Intelligence, Human-Computer Interaction, Computer Science, Computer Vision Systems, Multi-Modality Fusion" />
-        </attvalues>
-      </node>
-      <node id="2101.02082" label="2101.02082">
-        <attvalues>
-          <attvalue for="0" value="Artificial Intelligence Methods in In-Cabin Use Cases: A Survey" />
-          <attvalue for="1" value="  As interest in autonomous driving increases, efforts are being made to meet&#10;requirements for the high-level automation of vehicles. In this context, the&#10;functionality inside the vehicle cabin plays a key role in ensuring a safe and&#10;pleasant journey for driver and passenger alike. At the same time, recent&#10;advances in the field of artificial intelligence (AI) have enabled a whole&#10;range of new applications and assistance systems to solve automated problems in&#10;the vehicle cabin. This paper presents a thorough survey on existing work that&#10;utilizes AI methods for use-cases inside the driving cabin, focusing, in&#10;particular, on application scenarios related to (1) driving safety and (2)&#10;driving comfort. Results from the surveyed works show that AI technology has a&#10;promising future in tackling in-cabin tasks within the autonomous driving&#10;aspect.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.16727" label="2303.16727">
-        <attvalues>
-          <attvalue for="0" value="VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking" />
-          <attvalue for="1" value="  Scale is the primary factor for building a powerful foundation model that&#10;could well generalize to a variety of downstream tasks. However, it is still&#10;challenging to train video foundation models with billions of parameters. This&#10;paper shows that video masked autoencoder (VideoMAE) is a scalable and general&#10;self-supervised pre-trainer for building video foundation models. We scale the&#10;VideoMAE in both model and data with a core design. Specifically, we present a&#10;dual masking strategy for efficient pre-training, with an encoder operating on&#10;a subset of video tokens and a decoder processing another subset of video&#10;tokens. Although VideoMAE is very efficient due to high masking ratio in&#10;encoder, masking decoder can still further reduce the overall computational&#10;cost. This enables the efficient pre-training of billion-level models in video.&#10;We also use a progressive training paradigm that involves an initial&#10;pre-training on a diverse multi-sourced unlabeled dataset, followed by a&#10;post-pre-training on a mixed labeled dataset. Finally, we successfully train a&#10;video ViT model with a billion parameters, which achieves a new&#10;state-of-the-art performance on the datasets of Kinetics (90.0% on K400 and&#10;89.9% on K600) and Something-Something (68.7% on V1 and 77.0% on V2). In&#10;addition, we extensively verify the pre-trained video ViT models on a variety&#10;of downstream tasks, demonstrating its effectiveness as a general video&#10;representation learner. The code and model is available at&#10;\url{https://github.com/OpenGVLab/VideoMAEv2}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.08383" label="1811.08383">
-        <attvalues>
-          <attvalue for="0" value="TSM: Temporal Shift Module for Efficient Video Understanding" />
-          <attvalue for="1" value="  The explosive growth in video streaming gives rise to challenges on&#10;performing video understanding at high accuracy and low computation cost.&#10;Conventional 2D CNNs are computationally cheap but cannot capture temporal&#10;relationships; 3D CNN based methods can achieve good performance but are&#10;computationally intensive, making it expensive to deploy. In this paper, we&#10;propose a generic and effective Temporal Shift Module (TSM) that enjoys both&#10;high efficiency and high performance. Specifically, it can achieve the&#10;performance of 3D CNN but maintain 2D CNN's complexity. TSM shifts part of the&#10;channels along the temporal dimension; thus facilitate information exchanged&#10;among neighboring frames. It can be inserted into 2D CNNs to achieve temporal&#10;modeling at zero computation and zero parameters. We also extended TSM to&#10;online setting, which enables real-time low-latency online video recognition&#10;and video object detection. TSM is accurate and efficient: it ranks the first&#10;place on the Something-Something leaderboard upon publication; on Jetson Nano&#10;and Galaxy Note8, it achieves a low latency of 13ms and 35ms for online video&#10;recognition. The code is available at:&#10;https://github.com/mit-han-lab/temporal-shift-module.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2109.04021" label="2109.04021">
-        <attvalues>
-          <attvalue for="0" value="Supervised Contrastive Learning for Detecting Anomalous Driving&#10;  Behaviours from Multimodal Videos" />
-          <attvalue for="1" value="  Distracted driving is one of the major reasons for vehicle accidents.&#10;Therefore, detecting distracted driving behaviors is of paramount importance to&#10;reduce the millions of deaths and injuries occurring worldwide. Distracted or&#10;anomalous driving behaviors are deviations from 'normal' driving that need to&#10;be identified correctly to alert the driver. However, these driving behaviors&#10;do not comprise one specific type of driving style and their distribution can&#10;be different during the training and test phases of a classifier. We formulate&#10;this problem as a supervised contrastive learning approach to learn a visual&#10;representation to detect normal, and seen and unseen anomalous driving&#10;behaviors. We made a change to the standard contrastive loss function to adjust&#10;the similarity of negative pairs to aid the optimization. Normally, in a (self)&#10;supervised contrastive framework, the projection head layers are omitted during&#10;the test phase as the encoding layers are considered to contain general visual&#10;representative information. However, we assert that for a video-based&#10;supervised contrastive learning task, including a projection head can be&#10;beneficial. We showed our results on a driver anomaly detection dataset that&#10;contains 783 minutes of video recordings of normal and anomalous driving&#10;behaviors of 31 drivers from the various top and front cameras (both depth and&#10;infrared). Out of 9 video modalities combinations, our proposed contrastive&#10;approach improved the ROC AUC on 6 in comparison to the baseline models (from&#10;4.23% to 8.91% for different modalities). We performed statistical tests that&#10;showed evidence that our proposed method performs better than the baseline&#10;contrastive learning setup. Finally, the results showed that the fusion of&#10;depth and infrared modalities from the top and front views achieved the best&#10;AUC ROC of 0.9738 and AUC PR of 0.9772.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2210.09441" label="2210.09441">
-        <attvalues>
-          <attvalue for="0" value="Real-Time Driver Monitoring Systems through Modality and View Analysis" />
-          <attvalue for="1" value="  Driver distractions are known to be the dominant cause of road accidents.&#10;While monitoring systems can detect non-driving-related activities and&#10;facilitate reducing the risks, they must be accurate and efficient to be&#10;applicable. Unfortunately, state-of-the-art methods prioritize accuracy while&#10;ignoring latency because they leverage cross-view and multimodal videos in&#10;which consecutive frames are highly similar. Thus, in this paper, we pursue&#10;time-effective detection models by neglecting the temporal relation between&#10;video frames and investigate the importance of each sensing modality in&#10;detecting drives' activities. Experiments demonstrate that 1) our proposed&#10;algorithms are real-time and can achieve similar performances (97.5\% AUC-PR)&#10;with significantly reduced computation compared with video-based models; 2) the&#10;top view with the infrared channel is more informative than any other single&#10;modality. Furthermore, we enhance the DAD dataset by manually annotating its&#10;test set to enable multiclassification. We also thoroughly analyze the&#10;influence of visual sensor types and their placements on the prediction of each&#10;class. The code and the new labels will be released.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.04734" label="2204.04734">
-        <attvalues>
-          <attvalue for="0" value="A Comparative Analysis of Decision-Level Fusion for Multimodal Driver&#10;  Behaviour Understanding" />
-          <attvalue for="1" value="  Visual recognition inside the vehicle cabin leads to safer driving and more&#10;intuitive human-vehicle interaction but such systems face substantial obstacles&#10;as they need to capture different granularities of driver behaviour while&#10;dealing with highly limited body visibility and changing illumination.&#10;Multimodal recognition mitigates a number of such issues: prediction outcomes&#10;of different sensors complement each other due to different modality-specific&#10;strengths and weaknesses. While several late fusion methods have been&#10;considered in previously published frameworks, they constantly feature&#10;different architecture backbones and building blocks making it very hard to&#10;isolate the role of the chosen late fusion strategy itself. This paper presents&#10;an empirical evaluation of different paradigms for decision-level late fusion&#10;in video-based driver observation. We compare seven different mechanisms for&#10;joining the results of single-modal classifiers which have been both popular,&#10;(e.g. score averaging) and not yet considered (e.g. rank-level fusion) in the&#10;context of driver observation evaluating them based on different criteria and&#10;benchmark settings. This is the first systematic study of strategies for fusing&#10;outcomes of multimodal predictors inside the vehicles, conducted with the goal&#10;to provide guidance for fusion scheme selection.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2206.12912" label="2206.12912">
-        <attvalues>
-          <attvalue for="0" value="Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022&#10;  OmniCV Workshop Challenge" />
-          <attvalue for="1" value="  Object detection is a comprehensively studied problem in autonomous driving.&#10;However, it has been relatively less explored in the case of fisheye cameras.&#10;The strong radial distortion breaks the translation invariance inductive bias&#10;of Convolutional Neural Networks. Thus, we present the WoodScape fisheye object&#10;detection challenge for autonomous driving which was held as part of the CVPR&#10;2022 Workshop on Omnidirectional Computer Vision (OmniCV). This is one of the&#10;first competitions focused on fisheye camera object detection. We encouraged&#10;the participants to design models which work natively on fisheye images without&#10;rectification. We used CodaLab to host the competition based on the publicly&#10;available WoodScape fisheye dataset. In this paper, we provide a detailed&#10;analysis on the competition which attracted the participation of 120 global&#10;teams and a total of 1492 submissions. We briefly discuss the details of the&#10;winning methods and analyze their qualitative and quantitative results.&#10;" />
-          <attvalue for="2" value="&#10;Autonomous Driving is a challenging problem and it requires multiple sensors handling different aspects and robust sensor fusion algorithms which combine the senor information effectively \cite{yadav2020cnn, mohapatra2021bevdetnet, dasgupta2022spatio}. &#10;Surround-view systems employ four sensors to create a network with large overlapping zones to cover the car's near-field area \cite{eising2021near, kumar2022surround}. For near-field sensing, wide-angle images reaching $180^\degree$ are utilized. Any perception algorithm must consider the substantial fisheye distortion that such camera systems produce. Because most computer vision research relies on narrow field-of-view cameras with modest radial distortion, this is a substantial challenge. However, because camera systems are now more commonly used, development in this field has been attained. \ref{fig:svs} illustrates the typical automotive surround-view camera system comprising of four fisheye cameras covering the entire $360^\circ$ around the vehicle. &#10;Most commercial cars have fisheye cameras as a primary sensor for automated parking. Rear-view fisheye cameras have become a typical addition in low-cost vehicles for dashboard viewing and reverse parking. Despite its abundance, there are just a few public databases for fisheye images, so relatively little research is conducted. One such dataset is the Oxford RobotCar \cite{maddern20171} a large-scale dataset focusing on the long-term autonomy of autonomous vehicles. The key responsibilities of this dataset, which enables research into continuous learning for autonomous cars and mobile robotics, are localization and mapping. It includes approximately 100 repetitions of a continuous route around Oxford, UK, collected over a year and commonly used for long-term localization and mapping.\par&#10;&#10;WoodScape \cite{yogamani2019_woodscape} is a large dataset for $360^\degree$ sensing around an ego vehicle with four fisheye cameras. It is designed to complement existing automobile datasets with limited FOV images and encourage further research in multi-task multi-camera computer vision algorithms for self-driving vehicles. It is built based on industrialization needs addressing the diversity challenges \cite{uricar2019challenges}. The dataset sensor configuration consists of four surround-view fisheye cameras sampled randomly. The dataset comprises labels for geometry and segmentation tasks, including semantic segmentation, distance estimation, generalized bounding boxes, motion segmentation, and a novel lens soiling detection task (shown in \ref{fig:woodscape-tasks}). &#10;&#10;Instead of naive rectification, the WoodScape pushes researchers to create solutions that can work directly on raw fisheye images, modeling the underlying distortion. WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}.&#10;SynwoodScape \cite{sekkat2022synwoodscape} is a synthetic version of the Woodscape dataset.&#10;&#10;This paper discusses the second edition of WoodScape dataset challenge focused on the object detection task. The results of the first edition of the WoodScape challenge are discussed in \cite{ramachandran2021woodscape}. It is organized as part of the CVPR 2022 OmniCV workshop [\href{ Section \ref{sec:challenge} discusses the challenge setup including metrics and conditions. Section \ref{sec:outcome} discusses the participation information and the details of the winning solutions. Finally, Section \ref{sec:conc} provides concluding remarks. \par&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Object Detection, Autonomous Driving, Computer Science, Robotics" />
-        </attvalues>
-      </node>
-      <node id="2103.17001" label="2103.17001">
-        <attvalues>
-          <attvalue for="0" value="Near-field Perception for Low-Speed Vehicle Automation using&#10;  Surround-view Fisheye Cameras" />
-          <attvalue for="1" value="  Cameras are the primary sensor in automated driving systems. They provide&#10;high information density and are optimal for detecting road infrastructure cues&#10;laid out for human vision. Surround-view camera systems typically comprise of&#10;four fisheye cameras with 190{\deg}+ field of view covering the entire&#10;360{\deg} around the vehicle focused on near-field sensing. They are the&#10;principal sensors for low-speed, high accuracy, and close-range sensing&#10;applications, such as automated parking, traffic jam assistance, and low-speed&#10;emergency braking. In this work, we provide a detailed survey of such vision&#10;systems, setting up the survey in the context of an architecture that can be&#10;decomposed into four modular components namely Recognition, Reconstruction,&#10;Relocalization, and Reorganization. We jointly call this the 4R Architecture.&#10;We discuss how each component accomplishes a specific aspect and provide a&#10;positional argument that they can be synergized to form a complete perception&#10;system for low-speed automation. We support this argument by presenting results&#10;from previous works and by presenting architecture proposals for such a system.&#10;Qualitative results are presented in the video at https://youtu.be/ae8bCOF77uY.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.01489" label="1905.01489">
-        <attvalues>
-          <attvalue for="0" value="WoodScape: A multi-task, multi-camera fisheye dataset for autonomous&#10;  driving" />
-          <attvalue for="1" value="  Fisheye cameras are commonly employed for obtaining a large field of view in&#10;surveillance, augmented reality and in particular automotive applications. In&#10;spite of their prevalence, there are few public datasets for detailed&#10;evaluation of computer vision algorithms on fisheye images. We release the&#10;first extensive fisheye automotive dataset, WoodScape, named after Robert Wood&#10;who invented the fisheye camera in 1906. WoodScape comprises of four surround&#10;view cameras and nine tasks including segmentation, depth estimation, 3D&#10;bounding box detection and soiling detection. Semantic annotation of 40 classes&#10;at the instance level is provided for over 10,000 images and annotation for&#10;other tasks are provided for over 100,000 images. With WoodScape, we would like&#10;to encourage the community to adapt computer vision models for fisheye camera&#10;instead of using naive rectification.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1901.09270" label="1901.09270">
-        <attvalues>
-          <attvalue for="0" value="Challenges in Designing Datasets and Validation for Autonomous Driving" />
-          <attvalue for="1" value="  Autonomous driving is getting a lot of attention in the last decade and will&#10;be the hot topic at least until the first successful certification of a car&#10;with Level 5 autonomy. There are many public datasets in the academic&#10;community. However, they are far away from what a robust industrial production&#10;system needs. There is a large gap between academic and industrial setting and&#10;a substantial way from a research prototype, built on public datasets, to a&#10;deployable solution which is a challenging task. In this paper, we focus on bad&#10;practices that often happen in the autonomous driving from an industrial&#10;deployment perspective. Data design deserves at least the same amount of&#10;attention as the model design. There is very little attention paid to these&#10;issues in the scientific community, and we hope this paper encourages better&#10;formalization of dataset design. More specifically, we focus on the datasets&#10;design and validation scheme for autonomous driving, where we would like to&#10;highlight the common problems, wrong assumptions, and steps towards avoiding&#10;them, as well as some open problems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2105.12763" label="2105.12763">
-        <attvalues>
-          <attvalue for="0" value="An Online Learning System for Wireless Charging Alignment using&#10;  Surround-view Fisheye Cameras" />
-          <attvalue for="1" value="  Electric Vehicles are increasingly common, with inductive chargepads being&#10;considered a convenient and efficient means of charging electric vehicles.&#10;However, drivers are typically poor at aligning the vehicle to the necessary&#10;accuracy for efficient inductive charging, making the automated alignment of&#10;the two charging plates desirable. In parallel to the electrification of the&#10;vehicular fleet, automated parking systems that make use of surround-view&#10;camera systems are becoming increasingly popular. In this work, we propose a&#10;system based on the surround-view camera architecture to detect, localize, and&#10;automatically align the vehicle with the inductive chargepad. The visual design&#10;of the chargepads is not standardized and not necessarily known beforehand.&#10;Therefore, a system that relies on offline training will fail in some&#10;situations. Thus, we propose a self-supervised online learning method that&#10;leverages the driver's actions when manually aligning the vehicle with the&#10;chargepad and combine it with weak supervision from semantic segmentation and&#10;depth to learn a classifier to auto-annotate the chargepad in the video for&#10;further training. In this way, when faced with a previously unseen chargepad,&#10;the driver needs only manually align the vehicle a single time. As the&#10;chargepad is flat on the ground, it is not easy to detect it from a distance.&#10;Thus, we propose using a Visual SLAM pipeline to learn landmarks relative to&#10;the chargepad to enable alignment from a greater range. We demonstrate the&#10;working system on an automated vehicle as illustrated in the video at&#10;https://youtu.be/_cLCmkW4UYo. To encourage further research, we will share a&#10;chargepad dataset used in this work.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.02249" label="1912.02249">
-        <attvalues>
-          <attvalue for="0" value="Let's Get Dirty: GAN Based Data Augmentation for Camera Lens Soiling&#10;  Detection in Autonomous Driving" />
-          <attvalue for="1" value="  Wide-angle fisheye cameras are commonly used in automated driving for parking&#10;and low-speed navigation tasks. Four of such cameras form a surround-view&#10;system that provides a complete and detailed view of the vehicle. These cameras&#10;are directly exposed to harsh environmental settings and can get soiled very&#10;easily by mud, dust, water, frost. Soiling on the camera lens can severely&#10;degrade the visual perception algorithms, and a camera cleaning system&#10;triggered by a soiling detection algorithm is increasingly being deployed.&#10;While adverse weather conditions, such as rain, are getting attention recently,&#10;there is only limited work on general soiling. The main reason is the&#10;difficulty in collecting a diverse dataset as it is a relatively rare event. We&#10;propose a novel GAN based algorithm for generating unseen patterns of soiled&#10;images. Additionally, the proposed method automatically provides the&#10;corresponding soiling masks eliminating the manual annotation cost.&#10;Augmentation of the generated soiled images for training improves the accuracy&#10;of soiling detection tasks significantly by 18% demonstrating its usefulness.&#10;The manually annotated soiling dataset and the generated augmentation dataset&#10;will be made public. We demonstrate the generalization of our fisheye trained&#10;GAN model on the Cityscapes dataset. We provide an empirical evaluation of the&#10;degradation of the semantic segmentation algorithm with the soiled data.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.14042" label="2104.14042">
-        <attvalues>
-          <attvalue for="0" value="Weather and Light Level Classification for Autonomous Driving: Dataset,&#10;  Baseline and Active Learning" />
-          <attvalue for="1" value="  Autonomous driving is rapidly advancing, and Level 2 functions are becoming a&#10;standard feature. One of the foremost outstanding hurdles is to obtain robust&#10;visual perception in harsh weather and low light conditions where accuracy&#10;degradation is severe. It is critical to have a weather classification model to&#10;decrease visual perception confidence during these scenarios. Thus, we have&#10;built a new dataset for weather (fog, rain, and snow) classification and light&#10;level (bright, moderate, and low) classification. Furthermore, we provide&#10;street type (asphalt, grass, and cobblestone) classification, leading to 9&#10;labels. Each image has three labels corresponding to weather, light level, and&#10;street type. We recorded the data utilizing an industrial front camera of RCCC&#10;(red/clear) format with a resolution of $1024\times1084$. We collected 15k&#10;video sequences and sampled 60k images. We implement an active learning&#10;framework to reduce the dataset's redundancy and find the optimal set of frames&#10;for training a model. We distilled the 60k images further to 1.1k images, which&#10;will be shared publicly after privacy anonymization. There is no public dataset&#10;for weather and light level classification focused on autonomous driving to the&#10;best of our knowledge. The baseline ResNet18 network used for weather&#10;classification achieves state-of-the-art results in two non-automotive weather&#10;classification public datasets but significantly lower accuracy on our proposed&#10;dataset, demonstrating it is not saturated and needs further research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.06192" label="1803.06192">
-        <attvalues>
-          <attvalue for="0" value="Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision" />
-          <attvalue for="1" value="  Near field depth estimation around a self driving car is an important&#10;function that can be achieved by four wide angle fisheye cameras having a field&#10;of view of over 180. Depth estimation based on convolutional neural networks&#10;(CNNs) produce state of the art results, but progress is hindered because depth&#10;annotation cannot be obtained manually. Synthetic datasets are commonly used&#10;but they have limitations. For instance, they do not capture the extensive&#10;variability in the appearance of objects like vehicles present in real&#10;datasets. There is also a domain shift while performing inference on natural&#10;images illustrated by many attempts to handle the domain adaptation explicitly.&#10;In this work, we explore an alternate approach of training using sparse LiDAR&#10;data as ground truth for depth estimation for fisheye camera. We built our own&#10;dataset using our self driving car setup which has a 64 beam Velodyne LiDAR and&#10;four wide angle fisheye cameras. To handle the difference in view points of&#10;LiDAR and fisheye camera, an occlusion resolution mechanism was implemented. We&#10;started with Eigen's multiscale convolutional network architecture and improved&#10;by modifying activation function and optimizer. We obtained promising results&#10;on our dataset with RMSE errors comparable to the state of the art results&#10;obtained on KITTI.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2001.02223" label="2001.02223">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Task Weighting Methods for Multi-task Networks in Autonomous&#10;  Driving Systems" />
-          <attvalue for="1" value="  Deep multi-task networks are of particular interest for autonomous driving&#10;systems. They can potentially strike an excellent trade-off between predictive&#10;performance, hardware constraints and efficient use of information from&#10;multiple types of annotations and modalities. However, training such models is&#10;non-trivial and requires balancing learning over all tasks as their respective&#10;losses display different scales, ranges and dynamics across training. Multiple&#10;task weighting methods that adjust the losses in an adaptive way have been&#10;proposed recently on different datasets and combinations of tasks, making it&#10;difficult to compare them. In this work, we review and systematically evaluate&#10;nine task weighting strategies on common grounds on three automotive datasets&#10;(KITTI, Cityscapes and WoodScape). We then propose a novel method combining&#10;evolutionary meta-learning and task-based selective backpropagation, for&#10;computing task weights leading to reliable network training. Our method&#10;outperforms state-of-the-art methods by a significant margin on a two-task&#10;application.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2203.05056" label="2203.05056">
-        <attvalues>
-          <attvalue for="0" value="SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for&#10;  Autonomous Driving" />
-          <attvalue for="1" value="  Surround-view cameras are a primary sensor for automated driving, used for&#10;near-field perception. It is one of the most commonly used sensors in&#10;commercial vehicles primarily used for parking visualization and automated&#10;parking. Four fisheye cameras with a 190{\deg} field of view cover the&#10;360{\deg} around the vehicle. Due to its high radial distortion, the standard&#10;algorithms do not extend easily. Previously, we released the first public&#10;fisheye surround-view dataset named WoodScape. In this work, we release a&#10;synthetic version of the surround-view dataset, covering many of its weaknesses&#10;and extending it. Firstly, it is not possible to obtain ground truth for&#10;pixel-wise optical flow and depth. Secondly, WoodScape did not have all four&#10;cameras annotated simultaneously in order to sample diverse frames. However,&#10;this means that multi-camera algorithms cannot be designed to obtain a unified&#10;output in birds-eye space, which is enabled in the new dataset. We implemented&#10;surround-view fisheye geometric projections in CARLA Simulator matching&#10;WoodScape's configuration and created SynWoodScape. We release 80k images from&#10;the synthetic dataset with annotations for 10+ tasks. We also release the&#10;baseline code and supporting scripts.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2305.14207" label="2305.14207">
-        <attvalues>
-          <attvalue for="0" value="SAD: Segment Any RGBD" />
-          <attvalue for="1" value="  The Segment Anything Model (SAM) has demonstrated its effectiveness in&#10;segmenting any part of 2D RGB images. However, SAM exhibits a stronger emphasis&#10;on texture information while paying less attention to geometry information when&#10;segmenting RGB images. To address this limitation, we propose the Segment Any&#10;RGBD (SAD) model, which is specifically designed to extract geometry&#10;information directly from images. Inspired by the natural ability of humans to&#10;identify objects through the visualization of depth maps, SAD utilizes SAM to&#10;segment the rendered depth map, thus providing cues with enhanced geometry&#10;information and mitigating the issue of over-segmentation. We further include&#10;the open-vocabulary semantic segmentation in our framework, so that the 3D&#10;panoptic segmentation is fulfilled. The project is available on&#10;https://github.com/Jun-CEN/SegmentAnyRGBD.&#10;" />
-          <attvalue for="2" value="&#10;Recently, a prominent model for image segmentation called the Segment Anything model (SAM) has been introduced~\cite{kirillov2023segment}. SAM serves as a robust foundation model for effectively segmenting 2D images in various scenarios.&#10;However, during the segmentation of 2D RGB images, it has been observed that SAM primarily relies on texture information, such as color, leading to over-segmentation results. Consequently, the challenge lies in finding a way to obtain segmentation results that incorporate more geometric information through the utilization of SAM.&#10;&#10;To address this issue, we draw inspiration from the remarkable ability of humans to identify objects by visualizing depth maps.&#10;We first map a depth map ($\mathbb{R}^{H\times W}$) to the RGB space ($\mathbb{R}^{H\times W\times 3}$) by a colormap function and then feed the rendered depth image into SAM. Compared to RGB images, the rendered depth image ignores the texture information and focuses on the geometry information, as shown in Fig.~\ref{fig:example}. Notably, it is worth mentioning that while previous SAM-based projects~\cite{zhang2023comprehensive} like SSA~\cite{SSA}, Anything-3D~\cite{shen2023anything3d}, and SAM 3D~\cite{pointcept2023} primarily employ RGB images as inputs, we are the first to utilize SAM for directly segmenting rendered depth images.&#10; " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Semantic Segmentation, Computer Vision, Engineering, Machine Learning, Geometry Extraction, Mathematics, Artificial Intelligence, Computer Science, Image Segmentation" />
-        </attvalues>
-      </node>
-      <node id="2305.08196" label="2305.08196">
-        <attvalues>
-          <attvalue for="0" value="A Comprehensive Survey on Segment Anything Model for Vision and Beyond" />
-          <attvalue for="1" value="  Artificial intelligence (AI) is evolving towards artificial general&#10;intelligence, which refers to the ability of an AI system to perform a wide&#10;range of tasks and exhibit a level of intelligence similar to that of a human&#10;being. This is in contrast to narrow or specialized AI, which is designed to&#10;perform specific tasks with a high degree of efficiency. Therefore, it is&#10;urgent to design a general class of models, which we term foundation models,&#10;trained on broad data that can be adapted to various downstream tasks. The&#10;recently proposed segment anything model (SAM) has made significant progress in&#10;breaking the boundaries of segmentation, greatly promoting the development of&#10;foundation models for computer vision. To fully comprehend SAM, we conduct a&#10;survey study. As the first to comprehensively review the progress of segmenting&#10;anything task for vision and beyond based on the foundation model of SAM, this&#10;work focuses on its applications to various tasks and data types by discussing&#10;its historical development, recent progress, and profound impact on broad&#10;applications. We first introduce the background and terminology for foundation&#10;models including SAM, as well as state-of-the-art methods contemporaneous with&#10;SAM that are significant for segmenting anything task. Then, we analyze and&#10;summarize the advantages and limitations of SAM across various image processing&#10;applications, including software scenes, real-world scenes, and complex scenes.&#10;Importantly, many insights are drawn to guide future research to develop more&#10;versatile foundation models and improve the architecture of SAM. We also&#10;summarize massive other amazing applications of SAM in vision and beyond.&#10;Finally, we maintain a continuously updated paper list and an open-source&#10;project summary for foundation model SAM at&#10;\href{https://github.com/liliu-avril/Awesome-Segment-Anything}{\color{magenta}{here}}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2304.10261" label="2304.10261">
-        <attvalues>
-          <attvalue for="0" value="Anything-3D: Towards Single-view Anything Reconstruction in the Wild" />
-          <attvalue for="1" value="  3D reconstruction from a single-RGB image in unconstrained real-world&#10;scenarios presents numerous challenges due to the inherent diversity and&#10;complexity of objects and environments. In this paper, we introduce&#10;Anything-3D, a methodical framework that ingeniously combines a series of&#10;visual-language models and the Segment-Anything object segmentation model to&#10;elevate objects to 3D, yielding a reliable and versatile system for single-view&#10;conditioned 3D reconstruction task. Our approach employs a BLIP model to&#10;generate textural descriptions, utilizes the Segment-Anything model for the&#10;effective extraction of objects of interest, and leverages a text-to-image&#10;diffusion model to lift object into a neural radiance field. Demonstrating its&#10;ability to produce accurate and detailed 3D reconstructions for a wide array of&#10;objects, \emph{Anything-3D\footnotemark[2]} shows promise in addressing the&#10;limitations of existing methodologies. Through comprehensive experiments and&#10;evaluations on various datasets, we showcase the merits of our approach,&#10;underscoring its potential to contribute meaningfully to the field of 3D&#10;reconstruction. Demos and code will be available at&#10;\href{https://github.com/Anything-of-anything/Anything-3D}{https://github.com/Anything-of-anything/Anything-3D}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2309.05438" label="2309.05438">
-        <attvalues>
-          <attvalue for="0" value="Towards Content-based Pixel Retrieval in Revisited Oxford and Paris" />
-          <attvalue for="1" value="  This paper introduces the first two pixel retrieval benchmarks. Pixel&#10;retrieval is segmented instance retrieval. Like semantic segmentation extends&#10;classification to the pixel level, pixel retrieval is an extension of image&#10;retrieval and offers information about which pixels are related to the query&#10;object. In addition to retrieving images for the given query, it helps users&#10;quickly identify the query object in true positive images and exclude false&#10;positive images by denoting the correlated pixels. Our user study results show&#10;pixel-level annotation can significantly improve the user experience.&#10;  Compared with semantic and instance segmentation, pixel retrieval requires a&#10;fine-grained recognition capability for variable-granularity targets. To this&#10;end, we propose pixel retrieval benchmarks named PROxford and PRParis, which&#10;are based on the widely used image retrieval datasets, ROxford and RParis.&#10;Three professional annotators label 5,942 images with two rounds of&#10;double-checking and refinement. Furthermore, we conduct extensive experiments&#10;and analysis on the SOTA methods in image search, image matching, detection,&#10;segmentation, and dense matching using our pixel retrieval benchmarks. Results&#10;show that the pixel retrieval task is challenging to these approaches and&#10;distinctive from existing problems, suggesting that further research can&#10;advance the content-based pixel-retrieval and thus user search experience. The&#10;datasets can be downloaded from&#10;\href{https://github.com/anguoyuan/Pixel_retrieval-Segmented_instance_retrieval}{this&#10;link}.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Image retrieval is a long-standing and fundamental computer vision task and has achieved remarkable advances. However, because the retrieved ranking list contains false positive images and the true positive images contain complex co-occurring backgrounds, users may be difficult to identify the query object from the ranking list. In this paper, we execute a user study and show that providing pixel-level annotations can help users better understand the retrieved results. Therefore, this paper introduces the pixel retrieval task and its first benchmarks. Pixel retrieval is defined as searching pixels that depict the query object from the database. More specifically, it requires the machine to recognize, localize, and segment the query object in database images in run time, as shown in Figure~\ref{fig:user_study}.&#10;&#10;Similar to semantic segmentation, which works as an extension of classification and provides pixel-level category information to the machines, pixel retrieval is an extension of image retrieval. However, pixel retrieval differs from existing semantic segmentation~\cite{he2017mask,zhou2017scene,liu2018path} in two aspects: the fine-grained particular instance recognition and the variable-granularity recognition. &#10;&#10;On the one hand, pixel retrieval asks the machine to consider the fine-grained information to segment the same instance with the query, e.g., segment the particular query building in the street figures that contain many similar buildings. This is different from existing semantic segmentation~\cite{he2017mask} and instance segmentation~\cite{liu2018path,zhou2017scene}. Semantic segmentation only requires the category level information, e.g., to segment all the buildings in the street figures. On top of semantic segmentation, instance segmentation additionally requires demarcating individual instances, e.g., segmenting all the buildings and giving the boundary of each building separately. However, instance segmentation does not distinguish the differences among the buildings~\cite{zhou2017scene,liu2018path,bolya2019yolact}.&#10;&#10;On the other hand, pixel retrieval requires adjusting the recognition granularity as needed. The query image can be the whole building or only a part of the building. The search engine should understand the intention of the query and adjust the segmentation granularity in demand. This differs from existing segmentation benchmarks~\cite{zhou2017scene,cordts2016cityscapes,lin2014microsoft,everingham2010pascal,geiger2012we}, where the recognition granularity is fixed in advance. Therefore, the pixel retrieval task is supplementary to semantic and instance segmentation by considering the recognition and segmentation featured with fine-grained and variable-granularity properties, which are also fundamental visual abilities of humans. &#10;&#10;\Skip{&#10;Pixel retrieval is useful in many practical applications. Modern image retrieval techniques focus on improving&#10;the image-level ranking performance of the hard cases~\cite{cao2020unifying,an2021hypergraph,radenovic2018revisiting}. However, because of the difficulty of the hard case, users may not be conscious that it is truly positive in a short time, even though it is at the top of the ranking list.&#10;\WJ{Again, the connection between trends of recent ranking techniques and user experience is unclear.&#10;Can we explain how pixel retrieval can improve user experience? &#10;e.g., By highlighting the retrieved pixels on the searched images, we can guide the users to locate regions of interest more easily in such cases.}&#10;Our user study results show pixel-level annotation can significantly improve the user experience (Section~\ref{sec:Applications}).&#10;&#10;Besides the web search, improving the pixel retrieval performance helps the image level ranking techniques. Pixel retrieval can also be useful in medical diagnosis~\cite{yang2014parallel}, geographical information systems~\cite{zhu2021vigor}, and image matting~\cite{wei2021improved,xu2017deep,zhang2019late} (Section ~\ref{sec:Applications}). }&#10;&#10;In order to promote the study of pixel retrieval, we create the pixel retrieval benchmarks Pixel-Revisited-Oxford (PROxford) and Pixel-Revisited-Paris (PRParis) on top of the famous image retrieval benchmarks Revisited-Oxford (ROxford) and Revisited-Paris (RParis)~\cite{philbin2007object, philbin2008lost,radenovic2018revisiting}. There are three reasons to use ROxford and RParis as our base benchmarks. Firstly, they are notoriously difficult and can better reflect the search engines' performance. Secondly, each query in these datasets has up to hundreds of positive images, so they are suitable for evaluating the fine-grained recognition ability. Thirdly, every positive image is guaranteed to be identifiable by people without considering any contextual visual information~\cite{radenovic2018revisiting}. &#10;&#10;We provide the segmentation labels to a total of 5,942 images in ROxford and RParis. To ensure the label quality, three professional annotators independently label the query-index pairs and then refine and check the labels. The annotators are aged between 26 to 32 years old and have worked full-time on annotation for over two years. We then design new metrics, mAP@50:5:95, and mAP, to evaluate the pixel retrieval performance (Section~\ref{sec:Dataset}).&#10;&#10;We provide an extensive comparison of State-Of-The-Art (SOTA) methods in related fields, including image search, detection, segmentation, and dense matching with our benchmarks. We have some interesting findings from the experiment. For example, we find the SOTA spatial verification methods~\cite{cao2020unifying,Noh_2017_ICCV} give a high inlier number to some true query-index pairs but match the wrong regions. We find the dense and pixel-level approaches~\cite{min2021hypercorrelation, truong2021warp} helpful for the pixel retrieval task. Most importantly, our results show that pixel retrieval is difficult and further research is needed for advancing the user experience on the content-based search task. &#10;&#10;Our contributions are as follows: &#10;\begin{itemize}&#10; \item We introduced the pixel retrieval task and provided the first two landmark pixel retrieval benchmarks, PROxford and PRParis. Three professional annotators labeled, refined, and checked the labels.&#10; \item We executed the user study and showed that the pixel level annotation could significantly improve user experience.&#10; \item We performed extensive experiments with SOTA methods in image search, detection, segmentation, and dense matching. Our experiment results can be used as the baselines for future study.&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Information Science, Machine Learning, Computer Vision, Human-Computer Interaction, Cognitive Science, Computer Science, Image Retrieval, Pixel Segmentation" />
-        </attvalues>
-      </node>
-      <node id="1904.02689" label="1904.02689">
-        <attvalues>
-          <attvalue for="0" value="YOLACT: Real-time Instance Segmentation" />
-          <attvalue for="1" value="  We present a simple, fully-convolutional model for real-time instance&#10;segmentation that achieves 29.8 mAP on MS COCO at 33.5 fps evaluated on a&#10;single Titan Xp, which is significantly faster than any previous competitive&#10;approach. Moreover, we obtain this result after training on only one GPU. We&#10;accomplish this by breaking instance segmentation into two parallel subtasks:&#10;(1) generating a set of prototype masks and (2) predicting per-instance mask&#10;coefficients. Then we produce instance masks by linearly combining the&#10;prototypes with the mask coefficients. We find that because this process&#10;doesn't depend on repooling, this approach produces very high-quality masks and&#10;exhibits temporal stability for free. Furthermore, we analyze the emergent&#10;behavior of our prototypes and show they learn to localize instances on their&#10;own in a translation variant manner, despite being fully-convolutional.&#10;Finally, we also propose Fast NMS, a drop-in 12 ms faster replacement for&#10;standard NMS that only has a marginal performance penalty.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1803.11285" label="1803.11285">
-        <attvalues>
-          <attvalue for="0" value="Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking" />
-          <attvalue for="1" value="  In this paper we address issues with image retrieval benchmarking on standard&#10;and popular Oxford 5k and Paris 6k datasets. In particular, annotation errors,&#10;the size of the dataset, and the level of challenge are addressed: new&#10;annotation for both datasets is created with an extra attention to the&#10;reliability of the ground truth. Three new protocols of varying difficulty are&#10;introduced. The protocols allow fair comparison between different methods,&#10;including those using a dataset pre-processing stage. For each dataset, 15 new&#10;challenging queries are introduced. Finally, a new set of 1M hard,&#10;semi-automatically cleaned distractors is selected.&#10;  An extensive comparison of the state-of-the-art methods is performed on the&#10;new benchmark. Different types of methods are evaluated, ranging from&#10;local-feature-based to modern CNN based methods. The best results are achieved&#10;by taking the best of the two worlds. Most importantly, image retrieval appears&#10;far from being solved.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2011.12172" label="2011.12172">
-        <attvalues>
-          <attvalue for="0" value="VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval" />
-          <attvalue for="1" value="  Cross-view image geo-localization aims to determine the locations of&#10;street-view query images by matching with GPS-tagged reference images from&#10;aerial view. Recent works have achieved surprisingly high retrieval accuracy on&#10;city-scale datasets. However, these results rely on the assumption that there&#10;exists a reference image exactly centered at the location of any query image,&#10;which is not applicable for practical scenarios. In this paper, we redefine&#10;this problem with a more realistic assumption that the query image can be&#10;arbitrary in the area of interest and the reference images are captured before&#10;the queries emerge. This assumption breaks the one-to-one retrieval setting of&#10;existing datasets as the queries and reference images are not perfectly aligned&#10;pairs, and there may be multiple reference images covering one query location.&#10;To bridge the gap between this realistic setting and existing datasets, we&#10;propose a new large-scale benchmark -- VIGOR -- for cross-View Image&#10;Geo-localization beyond One-to-one Retrieval. We benchmark existing&#10;state-of-the-art methods and propose a novel end-to-end framework to localize&#10;the query in a coarse-to-fine manner. Apart from the image-level retrieval&#10;accuracy, we also evaluate the localization accuracy in terms of the actual&#10;distance (meters) using the raw GPS data. Extensive experiments are conducted&#10;under different application scenarios to validate the effectiveness of the&#10;proposed method. The results indicate that cross-view geo-localization in this&#10;realistic setting is still challenging, fostering new research in this&#10;direction. Our dataset and code will be released at&#10;\url{https://github.com/Jeff-Zilence/VIGOR}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2012.08323" label="2012.08323">
-        <attvalues>
-          <attvalue for="0" value="Improved Image Matting via Real-time User Clicks and Uncertainty&#10;  Estimation" />
-          <attvalue for="1" value="  Image matting is a fundamental and challenging problem in computer vision and&#10;graphics. Most existing matting methods leverage a user-supplied trimap as an&#10;auxiliary input to produce good alpha matte. However, obtaining high-quality&#10;trimap itself is arduous, thus restricting the application of these methods.&#10;Recently, some trimap-free methods have emerged, however, the matting quality&#10;is still far behind the trimap-based methods. The main reason is that, without&#10;the trimap guidance in some cases, the target network is ambiguous about which&#10;is the foreground target. In fact, choosing the foreground is a subjective&#10;procedure and depends on the user's intention. To this end, this paper proposes&#10;an improved deep image matting framework which is trimap-free and only needs&#10;several user click interactions to eliminate the ambiguity. Moreover, we&#10;introduce a new uncertainty estimation module that can predict which parts need&#10;polishing and a following local refinement module. Based on the computation&#10;budget, users can choose how many local parts to improve with the uncertainty&#10;guidance. Quantitative and qualitative results show that our method performs&#10;better than existing trimap-free methods and comparably to state-of-the-art&#10;trimap-based methods with minimal user effort.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1703.03872" label="1703.03872">
-        <attvalues>
-          <attvalue for="0" value="Deep Image Matting" />
-          <attvalue for="1" value="  Image matting is a fundamental computer vision problem and has many&#10;applications. Previous algorithms have poor performance when an image has&#10;similar foreground and background colors or complicated textures. The main&#10;reasons are prior methods 1) only use low-level features and 2) lack high-level&#10;context. In this paper, we propose a novel deep learning based algorithm that&#10;can tackle both these problems. Our deep model has two parts. The first part is&#10;a deep convolutional encoder-decoder network that takes an image and the&#10;corresponding trimap as inputs and predict the alpha matte of the image. The&#10;second part is a small convolutional network that refines the alpha matte&#10;predictions of the first network to have more accurate alpha values and sharper&#10;edges. In addition, we also create a large-scale image matting dataset&#10;including 49300 training images and 1000 testing images. We evaluate our&#10;algorithm on the image matting benchmark, our testing set, and a wide variety&#10;of real images. Experimental results clearly demonstrate the superiority of our&#10;algorithm over previous methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2303.06907" label="2303.06907">
-        <attvalues>
-          <attvalue for="0" value="ST360IQ: No-Reference Omnidirectional Image Quality Assessment with&#10;  Spherical Vision Transformers" />
-          <attvalue for="1" value="  Omnidirectional images, aka 360 images, can deliver immersive and interactive&#10;visual experiences. As their popularity has increased dramatically in recent&#10;years, evaluating the quality of 360 images has become a problem of interest&#10;since it provides insights for capturing, transmitting, and consuming this new&#10;media. However, directly adapting quality assessment methods proposed for&#10;standard natural images for omnidirectional data poses certain challenges.&#10;These models need to deal with very high-resolution data and implicit&#10;distortions due to the spherical form of the images. In this study, we present&#10;a method for no-reference 360 image quality assessment. Our proposed ST360IQ&#10;model extracts tangent viewports from the salient parts of the input&#10;omnidirectional image and employs a vision-transformers based module processing&#10;saliency selective patches/tokens that estimates a quality score from each&#10;viewport. Then, it aggregates these scores to give a final quality score. Our&#10;experiments on two benchmark datasets, namely OIQA and CVIQ datasets,&#10;demonstrate that as compared to the state-of-the-art, our approach predicts the&#10;quality of an omnidirectional image correlated with the human-perceived image&#10;quality. The code has been available on&#10;https://github.com/Nafiseh-Tofighi/ST360IQ&#10;" />
-          <attvalue for="2" value="&#10;&#10;Virtual reality (VR) data has recently occupied an increasing share of multimedia data as the corresponding hardware, including Head Mounted Displays (HMD) and capture devices, has become more widely accessible to the end-users.&#10;Therefore, developing methods for evaluating the quality of omnidirectional images has also gained much attention from the researchers working in this area.&#10;&#10;Image quality assessment (IQA) has been thoroughly studied in the past twenty years~\cite{DBLP:journals/corr/abs-2004-07728}. Generally speaking, IQA algorithms can be classified into full-reference IQA (FR IQA), reduced-reference IQA (RR IQA), and no-reference IQA (NR IQA)~\cite{Akhtar2017}. FR IQA and RR IQA models need full and part reference image information, respectively, while NR IQA takes only the distorted image as input. In that sense, NR IQA measures subjective, perceptual quality of images.&#10;&#10;With the recent advances in deep learning, deep methods started to dominate the IQA literature. While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}. In general, the quality assessment literature for 360$^\circ$ images follows the IQA literature as most of the existing work either suggest to directly use the same IQA methods proposed for natural images, or make small modification to those methods to deal with the spherical form~\cite{xu2020state}. Only very recently, CNN-based models specific to omnidirection images haves been proposed~\cite{8702664,xu2020blind,9506044,9766332,9859468}.&#10;&#10;In our work, we propose a ViT-based framework for omnidirectional image quality assessment called ST360IQ. Up to our knowledge, there are no prior work that explores ViTs for evaluating the perceptual quality of 360$^\circ$ images. Our framework estimates the overall image quality through a number of tangent viewports focusing on different parts of the spherical image. In particular, motivated by the relation between visual salience and image quality~\cite{zhang2014vsi}, we sample these viewports from the salient parts of the omnidirectional data. We process each viewport independently through a spherical ViT-based module, and extract individual quality scores, which are then aggregated to obtain the final quality score. Using tangent viewports instead of the commonly used projections such as equirectangular projection (ERP) or cube-mapping allows us to deal with distorted visual data due to the spherical form more effectively. Moreover, our ViT-based formulation specifically designed for the omnidirectional data domain results in more accurate predictions than the state-of-the-art. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision Techniques, Machine Learning, Engineering, Computer Vision, Information Technology, Image Quality Assessment, Omnidirectional Image Processing, Human-Computer Interaction, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2004.07728" label="2004.07728">
-        <attvalues>
-          <attvalue for="0" value="Image Quality Assessment: Unifying Structure and Texture Similarity" />
-          <attvalue for="1" value="  Objective measures of image quality generally operate by comparing pixels of&#10;a &quot;degraded&quot; image to those of the original. Relative to human observers, these&#10;measures are overly sensitive to resampling of texture regions (e.g., replacing&#10;one patch of grass with another). Here, we develop the first full-reference&#10;image quality model with explicit tolerance to texture resampling. Using a&#10;convolutional neural network, we construct an injective and differentiable&#10;function that transforms images to multi-scale overcomplete representations. We&#10;demonstrate empirically that the spatial averages of the feature maps in this&#10;representation capture texture appearance, in that they provide a set of&#10;sufficient statistical constraints to synthesize a wide variety of texture&#10;patterns. We then describe an image quality method that combines correlations&#10;of these spatial averages (&quot;texture similarity&quot;) with correlations of the&#10;feature maps (&quot;structure similarity&quot;). The parameters of the proposed measure&#10;are jointly optimized to match human ratings of image quality, while minimizing&#10;the reported distances between subimages cropped from the same texture images.&#10;Experiments show that the optimized method explains human perceptual scores,&#10;both on conventional image quality databases, as well as on texture databases.&#10;The measure also offers competitive performance on related tasks such as&#10;texture classification and retrieval. Finally, we show that our method is&#10;relatively insensitive to geometric transformations (e.g., translation and&#10;dilation), without use of any specialized training or data augmentation. Code&#10;is available at https://github.com/dingkeyan93/DISTS.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.01097" label="2101.01097">
-        <attvalues>
-          <attvalue for="0" value="Transformer for Image Quality Assessment" />
-          <attvalue for="1" value="  Transformer has become the new standard method in natural language processing&#10;(NLP), and it also attracts research interests in computer vision area. In this&#10;paper we investigate the application of Transformer in Image Quality (TRIQ)&#10;assessment. Following the original Transformer encoder employed in Vision&#10;Transformer (ViT), we propose an architecture of using a shallow Transformer&#10;encoder on the top of a feature map extracted by convolution neural networks&#10;(CNN). Adaptive positional embedding is employed in the Transformer encoder to&#10;handle images with arbitrary resolutions. Different settings of Transformer&#10;architectures have been investigated on publicly available image quality&#10;databases. We have found that the proposed TRIQ architecture achieves&#10;outstanding performance. The implementation of TRIQ is published on Github&#10;(https://github.com/junyongyou/triq).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.09140" label="2002.09140">
-        <attvalues>
-          <attvalue for="0" value="Blind Omnidirectional Image Quality Assessment with Viewport Oriented&#10;  Graph Convolutional Networks" />
-          <attvalue for="1" value="  Quality assessment of omnidirectional images has become increasingly urgent&#10;due to the rapid growth of virtual reality applications. Different from&#10;traditional 2D images and videos, omnidirectional contents can provide&#10;consumers with freely changeable viewports and a larger field of view covering&#10;the $360^{\circ}\times180^{\circ}$ spherical surface, which makes the objective&#10;quality assessment of omnidirectional images more challenging. In this paper,&#10;motivated by the characteristics of the human vision system (HVS) and the&#10;viewing process of omnidirectional contents, we propose a novel Viewport&#10;oriented Graph Convolution Network (VGCN) for blind omnidirectional image&#10;quality assessment (IQA). Generally, observers tend to give the subjective&#10;rating of a 360-degree image after passing and aggregating different viewports&#10;information when browsing the spherical scenery. Therefore, in order to model&#10;the mutual dependency of viewports in the omnidirectional image, we build a&#10;spatial viewport graph. Specifically, the graph nodes are first defined with&#10;selected viewports with higher probabilities to be seen, which is inspired by&#10;the HVS that human beings are more sensitive to structural information. Then,&#10;these nodes are connected by spatial relations to capture interactions among&#10;them. Finally, reasoning on the proposed graph is performed via graph&#10;convolutional networks. Moreover, we simultaneously obtain global quality using&#10;the entire omnidirectional image without viewport sampling to boost the&#10;performance according to the viewing experience. Experimental results&#10;demonstrate that our proposed model outperforms state-of-the-art full-reference&#10;and no-reference IQA metrics on two public omnidirectional IQA databases.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2407.12588" label="2407.12588">
-        <attvalues>
-          <attvalue for="0" value="Benchmarking Robust Self-Supervised Learning Across Diverse Downstream&#10;  Tasks" />
-          <attvalue for="1" value="  Large-scale vision models have become integral in many applications due to&#10;their unprecedented performance and versatility across downstream tasks.&#10;However, the robustness of these foundation models has primarily been explored&#10;for a single task, namely image classification. The vulnerability of other&#10;common vision tasks, such as semantic segmentation and depth estimation,&#10;remains largely unknown. We present a comprehensive empirical evaluation of the&#10;adversarial robustness of self-supervised vision encoders across multiple&#10;downstream tasks. Our attacks operate in the encoder embedding space and at the&#10;downstream task output level. In both cases, current state-of-the-art&#10;adversarial fine-tuning techniques tested only for classification significantly&#10;degrade clean and robust performance on other tasks. Since the purpose of a&#10;foundation model is to cater to multiple applications at once, our findings&#10;reveal the need to enhance encoder robustness more broadly. Our code is&#10;available at ${github.com/layer6ai-labs/ssl-robustness}$.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Foundation models trained through self-supervised learning (SSL) have become the backbone of many applications due to their versatility; one foundation model can be adapted to many downstream tasks with a small amount of data and training (or fine-tuning). Foundation models in the vision domain have even outperformed dedicated models on several tasks~\cite{caron2021dino,mae,oquab2024dinov2}.&#10;Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored. Recently, ~\cite{li2023adversarialnotbugs} showed that non-robust features extracted from adversarial examples for supervised models (and useful for classification) become largely useless when transferred to self-supervised learning paradigms. They advocated for a cross-paradigm examination of robustness, yet focused their analysis solely on classification. A major outstanding question is whether adversarial robustness transfers across downstream tasks.&#10;&#10;We present an in-depth empirical evaluation of the adversarial robustness of self-supervised vision encoders~\cite{simsiam,caron2021dino} for downstream tasks beyond classification. We use attacks that operate in the encoder's embedding space (\EmbedAttack) and those that leverage direct access to the downstream task outputs (\PGDAttacks), \eg \PGD for classification~\cite{madry2018towards} or \SegPGD for semantic segmentation~\cite{gu2022segpgd}. Our main observation is that the state-of-the-art adversarial full fine-tuning of encoders~\cite{zhang2022decoupled}: (1) substantially lowers clean performance, (2) increases robustness only against the \EmbedAttack, and (3) remains ineffective in improving robustness against the task-specific \PGDAttacks. We observe only a slight improvement against the \PGDAttacks for classification when the adversarial fine-tuning dataset and downstream dataset come from the same distribution. This indicates a need to rethink what it means for a foundation model to be robust. Finally, we offer potential approaches to bolster the cross-task robustness of SSL encoders. " />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Vision Model Robustness, Artificial Intelligence, Adversarial Attack Methods, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2006.04924" label="2006.04924">
-        <attvalues>
-          <attvalue for="0" value="A Self-supervised Approach for Adversarial Robustness" />
-          <attvalue for="1" value="  Adversarial examples can cause catastrophic mistakes in Deep Neural Network&#10;(DNNs) based vision systems e.g., for classification, segmentation and object&#10;detection. The vulnerability of DNNs against such attacks can prove a major&#10;roadblock towards their real-world deployment. Transferability of adversarial&#10;examples demand generalizable defenses that can provide cross-task protection.&#10;Adversarial training that enhances robustness by modifying target model's&#10;parameters lacks such generalizability. On the other hand, different input&#10;processing based defenses fall short in the face of continuously evolving&#10;attacks. In this paper, we take the first step to combine the benefits of both&#10;approaches and propose a self-supervised adversarial training mechanism in the&#10;input space. By design, our defense is a generalizable approach and provides&#10;significant robustness against the \textbf{unseen} adversarial attacks (\eg by&#10;reducing the success rate of translation-invariant \textbf{ensemble} attack&#10;from 82.6\% to 31.9\% in comparison to previous state-of-the-art). It can be&#10;deployed as a plug-and-play solution to protect a variety of vision systems, as&#10;we demonstrate for the case of classification, segmentation and detection. Code&#10;is available at: {\small\url{https://github.com/Muzammal-Naseer/NRP}}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.13337" label="2010.13337">
-        <attvalues>
-          <attvalue for="0" value="Robust Pre-Training by Adversarial Contrastive Learning" />
-          <attvalue for="1" value="  Recent work has shown that, when integrated with adversarial training,&#10;self-supervised pre-training can lead to state-of-the-art robustness In this&#10;work, we improve robustness-aware self-supervised pre-training by learning&#10;representations that are consistent under both data augmentations and&#10;adversarial perturbations. Our approach leverages a recent contrastive learning&#10;framework, which learns representations by maximizing feature consistency under&#10;differently augmented views. This fits particularly well with the goal of&#10;adversarial robustness, as one cause of adversarial fragility is the lack of&#10;feature invariance, i.e., small input perturbations can result in undesirable&#10;large changes in features or even predicted labels. We explore various options&#10;to formulate the contrastive task, and demonstrate that by injecting&#10;adversarial perturbations, contrastive pre-training can lead to models that are&#10;both label-efficient and robust. We empirically evaluate the proposed&#10;Adversarial Contrastive Learning (ACL) and show it can consistently outperform&#10;existing methods. For example on the CIFAR-10 dataset, ACL outperforms the&#10;previous state-of-the-art unsupervised robust pre-training approach by 2.99% on&#10;robust accuracy and 2.14% on standard accuracy. We further demonstrate that ACL&#10;pre-training can improve semi-supervised adversarial training, even when only a&#10;few labeled examples are available. Our codes and pre-trained models have been&#10;released at: https://github.com/VITA-Group/Adversarial-Contrastive-Learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2111.01124" label="2111.01124">
-        <attvalues>
-          <attvalue for="0" value="When Does Contrastive Learning Preserve Adversarial Robustness from&#10;  Pretraining to Finetuning?" />
-          <attvalue for="1" value="  Contrastive learning (CL) can learn generalizable feature representations and&#10;achieve the state-of-the-art performance of downstream tasks by finetuning a&#10;linear classifier on top of it. However, as adversarial robustness becomes&#10;vital in image classification, it remains unclear whether or not CL is able to&#10;preserve robustness to downstream tasks. The main challenge is that in the&#10;self-supervised pretraining + supervised finetuning paradigm, adversarial&#10;robustness is easily forgotten due to a learning task mismatch from pretraining&#10;to finetuning. We call such a challenge 'cross-task robustness&#10;transferability'. To address the above problem, in this paper we revisit and&#10;advance CL principles through the lens of robustness enhancement. We show that&#10;(1) the design of contrastive views matters: High-frequency components of&#10;images are beneficial to improving model robustness; (2) Augmenting CL with&#10;pseudo-supervision stimulus (e.g., resorting to feature clustering) helps&#10;preserve robustness without forgetting. Equipped with our new designs, we&#10;propose AdvCL, a novel adversarial contrastive pretraining framework. We show&#10;that AdvCL is able to enhance cross-task robustness transferability without&#10;loss of model accuracy and finetuning efficiency. With a thorough experimental&#10;study, we demonstrate that AdvCL outperforms the state-of-the-art&#10;self-supervised robust learning methods across multiple datasets (CIFAR-10,&#10;CIFAR-100, and STL-10) and finetuning schemes (linear evaluation and full model&#10;finetuning).&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.10899" label="2207.10899">
-        <attvalues>
-          <attvalue for="0" value="Decoupled Adversarial Contrastive Learning for Self-supervised&#10;  Adversarial Robustness" />
-          <attvalue for="1" value="  Adversarial training (AT) for robust representation learning and&#10;self-supervised learning (SSL) for unsupervised representation learning are two&#10;active research fields. Integrating AT into SSL, multiple prior works have&#10;accomplished a highly significant yet challenging task: learning robust&#10;representation without labels. A widely used framework is adversarial&#10;contrastive learning which couples AT and SSL, and thus constitute a very&#10;complex optimization problem. Inspired by the divide-and-conquer philosophy, we&#10;conjecture that it might be simplified as well as improved by solving two&#10;sub-problems: non-robust SSL and pseudo-supervised AT. This motivation shifts&#10;the focus of the task from seeking an optimal integrating strategy for a&#10;coupled problem to finding sub-solutions for sub-problems. With this said, this&#10;work discards prior practices of directly introducing AT to SSL frameworks and&#10;proposed a two-stage framework termed Decoupled Adversarial Contrastive&#10;Learning (DeACL). Extensive experimental results demonstrate that our DeACL&#10;achieves SOTA self-supervised adversarial robustness while significantly&#10;reducing the training time, which validates its effectiveness and efficiency.&#10;Moreover, our DeACL constitutes a more explainable solution, and its success&#10;also bridges the gap with semi-supervised AT for exploiting unlabeled samples&#10;for robust representation learning. The code is publicly accessible at&#10;https://github.com/pantheon5100/DeACL.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2310.18936" label="2310.18936">
-        <attvalues>
-          <attvalue for="0" value="Adversarial Examples Are Not Real Features" />
-          <attvalue for="1" value="  The existence of adversarial examples has been a mystery for years and&#10;attracted much interest. A well-known theory by \citet{ilyas2019adversarial}&#10;explains adversarial vulnerability from a data perspective by showing that one&#10;can extract non-robust features from adversarial examples and these features&#10;alone are useful for classification. However, the explanation remains quite&#10;counter-intuitive since non-robust features are mostly noise features to&#10;humans. In this paper, we re-examine the theory from a larger context by&#10;incorporating multiple learning paradigms. Notably, we find that contrary to&#10;their good usefulness under supervised learning, non-robust features attain&#10;poor usefulness when transferred to other self-supervised learning paradigms,&#10;such as contrastive learning, masked image modeling, and diffusion models. It&#10;reveals that non-robust features are not really as useful as robust or natural&#10;features that enjoy good transferability between these paradigms. Meanwhile,&#10;for robustness, we also show that naturally trained encoders from robust&#10;features are largely non-robust under AutoAttack. Our cross-paradigm&#10;examination suggests that the non-robust features are not really useful but&#10;more like paradigm-wise shortcuts, and robust features alone might be&#10;insufficient to attain reliable model robustness. Code is available at&#10;\url{https://github.com/PKU-ML/AdvNotRealFeatures}.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2207.12391" label="2207.12391">
-        <attvalues>
-          <attvalue for="0" value="SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and&#10;  Boosting Segmentation Robustness" />
-          <attvalue for="1" value="  Deep neural network-based image classifications are vulnerable to adversarial&#10;perturbations. The image classifications can be easily fooled by adding&#10;artificial small and imperceptible perturbations to input images. As one of the&#10;most effective defense strategies, adversarial training was proposed to address&#10;the vulnerability of classification models, where the adversarial examples are&#10;created and injected into training data during training. The attack and defense&#10;of classification models have been intensively studied in past years. Semantic&#10;segmentation, as an extension of classifications, has also received great&#10;attention recently. Recent work shows a large number of attack iterations are&#10;required to create effective adversarial examples to fool segmentation models.&#10;The observation makes both robustness evaluation and adversarial training on&#10;segmentation models challenging. In this work, we propose an effective and&#10;efficient segmentation attack method, dubbed SegPGD. Besides, we provide a&#10;convergence analysis to show the proposed SegPGD can create more effective&#10;adversarial examples than PGD under the same number of attack iterations.&#10;Furthermore, we propose to apply our SegPGD as the underlying attack method for&#10;segmentation adversarial training. Since SegPGD can create more effective&#10;adversarial examples, the adversarial training with our SegPGD can boost the&#10;robustness of segmentation models. Our proposals are also verified with&#10;experiments on popular Segmentation model architectures and standard&#10;segmentation datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1912.04950" label="1912.04950">
-        <attvalues>
-          <attvalue for="0" value="HyperCon: Image-To-Video Model Transfer for Video-To-Video Translation&#10;  Tasks" />
-          <attvalue for="1" value="  Video-to-video translation is more difficult than image-to-image translation&#10;due to the temporal consistency problem that, if unaddressed, leads to&#10;distracting flickering effects. Although video models designed from scratch&#10;produce temporally consistent results, training them to match the vast visual&#10;knowledge captured by image models requires an intractable number of videos. To&#10;combine the benefits of image and video models, we propose an image-to-video&#10;model transfer method called Hyperconsistency (HyperCon) that transforms any&#10;well-trained image model into a temporally consistent video model without&#10;fine-tuning. HyperCon works by translating a temporally interpolated video&#10;frame-wise and then aggregating over temporally localized windows on the&#10;interpolated video. It handles both masked and unmasked inputs, enabling&#10;support for even more video-to-video translation tasks than prior&#10;image-to-video model transfer techniques. We demonstrate HyperCon on video&#10;style transfer and inpainting, where it performs favorably compared to prior&#10;state-of-the-art methods without training on a single stylized or incomplete&#10;video. Our project website is available at&#10;https://ryanszeto.com/projects/hypercon .&#10;" />
-          <attvalue for="2" value="&#10;&#10;Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}. But compared to images, videos pose an additional challenge: not only do the video frames need to satisfy the intended translation, they must also be temporally consistent. Otherwise, they will exhibit flickering artifacts.&#10;&#10;Existing video-to-video translation techniques address temporal consistency through one of two types of strategies. The first incorporates temporal consistency directly into the method through optical flow-based losses~\cite{Kim2019,Ruder2016,Sajjadi2018,Gupta2017,Zhang2019} or network layers that operate on the time dimension, \eg, 3D convolutional layers~\cite{Chang2019} or recurrent layers~\cite{Kim2019}. These techniques leverage self-supervised video data and tailor models to specific tasks using relevant losses, \eg, reconstruction error~\cite{Liu2018,Yu2018,Kim2019} or style loss~\cite{Johnson2016,Gupta2017}. However, they require models and losses that are defined exclusively on videos and tuned for a specific application.&#10;&#10;The other type of strategy uses a blind video consistency model to reduce flicker in a frame-wise translated video as a post-processing step~\cite{Bonneel2015,Dong2015,Lai2018,Yao2017}. For example, Lai \etal~\cite{Lai2018} train a recurrent encoder-decoder network to improve the temporal consistency of independently processed frames via warping and structural preservation losses. Blind video consistency methods relax the need for task-specific video models and losses, and also enables image-to-image models to be applied immediately to videos without sacrificing consistency. However, they require dense correspondences between unprocessed frames during optimization, making them unsuitable for tasks in which certain input regions have no meaningful structure, \eg, video inpainting.&#10;&#10;Similarly to blind video consistency methods, we opt to impart image-to-image models with temporal consistency, motivated primarily by generalization issues inherent to video-tailored approaches. To elaborate, consider in Figure~\ref{fig:intro} a failure case from the otherwise impressive state-of-the-art video inpainting network VINet~\cite{Kim2019}; here, it fails to hallucinate a sensible texture for the missing region. Now consider a state-of-the-art image inpainting model, Contextual Attention~\cite{Yu2018}: this method produces realistic textures on the same example (but exhibits temporal inconsistency). The reason for this difference lies in the diversity of data used to train each model. Whereas the video model was trained on about 5,000 examples from one of the largest video segmentation datasets to date~\cite{Xu2018}, the image model was trained on over 1,000,000 images~\cite{Zhou2018}. Regardless of application, the vast scale of image datasets enables image models to encapsulate broader visual knowledge than video models trained from scratch and, as a result, better generalize to new data. Because storage and cost limitations make it intractable to collect high-quality video datasets as diverse as modern image datasets, video-tailored models are doomed to generalize poorly compared to image models.&#10;&#10;To overcome this challenging generalization issue, we propose a method of image-to-video model transfer for video-to-video translation tasks. Rather than optimize an image or video model from scratch, we aim to transform a black-box image-to-image translation model into a strong video-to-video translation model without fine-tuning---specifically, to automatically induce temporal consistency while achieving the same visual effect as the image-to-image model. Compared to prior techniques in blind video consistency~\cite{Bonneel2015,Lai2018}, which target these goals and therefore fall under the same problem space, ours broadens the scope of applicable tasks to include those in which the input and output videos have differing visual structure, such as video inpainting and edge-deforming style transfer (e.g., the mosaic style from PyTorch's examples repository).&#10;&#10;Key to our approach is the view of image-to-image models as noisy models in which small changes in the input lead to substantial changes in the output (\eg, Figure~\ref{fig:intro}, middle). To strengthen the desired signal and filter out noise, we synthesize several perturbed, but related predictions and aggregate over them. We obtain perturbations in a way that conditions on multiple neighboring frames to enhance temporal consistency. Our hyperconsistency approach (HyperCon for short) implements these principles by inserting frames into the video with a frame interpolation network, translating the interpolated video's frames independently, and aggregating within overlapping windows of appropriate stride to obtain a final video whose length matches the original (Figure~\ref{fig:method-overview}).&#10;&#10;As the first method among image-to-video model transfer techniques to reason in interpolated video space, HyperCon forgoes the traditional post-processing paradigm by integrating frame-wise translation within itself as an intermediate step, not as a step that precedes it. Since it does not require dense correspondences in the unprocessed input video, it performs well on video-to-video translation tasks with or without masked inputs---\eg, inpainting and style transfer respectively---as verified by our extensive experiments across these two widely differing applications. HyperCon outperforms a prior state-of-the-art video consistency model~\cite{Lai2018} in terms of reducing flicker and adhering to the intended translation; when combined with a strong image inpainting method, it also produces better predictions than a state-of-the-art video inpainting model~\cite{Kim2019}. It achieves competitive performance in both tasks despite not being trained with any masked or stylized videos.&#10;&#10;Our contributions are as follows. First, we motivate image-to-video model transfer as a way to leverage the superior generalization performance of image models for video-to-video translation without sacrificing temporal consistency. Second, we propose HyperCon, which supports a wider span of tasks than prior video consistency work thanks to its support for both masked and unmasked inputs. Finally, we show that HyperCon performs favorably compared to state-of-the-art video consistency and inpainting methods without the need to be fine-tuned on these tasks. Our project website is available at \href{&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Temporal Consistency, Machine Learning, Engineering, Computer Vision, Video Translation, Mathematics, Artificial Intelligence, Computer Science, Image Model Transfer" />
-        </attvalues>
-      </node>
-      <node id="2008.11882" label="2008.11882">
-        <attvalues>
-          <attvalue for="0" value="Crossing-Domain Generative Adversarial Networks for Unsupervised&#10;  Multi-Domain Image-to-Image Translation" />
-          <attvalue for="1" value="  State-of-the-art techniques in Generative Adversarial Networks (GANs) have&#10;shown remarkable success in image-to-image translation from peer domain X to&#10;domain Y using paired image data. However, obtaining abundant paired data is a&#10;non-trivial and expensive process in the majority of applications. When there&#10;is a need to translate images across n domains, if the training is performed&#10;between every two domains, the complexity of the training will increase&#10;quadratically. Moreover, training with data from two domains only at a time&#10;cannot benefit from data of other domains, which prevents the extraction of&#10;more useful features and hinders the progress of this research area. In this&#10;work, we propose a general framework for unsupervised image-to-image&#10;translation across multiple domains, which can translate images from domain X&#10;to any a domain without requiring direct training between the two domains&#10;involved in image translation. A byproduct of the framework is the reduction of&#10;computing time and computing resources, since it needs less time than training&#10;the domains in pairs as is done in state-of-the-art works. Our proposed&#10;framework consists of a pair of encoders along with a pair of GANs which learns&#10;high-level features across different domains to generate diverse and realistic&#10;samples from. Our framework shows competing results on many image-to-image&#10;tasks compared with state-of-the-art techniques.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.06601" label="1808.06601">
-        <attvalues>
-          <attvalue for="0" value="Video-to-Video Synthesis" />
-          <attvalue for="1" value="  We study the problem of video-to-video synthesis, whose goal is to learn a&#10;mapping function from an input source video (e.g., a sequence of semantic&#10;segmentation masks) to an output photorealistic video that precisely depicts&#10;the content of the source video. While its image counterpart, the&#10;image-to-image synthesis problem, is a popular topic, the video-to-video&#10;synthesis problem is less explored in the literature. Without understanding&#10;temporal dynamics, directly applying existing image synthesis approaches to an&#10;input video often results in temporally incoherent videos of low visual&#10;quality. In this paper, we propose a novel video-to-video synthesis approach&#10;under the generative adversarial learning framework. Through carefully-designed&#10;generator and discriminator architectures, coupled with a spatio-temporal&#10;adversarial objective, we achieve high-resolution, photorealistic, temporally&#10;coherent video results on a diverse set of input formats including segmentation&#10;masks, sketches, and poses. Experiments on multiple benchmarks show the&#10;advantage of our method compared to strong baselines. In particular, our model&#10;is capable of synthesizing 2K resolution videos of street scenes up to 30&#10;seconds long, which significantly advances the state-of-the-art of video&#10;synthesis. Finally, we apply our approach to future video prediction,&#10;outperforming several state-of-the-art competing systems.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.05174" label="1808.05174">
-        <attvalues>
-          <attvalue for="0" value="Recycle-GAN: Unsupervised Video Retargeting" />
-          <attvalue for="1" value="  We introduce a data-driven approach for unsupervised video retargeting that&#10;translates content from one domain to another while preserving the style native&#10;to a domain, i.e., if contents of John Oliver's speech were to be transferred&#10;to Stephen Colbert, then the generated content/speech should be in Stephen&#10;Colbert's style. Our approach combines both spatial and temporal information&#10;along with adversarial losses for content translation and style preservation.&#10;In this work, we first study the advantages of using spatiotemporal constraints&#10;over spatial constraints for effective retargeting. We then demonstrate the&#10;proposed approach for the problems where information in both space and time&#10;matters such as face-to-face translation, flower-to-flower, wind and cloud&#10;synthesis, sunrise and sunset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.07683" label="1908.07683">
-        <attvalues>
-          <attvalue for="0" value="Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video&#10;  Translation" />
-          <attvalue for="1" value="  In this paper, we investigate the problem of unpaired video-to-video&#10;translation. Given a video in the source domain, we aim to learn the&#10;conditional distribution of the corresponding video in the target domain,&#10;without seeing any pairs of corresponding videos. While significant progress&#10;has been made in the unpaired translation of images, directly applying these&#10;methods to an input video leads to low visual quality due to the additional&#10;time dimension. In particular, previous methods suffer from semantic&#10;inconsistency (i.e., semantic label flipping) and temporal flickering&#10;artifacts. To alleviate these issues, we propose a new framework that is&#10;composed of carefully-designed generators and discriminators, coupled with two&#10;core objective functions: 1) content preserving loss and 2) temporal&#10;consistency loss. Extensive qualitative and quantitative evaluations&#10;demonstrate the superior performance of the proposed method against previous&#10;approaches. We further apply our framework to a domain adaptation task and&#10;achieve favorable results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.09514" label="1908.09514">
-        <attvalues>
-          <attvalue for="0" value="Mocycle-GAN: Unpaired Video-to-Video Translation" />
-          <attvalue for="1" value="  Unsupervised image-to-image translation is the task of translating an image&#10;from one domain to another in the absence of any paired training examples and&#10;tends to be more applicable to practical applications. Nevertheless, the&#10;extension of such synthesis from image-to-image to video-to-video is not&#10;trivial especially when capturing spatio-temporal structures in videos. The&#10;difficulty originates from the aspect that not only the visual appearance in&#10;each frame but also motion between consecutive frames should be realistic and&#10;consistent across transformation. This motivates us to explore both appearance&#10;structure and temporal continuity in video synthesis. In this paper, we present&#10;a new Motion-guided Cycle GAN, dubbed as Mocycle-GAN, that novelly integrates&#10;motion estimation into unpaired video translator. Technically, Mocycle-GAN&#10;capitalizes on three types of constrains: adversarial constraint discriminating&#10;between synthetic and real frame, cycle consistency encouraging an inverse&#10;translation on both frame and motion, and motion translation validating the&#10;transfer of motion between consecutive frames. Extensive experiments are&#10;conducted on video-to-labels and labels-to-video translation, and superior&#10;results are reported when comparing to state-of-the-art methods. More&#10;remarkably, we qualitatively demonstrate our Mocycle-GAN for both&#10;flower-to-flower and ambient condition transfer.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.08060" label="1904.08060">
-        <attvalues>
-          <attvalue for="0" value="Deep Fusion Network for Image Completion" />
-          <attvalue for="1" value="  Deep image completion usually fails to harmonically blend the restored image&#10;into existing content, especially in the boundary area. This paper handles with&#10;this problem from a new perspective of creating a smooth transition and&#10;proposes a concise Deep Fusion Network (DFNet). Firstly, a fusion block is&#10;introduced to generate a flexible alpha composition map for combining known and&#10;unknown regions. The fusion block not only provides a smooth fusion between&#10;restored and existing content, but also provides an attention map to make&#10;network focus more on the unknown pixels. In this way, it builds a bridge for&#10;structural and texture information, so that information can be naturally&#10;propagated from known region into completion. Furthermore, fusion blocks are&#10;embedded into several decoder layers of the network. Accompanied by the&#10;adjustable loss constraints on each layer, more accurate structure information&#10;are achieved. We qualitatively and quantitatively compare our method with other&#10;state-of-the-art methods on Places2 and CelebA datasets. The results show the&#10;superior performance of DFNet, especially in the aspects of harmonious texture&#10;transition, texture detail and semantic structural consistency. Our source code&#10;will be avaiable at: \url{https://github.com/hughplay/DFNet}&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.01639" label="1905.01639">
-        <attvalues>
-          <attvalue for="0" value="Deep Video Inpainting" />
-          <attvalue for="1" value="  Video inpainting aims to fill spatio-temporal holes with plausible content in&#10;a video. Despite tremendous progress of deep neural networks for image&#10;inpainting, it is challenging to extend these methods to the video domain due&#10;to the additional time dimension. In this work, we propose a novel deep network&#10;architecture for fast video inpainting. Built upon an image-based&#10;encoder-decoder model, our framework is designed to collect and refine&#10;information from neighbor frames and synthesize still-unknown regions. At the&#10;same time, the output is enforced to be temporally consistent by a recurrent&#10;feedback and a temporal memory module. Compared with the state-of-the-art image&#10;inpainting algorithm, our method produces videos that are much more&#10;semantically correct and temporally smooth. In contrast to the prior video&#10;completion method which relies on time-consuming optimization, our method runs&#10;in near real-time while generating competitive video results. Finally, we&#10;applied our framework to video retargeting task, and obtain visually pleasing&#10;results.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1604.08610" label="1604.08610">
-        <attvalues>
-          <attvalue for="0" value="Artistic style transfer for videos" />
-          <attvalue for="1" value="  In the past, manually re-drawing an image in a certain artistic style&#10;required a professional artist and a long time. Doing this for a video sequence&#10;single-handed was beyond imagination. Nowadays computers provide new&#10;possibilities. We present an approach that transfers the style from one image&#10;(for example, a painting) to a whole video sequence. We make use of recent&#10;advances in style transfer in still images and propose new initializations and&#10;loss functions applicable to videos. This allows us to generate consistent and&#10;stable stylized video sequences, even in cases with large motion and strong&#10;occlusion. We show that the proposed method clearly outperforms simpler&#10;baselines both qualitatively and quantitatively.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1801.04590" label="1801.04590">
-        <attvalues>
-          <attvalue for="0" value="Frame-Recurrent Video Super-Resolution" />
-          <attvalue for="1" value="  Recent advances in video super-resolution have shown that convolutional&#10;neural networks combined with motion compensation are able to merge information&#10;from multiple low-resolution (LR) frames to generate high-quality images.&#10;Current state-of-the-art methods process a batch of LR frames to generate a&#10;single high-resolution (HR) frame and run this scheme in a sliding window&#10;fashion over the entire video, effectively treating the problem as a large&#10;number of separate multi-frame super-resolution tasks. This approach has two&#10;main weaknesses: 1) Each input frame is processed and warped multiple times,&#10;increasing the computational cost, and 2) each output frame is estimated&#10;independently conditioned on the input frames, limiting the system's ability to&#10;produce temporally consistent results.&#10;  In this work, we propose an end-to-end trainable frame-recurrent video&#10;super-resolution framework that uses the previously inferred HR estimate to&#10;super-resolve the subsequent frame. This naturally encourages temporally&#10;consistent results and reduces the computational cost by warping only one image&#10;in each step. Furthermore, due to its recurrent nature, the proposed method has&#10;the ability to assimilate a large number of previous frames without increased&#10;computational demands. Extensive evaluations and comparisons with previous&#10;methods validate the strengths of our approach and demonstrate that the&#10;proposed framework is able to significantly outperform the current state of the&#10;art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1705.02092" label="1705.02092">
-        <attvalues>
-          <attvalue for="0" value="Characterizing and Improving Stability in Neural Style Transfer" />
-          <attvalue for="1" value="  Recent progress in style transfer on images has focused on improving the&#10;quality of stylized images and speed of methods. However, real-time methods are&#10;highly unstable resulting in visible flickering when applied to videos. In this&#10;work we characterize the instability of these methods by examining the solution&#10;set of the style transfer objective. We show that the trace of the Gram matrix&#10;representing style is inversely related to the stability of the method. Then,&#10;we present a recurrent convolutional network for real-time video style transfer&#10;which incorporates a temporal consistency loss and overcomes the instability of&#10;prior methods. Our networks can be applied at any resolution, do not re- quire&#10;optical flow at test time, and produce high quality, temporally consistent&#10;stylized videos in real-time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1909.07957" label="1909.07957">
-        <attvalues>
-          <attvalue for="0" value="An Internal Learning Approach to Video Inpainting" />
-          <attvalue for="1" value="  We propose a novel video inpainting algorithm that simultaneously&#10;hallucinates missing appearance and motion (optical flow) information, building&#10;upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network&#10;architectures to enforce plausible texture in static images. In extending DIP&#10;to video we make two important contributions. First, we show that coherent&#10;video inpainting is possible without a priori training. We take a generative&#10;approach to inpainting based on internal (within-video) learning without&#10;reliance upon an external corpus of visual data to train a one-size-fits-all&#10;model for the large space of general videos. Second, we show that such a&#10;framework can jointly generate both appearance and flow, whilst exploiting&#10;these complementary modalities to ensure mutual consistency. We show that&#10;leveraging appearance statistics specific to each video achieves visually&#10;plausible results whilst handling the challenging problem of long-term&#10;consistency.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.01131" label="1907.01131">
-        <attvalues>
-          <attvalue for="0" value="Learnable Gated Temporal Shift Module for Deep Video Inpainting" />
-          <attvalue for="1" value="  How to efficiently utilize temporal information to recover videos in a&#10;consistent way is the main issue for video inpainting problems. Conventional 2D&#10;CNNs have achieved good performance on image inpainting but often lead to&#10;temporally inconsistent results where frames will flicker when applied to&#10;videos (see&#10;https://www.youtube.com/watch?v=87Vh1HDBjD0&amp;list=PLPoVtv-xp_dL5uckIzz1PKwNjg1yI0I94&amp;index=1);&#10;3D CNNs can capture temporal information but are computationally intensive and&#10;hard to train. In this paper, we present a novel component termed Learnable&#10;Gated Temporal Shift Module (LGTSM) for video inpainting models that could&#10;effectively tackle arbitrary video masks without additional parameters from 3D&#10;convolutions. LGTSM is designed to let 2D convolutions make use of neighboring&#10;frames more efficiently, which is crucial for video inpainting. Specifically,&#10;in each layer, LGTSM learns to shift some channels to its temporal neighbors so&#10;that 2D convolutions could be enhanced to handle temporal information.&#10;Meanwhile, a gated convolution is applied to the layer to identify the masked&#10;areas that are poisoning for conventional convolutions. On the FaceForensics&#10;and Free-form Video Inpainting (FVI) dataset, our model achieves&#10;state-of-the-art results with simply 33% of parameters and inference time.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1808.00449" label="1808.00449">
-        <attvalues>
-          <attvalue for="0" value="Learning Blind Video Temporal Consistency" />
-          <attvalue for="1" value="  Applying image processing algorithms independently to each frame of a video&#10;often leads to undesired inconsistent results over time. Developing temporally&#10;consistent video-based extensions, however, requires domain knowledge for&#10;individual tasks and is unable to generalize to other applications. In this&#10;paper, we present an efficient end-to-end approach based on deep recurrent&#10;network for enforcing temporal consistency in a video. Our method takes the&#10;original unprocessed and per-frame processed videos as inputs to produce a&#10;temporally consistent video. Consequently, our approach is agnostic to specific&#10;image processing algorithms applied on the original video. We train the&#10;proposed network by minimizing both short-term and long-term temporal losses as&#10;well as the perceptual loss to strike a balance between temporal stability and&#10;perceptual similarity with the processed frames. At test time, our model does&#10;not require computing optical flow and thus achieves real-time speed even for&#10;high-resolution videos. We show that our single model can handle multiple and&#10;unseen tasks, including but not limited to artistic style transfer,&#10;enhancement, colorization, image-to-image translation and intrinsic image&#10;decomposition. Extensive objective evaluation and subject study demonstrate&#10;that the proposed approach performs favorably against the state-of-the-art&#10;methods on various types of videos.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.00461" label="1809.00461">
-        <attvalues>
-          <attvalue for="0" value="YouTube-VOS: Sequence-to-Sequence Video Object Segmentation" />
-          <attvalue for="1" value="  Learning long-term spatial-temporal features are critical for many video&#10;analysis tasks. However, existing video segmentation methods predominantly rely&#10;on static image segmentation techniques, and methods capturing temporal&#10;dependency for segmentation have to depend on pretrained optical flow models,&#10;leading to suboptimal solutions for the problem. End-to-end sequential learning&#10;to explore spatial-temporal features for video segmentation is largely limited&#10;by the scale of available video segmentation datasets, i.e., even the largest&#10;video segmentation dataset only contains 90 short video clips. To solve this&#10;problem, we build a new large-scale video object segmentation dataset called&#10;YouTube Video Object Segmentation dataset (YouTube-VOS). Our dataset contains&#10;3,252 YouTube video clips and 78 categories including common objects and human&#10;activities. This is by far the largest video object segmentation dataset to our&#10;knowledge and we have released it at https://youtube-vos.org. Based on this&#10;dataset, we propose a novel sequence-to-sequence network to fully exploit&#10;long-term spatial-temporal information in videos for segmentation. We&#10;demonstrate that our method is able to achieve the best results on our&#10;YouTube-VOS test set and comparable results on DAVIS 2016 compared to the&#10;current state-of-the-art methods. Experiments show that the large scale dataset&#10;is indeed a key factor to the success of our model.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2108.08643" label="2108.08643">
-        <attvalues>
-          <attvalue for="0" value="Batch Curation for Unsupervised Contrastive Representation Learning" />
-          <attvalue for="1" value="  The state-of-the-art unsupervised contrastive visual representation learning&#10;methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data&#10;augmentations in order to construct a pretext task of instant discrimination&#10;consisting of similar and dissimilar pairs of images. Similar pairs are&#10;constructed by randomly extracting patches from the same image and applying&#10;several other transformations such as color jittering or blurring, while&#10;transformed patches from different image instances in a given batch are&#10;regarded as dissimilar pairs. We argue that this approach can result similar&#10;pairs that are \textit{semantically} dissimilar. In this work, we address this&#10;problem by introducing a \textit{batch curation} scheme that selects batches&#10;during the training process that are more inline with the underlying&#10;contrastive objective. We provide insights into what constitutes beneficial&#10;similar and dissimilar pairs as well as validate \textit{batch curation} on&#10;CIFAR10 by integrating it in the SimCLR model.&#10;" />
-          <attvalue for="2" value="&#10;Extracting data representation that are compact, expressive, and meaningful is a long term goal in the field of machine learning~\cite{bengio2013representation}. In recent years, most of the efforts have been put on learning general data representations that are suitable for a variety of different downstream task without using human annotation. In computer vision, prior work addressed this by solving pretext tasks defined in a self-supervised manner such as denoising autoencoders~\cite{vincent2008extracting}, solving jigsaw puzzles~\cite{noroozi2016unsupervised}, or motion segmentation~\cite{pathak2017learning}. A specific pretext task, namely instance discrimination~\cite{wu2018unsupervised}, has recently emerged as the state-of-the-art for unsupervised visual representation learning, spawning a polyhedral of new models based on contrastive learning such as SimCLR~\cite{chen2020simple}, MoCo~\cite{he2020momentum} or SwAV~\cite{caron2020unsupervised}.&#10;&#10;In these methods, the pretext tasks are constructed by applying various data transformations, for example, cropping, color jittering or gaussian blur, in order to obtain similar and dissimilar pairs in an unsupervised way. Thanks to the thorough ablation study performed by the respective methods, it is established that random resized cropping, i.e., extracting a patch of random size from the original image and resizing it back to the size of the original image, is one of the crucial data augmentation for learning useful representations~\cite{chen2020simple}. The intuition behind it is that patches drawn from the same image contain similar semantic information and should therefore be encoded close to each other. &#10;However, we hypothesise that this approach &#10;can produce patches containing semantically dissimilar information which decelerate learning as they violate this core similarity assumption.&#10;&#10;In this work, we first take a closer look at the random resize cropping and investigate how the size of the patches as well as their relative position to each other in the original image influences the learning process. Based on our findings, we then propose a batch curation scheme, which during training constructs similar and dissimilar pairs that are aligned with the contrastive objective. Our scheme is simple as it uses the given model in training to self-curate batches such that similar patches are semantically close as judged by the model itself.&#10;It can be easily adapted to any contrastive learning setting with arbitrary number of similar and dissimilar pairs, and can therefore be integrated into any of the current state-of-the-art methods. We validate our batch curation scheme in a small scale experiment by integrating it into the SimCLR~\cite{chen2020simple} model trained using CIFAR-10~\cite{krizhevsky2009learning} dataset. We show an improvement of $\approx 1.5\%$ on the K-NN classification evaluation as well as competitive performance on the linear classification protocol. To the best of our knowledge, we are the first to introduce a procedure for providing training batches of higher quality for unsupervised visual contrastive learning methods.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Image Data Augmentation, Mathematics, Artificial Intelligence, Contrastive Learning Methods, Cognitive Science, Computer Science, Visual Representation Learning" />
-        </attvalues>
-      </node>
-      <node id="1206.5538" label="1206.5538">
-        <attvalues>
-          <attvalue for="0" value="Representation Learning: A Review and New Perspectives" />
-          <attvalue for="1" value="  The success of machine learning algorithms generally depends on data&#10;representation, and we hypothesize that this is because different&#10;representations can entangle and hide more or less the different explanatory&#10;factors of variation behind the data. Although specific domain knowledge can be&#10;used to help design representations, learning with generic priors can also be&#10;used, and the quest for AI is motivating the design of more powerful&#10;representation-learning algorithms implementing such priors. This paper reviews&#10;recent work in the area of unsupervised feature learning and deep learning,&#10;covering advances in probabilistic models, auto-encoders, manifold learning,&#10;and deep networks. This motivates longer-term unanswered questions about the&#10;appropriate objectives for learning good representations, for computing&#10;representations (i.e., inference), and the geometrical connections between&#10;representation learning, density estimation and manifold learning.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1612.06370" label="1612.06370">
-        <attvalues>
-          <attvalue for="0" value="Learning Features by Watching Objects Move" />
-          <attvalue for="1" value="  This paper presents a novel yet intuitive approach to unsupervised feature&#10;learning. Inspired by the human visual system, we explore whether low-level&#10;motion-based grouping cues can be used to learn an effective visual&#10;representation. Specifically, we use unsupervised motion-based segmentation on&#10;videos to obtain segments, which we use as 'pseudo ground truth' to train a&#10;convolutional network to segment objects from a single frame. Given the&#10;extensive evidence that motion plays a key role in the development of the human&#10;visual system, we hope that this straightforward approach to unsupervised&#10;learning will be more effective than cleverly designed 'pretext' tasks studied&#10;in the literature. Indeed, our extensive experiments show that this is the&#10;case. When used for transfer learning on object detection, our representation&#10;significantly outperforms previous unsupervised approaches across multiple&#10;settings, especially when training data for the target task is scarce.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2006.09882" label="2006.09882">
-        <attvalues>
-          <attvalue for="0" value="Unsupervised Learning of Visual Features by Contrasting Cluster&#10;  Assignments" />
-          <attvalue for="1" value="  Unsupervised image representations have significantly reduced the gap with&#10;supervised pretraining, notably with the recent achievements of contrastive&#10;learning methods. These contrastive methods typically work online and rely on a&#10;large number of explicit pairwise feature comparisons, which is computationally&#10;challenging. In this paper, we propose an online algorithm, SwAV, that takes&#10;advantage of contrastive methods without requiring to compute pairwise&#10;comparisons. Specifically, our method simultaneously clusters the data while&#10;enforcing consistency between cluster assignments produced for different&#10;augmentations (or views) of the same image, instead of comparing features&#10;directly as in contrastive learning. Simply put, we use a swapped prediction&#10;mechanism where we predict the cluster assignment of a view from the&#10;representation of another view. Our method can be trained with large and small&#10;batches and can scale to unlimited amounts of data. Compared to previous&#10;contrastive methods, our method is more memory efficient since it does not&#10;require a large memory bank or a special momentum network. In addition, we also&#10;propose a new data augmentation strategy, multi-crop, that uses a mix of views&#10;with different resolutions in place of two full-resolution views, without&#10;increasing the memory or compute requirements much. We validate our findings by&#10;achieving 75.3% top-1 accuracy on ImageNet with ResNet-50, as well as&#10;surpassing supervised pretraining on all the considered transfer tasks.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2101.04544" label="2101.04544">
-        <attvalues>
-          <attvalue for="0" value="Resolution-invariant Person ReID Based on Feature Transformation and&#10;  Self-weighted Attention" />
-          <attvalue for="1" value="  Person Re-identification (ReID) is a critical computer vision task which aims&#10;to match the same person in images or video sequences. Most current works focus&#10;on settings where the resolution of images is kept the same. However, the&#10;resolution is a crucial factor in person ReID, especially when the cameras are&#10;at different distances from the person or the camera's models are different&#10;from each other. In this paper, we propose a novel two-stream network with a&#10;lightweight resolution association ReID feature transformation (RAFT) module&#10;and a self-weighted attention (SWA) ReID module to evaluate features under&#10;different resolutions. RAFT transforms the low resolution features to&#10;corresponding high resolution features. SWA evaluates both features to get&#10;weight factors for the person ReID. Both modules are jointly trained to get a&#10;resolution-invariant representation. Extensive experiments on five benchmark&#10;datasets show the effectiveness of our method. For instance, we achieve Rank-1&#10;accuracy of 43.3% and 83.2% on CAVIAR and MLR-CUHK03, outperforming the&#10;state-of-the-art.&#10;" />
-          <attvalue for="2" value="&#10;Person ReID is a vital computer vision task that aims to recognize the same person across the images or video sequences taken by different cameras \cite{zheng2016person, ye2020deep}.&#10;Due to the success of convolutional neural networks (CNNs), a variety of learning-based methods \cite{chang2018multi, sun2018beyond, zheng2017unlabeled} have been proposed to address person ReID problem and achieved good performance on many benchmarks.&#10;Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task.&#10;&#10;However, these methods are mostly developed under the settings where both query and gallery images are of high resolution (HR), which is not always true in practice since people may be captured by a camera in one resolution and then reappear under another camera in a drastically different resolution. In practical settings, query images are usually of low resolution (LR) captured by a surveillance system, while gallery images are mostly of HR which are chosen carefully. &#10;In practice, resolution changes can significantly affect the ReID results because of less discriminative appearance information of LR images and the gap between image features from different resolutions. Therefore, it is a necessity to study algorithms for handling resolution changes with high robustness in Person ReID.&#10;&#10;To address the above problem, existing methods \cite{jiao2018deep, chen2019learning, mao2019resolution,li2019recover} either use super-resolution (SR) models or generative adversarial network (GAN) to convert LR images into their corresponding HR images, followed by a person ReID model. However, these two kinds of methods suffer from different problems. SR is a one-on-one scheme, which means it is unable to handle arbitrary inputting resolution. Multiple SRs may be trained to deal with multiple inputting/outputting resolutions. However, the number of resolutions is still finite and needs to be pre-defined.&#10;GAN based methods require a large number of resources and training time, making them impractical in many resource-limited scenarios. Both of them regard generating HR images and person ReID as two separately consecutive phases, ignoring the relations between the features in both phases.&#10;&#10;To better resolve the above challenges, in this paper, we propose a novel network with the following three highlights:&#10;\begin{enumerate}[topsep = 5 pt]&#10;&#10;\item&#10;The input of the proposed network is two-stream, one for HR images and the other for LR images.&#10;\item&#10;We design a module to transform LR image features to their corresponding HR image features, rather than the transformation between images themselves. The module manages to connect the relations between ReID features from different resolutions.&#10;\item&#10;A self-weighted attention ReID module is proposed to evaluate the quality of each input HR and LR ReID features, making it capable of merging HR and LR features with a weighted distance metric.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Person Re-identification" />
-        </attvalues>
-      </node>
-      <node id="1803.09132" label="1803.09132">
-        <attvalues>
-          <attvalue for="0" value="Multi-Level Factorisation Net for Person Re-Identification" />
-          <attvalue for="1" value="  Key to effective person re-identification (Re-ID) is modelling discriminative&#10;and view-invariant factors of person appearance at both high and low semantic&#10;levels. Recently developed deep Re-ID models either learn a holistic single&#10;semantic level feature representation and/or require laborious human annotation&#10;of these factors as attributes. We propose Multi-Level Factorisation Net&#10;(MLFN), a novel network architecture that factorises the visual appearance of a&#10;person into latent discriminative factors at multiple semantic levels without&#10;manual annotation. MLFN is composed of multiple stacked blocks. Each block&#10;contains multiple factor modules to model latent factors at a specific level,&#10;and factor selection modules that dynamically select the factor modules to&#10;interpret the content of each input image. The outputs of the factor selection&#10;modules also provide a compact latent factor descriptor that is complementary&#10;to the conventional deeply learned features. MLFN achieves state-of-the-art&#10;results on three Re-ID datasets, as well as compelling results on the general&#10;object categorisation CIFAR-100 dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1907.08427" label="1907.08427">
-        <attvalues>
-          <attvalue for="0" value="VRSTC: Occlusion-Free Video Person Re-Identification" />
-          <attvalue for="1" value="  Video person re-identification (re-ID) plays an important role in&#10;surveillance video analysis. However, the performance of video re-ID&#10;degenerates severely under partial occlusion. In this paper, we propose a novel&#10;network, called Spatio-Temporal Completion network (STCnet), to explicitly&#10;handle partial occlusion problem. Different from most previous works that&#10;discard the occluded frames, STCnet can recover the appearance of the occluded&#10;parts. For one thing, the spatial structure of a pedestrian frame can be used&#10;to predict the occluded body parts from the unoccluded body parts of this&#10;frame. For another, the temporal patterns of pedestrian sequence provide&#10;important clues to generate the contents of occluded parts. With the&#10;Spatio-temporal information, STCnet can recover the appearance for the occluded&#10;parts, which could be leveraged with those unoccluded parts for more accurate&#10;video re-ID. By combining a re-ID network with STCnet, a video re-ID framework&#10;robust to partial occlusion (VRSTC) is proposed. Experiments on three&#10;challenging video re-ID databases demonstrate that the proposed approach&#10;outperforms the state-of-the-art.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1701.07717" label="1701.07717">
-        <attvalues>
-          <attvalue for="0" value="Unlabeled Samples Generated by GAN Improve the Person Re-identification&#10;  Baseline in vitro" />
-          <attvalue for="1" value="  The main contribution of this paper is a simple semi-supervised pipeline that&#10;only uses the original training set without collecting extra data. It is&#10;challenging in 1) how to obtain more training data only from the training set&#10;and 2) how to use the newly generated data. In this work, the generative&#10;adversarial network (GAN) is used to generate unlabeled samples. We propose the&#10;label smoothing regularization for outliers (LSRO). This method assigns a&#10;uniform label distribution to the unlabeled images, which regularizes the&#10;supervised model and improves the baseline. We verify the proposed method on a&#10;practical problem: person re-identification (re-ID). This task aims to retrieve&#10;a query person from other cameras. We adopt the deep convolutional generative&#10;adversarial network (DCGAN) for sample generation, and a baseline convolutional&#10;neural network (CNN) for representation learning. Experiments show that adding&#10;the GAN-generated data effectively improves the discriminative ability of&#10;learned CNN embeddings. On three large-scale datasets, Market-1501, CUHK03 and&#10;DukeMTMC-reID, we obtain +4.37%, +1.6% and +2.46% improvement in rank-1&#10;precision over the baseline CNN, respectively. We additionally apply the&#10;proposed method to fine-grained bird recognition and achieve a +0.6%&#10;improvement over a strong baseline. The code is available at&#10;https://github.com/layumi/Person-reID_GAN.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.07223" label="1904.07223">
-        <attvalues>
-          <attvalue for="0" value="Joint Discriminative and Generative Learning for Person&#10;  Re-identification" />
-          <attvalue for="1" value="  Person re-identification (re-id) remains challenging due to significant&#10;intra-class variations across different cameras. Recently, there has been a&#10;growing interest in using generative models to augment training data and&#10;enhance the invariance to input changes. The generative pipelines in existing&#10;methods, however, stay relatively separate from the discriminative re-id&#10;learning stages. Accordingly, re-id models are often trained in a&#10;straightforward manner on the generated data. In this paper, we seek to improve&#10;learned re-id embeddings by better leveraging the generated data. To this end,&#10;we propose a joint learning framework that couples re-id learning and data&#10;generation end-to-end. Our model involves a generative module that separately&#10;encodes each person into an appearance code and a structure code, and a&#10;discriminative module that shares the appearance encoder with the generative&#10;module. By switching the appearance or structure codes, the generative module&#10;is able to generate high-quality cross-id composed images, which are online fed&#10;back to the appearance encoder and used to improve the discriminative module.&#10;The proposed joint learning framework renders significant improvement over the&#10;baseline without using generated data, leading to the state-of-the-art&#10;performance on several benchmark datasets.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.03349" label="1904.03349">
-        <attvalues>
-          <attvalue for="0" value="Progressive Pose Attention Transfer for Person Image Generation" />
-          <attvalue for="1" value="  This paper proposes a new generative adversarial network for pose transfer,&#10;i.e., transferring the pose of a given person to a target pose. The generator&#10;of the network comprises a sequence of Pose-Attentional Transfer Blocks that&#10;each transfers certain regions it attends to, generating the person image&#10;progressively. Compared with those in previous works, our generated person&#10;images possess better appearance consistency and shape consistency with the&#10;input images, thus significantly more realistic-looking. The efficacy and&#10;efficiency of the proposed network are validated both qualitatively and&#10;quantitatively on Market-1501 and DeepFashion. Furthermore, the proposed&#10;architecture can generate training images for person re-identification,&#10;alleviating data insufficiency. Codes and models are available at:&#10;https://github.com/tengteng95/Pose-Transfer.git.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1712.02225" label="1712.02225">
-        <attvalues>
-          <attvalue for="0" value="Pose-Normalized Image Generation for Person Re-identification" />
-          <attvalue for="1" value="  Person Re-identification (re-id) faces two major challenges: the lack of&#10;cross-view paired training data and learning discriminative identity-sensitive&#10;and view-invariant features in the presence of large pose variations. In this&#10;work, we address both problems by proposing a novel deep person image&#10;generation model for synthesizing realistic person images conditional on the&#10;pose. The model is based on a generative adversarial network (GAN) designed&#10;specifically for pose normalization in re-id, thus termed pose-normalization&#10;GAN (PN-GAN). With the synthesized images, we can learn a new type of deep&#10;re-id feature free of the influence of pose variations. We show that this&#10;feature is strong on its own and complementary to features learned with the&#10;original images. Importantly, under the transfer learning setting, we show that&#10;our model generalizes well to any new re-id dataset without the need for&#10;collecting any training data for model fine-tuning. The model thus has the&#10;potential to make re-id model truly scalable.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1905.04525" label="1905.04525">
-        <attvalues>
-          <attvalue for="0" value="Illumination-Adaptive Person Re-identification" />
-          <attvalue for="1" value="  Most person re-identification (ReID) approaches assume that person images are&#10;captured under relatively similar illumination conditions. In reality,&#10;long-term person retrieval is common, and person images are often captured&#10;under different illumination conditions at different times across a day. In&#10;this situation, the performances of existing ReID models often degrade&#10;dramatically. This paper addresses the ReID problem with illumination&#10;variations and names it as {\em Illumination-Adaptive Person Re-identification&#10;(IA-ReID)}. We propose an Illumination-Identity Disentanglement (IID) network&#10;to dispel different scales of illuminations away while preserving individuals'&#10;identity information. To demonstrate the illumination issue and to evaluate our&#10;model, we construct two large-scale simulated datasets with a wide range of&#10;illumination variations. Experimental results on the simulated datasets and&#10;real-world images demonstrate the effectiveness of the proposed framework.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2002.01625" label="2002.01625">
-        <attvalues>
-          <attvalue for="0" value="Illumination adaptive person reid based on teacher-student model and&#10;  adversarial training" />
-          <attvalue for="1" value="  Most existing works in Person Re-identification (ReID) focus on settings&#10;where illumination either is kept the same or has very little fluctuation.&#10;However, the changes in the illumination degree may affect the robustness of a&#10;ReID algorithm significantly. To address this problem, we proposed a Two-Stream&#10;Network that can separate ReID features from lighting features to enhance ReID&#10;performance. Its innovations are threefold: (1) A discriminative entropy loss&#10;to ensure the ReID features contain no lighting information. (2) A ReID Teacher&#10;model trained by images under &quot;neutral&quot; lighting conditions to guide ReID&#10;classification. (3) An illumination Teacher model trained by the differences&#10;between the illumination-adjusted and original images to guide illumination&#10;classification. We construct two augmented datasets by synthetically changing a&#10;set of predefined lighting conditions in two of the most popular ReID&#10;benchmarks: Market1501 and DukeMTMC-ReID. Experiments demonstrate that our&#10;algorithm outperforms other state-of-the-art works and particularly potent in&#10;handling images under extremely low light.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.06052" label="1908.06052">
-        <attvalues>
-          <attvalue for="0" value="Recover and Identify: A Generative Dual Model for Cross-Resolution&#10;  Person Re-Identification" />
-          <attvalue for="1" value="  Person re-identification (re-ID) aims at matching images of the same identity&#10;across camera views. Due to varying distances between cameras and persons of&#10;interest, resolution mismatch can be expected, which would degrade person re-ID&#10;performance in real-world scenarios. To overcome this problem, we propose a&#10;novel generative adversarial network to address cross-resolution person re-ID,&#10;allowing query images with varying resolutions. By advancing adversarial&#10;learning techniques, our proposed model learns resolution-invariant image&#10;representations while being able to recover the missing details in&#10;low-resolution input images. The resulting features can be jointly applied for&#10;improving person re-ID performance due to preserving resolution invariance and&#10;recovering re-ID oriented discriminative details. Our experiments on five&#10;benchmark datasets confirm the effectiveness of our approach and its&#10;superiority over the state-of-the-art methods, especially when the input&#10;resolutions are unseen during training.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2106.06649" label="2106.06649">
-        <attvalues>
-          <attvalue for="0" value="1st Place Solution for YouTubeVOS Challenge 2021:Video Instance&#10;  Segmentation" />
-          <attvalue for="1" value="  Video Instance Segmentation (VIS) is a multi-task problem performing&#10;detection, segmentation, and tracking simultaneously. Extended from image set&#10;applications, video data additionally induces the temporal information, which,&#10;if handled appropriately, is very useful to identify and predict object&#10;motions. In this work, we design a unified model to mutually learn these tasks.&#10;Specifically, we propose two modules, named Temporally Correlated Instance&#10;Segmentation (TCIS) and Bidirectional Tracking (BiTrack), to take the benefit&#10;of the temporal correlation between the object's instance masks across adjacent&#10;frames. On the other hand, video data is often redundant due to the frame's&#10;overlap. Our analysis shows that this problem is particularly severe for the&#10;YoutubeVOS-VIS2021 data. Therefore, we propose a Multi-Source Data (MSD)&#10;training mechanism to compensate for the data deficiency. By combining these&#10;techniques with a bag of tricks, the network performance is significantly&#10;boosted compared to the baseline, and outperforms other methods by a&#10;considerable margin on the YoutubeVOS-VIS 2019 and 2021 datasets.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In this technical report, we present a solution for the task of Video Instance Segmentation (VIS), specifically targeting the VIS dataset hold by the CVPR2021-YoutubeVOS 2021 Workshop. VIS, first introduced in the YoutubeVOS 2019 challenge \cite{vis}, aims to perform object detection, instance segmentation, and object tracking across video frames. There are 2883 videos with 40 categories in the original 2019 version. In 2021, the dataset is enriched with more than 3800 videos, each has about 30 frames, and the categories are also refined.&#10;&#10;VIS by its nature is a multi-task learning problem, and generally there two main approaches. A straightforward way is to perform each individual task separately and sequentially \cite{2019vis_first}. However, since the components are trained and inferred independently, the full pipeline is complicated, slow, and sub-optimal. The second approach \cite{vis_2018_second, vis_2018_third, vis_2018_x} aims to build a single model that jointly learns and performs all the tasks simultaneously. This not only simplifies the pipeline, reduces the inference latency, but potentially improves the final performance.&#10;&#10;Our solution also follows the unified direction but is designed to address several specific technical challenges for the dataset. We also hope that it can serve as a strong baseline for more general applications. Our main contributions for the challenge are summarized as follows:&#10;&#10;\begin{itemize}&#10;\item Our data analysis shows that only a small portion ($17\%$) of the training images are useful, while the rest ($83\%$) are ineffective. We hence propose a training mechanism, named Multi-Source Data (MSD), which could both increase the diversity of data and improves model generalization.&#10;&#10;\item We exploit multi-task learning and propose the Temporally Correlated Instance Segmentation (TCIS) module to learn the temporal relation between instance masks over adjacent frames.&#10;&#10;\item We suggest a Bidirectional Tracking (BiTrack) post-processing step to track objects in both forward and backward order to recall more objects, before merging two sets of tracks to obtain a final result.&#10;&#10;\item Our method secures the 1st rank on the YoutubeVOS-VIS2021, with the score $0.575$ mAP on the public validation set, and $0.541$ mAP on the private test set. Evaluating the YouTubeVOS-VIS2019 dataset, our solution also obtains $0.543$ mAP, setting a new record for the benchmark.&#10;\end{itemize}&#10;&#10;The paper is organized as follows. Section \ref{sec:baseline} summarizes our baseline method, including the model architecture, training, and inference pipeline. Section \ref{sec:proposed-method} describes our main solutions, including the data analysis and techniques to improve data diversity, the TCIS component, and a bag of useful tricks to further improve the results. The implementation details, ablation study for different components, and comparison with other methods are presented in section\ref{sec:experiments}. Section \ref{sec:conclusion} concludes our paper.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Video Instance Segmentation, Artificial Intelligence, Temporal Information Analysis, Computer Science, Multi-Task Learning" />
-        </attvalues>
-      </node>
-      <node id="2304.02971" label="2304.02971">
-        <attvalues>
-          <attvalue for="0" value="Synthetic Hard Negative Samples for Contrastive Learning" />
-          <attvalue for="1" value="  Contrastive learning has emerged as an essential approach for self-supervised&#10;learning in visual representation learning. The central objective of&#10;contrastive learning is to maximize the similarities between two augmented&#10;versions of an image (positive pairs), while minimizing the similarities&#10;between different images (negative pairs). Recent studies have demonstrated&#10;that harder negative samples, i.e., those that are more difficult to&#10;differentiate from the anchor sample, perform a more crucial function in&#10;contrastive learning. This paper proposes a novel feature-level method, namely&#10;sampling synthetic hard negative samples for contrastive learning (SSCL), to&#10;exploit harder negative samples more effectively. Specifically, 1) we generate&#10;more and harder negative samples by mixing negative samples, and then sample&#10;them by controlling the contrast of anchor sample with the other negative&#10;samples; 2) considering the possibility of false negative samples, we further&#10;debias the negative samples. Our proposed method improves the classification&#10;performance on different image datasets and can be readily integrated into&#10;existing methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;During the recent years, self-supervised learning has undergone notable advancements, with the emergence of numerous noteworthy methodologies. Among these, contrastive learning has garnered significant attention, owing to its exceptional performance in many downstream tasks.&#10;&#10;Contrastive learning is mainly utilized to train the network model by maximizing the similarities of positive pairs and minimizing the similarities of negative pairs. Among many of the existing methods, positive samples usually come from different augmentation versions of the same image, as shown in Figure \ref{fig1}(b). In contrast, negative samples are obtained either from the current batch or memory bank. Because the number of negative samples will directly affect the training results, in view of this, SimCLR provides more negative samples by increasing the batch size \cite{b1}, while MoCo \cite{b2} and MoCo v2 \cite{b3} using momentum update mechanism keeps a queue as memory bank, which can dynamically maintain the count of negative samples.&#10;&#10;However, more negative samples do not necessarily mean harder negative samples. The study \cite{b4} found that only a small minority of negative samples are both necessary and sufficient for contrastive learning. The queue in momentum contrastive learning contains many normal negative samples and few hard ones (about 5$\%$). Only few hard negative samples actually work when the model is trained, while many existing methods have ignored the importance of hard negative samples. In Figure \ref{fig1}, it is observed that the normal negative sample and the anchor sample belong to different classes, and their semantic information is dissimilar. Furthermore, the hard negative sample has similar semantic information with the anchor sample, although they belong to different classes.&#10;&#10;In this study, we emphasize the significance of hard negative samples in the training process of contrastive learning.&#10;We propose to sample synthetic hard negative samples. Firstly, instead of generating negative samples by increasing the batch size or memory bank, we select the hardest existing negative samples in the feature space and synthesize new ones. Secondly, we control the weights of negative samples in loss function by applying weighting factors to negative samples. Finally, &#10;we also utilize the debiased sampling to deal with the issue of false negative sample in contrastive learning. The false negative sample in Figure \ref{fig1}(e) contains the similar semantic information to the anchor sample but is considered as a negative sample.&#10;&#10;All our contributions are listed as follow:&#10;\begin{itemize}&#10;\item A method for sampling synthetic hard negative samples in contrastive learning is proposed. It enables the network model to learn more challenging representations from harder negative samples, by generating more and harder negative samples and sampling synthetic hard negative samples in the feature space.&#10;&#10;\item &#10;Considering the issue of the false negative sample, we also debias negative samples to ensure the truthfulness of the negative samples.&#10;&#10;\item The proposed approach is compatible with existing contrastive learning methods with only few computational overheads, and is achieved by simple modifying the standard contrastive loss function.&#10;&#10;\end{itemize}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Computer Vision, Visual Representation, Mathematics, Artificial Intelligence, Contrastive Learning, Cognitive Science, Computer Science, Self-Supervised Learning" />
-        </attvalues>
-      </node>
-      <node id="2003.04297" label="2003.04297">
-        <attvalues>
-          <attvalue for="0" value="Improved Baselines with Momentum Contrastive Learning" />
-          <attvalue for="1" value="  Contrastive unsupervised learning has recently shown encouraging progress,&#10;e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the&#10;effectiveness of two of SimCLR's design improvements by implementing them in&#10;the MoCo framework. With simple modifications to MoCo---namely, using an MLP&#10;projection head and more data augmentation---we establish stronger baselines&#10;that outperform SimCLR and do not require large training batches. We hope this&#10;will make state-of-the-art unsupervised learning research more accessible. Code&#10;will be made public.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2010.06682" label="2010.06682">
-        <attvalues>
-          <attvalue for="0" value="Are all negatives created equal in contrastive instance discrimination?" />
-          <attvalue for="1" value="  Self-supervised learning has recently begun to rival supervised learning on&#10;computer vision tasks. Many of the recent approaches have been based on&#10;contrastive instance discrimination (CID), in which the network is trained to&#10;recognize two augmented versions of the same instance (a query and positive)&#10;while discriminating against a pool of other instances (negatives). The learned&#10;representation is then used on downstream tasks such as image classification.&#10;Using methodology from MoCo v2 (Chen et al., 2020), we divided negatives by&#10;their difficulty for a given query and studied which difficulty ranges were&#10;most important for learning useful representations. We found a minority of&#10;negatives -- the hardest 5% -- were both necessary and sufficient for the&#10;downstream task to reach nearly full accuracy. Conversely, the easiest 95% of&#10;negatives were unnecessary and insufficient. Moreover, the very hardest 0.1% of&#10;negatives were unnecessary and sometimes detrimental. Finally, we studied the&#10;properties of negatives that affect their hardness, and found that hard&#10;negatives were more semantically similar to the query, and that some negatives&#10;were more consistently easy or hard than we would expect by chance. Together,&#10;our results indicate that negatives vary in importance and that CID may benefit&#10;from more intelligent negative treatment.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1910.12322" label="1910.12322">
-        <attvalues>
-          <attvalue for="0" value="Multi-Resolution Overlapping Stripes Network for Person&#10;  Re-Identification" />
-          <attvalue for="1" value="  This paper addresses the person re-identification (PReID) problem by&#10;combining global and local information at multiple feature resolutions with&#10;different loss functions. Many previous studies address this problem using&#10;either part-based features or global features. In case of part-based&#10;representation, the spatial correlation between these parts is not considered,&#10;while global-based representation are not sensitive to spatial variations. This&#10;paper presents a part-based model with a multi-resolution network that uses&#10;different level of features. The output of the last two conv blocks is then&#10;partitioned horizontally and processed in pairs with overlapping stripes to&#10;cover the important information that might lie between parts. We use different&#10;loss functions to combine local and global information for classification.&#10;Experimental results on a benchmark dataset demonstrate that the presented&#10;method outperforms the state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Person re-identification (PReID) is the task of identifying the presence of a person from multiple surveillance cameras.&#10;Given a query image, the aim is to retrieve all images of the specified person in a gallery dataset.&#10;This task has attracted the attention of many researchers in computer vision for its great importance in multiple applications such as video surveillance for public security.&#10;With the recent success of deep convolution neural networks (CNNs), PReID performance has made significant progress.&#10;Deep representations provide high discriminative ability, especially when aggregated from part-based deep local features. &#10;&#10;Current related studies in PReID can be categorized to global feature-based and local part-based models. The local part-based models perform better with certain variations such as partial occlusion. &#10;Sun \etal \cite{Sun2018}, for instance, presented the part-based convolutional baseline (PCB) that horizontally divided the last feature maps into multiple stripes where each one contains part of the person's body in the input image. &#10;After that, a refinement mechanism was applied to each piece to guarantee that the feature map of this part focuses on the correct body part. &#10;PCB is a simple and effective framework that outperforms the other part-based models. &#10;However, it does not consider global features which play an important role in recognition and identification tasks and are normally robust to multiple variations. On the other hand, since their stripes have no overlaps, it loses important information that might lie at the edges of the divided stripes.&#10;&#10;Global feature-based models focus on contour, shape, and texture representations.&#10;For example, Wang \etal \cite{Wang2018} built the DaReNet model based only on global information&#10;using a multiple granularity network to extract global features at different resolutions.&#10;Hermans \etal \cite{Hermans2017} presented a ResNet-50 based classifier which uses global information.&#10;Shen \etal \cite{GSRW} combined global features with random walk algorithm. &#10;Li \etal \cite{Li2018} proposed an attention-based model.&#10;Luo \etal \cite{Luo_2019} reported a strong CNN based model with bag of learning tricks including augmentation and regularization. &#10;However, those methods may fail in the presence of object occlusion, multiple poses and lighting variations and usually depend on pre- and post-processing steps to boost their performances.&#10;&#10;To address the above problems, other groups combined both global and local features.&#10;Li \etal \cite {li2019pedestrian} fused local and global features while using mutual learning but they did not train the model with multiple loss functions.&#10;While He \etal \cite{he2019mfbn} used attention aware model that combines global and local features.&#10;Quan \etal \cite{Quan2019} introduced neural architecture search to PReID by focusing on searching the best CNN structure and applied the part-aware module in PReID search space that employs both part and global information.&#10;&#10;Different loss functions have also been presented to boost the performance of the PReID models.&#10;Two loss functions are widely used: triplet loss \cite{FaceNet} and cross-entropy loss \cite{wen2016discriminative}. &#10;Triplet loss is based on the feature metrics distances while cross-entropy loss is based on classification with fully connected (FC) layers.&#10;Hermans \etal \cite{Hermans2017} and Zhang \etal \cite{zhang2019learning} modified triplet loss to increase the training performance.&#10;Fan \etal \cite{Fan2019} presented a classification model based on an extended version of the cross-entropy loss function and a warming-up learning rate to learn a hypersphere manifold embedding. &#10;Recently, several models \cite{he2019mfbn, Quan2019} are trained using a combination of triplet loss and cross-entropy loss.&#10;&#10;In this paper, we propose a multi-resolution overlapping stripes (MROS) model by combining global and local information at multiple resolutions with different loss functions.&#10;First, based on the residual network (ResNet50) \cite{Resnet}, multiple levels are created each of which has different resolution.&#10;Inspired by the PCB model \cite{Sun2018}, the feature map from each level is divided horizontally into multi-stripes which will be processed later in pairs with overlapping rather than individually.&#10;The overlapping avoids lost of information&#10;at the boundaries/edges of stripes which usually occurs when using the part-based models.&#10;Secondly, instead of using the features from all multi-resolution levels for classification as in \cite{Wang2018}, only the features from the last two levels are considered. This is because the later levels of the model learn &#10;more semantic representations&#10;compared to the early layers. &#10;Thirdly, local and global features are combined using different loss functions.&#10;Experiments on the Market-1501 \cite{market1501} dataset -- a large-scale person dataset most widely used for the PReID task -- show &#10;the effectiveness of the presented approach.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Computer Science, Person Re-identification, Multi-resolution Networks" />
-        </attvalues>
-      </node>
-      <node id="1805.08805" label="1805.08805">
-        <attvalues>
-          <attvalue for="0" value="Resource Aware Person Re-identification across Multiple Resolutions" />
-          <attvalue for="1" value="  Not all people are equally easy to identify: color statistics might be enough&#10;for some cases while others might require careful reasoning about high- and&#10;low-level details. However, prevailing person re-identification(re-ID) methods&#10;use one-size-fits-all high-level embeddings from deep convolutional networks&#10;for all cases. This might limit their accuracy on difficult examples or makes&#10;them needlessly expensive for the easy ones. To remedy this, we present a new&#10;person re-ID model that combines effective embeddings built on multiple&#10;convolutional network layers, trained with deep-supervision. On traditional&#10;re-ID benchmarks, our method improves substantially over the previous&#10;state-of-the-art results on all five datasets that we evaluate on. We then&#10;propose two new formulations of the person re-ID problem under&#10;resource-constraints, and show how our model can be used to effectively trade&#10;off accuracy and computation in the presence of resource constraints. Code and&#10;pre-trained models are available at https://github.com/mileyan/DARENet.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.11178" label="1807.11178">
-        <attvalues>
-          <attvalue for="0" value="Deep Group-shuffling Random Walk for Person Re-identification" />
-          <attvalue for="1" value="  Person re-identification aims at finding a person of interest in an image&#10;gallery by comparing the probe image of this person with all the gallery&#10;images. It is generally treated as a retrieval problem, where the affinities&#10;between the probe image and gallery images (P2G affinities) are used to rank&#10;the retrieved gallery images. However, most existing methods only consider P2G&#10;affinities but ignore the affinities between all the gallery images (G2G&#10;affinity). Some frameworks incorporated G2G affinities into the testing&#10;process, which is not end-to-end trainable for deep neural networks. In this&#10;paper, we propose a novel group-shuffling random walk network for fully&#10;utilizing the affinity information between gallery images in both the training&#10;and testing processes. The proposed approach aims at end-to-end refining the&#10;P2G affinities based on G2G affinity information with a simple yet effective&#10;matrix operation, which can be integrated into deep neural networks. Feature&#10;grouping and group shuffle are also proposed to apply rich supervisions for&#10;learning better person features. The proposed approach outperforms&#10;state-of-the-art methods on the Market-1501, CUHK03, and DukeMTMC datasets by&#10;large margins, which demonstrate the effectiveness of our approach.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1802.08122" label="1802.08122">
-        <attvalues>
-          <attvalue for="0" value="Harmonious Attention Network for Person Re-Identification" />
-          <attvalue for="1" value="  Existing person re-identification (re-id) methods either assume the&#10;availability of well-aligned person bounding box images as model input or rely&#10;on constrained attention selection mechanisms to calibrate misaligned images.&#10;They are therefore sub-optimal for re-id matching in arbitrarily aligned person&#10;images potentially with large human pose variations and unconstrained&#10;auto-detection errors. In this work, we show the advantages of jointly learning&#10;attention selection and feature representation in a Convolutional Neural&#10;Network (CNN) by maximising the complementary information of different levels&#10;of visual attention subject to re-id discriminative learning constraints.&#10;Specifically, we formulate a novel Harmonious Attention CNN (HA-CNN) model for&#10;joint learning of soft pixel attention and hard regional attention along with&#10;simultaneous optimisation of feature representations, dedicated to optimise&#10;person re-id in uncontrolled (misaligned) images. Extensive comparative&#10;evaluations validate the superiority of this new HA-CNN model for person re-id&#10;over a wide variety of state-of-the-art methods on three large-scale benchmarks&#10;including CUHK03, Market-1501, and DukeMTMC-ReID.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1903.09776" label="1903.09776">
-        <attvalues>
-          <attvalue for="0" value="Auto-ReID: Searching for a Part-aware ConvNet for Person&#10;  Re-Identification" />
-          <attvalue for="1" value="  Prevailing deep convolutional neural networks (CNNs) for person&#10;re-IDentification (reID) are usually built upon ResNet or VGG backbones, which&#10;were originally designed for classification. Because reID is different from&#10;classification, the architecture should be modified accordingly. We propose to&#10;automatically search for a CNN architecture that is specifically suitable for&#10;the reID task. There are three aspects to be tackled. First, body structural&#10;information plays an important role in reID but it is not encoded in backbones.&#10;Second, Neural Architecture Search (NAS) automates the process of architecture&#10;design without human effort, but no existing NAS methods incorporate the&#10;structure information of input images. Third, reID is essentially a retrieval&#10;task but current NAS algorithms are merely designed for classification. To&#10;solve these problems, we propose a retrieval-based search algorithm over a&#10;specifically designed reID search space, named Auto-ReID. Our Auto-ReID enables&#10;the automated approach to find an efficient and effective CNN architecture for&#10;reID. Extensive experiments demonstrate that the searched architecture achieves&#10;state-of-the-art performance while reducing 50% parameters and 53% FLOPs&#10;compared to others.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1812.06576" label="1812.06576">
-        <attvalues>
-          <attvalue for="0" value="Learning Incremental Triplet Margin for Person Re-identification" />
-          <attvalue for="1" value="  Person re-identification (ReID) aims to match people across multiple&#10;non-overlapping video cameras deployed at different locations. To address this&#10;challenging problem, many metric learning approaches have been proposed, among&#10;which triplet loss is one of the state-of-the-arts. In this work, we explore&#10;the margin between positive and negative pairs of triplets and prove that large&#10;margin is beneficial. In particular, we propose a novel multi-stage training&#10;strategy which learns incremental triplet margin and improves triplet loss&#10;effectively. Multiple levels of feature maps are exploited to make the learned&#10;features more discriminative. Besides, we introduce global hard identity&#10;searching method to sample hard identities when generating a training batch.&#10;Extensive experiments on Market-1501, CUHK03, and DukeMTMCreID show that our&#10;approach yields a performance boost and outperforms most existing&#10;state-of-the-art methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1807.00537" label="1807.00537">
-        <attvalues>
-          <attvalue for="0" value="SphereReID: Deep Hypersphere Manifold Embedding for Person&#10;  Re-Identification" />
-          <attvalue for="1" value="  Many current successful Person Re-Identification(ReID) methods train a model&#10;with the softmax loss function to classify images of different persons and&#10;obtain the feature vectors at the same time. However, the underlying feature&#10;embedding space is ignored. In this paper, we use a modified softmax function,&#10;termed Sphere Softmax, to solve the classification problem and learn a&#10;hypersphere manifold embedding simultaneously. A balanced sampling strategy is&#10;also introduced. Finally, we propose a convolutional neural network called&#10;SphereReID adopting Sphere Softmax and training a single model end-to-end with&#10;a new warming-up learning rate schedule on four challenging datasets including&#10;Market-1501, DukeMTMC-reID, CHHK-03, and CUHK-SYSU. Experimental results&#10;demonstrate that this single model outperforms the state-of-the-art methods on&#10;all four datasets without fine-tuning or re-ranking. For example, it achieves&#10;94.4% rank-1 accuracy on Market-1501 and 83.9% rank-1 accuracy on&#10;DukeMTMC-reID. The code and trained weights of our model will be released.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2103.15105" label="2103.15105">
-        <attvalues>
-          <attvalue for="0" value="Single Object Tracking through a Fast and Effective Single-Multiple&#10;  Model Convolutional Neural Network" />
-          <attvalue for="1" value="  Object tracking becomes critical especially when similar objects are present&#10;in the same area. Recent state-of-the-art (SOTA) approaches are proposed based&#10;on taking a matching network with a heavy structure to distinguish the target&#10;from other objects in the area which indeed drastically downgrades the&#10;performance of the tracker in terms of speed. Besides, several candidates are&#10;considered and processed to localize the intended object in a region of&#10;interest for each frame which is time-consuming. In this article, a special&#10;architecture is proposed based on which in contrast to the previous approaches,&#10;it is possible to identify the object location in a single shot while taking&#10;its template into account to distinguish it from the similar objects in the&#10;same area. In brief, first of all, a window containing the object with twice&#10;the target size is considered. This window is then fed into a fully&#10;convolutional neural network (CNN) to extract a region of interest (RoI) in a&#10;form of a matrix for each of the frames. In the beginning, a template of the&#10;target is also taken as the input to the CNN. Considering this RoI matrix, the&#10;next movement of the tracker is determined based on a simple and fast method.&#10;Moreover, this matrix helps to estimate the object size which is crucial when&#10;it changes over time. Despite the absence of a matching network, the presented&#10;tracker performs comparatively with the SOTA in challenging situations while&#10;having a super speed compared to them (up to $120 FPS$ on 1080ti). To&#10;investigate this claim, a comparison study is carried out on the GOT-10k&#10;dataset. Results reveal the outstanding performance of the proposed method in&#10;fulfilling the task.&#10;" />
-          <attvalue for="2" value="&#10;&#10;Object tracking is a major task to be accomplished in computer vision applications. Mostly, it is utilized to take into account the information between several frames. Regarding the importance of this subject, numerous effort has been made to investigate an appropriate approach. &#10;Some of these methods are based on detecting objects and associating unique ids to the same objects in a sequence of frames \cite{mot_lit} while the others focus on the tracking task exclusively \cite{object_tracking}. &#10;Investigating the detail of the tracking-by-detection approaches, researchers commonly intend to solve the multiple object tracking problem. In this regard, \cite{tracklets} proposes to use tracklets based on a deep associate method while utilizing a tracking-by-detection paradigm. The other approach in \cite{deep_attentive} suggests a reciprocative learning algorithm while utilizing visual attention. In \cite{adversarial_}, adversarial learning is employed to find the most robust features of the targets in a form of a mask. Single object tracking, on the other hand, is of high importance for which plenty of researches are conducted in this area \cite{Lasot_}. In \cite{sot1}, a single object tracker is presented based on fuzzy least squares support vector machine. Another method is suggested based on an online fusion of trackers \cite{sot2}. Some of the approaches are developed to deal with special problems in a tracking scenario such as occlusions \cite{robust_tracking,robust_tracking2} while others extend the tracking task to a specific application \cite{drone,aircraft,surgical_lotfi}. &#10;&#10;A wide research has been conducted to perform the tracking without detection. To start with, a meta-updater is proposed in \cite{LTMU} by which an online-update-based tracker is designed to accomplish the local tracking. This is done by the use of a cascaded LSTM module. This idea helps the long-term tracking to benefit from the perfect short-term trackers. Another method presented in \cite{Global} suggests a baseline for performing a global search (over a very large area). By this means, there is no need for an online learning or any changes in each of the scale and the trajectory. Besides, this tracker is mainly developed based on the two-stage detectors. In \cite{SPLT}, on the other hand, a tracking framework is presented based on the proposed skimming and perusal modules. The latter one stands for a module consisting of two main parts. Firstly, a bounding box regressor generates several candidates. Then, a verifier is applied to identify the optimal candidate while taking its confidence score into account to infer about the object presence. Note, all the mentioned trackers employ the similar approach of using a global search along with a verifier module responsible for recognizing the intended object. The verifier module is often based on ResNet-50 architecture and acts as a matching network. They also utilize the SiamRPN-based network \cite{siamrpn} to produce some region candidates for the subsequent tasks. &#10;On the other hand, despite the other problems related to computer vision, the object tracking task requires an online prediction of the object appearance model which should be embedded into the tracker's architecture to realize an End-to-End training. In this regard a popular approach is the Siamese paradigm \cite{Siamese1,Siamese2} where a feature template of the target is predicted regardless of the background information. In \cite{DiMP}, however, the authors suggest to consider both the target and the background appearance information to predict the required model based on defining a discriminative learning loss. Previous to this method the same backbone had been employed by \cite{atom} which is basically proposed using a ResNet-18 architecture along with an online discriminative learning. &#10;&#10;As it is seen a vital issue in object tracking is to elicit exclusive features of the target to distinguish it from both the background and the other similar objects. In this work, our focus is on presenting an appropriate architecture based on a novel approach while addressing the challenging issues mainly the presence of similar objects. &#10;In this article, the proposed object tracker takes the advantage of using a light CNN model along with four input branches to track an object while taking both its template and its size into account. Besides, it is suggested to take only a small part of the image containing the specified object. This part is then updated and displaced by the proposed method. Based on this approach, the mentioned region of interest is enough to track the object even in a long-term tracking scenario. The challenge, then, is to determine an appropriate new RoI for each of the incoming frames. Assuming an initial RoI with a size twice the object one, we present a new object tracker to accomplish the object tracking task. In brief, an object localizer utilizes the RoI as its input to elicit a matrix representing the object location. This module is a fully CNN with several input branches to process each object regarding its specific size while taking the intended object template from the first frame into account, as well. To clarify, in each processing, two of four branches are activated one of which is the template related branch, and the other is one of the three branches corresponding to the object size. Taking the object template branch helps to integrate the exclusive features of the target to distinguish it from similar objects. Fig. \ref{fig:fig0} presents some sample outputs of the proposed model trained on GOT-10k dataset. The methodology is presented in detail in the next section. Then experimental results are reported while comparing the performance of the proposed tracker with state-of-the-art approaches. Finally, the last sectionconcludes the paper. &#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Machine Learning, Mathematics, Artificial Intelligence, Object Tracking, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="1409.7618" label="1409.7618">
-        <attvalues>
-          <attvalue for="0" value="Multiple Object Tracking: A Literature Review" />
-          <attvalue for="1" value="  Multiple Object Tracking (MOT) has gained increasing attention due to its&#10;academic and commercial potential. Although different approaches have been&#10;proposed to tackle this problem, it still remains challenging due to factors&#10;like abrupt appearance changes and severe object occlusions. In this work, we&#10;contribute the first comprehensive and most recent review on this problem. We&#10;inspect the recent advances in various aspects and propose some interesting&#10;directions for future research. To the best of our knowledge, there has not&#10;been any extensive review on this topic in the community. We endeavor to&#10;provide a thorough review on the development of this problem in recent decades.&#10;The main contributions of this review are fourfold: 1) Key aspects in an MOT&#10;system, including formulation, categorization, key principles, evaluation of&#10;MOT are discussed; 2) Instead of enumerating individual works, we discuss&#10;existing approaches according to various aspects, in each of which methods are&#10;divided into different groups and each group is discussed in detail for the&#10;principles, advances and drawbacks; 3) We examine experiments of existing&#10;publications and summarize results on popular datasets to provide quantitative&#10;and comprehensive comparisons. By analyzing the results from different&#10;perspectives, we have verified some basic agreements in the field; and 4) We&#10;provide a discussion about issues of MOT research, as well as some interesting&#10;directions which will become potential research effort in the future.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1810.03851" label="1810.03851">
-        <attvalues>
-          <attvalue for="0" value="Deep Attentive Tracking via Reciprocative Learning" />
-          <attvalue for="1" value="  Visual attention, derived from cognitive neuroscience, facilitates human&#10;perception on the most pertinent subset of the sensory data. Recently,&#10;significant efforts have been made to exploit attention schemes to advance&#10;computer vision systems. For visual tracking, it is often challenging to track&#10;target objects undergoing large appearance changes. Attention maps facilitate&#10;visual tracking by selectively paying attention to temporal robust features.&#10;Existing tracking-by-detection approaches mainly use additional attention&#10;modules to generate feature weights as the classifiers are not equipped with&#10;such mechanisms. In this paper, we propose a reciprocative learning algorithm&#10;to exploit visual attention for training deep classifiers. The proposed&#10;algorithm consists of feed-forward and backward operations to generate&#10;attention maps, which serve as regularization terms coupled with the original&#10;classification loss function for training. The deep classifier learns to attend&#10;to the regions of target objects robust to appearance changes. Extensive&#10;experiments on large-scale benchmark datasets show that the proposed attentive&#10;tracking method performs favorably against the state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.04273" label="1804.04273">
-        <attvalues>
-          <attvalue for="0" value="VITAL: VIsual Tracking via Adversarial Learning" />
-          <attvalue for="1" value="  The tracking-by-detection framework consists of two stages, i.e., drawing&#10;samples around the target object in the first stage and classifying each sample&#10;as the target object or as background in the second stage. The performance of&#10;existing trackers using deep classification networks is limited by two aspects.&#10;First, the positive samples in each frame are highly spatially overlapped, and&#10;they fail to capture rich appearance variations. Second, there exists extreme&#10;class imbalance between positive and negative samples. This paper presents the&#10;VITAL algorithm to address these two problems via adversarial learning. To&#10;augment positive samples, we use a generative network to randomly generate&#10;masks, which are applied to adaptively dropout input features to capture a&#10;variety of appearance changes. With the use of adversarial learning, our&#10;network identifies the mask that maintains the most robust features of the&#10;target objects over a long temporal span. In addition, to handle the issue of&#10;class imbalance, we propose a high-order cost sensitive loss to decrease the&#10;effect of easy negative samples to facilitate training the classification&#10;network. Extensive experiments on benchmark datasets demonstrate that the&#10;proposed tracker performs favorably against state-of-the-art approaches.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2004.00305" label="2004.00305">
-        <attvalues>
-          <attvalue for="0" value="High-Performance Long-Term Tracking with Meta-Updater" />
-          <attvalue for="1" value="  Long-term visual tracking has drawn increasing attention because it is much&#10;closer to practical applications than short-term tracking. Most top-ranked&#10;long-term trackers adopt the offline-trained Siamese architectures, thus, they&#10;cannot benefit from great progress of short-term trackers with online update.&#10;However, it is quite risky to straightforwardly introduce online-update-based&#10;trackers to solve the long-term problem, due to long-term uncertain and noisy&#10;observations. In this work, we propose a novel offline-trained Meta-Updater to&#10;address an important but unsolved problem: Is the tracker ready for updating in&#10;the current frame? The proposed meta-updater can effectively integrate&#10;geometric, discriminative, and appearance cues in a sequential manner, and then&#10;mine the sequential information with a designed cascaded LSTM module. Our&#10;meta-updater learns a binary output to guide the tracker's update and can be&#10;easily embedded into different trackers. This work also introduces a long-term&#10;tracking framework consisting of an online local tracker, an online verifier, a&#10;SiamRPN-based re-detector, and our meta-updater. Numerous experimental results&#10;on the VOT2018LT, VOT2019LT, OxUvALT, TLP, and LaSOT benchmarks show that our&#10;tracker performs remarkably better than other competing algorithms. Our project&#10;is available on the website: https://github.com/Daikenan/LTMU.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1809.04320" label="1809.04320">
-        <attvalues>
-          <attvalue for="0" value="Learning regression and verification networks for long-term visual&#10;  tracking" />
-          <attvalue for="1" value="  Compared with short-term tracking, the long-term tracking task requires&#10;determining the tracked object is present or absent, and then estimating the&#10;accurate bounding box if present or conducting image-wide re-detection if&#10;absent. Until now, few attempts have been done although this task is much&#10;closer to designing practical tracking systems. In this work, we propose a&#10;novel long-term tracking framework based on deep regression and verification&#10;networks. The offline-trained regression model is designed using the&#10;object-aware feature fusion and region proposal networks to generate a series&#10;of candidates and estimate their similarity scores effectively. The&#10;verification network evaluates these candidates to output the optimal one as&#10;the tracked object with its classification score, which is online updated to&#10;adapt to the appearance variations based on newly reliable observations. The&#10;similarity and classification scores are combined to obtain a final confidence&#10;value, based on which our tracker can determine the absence of the target&#10;accurately and conduct image-wide re-detection to capture the target&#10;successfully when it reappears. Extensive experiments show that our tracker&#10;achieves the best performance on the VOT2018 long-term challenge and&#10;state-of-the-art results on the OxUvA long-term dataset.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1904.07220" label="1904.07220">
-        <attvalues>
-          <attvalue for="0" value="Learning Discriminative Model Prediction for Tracking" />
-          <attvalue for="1" value="  The current strive towards end-to-end trainable computer vision systems&#10;imposes major challenges for the task of visual tracking. In contrast to most&#10;other vision problems, tracking requires the learning of a robust&#10;target-specific appearance model online, during the inference stage. To be&#10;end-to-end trainable, the online learning of the target model thus needs to be&#10;embedded in the tracking architecture itself. Due to the imposed challenges,&#10;the popular Siamese paradigm simply predicts a target feature template, while&#10;ignoring the background appearance information during inference. Consequently,&#10;the predicted model possesses limited target-background discriminability.&#10;  We develop an end-to-end tracking architecture, capable of fully exploiting&#10;both target and background appearance information for target model prediction.&#10;Our architecture is derived from a discriminative learning loss by designing a&#10;dedicated optimization process that is capable of predicting a powerful model&#10;in only a few iterations. Furthermore, our approach is able to learn key&#10;aspects of the discriminative loss itself. The proposed tracker sets a new&#10;state-of-the-art on 6 tracking benchmarks, achieving an EAO score of 0.440 on&#10;VOT2018, while running at over 40 FPS. The code and models are available at&#10;https://github.com/visionml/pytracking.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1811.07628" label="1811.07628">
-        <attvalues>
-          <attvalue for="0" value="ATOM: Accurate Tracking by Overlap Maximization" />
-          <attvalue for="1" value="  While recent years have witnessed astonishing improvements in visual tracking&#10;robustness, the advancements in tracking accuracy have been limited. As the&#10;focus has been directed towards the development of powerful classifiers, the&#10;problem of accurate target state estimation has been largely overlooked. In&#10;fact, most trackers resort to a simple multi-scale search in order to estimate&#10;the target bounding box. We argue that this approach is fundamentally limited&#10;since target estimation is a complex task, requiring high-level knowledge about&#10;the object.&#10;  We address this problem by proposing a novel tracking architecture,&#10;consisting of dedicated target estimation and classification components. High&#10;level knowledge is incorporated into the target estimation through extensive&#10;offline learning. Our target estimation component is trained to predict the&#10;overlap between the target object and an estimated bounding box. By carefully&#10;integrating target-specific information, our approach achieves previously&#10;unseen bounding box accuracy. We further introduce a classification component&#10;that is trained online to guarantee high discriminative power in the presence&#10;of distractors. Our final tracking framework sets a new state-of-the-art on&#10;five challenging benchmarks. On the new large-scale TrackingNet dataset, our&#10;tracker ATOM achieves a relative gain of 15% over the previous best approach,&#10;while running at over 30 FPS. Code and models are available at&#10;https://github.com/visionml/pytracking.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1908.00669" label="1908.00669">
-        <attvalues>
-          <attvalue for="0" value="Recognizing Image Objects by Relational Analysis Using Heterogeneous&#10;  Superpixels and Deep Convolutional Features" />
-          <attvalue for="1" value="  Superpixel-based methodologies have become increasingly popular in computer&#10;vision, especially when the computation is too expensive in time or memory to&#10;perform with a large number of pixels or features. However, rarely is&#10;superpixel segmentation examined within the context of deep convolutional&#10;neural network architectures. This paper presents a novel neural architecture&#10;that exploits the superpixel feature space. The visual feature space is&#10;organized using superpixels to provide the neural network with a substructure&#10;of the images. As the superpixels associate the visual feature space with parts&#10;of the objects in an image, the visual feature space is transformed into a&#10;structured vector representation per superpixel. It is shown that it is&#10;feasible to learn superpixel features using capsules and it is potentially&#10;beneficial to perform image analysis in such a structured manner. This novel&#10;deep learning architecture is examined in the context of an image&#10;classification task, highlighting explicit interpretability (explainability) of&#10;the network's decision making. The results are compared against a baseline deep&#10;neural model, as well as among superpixel capsule networks with a variety of&#10;hyperparameter settings.&#10;" />
-          <attvalue for="2" value="&#10;&#10;The landscape of computer vision has drastically changed since the resurgence of neural networks.&#10;Neural networks, now popularized as deep learning models, have achieved state-of-the-art results in areas such as object recognition and localization \cite{ILSVRC, COCO}, semantic segmentation \cite{FULLYCONV, SEMSEG}, RGB-D \cite{DCNF} and multiple view data analytics, etc. &#10;However, most deep learning methodologies usually require very high data processing bandwidth, and part of their overwhelming success can be attributed to the availability of large-scale image datasets \cite{ILSVRC, DL3D}. &#10;These deep learning models also create a large volume of features from their feature extraction process. &#10;The problem is that these models do not posses the following, regarding their generated feature space:&#10;(i) an intelligent methodology to organize feature maps in more meaningful and comprehensible way;&#10;(ii) a strong relationship between features and the spatial components of the observed samples. &#10;One of the forerunners \cite{DEFORM}, who is actively approaching these areas of concern, is the ideology of superpixel-enabled relational analysis. &#10;&#10;As Fig. \ref{fig:sp-entropy} shows, most superpixel models are confined to overly segmented superpixels associated with low local entropy and low information loss upon homogenization.&#10;Contrary to popular belief, one can also see the value of superpixel usage in the slighly higher entropy range, which affords the opportunity to establish a relationship between features and their spatial properties.&#10;The data points in Fig. \ref{fig:sp-entropy} are also listed in Tab. \ref{tab:sp-entropy}, which serve as an empirical evidence of the following observations:&#10;(i) Compared to a convolutional operation, superpixels induce a mechanism more efficient in orders of magnitudes to reduce mean local entropy, which makes the superpixel feature space highly organized.&#10;(ii) The de-facto popular usage of superpixel as been dimensionality reduction through oversegmentation, whereas deep leaning models without superpixels may use techniques as drastic as global pooling which would be an extreme undersegmentation.&#10;This means the homogeneity of superpixels alone shall not sufficiently determine the performance of a superpixel based model.&#10;(iii) Empirically, the number of superpixels and the mean local entropy exhibit a strong anticorrelation (as demonstrated in Fig. \ref{fig:sp-entropy}) across a board range (from 1 to 10,000) of number of superpixels.&#10;These are the observations that give rise to the model that is proposed herein, &#10;and the reasons that heterogeneous or high-entropy superpixel usage deserves to be classed as an entirely different type of model,&#10;which is further discuss in Sect. \ref{sect:sp-feature}.&#10;&#10;Prior researches \cite{DCNF} \cite{SPSEG} in the field of deep learning have used superpixels mainly to reduce the computational intensity.&#10;Superpixel pooling is one of the primary recognized techniques of dimensionality reduction for deep convolutional networks.&#10;This method allows perceptually homogeneous regions of pixels to be grouped together and represented more concisely \cite{SLIC},&#10;and then aggregated according to such regions. &#10;As reported in the literature, such as \cite{DCNF, SPCNN}, it is feasible to adapt superpixel features to many deep convolutional neural network (DCNN) architectures to extract features from convolutional maps,&#10;making a potentially very expensive computation feasible.&#10;More interestingly, however, in some literature it is reported that not only have superpixels made the computation more efficient, but there has been model performance benefits as well.&#10;For example, in the work of Mostajabi, et al. \cite{SPSEMSEG} superpixels were used to exploit statistical structure in the image and in the label space and to avoid complex and expensive inference.&#10;More recently, in the work of Wu, et al. \cite{SPCNN} superpixel based analysis is incorporated to perform highly efficient cloud image segmentation. &#10;The local consistency of the segmentation was preserved and faster DCNN learning was reported.&#10;&#10;Consequently, with respect to the above, superpixels can also provide an alternative organization of the convolutional features. &#10;Herein, superpixels are utilized to reorganize the feature space by associating convolutional features to discrete parts of objects within images. &#10;Afterwards, relational analysis is explored, in the form of capsule routing, to fuse structurally organized convolutional features. &#10;Relational analysis in this sense, will prioritize the spatial relationship between the features while combining them into a set of model predictions \cite{CAPS}.&#10;&#10;The challenge with processing superpixel features and learning a model that makes inferences based on superpixels,&#10;is to fully exploit the superpixel features with some capable mathematical machinery; &#10;as opposed to performing naive aggregations too early and losing a significant amount of information.&#10;To this end, capsules \cite{CAPS} are experimented with to create a model that analyzes visual images by putting together pieces of superpixels and processing the superpixel vector features.&#10;As each superpixel segmentation of the image is completely different, there is no specific role assigned to the neurons handling the superpixels and this makes the superpixel feature space difficult to learn especially with high dimensionality.&#10;&#10;Related state-of-the-art research actively addressing the connection between feature space and feature spatial properties also includes Deformable ConverNets \cite{DEFORM},&#10;which has allowed the region of interest to adapt to the geometric variations of objects.&#10;Compared to the superpixel approach this geometrical adaptation is not limited to perceptually homogeneous regions and it would also relax the disjoint constraint.&#10;The similarity shared with the superpixel approach, including the proposed method, is the acknowledgment that introducing irregular regions that account for geometrical cues is considerably&#10;enhancing of the modeling capability of a deep convolutional neural network.&#10;&#10;The strength of the proposed model primarily lies in the heterogeneous superpixel usage, which has facilitated image object relational analysis,&#10;thanks to its capability to associate convolutional features to their spatial locality, and in turn, parts of objects in the image.&#10;The usage of heterogeneous superpixel does not follow the popular homogeneity assumption, but have demonstrated a preliminary success in the effort of image object recognition based on relational analysis,&#10;while most DCNNs that do not use superpixel either aggregate across the entire image or process at the pixel level, as illustrated in Fig. \ref{fig:sp-entropy} (a),&#10;and most current superpixel based DCNNs or PGMs rely on oversegmentation, as illustrated in Fig. \ref{fig:sp-entropy} (c).&#10;Although the relational analysis appears to be similar to segmentation, it is worth pointing out that it is not feasible to compare with the state-of-art segmentation DCNN such as U-Net \cite{UNET} or various other segmentation networks \cite{pal2018capsdemm} \cite{lalonde2018capsules} that rely on it, because those types of neural networks are using segmentation labels for training while the proposed model has none, instead implicitly learning through many occurrences of the image objects given non-localized class labels only.&#10;Therefore, the model would be more accurately described as multiple-instance learning from a segmentation model point of view.&#10;By computing the superpixel-wise feature contribution to the class vector, visualizations are generated that represent such part-whole relations and make it more feasible to segment the object of interest as a result.&#10;In addition, the image substructure provided by superpixels can be beneficial to other related problem domains such as localization, segmentation, and data fusion.&#10;&#10;The remainder of this paper is structured as follows.&#10;Sect. \ref{sect:sp-feature} details the mathematics and theoretical considerations of the superpixel feature extraction, followed by superpixel feature aggregation in Sect. \ref{sect:sp-agg}.&#10;Sect. \ref{sect:eval} details the evaluation framework, followed by a discussion of the dataset and novel neural architecture.&#10;Then, experimental results are discussed in Sect. \ref{sect:results}.&#10;Finally, Sect. \ref{sect:conclusion} provides concluding remarks and discusses potential future investigations.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Mathematics, Artificial Intelligence, Deep Learning, Cognitive Science, Computer Science, Image Analysis" />
-        </attvalues>
-      </node>
-      <node id="1811.11168" label="1811.11168">
-        <attvalues>
-          <attvalue for="0" value="Deformable ConvNets v2: More Deformable, Better Results" />
-          <attvalue for="1" value="  The superior performance of Deformable Convolutional Networks arises from its&#10;ability to adapt to the geometric variations of objects. Through an examination&#10;of its adaptive behavior, we observe that while the spatial support for its&#10;neural features conforms more closely than regular ConvNets to object&#10;structure, this support may nevertheless extend well beyond the region of&#10;interest, causing features to be influenced by irrelevant image content. To&#10;address this problem, we present a reformulation of Deformable ConvNets that&#10;improves its ability to focus on pertinent image regions, through increased&#10;modeling power and stronger training. The modeling power is enhanced through a&#10;more comprehensive integration of deformable convolution within the network,&#10;and by introducing a modulation mechanism that expands the scope of deformation&#10;modeling. To effectively harness this enriched modeling capability, we guide&#10;network training via a proposed feature mimicking scheme that helps the network&#10;to learn features that reflect the object focus and classification power of&#10;R-CNN features. With the proposed contributions, this new version of Deformable&#10;ConvNets yields significant performance gains over the original model and&#10;produces leading results on the COCO benchmark for object detection and&#10;instance segmentation.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1806.02705" label="1806.02705">
-        <attvalues>
-          <attvalue for="0" value="Efficient semantic image segmentation with superpixel pooling" />
-          <attvalue for="1" value="  In this work, we evaluate the use of superpixel pooling layers in deep&#10;network architectures for semantic segmentation. Superpixel pooling is a&#10;flexible and efficient replacement for other pooling strategies that&#10;incorporates spatial prior information. We propose a simple and efficient&#10;GPU-implementation of the layer and explore several designs for the integration&#10;of the layer into existing network architectures. We provide experimental&#10;results on the IBSR and Cityscapes dataset, demonstrating that superpixel&#10;pooling can be leveraged to consistently increase network accuracy with minimal&#10;computational overhead. Source code is available at&#10;https://github.com/bermanmaxim/superpixPool&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1412.0774" label="1412.0774">
-        <attvalues>
-          <attvalue for="0" value="Feedforward semantic segmentation with zoom-out features" />
-          <attvalue for="1" value="  We introduce a purely feed-forward architecture for semantic segmentation. We&#10;map small image elements (superpixels) to rich feature representations&#10;extracted from a sequence of nested regions of increasing extent. These regions&#10;are obtained by &quot;zooming out&quot; from the superpixel all the way to scene-level&#10;resolution. This approach exploits statistical structure in the image and in&#10;the label space without setting up explicit structured prediction mechanisms,&#10;and thus avoids complex and expensive inference. Instead superpixels are&#10;classified by a feedforward multilayer network. Our architecture achieves new&#10;state of the art performance in semantic segmentation, obtaining 64.4% average&#10;accuracy on the PASCAL VOC 2012 test set.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1710.09829" label="1710.09829">
-        <attvalues>
-          <attvalue for="0" value="Dynamic Routing Between Capsules" />
-          <attvalue for="1" value="  A capsule is a group of neurons whose activity vector represents the&#10;instantiation parameters of a specific type of entity such as an object or an&#10;object part. We use the length of the activity vector to represent the&#10;probability that the entity exists and its orientation to represent the&#10;instantiation parameters. Active capsules at one level make predictions, via&#10;transformation matrices, for the instantiation parameters of higher-level&#10;capsules. When multiple predictions agree, a higher level capsule becomes&#10;active. We show that a discrimininatively trained, multi-layer capsule system&#10;achieves state-of-the-art performance on MNIST and is considerably better than&#10;a convolutional net at recognizing highly overlapping digits. To achieve these&#10;results we use an iterative routing-by-agreement mechanism: A lower-level&#10;capsule prefers to send its output to higher level capsules whose activity&#10;vectors have a big scalar product with the prediction coming from the&#10;lower-level capsule.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1804.04241" label="1804.04241">
-        <attvalues>
-          <attvalue for="0" value="Capsules for Object Segmentation" />
-          <attvalue for="1" value="  Convolutional neural networks (CNNs) have shown remarkable results over the&#10;last several years for a wide range of computer vision tasks. A new&#10;architecture recently introduced by Sabour et al., referred to as a capsule&#10;networks with dynamic routing, has shown great initial results for digit&#10;recognition and small image classification. The success of capsule networks&#10;lies in their ability to preserve more information about the input by replacing&#10;max-pooling layers with convolutional strides and dynamic routing, allowing for&#10;preservation of part-whole relationships in the data. This preservation of the&#10;input is demonstrated by reconstructing the input from the output capsule&#10;vectors. Our work expands the use of capsule networks to the task of object&#10;segmentation for the first time in the literature. We extend the idea of&#10;convolutional capsules with locally-connected routing and propose the concept&#10;of deconvolutional capsules. Further, we extend the masked reconstruction to&#10;reconstruct the positive input class. The proposed&#10;convolutional-deconvolutional capsule network, called SegCaps, shows strong&#10;results for the task of object segmentation with substantial decrease in&#10;parameter space. As an example application, we applied the proposed SegCaps to&#10;segment pathological lungs from low dose CT scans and compared its accuracy and&#10;efficiency with other U-Net-based architectures. SegCaps is able to handle&#10;large image sizes (512 x 512) as opposed to baseline capsules (typically less&#10;than 32 x 32). The proposed SegCaps reduced the number of parameters of U-Net&#10;architecture by 95.4% while still providing a better segmentation accuracy.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2102.03558" label="2102.03558">
-        <attvalues>
-          <attvalue for="0" value="SM+: Refined Scale Match for Tiny Person Detection" />
-          <attvalue for="1" value="  Detecting tiny objects ( e.g., less than 20 x 20 pixels) in large-scale&#10;images is an important yet open problem. Modern CNN-based detectors are&#10;challenged by the scale mismatch between the dataset for network pre-training&#10;and the target dataset for detector training. In this paper, we investigate the&#10;scale alignment between pre-training and target datasets, and propose a new&#10;refined Scale Match method (termed SM+) for tiny person detection. SM+ improves&#10;the scale match from image level to instance level, and effectively promotes&#10;the similarity between pre-training and target dataset. Moreover, considering&#10;SM+ possibly destroys the image structure, a new probabilistic structure&#10;inpainting (PSI) method is proposed for the background processing. Experiments&#10;conducted across various detectors show that SM+ noticeably improves the&#10;performance on TinyPerson, and outperforms the state-of-the-art detectors with&#10;a significant margin.&#10;" />
-          <attvalue for="2" value="&#10;Person detection is an important topic in the computer vision area. It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc. The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs). However, the detectors perform poorly when detecting tiny objects with few pixels (e.g., less than $20\times20$ pixels), such as traffic signs \cite{Lu2018TrafficSD}, persons in aerial images \cite{Yu2020ScaleMF}, etc.&#10;&#10;To better exploit the CNN-based detectors, a large number of person datasets \cite{Everingham2009ThePV}\cite{lin2014microsoft}\cite{Kuznetsova_2020} for detection with human manual annotations have been proposed and made publicly available. However, datasets for specific object detection, such as tiny person detection \cite{Yu2020ScaleMF}, are not as large as other counterparts \cite{deng2009imagenet}\cite{lin2014microsoft}, due to the cost of collecting and annotating the data. With the insufficient data for a specific application, an alternative way is to pre-train a model on the extra-large datasets (e.g., ImageNet \cite{deng2009imagenet}, COCO \cite{lin2014microsoft}), and then fine-tune the model on a task-specific dataset.&#10;&#10;However, a new question arises: Could we take better advantage of existing large datasets for a task-specific application, particularly when object sizes significantly differ between the datasets? SM algorithm\cite{Yu2020ScaleMF}, Random Scale Match (RSM) and Monotone Scale Match (MSM), gave simple yet effective ways. With a sampled scale factor, the SM algorithm directly resizes the images and aligns the scale distribution of pre-training dataset to that of the target dataset.&#10;The SM algorithm, with image-level scaling, is merely a simple approximation for scale match by simply regarding the average size of all objects in an image as the size of the image, where there may be many labeled objects with multi-scales. &#10;&#10;In this paper, we propose a newly refined SM method (SM+), in which we transform the scale distribution of pre-training dataset by instance-level scaling instead of resizing the whole image. Intuitively, compared with the vanilla SM algorithm, SM+ is a finer-scale scaling and alleviates the uncertainty and inaccuracy caused by the approximation introduced in the SM algorithm. The differences between SM and SM+ are illustrated in Fig. \ref{fig:different}. SM+ algorithm separates the image into two parts: the annotated instances and their background. The instances are utilized for the instance-level scale match, while the images are destroyed by some instance-shaped holes (background). However, the traditional method \cite{Bertalmo2001NavierstokesFD} blurs the images by direct inpainting the holes and generates some unreal images, leading to the performance drop of the pre-trained model. To solve this problem, a probabilistic structure inpainting (PSI) method is further proposed to dynamically inpaint the images by suppressing the image blur and preserving context consistency around the holes. Compared with state-of-the-art detectors on TinyPerson, SM+ algorithm leads to significant performance improvement in AP. The main contributions of our work include:&#10;&#10;1. We comprehensively analyze the scale information of TinyPerson, and propose a new refined scale match method, dubbed as SM+, which achieves better scale distribution alignment by finer-scale scaling.&#10;&#10;2. We propose the probabilistic structure inpainting for the SM+ algorithm. PSI can effectively inpaint the images.&#10;&#10;3. The proposed SM+ algorithm improves the detection performance over the state-of-the-art detectors with a large margin. Codes will be available upon acceptance.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Machine Learning, Engineering, Computer Vision, Mathematics, Artificial Intelligence, Object Detection, Image Processing, Deep Learning, Computer Science" />
-        </attvalues>
-      </node>
-      <node id="2006.05196" label="2006.05196">
-        <attvalues>
-          <attvalue for="0" value="Multi-spectral Facial Landmark Detection" />
-          <attvalue for="1" value="  Thermal face image analysis is favorable for certain circumstances. For&#10;example, illumination-sensitive applications, like nighttime surveillance; and&#10;privacy-preserving demanded access control. However, the inadequate study on&#10;thermal face image analysis calls for attention in responding to the industry&#10;requirements. Detecting facial landmark points are important for many face&#10;analysis tasks, such as face recognition, 3D face reconstruction, and face&#10;expression recognition. In this paper, we propose a robust neural network&#10;enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL).&#10;Briefly, DMSL consists of two sub-models, i.e. face boundary detection, and&#10;landmark coordinates detection. Such an architecture demonstrates the&#10;capability of detecting the facial landmarks on both visible and thermal&#10;images. Particularly, the proposed DMSL model is robust in facial landmark&#10;detection where the face is partially occluded, or facing different directions.&#10;The experiment conducted on Eurecom's visible and thermal paired database shows&#10;the superior performance of DMSL over the state-of-the-art for thermal facial&#10;landmark detection. In addition to that, we have annotated a thermal face&#10;dataset with their respective facial landmark for the purpose of&#10;experimentation.&#10;" />
-          <attvalue for="2" value="&#10;&#10;In recent years, infrared spectrum (IR) imagery has been utilized to facilitate the development of computer vision (CV) applications mainly due to its light-insensitive nature. For instance, IR camera can capture images under unfavorable illumination situations, hence it can be used at night or even under completely dark conditions. Most conventional IR imagery today uses an active approach which senses reflected electromagnetic radiation (EMR) within the Near infrared (NIR) or Short-Wave infrared (SWIR) spectrum. This approach relies on the external IR illumination \cite{maldague2002introduction}. As a result, it can be deteriorated by different illumination conditions. Thermal, a.k.a Long-Wave infrared (\SI{7}{\micro\metre} - \SI{14}{\micro\metre} wavelength), is regarded as a promising direction to extend imaging technology \cite{ghiass2014infrared}. Thermal imagery is a passive IR technology that primarily captures the emitted EMR specifically the heat energy from the object, hence thermal spectrum is insensitive to illumination variations. However, it can be affected by ambience temperature.&#10;&#10;Bourlai and Cukik \cite{bourlai2012multi} presented some scenarios which require images captured from multiple electromagnetic spectra to perform facial recognition. This suggests that multi-spectral imagery technology can be employed to tackle some CV problems that involve working under a variety of illumination conditions. Chang et al. \cite{chang2008multispectral} showed that the images from multiple spectra can be fused together to perform face recognition under poor illumination conditions. They also presented a way to fuse visible and thermal face images. There are some scenarios where the visible and thermal imagery can work in conjunction with each other for a more robust performance than a single spectral system. For instance, in real time facial image analysis, we can have a dual camera setup equipped with visible and thermal cameras such as (FLIR Duo Pro R 640), this camera has a thermal sensor as well as visible light sensor designed for camera drone. Depending on the illumination conditions, we can switch in between both configurations, or combine the input to perform a seamless analysis.&#10;&#10;One of the main prerequisites of facial image analysis is facial landmark detection as it plays a key role in many tasks, such as automatic face recognition \cite{wiskott1997face} \cite{husken2005strategies}, expression recognition \cite{jung2015joint}, 3D face reconstruction \cite{choi20103d}. Many facial landmark detectors have been designed for face images in the visible spectrum and they are proved to be very reliable. However, these detectors fail to achieve good performance for thermal images \cite{Chu2019}. To achieve multi-spectral analysis, there is a need for an efficient thermal facial landmark detector as well. In recent years, deep learning methods have been successfully applied to many areas such as image recognition due to their automatic learning capabilities, and have brought significant improvements in these areas. Therefore, this paper focuses on the thermal facial landmark detection based on deep learning methods.&#10;&#10;Due to the expensive capture devices and limited dataset, there are only few works on thermal facial landmark detection. Active Appearance Models \cite{kopaczka2016robust} is a relatively traditional method initially used to model faces given facial landmark positions \cite{cootes1998active}. This method is a handcrafted model designed to compute the mean shape and appearance metric of a given dataset and use to it to predict the landmark of an unseen image. Like most handcrafted techniques, this method does not work well with high data variability.&#10;&#10;A current trend towards thermal facial landmark detection is to apply deep learning models that are designed for detecting visible facial landmarks and adapted to the thermal image domain. Poster et al.\cite{poster2019examination} noted that the state-of-the-art visible facial landmark detection models work poorly with thermal images due to relatively low information contained in thermal images. This work shows that thermal images aligned with modern landmark detection algorithms often fail to achieve thermal-to-visible face verification results compared to manually aligned imagery. The models being tested in this study include Deep Alignment Network (DAN) \cite{kowalski2017deep}, Multi-task Convolutional Neural Network (MTCNN) \cite{zhang2016joint}, and Multi-class Patch-based fully convolutional neural network (PBC) \cite{poster2019examination}.&#10;&#10;Another solution is to transform thermal images into visible images, and then to detect the landmarks from the transformed images using existing landmark detectors designed for visible images \cite{Chu2019}. The transformation is achieved by developing a CycleGAN model \cite{Zhu_2017_ICCV}. This method is shown to be unreliable, for example, CycleGAN mode collapse. This solution also offers room for inconsistency as it involves transforming a low information image into a higher information domain. To tackle this problem, an end-to-end landmark detection model based on two-stage training mechanism is proposed in \cite{Chu2019}. The first stage of this solution is to train a U-Net \cite{Ronneberger2015} to outline the face of a given image. A fully connected network is attached to the U-Net at the second stage to estimate facial landmark coordinates from the outline. An auxiliary output layer is used to detect the facial expression to enhance the training outcome. This solution however, are only able to detect front facing faces with no occlusion. Even self occlusion from slight face tilting to different direction results in landmark distortion.&#10;&#10;Based on the aforementioned discussion, thermal facial landmark detection is still a challenging task. In this paper, we proposed a Deep Multi-Spectral Learning (DMSL) model for facial landmark detection. The proposed model is a stack of two independent sub-models, an auxiliary model for detecting face boundary, and a main model for detecting facial landmarks. DMSL is a unified model that can detect facial landmarks from both thermal and visible images, this leads to efficient and convenient solution for multi-spectral facial image analysis. Another core strength is that this model is robust for thermal facial landmark detection when a face is not front facing or when there is occlusion. Additionally, we annotated a thermal image dataset by a 68 points landmark configuration. This dataset is used for the experimentation of our proposed deep learning model based on U-Net to detect facial landmarks on thermal images. &#10;&#10;The rest of this paper is organized as follows. In Section 2, we present the proposed model architecture and pipeline. In section3, we describe the dataset used for experimentation. The steps for pre-processing, and dataset annotation, are outlined in section4. In section5, we describe the metrics used to evaluate our model. In section6, we explain the experiment process in detail as well as illustrate the performance and effectiveness of the model. Finally we conclude with a discussion in section7.&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Facial Landmark Detection, Artificial Intelligence, Biometrics, Physics, Computer Science, Neural Network Architecture, Face Image Analysis" />
-        </attvalues>
-      </node>
-      <node id="1401.8261" label="1401.8261">
-        <attvalues>
-          <attvalue for="0" value="Infrared face recognition: a comprehensive review of methodologies and&#10;  databases" />
-          <attvalue for="1" value="  Automatic face recognition is an area with immense practical potential which&#10;includes a wide range of commercial and law enforcement applications. Hence it&#10;is unsurprising that it continues to be one of the most active research areas&#10;of computer vision. Even after over three decades of intense research, the&#10;state-of-the-art in face recognition continues to improve, benefitting from&#10;advances in a range of different research fields such as image processing,&#10;pattern recognition, computer graphics, and physiology. Systems based on&#10;visible spectrum images, the most researched face recognition modality, have&#10;reached a significant level of maturity with some practical success. However,&#10;they continue to face challenges in the presence of illumination, pose and&#10;expression changes, as well as facial disguises, all of which can significantly&#10;decrease recognition accuracy. Amongst various approaches which have been&#10;proposed in an attempt to overcome these limitations, the use of infrared (IR)&#10;imaging has emerged as a particularly promising research direction. This paper&#10;presents a comprehensive and timely review of the literature on this subject.&#10;Our key contributions are: (i) a summary of the inherent properties of infrared&#10;imaging which makes this modality promising in the context of face recognition,&#10;(ii) a systematic review of the most influential approaches, with a focus on&#10;emerging common trends as well as key differences between alternative&#10;methodologies, (iii) a description of the main databases of infrared facial&#10;images available to the researcher, and lastly (iv) a discussion of the most&#10;promising avenues for future research.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="1706.01789" label="1706.01789">
-        <attvalues>
-          <attvalue for="0" value="Deep Alignment Network: A convolutional neural network for robust face&#10;  alignment" />
-          <attvalue for="1" value="  In this paper, we propose Deep Alignment Network (DAN), a robust face&#10;alignment method based on a deep neural network architecture. DAN consists of&#10;multiple stages, where each stage improves the locations of the facial&#10;landmarks estimated by the previous stage. Our method uses entire face images&#10;at all stages, contrary to the recently proposed face alignment methods that&#10;rely on local patches. This is possible thanks to the use of landmark heatmaps&#10;which provide visual information about landmark locations estimated at the&#10;previous stages of the algorithm. The use of entire face images rather than&#10;patches allows DAN to handle face images with large variation in head pose and&#10;difficult initializations. An extensive evaluation on two publicly available&#10;datasets shows that DAN reduces the state-of-the-art failure rate by up to 70%.&#10;Our method has also been submitted for evaluation as part of the Menpo&#10;challenge.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2208.14039" label="2208.14039">
-        <attvalues>
-          <attvalue for="0" value="CAIR: Fast and Lightweight Multi-Scale Color Attention Network for&#10;  Instagram Filter Removal" />
-          <attvalue for="1" value="  Image restoration is an important and challenging task in computer vision.&#10;Reverting a filtered image to its original image is helpful in various computer&#10;vision tasks. We employ a nonlinear activation function free network (NAFNet)&#10;for a fast and lightweight model and add a color attention module that extracts&#10;useful color information for better accuracy. We propose an accurate, fast,&#10;lightweight network with multi-scale and color attention for Instagram filter&#10;removal (CAIR). Experiment results show that the proposed CAIR outperforms&#10;existing Instagram filter removal networks in fast and lightweight ways, about&#10;11$\times$ faster and 2.4$\times$ lighter while exceeding 3.69 dB PSNR on IFFI&#10;dataset. CAIR can successfully remove the Instagram filter with high quality&#10;and restore color information in qualitative results. The source code and&#10;pretrained weights are available at \url{https://github.com/HnV-Lab/CAIR}.&#10;" />
-          <attvalue for="2" value="&#10;Photographic filters have been widely used to control the feel of photos when using analog and digital cameras. Filter as a camera accessory can be attached to the optical lens, which gives various effects to the photo (i.e., image), such as color conversion, color subtraction, and contrast enhancement. The popularity of smartphones has made it easier for people to take and share photos than in the days of analog and digital cameras. It has become a daily life for many people to share images on Instagram after applying various photographic effects and corrections to an image taken. These filters can digitally modify the original image (without a camera accessory) by adjusting contrast, hue, or saturation and applying blur or noise. Filters applied to images can make people feel emotional and change the photo's mood, as shown in Fig. \ref{fig:filter}. &#10;&#10;However, filtered images can be noise data that distracts training deep learning models from perspective of computer vision researchers. Computer vision requires a pure image state with no filters. Applying various filtering effects to raw images for data augmentation techniques can improve the generalizability of a model, but filtered images significantly degrade performance in major computer vision tasks~\cite{bianco2017artistic,hendrycks2019benchmarking} such as image classification~\cite{chen2015filter,wu2020recognizing}, detection, and segmentation~\cite{kinli2021instagram}. Therefore, filters (e.g., Instagram filter) need to be removed from filter images for a machine vision system to achieve non-degraded performance.&#10;&#10;Previous studies have employed the encoder-decoder structure and adversarial training strategy to solve the filter removal problem \cite{isola2017image}. Instagram filter removal network (IFRNet) \cite{kinli2021instagram} defines the problem of removing filters from photos as a reverse style transfer \cite{huang2017arbitrary}. IFRNet consists of an encoder with VGGNet \cite{simonyan2014very} as a style extractor module and a decoder inspired by PatchGAN \cite{isola2017image}, regarding filter as additional style information. The visual effects of a filter can be directly removed by adaptively normalizing external style information in each level of the encoder \cite{kinli2021instagram}. Contrastive Instagram Filter Removal Network (CIFR)~\cite{kinli2022patch} tackles the problem by normalizing the affine parameters using VGGNet with the help of adaptive normalization. Kinli et al.~\cite{kinli2022patch} employ a multi-layer patch-wise contrastive learning strategy \cite{park2020contrastive} and propose an isolated patch sampling module that distills the content and style information \cite{kinli2022patch}. These studies showed the outstanding performance of visual understanding and removing filters; however, their structure and learning process is exceedingly complex. Thus, we desire to address the problem of filter removal efficiently with a straightforward approach.&#10;&#10;We propose a multi-scale color attention network based on NAFNet (nonlinear activation function free network)~\cite{chen2022simple} for removing Instagram filters. The color attention modules explore different color-space between an original (non-filtered) image and a filtered image. First, we modify the architecture of NAFNet with less computational costs than the original NAFNet. Second, a color attention module inspired by~\cite{zamir2020cycleisp} is proposed to utilize the color-informative features from multi-scale input images. Third, the ensemble learning strategy is adopted to improve the performance of the results further. The proposed model is accurate, fast, and lightweight. Moreover, we consider carbon emission reduction through the lightweight model. The main contributions of this work are as follows:&#10;&#10;\begin{enumerate}&#10; \item We propose an accurate, fast, and lightweight multi-scale color attention network for Instagram filter removal named CAIR, which has the remarkable qualitative and quantitative ability for Instagram filter removal.&#10; \item A color attention module is proposed that captures color-informative features from multi-scale input images, resulting in better filter removal performance.&#10; \item CAIR is a lightweight model in terms of the number of operations and parameters compared to the previous studies on filter removal.&#10; \item CAIR's low computational complexity improves inference speed. CAIR achieves about 11x faster runtime compared to~\cite{kinli2021instagram,kinli2022patch} without performance penalty.&#10;\end{enumerate}&#10;&#10;" />
-          <attvalue for="3" value="" />
-          <attvalue for="4" value="Computer Vision, Engineering, Mathematics, Artificial Intelligence, Deep Learning, Computer Science, Image Restoration, Signal Processing" />
-        </attvalues>
-      </node>
-      <node id="1912.13000" label="1912.13000">
-        <attvalues>
-          <attvalue for="0" value="Recognizing Instagram Filtered Images with Feature De-stylization" />
-          <attvalue for="1" value="  Deep neural networks have been shown to suffer from poor generalization when&#10;small perturbations are added (like Gaussian noise), yet little work has been&#10;done to evaluate their robustness to more natural image transformations like&#10;photo filters. This paper presents a study on how popular pretrained models are&#10;affected by commonly used Instagram filters. To this end, we introduce&#10;ImageNet-Instagram, a filtered version of ImageNet, where 20 popular Instagram&#10;filters are applied to each image in ImageNet. Our analysis suggests that&#10;simple structure preserving filters which only alter the global appearance of&#10;an image can lead to large differences in the convolutional feature space. To&#10;improve generalization, we introduce a lightweight de-stylization module that&#10;predicts parameters used for scaling and shifting feature maps to &quot;undo&quot; the&#10;changes incurred by filters, inverting the process of style transfer tasks. We&#10;further demonstrate the module can be readily plugged into modern CNN&#10;architectures together with skip connections. We conduct extensive studies on&#10;ImageNet-Instagram, and show quantitatively and qualitatively, that the&#10;proposed module, among other things, can effectively improve generalization by&#10;simply learning normalization parameters without retraining the entire network,&#10;thus recovering the alterations in the feature space caused by the filters.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2104.05072" label="2104.05072">
-        <attvalues>
-          <attvalue for="0" value="Instagram Filter Removal on Fashionable Images" />
-          <attvalue for="1" value="  Social media images are generally transformed by filtering to obtain&#10;aesthetically more pleasing appearances. However, CNNs generally fail to&#10;interpret both the image and its filtered version as the same in the visual&#10;analysis of social media images. We introduce Instagram Filter Removal Network&#10;(IFRNet) to mitigate the effects of image filters for social media analysis&#10;applications. To achieve this, we assume any filter applied to an image&#10;substantially injects a piece of additional style information to it, and we&#10;consider this problem as a reverse style transfer problem. The visual effects&#10;of filtering can be directly removed by adaptively normalizing external style&#10;information in each level of the encoder. Experiments demonstrate that IFRNet&#10;outperforms all compared methods in quantitative and qualitative comparisons,&#10;and has the ability to remove the visual effects to a great extent.&#10;Additionally, we present the filter classification performance of our proposed&#10;model, and analyze the dominant color estimation on the images unfiltered by&#10;all compared methods.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2204.07486" label="2204.07486">
-        <attvalues>
-          <attvalue for="0" value="Patch-wise Contrastive Style Learning for Instagram Filter Removal" />
-          <attvalue for="1" value="  Image-level corruptions and perturbations degrade the performance of CNNs on&#10;different downstream vision tasks. Social media filters are one of the most&#10;common resources of various corruptions and perturbations for real-world visual&#10;analysis applications. The negative effects of these distractive factors can be&#10;alleviated by recovering the original images with their pure style for the&#10;inference of the downstream vision tasks. Assuming these filters substantially&#10;inject a piece of additional style information to the social media images, we&#10;can formulate the problem of recovering the original versions as a reverse&#10;style transfer problem. We introduce Contrastive Instagram Filter Removal&#10;Network (CIFR), which enhances this idea for Instagram filter removal by&#10;employing a novel multi-layer patch-wise contrastive style learning mechanism.&#10;Experiments show our proposed strategy produces better qualitative and&#10;quantitative results than the previous studies. Moreover, we present the&#10;results of our additional experiments for proposed architecture within&#10;different settings. Finally, we present the inference outputs and quantitative&#10;comparison of filtered and recovered images on localization and segmentation&#10;tasks to encourage the main motivation for this problem.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2007.15651" label="2007.15651">
-        <attvalues>
-          <attvalue for="0" value="Contrastive Learning for Unpaired Image-to-Image Translation" />
-          <attvalue for="1" value="  In image-to-image translation, each patch in the output should reflect the&#10;content of the corresponding patch in the input, independent of domain. We&#10;propose a straightforward method for doing so -- maximizing mutual information&#10;between the two, using a framework based on contrastive learning. The method&#10;encourages two elements (corresponding patches) to map to a similar point in a&#10;learned feature space, relative to other elements (other patches) in the&#10;dataset, referred to as negatives. We explore several critical design choices&#10;for making contrastive learning effective in the image synthesis setting.&#10;Notably, we use a multilayer, patch-based approach, rather than operate on&#10;entire images. Furthermore, we draw negatives from within the input image&#10;itself, rather than from the rest of the dataset. We demonstrate that our&#10;framework enables one-sided translation in the unpaired image-to-image&#10;translation setting, while improving quality and reducing training time. In&#10;addition, our method can even be extended to the training setting where each&#10;&quot;domain&quot; is only a single image.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-      <node id="2003.07761" label="2003.07761">
-        <attvalues>
-          <attvalue for="0" value="CycleISP: Real Image Restoration via Improved Data Synthesis" />
-          <attvalue for="1" value="  The availability of large-scale datasets has helped unleash the true&#10;potential of deep convolutional neural networks (CNNs). However, for the&#10;single-image denoising problem, capturing a real dataset is an unacceptably&#10;expensive and cumbersome procedure. Consequently, image denoising algorithms&#10;are mostly developed and evaluated on synthetic data that is usually generated&#10;with a widespread assumption of additive white Gaussian noise (AWGN). While the&#10;CNNs achieve impressive results on these synthetic datasets, they do not&#10;perform well when applied on real camera images, as reported in recent&#10;benchmark datasets. This is mainly because the AWGN is not adequate for&#10;modeling the real camera noise which is signal-dependent and heavily&#10;transformed by the camera imaging pipeline. In this paper, we present a&#10;framework that models camera imaging pipeline in forward and reverse&#10;directions. It allows us to produce any number of realistic image pairs for&#10;denoising both in RAW and sRGB spaces. By training a new image denoising&#10;network on realistic synthetic data, we achieve the state-of-the-art&#10;performance on real camera benchmark datasets. The parameters in our model are&#10;~5 times lesser than the previous best method for RAW denoising. Furthermore,&#10;we demonstrate that the proposed framework generalizes beyond image denoising&#10;problem e.g., for color matching in stereoscopic cinema. The source code and&#10;pre-trained models are available at https://github.com/swz30/CycleISP.&#10;" />
-          <attvalue for="2" value="" />
-          <attvalue for="3" value="" />
-        </attvalues>
-      </node>
-    </nodes>
-    <edges>
-      <edge source="2009.07506" target="1405.0312" id="0">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1506.01497" id="1">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1803.06799" id="2">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1810.04002" id="3">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1712.00726" id="4">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1712.00886" id="5">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1909.09709" id="6">
-        <attvalues>
-          <attvalue for="5" value=" With the rise of deep convolutional neural networks, research in object detection has seen unprecedented progress~\cite{lin2014microsoft,ren2015faster,cheng2018revisiting,cheng2018decoupled,cai2018cascade,shen2017improving,zhang2019skynet}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07506" target="1912.10664" id="7">
-        <attvalues>
-          <attvalue for="5" value=" The TinyPerson dataset~\cite{yu2020scale} was adopted in this challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1812.08008" id="8">
-        <attvalues>
-          <attvalue for="5" value=" Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1612.00137" id="9">
-        <attvalues>
-          <attvalue for="5" value=" Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2103.02440" id="10">
-        <attvalues>
-          <attvalue for="5" value=" Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1809.01354" id="11">
-        <attvalues>
-          <attvalue for="5" value=" Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1804.03287" id="12">
-        <attvalues>
-          <attvalue for="5" value=" Over the past two decades, the research community has focused on predicting 2D contents such as keypoints~\cite{cao2019openpose,fang2017rmpe,kreiss2021openpifpaf}, silhouettes~\cite{chen2018semantic}, and part segmentations~\cite{zhao2018understanding} from RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2008.09457" id="13">
-        <attvalues>
-          <attvalue for="5" value=" With these advances, researchers further seek to estimate human pose in 3D space~\cite{grauman2003inferring,agarwal2005recovering, martinez_2017_3dbaseline, pavlakos2017coarse, sun2018integral, mehta2020xnect,Weinzaepfel2020_dope}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1607.08128" id="14">
-        <attvalues>
-          <attvalue for="5" value=" Following the optimization-based paradigm~\cite{bogo2016keep,lassner2017unite,zanfir2018monocular}, methods explicitly fit body models to 2D observations in an iterative manner." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1712.06584" id="15">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1805.04092" id="16">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1808.05942" id="17">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2103.16507" id="18">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2104.08527" id="19">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1904.05866" id="20">
-        <attvalues>
-          <attvalue for="5" value="~\ref{fig:citations}, since the release of the SMPL model~\cite{loper2015smpl} in 2015 and the SMPL-X model~\cite{pavlakos2019expressive} in 2019, they have gained increasing interest as their annual citations grow rapidly year by year." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2108.10378" id="21">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the community has shifted its interests towards 3D mesh recovery of human bodies~\cite{bogo2016keep,huang2017towards,zanfir2018monocular,kanazawa2018end,pavlakos2018learning,omran2018neural,zhang2021pymaf,kocabas2021pare,joo2021exemplar} along with expressive face and hands~\cite{pavlakos2019expressive, choutas2020monocular, feng2021collaborative, moon2022accurate,zhang2021lightweight}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1804.06023" id="22">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2007.03858" id="23">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2105.00261" id="24">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2206.02194" id="25">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2112.09127" id="26">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2212.07422" id="27">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2012.15838" id="28">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2112.10203" id="29">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2004.04572" id="30">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2109.01137" id="31">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2203.14478" id="32">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="2305.04789" id="33">
-        <attvalues>
-          <attvalue for="5" value=" The recovery of human body meshes plays a key role in facilitating the downstream tasks such as clothed human reconstruction~\cite{yu2018doublefusion,zheng2021pamir,zheng2021deepmulticap,li2021image,feng2022fof,xiu2022icon,xiu2023econ}, rendering~\cite{peng2021neural,hu2022hvtr}, and avatar modeling~\cite{huang2020arch,ma2021power,zheng2022structured,zheng2023avatarrex}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1701.02468" id="34">
-        <attvalues>
-          <attvalue for="5" value=" Following the optimization-based paradigm~\cite{bogo2016keep,lassner2017unite,zanfir2018monocular}, methods explicitly fit body models to 2D observations in an iterative manner." />
-        </attvalues>
-      </edge>
-      <edge source="2203.01923" target="1712.01337" id="35">
-        <attvalues>
-          <attvalue for="5" value=" Alternatively, the regression-based paradigm~\cite{tung2017self, kanazawa2018end, pavlakos2018learning, omran2018neural, kolotouros2019learning} takes advantage of the powerful nonlinear mapping capability of neural networks and directly predicts model parameters from raw image pixels." />
-        </attvalues>
-      </edge>
-      <edge source="2303.13794" target="1712.07629" id="36">
-        <attvalues>
-          <attvalue for="5" value=" While most current works focus on interest point detectors~\cite{detone2018superpoint, tyszkiewicz2020disk}, descriptors~\cite{detone2018superpoint, tyszkiewicz2020disk}, matching, or implementing in an end-to-end way~\cite{edstedt2022deep, sun2021loftr, tang2022quadtree, chen2022aspanformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.13794" target="1911.11763" id="37">
-        <attvalues>
-          <attvalue for="5" value=" In the task of image matching, the objective is to establish the relationship between two given images by employing methods such as feature point extraction~\cite{detone2018superpoint, tyszkiewicz2020disk}, feature point description~\cite{detone2018superpoint, tyszkiewicz2020disk}, and feature point matching~\cite{sarlin2020superglue}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.13794" target="1503.01817" id="38">
-        <attvalues>
-          <attvalue for="5" value=" Our experiments demonstrate that the two-stage pipeline with MKPC as the core consistently improves image matching performance on the PhotoTourism dataset, which is a subset of YFCC100M dataset~\cite{thomee2016yfcc100m} with ground truth poses." />
-        </attvalues>
-      </edge>
-      <edge source="2011.03367" target="1703.06868" id="39">
-        <attvalues>
-          <attvalue for="5" value=" We use adaptive instance normalization layers \cite{DBLP:journals/corr/HuangB17} to encourage shape/style disentanglement within each object." />
-        </attvalues>
-      </edge>
-      <edge source="2011.03367" target="1904.12584" id="40">
-        <attvalues>
-          <attvalue for="5" value=" We show that a VQA modular network that incorporates our concept classifiers shows improved generalization over the state-of-the-art \cite{Mao2019NeuroSymbolic} with dramatically fewer examples." />
-        </attvalues>
-      </edge>
-      <edge source="2011.03367" target="1611.09978" id="41">
-        <attvalues>
-          <attvalue for="5" value=" We train concept classifiers for object shapes, object colors/materials, and spatial relationships on our inferred disentangled feature spaces, and show they outperform current state-of-the-art \cite{Mao2019NeuroSymbolic,DBLP:journals/corr/HuRADS16}, which use 2D representations." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14865" target="1904.04812" id="42">
-        <attvalues>
-          <attvalue for="5" value=" Due to fundamental perspective ambiguity however, the absolute metric of depth cannot be obtained from a single view alone \cite{amazon_paper_2, depth_impossible}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1701.02468" id="43">
-        <attvalues>
-          <attvalue for="5" value=" Some approaches were proposed to annotate images with the SMPL model constraining its optimization with 2D pose detections combined with manual curation~\cite{unite}, Inertial Measurement Units (IMUs)~\cite{3dpw} or using multiple views~\cite{HuangBLKGRAB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1801.01615" id="44">
-        <attvalues>
-          <attvalue for="5" value=" Primarily driven by the availability (or absence) of training data, the problem has been originally tackled either as 2D pose estimation in-the-wild~\cite{mpii} or 3D pose estimation in either synthetic~\cite{unite} or constrained scenarios~\cite{h36,totalcapture}, \eg using Motion Capture (MoCap) rooms." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1712.06584" id="45">
-        <attvalues>
-          <attvalue for="5" value=" Interestingly, when evaluating the 2D reprojection error of the body joints, HMR~\cite{hmr} outperforms more recent approaches." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1905.03244" id="46">
-        <attvalues>
-          <attvalue for="5" value=" As a second contribution, we benchmark recent methods that predict SMPL parameters in images~\cite{hmr,nbf,spin,texturepose,graphcmr,dct} or videos~\cite{vibe}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1910.11322" id="47">
-        <attvalues>
-          <attvalue for="5" value=" As a second contribution, we benchmark recent methods that predict SMPL parameters in images~\cite{hmr,nbf,spin,texturepose,graphcmr,dct} or videos~\cite{vibe}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1908.06442" id="48">
-        <attvalues>
-          <attvalue for="5" value=" As a second contribution, we benchmark recent methods that predict SMPL parameters in images~\cite{hmr,nbf,spin,texturepose,graphcmr,dct} or videos~\cite{vibe}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1908.00439" id="49">
-        <attvalues>
-          <attvalue for="5" value=" A few recent works make direct predictions of the human 3D shape without any parametric representation~\cite{moulding,zhu2019detailed,deephuman}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1904.10506" id="50">
-        <attvalues>
-          <attvalue for="5" value=" A few recent works make direct predictions of the human 3D shape without any parametric representation~\cite{moulding,zhu2019detailed,deephuman}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1903.06473" id="51">
-        <attvalues>
-          <attvalue for="5" value=" A few recent works make direct predictions of the human 3D shape without any parametric representation~\cite{moulding,zhu2019detailed,deephuman}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1712.03453" id="52">
-        <attvalues>
-          <attvalue for="5" value=" Even if some efforts have been made to produce outdoor evaluation datasets~\cite{mupots,3dpw,unite}, one can argue that the accuracy of the ground truth is not sufficient or that the variety in terms of scenes, subjects, and backgrounds is not large enough to properly benchmark state-of-the-art methods." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1707.07548" id="53">
-        <attvalues>
-          <attvalue for="5" value=" Some approaches were proposed to annotate images with the SMPL model constraining its optimization with 2D pose detections combined with manual curation~\cite{unite}, Inertial Measurement Units (IMUs)~\cite{3dpw} or using multiple views~\cite{HuangBLKGRAB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1904.11111" id="54">
-        <attvalues>
-          <attvalue for="5" value=" The Mannequin Challenge dataset~\cite{mannequinchallenge}, recently introduced to predict the depth of humans in an image, shows in-the-wild videos of people, often several persons, frozen in action like mannequins and captured from a moving camera." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1802.00434" id="55">
-        <attvalues>
-          <attvalue for="5" value=" More concretely, our pipeline includes the following steps: (a) the human instances are tracked and segmented over the frames to find individual instances, (b) we run a structure-from-motion reconstruction to obtain a 3D point cloud and clean it using the human segmentations, (c) we use DensePose~\cite{densepose} to obtain correspondences between pixels and SMPL surface, and fit SMPL with the constraint of the pose being constant over frames while minimizing the DensePose reconstruction error." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1909.12828" id="56">
-        <attvalues>
-          <attvalue for="5" value=" As a second contribution, we benchmark recent methods that predict SMPL parameters in images~\cite{hmr,nbf,spin,texturepose,graphcmr,dct} or videos~\cite{vibe}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.02743" target="1912.05656" id="57">
-        <attvalues>
-          <attvalue for="5" value=" We find that the best method regarding 3D pose estimation is VIBE~\cite{vibe} which benefits from leveraging temporal information compared to the other ones that predict SMPL parameters from a static image." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08270" target="2003.08934" id="58">
-        <attvalues>
-          <attvalue for="5" value=" Central to this endeavor is the task of accurately perceiving and segmenting 3D structures~\cite{NeRF, dvgo, tensorf, mipnerf360, InstantNGP, plenoxel}, a task that becomes increasingly complex as we delve into more sophisticated representations of 3D scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08270" target="2308.04079" id="59">
-        <attvalues>
-          <attvalue for="5" value=" Recently, 3D Gaussian Splatting (3D-GS)~\cite{3dgs} emerges as a cutting-edge approach that promises to revolutionize how we render and reconstruct 3D spaces." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08270" target="2312.00732" id="60">
-        <attvalues>
-          <attvalue for="5" value=" Existing approaches~\cite{gsgroup, saga} to this challenge have largely depended on iterative gradient descent methods for labeling 3D Gaussian." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05014" target="1804.01654" id="61">
-        <attvalues>
-          <attvalue for="5" value=" Such a surface representation differs from explicit mesh or implicit level set used in many previous works \cite{wang2018pixel2mesh,jiang2020sdfdiff}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05014" target="1912.07109" id="62">
-        <attvalues>
-          <attvalue for="5" value=" Such a surface representation differs from explicit mesh or implicit level set used in many previous works \cite{wang2018pixel2mesh,jiang2020sdfdiff}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="2006.12057" id="63">
-        <attvalues>
-          <attvalue for="5" value=" Differentiable rendering (DR)~\cite{kato2020differentiable,zhao2020physics} is an emerging tool for multi-view stereo." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="1912.07372" id="64">
-        <attvalues>
-          <attvalue for="5" value=" A high-fidelity reconstruction can be achieved~\cite{dvr,idr} when combined with 3D representations using neural networks ({neural fields}~\cite{neuralfields})." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="2003.09852" id="65">
-        <attvalues>
-          <attvalue for="5" value=" On the one hand, compared to previous DR frameworks~\cite{idr,neus,volsdf} which only use photometric data to supervise the training, the SL supervision can provide important cues to reduce the geometrical ambiguity." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="2111.11426" id="66">
-        <attvalues>
-          <attvalue for="5" value=" A high-fidelity reconstruction can be achieved~\cite{dvr,idr} when combined with 3D representations using neural networks ({neural fields}~\cite{neuralfields})." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="1406.6595" id="67">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, by establishing a geometric relationship between the neural implicit surface and camera poses, the camera extrinsic parameters are also refined during optimization; therefore, our method does not require strict calibration in contrast to conventional SL methods~\cite{Caspi,GuQ,Nguyen,Salvi}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="1908.08185" id="68">
-        <attvalues>
-          <attvalue for="5" value=" A representative method is the use structured light (SL)~\cite{Caspi,GuQ,Nguyen,Salvi,li2019pro,Li7}, in which multiple texture patterns are projected onto objects using a projector." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="2106.10689" id="69">
-        <attvalues>
-          <attvalue for="5" value=" On the one hand, compared to previous DR frameworks~\cite{idr,neus,volsdf} which only use photometric data to supervise the training, the SL supervision can provide important cues to reduce the geometrical ambiguity." />
-        </attvalues>
-      </edge>
-      <edge source="2211.11971" target="2106.12052" id="70">
-        <attvalues>
-          <attvalue for="5" value=" On the one hand, compared to previous DR frameworks~\cite{idr,neus,volsdf} which only use photometric data to supervise the training, the SL supervision can provide important cues to reduce the geometrical ambiguity." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01013" target="2211.09791" id="71">
-        <attvalues>
-          <attvalue for="5" value=" Once the camera is calibrated, it can accurately measure distances, angles, and sizes of objects in the 3D world and perform other image-based computer vision tasks such as object tracking \cite{zhang2023motrv2}, 3D reconstruction \cite{kang2020review,wu2023multiview}, medical imaging \cite{barbero2019smartphone}, and autonomous driving \cite{feng2019can}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01013" target="2301.08247" id="72">
-        <attvalues>
-          <attvalue for="5" value=" Once the camera is calibrated, it can accurately measure distances, angles, and sizes of objects in the 3D world and perform other image-based computer vision tasks such as object tracking \cite{zhang2023motrv2}, 3D reconstruction \cite{kang2020review,wu2023multiview}, medical imaging \cite{barbero2019smartphone}, and autonomous driving \cite{feng2019can}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01013" target="1909.12358" id="73">
-        <attvalues>
-          <attvalue for="5" value=" Once the camera is calibrated, it can accurately measure distances, angles, and sizes of objects in the 3D world and perform other image-based computer vision tasks such as object tracking \cite{zhang2023motrv2}, 3D reconstruction \cite{kang2020review,wu2023multiview}, medical imaging \cite{barbero2019smartphone}, and autonomous driving \cite{feng2019can}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01013" target="1810.10658" id="74">
-        <attvalues>
-          <attvalue for="5" value=" The idea of generating synthetic calibration data has been previously applied in other works, such as sports-based synthetic calibration \cite{chen2019sports} and evaluating closed-form solutions of principal line calibration \cite{chuang2021geometry}, but not necessarily for comparing calibration algorithms." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="1506.01497" id="75">
-        <attvalues>
-          <attvalue for="5" value=" The rise of deep learning methods has led to impressive progress in object detection in the past decades, mostly thanks to their ability to extract non-linear features well adapted to the downstream task \cite{ren2015faster,redmon2016you}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="1506.02640" id="76">
-        <attvalues>
-          <attvalue for="5" value=" The rise of deep learning methods has led to impressive progress in object detection in the past decades, mostly thanks to their ability to extract non-linear features well adapted to the downstream task \cite{ren2015faster,redmon2016you}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="2012.08573" id="77">
-        <attvalues>
-          <attvalue for="5" value=" {Indeed, neural networks (NN) based on bounding box regression either have anchors too large for detecting small objects, or require a significant decrease in the IoU threshold, which leads to {multiple detections of a same target~\cite{dai2021attentional}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="1902.07296" id="78">
-        <attvalues>
-          <attvalue for="5" value=" Some of them focus on augmenting or oversampling the dataset~\cite{kisantal2019augmentation, akyon2022sahi}, while others focus on improving small object feature-enhancement." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="1612.03144" id="79">
-        <attvalues>
-          <attvalue for="5" value=" {Among} the latter, we can cite Feature Pyramid Networks (FPN) and their variants~\cite{lin2017feature,deng2021extended}, {whose} multi-scale approach is beneficial for the detection of objects of various sizes." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="2003.07021" id="80">
-        <attvalues>
-          <attvalue for="5" value=" Other works include attention mechanisms to learn long-range dependencies~\cite{wu2022uiu}, or use super-resolution to enhance feature responses of small objects~\cite{deng2021extended}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.01363" target="2212.00968" id="81">
-        <attvalues>
-          <attvalue for="5" value=" Other works include attention mechanisms to learn long-range dependencies~\cite{wu2022uiu}, or use super-resolution to enhance feature responses of small objects~\cite{deng2021extended}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.11077" target="1708.07987" id="82">
-        <attvalues>
-          <attvalue for="5" value=" The energy-based algorithms \cite{gu2008local} are global methods with a cost function defined as a function of the entire image extent \cite{yang2008stereo}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1907.05272" id="83">
-        <attvalues>
-          <attvalue for="5" value=" Camera pose estimation is used in a variety of applications such as autonomous navigation, augmented reality, structure from motion, and simultaneous localization and mapping \cite{shavit2019introduction}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1603.09114" id="84">
-        <attvalues>
-          <attvalue for="5" value=" Classically, a feature descriptor (\cite{lowe2004Sift,bay2008Surf,calonder2010BRIEFDescriptor,rublee2011ORB,yi2016LIFT}) would be used to match 2D points to 3D points." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1505.07427" id="85">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kendall2016modellingBayesianPoseNet,kendall2017geometricLossFunctions,walch2017LSTMPose} proposed small variations to the PoseNet \cite{kendall2015posenet} architecture to reduce median localization error." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1409.4842" id="86">
-        <attvalues>
-          <attvalue for="5" value=" They use a GoogLeNet \cite{szegedy2015goingGoogLeNet} backbone to transform scene images to feature vectors; A fully-connected layer regresses both absolute translations and rotation unit quaternions." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1509.05909" id="87">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kendall2016modellingBayesianPoseNet,kendall2017geometricLossFunctions,walch2017LSTMPose} proposed small variations to the PoseNet \cite{kendall2015posenet} architecture to reduce median localization error." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1704.00390" id="88">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kendall2016modellingBayesianPoseNet,kendall2017geometricLossFunctions,walch2017LSTMPose} proposed small variations to the PoseNet \cite{kendall2015posenet} architecture to reduce median localization error." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1611.07890" id="89">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kendall2016modellingBayesianPoseNet,kendall2017geometricLossFunctions,walch2017LSTMPose} proposed small variations to the PoseNet \cite{kendall2015posenet} architecture to reduce median localization error." />
-        </attvalues>
-      </edge>
-      <edge source="2108.03819" target="1707.09733" id="90">
-        <attvalues>
-          <attvalue for="5" value="18\% (from 37ms to 4ms), \item Outperform NN-Net \cite{laskar2017camera} on information retrieval while retaining the performance on pose regression." />
-        </attvalues>
-      </edge>
-      <edge source="2010.13508" target="1503.05860" id="91">
-        <attvalues>
-          <attvalue for="5" value=" Representing the physical world in 3D, including shape and colour, is key for industrial and research purposes~\cite{anguelov2005scape,yang2014semantic,hasler2009statistical,pishchulin2017building,bogo2014faust,xu2018multilevel,bronstein2008numerical,robinette1999caesar}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.13508" target="1811.10175" id="92">
-        <attvalues>
-          <attvalue for="5" value=" Representing the physical world in 3D, including shape and colour, is key for industrial and research purposes~\cite{anguelov2005scape,yang2014semantic,hasler2009statistical,pishchulin2017building,bogo2014faust,xu2018multilevel,bronstein2008numerical,robinette1999caesar}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1904.08963" id="93">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1712.02294" id="94">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1711.10275" id="95">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1611.07759" id="96">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="2012.03194" id="97">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="2008.00072" id="98">
-        <attvalues>
-          <attvalue for="5" value=" 3D object detection~\cite{ding2019votenet,ku2018joint,graham20183d} is an important task in computer vision and has various applications such as autonomous driving~\cite{chen2017multi} and robotics~\cite{fan2020computer,vincent2020dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="2001.10692" id="99">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, great success has been made with the help of deep networks and large-scale datasets ~\cite{qi2020imvotenet,yang2018pixor,yan2018second,zhou2018voxelnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1902.06326" id="100">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, great success has been made with the help of deep networks and large-scale datasets ~\cite{qi2020imvotenet,yang2018pixor,yan2018second,zhou2018voxelnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1711.06396" id="101">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, great success has been made with the help of deep networks and large-scale datasets ~\cite{qi2020imvotenet,yang2018pixor,yan2018second,zhou2018voxelnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1507.02672" id="102">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1610.02242" id="103">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1704.03976" id="104">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="2004.08514" id="105">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="2004.08878" id="106">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of lacking annotated data, semi-supervised learning (SSL)~\cite{rasmus2015semi,samuli2017temporal,miyato2018virtual,feng2020dmt,feng2020semi} and unsupervised domain adaptation (UDA)~\cite{zhou2020uncertainty, zhou2021sad, guo2021label, zhou2021context} brings a feasible solution." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1912.11803" id="107">
-        <attvalues>
-          <attvalue for="5" value=" To the best of our knowledge,~\cite{zhao2020sess} is the only deep learning method for 3D object detection under semi-supervised conditions." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1702.04405" id="108">
-        <attvalues>
-          <attvalue for="5" value=" Promising results have been achieved on indoor scenes~\cite{sunrgb-d,scannet}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06649" target="1904.04717" id="109">
-        <attvalues>
-          <attvalue for="5" value="Considering that the direct use of pseudo-labeling~\cite{iscen2019label} will produce inevitable noise that may accumulate errors and affect the performance, we adopt Adaptive Class Confidence Selection scheme (ACCS) for pseudo-label filtering." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1503.03585" id="110">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2009.00713" id="111">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2105.05233" id="112">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2101.09258" id="113">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2107.00630" id="114">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion Probabilistic Models (DPMs)~\cite{sohl2015deep, song2019generative, ho2020denoising}, also known as simply diffusion models, have recently emerged as a powerful family of generative models, achieving state-of-the-art performance on audio and image synthesis \cite{chen2020wavegrad,dhariwal2021diffusion}, while admitting better training stability over adversarial approaches~\cite{goodfellow2014generative}, as well as likelihood computation, which enables further applications such as compression and density estimation \cite{song2021maximum,kingma2021variational}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2011.13456" id="115">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have achieved impressive empirical results in a variety of image-to-image translation tasks not limited to text-to-image, super-resolution, inpainting, colorization, uncropping, and artifact removal \cite{song2020score,saharia2021palette,ramesh2022hierarchical, saharia2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2111.05826" id="116">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have achieved impressive empirical results in a variety of image-to-image translation tasks not limited to text-to-image, super-resolution, inpainting, colorization, uncropping, and artifact removal \cite{song2020score,saharia2021palette,ramesh2022hierarchical, saharia2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2204.06125" id="117">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have achieved impressive empirical results in a variety of image-to-image translation tasks not limited to text-to-image, super-resolution, inpainting, colorization, uncropping, and artifact removal \cite{song2020score,saharia2021palette,ramesh2022hierarchical, saharia2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1906.01618" id="118">
-        <attvalues>
-          <attvalue for="5" value=" Before the recent emergence of Scene Representation Networks (SRN)~\cite{sitzmann2019scene} and Neural Radiance Fields (NeRF)~\cite{mildenhall2020nerf}, state-of-the-art approaches to novel view synthesis were typically built on generative models~\cite{sun2018multi} or more classical techniques on interpolation or disparity estimation \cite{park2017transformation,zhou2018stereo}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2003.08934" id="119">
-        <attvalues>
-          <attvalue for="5" value=" Before the recent emergence of Scene Representation Networks (SRN)~\cite{sitzmann2019scene} and Neural Radiance Fields (NeRF)~\cite{mildenhall2020nerf}, state-of-the-art approaches to novel view synthesis were typically built on generative models~\cite{sun2018multi} or more classical techniques on interpolation or disparity estimation \cite{park2017transformation,zhou2018stereo}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1703.02921" id="120">
-        <attvalues>
-          <attvalue for="5" value=" Before the recent emergence of Scene Representation Networks (SRN)~\cite{sitzmann2019scene} and Neural Radiance Fields (NeRF)~\cite{mildenhall2020nerf}, state-of-the-art approaches to novel view synthesis were typically built on generative models~\cite{sun2018multi} or more classical techniques on interpolation or disparity estimation \cite{park2017transformation,zhou2018stereo}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1805.09817" id="121">
-        <attvalues>
-          <attvalue for="5" value=" Before the recent emergence of Scene Representation Networks (SRN)~\cite{sitzmann2019scene} and Neural Radiance Fields (NeRF)~\cite{mildenhall2020nerf}, state-of-the-art approaches to novel view synthesis were typically built on generative models~\cite{sun2018multi} or more classical techniques on interpolation or disparity estimation \cite{park2017transformation,zhou2018stereo}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2012.02190" id="122">
-        <attvalues>
-          <attvalue for="5" value=" PixelNeRF~\cite{yu2021pixelnerf} and VisionNeRF~\cite{lin2022vision} address this by training NeRF-like models conditioned on feature maps that encode the novel input view(s)." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2112.00724" id="123">
-        <attvalues>
-          <attvalue for="5" value=" Heavily regularized NeRFs for novel view synthesis with few images such as RegNeRF~\cite{niemeyer2021regnerf} produce undesired artifacts when given very few images, and fail to leverage knowledge from multiple scenes (recall NeRFs are trained on a single scene, eg, one model per scene), and given one or very few views of a novel scene, a reasonable model must extrapolate to complete the occluded parts of the scene." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2109.01750" id="124">
-        <attvalues>
-          <attvalue for="5" value=" Other works such as CodeNeRF~\cite{jang2021codenerf} and LoLNeRF~\cite{rebain2021lolnerf} instead employ test-time optimization to handle novel scenes, but still have issues with sample quality." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2207.05736" id="125">
-        <attvalues>
-          <attvalue for="5" value=" PixelNeRF~\cite{yu2021pixelnerf} and VisionNeRF~\cite{lin2022vision} address this by training NeRF-like models conditioned on feature maps that encode the novel input view(s)." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2104.07636" id="126">
-        <attvalues>
-          <attvalue for="5" value=" This type of failure has also been previously observed in regression-based models~\cite{saharia2021image}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2111.09996" id="127">
-        <attvalues>
-          <attvalue for="5" value=" Other works such as CodeNeRF~\cite{jang2021codenerf} and LoLNeRF~\cite{rebain2021lolnerf} instead employ test-time optimization to handle novel scenes, but still have issues with sample quality." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2106.02634" id="128">
-        <attvalues>
-          <attvalue for="5" value=" In recent literature, geometry-free approaches (eg, methods without explicit geometric inductive biases like those introduced by volume rendering) such as Light Field Networks (LFN)~\cite{sitzmann2021light} and Scene Representation Transformers (SRT)~\cite{sajjadi2021scene} have achieved results competitive with 3D-aware methods in the ``few-shot'' setting, where the number of conditioning views is limited (eg, 1-10 images vs." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2111.13152" id="129">
-        <attvalues>
-          <attvalue for="5" value=" Drawing inspiration from Scene Representation Transformers~\cite{sajjadi2021scene}, 3DiMs are trained to build a conditional generative model of one view given another view and their poses." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2112.07945" id="130">
-        <attvalues>
-          <attvalue for="5" value=" Similarly to our approach, EG3D \cite{chan2022efficient} provides approximate 3D consistency by leveraging generative models." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1812.04948" id="131">
-        <attvalues>
-          <attvalue for="5" value=" EG3D employs a StyleGAN \cite{karras2019style} with volumetric rendering, followed by generative super-resolution (the latter being responsible for the approximation)." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1801.04406" id="132">
-        <attvalues>
-          <attvalue for="5" value=" In comparison to this complex setup, diffusion not only provides a significantly simpler architecture, but also a simpler hyper-parameter tuning experience compared to GANs, which are well-known to be notoriously difficult to tune \cite{mescheder2018training}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2103.01458" id="133">
-        <attvalues>
-          <attvalue for="5" value=" Notably, diffusion models have already seen some success on 3D point-cloud generation \cite{luo2021diffusion,waibel2022diffusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="2208.14125" id="134">
-        <attvalues>
-          <attvalue for="5" value=" Notably, diffusion models have already seen some success on 3D point-cloud generation \cite{luo2021diffusion,waibel2022diffusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.04628" target="1505.04597" id="135">
-        <attvalues>
-          <attvalue for="5" value=" \item We introduce X-UNet, a new UNet architecture~\cite{ronneberger2015u} variant for 3D novel view synthesis, demonstrating that changes in architecture are critical for high fidelity results." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1906.02739" id="136">
-        <attvalues>
-          <attvalue for="5" value=" To deal with this ill-posed problem, a promising approach is to use supervised learning approaches requiring 3D ground truth, such as Mesh R-CNN~\cite{Gkioxari_2019_ICCV} and Pixel2Mesh~\cite{wang2018pixel2mesh}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1804.01654" id="137">
-        <attvalues>
-          <attvalue for="5" value=" To deal with this ill-posed problem, a promising approach is to use supervised learning approaches requiring 3D ground truth, such as Mesh R-CNN~\cite{Gkioxari_2019_ICCV} and Pixel2Mesh~\cite{wang2018pixel2mesh}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1506.01497" id="138">
-        <attvalues>
-          <attvalue for="5" value=" Although supervised training has been quite successful in 2D tasks such as image classification~\cite{ mei2021videobased}, object detection~\cite{ren2016faster} and segmentation~\cite{ronneberger2015u}, it is much more time-consuming and expensive to collect 3D ground truth data for 3D supervised training." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1611.07700" id="139">
-        <attvalues>
-          <attvalue for="5" value=" For example, SMPL~\cite{loper2015smpl} uses thousands of registered 3D scans of humans for training, SMAL~\cite{zuffi20173d} utilizes scans of animal toys and a manually rigged mesh model." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1803.07549" id="140">
-        <attvalues>
-          <attvalue for="5" value=" Why single-view? 3D reconstruction models, such as Nerf~\cite{mildenhall2020nerf}, uses multi-view images as input while CMR~\cite{kanazawa2018learning}, ACSM~\cite{kulkarni2020articulationaware} and LASR~\cite{yang2021lasr} use a single view image." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="2004.00614" id="141">
-        <attvalues>
-          <attvalue for="5" value=" Why single-view? 3D reconstruction models, such as Nerf~\cite{mildenhall2020nerf}, uses multi-view images as input while CMR~\cite{kanazawa2018learning}, ACSM~\cite{kulkarni2020articulationaware} and LASR~\cite{yang2021lasr} use a single view image." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="2003.08934" id="142">
-        <attvalues>
-          <attvalue for="5" value=" Why single-view? 3D reconstruction models, such as Nerf~\cite{mildenhall2020nerf}, uses multi-view images as input while CMR~\cite{kanazawa2018learning}, ACSM~\cite{kulkarni2020articulationaware} and LASR~\cite{yang2021lasr} use a single view image." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="2105.02976" id="143">
-        <attvalues>
-          <attvalue for="5" value=" Why single-view? 3D reconstruction models, such as Nerf~\cite{mildenhall2020nerf}, uses multi-view images as input while CMR~\cite{kanazawa2018learning}, ACSM~\cite{kulkarni2020articulationaware} and LASR~\cite{yang2021lasr} use a single view image." />
-        </attvalues>
-      </edge>
-      <edge source="2201.09373" target="1904.01786" id="144">
-        <attvalues>
-          <attvalue for="5" value=" We utilize a differentiable render~\cite{liu2019softras} to make whole learning unsupervised, given an initialized 3D template that can be deformed based on some learnable parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04479" target="1110.5102" id="145">
-        <attvalues>
-          <attvalue for="5" value=" Better depth estimates have been achieved by incorporating semantic labels~\cite{Liu:cvpr10}, or more sophisticated models~\cite{Li:2010}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1611.07759" id="146">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the rapid development of the 3D point cloud has shown crucial importance to many real-world applications in different areas, such as robotics~\cite{pomerleau2015review}, autonomous driving~\cite{chen2017multi}, augmented and virtual reality~\cite{reitinger2007augmented}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1612.00593" id="147">
-        <attvalues>
-          <attvalue for="5" value=" PointNet firstly utilizes deep learning on raw point clouds, which is the bedrock for overcoming the challenge of the irregular nature of point cloud~\cite{qi2017pointnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1804.01495" id="148">
-        <attvalues>
-          <attvalue for="5" value=" The 3D point cloud learning has attracted increasing attention on computer vision areas to process the various vision tasks, such as classification~\cite{qi2017pointnet}, segmentation~\cite{ye20183d}, registration~\cite{lawin2018density} and detection~\cite{vora2020pointpainting}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1911.10150" id="149">
-        <attvalues>
-          <attvalue for="5" value=" The 3D point cloud learning has attracted increasing attention on computer vision areas to process the various vision tasks, such as classification~\cite{qi2017pointnet}, segmentation~\cite{ye20183d}, registration~\cite{lawin2018density} and detection~\cite{vora2020pointpainting}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1505.00880" id="150">
-        <attvalues>
-          <attvalue for="5" value=" For example, the traditional deep learning is used with the multi-view~\cite{su2015multi} or 3D voxels~\cite{maturana2015voxnet} methods on point cloud tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1811.07246" id="151">
-        <attvalues>
-          <attvalue for="5" value=" Many follow up works such as PointCNN~\cite{li2018pointcnn}, PointConv~\cite{wu2019pointconv} and DGCNN~\cite{wang2019dynamic} also focus on the aggregation of local regions to further improve the feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1801.07829" id="152">
-        <attvalues>
-          <attvalue for="5" value=" Many follow up works such as PointCNN~\cite{li2018pointcnn}, PointConv~\cite{wu2019pointconv} and DGCNN~\cite{wang2019dynamic} also focus on the aggregation of local regions to further improve the feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1910.08207" id="153">
-        <attvalues>
-          <attvalue for="5" value=" The unlabeled data is relatively easy to obtain abundantly in real applications, thus some recent studies attempt unsupervised learning for 3D point clouds~\cite{hassani2019unsupervised,chen2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="2003.01661" id="154">
-        <attvalues>
-          <attvalue for="5" value=" The unlabeled data is relatively easy to obtain abundantly in real applications, thus some recent studies attempt unsupervised learning for 3D point clouds~\cite{hassani2019unsupervised,chen2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1712.07262" id="155">
-        <attvalues>
-          <attvalue for="5" value=" To get discriminative information in unsupervised manner, recent works focus on constructing a new structural representation in the form of 3D structure points~\cite{yang2018foldingnet, deng2018ppf}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.05301" target="1911.02744" id="156">
-        <attvalues>
-          <attvalue for="5" value=" Currently, PointDAN~\cite{qin2019pointdan} introduces a point-based unsupervised domain adaptation network by aligning the local and global distribution across the different domains for classification tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2312.13090" target="2305.13786" id="157">
-        <attvalues>
-          <attvalue for="5" value=" The Perception Test~\cite{patraucean2023perception} is the first benchmark to use real world videos and focus on comprehensively diagnosing perception capabilities, like memory, understanding of intuitive physics and geometry, abstractions, or semantics." />
-        </attvalues>
-      </edge>
-      <edge source="2012.14833" target="1903.01656" id="158">
-        <attvalues>
-          <attvalue for="5" value=" In previous work, the research community has investigated - among others - the benefits of thermal vision specifically for the problem of thermal-inertial odometry estimation~\cite{KTIO_ICRA_2019,VTIMUDVE_AEROCONF_2019,khattak2019robust,zhao2020tp,khattak2018marker} and object detection~\cite{treptow2006real,ciric2013computationally,yin2006moving}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.14833" target="2012.03455" id="159">
-        <attvalues>
-          <attvalue for="5" value=" In previous work, the research community has investigated - among others - the benefits of thermal vision specifically for the problem of thermal-inertial odometry estimation~\cite{KTIO_ICRA_2019,VTIMUDVE_AEROCONF_2019,khattak2019robust,zhao2020tp,khattak2018marker} and object detection~\cite{treptow2006real,ciric2013computationally,yin2006moving}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.14833" target="1903.00782" id="160">
-        <attvalues>
-          <attvalue for="5" value=" In previous work, the research community has investigated - among others - the benefits of thermal vision specifically for the problem of thermal-inertial odometry estimation~\cite{KTIO_ICRA_2019,VTIMUDVE_AEROCONF_2019,khattak2019robust,zhao2020tp,khattak2018marker} and object detection~\cite{treptow2006real,ciric2013computationally,yin2006moving}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1611.00850" id="161">
-        <attvalues>
-          <attvalue for="5" value=" For instance, SpyNet \cite{Ranjan2017_SPyNet} proposes to use a spatial pyramid network, where the downsampled input frames are warped by the current estimate and iterative updates proceed in a coarse-to-fine manner \cite{Brox2004}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1709.02371" id="162">
-        <attvalues>
-          <attvalue for="5" value=" While \cite{Saxena2019_pwoc,Rishav2021_ResFPN} are also based on FPN ideas to integrate semantically important information from other scales, they rely on PWC-Net \cite{Sun2018_PWC} and hence neither exploit recurrent updates nor a non-local lookup in the cost volume." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1904.05290" id="163">
-        <attvalues>
-          <attvalue for="5" value=" Finally, IRR \cite{Hur2019} suggests to share the estimation network across all scales, leading to a more compact architecture which can be trained more efficiently." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="2003.12039" id="164">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, RAFT \cite{Teed2020_RAFT} follows a completely different ap- proach." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1912.10739" id="165">
-        <attvalues>
-          <attvalue for="5" value=" While the correlation pyramid adds non-local cost information to the matching process, the sampling allows to overcome some problems of warping \cite{Hofinger2020_IOFPL}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1612.03144" id="166">
-        <attvalues>
-          <attvalue for="5" value="\ \cite{Lin2017_FPN}) and the actual loss is not specified." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="1904.06116" id="167">
-        <attvalues>
-          <attvalue for="5" value=" While \cite{Saxena2019_pwoc,Rishav2021_ResFPN} are also based on FPN ideas to integrate semantically important information from other scales, they rely on PWC-Net \cite{Sun2018_PWC} and hence neither exploit recurrent updates nor a non-local lookup in the cost volume." />
-        </attvalues>
-      </edge>
-      <edge source="2207.12163" target="2006.12235" id="168">
-        <attvalues>
-          <attvalue for="5" value=" While \cite{Saxena2019_pwoc,Rishav2021_ResFPN} are also based on FPN ideas to integrate semantically important information from other scales, they rely on PWC-Net \cite{Sun2018_PWC} and hence neither exploit recurrent updates nor a non-local lookup in the cost volume." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15820" target="2304.04672" id="169">
-        <attvalues>
-          <attvalue for="5" value=" Without a proper trimap, the matting problem becomes ill-posed, as distinguishing between the ``foreground&quot; and ``background&quot; can be highly ambiguous \cite{li2023deep, xu2017deep, fang2022user, sun2023ultrahigh, park2022matteformer, liu2021long, zheng2022image, liu2021tripartite}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15820" target="2004.00626" id="170">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue, pioneering work such as the &quot;Background Matting&quot; (BGM) algorithm \cite{sengupta2020background, BGMv2} was introduced." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15820" target="2012.07810" id="171">
-        <attvalues>
-          <attvalue for="5" value=" Additionally, we replace the use of traditional RGB patches with RGB-D patches in the refining network \cite{BGMv2}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15820" target="1503.02531" id="172">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, we introduce the concept of distillation, where we derive a smaller base network \cite{hinton2015distilling} from the original ResNet$50$-based model used in BGMv$2$." />
-        </attvalues>
-      </edge>
-      <edge source="2401.08357" target="2103.09943" id="173">
-        <attvalues>
-          <attvalue for="5" value=" Existing image fusion algorithms can be divided into two main categories: deep learning (DL)-based methods \cite{R4,R5,R27} and traditional methods \cite{R7,R8,R9}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="1906.06423" id="174">
-        <attvalues>
-          <attvalue for="5" value=" Increasing resolution is a reliable way to improve model accuracy \cite{fixres, gpipe, efficientnet, threethings, revenge, apple_vlm, monkey, vstar}, but this comes at a cost; training models for hundreds of epochs on large-scale datasets is expensive, especially at high-resolutions." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2010.11929" id="175">
-        <attvalues>
-          <attvalue for="5" value=" Vision transformers (ViTs \cite{vit}) offer SoTA performance on many computer vision tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2210.05844" id="176">
-        <attvalues>
-          <attvalue for="5" value=" This non-hierarchical design enables learning patch representations, which are useful for dense prediction tasks \cite{segvit, open_vocab, plain_obj} and are fundamental for vision-language models \cite{vlm_1, vlm_2, vlm_3}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2304.08485" id="177">
-        <attvalues>
-          <attvalue for="5" value=" This non-hierarchical design enables learning patch representations, which are useful for dense prediction tasks \cite{segvit, open_vocab, plain_obj} and are fundamental for vision-language models \cite{vlm_1, vlm_2, vlm_3}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2106.04560" id="178">
-        <attvalues>
-          <attvalue for="5" value=" Lastly, it enables model scaling by increasing the embedding size and the layer count \cite{scaling, sovit}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2108.12409" id="179">
-        <attvalues>
-          <attvalue for="5" value=" We strongly believe that patch position encoding is a primary cause of the poor sequence length extrapolation ability of ViTs --- like it is in NLP, where significant advancements have been made by improving position encoding \cite{alibi_nlp, rope_scaling, nope, skip_pose}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2212.08013" id="180">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, methods that improve sequence length extrapolation, like our proposed method, can be fused with methods that adjust patch sizes, like FlexiViT \cite{flexivit}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2104.09864" id="181">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the rotary position embeddings (RoPE \cite{rope_nlp}) used in SoTA language models \cite{llama, mixtral} were extended to ViTs, as $2$D-RoPE \cite{rope}, showing exciting results." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2302.13971" id="182">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the rotary position embeddings (RoPE \cite{rope_nlp}) used in SoTA language models \cite{llama, mixtral} were extended to ViTs, as $2$D-RoPE \cite{rope}, showing exciting results." />
-        </attvalues>
-      </edge>
-      <edge source="2405.13985" target="2211.10408" id="183">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the rotary position embeddings (RoPE \cite{rope_nlp}) used in SoTA language models \cite{llama, mixtral} were extended to ViTs, as $2$D-RoPE \cite{rope}, showing exciting results." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1602.00134" id="184">
-        <attvalues>
-          <attvalue for="5" value=" Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1611.07828" id="185">
-        <attvalues>
-          <attvalue for="5" value=" Some methods are capable of predicting multiple poses~\cite{Cao:etal:cvpr17:openpose,Mehta:etal:XNECT:TOG2020} while others are focusing on single person~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PavlakosZDD17a}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1705.01583" id="186">
-        <attvalues>
-          <attvalue for="5" value=" As a result, a depth image-based approach may not necessarily require as many sophisticated designs as methods aiming to estimate 3D poses from a single RGB image~\cite{Mehta:etal:ACM2017:VNect,Mehta:etal:XNECT:TOG2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1607.08659" id="187">
-        <attvalues>
-          <attvalue for="5" value=" There are methods mostly relying on a single image to predict human poses~\cite{Wei:etal:CVPR2016:CPM,DBLP:conf/cvpr/PavlakosZDD17a,Mehta:etal:ACM2017:VNect} and others based on multiple cameras~\cite{DBLP:conf/eccv/RhodinRCRST16,Elhayek:etal:PAMI17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1611.08050" id="188">
-        <attvalues>
-          <attvalue for="5" value=" While some RGB-based methods spend more effort in differentiating people cluttered in a large scene~\cite{Cao:etal:cvpr17:openpose,BenzineCLPA:CVPR2020}, depth-based methods~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Xiong:etal:iccv2019:a2j} focus more on recovering highly accurate 3D poses of fewer people presented in a closer range." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1907.00837" id="189">
-        <attvalues>
-          <attvalue for="5" value=" As a result, a depth image-based approach may not necessarily require as many sophisticated designs as methods aiming to estimate 3D poses from a single RGB image~\cite{Mehta:etal:ACM2017:VNect,Mehta:etal:XNECT:TOG2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1603.06937" id="190">
-        <attvalues>
-          <attvalue for="5" value=" Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1605.05180" id="191">
-        <attvalues>
-          <attvalue for="5" value=" Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1705.03098" id="192">
-        <attvalues>
-          <attvalue for="5" value=" Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1805.04095" id="193">
-        <attvalues>
-          <attvalue for="5" value=" Some methods estimate 3D poses~\cite{DBLP:conf/bmvc/TekinKSLF16,DBLP:conf/iccv/MartinezHRL17,DBLP:conf/cvpr/PavlakosZD18,Xiong:etal:iccv2019:a2j} while others only predict 2D poses~\cite{Wei:etal:CVPR2016:CPM,Newell:etal:ECCV2016:hourglass,DBLP:conf/cvpr/PapandreouZKTTB17}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1908.09999" id="194">
-        <attvalues>
-          <attvalue for="5" value=" While some RGB-based methods spend more effort in differentiating people cluttered in a large scene~\cite{Cao:etal:cvpr17:openpose,BenzineCLPA:CVPR2020}, depth-based methods~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Xiong:etal:iccv2019:a2j} focus more on recovering highly accurate 3D poses of fewer people presented in a closer range." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1701.01779" id="195">
-        <attvalues>
-          <attvalue for="5" value=" Developing a depth image-based multi-person 3D pose estimation from a well-established RGB image-based 2D method~\cite{DBLP:conf/cvpr/PapandreouZKTTB17,Ren:etal:pami2017:faster:rcnn,Cao:etal:cvpr17:openpose} is conceptually simple for two reasons: some schemes to handle multi-person detection can be shared; 3D information is partially available from the input~\cite{Haque:etal:eccv2016:itop,Xiong:etal:iccv2019:a2j}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1910.13911" id="196">
-        <attvalues>
-          <attvalue for="5" value=" While some RGB-based methods spend more effort in differentiating people cluttered in a large scene~\cite{Cao:etal:cvpr17:openpose,BenzineCLPA:CVPR2020}, depth-based methods~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Xiong:etal:iccv2019:a2j} focus more on recovering highly accurate 3D poses of fewer people presented in a closer range." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1608.03932" id="197">
-        <attvalues>
-          <attvalue for="5" value=" Methods can also be differentiated by the input, while most methods use RGB images~\cite{Cao:etal:cvpr17:openpose,DBLP:conf/cvpr/PapandreouZKTTB17,Mehta:etal:ACM2017:VNect,BenzineCLPA:CVPR2020}, some use depth maps~\cite{Martiez-Gonzalez:etal:iros2018:rpm,Wang:etal:ACM2016,Xiong:etal:iccv2019:a2j}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1512.02325" id="198">
-        <attvalues>
-          <attvalue for="5" value=" The simplest idea is to directly extend a single-shot object detector~\cite{Liu:etal:eccv2016:ssd,Redmon:etal:cvpr16:yolo,Redmon:Farhadi:cvpr2017:yolo2} with additional pose attributes, so that the network can output human poses." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1506.02640" id="199">
-        <attvalues>
-          <attvalue for="5" value=" In addition, more sophisticated design is required to solve the compatibility issue between pose estimation and bounding box detection~\cite{He:etal:iccv2017:mask:rcnn,Redmon:etal:cvpr16:yolo}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1612.08242" id="200">
-        <attvalues>
-          <attvalue for="5" value=" The simplest idea is to directly extend a single-shot object detector~\cite{Liu:etal:eccv2016:ssd,Redmon:etal:cvpr16:yolo,Redmon:Farhadi:cvpr2017:yolo2} with additional pose attributes, so that the network can output human poses." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1703.06870" id="201">
-        <attvalues>
-          <attvalue for="5" value=" In addition, more sophisticated design is required to solve the compatibility issue between pose estimation and bounding box detection~\cite{He:etal:iccv2017:mask:rcnn,Redmon:etal:cvpr16:yolo}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1608.08526" id="202">
-        <attvalues>
-          <attvalue for="5" value=" The third idea is to detect human poses from part detection and association~\cite{DBLP:conf/eccv/IqbalG16,DBLP:conf/nips/NewellHD17,Cao:etal:cvpr17:openpose,Martiez-Gonzalez:etal:iros2018:rpm,Mehta:etal:XNECT:TOG2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1611.05424" id="203">
-        <attvalues>
-          <attvalue for="5" value=" The third idea is to detect human poses from part detection and association~\cite{DBLP:conf/eccv/IqbalG16,DBLP:conf/nips/NewellHD17,Cao:etal:cvpr17:openpose,Martiez-Gonzalez:etal:iros2018:rpm,Mehta:etal:XNECT:TOG2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1506.01497" id="204">
-        <attvalues>
-          <attvalue for="5" value=" Developing a depth image-based multi-person 3D pose estimation from a well-established RGB image-based 2D method~\cite{DBLP:conf/cvpr/PapandreouZKTTB17,Ren:etal:pami2017:faster:rcnn,Cao:etal:cvpr17:openpose} is conceptually simple for two reasons: some schemes to handle multi-person detection can be shared; 3D information is partially available from the input~\cite{Haque:etal:eccv2016:itop,Xiong:etal:iccv2019:a2j}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1603.07076" id="205">
-        <attvalues>
-          <attvalue for="5" value=" Developing a depth image-based multi-person 3D pose estimation from a well-established RGB image-based 2D method~\cite{DBLP:conf/cvpr/PapandreouZKTTB17,Ren:etal:pami2017:faster:rcnn,Cao:etal:cvpr17:openpose} is conceptually simple for two reasons: some schemes to handle multi-person detection can be shared; 3D information is partially available from the input~\cite{Haque:etal:eccv2016:itop,Xiong:etal:iccv2019:a2j}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.06734" target="1405.0312" id="206">
-        <attvalues>
-          <attvalue for="5" value=" Although there are a decent amount of RGB datasets~\cite{DBLP:conf/eccv/LinMBHPRDZ14,DBLP:journals/pami/IonescuPOS14,DBLP:conf/cvpr/AndrilukaPGS14:mpii} in prior art, there are limited high-quality depth datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1404.1777" id="207">
-        <attvalues>
-          <attvalue for="5" value=" Unluckily, compared to content-based image retrieval \cite{wu2018unsupervised,babenko2014neural,radenovic2018fine,liu2016deep,caron2018deep}, IBSR is challenging due to the large appearance gap between 3D shapes and 2D images." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1711.02512" id="208">
-        <attvalues>
-          <attvalue for="5" value=" Unluckily, compared to content-based image retrieval \cite{wu2018unsupervised,babenko2014neural,radenovic2018fine,liu2016deep,caron2018deep}, IBSR is challenging due to the large appearance gap between 3D shapes and 2D images." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1807.05520" id="209">
-        <attvalues>
-          <attvalue for="5" value=" Unluckily, compared to content-based image retrieval \cite{wu2018unsupervised,babenko2014neural,radenovic2018fine,liu2016deep,caron2018deep}, IBSR is challenging due to the large appearance gap between 3D shapes and 2D images." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1804.04610" id="210">
-        <attvalues>
-          <attvalue for="5" value=" Our approach shows state-of-the-art performance on a recently released large-scale 3D-FUTURE \cite{fu20203d} repository, as well as three widely studied benchmarks, including Pix3D \cite{sun2018pix3d}, Stanford Cars \cite{KrauseStarkDengFei-Fei_3DRR2013}, and Comp Cars \cite{wang20183d}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1808.02201" id="211">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from images, such as surface normal, depth, and location filed, to bridge the gaps between 3D and 2D domains \cite{sun2018pix3d,huang2018holistic,wu2017marrnet,bansal2016marr,bachman1978data,grabner20183d,grabner2019location}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1604.01347" id="212">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from images, such as surface normal, depth, and location filed, to bridge the gaps between 3D and 2D domains \cite{sun2018pix3d,huang2018holistic,wu2017marrnet,bansal2016marr,bachman1978data,grabner20183d,grabner2019location}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1803.11493" id="213">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from images, such as surface normal, depth, and location filed, to bridge the gaps between 3D and 2D domains \cite{sun2018pix3d,huang2018holistic,wu2017marrnet,bansal2016marr,bachman1978data,grabner20183d,grabner2019location}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1908.02853" id="214">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from images, such as surface normal, depth, and location filed, to bridge the gaps between 3D and 2D domains \cite{sun2018pix3d,huang2018holistic,wu2017marrnet,bansal2016marr,bachman1978data,grabner20183d,grabner2019location}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1411.4734" id="215">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from 2D images is itself an open problem \cite{eigen2015predicting,wang2015designing,chen2016single,qi2018geonet,fu2018deep,wang2016surge}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1411.4958" id="216">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from 2D images is itself an open problem \cite{eigen2015predicting,wang2015designing,chen2016single,qi2018geonet,fu2018deep,wang2016surge}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1604.03901" id="217">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from 2D images is itself an open problem \cite{eigen2015predicting,wang2015designing,chen2016single,qi2018geonet,fu2018deep,wang2016surge}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1806.02446" id="218">
-        <attvalues>
-          <attvalue for="5" value="5D sketches from 2D images is itself an open problem \cite{eigen2015predicting,wang2015designing,chen2016single,qi2018geonet,fu2018deep,wang2016surge}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="2009.09633" id="219">
-        <attvalues>
-          <attvalue for="5" value=" Our approach shows state-of-the-art performance on a recently released large-scale 3D-FUTURE \cite{fu20203d} repository, as well as three widely studied benchmarks, including Pix3D \cite{sun2018pix3d}, Stanford Cars \cite{KrauseStarkDengFei-Fei_3DRR2013}, and Comp Cars \cite{wang20183d}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.12238" target="1806.04314" id="220">
-        <attvalues>
-          <attvalue for="5" value=" Our approach shows state-of-the-art performance on a recently released large-scale 3D-FUTURE \cite{fu20203d} repository, as well as three widely studied benchmarks, including Pix3D \cite{sun2018pix3d}, Stanford Cars \cite{KrauseStarkDengFei-Fei_3DRR2013}, and Comp Cars \cite{wang20183d}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="1903.12261" id="221">
-        <attvalues>
-          <attvalue for="5" value=" Our main contributions can be summarized as follows: (1) we propose a novel test-time \underline{F}FT-based \underline{RO}bust \underline{ST}atistics selection method (FROST) based on a codebook mapping FFT features to corruption-specific statistics; (2)&#09;we achieve model robustness to image corruptions at test-time, making it useful for mobile and robotic applications in the wild; (3)&#09;we improve accuracy against competitors on ImageNet-C \cite{hendrycks2019robustness}, especially in case of severe corruptions, while preserving accuracy on clean data; (4)&#09;our solution is portable to different models and architectures, optimizing statistics of feature normalization layers (Batch/Layer Normalization), with very limited storage requirements." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2307.11823" id="222">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies leveraged frequency spectra insights to improve model robustness \cite{Chen_2021_ICCV,yucel2023hybridaugment,long2022frequency}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2302.05155" id="223">
-        <attvalues>
-          <attvalue for="5" value=" Some methods focus on resolving shift of data statistics to improve model robustness, developing modules on top of feature normalization layers \cite{lim2023ttn,Schneider2020ImprovingRA,Khurana2021SITASI}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2108.08487" id="224">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies leveraged frequency spectra insights to improve model robustness \cite{Chen_2021_ICCV,yucel2023hybridaugment,long2022frequency}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2207.05382" id="225">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods often use cumbersome ensemble models \cite{Saikia_2021_ICCV} or formulate complex augmentation regimes \cite{Sun2022ASV,long2022frequency} at training time, with no possibility to adapt the models to test data." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="1409.0575" id="226">
-        <attvalues>
-          <attvalue for="5" value=" While the models have achieved impressive results on several benchmarks \cite{russakovsky2015imagenet, Krizhevsky2009LearningML}, recent works show that their performance is severely degraded when dealing with corrupted images." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="1707.07830" id="227">
-        <attvalues>
-          <attvalue for="5" value=" Object recognition using distorted image datasets can be posed as a dataset shift problem \cite{Sun2017ImprovingRO}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2110.13771" id="228">
-        <attvalues>
-          <attvalue for="5" value=" The most successful approaches make use of data augmentation \cite{wang2021augmax,autoaugment,Hendrycks2021PixMixDP,hendrycks2020augmix} and adversarial training \cite{bai2021transformers,long2022frequency,yucel2023hybridaugment}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2112.05135" id="229">
-        <attvalues>
-          <attvalue for="5" value=" The most successful approaches make use of data augmentation \cite{wang2021augmax,autoaugment,Hendrycks2021PixMixDP,hendrycks2020augmix} and adversarial training \cite{bai2021transformers,long2022frequency,yucel2023hybridaugment}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="1912.02781" id="230">
-        <attvalues>
-          <attvalue for="5" value=" The most successful approaches make use of data augmentation \cite{wang2021augmax,autoaugment,Hendrycks2021PixMixDP,hendrycks2020augmix} and adversarial training \cite{bai2021transformers,long2022frequency,yucel2023hybridaugment}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2111.05464" id="231">
-        <attvalues>
-          <attvalue for="5" value=" The most successful approaches make use of data augmentation \cite{wang2021augmax,autoaugment,Hendrycks2021PixMixDP,hendrycks2020augmix} and adversarial training \cite{bai2021transformers,long2022frequency,yucel2023hybridaugment}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2006.16971" id="232">
-        <attvalues>
-          <attvalue for="5" value=" Some methods focus on resolving shift of data statistics to improve model robustness, developing modules on top of feature normalization layers \cite{lim2023ttn,Schneider2020ImprovingRA,Khurana2021SITASI}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2112.02355" id="233">
-        <attvalues>
-          <attvalue for="5" value=" Some methods focus on resolving shift of data statistics to improve model robustness, developing modules on top of feature normalization layers \cite{lim2023ttn,Schneider2020ImprovingRA,Khurana2021SITASI}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.14335" target="2103.16241" id="234">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods often use cumbersome ensemble models \cite{Saikia_2021_ICCV} or formulate complex augmentation regimes \cite{Sun2022ASV,long2022frequency} at training time, with no possibility to adapt the models to test data." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2311.15260" id="235">
-        <attvalues>
-          <attvalue for="5" value=" The synthesis of high-quality 3D scenes from sparse image data has emerged as a crucial research area in computer vision and graphics, driven by applications such as virtual reality, augmented reality, autonomous driving, and robotics \cite{tonderski2024neurad,pan2023uniocc,zhou2024drivinggaussian}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2306.09117" id="236">
-        <attvalues>
-          <attvalue for="5" value=" The synthesis of high-quality 3D scenes from sparse image data has emerged as a crucial research area in computer vision and graphics, driven by applications such as virtual reality, augmented reality, autonomous driving, and robotics \cite{tonderski2024neurad,pan2023uniocc,zhou2024drivinggaussian}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2312.07920" id="237">
-        <attvalues>
-          <attvalue for="5" value=" The synthesis of high-quality 3D scenes from sparse image data has emerged as a crucial research area in computer vision and graphics, driven by applications such as virtual reality, augmented reality, autonomous driving, and robotics \cite{tonderski2024neurad,pan2023uniocc,zhou2024drivinggaussian}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2008.02268" id="238">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods like Neural Radiance Fields (NeRF) \cite{martin2021nerf} have shown impressive performance in generating detailed 3D scenes and novel views, they rely on implicit volumetric representations that demand substantial computational resources for pixel-level ray tracing and optimization \cite{mildenhall2021nerf,wang2023sparsenerf,truong2023sparf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2003.08934" id="239">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods like Neural Radiance Fields (NeRF) \cite{martin2021nerf} have shown impressive performance in generating detailed 3D scenes and novel views, they rely on implicit volumetric representations that demand substantial computational resources for pixel-level ray tracing and optimization \cite{mildenhall2021nerf,wang2023sparsenerf,truong2023sparf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2303.16196" id="240">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods like Neural Radiance Fields (NeRF) \cite{martin2021nerf} have shown impressive performance in generating detailed 3D scenes and novel views, they rely on implicit volumetric representations that demand substantial computational resources for pixel-level ray tracing and optimization \cite{mildenhall2021nerf,wang2023sparsenerf,truong2023sparf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2211.11738" id="241">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods like Neural Radiance Fields (NeRF) \cite{martin2021nerf} have shown impressive performance in generating detailed 3D scenes and novel views, they rely on implicit volumetric representations that demand substantial computational resources for pixel-level ray tracing and optimization \cite{mildenhall2021nerf,wang2023sparsenerf,truong2023sparf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2308.04079" id="242">
-        <attvalues>
-          <attvalue for="5" value=" Recently, 3D Gaussian Splatting (3DGS) \cite{kerbl20233d,li2024dngaussian,lu2024scaffold} has emerged as a promising alternative, offering explicit representations for 3D scene synthesis that bypass the computational burden of voxel-based rendering of NeRF." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2403.06912" id="243">
-        <attvalues>
-          <attvalue for="5" value=" By utilizing Gaussian primitives, 3DGS facilitates faster and more efficient scene reconstruction and novel view synthesis \cite{li2024dngaussian,zhang2024pixel,yu2024mip}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2312.00109" id="244">
-        <attvalues>
-          <attvalue for="5" value=" Recently, 3D Gaussian Splatting (3DGS) \cite{kerbl20233d,li2024dngaussian,lu2024scaffold} has emerged as a promising alternative, offering explicit representations for 3D scene synthesis that bypass the computational burden of voxel-based rendering of NeRF." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2403.15530" id="245">
-        <attvalues>
-          <attvalue for="5" value=" By utilizing Gaussian primitives, 3DGS facilitates faster and more efficient scene reconstruction and novel view synthesis \cite{li2024dngaussian,zhang2024pixel,yu2024mip}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2311.16493" id="246">
-        <attvalues>
-          <attvalue for="5" value=" By utilizing Gaussian primitives, 3DGS facilitates faster and more efficient scene reconstruction and novel view synthesis \cite{li2024dngaussian,zhang2024pixel,yu2024mip}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08613" target="2409.03213" id="247">
-        <attvalues>
-          <attvalue for="5" value=" In scenarios with sparse viewpoints and limited input images, the initial point clouds often lack sufficient detail and accuracy, constraining the performance of 3DGS and limiting its applicability to sparse-viewpoint scene reconstruction \cite{xiong2023sparsegs,chen2024optimizing,fan2024instantsplat}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2003.08934" id="248">
-        <attvalues>
-          <attvalue for="5" value=" Traditional Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf,barron2021mip,Barron_2023_ICCV} have shown strong performance in NVS, but their pixel-level ray rendering is computationally intensive and not well-suited for scenarios with sparse input data, requiring substantial resources and processing time." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2103.13415" id="249">
-        <attvalues>
-          <attvalue for="5" value=" Traditional Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf,barron2021mip,Barron_2023_ICCV} have shown strong performance in NVS, but their pixel-level ray rendering is computationally intensive and not well-suited for scenarios with sparse input data, requiring substantial resources and processing time." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2304.06706" id="250">
-        <attvalues>
-          <attvalue for="5" value=" Traditional Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf,barron2021mip,Barron_2023_ICCV} have shown strong performance in NVS, but their pixel-level ray rendering is computationally intensive and not well-suited for scenarios with sparse input data, requiring substantial resources and processing time." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2308.04079" id="251">
-        <attvalues>
-          <attvalue for="5" value=" Leveraging the smooth, differentiable properties of Gaussian distributions, 3DGS enables rapid rasterization by projecting 3D Gaussians onto 2D image planes, supporting efficient rendering and interpolation \cite{kerbl20233d,tang2023dreamgaussian,yi2024gaussiandreamer}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2309.16653" id="252">
-        <attvalues>
-          <attvalue for="5" value=" Leveraging the smooth, differentiable properties of Gaussian distributions, 3DGS enables rapid rasterization by projecting 3D Gaussians onto 2D image planes, supporting efficient rendering and interpolation \cite{kerbl20233d,tang2023dreamgaussian,yi2024gaussiandreamer}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2310.08529" id="253">
-        <attvalues>
-          <attvalue for="5" value=" Leveraging the smooth, differentiable properties of Gaussian distributions, 3DGS enables rapid rasterization by projecting 3D Gaussians onto 2D image planes, supporting efficient rendering and interpolation \cite{kerbl20233d,tang2023dreamgaussian,yi2024gaussiandreamer}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2312.03203" id="254">
-        <attvalues>
-          <attvalue for="5" value=" This limitation is particularly evident when dealing with extremely thin lines, where it can result in inaccuracies that hinder the precise capture and reproduction of slender structures and small features, thereby compromising the overall visual realism and detail fidelity of the scene \cite{zhou2024feature,yu2024mip}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2311.16493" id="255">
-        <attvalues>
-          <attvalue for="5" value=" To impose size constraints on the 3D Gaussian primitives, we introduce a 3D smoothing filter \cite{yu2024mip}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2403.15530" id="256">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in practical applications, 3D Gaussian Splatting (3DGS) requires densely sampled multi-view scenes to achieve optimal results \cite{zhang2024pixel,fan2024instantsplat,li2024dngaussian}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2403.06912" id="257">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in practical applications, 3D Gaussian Splatting (3DGS) requires densely sampled multi-view scenes to achieve optimal results \cite{zhang2024pixel,fan2024instantsplat,li2024dngaussian}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03213" target="2209.14988" id="258">
-        <attvalues>
-          <attvalue for="5" value=" For the optimization of the 3D Gaussian primitives, we employ score distillation sampling (SDS) loss \cite{poole2022dreamfusion} to integrate 3DGS with 2D diffusion, incorporating depth prior information to constrain the positions and sizes of the 3D Gaussian primitives." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02837" target="1610.08120" id="259">
-        <attvalues>
-          <attvalue for="5" value=" Some segmentation techniques for fruit detection are discussed in~\cite{SucJam16}, and some works are particularly tuned for a specific type of fruits such as grapes, mangoes, apples, kiwifruit, peaches, etc Fruit detection, tracking, and 3D reconstruction for crop mapping and yield estimation was developed in~\cite{MooCha10}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02837" target="2212.13844" id="260">
-        <attvalues>
-          <attvalue for="5" value=" More importantly the depth accuracy of Kinect version 2 is almost constant over different distances~\cite{YanZhaDon15}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02837" target="1507.06821" id="261">
-        <attvalues>
-          <attvalue for="5" value=" The maximum range of D435 varies depending on calibration, scene, lighting condition, etc The emergence of these low-cost RGB-D sensors motivated many computer vision researchers to use them for many applications such as object recognition~\cite{PraAld15}\cite{AndJos15}, object detection~\cite{XiaYun17}, pose estimation~\cite{SchBeh17}\cite{NguKoJeo15}\cite{ChoTagTuz12}, 3D reconstruction~\cite{IzadiKim11}, simultaneous localization and mapping (SLAM)~\cite{EndCre14}, etc However, to the best of our knowledge, only the work in~\cite{Mas13} gave attention to the application of computer vision for mushrooms harvesting." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="1703.06870" id="262">
-        <attvalues>
-          <attvalue for="5" value=" Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="2003.10152" id="263">
-        <attvalues>
-          <attvalue for="5" value=" Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="1903.12174" id="264">
-        <attvalues>
-          <attvalue for="5" value=" Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="2210.03551" id="265">
-        <attvalues>
-          <attvalue for="5" value=" Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="2110.07057" id="266">
-        <attvalues>
-          <attvalue for="5" value=" Recently, considerable work has been conducted in instance segmentation due to its wide scope of application~\cite{he2017mask,wang2020solov2,chen2019tensormask,chen2022instance}, such as autonomous driving~\cite{de2017semantic}, medical diagnosis~\cite{kumar2017dataset} and agricultural phenotyping~\cite{scharr2016leaf,chen2022high}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.04887" target="1801.00868" id="267">
-        <attvalues>
-          <attvalue for="5" value=" For match-based approaches, such as Average Precision (AP)~\cite{caicedo2019nucleus} and Panoptic Quality (PQ)~\cite{kirillov2019panoptic}, the score will change abruptly at the match threshold." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1703.06870" id="268">
-        <attvalues>
-          <attvalue for="5" value=" As a paradigm, Mask-RCNN~\cite{mrcnn} refines bounding boxes obtained from Region-based Convolutional Neural Networks (R-CNN)~\cite{rcnn,frcnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1311.2524" id="269">
-        <attvalues>
-          <attvalue for="5" value=" As a paradigm, Mask-RCNN~\cite{mrcnn} refines bounding boxes obtained from Region-based Convolutional Neural Networks (R-CNN)~\cite{rcnn,frcnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1506.01497" id="270">
-        <attvalues>
-          <attvalue for="5" value=" As a paradigm, Mask-RCNN~\cite{mrcnn} refines bounding boxes obtained from Region-based Convolutional Neural Networks (R-CNN)~\cite{rcnn,frcnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1806.03535" id="271">
-        <attvalues>
-          <attvalue for="5" value=" A finer polygon representation approach was proposed by~\cite{StarDist} to reduce false suppressions." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1604.02677" id="272">
-        <attvalues>
-          <attvalue for="5" value=" DCAN~\cite{dcan} predicts the object boundary explicitly and groups connected pixels that are separated by these boundaries as instances." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1909.01616" id="273">
-        <attvalues>
-          <attvalue for="5" value=" Graph partition based on pixel-pair affinity~\cite{ssap} and the watershed transform~\cite{deepWatershed} are more robust in terms of grouping, but the pipeline as a whole relies heavily on post-processing, with the learning model only optimized with intermediate results." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1611.08303" id="274">
-        <attvalues>
-          <attvalue for="5" value=" Graph partition based on pixel-pair affinity~\cite{ssap} and the watershed transform~\cite{deepWatershed} are more robust in terms of grouping, but the pipeline as a whole relies heavily on post-processing, with the learning model only optimized with intermediate results." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="2004.09821" id="275">
-        <attvalues>
-          <attvalue for="5" value=" Recent research~\cite{disEmb,cosEmb,rnnEmb,impemb} introduces pixel embedding for grouping." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="1712.08273" id="276">
-        <attvalues>
-          <attvalue for="5" value=" Recent research~\cite{disEmb,cosEmb,rnnEmb,impemb} introduces pixel embedding for grouping." />
-        </attvalues>
-      </edge>
-      <edge source="2210.03551" target="2007.06660" id="277">
-        <attvalues>
-          <attvalue for="5" value=" Recent research~\cite{disEmb,cosEmb,rnnEmb,impemb} introduces pixel embedding for grouping." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1605.06211" id="278">
-        <attvalues>
-          <attvalue for="5" value=" The task of semantic segmentation~\cite{longFullyConvolutionalNetworks2015} aims at, given an input image, performing pixel-wise classification over a predefined set of categories." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1505.04597" id="279">
-        <attvalues>
-          <attvalue for="5" value=" As in many other dense prediction problems, the end-to-end convolutional neural networks (CNN)-based fully supervised approach to this task has become the de facto standard to solve it, leading to robustly performing models~\cite{ronnebergerUNetConvolutionalNetworks2015} at the expense of a large amount of human annotations." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1806.00104" id="280">
-        <attvalues>
-          <attvalue for="5" value=" Advances on multi-view human pose estimation were possible by leveraging various wide baseline datasets over RGB~\cite{kazemiMultiviewBodyPart2013,yaoMONETMultiviewSemiSupervised2019,ionescuHuman36MLarge2014} and depth~\cite{haque2016viewpoint} images of both groups~\cite{jooPanopticStudioMassively2015} and individuals." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1603.07076" id="281">
-        <attvalues>
-          <attvalue for="5" value=" Advances on multi-view human pose estimation were possible by leveraging various wide baseline datasets over RGB~\cite{kazemiMultiviewBodyPart2013,yaoMONETMultiviewSemiSupervised2019,ionescuHuman36MLarge2014} and depth~\cite{haque2016viewpoint} images of both groups~\cite{jooPanopticStudioMassively2015} and individuals." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1612.02709" id="282">
-        <attvalues>
-          <attvalue for="5" value=" A few works focus on cross-view semantic transfer, with an unsupervised transfer of the semantic annotations to new label-free views, eg ground to aerial views~\cite{zhaiPredictingGroundLevelScene2017} or among distinct vehicle-mounted cameras~\cite{Coors2019THREEDV} in close-baseline footage from~\cite{dosovitskiy2017carla}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1711.03938" id="283">
-        <attvalues>
-          <attvalue for="5" value=" The House3D~\cite{wuBuildingGeneralizableAgents2018}, Gibson~\cite{xiaGibsonEnvRealWorld2018} and CARLA~\cite{dosovitskiy2017carla} environments are some relevant examples, although only CARLA, being fully virtual, could yield high object densities via its API." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1904.04571" id="284">
-        <attvalues>
-          <attvalue for="5" value=" They both provide body part~\cite{ionescuHuman36MLarge2014,pumarola20193dpeople} or clothing~\cite{pumarola20193dpeople} segmentations, but~\cite{pumarola20193dpeople} has immutable 2D backgrounds, and they are both restricted to single subjects and thus limited in the severity of the occlusions and subject size variation across views." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1703.08866" id="285">
-        <attvalues>
-          <attvalue for="5" value=" Several recent papers~\cite{riemenschneiderLearningWhereClassify2014,maMultiviewDeepLearning2017,dai3DMVJoint3DMultiView2018} leverage the spatial consistencies in temporal sequences of RGB or RGB-D images with small relative baselines among them to address semantic segmentation of either 2D images or their reconstructed 3D representations." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1803.10409" id="286">
-        <attvalues>
-          <attvalue for="5" value=" Several recent papers~\cite{riemenschneiderLearningWhereClassify2014,maMultiviewDeepLearning2017,dai3DMVJoint3DMultiView2018} leverage the spatial consistencies in temporal sequences of RGB or RGB-D images with small relative baselines among them to address semantic segmentation of either 2D images or their reconstructed 3D representations." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1702.04405" id="287">
-        <attvalues>
-          <attvalue for="5" value=" The raw sequences of the NYUv2~\cite{silbermanIndoorSegmentationSupport2012}, Camvid, ETHZ RueMonge 2014~\cite{riemenschneiderLearningWhereClassify2014} or the ScanNet~\cite{daiScanNetRichlyAnnotated3D2017} datasets are commonly used to achieve this." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1801.02209" id="288">
-        <attvalues>
-          <attvalue for="5" value=" The House3D~\cite{wuBuildingGeneralizableAgents2018}, Gibson~\cite{xiaGibsonEnvRealWorld2018} and CARLA~\cite{dosovitskiy2017carla} environments are some relevant examples, although only CARLA, being fully virtual, could yield high object densities via its API." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1808.10654" id="289">
-        <attvalues>
-          <attvalue for="5" value=" The House3D~\cite{wuBuildingGeneralizableAgents2018}, Gibson~\cite{xiaGibsonEnvRealWorld2018} and CARLA~\cite{dosovitskiy2017carla} environments are some relevant examples, although only CARLA, being fully virtual, could yield high object densities via its API." />
-        </attvalues>
-      </edge>
-      <edge source="2205.15452" target="1906.03560" id="290">
-        <attvalues>
-          <attvalue for="5" value=" This was shown in~\cite{panCrossviewSemanticSegmentation2019} for close baseline setups, proposing a multi-view semantic fusion scheme from up to 8 input views onto a new virtual zenithal view." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="1706.03762" id="291">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by the recent success of Transformers\,\cite{vaswani2017attention} in NLP, numerous models have been developed for various vision tasks, especially in recognition and detection." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2005.12872" id="292">
-        <attvalues>
-          <attvalue for="5" value="\,\cite{carion2020end} propose the Detection Transformers\,(DETR) to replace the meticulously designed components with a transformer encoder and decoder architecture, which serves as a neck component to bridge a CNN body for feature extraction and a detector head for prediction." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2010.11929" id="293">
-        <attvalues>
-          <attvalue for="5" value="\,\cite{dosovitskiy2021image} show that a fully-transformer backbone without any convolutional layers, Vision Transformer (ViT), achieves state-of-the-art results on image classification benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2106.00666" id="294">
-        <attvalues>
-          <attvalue for="5" value="\,\cite{fang2021you} propose the YOLOS model by appending the detection tokens $[\mathtt{DET}]$ to the patch tokens $[\mathtt{PATCH}]$\, (see Figure \ref{fig:pipelines}(b)), where $[\mathtt{DET}]$ tokens are learnable embeddings to specify different objects to detect." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2103.14030" id="295">
-        <attvalues>
-          <attvalue for="5" value=" Thus, we can integrate the Swin Transformer~\cite{liu2021swin} backbone with RAM to be an object detector and obtain high scalability using its local attention mechanism with linear complexity." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="1912.05992" id="296">
-        <attvalues>
-          <attvalue for="5" value=" For effective multi-task learning, ViDT+ equips efficient feature pyramid layers on top of its body for multi-scale feature fusion, and leverages two additional training losses eg, IoU-aware loss\,\cite{wu2020iou} and token labeling loss\,\cite{jiang2021all}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2104.10858" id="297">
-        <attvalues>
-          <attvalue for="5" value=" For effective multi-task learning, ViDT+ equips efficient feature pyramid layers on top of its body for multi-scale feature fusion, and leverages two additional training losses eg, IoU-aware loss\,\cite{wu2020iou} and token labeling loss\,\cite{jiang2021all}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2104.06399" id="298">
-        <attvalues>
-          <attvalue for="5" value=" It can be extended to be an end-to-end architecture for multi-task learning, and easily combined with ViT variants such as CoaT\,\cite{xu2021co} and PVT-v2\,\cite{wang2021pvtv2} other than the Swin Transformer." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="1405.0312" id="299">
-        <attvalues>
-          <attvalue for="5" value=" } Extensive experiments on Microsoft COCO~\cite{lin2014microsoft} show that ViDT is highly scalable even for large ViT models, such as Swin-base with 0." />
-        </attvalues>
-      </edge>
-      <edge source="2204.07962" target="2110.03921" id="300">
-        <attvalues>
-          <attvalue for="5" value=" } This work is an extensive version of our ICLR 2022\,\cite{song2022vidt}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1902.07381" id="301">
-        <attvalues>
-          <attvalue for="5" value=" Such feature extractors are used in several applications such as pose estimation \cite{godard2018digging}, Visual Place Recognition (VPR)\cite{garg2020delta}\cite{garg2019look}, object detection \cite{zhao2019object} and segmentation \cite{badrinarayanan2017segnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1406.2283" id="302">
-        <attvalues>
-          <attvalue for="5" value=" The supervised learning methods \cite{eigen2014depth} \cite{cao2018estimating} necessitate explicit availability of ground truth information (Laser or LiDAR range data) which may not always be feasible in many real-world scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1605.02305" id="303">
-        <attvalues>
-          <attvalue for="5" value=" The supervised learning methods \cite{eigen2014depth} \cite{cao2018estimating} necessitate explicit availability of ground truth information (Laser or LiDAR range data) which may not always be feasible in many real-world scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1803.02276" id="304">
-        <attvalues>
-          <attvalue for="5" value=" This is overcome by the unsupervised methods \cite{zhou2017unsupervised} \cite{yin2018geonet} \cite{babu2018undemon} that harness the spatial and/or temporal consistency present in image sequences to extract the underlying geometry to be used as the implicit supervision signal required for training the models." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1604.01685" id="305">
-        <attvalues>
-          <attvalue for="5" value=" Many of these methods were shown to provide very impressive results on several popular datasets such as KITTI \cite{geiger2013vision} and Cityscapes \cite{cordts2016cityscapes} containing only day-time images." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1803.07702" id="306">
-        <attvalues>
-          <attvalue for="5" value=" For instance, authors in \cite{im2018robust} exploit the inherent motion component in burst shot (several successive shots with varying camera settings, also known as ``auto-bracketing'') to estimate depth from images taken under low-light condition." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1812.08156" id="307">
-        <attvalues>
-          <attvalue for="5" value=" \cite{zhu2019unsupervised} present a deep learning based method for estimating motion flow, depth and pose from images obtained from event cameras that return a time-stamped event tuple whenever a change in pixel intensity is detected." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1703.10593" id="308">
-        <attvalues>
-          <attvalue for="5" value=" \item We also propose an image translation-based method for night-time depth estimation by using a combination of an image translating network (eg CycleGAN \cite{zhu2017unpaired}) and a standard day-time depth estimation network (such as \cite{godard2018digging}) in cascade." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1804.04732" id="309">
-        <attvalues>
-          <attvalue for="5" value=" One possible solution will be to apply image-to-image translation methods, such as Cycle-GAN \cite{zhu2017unpaired} or MUNIT \cite{huang2018multimodal}, to map night-time images to day-time images and then use a pre-trained day-time depth model to estimate depth from these translated images." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1809.09767" id="310">
-        <attvalues>
-          <attvalue for="5" value=" For instance, the authors in \cite{anoosheh2019night} use night-to-day image translation for solving the place recognition problem required for localization." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1904.01870" id="311">
-        <attvalues>
-          <attvalue for="5" value=" Since the domain features are adapted through adversarial learning, this method is termed as ``Adversarial Domain Feature Adaptation (ADFA)'' method to distinguish it from other methods that attempt to adapt depth predictions directly \cite{atapour2018real} \cite{zhao2019geometry} \cite{nath2018adadepth}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1803.01599" id="312">
-        <attvalues>
-          <attvalue for="5" value=" Since the domain features are adapted through adversarial learning, this method is termed as ``Adversarial Domain Feature Adaptation (ADFA)'' method to distinguish it from other methods that attempt to adapt depth predictions directly \cite{atapour2018real} \cite{zhao2019geometry} \cite{nath2018adadepth}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1702.05464" id="313">
-        <attvalues>
-          <attvalue for="5" value=" This is achieved by using an adversarial discriminative learning \cite{tzeng2017adversarial} that uses day-time encoded features as the reference." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1611.07004" id="314">
-        <attvalues>
-          <attvalue for="5" value=" Patch{GAN} networks \cite{isola2017image} \cite{vankadari2019unsupervised} have been shown to provide superior performance compared to conventional GANs by capturing high frequency local structural information and hence, form a natural choice of GAN architecture for the proposed method." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1806.01260" id="315">
-        <attvalues>
-          <attvalue for="5" value=" \item We also propose an image translation-based method for night-time depth estimation by using a combination of an image translating network (eg CycleGAN \cite{zhu2017unpaired}) and a standard day-time depth estimation network (such as \cite{godard2018digging}) in cascade." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="2006.05700" id="316">
-        <attvalues>
-          <attvalue for="5" value=" Such feature extractors are used in several applications such as pose estimation \cite{godard2018digging}, Visual Place Recognition (VPR)\cite{garg2020delta}\cite{garg2019look}, object detection \cite{zhao2019object} and segmentation \cite{badrinarayanan2017segnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1807.05511" id="317">
-        <attvalues>
-          <attvalue for="5" value=" Such feature extractors are used in several applications such as pose estimation \cite{godard2018digging}, Visual Place Recognition (VPR)\cite{garg2020delta}\cite{garg2019look}, object detection \cite{zhao2019object} and segmentation \cite{badrinarayanan2017segnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.01402" target="1511.00561" id="318">
-        <attvalues>
-          <attvalue for="5" value=" Such feature extractors are used in several applications such as pose estimation \cite{godard2018digging}, Visual Place Recognition (VPR)\cite{garg2020delta}\cite{garg2019look}, object detection \cite{zhao2019object} and segmentation \cite{badrinarayanan2017segnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2205.01917" id="319">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2106.04560" id="320">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2111.09883" id="321">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2106.04803" id="322">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2112.03857" id="323">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2106.08322" id="324">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2106.09018" id="325">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2205.05740" id="326">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2204.08721" id="327">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2204.02251" id="328">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2203.01509" id="329">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2104.00678" id="330">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2104.13586" id="331">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2106.02036" id="332">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="1905.09035" id="333">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2201.08383" id="334">
-        <attvalues>
-          <attvalue for="5" value=" Most existing computer vision techniques aim at understanding the visual world from a third-person perspective \cite{yu2022coca,zhai2022scaling,liu2022swin,dai2021coatnet,liu2022swin,li2022grounded,dai2021dynamic,xu2021end,ran2022surface,wang2022multimodal,wang2022rbgnet,vu2022softgroup,liu2021group,duan2022revisiting, wei2022masked, yan2022multiview,girdhar2021anticipative,furnari2019would,wu2022memvit}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2110.07058" id="335">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="1812.04558" id="336">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="1908.08498" id="337">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="2104.07905" id="338">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.16606" target="1904.09882" id="339">
-        <attvalues>
-          <attvalue for="5" value=" \\ Egocentric video understanding offers a variety of tasks centered around recognizing visual experience in the past, present, and future \cite{grauman2022ego4d,damen2016you,nagarajan2019grounded,kazakos2019epic,li2021ego,del2016summarization,ng2020you2me,yonetani2016recognizing,abu2018will,girdhar2021anticipative}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1904.01390" id="340">
-        <attvalues>
-          <attvalue for="5" value=" The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1805.04176" id="341">
-        <attvalues>
-          <attvalue for="5" value=" The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1902.06701" id="342">
-        <attvalues>
-          <attvalue for="5" value=" The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1901.03554" id="343">
-        <attvalues>
-          <attvalue for="5" value=" The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1810.02797" id="344">
-        <attvalues>
-          <attvalue for="5" value=" The depth map from images can be estimated in various ways like structure from motion \cite{nister2005preemptive}, multi-view stereo \cite{seitz2006comparison}, monocular methods \cite{saxena2006learning}, single-image methods \cite{saxena20083}, etc The deep learning and convolutional neural networks (CNNs) based methods perform outstanding in most of the problems of computer vision such as image classification \cite{alexnet}, facial micro-expression recognition \cite{reddy2019spontaneous}, face anti-spoofing \cite{nagpal2018performance}, hyper-spectral image classification \cite{roy2019hybridsn}, image-to-image transformation \cite{kancharagunta2019csgan}, colon cancer nuclei classification \cite{basha2018rccnet}, etc Inspired from the success of deep learning, several researchers also tried to utilize the CNN for the depth prediction, specially in monocular imaging conditions." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1604.05383" id="345">
-        <attvalues>
-          <attvalue for="5" value=" These approaches are classified mainly in three categories namely learning-based stereo \cite{zhou2016learning}, \cite{zbontar2016stereo}, supervised single view depth estimation \cite{eigen2014depth}, \cite{liu2016learning}, and unsupervised depth estimation \cite{garg2016unsupervised}, \cite{godard2017unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1510.05970" id="346">
-        <attvalues>
-          <attvalue for="5" value=" These approaches are classified mainly in three categories namely learning-based stereo \cite{zhou2016learning}, \cite{zbontar2016stereo}, supervised single view depth estimation \cite{eigen2014depth}, \cite{liu2016learning}, and unsupervised depth estimation \cite{garg2016unsupervised}, \cite{godard2017unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1406.2283" id="347">
-        <attvalues>
-          <attvalue for="5" value=" \cite{zhou2017unsupervised} utilized the concepts of unsupervised image depth estimation proposed in \cite{eigen2014depth} and \cite{godard2017unsupervised} to tackle the monocular depth and camera motion estimation in unstructured video sequences in unsupervised learning framework." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1502.07411" id="348">
-        <attvalues>
-          <attvalue for="5" value=" These approaches are classified mainly in three categories namely learning-based stereo \cite{zhou2016learning}, \cite{zbontar2016stereo}, supervised single view depth estimation \cite{eigen2014depth}, \cite{liu2016learning}, and unsupervised depth estimation \cite{garg2016unsupervised}, \cite{godard2017unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1603.04992" id="349">
-        <attvalues>
-          <attvalue for="5" value=" used auto-encoder deep CNN to predict the inverse depth map (eg disparity) from left image \cite{garg2016unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1609.03677" id="350">
-        <attvalues>
-          <attvalue for="5" value=" \cite{zhou2017unsupervised} utilized the concepts of unsupervised image depth estimation proposed in \cite{eigen2014depth} and \cite{godard2017unsupervised} to tackle the monocular depth and camera motion estimation in unstructured video sequences in unsupervised learning framework." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1704.07813" id="351">
-        <attvalues>
-          <attvalue for="5" value=" \cite{zhou2017unsupervised} utilized the concepts of unsupervised image depth estimation proposed in \cite{eigen2014depth} and \cite{godard2017unsupervised} to tackle the monocular depth and camera motion estimation in unstructured video sequences in unsupervised learning framework." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06324" target="1802.05522" id="352">
-        <attvalues>
-          <attvalue for="5" value=" In one of the recent work, the 3D loss such as photometric quality of frame reconstructions is combined with 2D loss such as pixel-wise or gradient-based loss for learning the depth and ego-motion from monocular video in unsupervised manner \cite{mahjourian2018unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2209.14988" id="353">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, the generative approach, which leverages large-scale 2D diffusion models~\cite{dreamfusion, make3d}, introduces its own set of challenges." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2305.02463" id="354">
-        <attvalues>
-          <attvalue for="5" value=" Numerous attempts~\cite{shap-e, point-e, diffTF3D} have been made to train 3D diffusion models on relatively small 3D datasets, condition on textual or visual prompts; Yet, these endeavors often fall short in creating 3D objects with structural integrity and textural consistency." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2205.06267" id="355">
-        <attvalues>
-          <attvalue for="5" value=" However, this often leads to blurred 3D outputs due to the neglect of unseen elements, resulting in excessively blurred 3D objects~\cite{topology3d, pixelnerf} as most views remain unseen." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2303.11328" id="356">
-        <attvalues>
-          <attvalue for="5" value=" Efforts to develop 3D-aware 2D diffusion models~\cite{zero123, zero123++, zeronvs, wonder3D, magic123, anything3d, dtc123, dreamcraft3d} involve fine-tuning 2D models with camera transformation modeling on 3D datasets~\cite{objaverse, objaverseXL}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2212.08051" id="357">
-        <attvalues>
-          <attvalue for="5" value=" Efforts to develop 3D-aware 2D diffusion models~\cite{zero123, zero123++, zeronvs, wonder3D, magic123, anything3d, dtc123, dreamcraft3d} involve fine-tuning 2D models with camera transformation modeling on 3D datasets~\cite{objaverse, objaverseXL}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2308.04079" id="358">
-        <attvalues>
-          <attvalue for="5" value=" In the coarse phase, we leverage 3D Gaussian splatting~\cite{3dgs} to swiftly create basic geometry and textures." />
-        </attvalues>
-      </edge>
-      <edge source="2409.12193" target="2308.05371" id="359">
-        <attvalues>
-          <attvalue for="5" value=" Here, we employ FlexiCubes~\cite{flexicubes}, an advanced differentiable isosurface technique, to refine the geometry." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1703.06241" id="360">
-        <attvalues>
-          <attvalue for="5" value=" Various works \cite{delage2006dynamic,hedau2009recovering,lee2017roomnet,zhang2014panocontext} have been developed for room layout estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1609.09270" id="361">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, Xu \etal \cite{xu2017pano2cad} estimated the room layout and the pose of objects by detecting surface normal orientations." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1811.11977" id="362">
-        <attvalues>
-          <attvalue for="5" value=" Yang \etal \cite{yang2019dula} performed semantic segmentation of floor plan segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1903.08094" id="363">
-        <attvalues>
-          <attvalue for="5" value=" Fernandez-Labrador \etal \cite{fernandez2020corners} and Zou \etal \cite{zou2018layoutnet} predicted the probability map of boundaries and corners as 2D image, while Sun \etal \cite{SunHSC19} predicted as 1D vector." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1803.08999" id="364">
-        <attvalues>
-          <attvalue for="5" value=" Fernandez-Labrador \etal \cite{fernandez2020corners} and Zou \etal \cite{zou2018layoutnet} predicted the probability map of boundaries and corners as 2D image, while Sun \etal \cite{SunHSC19} predicted as 1D vector." />
-        </attvalues>
-      </edge>
-      <edge source="2009.02857" target="1901.03861" id="365">
-        <attvalues>
-          <attvalue for="5" value=" \ref{fig:qualitative}, our method performs better in accuracy than HorizonNet \cite{SunHSC19}, which is the current state-of-the-art method." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="2203.12979" id="366">
-        <attvalues>
-          <attvalue for="5" value=" The recently proposed GoMatch~\cite{zhou2022geometry} represents significant progress in descriptor-free 2D-3D keypoints matching, achieving reasonable matching performance on a variety of real-world datasets~\cite{li2018megadepth, shotton2013scene, kendall2015posenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="1506.02753" id="367">
-        <attvalues>
-          <attvalue for="5" value=" These methods are susceptible to privacy attacks~\cite{dosovitskiy2016inverting, dosovitskiy2016generating, chelani2021privacy, pan2023privacy} and necessitate computationally expensive model maintenance and descriptor update procedures~\cite{zhou2022geometry} when incorporating new descriptors or points into the model." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="1909.06216" id="368">
-        <attvalues>
-          <attvalue for="5" value=" Other methods~\cite{li2020hierarchical, brachmann2021dsacstar} directly learn a function that maps 2D pixels to 3D coordinates without explicitly storing the 3D scene." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="2112.12785" id="369">
-        <attvalues>
-          <attvalue for="5" value=" Additionally, \cite{ng2022ninjadesc} introduces an adversarial learning framework to develop content-concealing descriptors that prevent privacy leakage." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="2007.14628" id="370">
-        <attvalues>
-          <attvalue for="5" value=" Recently, researchers~\cite{campbell2020solving, liu2020learning} have begun exploring deep learning techniques for cross-domain direct 2D-3D matching and pose estimation without visual descriptors, showcasing the potential of descriptor-free matching through differentiable geometric optimization." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="1804.00607" id="371">
-        <attvalues>
-          <attvalue for="5" value=" The recently proposed GoMatch~\cite{zhou2022geometry} represents significant progress in descriptor-free 2D-3D keypoints matching, achieving reasonable matching performance on a variety of real-world datasets~\cite{li2018megadepth, shotton2013scene, kendall2015posenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="1911.11763" id="372">
-        <attvalues>
-          <attvalue for="5" value=" While GoMatch attains reasonable accuracy, its performance still significantly lags behind its descriptor-based counterparts~\cite{sarlin2020superglue, sarlin2019coarse, sattler2016efficient}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.12547" target="2204.11700" id="373">
-        <attvalues>
-          <attvalue for="5" value=" Taking inspiration from~\cite{shi2022clustergnn}, we employ a cluster-based transformer to constrain information flow within local clusters." />
-        </attvalues>
-      </edge>
-      <edge source="2302.04815" target="1603.06937" id="374">
-        <attvalues>
-          <attvalue for="5" value=" in \cite{hourglass} is one of the first compelling deep learning-based approaches to HPE, as classical approaches dominated HPE literature prior to it." />
-        </attvalues>
-      </edge>
-      <edge source="2201.06775" target="1911.06283" id="375">
-        <attvalues>
-          <attvalue for="5" value=" The authors of \cite{8972568} proposed a method that trains on rendered images and fine-tunes on real images to detect and track the state of a rope." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1707.09092" id="376">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1907.00338" id="377">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1502.00956" id="378">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1707.07410" id="379">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1712.07629" id="380">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1511.07247" id="381">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1612.06321" id="382">
-        <attvalues>
-          <attvalue for="5" value=" Detecting correspondences between images is a primary step towards solving many problems in computer vision: scene reconstruction~\cite{RomeInDay2009,Heinly15,COLMAP2016}, visual localization~\cite{ActiveSearch2012,Sattler18,Lynen19}, SLAM~\cite{Mur15,DeTone17a,SuperPoint2017}, image retrieval~\cite{DBLP:journals/ijcv/Lowe04,NetVLAD2016,DELF2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1911.11763" id="383">
-        <attvalues>
-          <attvalue for="5" value=" \par The SuperGlue \cite{sarlin20superglue} approach to image matching fits into the standard image matching pipeline replacing classical Nearest Neighbor-based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="1706.03762" id="384">
-        <attvalues>
-          <attvalue for="5" value=" It tackles keypoints matching by formulating it as an end-to-end classification task solved by an Attention-based \cite{DBLP:conf/nips/VaswaniSPUJGKP17} Graph Neural Network (AGNN)." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="2006.16236" id="385">
-        <attvalues>
-          <attvalue for="5" value=" We mainly focus on linear~\cite{katharopoulos-et-al-2020} and FAVOR+~\cite{choromanski2020rethinking} versions of attention and show how they contribute to the faster computation time and memory usage." />
-        </attvalues>
-      </edge>
-      <edge source="2204.08870" target="2009.14794" id="386">
-        <attvalues>
-          <attvalue for="5" value=" We mainly focus on linear~\cite{katharopoulos-et-al-2020} and FAVOR+~\cite{choromanski2020rethinking} versions of attention and show how they contribute to the faster computation time and memory usage." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="2110.03143" id="387">
-        <attvalues>
-          <attvalue for="5" value=" These ATR algorithms are capable of detecting targets at different scales and ranges (Range:1000m - 5000m) \cite{vs2022meta}, which are often not recognized by the naked eye." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="1506.01497" id="388">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there has been significant interest in improving ATR algorithms using DNNs \cite{kazemi2019deep, vs2022meta} where the DNN frameworks are employed to improve detection and classification components of the ATR algorithms \cite{ren2015faster, he2016deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="1811.08581" id="389">
-        <attvalues>
-          <attvalue for="5" value=" Open-set recognition is a problem of handling `unknown' classes that are not seen during training, whereas traditional closed-set classifiers assume that only known classes appear during testing \cite{geng2020recent, vaze2021open}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="1707.07418" id="390">
-        <attvalues>
-          <attvalue for="5" value=" G-OpenMax \cite{ge2017generative} is a generative method which trains a model with synthesized unknown data." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="1511.06233" id="391">
-        <attvalues>
-          <attvalue for="5" value=" Our experiments have shown that for open-set classification, employing OpenMax \cite{bendale2016towards} or SoftMax \cite{Neal_2018_ECCV} for classifying targets in short ranges (0m-1000m) works reasonably well." />
-        </attvalues>
-      </edge>
-      <edge source="2211.05883" target="1512.03385" id="392">
-        <attvalues>
-          <attvalue for="5" value=" In this work, we employ ResNet 18 \cite{he2016deep} as the base network for ATR closed-set classification." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2308.04079" id="393">
-        <attvalues>
-          <attvalue for="5" value=" While 3D Gaussian Splatting (3DGS) \cite{3dgs} has made significant strides in achieving high-fidelity, real-time rendering through its differentiability, its primary focus is on optimized visual representation rather than the reconstruction of scenes from diverse, unstructured image sources \cite{10630672}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2406.16289" id="394">
-        <attvalues>
-          <attvalue for="5" value=" By harnessing the diversity and widespread availability of crowdsourced imagery, researchers can achieve more comprehensive and varied datasets, laying the groundwork for more detailed and nuanced 3D reconstructions \cite{csnerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2312.07504" id="395">
-        <attvalues>
-          <attvalue for="5" value=" Such inconsistencies disrupt traditional methods like COLMAP \cite{colmap1,colmap2}, which rely on accurate Structure from Motion (SfM) \cite{DBLP:journals/corr/abs-2312-07504,fan2024instantsplat,10446550}, and they particularly affect the ability of novel view synthesis methods to maintain consistent color and texture across perspectives \cite{Gao_2024_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2008.02268" id="396">
-        <attvalues>
-          <attvalue for="5" value=" The primary issues include the lack of precise camera poses \cite{colmapfree}, sparse and limited viewpoints \cite{10446844}, and inconsistent lighting conditions across images \cite{wildinnerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2311.17123" id="397">
-        <attvalues>
-          <attvalue for="5" value=" Such inconsistencies disrupt traditional methods like COLMAP \cite{colmap1,colmap2}, which rely on accurate Structure from Motion (SfM) \cite{DBLP:journals/corr/abs-2312-07504,fan2024instantsplat,10446550}, and they particularly affect the ability of novel view synthesis methods to maintain consistent color and texture across perspectives \cite{Gao_2024_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="1907.09358" id="398">
-        <attvalues>
-          <attvalue for="5" value=" As a result, synthesizing accurate and visually coherent new viewpoints from crowdsourced data remains a significant challenge \cite{DBLP:journals/jair/MogadalaKK21}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2406.09756" id="399">
-        <attvalues>
-          <attvalue for="5" value=" Addressing these issues requires more robust approaches, such as leveraging large visual models \cite{mast3r}, which can better generalize across diverse and noisy inputs to enhance pose estimation \cite{Wang_2024_CVPR} and illumination consistency \cite{10480646}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.08562" target="2312.14132" id="400">
-        <attvalues>
-          <attvalue for="5" value=" Addressing these issues requires more robust approaches, such as leveraging large visual models \cite{mast3r}, which can better generalize across diverse and noisy inputs to enhance pose estimation \cite{Wang_2024_CVPR} and illumination consistency \cite{10480646}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.01949" target="1411.6031" id="401">
-        <attvalues>
-          <attvalue for="5" value=" The predictions are mainly evaluated by spatio-temporal Average-Precision (stAP)~\cite{gkioxari2015finding}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.04310" target="1704.07813" id="402">
-        <attvalues>
-          <attvalue for="5" value=" However, conventional self-supervised monocular depth estimation methods \cite{zhou2017unsupervised, yang2017unsupervised, yin2018geonet} often adopt self-supervision losses, such as a photometric consistency loss and a feature consistency loss despite the underlying side effects." />
-        </attvalues>
-      </edge>
-      <edge source="2111.04310" target="2007.10603" id="403">
-        <attvalues>
-          <attvalue for="5" value=" A recent self-supervised depth estimation method \cite{shu2020feature} proposes a feature-metric loss that utilizes the discriminability of auto-encoded feature representations, expecting explicitly trained auto-encoder to have better capacity on encoding more discriminative information in textureless or occluded regions." />
-        </attvalues>
-      </edge>
-      <edge source="2111.04310" target="2003.13017" id="404">
-        <attvalues>
-          <attvalue for="5" value=" We adopt the Gauss-Newton refinement approach \cite{yu2020fast} to derive residual depths from the auto-encoded feature and depth feature." />
-        </attvalues>
-      </edge>
-      <edge source="2109.10563" target="1908.00222" id="405">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, to acquire diverse and realistic synthesized data, numerous things should be set exquisitely, which often requires professional designers and tools \cite{structure3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="2008.05787" id="406">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="2003.07064" id="407">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1711.05471" id="408">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1807.00119" id="409">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1708.02863" id="410">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="2001.03152" id="411">
-        <attvalues>
-          <attvalue for="5" value=" Object hallucination by deep detectors can be causes by sensitivity to the absolute position in the image~\cite{manfredi2020shift,kayhan2020translation} while also affected by scene context~\cite{Barnea_2019_CVPR, gidaris2015object,Liu_2018,zhu2017couplenet,singh2020don}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1405.0312" id="412">
-        <attvalues>
-          <attvalue for="5" value=" Existing object detection datasets such as PASCAL VOC~\cite{pascal-voc-2012}, MS-COCO\cite{lin2014microsoft}, Imagenet-det~\cite{russakovsky2015imagenet}, and Open Image~\cite{DBLP:journals/corr/abs-1811-00982} have no annotated object parts." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1409.0575" id="413">
-        <attvalues>
-          <attvalue for="5" value=" Existing object detection datasets such as PASCAL VOC~\cite{pascal-voc-2012}, MS-COCO\cite{lin2014microsoft}, Imagenet-det~\cite{russakovsky2015imagenet}, and Open Image~\cite{DBLP:journals/corr/abs-1811-00982} have no annotated object parts." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1811.00982" id="414">
-        <attvalues>
-          <attvalue for="5" value=" Existing object detection datasets such as PASCAL VOC~\cite{pascal-voc-2012}, MS-COCO\cite{lin2014microsoft}, Imagenet-det~\cite{russakovsky2015imagenet}, and Open Image~\cite{DBLP:journals/corr/abs-1811-00982} have no annotated object parts." />
-        </attvalues>
-      </edge>
-      <edge source="2106.02523" target="1406.2031" id="415">
-        <attvalues>
-          <attvalue for="5" value=" Pascal-Parts~\cite{chen2014detect} and GoCaRD~\cite{stappen2020gocard} include part labels, yet lack information if a part is missing and where, as is required to evaluate visual verification." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2003.08934" id="416">
-        <attvalues>
-          <attvalue for="5" value=" The advent of Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf} has brought revolutionary advancements in photo-realistic novel view synthesis." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2308.04079" id="417">
-        <attvalues>
-          <attvalue for="5" value=" Building on this progress, the recent introduction of 3D Gaussian Splatting (3DGS) \cite{kerbl20233d} has further enhanced 3D scene representation using Gaussian ellipsoids, achieving high-quality 2D rendering and faster training and rendering speeds." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2111.14292" id="418">
-        <attvalues>
-          <attvalue for="5" value=" NeRF's deblurring technology has undergone relatively early development, with Deblur-NeRF \cite{ma2022deblur} being the first framework to tackle this issue, which utilized an analysis-by-synthesis approach to recover sharp NeRFs from blurry inputs." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2401.00834" id="419">
-        <attvalues>
-          <attvalue for="5" value="~\cite{lee2024deblurring} assign the corrections on the rotation and scaling matrix of 3D gaussians by using a small MLP, enhancing the clarity of scenes reconstructed from blurred images." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2206.11896" id="420">
-        <attvalues>
-          <attvalue for="5" value=" For NeRFs, EventNeRF \cite{rudnev2023eventnerf} used event integration and color simulation for colored 3D representations, while Qi et al." />
-        </attvalues>
-      </edge>
-      <edge source="2407.13520" target="2405.20224" id="421">
-        <attvalues>
-          <attvalue for="5" value=" For 3DGS, Yu proposed EvaGaussian\cite{yu2024evagaussians}, optimizing both scene reconstruction and camera trajectory estimation to achieve unparalleled detail and fidelity in real-time 3D scene synthesis." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2207.07386" id="422">
-        <attvalues>
-          <attvalue for="5" value=" The 3D Human Pose Estimation (HPE) technique is essential, as it encompasses wide-ranging applications in security surveillance, film making as well as emerging trends such as dance generation~\cite{au2022choreograph} and pedestrian behaviour prediction~\cite{papaioannidis2023fast}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2207.07381" id="423">
-        <attvalues>
-          <attvalue for="5" value=" Despite the considerable progress on multi-view 3D HPE~\cite{jiang2022dual,zhou2022quickpose,pavlakos2019expressive,gong2022posetriplet}, monocular pose estimation remains challenging because of the depth ambiguity caused by uncertain 2D-to-3D projection and occlusion of body parts." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="1904.05866" id="424">
-        <attvalues>
-          <attvalue for="5" value=" Despite the considerable progress on multi-view 3D HPE~\cite{jiang2022dual,zhou2022quickpose,pavlakos2019expressive,gong2022posetriplet}, monocular pose estimation remains challenging because of the depth ambiguity caused by uncertain 2D-to-3D projection and occlusion of body parts." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2203.15625" id="425">
-        <attvalues>
-          <attvalue for="5" value=" Despite the considerable progress on multi-view 3D HPE~\cite{jiang2022dual,zhou2022quickpose,pavlakos2019expressive,gong2022posetriplet}, monocular pose estimation remains challenging because of the depth ambiguity caused by uncertain 2D-to-3D projection and occlusion of body parts." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2103.16385" id="426">
-        <attvalues>
-          <attvalue for="5" value=" Following \cite{xu2021graph}, Zhao et al." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="1811.11742" id="427">
-        <attvalues>
-          <attvalue for="5" value=" Despite multi-frame estimation frameworks~\cite{pavllo20193d,zhang2022mixste,du2023flowpose} leveraging temporal information to effectively address ambiguity, those frameworks rely on future frames that hinder real-time performance." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2203.00859" id="428">
-        <attvalues>
-          <attvalue for="5" value=" Despite multi-frame estimation frameworks~\cite{pavllo20193d,zhang2022mixste,du2023flowpose} leveraging temporal information to effectively address ambiguity, those frameworks rely on future frames that hinder real-time performance." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2010.02502" id="429">
-        <attvalues>
-          <attvalue for="5" value="~\cite{gong2023diffpose} also adopt a GCN structure~\cite{zhao2022graformer} and accelerate the diffusion process using DDIM~\cite{song2021denoising}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2006.11239" id="430">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Denoising Diffusion Probabilistic Models~\cite{song2021denoising,ho2020denoising,gong2023diffpose,choi2022diffupose} (DDPMs, a." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2211.16940" id="431">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by ~\cite{gong2023diffpose, choi2022diffupose}, we propose a monocular single-frame 3D HPE framework based on diffusion modelling." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="2212.02796" id="432">
-        <attvalues>
-          <attvalue for="5" value="~\cite{choi2022diffupose} suggest replacing the standard DDPM's U-Net structure with a GCN." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09836" target="1611.09813" id="433">
-        <attvalues>
-          <attvalue for="5" value=" 3) The proposed framework achieves state-of-the-art performance on two widely-used benchmarks~\cite{ionescu2013human3,mehta2017monocular} demonstrating our superiority." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13257" target="1912.08804" id="434">
-        <attvalues>
-          <attvalue for="5" value=" Such representations arise, for example, in differentiable rendering pipelines \cite{SynSin, SoftRasterizer} or implicit representations of images \cite{siren, fourier_inr, inr_gan, cips}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13257" target="2006.09661" id="435">
-        <attvalues>
-          <attvalue for="5" value=" Such representations arise, for example, in differentiable rendering pipelines \cite{SynSin, SoftRasterizer} or implicit representations of images \cite{siren, fourier_inr, inr_gan, cips}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1506.06579" id="436">
-        <attvalues>
-          <attvalue for="5" value=" The strength of these methods comes from the abstract features that emerge at the deeper layers of CNNs \cite{Yosinski2015}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1409.1556" id="437">
-        <attvalues>
-          <attvalue for="5" value=" The earlier approaches of this category \cite{Simonyan15,Szegedy15,Krizhevsky12,Sermanet13} performs particularly good at problems like image category classification, object detection and/or localization, mainly because of their capability to convolve abstract features into image categories or object definitions." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1409.4842" id="438">
-        <attvalues>
-          <attvalue for="5" value=" The earlier approaches of this category \cite{Simonyan15,Szegedy15,Krizhevsky12,Sermanet13} performs particularly good at problems like image category classification, object detection and/or localization, mainly because of their capability to convolve abstract features into image categories or object definitions." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1312.6229" id="439">
-        <attvalues>
-          <attvalue for="5" value=" The earlier approaches of this category \cite{Simonyan15,Szegedy15,Krizhevsky12,Sermanet13} performs particularly good at problems like image category classification, object detection and/or localization, mainly because of their capability to convolve abstract features into image categories or object definitions." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1404.1777" id="440">
-        <attvalues>
-          <attvalue for="5" value=" There are also attempts with promising results, which aim at transferring pre-trained and well-tuned CNNs into image retrieval frameworks \cite{Babenko14,Chandrasekhar16}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05355" target="1508.02496" id="441">
-        <attvalues>
-          <attvalue for="5" value=" There are also attempts with promising results, which aim at transferring pre-trained and well-tuned CNNs into image retrieval frameworks \cite{Babenko14,Chandrasekhar16}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="1805.00330" id="442">
-        <attvalues>
-          <attvalue for="5" value=" As an example, in \cite{nikouei2018real}, the authors used a lightweight Convolutional Neural Network (L-CNN) to detect humans in surveillance video frames." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="2203.07093" id="443">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{shi2016human}, we considered using K-NN classifiers with AM-FM representations for person detection." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="2110.07646" id="444">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{shi2021talking}, the authors used head detection to detect talking activities." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="2110.07070" id="445">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{teeparthi2021fast}, the authors developed methods for hand detection." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="2112.13463" id="446">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{sanchez2021bilingual}, the authors considered the use of YOLO \cite{redmon2018yolov3} for head detection to build a bilingual speech recognition system." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="1804.02767" id="447">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{sanchez2021bilingual}, the authors considered the use of YOLO \cite{redmon2018yolov3} for head detection to build a bilingual speech recognition system." />
-        </attvalues>
-      </edge>
-      <edge source="2112.12217" target="2110.13269" id="448">
-        <attvalues>
-          <attvalue for="5" value=" Fast video face detection was recently described in \cite{tran2021facial}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07070" target="2112.13463" id="449">
-        <attvalues>
-          <attvalue for="5" value=" In addtion, we note that speech recognition using speaker geometry is studied in \cite{Luis2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07070" target="1506.01497" id="450">
-        <attvalues>
-          <attvalue for="5" value=" We tested Faster R-CNN \cite{ren2016faster}, YOLO \cite{redmon2016you}, and SSD \cite{liu2016ssd}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07070" target="1506.02640" id="451">
-        <attvalues>
-          <attvalue for="5" value=" We tested Faster R-CNN \cite{ren2016faster}, YOLO \cite{redmon2016you}, and SSD \cite{liu2016ssd}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07070" target="1512.02325" id="452">
-        <attvalues>
-          <attvalue for="5" value=" We tested Faster R-CNN \cite{ren2016faster}, YOLO \cite{redmon2016you}, and SSD \cite{liu2016ssd}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.04392" target="1605.06211" id="453">
-        <attvalues>
-          <attvalue for="5" value=" Object segmentation, in its various forms, is a widely studied problem in computer vision~\cite{long2015fully}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.04392" target="1604.05144" id="454">
-        <attvalues>
-          <attvalue for="5" value=" These methods function with weak supervision provided through scribbles~\cite{lin2016scribblesup, tang2018normalized, tang2018regularized, vernaza2017learning, xu2015learning} or clicks~\cite{bearman2016s} or image-level tags~\cite{papandreou2015weakly, tang2018regularized, xu2015learning} or even bounds~\cite{dai2015boxsup}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.04392" target="1503.01640" id="455">
-        <attvalues>
-          <attvalue for="5" value=" Further, semi-supervised segmentation techniques~\cite{dai2015boxsup, hong2015decoupled, hung2018adversarial, papandreou2015weakly, pathak2015constrained} were proposed that attempt segmentation in a setting where only a fraction of the image datasets are human-labelled." />
-        </attvalues>
-      </edge>
-      <edge source="2307.04392" target="2104.14294" id="456">
-        <attvalues>
-          <attvalue for="5" value=" We propose an end-to-end pipeline as follows -- for a given video frame, we encode its frames and their optical flows in RGB format using a self-supervised ViT like DINO~\cite{caron2021emerging}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.14581" target="1603.09114" id="457">
-        <attvalues>
-          <attvalue for="5" value=" Over the recent years, the deep learning-based methods have achieved significant progress in general visual tasks, and have also been introduced into the field of image matching \cite{yi2016lift,ono2018lf}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.14581" target="1805.09662" id="458">
-        <attvalues>
-          <attvalue for="5" value=" Over the recent years, the deep learning-based methods have achieved significant progress in general visual tasks, and have also been introduced into the field of image matching \cite{yi2016lift,ono2018lf}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.14581" target="2002.05709" id="459">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL), which helps the model obtain easy invariance with augmented data, is one of the most popular technics in natural language processing and computer vision\cite{chen2020simple,sermanet2017time}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.14581" target="1712.07629" id="460">
-        <attvalues>
-          <attvalue for="5" value=" As for local feature representation learning, the well-known Superpoint \cite{detone2018superpoint} proposed a novel Homographic Adaptation procedure, which is a form of self-supervision, to tackle the ill-posed problem of keypoint extraction." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01549" target="1902.03604" id="461">
-        <attvalues>
-          <attvalue for="5" value=" In this way, different from previous works \cite{voigtlaender2019mots} which requires consecutive frames as inputs, PointTrack enables a more flexible training strategy since both image and video level segmentation labels can be used." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01549" target="1912.02096" id="462">
-        <attvalues>
-          <attvalue for="5" value=" \cite{porzi2019learning} introduce mask pooling rather than ROI Align for instance feature extraction." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01549" target="2007.01550" id="463">
-        <attvalues>
-          <attvalue for="5" value=" In our previous work, we propose a simple yet highly effective method named PointTrack \cite{xu2020Segment} to learn instance embeddings on segments." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01549" target="1906.11109" id="464">
-        <attvalues>
-          <attvalue for="5" value=" As bbox-proposal based instance segmentation methods always miss bboxes when instances multiple bboxes are heavily overlapped, PointTrack adopts a proposal-free instance segmentation network \cite{Neven_2019_CVPR} following the encoder-decoder architecture for efficient instance segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="1506.07363" id="465">
-        <attvalues>
-          <attvalue for="5" value=" Traditional methods mainly use low-level features such as color, brightness, orientation, and contrast of an image to compute saliency maps, but they often fail to deal with complex background and multiple objects \cite{yang2013saliency,zhu2014saliency,srivatsa2015salient}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2012.13095" id="466">
-        <attvalues>
-          <attvalue for="5" value="\cite{wu2021mobilesal}, Lee et al." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2112.07380" id="467">
-        <attvalues>
-          <attvalue for="5" value="\cite{lee2021tracer}, Wang et al." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2104.07877" id="468">
-        <attvalues>
-          <attvalue for="5" value="\cite{wang2021raining} designed lightweight structures to accelerate inference." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="1904.09569" id="469">
-        <attvalues>
-          <attvalue for="5" value="\cite{Liu2019PoolSal}, Wu et al." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2106.12011" id="470">
-        <attvalues>
-          <attvalue for="5" value="\cite{Wu_2022}, Qin et al." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2101.04704" id="471">
-        <attvalues>
-          <attvalue for="5" value="\cite{qin2021boundary} used coarse-to-fine framework to obtain finer results." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2204.05041" id="472">
-        <attvalues>
-          <attvalue for="5" value="\cite{xie2022pyramid}, Kim et al." />
-        </attvalues>
-      </edge>
-      <edge source="2312.07100" target="2209.09475" id="473">
-        <attvalues>
-          <attvalue for="5" value="\cite{kim2022revisiting} designed high resolution modules to obtain accurate high resolution saliency maps." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="2103.00020" id="474">
-        <attvalues>
-          <attvalue for="5" value=" In today's data-driven era, managing the sheer volume and variety of data presents a crucial challenge, particularly in areas like computer vision and deep learning, which deal with a tremendous amount of data~\cite{radford2021learning,codetr2022,cheng2021mask2former}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="2211.12860" id="475">
-        <attvalues>
-          <attvalue for="5" value=" In today's data-driven era, managing the sheer volume and variety of data presents a crucial challenge, particularly in areas like computer vision and deep learning, which deal with a tremendous amount of data~\cite{radford2021learning,codetr2022,cheng2021mask2former}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="2112.01527" id="476">
-        <attvalues>
-          <attvalue for="5" value=" In today's data-driven era, managing the sheer volume and variety of data presents a crucial challenge, particularly in areas like computer vision and deep learning, which deal with a tremendous amount of data~\cite{radford2021learning,codetr2022,cheng2021mask2former}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="1903.04351" id="477">
-        <attvalues>
-          <attvalue for="5" value=" However, as illustrated in Figure \ref{fig:overview}, traditional coreset selection methods are unrealistic because they were developed under the naïve assumption of a single object per image, a condition that real-world images do not often meet~\cite{braverman2019coresets,coleman2019selection,guo2022deepcore}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="1906.11829" id="478">
-        <attvalues>
-          <attvalue for="5" value=" However, as illustrated in Figure \ref{fig:overview}, traditional coreset selection methods are unrealistic because they were developed under the naïve assumption of a single object per image, a condition that real-world images do not often meet~\cite{braverman2019coresets,coleman2019selection,guo2022deepcore}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="2204.08499" id="479">
-        <attvalues>
-          <attvalue for="5" value=" However, as illustrated in Figure \ref{fig:overview}, traditional coreset selection methods are unrealistic because they were developed under the naïve assumption of a single object per image, a condition that real-world images do not often meet~\cite{braverman2019coresets,coleman2019selection,guo2022deepcore}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="1805.04687" id="480">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, we also evaluated it on the BDD100k~\cite{yu2020bdd100k} and MS COCO2017~\cite{lin2014microsoft} datasets and confirmed that our method outperforms random selection." />
-        </attvalues>
-      </edge>
-      <edge source="2404.09161" target="1405.0312" id="481">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, we also evaluated it on the BDD100k~\cite{yu2020bdd100k} and MS COCO2017~\cite{lin2014microsoft} datasets and confirmed that our method outperforms random selection." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1502.00956" id="482">
-        <attvalues>
-          <attvalue for="5" value=" Classic approaches to two-view SfM follow a standard pipeline of first matching features/edges between the two images, then inferring motion and geometry from those matches~\cite{davison2007pami:monoslam,klein2007ismar:ptam,mur2015orbslam,forster2014icra:svo,engel14eccv:lsdslam,engel2016dso,zhong2017self,zhong2020displacement}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1607.02565" id="483">
-        <attvalues>
-          <attvalue for="5" value=" Classic approaches to two-view SfM follow a standard pipeline of first matching features/edges between the two images, then inferring motion and geometry from those matches~\cite{davison2007pami:monoslam,klein2007ismar:ptam,mur2015orbslam,forster2014icra:svo,engel14eccv:lsdslam,engel2016dso,zhong2017self,zhong2020displacement}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1709.00930" id="484">
-        <attvalues>
-          <attvalue for="5" value=" Classic approaches to two-view SfM follow a standard pipeline of first matching features/edges between the two images, then inferring motion and geometry from those matches~\cite{davison2007pami:monoslam,klein2007ismar:ptam,mur2015orbslam,forster2014icra:svo,engel14eccv:lsdslam,engel2016dso,zhong2017self,zhong2020displacement}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="2012.00899" id="485">
-        <attvalues>
-          <attvalue for="5" value=" Classic approaches to two-view SfM follow a standard pipeline of first matching features/edges between the two images, then inferring motion and geometry from those matches~\cite{davison2007pami:monoslam,klein2007ismar:ptam,mur2015orbslam,forster2014icra:svo,engel14eccv:lsdslam,engel2016dso,zhong2017self,zhong2020displacement}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1808.08690" id="486">
-        <attvalues>
-          <attvalue for="5" value=" Without a priori knowledge of scale or recognizable objects in the scene, it is impossible to recover scene geometry from a single view~\cite{hassner2006cvprw,zhong18eccvmono,xie2019iccv:pix2vox,li2019bmvc:svosr}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1901.11153" id="487">
-        <attvalues>
-          <attvalue for="5" value=" Without a priori knowledge of scale or recognizable objects in the scene, it is impossible to recover scene geometry from a single view~\cite{hassner2006cvprw,zhong18eccvmono,xie2019iccv:pix2vox,li2019bmvc:svosr}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1811.11921" id="488">
-        <attvalues>
-          <attvalue for="5" value=" Without a priori knowledge of scale or recognizable objects in the scene, it is impossible to recover scene geometry from a single view~\cite{hassner2006cvprw,zhong18eccvmono,xie2019iccv:pix2vox,li2019bmvc:svosr}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1704.07813" id="489">
-        <attvalues>
-          <attvalue for="5" value=" In the first category, which we shall call Type $\mathbf{I}$, the problem is treated as a joint optimization task of monocular depth and pose regression~\cite{sfmlearner,yin2018geonet,Ranjan_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1803.02276" id="490">
-        <attvalues>
-          <attvalue for="5" value=" In the first category, which we shall call Type $\mathbf{I}$, the problem is treated as a joint optimization task of monocular depth and pose regression~\cite{sfmlearner,yin2018geonet,Ranjan_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1805.09806" id="491">
-        <attvalues>
-          <attvalue for="5" value=" In the first category, which we shall call Type $\mathbf{I}$, the problem is treated as a joint optimization task of monocular depth and pose regression~\cite{sfmlearner,yin2018geonet,Ranjan_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1612.02401" id="492">
-        <attvalues>
-          <attvalue for="5" value=" In the second category, denoted Type $\mathbf{II}$, the scaled camera pose and the scaled depth are inferred from the image pair, and are iteratively refined via multi-view geometry~\cite{ummenhofer2017demon,tang2018ba,teed2018deepv2d}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1806.04807" id="493">
-        <attvalues>
-          <attvalue for="5" value=" In the second category, denoted Type $\mathbf{II}$, the scaled camera pose and the scaled depth are inferred from the image pair, and are iteratively refined via multi-view geometry~\cite{ummenhofer2017demon,tang2018ba,teed2018deepv2d}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1812.04605" id="494">
-        <attvalues>
-          <attvalue for="5" value=" In the second category, denoted Type $\mathbf{II}$, the scaled camera pose and the scaled depth are inferred from the image pair, and are iteratively refined via multi-view geometry~\cite{ummenhofer2017demon,tang2018ba,teed2018deepv2d}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="1904.03848" id="495">
-        <attvalues>
-          <attvalue for="5" value=" Our method operates by first estimating dense matching points between two frames using a deep optical flow network~\cite{zhong2019unsupervised,zhong2020nipsflow}, from which a set of highly reliable matches are sampled in order to compute the relative camera pose via a GPU-accelerated classic five-point algorithm~\cite{li2006five} with RANSAC~\cite{fischler1981random}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.00556" target="2010.14851" id="496">
-        <attvalues>
-          <attvalue for="5" value=" Our method operates by first estimating dense matching points between two frames using a deep optical flow network~\cite{zhong2019unsupervised,zhong2020nipsflow}, from which a set of highly reliable matches are sampled in order to compute the relative camera pose via a GPU-accelerated classic five-point algorithm~\cite{li2006five} with RANSAC~\cite{fischler1981random}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.05423" target="1908.08854" id="497">
-        <attvalues>
-          <attvalue for="5" value=" Although some notable 3D segmentation surveys have been released including RGB-D semantic segmentation~\cite{fooladgar2020survey}, remote sensing imagery segmentation~\cite{yuan2021review}, point clouds segmentation~\cite{xie2020linking},~\cite{guo2020deep},~\cite{liu2019deep},~\cite{bello2020deep},~\cite{naseer2018indoor},~\cite{ioannidou2017deep}, these surveys do not comprehensively cover all 3D data types and typical application domains." />
-        </attvalues>
-      </edge>
-      <edge source="2103.05423" target="1912.12033" id="498">
-        <attvalues>
-          <attvalue for="5" value=" Most importantly, these surveys do not focus on 3D segmentation but give a general survey of deep learning from point clouds~\cite{guo2020deep},~\cite{liu2019deep},~\cite{bello2020deep},~\cite{naseer2018indoor},~\cite{ioannidou2017deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.05423" target="1803.03352" id="499">
-        <attvalues>
-          <attvalue for="5" value=" Most importantly, these surveys do not focus on 3D segmentation but give a general survey of deep learning from point clouds~\cite{guo2020deep},~\cite{liu2019deep},~\cite{bello2020deep},~\cite{naseer2018indoor},~\cite{ioannidou2017deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2106.08261" id="500">
-        <attvalues>
-          <attvalue for="5" value=" Both key ingredients are harder to formulate and successfully implement: masked prediction in vision, in the guise of future prediction of video frames~\cite{voleti2022masked, yan2022temporally} (or meaningful image-derived latents~\cite{nair2022r3m}) from previous context frames (or latents), has remained a largely unsolved problem~\cite{bear2021physion}; and it is not at all clear how to elicit most of the ``intermediate'' computations above through an interface to a single vision model." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2108.07258" id="501">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the ``foundation model'' approach~\cite{bommasani2021opportunities} has driven rapid progress in natural language generation and understanding tasks, without breaking them down into pieces that require special methods." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2203.12601" id="502">
-        <attvalues>
-          <attvalue for="5" value=" Both key ingredients are harder to formulate and successfully implement: masked prediction in vision, in the guise of future prediction of video frames~\cite{voleti2022masked, yan2022temporally} (or meaningful image-derived latents~\cite{nair2022r3m}) from previous context frames (or latents), has remained a largely unsolved problem~\cite{bear2021physion}; and it is not at all clear how to elicit most of the ``intermediate'' computations above through an interface to a single vision model." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2206.08916" id="503">
-        <attvalues>
-          <attvalue for="5" value=" For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments? Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2206.07669" id="504">
-        <attvalues>
-          <attvalue for="5" value=" For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments? Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2303.04671" id="505">
-        <attvalues>
-          <attvalue for="5" value=" For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments? Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2211.09808" id="506">
-        <attvalues>
-          <attvalue for="5" value=" For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments? Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2212.02499" id="507">
-        <attvalues>
-          <attvalue for="5" value=" For instance, what sort of input -- which must be visual in nature, if training were purely on unlabeled visual data -- would cause a vision model to output object segments? Recent unified vision models emphasize a shared architectural stem from which many outputs are produced, but they all require some combination of (i) supervised training on labeled datasets for each supported task, (ii) task-specific decoder architectures, and (iii) a language prompting interface (and therefore training on joint visual-textual data at some stage)~\cite{jaegle2021perceiver,lu2022unified,chen2022unified,wu2023visual,li2023uni,wang2023images}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.01828" target="2111.06377" id="508">
-        <attvalues>
-          <attvalue for="5" value="] The recent Masked Autoencoder models~\cite{he2022masked} directly translate the masked prediction task from text to visual data (images and video); however, these models do not appear to learn representations as general as language models." />
-        </attvalues>
-      </edge>
-      <edge source="2110.00803" target="1912.06704" id="509">
-        <attvalues>
-          <attvalue for="5" value=" Most existing approaches focus on stereo pairs and excellent results have been achieved in this context \cite{Scharstein2001} \cite{yang2019hierarchical} \cite{song2020edgestereo} \cite{Xu2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.00803" target="1903.01700" id="510">
-        <attvalues>
-          <attvalue for="5" value=" Most existing approaches focus on stereo pairs and excellent results have been achieved in this context \cite{Scharstein2001} \cite{yang2019hierarchical} \cite{song2020edgestereo} \cite{Xu2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04862" target="1907.02890" id="511">
-        <attvalues>
-          <attvalue for="5" value=" However, initial correspondences generated by matching local geometric descriptors are usually contaminated by heavy outliers owing to nuisances such as noise, varying data resolutions, clutter, and occlusion~\cite{Yang2020corr_group_eval}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04862" target="1706.00984" id="512">
-        <attvalues>
-          <attvalue for="5" value=" Although for RANSAC estimators applied to image registration, some metrics different from the classical inlier count metric have bee proposed~\cite{barath2018graph,lebeda2012fixing}, they fail to comprehensively analyze the contributions of inliers and outliers." />
-        </attvalues>
-      </edge>
-      <edge source="1909.06884" target="1304.3192" id="513">
-        <attvalues>
-          <attvalue for="5" value="1927035}, SI \cite{johnson1999using}, FPFH \cite{rusu2010fast}, USC\cite{tombari2010unique}, RoPS \cite{guo2013rotational}) as well as for a recently proposed descriptor based on deep learning (CGF-32 \cite{khoury2017learning})." />
-        </attvalues>
-      </edge>
-      <edge source="1909.06884" target="1709.05056" id="514">
-        <attvalues>
-          <attvalue for="5" value="1927035}, SI \cite{johnson1999using}, FPFH \cite{rusu2010fast}, USC\cite{tombari2010unique}, RoPS \cite{guo2013rotational}) as well as for a recently proposed descriptor based on deep learning (CGF-32 \cite{khoury2017learning})." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="1604.01685" id="515">
-        <attvalues>
-          <attvalue for="5" value=" It takes more than an hour to obtain pixel-level annotation of an image~\cite{cordts2016cityscapes}, which makes the labor cost of large-scale pixel-level annotated datasets expensive." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="1709.03410" id="516">
-        <attvalues>
-          <attvalue for="5" value=" \item Experiments show that our method achieves a new state-of-the-art on PASCAL-5$^i$~\cite{shaban2017one} and COCO-20$^i$~\cite{lin2014microsoft} under both mask FSS and class-aware mask FSS settings." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2104.01538" id="517">
-        <attvalues>
-          <attvalue for="5" value=" Based on different strategies for mining query-support information, existing FSS methods can be divided into two technical approaches: support-query matching mechanism~\cite{min2021hypercorrelation, hong2022cost, moon2023msi, peng2023hierarchical, shuai2023pgmanet, yang2023mianet} and self-support matching mechanism~\cite{fan2022ssp, wang2023focus}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2207.11549" id="518">
-        <attvalues>
-          <attvalue for="5" value=" Based on different strategies for mining query-support information, existing FSS methods can be divided into two technical approaches: support-query matching mechanism~\cite{min2021hypercorrelation, hong2022cost, moon2023msi, peng2023hierarchical, shuai2023pgmanet, yang2023mianet} and self-support matching mechanism~\cite{fan2022ssp, wang2023focus}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2111.06377" id="519">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2104.14294" id="520">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2304.07193" id="521">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2103.00020" id="522">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2112.10752" id="523">
-        <attvalues>
-          <attvalue for="5" value=" SD-DINO~\cite{zhang2024sddino} proposes to fuse the features of Stable Diffusion (SD)~\cite{rombach2022sd} and DINOv2~\cite{oquab2023dinov2} for semantic and dense correspondence." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2306.03881" id="524">
-        <attvalues>
-          <attvalue for="5" value=" DIFT~\cite{tang2023dift} extracts diffusion features to establish correspondences between images." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2305.15347" id="525">
-        <attvalues>
-          <attvalue for="5" value=" SD-DINO~\cite{zhang2024sddino} proposes to fuse the features of Stable Diffusion (SD)~\cite{rombach2022sd} and DINOv2~\cite{oquab2023dinov2} for semantic and dense correspondence." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2407.11503" id="526">
-        <attvalues>
-          <attvalue for="5" value=" UniFSS~\cite{chang2024unifss} utilizes CLIP to build a universal vision-language framework to accomplish seven FSS tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2212.07143" id="527">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2210.08402" id="528">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2303.15343" id="529">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="2309.17425" id="530">
-        <attvalues>
-          <attvalue for="5" value=" We systematically summarize the performance of various foundation models in FSS from both quantitative and qualitative perspectives, including DINO~\cite{caron2021dinov1}, DINOv2~\cite{oquab2023dinov2}, MAE~\cite{he2022mae}, CLIP~\cite{radford2021clip}, Open CLIP~\cite{ilharco_gabriel_2021_5143773,cherti2023reproducible,schuhmann2022laionb}, SigLIP~\cite{zhai2023siglip}, and DFN~\cite{fang2023dfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06305" target="1405.0312" id="531">
-        <attvalues>
-          <attvalue for="5" value=" \item Experiments show that our method achieves a new state-of-the-art on PASCAL-5$^i$~\cite{shaban2017one} and COCO-20$^i$~\cite{lin2014microsoft} under both mask FSS and class-aware mask FSS settings." />
-        </attvalues>
-      </edge>
-      <edge source="2406.11570" target="2003.02050" id="532">
-        <attvalues>
-          <attvalue for="5" value=" Work on texture transfer includes mesh methods such as using adaptive triangulations of meshes, which completes 11 texture transfers in 38 minutes \cite{schmidt2023surface}; or mapping pixels from the UV parameterization of a surface to the 3D avatar which was used to transfer clothing onto avatars \cite{mir2020learning}; or deep learning methods that perform direct image-to-image translation \cite{shysheya2019textured}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.11570" target="1905.08776" id="533">
-        <attvalues>
-          <attvalue for="5" value=" Work on texture transfer includes mesh methods such as using adaptive triangulations of meshes, which completes 11 texture transfers in 38 minutes \cite{schmidt2023surface}; or mapping pixels from the UV parameterization of a surface to the 3D avatar which was used to transfer clothing onto avatars \cite{mir2020learning}; or deep learning methods that perform direct image-to-image translation \cite{shysheya2019textured}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.11570" target="2308.04079" id="534">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kerbl20233d} used 3DG as volumetric representation for real-time scene rendering." />
-        </attvalues>
-      </edge>
-      <edge source="2406.11570" target="2402.19441" id="535">
-        <attvalues>
-          <attvalue for="5" value=" \cite{wang20243d} generated 3DG Textures for volumetric texturing in animations." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04123" target="1704.03489" id="536">
-        <attvalues>
-          <attvalue for="5" value=" Such depth information helps better understand 3D scenes and also facilitates many computer vision tasks such as indoor localization \cite{li20203d}, height estimation \cite{amirkolaee2019height}, simultaneous localization and mapping (SLAM) \cite{tateno2017cnn}, visual odometry \cite{yang2018deep}, classification \cite{he2017estimated}, etc Usually, the depth information is obtained through commercial depth sensors such as various LiDAR devices and Kinects." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04123" target="1807.02570" id="537">
-        <attvalues>
-          <attvalue for="5" value=" Such depth information helps better understand 3D scenes and also facilitates many computer vision tasks such as indoor localization \cite{li20203d}, height estimation \cite{amirkolaee2019height}, simultaneous localization and mapping (SLAM) \cite{tateno2017cnn}, visual odometry \cite{yang2018deep}, classification \cite{he2017estimated}, etc Usually, the depth information is obtained through commercial depth sensors such as various LiDAR devices and Kinects." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04123" target="1709.07077" id="538">
-        <attvalues>
-          <attvalue for="5" value=" Such depth information helps better understand 3D scenes and also facilitates many computer vision tasks such as indoor localization \cite{li20203d}, height estimation \cite{amirkolaee2019height}, simultaneous localization and mapping (SLAM) \cite{tateno2017cnn}, visual odometry \cite{yang2018deep}, classification \cite{he2017estimated}, etc Usually, the depth information is obtained through commercial depth sensors such as various LiDAR devices and Kinects." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04123" target="1809.05786" id="539">
-        <attvalues>
-          <attvalue for="5" value=" Various network architecture based on convolutional neural network (CNN) \cite{liu2010single}, recurrent neural network (RNN) \cite{wang2019recurrent} and generative adversarial network (GAN) \cite{almalioglu2019ganvo:} have been proposed to exploit RGB images." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1704.08345" id="540">
-        <attvalues>
-          <attvalue for="5" value=" It aims to recognize unseen (target) categories by transferring knowledge from seen (source) categories~\cite{DBLP:conf/cvpr/KodirovXG17,DBLP:conf/cvpr/XianLSA18,DBLP:conf/cvpr/PaulKM19}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1712.00981" id="541">
-        <attvalues>
-          <attvalue for="5" value=" It aims to recognize unseen (target) categories by transferring knowledge from seen (source) categories~\cite{DBLP:conf/cvpr/KodirovXG17,DBLP:conf/cvpr/XianLSA18,DBLP:conf/cvpr/PaulKM19}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1904.07659" id="542">
-        <attvalues>
-          <attvalue for="5" value=" It aims to recognize unseen (target) categories by transferring knowledge from seen (source) categories~\cite{DBLP:conf/cvpr/KodirovXG17,DBLP:conf/cvpr/XianLSA18,DBLP:conf/cvpr/PaulKM19}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1803.11320" id="543">
-        <attvalues>
-          <attvalue for="5" value=" \ref{intro}, the existing zero-shot semantic segmentation approaches reveal strong bias towards seen categories in Generalized ZSL(GZSL)~\cite{DBLP:conf/cvpr/SongSYLS18}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1606.00915" id="544">
-        <attvalues>
-          <attvalue for="5" value=" Hence, the semantic segmentation model tends to map the pixels of unseen images to the semantic embeddings of source categories and obtains poor generalization performance~\cite{DBLP:journals/pami/ChenPKMY18}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.00515" target="1906.00817" id="545">
-        <attvalues>
-          <attvalue for="5" value=" It should be noted that our method is also useful for the conventional ZSL setting since it can regularize the segmentation model to make more reasonable predictions for target images, which produces gradients to make the semantic segmentation neural network better fit them~\cite{DBLP:journals/corr/abs-1906-00817}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1506.01497" id="546">
-        <attvalues>
-          <attvalue for="5" value=" First, they use a mask-out strategy to filter the noisy object proposals; then, they use a Faster RCNN~\cite{ren2015faster} for detection using bags of instances." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1807.09856" id="547">
-        <attvalues>
-          <attvalue for="5" value=" Since no direct relevant work exists for this particular setup, we compare our methods against Glance~\cite{chattopadhyay2016counting} and the fully supervised LCFCN~\cite{laradji2018blobs}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1711.05282" id="548">
-        <attvalues>
-          <attvalue for="5" value=" C-WSL~\cite{Gao2018cwsl} is the most relevant to our work as they use count information to obtain the highest scoring proposals." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1604.03505" id="549">
-        <attvalues>
-          <attvalue for="5" value=" Since no direct relevant work exists for this particular setup, we compare our methods against Glance~\cite{chattopadhyay2016counting} and the fully supervised LCFCN~\cite{laradji2018blobs}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1907.01430" id="550">
-        <attvalues>
-          <attvalue for="5" value=" Many methods exist that can perform object localization but they need to be trained on point-level annotations~\cite{lempitsky2010learning, laradji2018blobs, li2018csrnet, laradji2019instance} or image-level~\cite{laradji2019masks}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1512.03385" id="551">
-        <attvalues>
-          <attvalue for="5" value=" This method is an ImageNet~\cite{deng2009imagenet} pre-trained model such as ResNet50~\cite{he2016deep} with a regression layer as its output layer." />
-        </attvalues>
-      </edge>
-      <edge source="2007.01837" target="1904.05647" id="552">
-        <attvalues>
-          <attvalue for="5" value=" C-MIL~\cite{Wan2019CMILCM} introduces a continuation optimization method to avoid getting stuck in a local minima." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1704.00675" id="553">
-        <attvalues>
-          <attvalue for="5" value=" Popular benchmarks for this task include DAVIS~\cite{pont20172017}, JHMDB~\cite{jhuang2013towards}, and VIP~\cite{zhou2018adaptive}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1808.00661" id="554">
-        <attvalues>
-          <attvalue for="5" value=" Popular benchmarks for this task include DAVIS~\cite{pont20172017}, JHMDB~\cite{jhuang2013towards}, and VIP~\cite{zhou2018adaptive}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1803.09453" id="555">
-        <attvalues>
-          <attvalue for="5" value=" Obtaining the best accuracy on such benchmarks requires fully supervised training from large amounts of dense pixel annotations, often with an additional step of online adaptation to each test sequence~\cite{bao2018cnn,caelles2017one,luiten2018premvos,DMMNet19}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1611.05198" id="556">
-        <attvalues>
-          <attvalue for="5" value=" Obtaining the best accuracy on such benchmarks requires fully supervised training from large amounts of dense pixel annotations, often with an additional step of online adaptation to each test sequence~\cite{bao2018cnn,caelles2017one,luiten2018premvos,DMMNet19}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1807.09190" id="557">
-        <attvalues>
-          <attvalue for="5" value=" Obtaining the best accuracy on such benchmarks requires fully supervised training from large amounts of dense pixel annotations, often with an additional step of online adaptation to each test sequence~\cite{bao2018cnn,caelles2017one,luiten2018premvos,DMMNet19}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1806.09594" id="558">
-        <attvalues>
-          <attvalue for="5" value=" Popular self-supervised pretext tasks include frame-to-frame colorization \cite{vondrick2018tracking, lai2020mast,lai2019self,li2019joint} and patch tracking with cycle consistency~\cite{wang2019learning,jabri2020walk}, while the propagation step relies on some variant of $k$-nearest-neighbors to label a location based on the most similar locations in previous frames, with the similarity computed from the pre-trained feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1903.07593" id="559">
-        <attvalues>
-          <attvalue for="5" value=" To underscore this point, Figure \ref{fig:imagenet-history} shows the performance of a supervised ImageNet-pretrained baseline on DAVIS as reported by three recent papers~\cite{wang2019learning,li2019joint,jabri2020walk}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1905.00875" id="560">
-        <attvalues>
-          <attvalue for="5" value=" Popular self-supervised pretext tasks include frame-to-frame colorization \cite{vondrick2018tracking, lai2020mast,lai2019self,li2019joint} and patch tracking with cycle consistency~\cite{wang2019learning,jabri2020walk}, while the propagation step relies on some variant of $k$-nearest-neighbors to label a location based on the most similar locations in previous frames, with the similarity computed from the pre-trained feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1909.11895" id="561">
-        <attvalues>
-          <attvalue for="5" value=" We introduce a hybrid method relying on signals from both UVC~\cite{li2019joint} video-based learning and a self-supervised image objective." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2002.07793" id="562">
-        <attvalues>
-          <attvalue for="5" value=" However, since large-scale pixel-level annotations are difficult to obtain, a number of recent works have focused instead on {self-supervised} approaches for learning temporal correspondence, where a feature representation pre-trained on a different dataset and pretext task is transferred to the desired label propagation benchmark with the help of a custom-designed propagation algorithm \cite{vondrick2018tracking,wang2019learning,lai2019self,li2019joint,lai2020mast,jabri2020walk,xu2021rethinking,araslanov2021dense}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2006.14613" id="563">
-        <attvalues>
-          <attvalue for="5" value=" Here, we find that several of the most recent methods, including our UVC+, CRW~\cite{jabri2020walk}, and VFS~\cite{xu2021rethinking}, achieve very similar levels of performance, close to our strong ImageNet baseline, despite relying on different combinations of cues and training particulars." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2103.17263" id="564">
-        <attvalues>
-          <attvalue for="5" value=" Here, we find that several of the most recent methods, including our UVC+, CRW~\cite{jabri2020walk}, and VFS~\cite{xu2021rethinking}, achieve very similar levels of performance, close to our strong ImageNet baseline, despite relying on different combinations of cues and training particulars." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2111.06265" id="565">
-        <attvalues>
-          <attvalue for="5" value=" However, since large-scale pixel-level annotations are difficult to obtain, a number of recent works have focused instead on {self-supervised} approaches for learning temporal correspondence, where a feature representation pre-trained on a different dataset and pretext task is transferred to the desired label propagation benchmark with the help of a custom-designed propagation algorithm \cite{vondrick2018tracking,wang2019learning,lai2019self,li2019joint,lai2020mast,jabri2020walk,xu2021rethinking,araslanov2021dense}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2003.08505" id="566">
-        <attvalues>
-          <attvalue for="5" value=" In the spirit of comparative studies that have taken a critical look at areas such as metric learning~\cite{musgrave2020metric}, self-supervised representation learning~\cite{kolesnikov2019revisiting}, and tracking~\cite{wang2021different}, we conduct a study to clarify performance differences between recent self-supervised video correspondence works by comparing several feature representations trained on still images and video." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="1901.09005" id="567">
-        <attvalues>
-          <attvalue for="5" value=" In the spirit of comparative studies that have taken a critical look at areas such as metric learning~\cite{musgrave2020metric}, self-supervised representation learning~\cite{kolesnikov2019revisiting}, and tracking~\cite{wang2021different}, we conduct a study to clarify performance differences between recent self-supervised video correspondence works by comparing several feature representations trained on still images and video." />
-        </attvalues>
-      </edge>
-      <edge source="2203.05553" target="2107.02156" id="568">
-        <attvalues>
-          <attvalue for="5" value=" In the spirit of comparative studies that have taken a critical look at areas such as metric learning~\cite{musgrave2020metric}, self-supervised representation learning~\cite{kolesnikov2019revisiting}, and tracking~\cite{wang2021different}, we conduct a study to clarify performance differences between recent self-supervised video correspondence works by comparing several feature representations trained on still images and video." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2006.00168" id="569">
-        <attvalues>
-          <attvalue for="5" value=" It is one of the oldest problems in computer vision and has essential applications in autonomous driving\cite{capito2020optical,wang2021end,wang2021learning}, video understanding\cite{fan2018end}, and human action recognition\cite{lertniphonphan2011human,arshad2022human}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2104.08862" id="570">
-        <attvalues>
-          <attvalue for="5" value=" It is one of the oldest problems in computer vision and has essential applications in autonomous driving\cite{capito2020optical,wang2021end,wang2021learning}, video understanding\cite{fan2018end}, and human action recognition\cite{lertniphonphan2011human,arshad2022human}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2104.12861" id="571">
-        <attvalues>
-          <attvalue for="5" value=" It is one of the oldest problems in computer vision and has essential applications in autonomous driving\cite{capito2020optical,wang2021end,wang2021learning}, video understanding\cite{fan2018end}, and human action recognition\cite{lertniphonphan2011human,arshad2022human}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="1804.00413" id="572">
-        <attvalues>
-          <attvalue for="5" value=" It is one of the oldest problems in computer vision and has essential applications in autonomous driving\cite{capito2020optical,wang2021end,wang2021learning}, video understanding\cite{fan2018end}, and human action recognition\cite{lertniphonphan2011human,arshad2022human}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="1709.02371" id="573">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning methods\cite{sun2018pwc,teed2020raft} have become the mainstream solution for optical flow problem and have achieved performance far superior to traditional handcrafted methods \cite{beauchemin1995computation} in benchmark tests\cite{Menze2015ISA,butler2012naturalistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2003.12039" id="574">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning methods\cite{sun2018pwc,teed2020raft} have become the mainstream solution for optical flow problem and have achieved performance far superior to traditional handcrafted methods \cite{beauchemin1995computation} in benchmark tests\cite{Menze2015ISA,butler2012naturalistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2104.02409" id="575">
-        <attvalues>
-          <attvalue for="5" value=" Compared to traditional methods, deep optical flow methods can learn robust matching features and contextual features from large-scale data, as well as infer the optical flow of occluded parts\cite{jiang2021learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2111.13680" id="576">
-        <attvalues>
-          <attvalue for="5" value=" The filtered optical flow results can be directly used to train complex geometric matching tasks\cite{sun2021loftr,xu2022gmflow,liu2022camliflow,ling2023learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="1512.02134" id="577">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we only used about 300 collected scenarios, which has made the ADF-supervised optical flow methods far superior in zero-shot generalization to self-supervised and supervised models trained on existing datasets \cite{mayer2016large,butler2012naturalistic,Menze2015ISA}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2303.01943" id="578">
-        <attvalues>
-          <attvalue for="5" value=" Although recent works in synthetic datasets\cite{mayer2016large,butler2012naturalistic,mehl2023spring} and self-supervised learning\cite{jonschkowski2020matters,kong2022mdflow} have alleviated this limitation to some extent, they are still subject to constraints in domain transferability, lighting variations, and occlusions." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2006.04902" id="579">
-        <attvalues>
-          <attvalue for="5" value=" Although recent works in synthetic datasets\cite{mayer2016large,butler2012naturalistic,mehl2023spring} and self-supervised learning\cite{jonschkowski2020matters,kong2022mdflow} have alleviated this limitation to some extent, they are still subject to constraints in domain transferability, lighting variations, and occlusions." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2211.06018" id="580">
-        <attvalues>
-          <attvalue for="5" value=" Although recent works in synthetic datasets\cite{mayer2016large,butler2012naturalistic,mehl2023spring} and self-supervised learning\cite{jonschkowski2020matters,kong2022mdflow} have alleviated this limitation to some extent, they are still subject to constraints in domain transferability, lighting variations, and occlusions." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2005.14165" id="581">
-        <attvalues>
-          <attvalue for="5" value=" In this era where data is the gold mine, massive training datasets are a necessary factor for the success of powerful models such as ChatGPT\cite{brown2020language} and SAM\cite{kirillov2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2304.02643" id="582">
-        <attvalues>
-          <attvalue for="5" value=" In this era where data is the gold mine, massive training datasets are a necessary factor for the success of powerful models such as ChatGPT\cite{brown2020language} and SAM\cite{kirillov2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2304.06706" id="583">
-        <attvalues>
-          <attvalue for="5" value=" Data generation: We build a data generator based on the latest radiation rendering technology (Nerf)\cite{barron2023zip}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2211.11738" id="584">
-        <attvalues>
-          <attvalue for="5" value=" Before us, the only evaluation metric used was ambient occlusion (AO) \cite{truong2023sparf,tosi2023nerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2303.17603" id="585">
-        <attvalues>
-          <attvalue for="5" value=" Before us, the only evaluation metric used was ambient occlusion (AO) \cite{truong2023sparf,tosi2023nerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2104.00680" id="586">
-        <attvalues>
-          <attvalue for="5" value=" The filtered optical flow results can be directly used to train complex geometric matching tasks\cite{sun2021loftr,xu2022gmflow,liu2022camliflow,ling2023learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.04246" target="2111.10502" id="587">
-        <attvalues>
-          <attvalue for="5" value=" The filtered optical flow results can be directly used to train complex geometric matching tasks\cite{sun2021loftr,xu2022gmflow,liu2022camliflow,ling2023learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1911.01223" target="1612.01601" id="588">
-        <attvalues>
-          <attvalue for="5" value=" Unlike patches split without specific meanings, superpixels \cite{superpixelsstateoftheart} are visually meaningful entities." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11038" target="1707.07184" id="589">
-        <attvalues>
-          <attvalue for="5" value=" We refer to \cite{tuceryan1993texture,raad2018survey} for a comprehensive review of classical models." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11038" target="1412.1123" id="590">
-        <attvalues>
-          <attvalue for="5" value=" More recent network based approaches with data-adaptive property is capable of accomplishing high level tasks as semantic segmentation, eg, \cite{bertasius2015deepedge, 9093290, xie15hed, he2019bi, liu2017richer}, assuming the network is well-trained." />
-        </attvalues>
-      </edge>
-      <edge source="2309.11326" target="2104.06456" id="591">
-        <attvalues>
-          <attvalue for="5" value=" Camera calibration is a vital first step in numerous computer vision applications, ranging from photogrammetry \cite{li2022review} and depth estimation \cite{mertan2022single} to robotics \cite{khan2019vision} and SLAM \cite{smith2006real,devernay2001straight}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.11326" target="2303.10559" id="592">
-        <attvalues>
-          <attvalue for="5" value=" Several attempts have been made to unify the calibration procedures for different camera types and camera systems \cite{Ramalingam2017,Hartley2004,liao2023deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.11326" target="2202.00158" id="593">
-        <attvalues>
-          <attvalue for="5" value=" A deep learning variant of this can be found in \cite{zhang2022learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.10559" target="2011.12108" id="594">
-        <attvalues>
-          <attvalue for="5" value=" \cite{fan2022wide}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.08040" target="2005.10356" id="595">
-        <attvalues>
-          <attvalue for="5" value=" Tracking Any Object (TAO)\cite{2020TAO} dataset is the first work to bring more sophisticated categories in an open world to the multiple-objects-tracking (MOT) community." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="1506.01497" id="596">
-        <attvalues>
-          <attvalue for="5" value=" Object localization in 2D images is a key task for many perception systems, eg, autonomous robots and cars, augmented reality headsets, visual search engines, etc Depending on the application, the task can take different forms such as object detection \cite{ren2015fasterrcnn,carion2020detr} and instance segmentation \cite{he2017mask, chen2018deeplab}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2005.12872" id="597">
-        <attvalues>
-          <attvalue for="5" value=" Object localization in 2D images is a key task for many perception systems, eg, autonomous robots and cars, augmented reality headsets, visual search engines, etc Depending on the application, the task can take different forms such as object detection \cite{ren2015fasterrcnn,carion2020detr} and instance segmentation \cite{he2017mask, chen2018deeplab}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="1703.06870" id="598">
-        <attvalues>
-          <attvalue for="5" value=" Object localization in 2D images is a key task for many perception systems, eg, autonomous robots and cars, augmented reality headsets, visual search engines, etc Depending on the application, the task can take different forms such as object detection \cite{ren2015fasterrcnn,carion2020detr} and instance segmentation \cite{he2017mask, chen2018deeplab}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="1405.0312" id="599">
-        <attvalues>
-          <attvalue for="5" value=" Achieving good results in these tasks has traditionally hinged on one critical factor: access to extensive, meticulously annotated datasets \cite{coco2014,pascal-voc-2007} to build and train deep neural networks." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2304.02643" id="600">
-        <attvalues>
-          <attvalue for="5" value=" The high interest around the recent Segment Anything (SAM) \cite{kirillov2023segment} model shows the desire for the community to segment any object in an image in a class-agnostic fashion." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2304.05387" id="601">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, as shown in \ref{fig:res-in-time}, recent methods (eg, MOST \cite{rambhatla2023most} and MOVE \cite{bielski2022move}) have achieved impressive results without any annotations, predicting accurate object boxes in over 75\% of the images in the VOC07 \cite{pascal-voc-2007} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2210.07920" id="602">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, as shown in \ref{fig:res-in-time}, recent methods (eg, MOST \cite{rambhatla2023most} and MOVE \cite{bielski2022move}) have achieved impressive results without any annotations, predicting accurate object boxes in over 75\% of the images in the VOC07 \cite{pascal-voc-2007} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2010.11929" id="603">
-        <attvalues>
-          <attvalue for="5" value=" First, Vision Transformer (ViT) models \cite{dosovitskiy2021vit} provide global correlations between patches when Convolutional Neural Networks (CNNs) only correlate pixels in a local receptive field." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2104.14294" id="604">
-        <attvalues>
-          <attvalue for="5" value=" Second, self-supervised representation learning \cite{caron2021dino, chen2021mocov3, he2022mae} has improved and scaled to massive datasets for feature learning." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2105.08127" id="605">
-        <attvalues>
-          <attvalue for="5" value=" With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="1707.06397" id="606">
-        <attvalues>
-          <attvalue for="5" value=" With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="1904.03148" id="607">
-        <attvalues>
-          <attvalue for="5" value=" With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2007.02662" id="608">
-        <attvalues>
-          <attvalue for="5" value=" With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2106.06650" id="609">
-        <attvalues>
-          <attvalue for="5" value=" With such strategies, there is no need to \OS{carefully design hand-crafted methods \cite{yan2013hs, zhu2014wctr}, use generative adversarial models \cite{melas_kyriazi2021_imageseg}, refine noisy labels \cite{nguyen2019deepusps},} nor to interpret the thousands of object proposals generated by handcrafted methods~\cite{zitnick2014edgebox, uijlings2013selectivesearch} (with a high-recall but low precision) using expensive dataset-level quadratic pattern repetition search \cite{Wei2019ddtplus,vo2019unsup_image_matching,vo2020unsup_multi_object_discovery,vo2021largescale}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2002.08721" id="610">
-        <attvalues>
-          <attvalue for="5" value=" To our knowledge, this is the first survey on such topic and we would like to point the reader to related surveys on image classification with limited annotations~\cite{schmarje2021survey_classification}, weakly supervised object localization~\cite{zhang2022weakly_supervised_survey,shao2022weakly_supervised_survey}, object localization on natural scenes \cite{choudhuri2018survey_localization_natural}, object instance segmentation \cite{sharma2022survey_segmentation}, object segmentation \cite{wang2022review_segmentation}, and object detection \cite{amjoud2023review_detection,shehzadi2023detection_transformers}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2104.07918" id="611">
-        <attvalues>
-          <attvalue for="5" value=" To our knowledge, this is the first survey on such topic and we would like to point the reader to related surveys on image classification with limited annotations~\cite{schmarje2021survey_classification}, weakly supervised object localization~\cite{zhang2022weakly_supervised_survey,shao2022weakly_supervised_survey}, object localization on natural scenes \cite{choudhuri2018survey_localization_natural}, object instance segmentation \cite{sharma2022survey_segmentation}, object segmentation \cite{wang2022review_segmentation}, and object detection \cite{amjoud2023review_detection,shehzadi2023detection_transformers}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2301.07499" id="612">
-        <attvalues>
-          <attvalue for="5" value=" To our knowledge, this is the first survey on such topic and we would like to point the reader to related surveys on image classification with limited annotations~\cite{schmarje2021survey_classification}, weakly supervised object localization~\cite{zhang2022weakly_supervised_survey,shao2022weakly_supervised_survey}, object localization on natural scenes \cite{choudhuri2018survey_localization_natural}, object instance segmentation \cite{sharma2022survey_segmentation}, object segmentation \cite{wang2022review_segmentation}, and object detection \cite{amjoud2023review_detection,shehzadi2023detection_transformers}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.12904" target="2306.04670" id="613">
-        <attvalues>
-          <attvalue for="5" value=" To our knowledge, this is the first survey on such topic and we would like to point the reader to related surveys on image classification with limited annotations~\cite{schmarje2021survey_classification}, weakly supervised object localization~\cite{zhang2022weakly_supervised_survey,shao2022weakly_supervised_survey}, object localization on natural scenes \cite{choudhuri2018survey_localization_natural}, object instance segmentation \cite{sharma2022survey_segmentation}, object segmentation \cite{wang2022review_segmentation}, and object detection \cite{amjoud2023review_detection,shehzadi2023detection_transformers}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1506.01497" id="614">
-        <attvalues>
-          <attvalue for="5" value=" The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1504.08083" id="615">
-        <attvalues>
-          <attvalue for="5" value=" The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1804.02767" id="616">
-        <attvalues>
-          <attvalue for="5" value=" The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1708.02002" id="617">
-        <attvalues>
-          <attvalue for="5" value=" The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2010.11929" id="618">
-        <attvalues>
-          <attvalue for="5" value=" However, the recent success of transformers in natural language processing (NLP) has led researchers to explore their potential in computer vision as well \cite{ViT23}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1706.03762" id="619">
-        <attvalues>
-          <attvalue for="5" value=" The transformer architecture \cite{att75} has been shown to be effective in capturing long-range dependencies in sequential data \cite{att75}, making it an attractive candidate for object detection tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="1311.2524" id="620">
-        <attvalues>
-          <attvalue for="5" value=" The DETR network shows promising results, outperforming conventional CNN-based object detectors \cite{fasterrcnn3,fastrcnn5,yolov3,retinaNet68} while also eliminating the need for hand-crafted components such as region proposal networks and post-processing steps such as non-maximum suppression (NMS) \cite{rcnn13}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2010.04159" id="621">
-        <attvalues>
-          <attvalue for="5" value=" Deformable-DETR \cite{Deformable54} modifies the attention modules to process the image feature maps by considering the attention mechanism as the main reason for slow training convergence." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2011.09094" id="622">
-        <attvalues>
-          <attvalue for="5" value=" UP-DETR \cite{updetr23} proposes a few modifications to Pre-train the DETR similar to the pretraining of transformers in natural language processing." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2104.01318" id="623">
-        <attvalues>
-          <attvalue for="5" value=" Efficient-DETR \cite{efficientDE} based on original DETR and Deformable-DETR examines the randomly initialized object probabilities, including reference points and object queries, which is one of the reasons for multiple training iterations." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2108.02404" id="624">
-        <attvalues>
-          <attvalue for="5" value=" SMCA-DETR \cite{smca23} introduces a Spatially-Modulated Co-attention module that replaces the existing co-attention mechanism in DETR to overcome the slow training convergence of DETR." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2011.10881" id="625">
-        <attvalues>
-          <attvalue for="5" value=" TSP-DETR \cite{Reth78} deals with the cross-attention and the instability of bipartite matching to overcome the slow training convergence of DETR." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2108.06152" id="626">
-        <attvalues>
-          <attvalue for="5" value=" Conditional-DETR \cite{CondDE} presents a conditional cross-attention mechanism to solve the training convergence issue of DETR." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2109.07036" id="627">
-        <attvalues>
-          <attvalue for="5" value=" PnP-DETR \cite{pnp6} proposes a PnP sampling module to reduce spatial redundancy and make the transformer network computationally more efficient." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2106.00666" id="628">
-        <attvalues>
-          <attvalue for="5" value=" YOLOS-DETR \cite{yolos6} presents the transferability and versatility of the Transformer from image recognition to detection in the sequence aspect using the least information about the spatial design of the input and improves performance." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2111.14330" id="629">
-        <attvalues>
-          <attvalue for="5" value=" Sparse-DETR \cite{sparsedetr} reduces the computational cost by filtering encoder tokens with learnable cross-attention maps." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2203.01305" id="630">
-        <attvalues>
-          <attvalue for="5" value=" DN-DETR \cite{dn42} uses noised object queries as additional decoder input to reduce the instability of the bipartite-matching mechanism in DETR, which causes the slow convergence problem." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2203.16507" id="631">
-        <attvalues>
-          <attvalue for="5" value=" AdaMixer \cite{adamixer7} considers the encoder an extra network between the backbone and decoder that limits the performance and slower the training convergence because of its design complexity." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2112.04632" id="632">
-        <attvalues>
-          <attvalue for="5" value=" REGO-DETR \cite{rego2} proposes an RoI-based method for detection refinement to improve the attention mechanism in the detection transformer." />
-        </attvalues>
-      </edge>
-      <edge source="2306.04670" target="2203.03605" id="633">
-        <attvalues>
-          <attvalue for="5" value=" DINO \cite{dino23} considers positive and negative noised object queries to make training convergence faster and to enhance the performance on small objects." />
-        </attvalues>
-      </edge>
-      <edge source="2406.12285" target="2107.05475" id="634">
-        <attvalues>
-          <attvalue for="5" value=" Object detection is now widely applied across various fields, such as smart transportation~\cite{jiao2023dsam}, medical diagnosis~\cite{ni2018multiple}, industrial manufacturing~\cite{zhao2018surface}, and re-identification~\cite{shen2023git}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.12285" target="1405.0312" id="635">
-        <attvalues>
-          <attvalue for="5" value=" According to the MS COCO dataset~\cite{lin2014microsoft} definition in the field of object detection, small objects are those smaller than 32 × 32 pixels." />
-        </attvalues>
-      </edge>
-      <edge source="2406.12285" target="2404.01614" id="636">
-        <attvalues>
-          <attvalue for="5" value=" LR-FPN~\cite{li2024lr} improves remote sensing target detection by enhancing low-level positional information and fine-grained context interaction, leading to superior performance in agricultural and urban planning applications." />
-        </attvalues>
-      </edge>
-      <edge source="2111.12315" target="1511.00423" id="637">
-        <attvalues>
-          <attvalue for="5" value=" DT has been widely used in various applications such as video retrieval \cite{RelevanceFeedback}, fire detection \cite{5208233}, and micro-expression analysis \cite{7851001}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.12315" target="1502.00303" id="638">
-        <attvalues>
-          <attvalue for="5" value=" Many methods have been developed for dynamic texture recognition, \eg, geometric properties computation \cite{QUAN201785}, local spatio-temporal filtering \cite{Ramirez2015}, dictionary learning~\cite{7780409}, deep convolutional neural network~\cite{QI20161230, shervin2017dynamic}, and various spatial-temporal local binary patterns~\cite{4160945, MBSIFTOP, LPQTOP,ren2013dynamic, ren2015chi, ren2015learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="2011.08612" id="639">
-        <attvalues>
-          <attvalue for="5" value=" Natural image matting refers to estimating a soft foreground from a natural image, which is a fundamental process for many applications, eg, film post-production and image editing~\cite{chen2013knn,zhang2020empowering}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="1909.04686" id="640">
-        <attvalues>
-          <attvalue for="5" value=" Since image matting is a highly ill-posed problem, previous methods usually adopt auxiliary user input, eg trimap~\cite{sun2004poisson,cai2019disentangled}, scribble~\cite{levin2007closed}, or background image~\cite{backgroundmatting} as constraints." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="2004.00626" id="641">
-        <attvalues>
-          <attvalue for="5" value=" Since image matting is a highly ill-posed problem, previous methods usually adopt auxiliary user input, eg trimap~\cite{sun2004poisson,cai2019disentangled}, scribble~\cite{levin2007closed}, or background image~\cite{backgroundmatting} as constraints." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="1908.00672" id="642">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods estimate the alpha value by sampling neighboring pixels~\cite{wang2007optimized} or defining affinity metrics for alpha propagation~\cite{levin2008spectral}, deep learning-based approaches solve it by learning discriminative representations from a large amount of labeled data and predicting alpha matte directly~\cite{lu2019indices,li2020natural}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="2001.04069" id="643">
-        <attvalues>
-          <attvalue for="5" value=" While traditional methods estimate the alpha value by sampling neighboring pixels~\cite{wang2007optimized} or defining affinity metrics for alpha propagation~\cite{levin2008spectral}, deep learning-based approaches solve it by learning discriminative representations from a large amount of labeled data and predicting alpha matte directly~\cite{lu2019indices,li2020natural}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="1809.01354" id="644">
-        <attvalues>
-          <attvalue for="5" value=" Prior AIM methods~\cite{Qiao_2020_CVPR,gfm} solve this problem by learning semantic features from the image to aid the matting process but is limited to images with salient opaque foregrounds, eg, human~\cite{shen2016deep,chen2018semantic}, animal~\cite{gfm}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="1709.01507" id="645">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, 1) we use the generalized trimap as the semantic representation in the semantic decoder to adapt it for all types of images; 2) we exploit the effective SE attention \cite{hu2018squeeze} in the semantic decoder to learn better semantic features to handle different characteristics of foreground alpha mattes; 3) we improve the interaction between the semantic decoder and matting decoder by devising a spatial attention module, which guides the matting decoder to focus on the details in the transition areas." />
-        </attvalues>
-      </edge>
-      <edge source="2107.07235" target="1405.0312" id="646">
-        <attvalues>
-          <attvalue for="5" value=" Previous methods use composition images by pasting different foregrounds on background images from COCO dataset~\cite{lin2014microsoft} for evaluation, which may introduce composition artifacts and are not representative for natural images as mentioned in~\cite{gfm}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.06868" target="1703.06870" id="647">
-        <attvalues>
-          <attvalue for="5" value=" Instance segmentation methods such as Mask R-CNN \cite{He2017} perform better on images of size $1024\times1024$ pixels but on smaller size images, the outputted Region of Interests (RoI) do not align well with long, 1D-barcode structures." />
-        </attvalues>
-      </edge>
-      <edge source="2102.06868" target="2004.10934" id="648">
-        <attvalues>
-          <attvalue for="5" value=" In the same way, dedicated object detection pipelines, such as YOLOv4 \cite{Bochkovskiy2020}, though they perform well on lower Intersectionover Union (IoU) thresholds, suffer accuracy at higher IoU thresholds." />
-        </attvalues>
-      </edge>
-      <edge source="2102.06868" target="1906.06281" id="649">
-        <attvalues>
-          <attvalue for="5" value=" Among those using segmentation on LR images as a means for detection, \cite{Zharkov2019} also tends to not perform well at higher IoU thresholds." />
-        </attvalues>
-      </edge>
-      <edge source="2102.06868" target="1506.01497" id="650">
-        <attvalues>
-          <attvalue for="5" value=" \ref{ynet-full-pipeline}}, in the first stage, we expand on the Region Proposal Network (RPN) introduced in Faster R-CNN \cite{Girshick2015} to extract high definition regions of potential locations where barcodes might be." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05044" target="1503.02619" id="651">
-        <attvalues>
-          <attvalue for="5" value=" RANSAC and many of its variants have been successfully applied to a wide range of vision tasks, for instance, short baseline stereo~\cite{torr1993outlier,torr1998robust}, motion segmentation~\cite{torr1993outlier}, detection of geometric primitives~\cite{sminchisescu2005incremental}, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, in structure-from-motion~\cite{agarwal2010bundle,wilson2014robust, schonberger2016structure} (SfM) or simultaneous localization and mapping~\cite{engel2014lsd,mur2015orb} (SLAM) pipelines, image mosaicing~\cite{ghosh2016survey}, and to perform~\cite{zuliani2005multiransac} or initialize multi-model fitting~\cite{isack2012energy,pham2014interacting}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05044" target="1502.00956" id="652">
-        <attvalues>
-          <attvalue for="5" value=" RANSAC and many of its variants have been successfully applied to a wide range of vision tasks, for instance, short baseline stereo~\cite{torr1993outlier,torr1998robust}, motion segmentation~\cite{torr1993outlier}, detection of geometric primitives~\cite{sminchisescu2005incremental}, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, in structure-from-motion~\cite{agarwal2010bundle,wilson2014robust, schonberger2016structure} (SfM) or simultaneous localization and mapping~\cite{engel2014lsd,mur2015orb} (SLAM) pipelines, image mosaicing~\cite{ghosh2016survey}, and to perform~\cite{zuliani2005multiransac} or initialize multi-model fitting~\cite{isack2012energy,pham2014interacting}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05044" target="1706.00984" id="653">
-        <attvalues>
-          <attvalue for="5" value=" This approach had been further improved in Graph-Cut RANSAC~\cite{barath2018graph} considering the fact that real-world data often form spatially coherent structures." />
-        </attvalues>
-      </edge>
-      <edge source="2104.05044" target="1906.02295" id="654">
-        <attvalues>
-          <attvalue for="5" value=" This criterion was then relaxed by Progressive NAPSAC~\cite{barath2019progressive} by terminating if the probability of finding a model which has significantly more inliers than the previous best falls below a threshold." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1611.01704" id="655">
-        <attvalues>
-          <attvalue for="5" value=" The data include images generated from popular engineered \cite{skodras2001jpeg,wallace1992jpeg,bpg} and learned compression methods \cite{mentzer2018conditional,balle2016end,balle2018variational,patel2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1802.01436" id="656">
-        <attvalues>
-          <attvalue for="5" value=" The data include images generated from popular engineered \cite{skodras2001jpeg,wallace1992jpeg,bpg} and learned compression methods \cite{mentzer2018conditional,balle2016end,balle2018variational,patel2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1801.04260" id="657">
-        <attvalues>
-          <attvalue for="5" value=" The data include images generated from popular engineered \cite{skodras2001jpeg,wallace1992jpeg,bpg} and learned compression methods \cite{mentzer2018conditional,balle2016end,balle2018variational,patel2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1705.05823" id="658">
-        <attvalues>
-          <attvalue for="5" value=" Despite of the short comings, recent image compression literature continues to evaluate models using MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context,patel2019deep,rippel2017real,theis2015generative,theis2017lossy}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1809.10452" id="659">
-        <attvalues>
-          <attvalue for="5" value=" Despite of the short comings, recent image compression literature continues to evaluate models using MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context,patel2019deep,rippel2017real,theis2015generative,theis2017lossy}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1907.08310" id="660">
-        <attvalues>
-          <attvalue for="5" value=" The data include images generated from popular engineered \cite{skodras2001jpeg,wallace1992jpeg,bpg} and learned compression methods \cite{mentzer2018conditional,balle2016end,balle2018variational,patel2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1703.00395" id="661">
-        <attvalues>
-          <attvalue for="5" value=" Despite of the short comings, recent image compression literature continues to evaluate models using MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context,patel2019deep,rippel2017real,theis2015generative,theis2017lossy}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1312.6114" id="662">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning models for image compression can be broadly categorized into three generative models: a) Variational Auto-Encoders \cite{kingma2013auto,balle2018variational,balle2016end,lee2018context}; b) Generative Adversarial Networks \cite{rippel2017real,agustsson2018generative} (GAN) \cite{goodfellow2014generative}; c) Auto-Regressive (AR) \cite{van2016conditional,mentzer2018conditional}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1804.02958" id="663">
-        <attvalues>
-          <attvalue for="5" value=" GANs are useful in very low bit-rate settings as they can learn to synthesize the images \cite{agustsson2018generative}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1606.05328" id="664">
-        <attvalues>
-          <attvalue for="5" value=" A hierarchical auto-regressive model with two-stages is designed, which is realized by two 3D Pixel-CNN \cite{van2016conditional}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1908.04187" id="665">
-        <attvalues>
-          <attvalue for="5" value=" Both of these metrics do not align well with human perception of similarity \cite{patel2019human}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1801.03924" id="666">
-        <attvalues>
-          <attvalue for="5" value=" As Patel ~\etal \cite{patel2019deep} found, using the model \cite{zhang18} has limitations for compression." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1711.06077" id="667">
-        <attvalues>
-          <attvalue for="5" value=" The super-resolution methods have started using a more sophisticated learned perceptual similarity metric \cite{blau2018perception,blau20182018} for evaluation, exactly for the mentioned reasons." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1809.07517" id="668">
-        <attvalues>
-          <attvalue for="5" value=" The super-resolution methods have started using a more sophisticated learned perceptual similarity metric \cite{blau2018perception,blau20182018} for evaluation, exactly for the mentioned reasons." />
-        </attvalues>
-      </edge>
-      <edge source="2002.04988" target="1506.03478" id="669">
-        <attvalues>
-          <attvalue for="5" value=" Despite of the short comings, recent image compression literature continues to evaluate models using MS-SSIM and PSNR \cite{balle2016end,balle2018variational,mentzer2018conditional,lee2018context,patel2019deep,rippel2017real,theis2015generative,theis2017lossy}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="1604.01685" id="670">
-        <attvalues>
-          <attvalue for="5" value=" Despite its importance, semantic segmentation faces challenges due to the requirement for highly precise pixel-level labels, which makes data preparation time-consuming and costly~\cite{cordts2016cityscapes}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="1811.08305" id="671">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation is widely utilized in various fields that demand accurate and detailed predictions, such as autonomous driving and medical imaging~\cite{cordts2016cityscapes,dolz2018ivd}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2102.09896" id="672">
-        <attvalues>
-          <attvalue for="5" value=" These include methods ranging from scribble-level~\cite{pan2021scribble} to point-level~\cite{tel_liang2022tree,bearman2016s}, image-level~\cite{eps_lee2021railroad,SEAM_wang2020self}, and zero-shot approaches based on Vision-Language (VL) models~\cite{maskclip_zhou2022extract} (extracting templates from text)." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2203.10739" id="673">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, we observe performance improvements over the state-of-the-art TEL~\cite{tel_liang2022tree} for scribble and point level labels by 4." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2105.08965" id="674">
-        <attvalues>
-          <attvalue for="5" value=" These include methods ranging from scribble-level~\cite{pan2021scribble} to point-level~\cite{tel_liang2022tree,bearman2016s}, image-level~\cite{eps_lee2021railroad,SEAM_wang2020self}, and zero-shot approaches based on Vision-Language (VL) models~\cite{maskclip_zhou2022extract} (extracting templates from text)." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2004.04581" id="675">
-        <attvalues>
-          <attvalue for="5" value=" These include methods ranging from scribble-level~\cite{pan2021scribble} to point-level~\cite{tel_liang2022tree,bearman2016s}, image-level~\cite{eps_lee2021railroad,SEAM_wang2020self}, and zero-shot approaches based on Vision-Language (VL) models~\cite{maskclip_zhou2022extract} (extracting templates from text)." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2112.01071" id="676">
-        <attvalues>
-          <attvalue for="5" value=" Lastly, when using text-driven labels in a vision-language model like MaskCLIP~\cite{maskclip_zhou2022extract}, our method outperforms existing method~\cite{segformer_xie2021segformer} by 11." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2110.03740" id="677">
-        <attvalues>
-          <attvalue for="5" value="1\%p, and over ADELE~\cite{adele_liu2022adaptive} by 1." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2104.14294" id="678">
-        <attvalues>
-          <attvalue for="5" value=" We also confirm that various types of self-supervised vision transformers~\cite{ibot_zhou2021image,dino_caron2021emerging,dinov2_oquab2023dinov2} are effective in learning from imperfect labels compared to traditional methods for training segmentation networks." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2304.07193" id="679">
-        <attvalues>
-          <attvalue for="5" value=" We also confirm that various types of self-supervised vision transformers~\cite{ibot_zhou2021image,dino_caron2021emerging,dinov2_oquab2023dinov2} are effective in learning from imperfect labels compared to traditional methods for training segmentation networks." />
-        </attvalues>
-      </edge>
-      <edge source="2401.12535" target="2105.15203" id="680">
-        <attvalues>
-          <attvalue for="5" value=" Lastly, when using text-driven labels in a vision-language model like MaskCLIP~\cite{maskclip_zhou2022extract}, our method outperforms existing method~\cite{segformer_xie2021segformer} by 11." />
-        </attvalues>
-      </edge>
-      <edge source="2210.16057" target="1803.03243" id="681">
-        <attvalues>
-          <attvalue for="5" value=" Images captured in hazy weather often suffer from noticeable visibility degradation, color distortions, and contrast reduction, which further drop the performance of downstream vision-based systems such as object detection, autonomous driving, and traffic surveillance \cite{chen2018domain,choi2017sharpness}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.16057" target="1811.08747" id="682">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning open up huge opportunities for image dehazing tasks, and a large number of learning-based approaches have sprung up \cite{li2017aod,chen2019gated,qin2020ffa,song2022vision}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.16057" target="1911.07559" id="683">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning open up huge opportunities for image dehazing tasks, and a large number of learning-based approaches have sprung up \cite{li2017aod,chen2019gated,qin2020ffa,song2022vision}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.16057" target="2204.03883" id="684">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning open up huge opportunities for image dehazing tasks, and a large number of learning-based approaches have sprung up \cite{li2017aod,chen2019gated,qin2020ffa,song2022vision}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.16057" target="2006.16829" id="685">
-        <attvalues>
-          <attvalue for="5" value=" Recently, several semi-supervised \cite{li2019semi,chen2021psd} and unsupervised \cite{li2021you,wang2022cycle} methods have attempted to solve the domain shift issue via training models on real-world images." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09585" target="1901.03577" id="686">
-        <attvalues>
-          <attvalue for="5" value=" {Moving Object Segmentation (MOS) is an important problem in computer vision, particularly in surveillance system applications \cite{garcia2020background}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09585" target="1811.05255" id="687">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning models \cite{BOUWMANS20198} have demonstrated strong performance on large-scale datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09585" target="2002.04993" id="688">
-        <attvalues>
-          <attvalue for="5" value=" In addition to these, some state-of-the-art (SOTA) techniques have been combined with deep methods to create novel approaches, such as MotionRec \cite{mandal2020motionrec}, RT-SBS \cite{cioppa2020real}, and GraphMOS \cite{giraldo2020graph}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09585" target="1712.00468" id="689">
-        <attvalues>
-          <attvalue for="5" value=" These methods are inspired by the theory of graph signal processing \cite{ortega2018graph} and have shown promising results." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09585" target="2104.13478" id="690">
-        <attvalues>
-          <attvalue for="5" value="} In this work, we propose a novel Graph Inductive Moving Object Segmentation (GraphIMOS) algorithm based on Graph Neural Networks (GNNs) \cite{bronstein2021geometric}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.09354" target="1910.11336" id="691">
-        <attvalues>
-          <attvalue for="5" value=" The changes and optimizations of this mode were presented in a new publication in 2019~\cite{liba2019handheld}." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1407.5035" id="692">
-        <attvalues>
-          <attvalue for="5" value=" To deal with the partial lackness of bounding-box-level annotations, LSDA \cite{hoffman2014lsda,tang2018visual} adopts the method of domain adaptation and knowledge transfer, which trains the classifier for all categories and transfers the classifier to object detector." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1801.03145" id="693">
-        <attvalues>
-          <attvalue for="5" value=" To deal with the partial lackness of bounding-box-level annotations, LSDA \cite{hoffman2014lsda,tang2018visual} adopts the method of domain adaptation and knowledge transfer, which trains the classifier for all categories and transfers the classifier to object detector." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1506.02640" id="694">
-        <attvalues>
-          <attvalue for="5" value=" Based on YOLO v2 \cite{redmon2016you, redmon2017yolo9000}, YOLO-9000 proposes an end-to-end one-stage object detection pipeline to jointly train the image-level annotated and bounding-box-level annotated images simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1504.08083" id="695">
-        <attvalues>
-          <attvalue for="5" value=" Due to the superiority in the accuracy of two-stage object detection, such as Fast R-CNN \cite{girshick2015fast}, Faster R-CNN \cite{ren2015faster}, R-FCN \cite{dai2016r} and R-FCN-3000 \cite{singh2018r}, it is valuable to realize more accurate cross-supervised learning based on the two-stage pipeline." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1506.01497" id="696">
-        <attvalues>
-          <attvalue for="5" value=" Due to the superiority in the accuracy of two-stage object detection, such as Fast R-CNN \cite{girshick2015fast}, Faster R-CNN \cite{ren2015faster}, R-FCN \cite{dai2016r} and R-FCN-3000 \cite{singh2018r}, it is valuable to realize more accurate cross-supervised learning based on the two-stage pipeline." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1605.06409" id="697">
-        <attvalues>
-          <attvalue for="5" value=" Due to the superiority in the accuracy of two-stage object detection, such as Fast R-CNN \cite{girshick2015fast}, Faster R-CNN \cite{ren2015faster}, R-FCN \cite{dai2016r} and R-FCN-3000 \cite{singh2018r}, it is valuable to realize more accurate cross-supervised learning based on the two-stage pipeline." />
-        </attvalues>
-      </edge>
-      <edge source="1905.12863" target="1712.01802" id="698">
-        <attvalues>
-          <attvalue for="5" value=" Due to the superiority in the accuracy of two-stage object detection, such as Fast R-CNN \cite{girshick2015fast}, Faster R-CNN \cite{ren2015faster}, R-FCN \cite{dai2016r} and R-FCN-3000 \cite{singh2018r}, it is valuable to realize more accurate cross-supervised learning based on the two-stage pipeline." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2006.11339" id="699">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, many VPS datasets have emerged, including Cityscapes-VPS~\cite{kim2020video}, KITTI-STEP~\cite{weber2021step}, VIPSeg~\cite{miao2022large}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2102.11859" id="700">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, many VPS datasets have emerged, including Cityscapes-VPS~\cite{kim2020video}, KITTI-STEP~\cite{weber2021step}, VIPSeg~\cite{miao2022large}." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2204.04656" id="701">
-        <attvalues>
-          <attvalue for="5" value=" For example, Video K-Net~\cite{li2022video} is built upon K-Net, a method that unifies image segmentation via a group of learnable kernels." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2301.02657" id="702">
-        <attvalues>
-          <attvalue for="5" value=" Tarvis~\cite{athar2023tarvis} proposes a novel, unified network architecture that can be applied to any task that requires segmenting a set of arbitrarily defined `targets' in the video." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2304.04694" id="703">
-        <attvalues>
-          <attvalue for="5" value=" Video-KMax~\cite{shin2023video} propose a unified approach for online and near-online VPS." />
-        </attvalues>
-      </edge>
-      <edge source="2306.06753" target="2205.08534" id="704">
-        <attvalues>
-          <attvalue for="5" value=" For the second challenge, we add an additional image semantic segmentation model ViT-Adapter~\cite{chen2022vision} trained on VSPW~\cite{miao2021vspw} to further improve the performance of semantic classes (stuff classes)." />
-        </attvalues>
-      </edge>
-      <edge source="2002.06429" target="1807.04067" id="705">
-        <attvalues>
-          <attvalue for="5" value=" Research studies in this area have recently achieved significant progress due to the development of different sophisticated deep learning architectures \cite{kocabas2018multiposenet, insafutdinov2016deeper, sun2019deep, xiao2018simple, zhang2019semantics} and several large-scale datasets such as \cite{andriluka20142d, lin2014microsoft, li2019crowdpose}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.06429" target="1711.10872" id="706">
-        <attvalues>
-          <attvalue for="5" value=" \cite{ye2018occlusion} considered hand pose estimation or Reddy et al." />
-        </attvalues>
-      </edge>
-      <edge source="2002.06429" target="1702.05693" id="707">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of non-availability of required pose annotations for pedestrians from training datasets such as CityPersons \cite{zhang2017citypersons}, we make use of relevant samples of MS-COCO dataset \cite{lin2014microsoft} for pose supervision during the training phase and employ instance-level domain adaptation strategy for this purpose." />
-        </attvalues>
-      </edge>
-      <edge source="2002.06429" target="1405.0312" id="708">
-        <attvalues>
-          <attvalue for="5" value=" To alleviate the problem of non-availability of required pose annotations for pedestrians from training datasets such as CityPersons \cite{zhang2017citypersons}, we make use of relevant samples of MS-COCO dataset \cite{lin2014microsoft} for pose supervision during the training phase and employ instance-level domain adaptation strategy for this purpose." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="1608.01807" id="709">
-        <attvalues>
-          <attvalue for="5" value=" Three eras that vary in how researchers export the different features that define an image's visual content characterize content-based image retrieval \cite{zheng2018sift}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="1706.03762" id="710">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, in natural language processing (NLP), the self-attention-based architecture, particularly Transformers, is now considered as the new standard \cite{vaswani2017attention}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="1711.07971" id="711">
-        <attvalues>
-          <attvalue for="5" value=" The Transformer is a type of deep-neural network mainly based on self-attention mechanism \cite{wang2018non}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="2012.12556" id="712">
-        <attvalues>
-          <attvalue for="5" value=" Recently, researchers have expanded transformers for computer vision tasks inspired by the influence of the Transformer in NLP \cite{han2020survey,locatello2020object,hu2018relation}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="2006.15055" id="713">
-        <attvalues>
-          <attvalue for="5" value=" Recently, researchers have expanded transformers for computer vision tasks inspired by the influence of the Transformer in NLP \cite{han2020survey,locatello2020object,hu2018relation}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="1711.11575" id="714">
-        <attvalues>
-          <attvalue for="5" value=" Recently, researchers have expanded transformers for computer vision tasks inspired by the influence of the Transformer in NLP \cite{han2020survey,locatello2020object,hu2018relation}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.03771" target="2010.11929" id="715">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the authors in \cite{dosovitskiy2020image} have attempted to apply a typical transformer directly to images, with the fewest adjustments possible shaping an image recognition network." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2312.13763" id="716">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there has been a significant focus on the use of text/image-prompts to describe appearance and motion~\cite{ling2023align, ren2023dreamgaussian4d, zhao2023animate124}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2303.11328" id="717">
-        <attvalues>
-          <attvalue for="5" value=" The general pipeline of such existing consists of two stpdf: (i) acquiring static geometry through a 3D generation (eg, a text/image-to-3D) model~\cite{liu2023zero}, which generates 3D representation such as meshes/implicit fields according to text/image prompts, and (ii) obtaining motion information via a video generation model~\cite{blattmann2023stable}, which generates video according to text/image prompts." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2311.15127" id="718">
-        <attvalues>
-          <attvalue for="5" value=" The general pipeline of such existing consists of two stpdf: (i) acquiring static geometry through a 3D generation (eg, a text/image-to-3D) model~\cite{liu2023zero}, which generates 3D representation such as meshes/implicit fields according to text/image prompts, and (ii) obtaining motion information via a video generation model~\cite{blattmann2023stable}, which generates video according to text/image prompts." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2403.17920" id="719">
-        <attvalues>
-          <attvalue for="5" value="\ref{compare}, most existing methods use text as prompts to describe the desired motion, with some efforts~\cite{bahmani2024tc4d} allowing trajectory (root body transformation) control." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2401.08809" id="720">
-        <attvalues>
-          <attvalue for="5" value=" Also, we leverage dynamic rigidity~\cite{zhang2024learning} to guarantee smoothness between frames, unlike the existing approaches~\cite{song20213d, 10076900}, which perform frame-independent pose transfer." />
-        </attvalues>
-      </edge>
-      <edge source="2405.14017" target="2109.15025" id="721">
-        <attvalues>
-          <attvalue for="5" value=" Also, we leverage dynamic rigidity~\cite{zhang2024learning} to guarantee smoothness between frames, unlike the existing approaches~\cite{song20213d, 10076900}, which perform frame-independent pose transfer." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="1807.06403" id="722">
-        <attvalues>
-          <attvalue for="5" value=" Model-based methods, such as bilinear interpolation, Malvar \cite{malvar2004high}, or Menon \cite{menon2006demosaicing}, are still widely used, but they fail to match the performance of recent deep learning-based approaches using deep convolutional networks (CNN) \cite{kokkinos2019iterative, gharbi2016deep, kerepecky2021d3net} or Swin Transformers \cite{xing2022residual}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2204.07098" id="723">
-        <attvalues>
-          <attvalue for="5" value=" Model-based methods, such as bilinear interpolation, Malvar \cite{malvar2004high}, or Menon \cite{menon2006demosaicing}, are still widely used, but they fail to match the performance of recent deep learning-based approaches using deep convolutional networks (CNN) \cite{kokkinos2019iterative, gharbi2016deep, kerepecky2021d3net} or Swin Transformers \cite{xing2022residual}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2111.11426" id="724">
-        <attvalues>
-          <attvalue for="5" value=" NF has also been used in image-to-image translation tasks such as superresolution, denoising, inpainting, and generative modeling \cite{xie2022neural}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2003.08934" id="725">
-        <attvalues>
-          <attvalue for="5" value=" NF has been applied in various domains and applications including Neural Radiance Fields (NeRF) \cite{mildenhall2021nerf} which achieved state-of-the-art results in representing complex 3D scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2110.13903" id="726">
-        <attvalues>
-          <attvalue for="5" value=" NeRV \cite{chen2021nerv} encodes entire videos in neural networks." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2012.09161" id="727">
-        <attvalues>
-          <attvalue for="5" value=" The Local Implicit Image Function (LIIF) \cite{chen2021learning} represents an image as a neural field capable of extrapolating to 30 times higher resolution." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2006.09661" id="728">
-        <attvalues>
-          <attvalue for="5" value=" SIREN \cite{sitzmann2020implicit} uses a sinusoidal neural representation and demonstrates superiority over classical ReLU MLP in representing complex natural signals such as images." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2011.12026" id="729">
-        <attvalues>
-          <attvalue for="5" value=" Prior information from training data can be encoded into neural representation through conditioning (local or global) using methods such as concatenation, modulation of activation functions \cite{dupont2022data}, or hypernetworks \cite{skorokhodov2021adversarial}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06566" target="2203.00137" id="730">
-        <attvalues>
-          <attvalue for="5" value=" For example, CURE \cite{shangguan2022learning}, a state-of-the-art method for video interpolation based on NF, uses an encoder to impose space-time consistency using local feature codes." />
-        </attvalues>
-      </edge>
-      <edge source="2010.10008" target="1902.09212" id="731">
-        <attvalues>
-          <attvalue for="5" value=" Recently, significant progress has been made in this area~\cite{sun2019HRNet}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.10008" target="2005.04490" id="732">
-        <attvalues>
-          <attvalue for="5" value=" However, the pose estimation in complex events~\cite{lin2020human} is still relatively new and a challenging problem." />
-        </attvalues>
-      </edge>
-      <edge source="2010.10008" target="1804.06208" id="733">
-        <attvalues>
-          <attvalue for="5" value=" In the second step, we perform pose estimation on every box by two state-of-the-art single-person pose estimation models~\cite{xiao2018SimpleNet, sun2019HRNet}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2202.13514" id="734">
-        <attvalues>
-          <attvalue for="5" value=" Real-time TbD methods commonly hold a Kalman Filter \cite{kalman} for each tracked object, match the estimated states of the objects with the detections via the Hungarian algorithm \cite{hungarian} and update the states of the tracklets with matched detection\cite{sort, deepsort, bytetrack, ocsort, deepocsort,strongsort,botsort,boostTrack,pia2,imprasso,lgtrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2206.04656" id="735">
-        <attvalues>
-          <attvalue for="5" value=" However, they are criticized for lack of modularity and computational demand of training and deployment \cite{Seidenschwarz_2023_CVPR,moyolo}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1602.00763" id="736">
-        <attvalues>
-          <attvalue for="5" value=" Real-time TbD methods commonly hold a Kalman Filter \cite{kalman} for each tracked object, match the estimated states of the objects with the detections via the Hungarian algorithm \cite{hungarian} and update the states of the tracklets with matched detection\cite{sort, deepsort, bytetrack, ocsort, deepocsort,strongsort,botsort,boostTrack,pia2,imprasso,lgtrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2206.14651" id="737">
-        <attvalues>
-          <attvalue for="5" value=" Real-time TbD methods commonly hold a Kalman Filter \cite{kalman} for each tracked object, match the estimated states of the objects with the detections via the Hungarian algorithm \cite{hungarian} and update the states of the tracklets with matched detection\cite{sort, deepsort, bytetrack, ocsort, deepocsort,strongsort,botsort,boostTrack,pia2,imprasso,lgtrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2309.09765" id="738">
-        <attvalues>
-          <attvalue for="5" value=" Real-time TbD methods commonly hold a Kalman Filter \cite{kalman} for each tracked object, match the estimated states of the objects with the detections via the Hungarian algorithm \cite{hungarian} and update the states of the tracklets with matched detection\cite{sort, deepsort, bytetrack, ocsort, deepocsort,strongsort,botsort,boostTrack,pia2,imprasso,lgtrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1703.07402" id="739">
-        <attvalues>
-          <attvalue for="5" value="The cost matrix of the matching algorithms can be based on the object's position, motion, appearance, or a combination of multiple factors \cite{deepsort, ocsort, deepocsort}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1903.07071" id="740">
-        <attvalues>
-          <attvalue for="5" value=" ReID features \cite{reid-bot, fastreid} are widely used in the matching phase of state-of-the-art (SOTA) methods due to their robustness against occlusions and clutter \cite{deepocsort, deepsort, strongsort, botsort, fairmot, rethink}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2302.11813" id="741">
-        <attvalues>
-          <attvalue for="5" value=" ReID features \cite{reid-bot, fastreid} are widely used in the matching phase of state-of-the-art (SOTA) methods due to their robustness against occlusions and clutter \cite{deepocsort, deepsort, strongsort, botsort, fairmot, rethink}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1603.00831" id="742">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, existing MOT methods prioritize achieving high accuracy on prerecorded datasets such as MOT17 \cite{mot16-review}, MOT20 \cite{mot20}, and DanceTrack \cite{dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2003.09003" id="743">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, existing MOT methods prioritize achieving high accuracy on prerecorded datasets such as MOT17 \cite{mot16-review}, MOT20 \cite{mot20}, and DanceTrack \cite{dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2111.14690" id="744">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, existing MOT methods prioritize achieving high accuracy on prerecorded datasets such as MOT17 \cite{mot16-review}, MOT20 \cite{mot20}, and DanceTrack \cite{dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1909.12605" id="745">
-        <attvalues>
-          <attvalue for="5" value=" The idea of joint detection and embedding (JDE) \cite{jde-review} is one such approach, which enables detection and feature extraction in a shared backbone \cite{jde1, jde-review, rethink, retinatrack,fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1604.01850" id="746">
-        <attvalues>
-          <attvalue for="5" value=" The idea of joint detection and embedding (JDE) \cite{jde-review} is one such approach, which enables detection and feature extraction in a shared backbone \cite{jde1, jde-review, rethink, retinatrack,fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2004.01888" id="747">
-        <attvalues>
-          <attvalue for="5" value=" The idea of joint detection and embedding (JDE) \cite{jde-review} is one such approach, which enables detection and feature extraction in a shared backbone \cite{jde1, jde-review, rethink, retinatrack,fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="1903.05625" id="748">
-        <attvalues>
-          <attvalue for="5" value=" Some other approaches replaces data association by regression of previous detections \cite{tbreg1, tbreg2}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2005.12872" id="749">
-        <attvalues>
-          <attvalue for="5" value=" With the introduction of DETR \cite{detr}, using transformers, which learn spatiotemporal relation of detections and tracklets, for end-to-end object detection and tracking gained popularity \cite{trans1, trans2, trans3, trans4}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06617" target="2310.17170" id="750">
-        <attvalues>
-          <attvalue for="5" value=" However, they are criticized for lack of modularity and computational demand of training and deployment \cite{Seidenschwarz_2023_CVPR,moyolo}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.17296" target="2003.11596" id="751">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, fueled by various paired image exposure dataset~\cite{cai2018learning,afifi2021learning}, several methods~\cite{huang2022ecl,huang2022deep,wang2023decoupling,wang2022local,yao2023generalized} that focus on learning image exposure correction have been proposed." />
-        </attvalues>
-      </edge>
-      <edge source="2402.17296" target="2308.13783" id="752">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, fueled by various paired image exposure dataset~\cite{cai2018learning,afifi2021learning}, several methods~\cite{huang2022ecl,huang2022deep,wang2023decoupling,wang2022local,yao2023generalized} that focus on learning image exposure correction have been proposed." />
-        </attvalues>
-      </edge>
-      <edge source="2402.17296" target="2104.10729" id="753">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods generally do not handle videos of dynamic scenes, as directly applying them in a frame-by-frame manner often results in inter-frame exposure inconsistency with low visual quality~\cite{lv2018mbllen,li2021low}, as shown in the below sub-figures of Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2402.17296" target="1808.04560" id="754">
-        <attvalues>
-          <attvalue for="5" value=" Some methods~\cite{wei2018deep, wang2021seeing, fu2023dancing} focus on enhancing underexposed videos taken from low-light dynamic scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2006.11392" id="755">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2105.08468" id="756">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2203.14291" id="757">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2004.14133" id="758">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2004.07054" id="759">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2106.06485" id="760">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="1807.02011" id="761">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="1903.08536" id="762">
-        <attvalues>
-          <attvalue for="5" value=" This natural phenomenon has intrigued scientists and inspired diverse practical applications, such as detecting medical lesions \cite{fan2020pranet,chiao2011hyperspectral,ji2021progressively,ji2022video,fan2020inf,wu2021jcs,liu2021covid,chen2022pedestrian} and identifying industrial defects \cite{bergmann2018defect1,tabernik2020defect2,tsai2021defect3}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2111.03216" id="763">
-        <attvalues>
-          <attvalue for="5" value=" Recent deep learning based methods \cite{edge_aware_SCRN,errnet,mutualGL,edge_reconstruion,edge_aware_mirrornet} effectively tackle this challenge by introducing an additional branch to integrate extra prior information (such as edge, depth, etc)." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2104.02613" id="764">
-        <attvalues>
-          <attvalue for="5" value=" Recent deep learning based methods \cite{edge_aware_SCRN,errnet,mutualGL,edge_reconstruion,edge_aware_mirrornet} effectively tackle this challenge by introducing an additional branch to integrate extra prior information (such as edge, depth, etc)." />
-        </attvalues>
-      </edge>
-      <edge source="2404.08936" target="2307.03932" id="765">
-        <attvalues>
-          <attvalue for="5" value=" Recent deep learning based methods \cite{edge_aware_SCRN,errnet,mutualGL,edge_reconstruion,edge_aware_mirrornet} effectively tackle this challenge by introducing an additional branch to integrate extra prior information (such as edge, depth, etc)." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1704.07333" id="766">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1808.07962" id="767">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1808.10437" id="768">
-        <attvalues>
-          <attvalue for="5" value=" Instead of engineering a new model, we try to understand the detectability and identifiability of rare interactions with the help of the state-of-the-art HOI detectors~\cite{gao2018ican,chao2018learning,li2019transferable}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1812.05736" id="769">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1904.03181" id="770">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1910.07721" id="771">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="2001.04360" id="772">
-        <attvalues>
-          <attvalue for="5" value=" The task received an increasing amount of attention in recent years~\cite{gkioxari2018detecting,qi2018learning,gao2018ican,xu2019learning,peyre2018detecting,gupta2018no,bansal2019detecting,wang2019deep,chafik2020classifying} thanks to the benchmark dataset HICO-Det~\cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="1702.05448" id="773">
-        <attvalues>
-          <attvalue for="5" value=" Instead of engineering a new model, we try to understand the detectability and identifiability of rare interactions with the help of the state-of-the-art HOI detectors~\cite{gao2018ican,chao2018learning,li2019transferable}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05728" target="2101.10292" id="774">
-        <attvalues>
-          <attvalue for="5" value=" Instead of engineering a new model, we try to understand the detectability and identifiability of rare interactions with the help of the state-of-the-art HOI detectors~\cite{gao2018ican,chao2018learning,li2019transferable}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="1711.00937" id="775">
-        <attvalues>
-          <attvalue for="5" value=" Despite the achievements of VQ-VAE \cite{VQ-VAE} and VQGAN \cite{vqgan}, they face limitations when employed for complex scene generation tasks that demand high resolution, emphasize real-world dynamic scenarios, incorporate abundant contextual semantics, and take into account temporal aspects." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="2309.17080" id="776">
-        <attvalues>
-          <attvalue for="5" value=" Given that the decoder of VQGAN\cite{vqgan} is often not utilized in the second stage in complex scene generation applications\cite{gaia1, worlddreamer}, our pyramid feature learning approach focuses on simplicity without requiring additional parameter learning." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="2304.07193" id="777">
-        <attvalues>
-          <attvalue for="5" value=" To address this, they employ the DINOv2\cite{oquab2023dinov2} model to guide compression." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="2002.08111" id="778">
-        <attvalues>
-          <attvalue for="5" value=" \cite{williams2020hierarchical} aims to reset infrequently used or unused entries by leveraging frequently accessed ones, while CVQ-VAE \cite{zheng2023online} proposes an online clustering algorithm that updates codebook entries using associated encoded features during the training stage." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="2307.15139" id="779">
-        <attvalues>
-          <attvalue for="5" value=" VQGAN\cite{vqgan} and CVQ-VAE\cite{zheng2023online} exhibits an alarming $0." />
-        </attvalues>
-      </edge>
-      <edge source="2409.06105" target="2105.15203" id="780">
-        <attvalues>
-          <attvalue for="5" value=" By leveraging the inference results from the segmentation model\cite{xie2021segformer}, our approach achieves temporospatially consistent semantic learning, as the semantic IDs remain consistent across both time and space." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1611.09224" id="781">
-        <attvalues>
-          <attvalue for="5" value=" Previous methods can be grouped into either online-update trackers\cite{eco, dimp} and Siamese trackers\cite{siamesefc, siamrcnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1606.09549" id="782">
-        <attvalues>
-          <attvalue for="5" value=" Previous methods can be grouped into either online-update trackers\cite{eco, dimp} and Siamese trackers\cite{siamesefc, siamrcnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1706.03762" id="783">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Transformer\cite{attention_is_all} sweeps in computer vision, the dominant tracking methods are Transformer-based trackers\cite{transt,stark,mixformer,ostrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2103.15436" id="784">
-        <attvalues>
-          <attvalue for="5" value=" TransT\cite{transt} proposes transformer-based ECA and CFA modules to replace the long-used correlation calculation." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2103.17154" id="785">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Transformer\cite{attention_is_all} sweeps in computer vision, the dominant tracking methods are Transformer-based trackers\cite{transt,stark,mixformer,ostrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2302.02814" id="786">
-        <attvalues>
-          <attvalue for="5" value=" More recently, some trackers\cite{mixformer, ostrack} introduce pure transformer architecture, and the feature extracting and template-search region interaction is completed in a single backbone, tracking performance is pushed to new records." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2203.11991" id="787">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Transformer\cite{attention_is_all} sweeps in computer vision, the dominant tracking methods are Transformer-based trackers\cite{transt,stark,mixformer,ostrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1709.06750" id="788">
-        <attvalues>
-          <attvalue for="5" value=" Early VOS methods propagate object masks over video frames via motion clues\cite{segflow,tsai2016video} or adopt online learning strategies\cite{caelles2017one,li2018video}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1611.05198" id="789">
-        <attvalues>
-          <attvalue for="5" value=" Early VOS methods propagate object masks over video frames via motion clues\cite{segflow,tsai2016video} or adopt online learning strategies\cite{caelles2017one,li2018video}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1803.04242" id="790">
-        <attvalues>
-          <attvalue for="5" value=" Early VOS methods propagate object masks over video frames via motion clues\cite{segflow,tsai2016video} or adopt online learning strategies\cite{caelles2017one,li2018video}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="1904.00607" id="791">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Space-Temporal Memory (STM) network~\cite{oh2019video, wang2017learning} extracts the spatio-temporal context from a memory bank to handle the appearance changes and occlusions, offering a promising solution for semi-supervised video object segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2106.02638" id="792">
-        <attvalues>
-          <attvalue for="5" value=" Based on AOT\cite{aot}, DeAOT\cite{deaot} decouples the hierarchical propagation of object-agnostic and object-specific embeddings from previous frames to the current frame, further improving the VOS accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2210.09782" id="793">
-        <attvalues>
-          <attvalue for="5" value=" VMOS is an improved variant of DeAOT~\cite{deaot}, we cascade a 1/8 scale gated propagation module (GPM) for perceiving small objects in complex scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2211.05778" id="794">
-        <attvalues>
-          <attvalue for="5" value=" Besides, Intern-T\cite{internimage} is employed as our feature extractor to enhance object discrimination capabilities." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2304.02643" id="795">
-        <attvalues>
-          <attvalue for="5" value=" SAM\cite{sam} is prone to failure when predicting objects with complex structures\cite{sam_hq}, and these difficult cases are common in VOTS chanllenge." />
-        </attvalues>
-      </edge>
-      <edge source="2307.13974" target="2306.01567" id="796">
-        <attvalues>
-          <attvalue for="5" value=" To further improve the quality of tracking masks, a pre-trained HQ-SAM\cite{sam_hq} model is employed to refine the tracking masks." />
-        </attvalues>
-      </edge>
-      <edge source="1910.13136" target="1908.01703" id="797">
-        <attvalues>
-          <attvalue for="5" value=" End-to-end algorithms \cite{ma2019sesf, yan2018unsupervised, zhao2018multi} directly obtain the fusion results, but the results are unfortunately not realistic, as with the transfer domain algorithms." />
-        </attvalues>
-      </edge>
-      <edge source="2108.13055" target="1405.0312" id="798">
-        <attvalues>
-          <attvalue for="5" value=" They have largely improved the performances of long-lasting problems such as handwritten digit recognition \cite{Lecun_1998_MNIST_PIEEE}, face recognition \cite{Hinton_2006_Science}, image categorization \cite{Krizhevsky_2012_AlexNet_NIPS}, etc They are also enabling exploring new boundaries including the studies on image and video captioning \cite{Lin_2014_COCO_ECCV,Song_2019_MS-RNNs_TNNLS,Gao_2019_HLSTMs_PAMI}, body pose estimation \cite{Wei_2016_CPM_CVPR}, and many others." />
-        </attvalues>
-      </edge>
-      <edge source="2108.13055" target="1812.11004" id="799">
-        <attvalues>
-          <attvalue for="5" value=" They have largely improved the performances of long-lasting problems such as handwritten digit recognition \cite{Lecun_1998_MNIST_PIEEE}, face recognition \cite{Hinton_2006_Science}, image categorization \cite{Krizhevsky_2012_AlexNet_NIPS}, etc They are also enabling exploring new boundaries including the studies on image and video captioning \cite{Lin_2014_COCO_ECCV,Song_2019_MS-RNNs_TNNLS,Gao_2019_HLSTMs_PAMI}, body pose estimation \cite{Wei_2016_CPM_CVPR}, and many others." />
-        </attvalues>
-      </edge>
-      <edge source="2108.13055" target="1602.00134" id="800">
-        <attvalues>
-          <attvalue for="5" value=" They have largely improved the performances of long-lasting problems such as handwritten digit recognition \cite{Lecun_1998_MNIST_PIEEE}, face recognition \cite{Hinton_2006_Science}, image categorization \cite{Krizhevsky_2012_AlexNet_NIPS}, etc They are also enabling exploring new boundaries including the studies on image and video captioning \cite{Lin_2014_COCO_ECCV,Song_2019_MS-RNNs_TNNLS,Gao_2019_HLSTMs_PAMI}, body pose estimation \cite{Wei_2016_CPM_CVPR}, and many others." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1902.05978" id="801">
-        <attvalues>
-          <attvalue for="5" value=" To model textures in high fidelity, GANFit \cite{ganfit} harnesses Generative Adversarial Networks (GANs) to train a very powerful generator of facial texture in UV space and constrains the latent parameter by state-of-the-art face recognition model~\cite{deng2019arcface,arcface,deng2021variational,an2022killing,zhu2021webface260m,zhu2022webface260m}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1511.07212" id="802">
-        <attvalues>
-          <attvalue for="5" value="~\cite{zhu20153ddfa} propose 3D Dense Face Alignment (3DDFA) by taking advantage of Projected Normalized Coordinate Code (PNCC)." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1803.07835" id="803">
-        <attvalues>
-          <attvalue for="5" value=" PRN~\cite{feng2018joint} predicts a position map in the UV space of a template mesh." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1903.08527" id="804">
-        <attvalues>
-          <attvalue for="5" value="~\cite{deng2019accurate} train a 3DMM parameter regressor based on photometric reconstruction loss with skin attention masks, a perception loss based on FaceNet~\cite{facenet}, and multi-image consistency losses." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1801.07698" id="805">
-        <attvalues>
-          <attvalue for="5" value=" To model textures in high fidelity, GANFit \cite{ganfit} harnesses Generative Adversarial Networks (GANs) to train a very powerful generator of facial texture in UV space and constrains the latent parameter by state-of-the-art face recognition model~\cite{deng2019arcface,arcface,deng2021variational,an2022killing,zhu2021webface260m,zhu2022webface260m}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="2103.04098" id="806">
-        <attvalues>
-          <attvalue for="5" value=" To model textures in high fidelity, GANFit \cite{ganfit} harnesses Generative Adversarial Networks (GANs) to train a very powerful generator of facial texture in UV space and constrains the latent parameter by state-of-the-art face recognition model~\cite{deng2019arcface,arcface,deng2021variational,an2022killing,zhu2021webface260m,zhu2022webface260m}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="2204.10149" id="807">
-        <attvalues>
-          <attvalue for="5" value=" To model textures in high fidelity, GANFit \cite{ganfit} harnesses Generative Adversarial Networks (GANs) to train a very powerful generator of facial texture in UV space and constrains the latent parameter by state-of-the-art face recognition model~\cite{deng2019arcface,arcface,deng2021variational,an2022killing,zhu2021webface260m,zhu2022webface260m}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1806.06098" id="808">
-        <attvalues>
-          <attvalue for="5" value="~\cite{genova2018unsupervised} propose a 3DMM parameter regression technique that is based on synthetic renderings and Tran et al." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="2204.02776" id="809">
-        <attvalues>
-          <attvalue for="5" value=" However, we avoid explicit face pose estimation as in RetinaFace~\cite{deng2020retinaface} and DenseLandmark~\cite{wood20223d}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1612.04904" id="810">
-        <attvalues>
-          <attvalue for="5" value="~\cite{tran2016regressing} directly regress 3DMM parameters using a CNN trained on fitted 3DMM data." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1804.03786" id="811">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{tran2018nonlinear}, CNN-based shape and texture decoders are trained on unwrapped UV space for non-linear 3D morphable face modelling." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1703.07834" id="812">
-        <attvalues>
-          <attvalue for="5" value="~\cite{jackson2017large} propose a model-free approach that reconstructs a voxel-based representation of the human face." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1803.02188" id="813">
-        <attvalues>
-          <attvalue for="5" value=" DenseReg~\cite{gueler2017densereg} regresses horizontal and vertical tessellation which is obtained by unwrapping the template shape and transferring it to the image domain." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1503.03832" id="814">
-        <attvalues>
-          <attvalue for="5" value="~\cite{deng2019accurate} train a 3DMM parameter regressor based on photometric reconstruction loss with skin attention masks, a perception loss based on FaceNet~\cite{facenet}, and multi-image consistency losses." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="2012.04012" id="815">
-        <attvalues>
-          <attvalue for="5" value=" DECA~\cite{deca} robustly produces a UV displacement map from a low-dimensional latent representation." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="2204.06607" id="816">
-        <attvalues>
-          <attvalue for="5" value=" When the face is close to the camera, the distortion caused by perspective projection can not be ignored \cite{kao2022single,zielonka2022towards}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.07142" target="1902.10744" id="817">
-        <attvalues>
-          <attvalue for="5" value=" Most important, explicit pose parameter regression can result in drift-alignment problem \cite{chaudhuri2019joint}, which will make the 2D visualization of face reconstruction unsatisfying." />
-        </attvalues>
-      </edge>
-      <edge source="2101.10801" target="1605.06211" id="818">
-        <attvalues>
-          <attvalue for="5" value=" Early approaches \cite{long2015fully} attempt to use a two-stream network to extract features from RGB and depth modality respectively and fuse them at the last layer to predict the final segmentation results." />
-        </attvalues>
-      </edge>
-      <edge source="2101.10801" target="1806.01054" id="819">
-        <attvalues>
-          <attvalue for="5" value=" Later, researchers tend to propagate features of the depth branch to the RGB branch in a multi-stage manner, eg, add depth feature to the RGB branch at the end of every stage of the encoder network \cite{hazirbas2016fusenet,jiang2018rednet}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.10801" target="2002.10120" id="820">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, the alignment process is to simultaneously warp the feature maps of both modalities according to the offsets predicted through a convolution layer which is inspired by the optical flow in video processing field and Semantic Flow \cite{li2020semantic}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2103.00020" id="821">
-        <attvalues>
-          <attvalue for="5" value=" Recently, large visual-language models (VLMs), such as CLIP~\cite{clip}, have demonstrated impressive generalization capabilities trained on internet-scale image-language pairs." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2211.15654" id="822">
-        <attvalues>
-          <attvalue for="5" value=" On one hand, there are explicit point-cloud based approaches such as OpenScene~\cite{Peng2022OpenScene}, PLA~\cite{Ding2022PLALO}, CLIP-FO3D~\cite{Zhang2023CLIPFO3DLF} and ConceptFusion~\cite{conceptfusion} which aim to create scene representations that allow querying the scene based on arbitrary open-vocabulary descriptions." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2211.16312" id="823">
-        <attvalues>
-          <attvalue for="5" value=" On one hand, there are explicit point-cloud based approaches such as OpenScene~\cite{Peng2022OpenScene}, PLA~\cite{Ding2022PLALO}, CLIP-FO3D~\cite{Zhang2023CLIPFO3DLF} and ConceptFusion~\cite{conceptfusion} which aim to create scene representations that allow querying the scene based on arbitrary open-vocabulary descriptions." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2303.04748" id="824">
-        <attvalues>
-          <attvalue for="5" value=" On one hand, there are explicit point-cloud based approaches such as OpenScene~\cite{Peng2022OpenScene}, PLA~\cite{Ding2022PLALO}, CLIP-FO3D~\cite{Zhang2023CLIPFO3DLF} and ConceptFusion~\cite{conceptfusion} which aim to create scene representations that allow querying the scene based on arbitrary open-vocabulary descriptions." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2302.07241" id="825">
-        <attvalues>
-          <attvalue for="5" value=" On one hand, there are explicit point-cloud based approaches such as OpenScene~\cite{Peng2022OpenScene}, PLA~\cite{Ding2022PLALO}, CLIP-FO3D~\cite{Zhang2023CLIPFO3DLF} and ConceptFusion~\cite{conceptfusion} which aim to create scene representations that allow querying the scene based on arbitrary open-vocabulary descriptions." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2303.09553" id="826">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, there are works such as LERF~\cite{Kerr2023LERFLE}, OpenNeRF~\cite{opennerf} and DFF~\cite{kobayashi2022distilledfeaturefields} that embed open-vocabulary features on an implicit scene representation." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2205.15585" id="827">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, there are works such as LERF~\cite{Kerr2023LERFLE}, OpenNeRF~\cite{opennerf} and DFF~\cite{kobayashi2022distilledfeaturefields} that embed open-vocabulary features on an implicit scene representation." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2306.13631" id="828">
-        <attvalues>
-          <attvalue for="5" value=" Methods focusing on 3D instance segmentation include OpenMask3D~\cite{takmaz2023openmask3d}, OpenIns3D~\cite{Huang2023OpenIns3DSA} and Open3DIS~\cite{Nguyen2023Open3DISO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2309.00616" id="829">
-        <attvalues>
-          <attvalue for="5" value=" Methods focusing on 3D instance segmentation include OpenMask3D~\cite{takmaz2023openmask3d}, OpenIns3D~\cite{Huang2023OpenIns3DSA} and Open3DIS~\cite{Nguyen2023Open3DISO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2312.10671" id="830">
-        <attvalues>
-          <attvalue for="5" value=" Methods focusing on 3D instance segmentation include OpenMask3D~\cite{takmaz2023openmask3d}, OpenIns3D~\cite{Huang2023OpenIns3DSA} and Open3DIS~\cite{Nguyen2023Open3DISO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2304.14880" id="831">
-        <attvalues>
-          <attvalue for="5" value=" There has also been a growing interest in creating a 3D scene graph representation \cite{sarkar2023sgaligner,Armeni_2019_ICCV} that allows for open-vocabulary querying such as ConceptGraphs~\cite{Gu2023ConceptGraphsO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="1910.02527" id="832">
-        <attvalues>
-          <attvalue for="5" value=" There has also been a growing interest in creating a 3D scene graph representation \cite{sarkar2023sgaligner,Armeni_2019_ICCV} that allows for open-vocabulary querying such as ConceptGraphs~\cite{Gu2023ConceptGraphsO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.15321" target="2309.16650" id="833">
-        <attvalues>
-          <attvalue for="5" value=" There has also been a growing interest in creating a 3D scene graph representation \cite{sarkar2023sgaligner,Armeni_2019_ICCV} that allows for open-vocabulary querying such as ConceptGraphs~\cite{Gu2023ConceptGraphsO3}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="1512.03385" id="834">
-        <attvalues>
-          <attvalue for="5" value=" Deep convolutional neural networks~\cite{Alexnet,Resnet} have revolutionized computer vision in the last decade, making it possible to automatically learn representations from a large number of images." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="1909.09157" id="835">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="2007.11498" id="836">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="1909.02729" id="837">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="2003.11539" id="838">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="2103.01315" id="839">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="2003.04390" id="840">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="2110.13953" id="841">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies of few-shot image classification have highlighted the importance of the quality of learned image representations~\cite{mamlrepresentation,crosstransformer,baseline,allyouneed,IER}, and also showed that representations learned by neural networks do not generalize well to novel few-shot classification tasks when there is task distribution shift~\cite{metabaseline,crosstransformer,sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="1412.6856" id="842">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, in the layer after global pooling, different channels in the learned feature seek for different patterns~(as verified in~\cite{emergingobject,dissection}) during training, and the channels are weighted (in a biased way) based on their importance to the training task." />
-        </attvalues>
-      </edge>
-      <edge source="2206.08126" target="1704.05796" id="843">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, in the layer after global pooling, different channels in the learned feature seek for different patterns~(as verified in~\cite{emergingobject,dissection}) during training, and the channels are weighted (in a biased way) based on their importance to the training task." />
-        </attvalues>
-      </edge>
-      <edge source="2311.16589" target="1710.03077" id="844">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies~\cite{dg1,dg2,dg3,dg4,dg5} have attempted to address this performance degradation." />
-        </attvalues>
-      </edge>
-      <edge source="2311.16589" target="1301.2115" id="845">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies~\cite{dg1,dg2,dg3,dg4,dg5} have attempted to address this performance degradation." />
-        </attvalues>
-      </edge>
-      <edge source="2311.16589" target="1910.13580" id="846">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies~\cite{dg1,dg2,dg3,dg4,dg5} have attempted to address this performance degradation." />
-        </attvalues>
-      </edge>
-      <edge source="2311.16589" target="1912.02781" id="847">
-        <attvalues>
-          <attvalue for="5" value=" To achieve this diversity, previous works~\cite{hendrycks2019augmix, volpi2018generalizing, qiao2020learning, ssdg1, ssdg2, ssdg3} actively adopt augmentation techniques such as Augmix~\cite{hendrycks2019augmix} or adversarial methods~\cite{volpi2018generalizing, qiao2020learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.16589" target="1805.12018" id="848">
-        <attvalues>
-          <attvalue for="5" value=" To achieve this diversity, previous works~\cite{hendrycks2019augmix, volpi2018generalizing, qiao2020learning, ssdg1, ssdg2, ssdg3} actively adopt augmentation techniques such as Augmix~\cite{hendrycks2019augmix} or adversarial methods~\cite{volpi2018generalizing, qiao2020learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1506.01497" id="849">
-        <attvalues>
-          <attvalue for="5" value=" First, the detection framework (eg Faster R-CNN \cite{ren2015faster} or YOLO \cite{redmon2016you}) and its backbone (eg ResNet-50 or 101), then the different loss functions (eg L1, IoU, Focal Loss) to train each part of the network and finally all the hyperparameters that are tied to these methods (eg from the learning rate to the class splits for evaluation)." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1506.02640" id="850">
-        <attvalues>
-          <attvalue for="5" value=" First, the detection framework (eg Faster R-CNN \cite{ren2015faster} or YOLO \cite{redmon2016you}) and its backbone (eg ResNet-50 or 101), then the different loss functions (eg L1, IoU, Focal Loss) to train each part of the network and finally all the hyperparameters that are tied to these methods (eg from the learning rate to the class splits for evaluation)." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1612.00796" id="851">
-        <attvalues>
-          <attvalue for="5" value=" Of course, plenty of techniques have been introduced so that the fine-tuning does not disrupt base learning (see eg \cite{kirkpatrick2017overcoming})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1812.01866" id="852">
-        <attvalues>
-          <attvalue for="5" value=" Although the attention mechanisms, deployed in most FSOD methods, might differ from the original one proposed in \cite{kang2019few}, the main principle remains the same." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="2006.07826" id="853">
-        <attvalues>
-          <attvalue for="5" value=" Plenty of methods based on the same idea have been introduced (see eg \cite{deng2020few,fan2020few,wallach2019one,li2020one})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1908.01998" id="854">
-        <attvalues>
-          <attvalue for="5" value=" Plenty of methods based on the same idea have been introduced (see eg \cite{deng2020few,fan2020few,wallach2019one,li2020one})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="1911.12529" id="855">
-        <attvalues>
-          <attvalue for="5" value=" Plenty of methods based on the same idea have been introduced (see eg \cite{deng2020few,fan2020few,wallach2019one,li2020one})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="2005.03819" id="856">
-        <attvalues>
-          <attvalue for="5" value=" Plenty of methods based on the same idea have been introduced (see eg \cite{deng2020few,fan2020few,wallach2019one,li2020one})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="2109.13027" id="857">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, there exist some papers that use metric learning to solve FSOD (see eg \cite{karlinsky2019repmet,jeune2021experience,sun2021fsce})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="2103.05950" id="858">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, there exist some papers that use metric learning to solve FSOD (see eg \cite{karlinsky2019repmet,jeune2021experience,sun2021fsce})." />
-        </attvalues>
-      </edge>
-      <edge source="2201.02052" target="2110.14711" id="859">
-        <attvalues>
-          <attvalue for="5" value=" To our knowledge, there exist two papers that review FSOD methods \cite{huang2021survey, jiaxu2021comparative}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="1405.0312" id="860">
-        <attvalues>
-          <attvalue for="5" value=" T{raditional} object detectors rely on large supervised object detection datasets such as PASCAL VOC~\cite{pascal} and MS COCO~\cite{coco}, which have over hundreds and thousands of annotated examples per object category." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="1812.01866" id="861">
-        <attvalues>
-          <attvalue for="5" value=" Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr} However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2003.06957" id="862">
-        <attvalues>
-          <attvalue for="5" value=" Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr} However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2007.09384" id="863">
-        <attvalues>
-          <attvalue for="5" value=" Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr} However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2106.04550" id="864">
-        <attvalues>
-          <attvalue for="5" value=" In particular, the current state-of-the-art for FSOD on MS COCO is a method which does self-supervised pretraining of both the backbone and the object detector~\cite{detreg}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2106.02637" id="865">
-        <attvalues>
-          <attvalue for="5" value=" These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2102.08318" id="866">
-        <attvalues>
-          <attvalue for="5" value=" These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2103.12902" id="867">
-        <attvalues>
-          <attvalue for="5" value=" These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2106.09785" id="868">
-        <attvalues>
-          <attvalue for="5" value=" Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr} However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2011.05499" id="869">
-        <attvalues>
-          <attvalue for="5" value=" Until recently, the standard approach in few-shot object detection was to pretrain a backbone for ImageNet classification, then train an object detector on top of this backbone on the base classes, and finally finetune on the novel classes~\cite{metayolo,metarcnn,tfa,mpsr,metadetr} However, because of the tremendous progress in learning self-supervised representations, several (few-shot) detection methods now initialize their backbone from representations pretrained with unsupervised pretext tasks on ImageNet and MS COCO~\cite{detreg,soco,insloc,resim,esvit,pinheiro2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2002.05709" id="870">
-        <attvalues>
-          <attvalue for="5" value=" The problem with typical self-supervised pretrained methods such as SimCLR~\cite{simclr} or MoCo~\cite{moco} is that they are geared towards classification, and often engineered to maximize Top-1 performance on ImageNet~\cite{densecl}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="1911.05722" id="871">
-        <attvalues>
-          <attvalue for="5" value=" The problem with typical self-supervised pretrained methods such as SimCLR~\cite{simclr} or MoCo~\cite{moco} is that they are geared towards classification, and often engineered to maximize Top-1 performance on ImageNet~\cite{densecl}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2011.09157" id="872">
-        <attvalues>
-          <attvalue for="5" value=" These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2110.14711" target="2011.09094" id="873">
-        <attvalues>
-          <attvalue for="5" value=" These methods~\cite{updetr,densecl,insloc,resim,soco} not only attempt to remedy the shortcomings of classification-geared representations, but also pretrain more components in addition to the feature extractor, such as the region proposal network (RPN) and detection head, in the case of Faster R-CNN based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2401.00343" target="1908.09464" id="874">
-        <attvalues>
-          <attvalue for="5" value=" The reconstruction of human body pose and shape (HPS) has gained attention from industries like fashion, healthcare, special effects, surveillance, computer animation, and virtual and augmented reality~\cite{Liang01,hu2018personalized,ghadi2021syntactic}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.00343" target="1908.05794" id="875">
-        <attvalues>
-          <attvalue for="5" value=" Common issues affecting HPS reconstruction results, such as self-occlusion, low-contrast lighting or poor depth perception are often due to suboptimal camera poses~\cite{Puscas01,kocabas2021spec,pang2022benchmarking}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.00343" target="2110.00620" id="876">
-        <attvalues>
-          <attvalue for="5" value=" Common issues affecting HPS reconstruction results, such as self-occlusion, low-contrast lighting or poor depth perception are often due to suboptimal camera poses~\cite{Puscas01,kocabas2021spec,pang2022benchmarking}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.00343" target="2209.10529" id="877">
-        <attvalues>
-          <attvalue for="5" value=" Common issues affecting HPS reconstruction results, such as self-occlusion, low-contrast lighting or poor depth perception are often due to suboptimal camera poses~\cite{Puscas01,kocabas2021spec,pang2022benchmarking}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1703.03400" id="878">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1810.09502" id="879">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1703.05175" id="880">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1711.06025" id="881">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1711.04340" id="882">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1806.04734" id="883">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1903.10132" id="884">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="2401.15636" id="885">
-        <attvalues>
-          <attvalue for="5" value=" Generally, these methods can be divided into three groups: optimization-based methods \cite{finn2017model,antoniou2018train,jiang2022continual}, metric-based methods \cite{snell2017prototypical,sung2018learning,jiang2022lie}, and data augmentation-based methods \cite{antoniou2017data,schwartz2018delta,xian2019f,huang2023diffusion,he2024freestyle}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="1903.12290" id="886">
-        <attvalues>
-          <attvalue for="5" value=" In previous works, \cite{li2019revisiting} proposed DN4, which directly utilizes all query descriptors." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="2103.08160" id="887">
-        <attvalues>
-          <attvalue for="5" value=" Based on DN4, \cite{liu2022dmn4} introduced DMN4, which believes that not all query descriptors are task-relevant and contain significant background noise." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05953" target="2312.05449" id="888">
-        <attvalues>
-          <attvalue for="5" value=" \cite{qiao2024talds} introduced TALDS-Net, recognizing background noise in query descriptors." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1511.04587" id="889">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1707.02921" id="890">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1609.04802" id="891">
-        <attvalues>
-          <attvalue for="5" value=" Following them, two primary directions to improve GAN-based IR arise from enhancing the generator (\ieno, the restoration network)~\cite{SRGAN,zhang2019ranksrgan-gene1,wang2018esrgan,wang2021realesrgan,zhang2021designing-gene2} and discriminator~\cite{schonfeld2020u-pixel-discri,isola2017image-patch-wise-discri,simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1809.00219" id="892">
-        <attvalues>
-          <attvalue for="5" value=" In particular, ESRGAN~\cite{wang2018esrgan} introduces the powerful RRDB~\cite{wang2018esrgan} as a generator for GAN-based SR tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2108.11084" id="893">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2208.11247" id="894">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1612.02177" id="895">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1711.07064" id="896">
-        <attvalues>
-          <attvalue for="5" value=" For instance, SRGAN~\cite{SRGAN} and DeblurGAN~\cite{kupyn2018deblurgan-old2} utilize a combination of pixel-wise loss and adversarial loss to achieve perception-oriented SR network and deblurring network, respectively." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2204.07820" id="897">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2202.09652" id="898">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2111.09881" id="899">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2204.04627" id="900">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1608.03981" id="901">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1508.02848" id="902">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1907.08448" id="903">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2202.14009" id="904">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2209.12394" id="905">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2304.01598" id="906">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1604.07379" id="907">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1801.07892" id="908">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2304.06790" id="909">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2203.15270" id="910">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1504.06993" id="911">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1605.00366" id="912">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2004.09320" id="913">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2109.14573" id="914">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2207.07335" id="915">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2305.02760" id="916">
-        <attvalues>
-          <attvalue for="5" value=" The popular IR tasks contain image super-resolution (SR)~\cite{SR1_SRCNN-cnn,SR2——VDSR-cnn,SR3_EDSR-cnn,SR4_SAN,SRGAN,wang2018esrgan,sr_transformer1,sr2_transformer2,sr2_transformer3,sr3_swinfir-transformer}, deblurring~\cite{nah2017deep-deblur-old1-cnn,kupyn2018deblurgan-old2,nimisha2017blur-old3-cnn,zhao2022fcl-deblur,kim2023mssnet-deblur-cnn,zamir2022restormer_deblur-transformer,tsai2022stripformer_deblur-transformer}, denoising~\cite{dabov2007image-traditional1,zhang2017beyond-denoise2-cnn,chen2016trainable-denoise3,valsesia2020deep-denoise4-cnn,fan2022sunet-denoise-new1-transformer,liu2022dnt-denoise-new2-transformer,tian2023multi-denoise-new3,zhang2023mm-new4}, inpainting~\cite{xie2012image-inpainting-old1,pathak2016context-inpainting-old2,yu2018generative-inpainting-new1,inpainting-new2,yu2023inpaint-new3,li2022mat-inpainting-new4} and compression artifacts removing~\cite{dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,jiang2021towards-jpeg-old4-cnn,jiang2022learning-jpeg-new1-transformer,wang2022jpeg-new2,jiang2023multi-jpeg-new3}, etc The visual illustration of some IR tasks are shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1701.05957" id="917">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1711.10098" id="918">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1802.07412" id="919">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2103.14030" id="920">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2111.09883" id="921">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2106.03106" id="922">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1706.03762" id="923">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, numerous IR works collected a series of datasets tailored for various IR tasks, \egno, DIV2K~\cite{DIV2K}, Set5~\cite{set5}, and Set14~\cite{Set14} for SR, Rain800~\cite{Rain800}, Rain200~\cite{rain100}, Raindrop~\cite{rain-drop} and DID-MDN~\cite{zhang2018density-DID-MDN} for draining, REDS~\cite{Nah_2019_CVPR_Workshops_REDS}, and Gopro~\cite{Gopro} for motion deblurring, etc Leveraging these datasets, the majority of recent works~\cite{SR3_EDSR-cnn,SR1_SRCNN-cnn,SR2——VDSR-cnn,nimisha2017blur-old3-cnn,nah2017deep-deblur-old1-cnn,valsesia2020deep-denoise4-cnn,dong2015compression-jpeg-old1-cnn,svoboda2016compression-jpeg-old2-cnn,ehrlich2020quantization-jpeg-old3-cnn,zhang2017beyond-denoise2-cnn,sr_transformer1,sr2_transformer2,sr2_transformer3,swinIR-transformer,sr3_swinfir-transformer,swinv2-transformer,wang2022uformer-transformer,liu2022dnt-denoise-new2-transformer,fan2022sunet-denoise-new1-transformer,zamir2022restormer_deblur-transformer} focused on improving the representation capability of IR networks for complicated degradation through well-designed backbones based on Convolutional neural networks (CNNs)~\cite{krizhevsky2017imagenet-cnn1} or Transformer~\cite{vaswani2017attention-transformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1312.6114" id="924">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1601.06759" id="925">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1606.05328" id="926">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1701.05517" id="927">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1505.05770" id="928">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1705.07057" id="929">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1710.07035" id="930">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1710.10196" id="931">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1609.03126" id="932">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1602.02644" id="933">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1603.08155" id="934">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1511.05666" id="935">
-        <attvalues>
-          <attvalue for="5" value=" Thanks to the development of generative models~\cite{kingma2013auto-VAE,van2016pixel-autoregressive1,van2016conditional-autoregressive2,salimans2017pixelcnn++-autoregressive3,rezende2015variational-flow,papamakarios2017masked-flow2,GAN,karras2017progressive-GAN1,zhao2016energy-GAN2}, especially the generative adversarial network (GAN)~\cite{GAN}, some pioneering IR studies~\cite{dosovitskiy2016generating-loss1,yu2016ultra-loss2,johnson2016perceptual-loss3,bruna2015super-loss4,SRGAN,wang2018esrgan} pinpoint that previous pixel-wise losses, \egno, MSE loss, and L1 loss are susceptible to the blurring textures, and introduce the adversarial loss from GAN to the optimization of IR network, thereby enhancing its texture generation capability." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1908.06382" id="936">
-        <attvalues>
-          <attvalue for="5" value=" Following them, two primary directions to improve GAN-based IR arise from enhancing the generator (\ieno, the restoration network)~\cite{SRGAN,zhang2019ranksrgan-gene1,wang2018esrgan,wang2021realesrgan,zhang2021designing-gene2} and discriminator~\cite{schonfeld2020u-pixel-discri,isola2017image-patch-wise-discri,simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2107.10833" id="937">
-        <attvalues>
-          <attvalue for="5" value=" Following them, two primary directions to improve GAN-based IR arise from enhancing the generator (\ieno, the restoration network)~\cite{SRGAN,zhang2019ranksrgan-gene1,wang2018esrgan,wang2021realesrgan,zhang2021designing-gene2} and discriminator~\cite{schonfeld2020u-pixel-discri,isola2017image-patch-wise-discri,simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2103.14006" id="938">
-        <attvalues>
-          <attvalue for="5" value=" Following them, two primary directions to improve GAN-based IR arise from enhancing the generator (\ieno, the restoration network)~\cite{SRGAN,zhang2019ranksrgan-gene1,wang2018esrgan,wang2021realesrgan,zhang2021designing-gene2} and discriminator~\cite{schonfeld2020u-pixel-discri,isola2017image-patch-wise-discri,simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2002.12655" id="939">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1611.07004" id="940">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1409.1556" id="941">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1912.04958" id="942">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2203.04962" id="943">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2205.03524" id="944">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1804.02900" id="945">
-        <attvalues>
-          <attvalue for="5" value=" Three popular discriminators, including pixel-wise discriminator (U-Net shape)~\cite{schonfeld2020u-pixel-discri}, patch-wise discriminator~\cite{isola2017image-patch-wise-discri,luo2022learning-patch-wise-2,xu2022dual-patch-wise-3,wang2018fully-patch-wise-4}, and image-wise discriminator~\cite{simonyan2014very-image-wise-discri,karras2020analyzing-image-wise-discri2} (\ieno, VGG-like architecture) are designed to focus on the subjective quality at different levels of granularity, (\ieno, from local to global)." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1503.03585" id="946">
-        <attvalues>
-          <attvalue for="5" value=" The prototype of the diffusion model can be traced back to the work~\cite{sohl2015deep-dpm}, and has been developed by DDPM~\cite{DDPM}, NCSN~\cite{NCSN} and SDE~\cite{SDE}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2006.11239" id="947">
-        <attvalues>
-          <attvalue for="5" value="~\ref{sec:background}, highlighting three foundational modeling methods, \ieno, NCSN~\cite{NCSN}, DDPM~\cite{DDPM}, and SDE~\cite{SDE}, along with the further improvements on diffusion models from the perspectives of optimization strategy, sampling efficiency, model architecture, and conditional strategies." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="1907.05600" id="948">
-        <attvalues>
-          <attvalue for="5" value="~\ref{sec:background}, highlighting three foundational modeling methods, \ieno, NCSN~\cite{NCSN}, DDPM~\cite{DDPM}, and SDE~\cite{SDE}, along with the further improvements on diffusion models from the perspectives of optimization strategy, sampling efficiency, model architecture, and conditional strategies." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2011.13456" id="949">
-        <attvalues>
-          <attvalue for="5" value="~\ref{sec:background}, highlighting three foundational modeling methods, \ieno, NCSN~\cite{NCSN}, DDPM~\cite{DDPM}, and SDE~\cite{SDE}, along with the further improvements on diffusion models from the perspectives of optimization strategy, sampling efficiency, model architecture, and conditional strategies." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2010.02502" id="950">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2303.17076" id="951">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2205.03859" id="952">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2105.06337" id="953">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2206.02246" id="954">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2209.14792" id="955">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2211.10440" id="956">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2302.05543" id="957">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2211.08332" id="958">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2111.14822" id="959">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2302.08453" id="960">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2305.05768" id="961">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2304.08818" id="962">
-        <attvalues>
-          <attvalue for="5" value=" Compared to GANs, the diffusion model yields high-fidelity and diverse generation results, thereby successfully replacing the GANs in a series of fields, such as visual generation~\cite{NCSN,DDIM,zhang2023diffcollage-app10,SDE,DDPM} and conditional visual generation~\cite{singh2022conditioning-noisecondition,popov2021grad-app1,levkovitch2022zero-app2,singer2022make-app4,lin2023magic3d-app5,zhang2023adding-app6,xu2022versatile-app7,gu2022vector-vqdiffusion-app8,mou2023t2i-app9,zhang2023diffcollage-app10,babnik2023diffiqa-app11,blattmann2023align-app12}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2112.10752" id="963">
-        <attvalues>
-          <attvalue for="5" value=" With the advancement of the vision-language model, diffusion model has been extended to the cross-modality generation, such as StableDiffusion~\cite{LDM}, and DALLE-2~\cite{ramesh2022hierarchical-dalle2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2204.06125" id="964">
-        <attvalues>
-          <attvalue for="5" value=" With the advancement of the vision-language model, diffusion model has been extended to the cross-modality generation, such as StableDiffusion~\cite{LDM}, and DALLE-2~\cite{ramesh2022hierarchical-dalle2}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2104.07636" id="965">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2104.14951" id="966">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2302.12831" id="967">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2209.12064" id="968">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2207.14626" id="969">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2112.02475" id="970">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2304.08291" id="971">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2305.10028" id="972">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2303.06682" id="973">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2105.14951" id="974">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2209.11888" id="975">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2201.11793" id="976">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2212.00490" id="977">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2209.14687" id="978">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2111.08005" id="979">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2304.11105" id="980">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2212.03221" id="981">
-        <attvalues>
-          <attvalue for="5" value=" According to the training strategy, these works can be roughly divided into two categories: 1) the first category~\cite{SR3,li2022srdiff,niu2023cdpmsr,sde-sr,weather-diff,deblur-DPM,luo2023refusion,zhou2023pyramid-pydiff,chan2023sud,miao2023dds2m} is dedicated to optimizing the diffusion model for IR from scratch via supervised learning, and 2) the second one (\ieno, zero-shot one)~\cite{kawar2021snips,jpegddrm,DDRM,ddnm,dps,SIM-SGM,liu2023improved,abu2022adir} strives to exploit the generative priors in the pre-trained diffusion models for IR." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2302.07864" id="982">
-        <attvalues>
-          <attvalue for="5" value=" To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2301.12686" id="983">
-        <attvalues>
-          <attvalue for="5" value=" To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2211.10656" id="984">
-        <attvalues>
-          <attvalue for="5" value=" To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2305.07015" id="985">
-        <attvalues>
-          <attvalue for="5" value=" To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning." />
-        </attvalues>
-      </edge>
-      <edge source="2308.09388" target="2303.06885" id="986">
-        <attvalues>
-          <attvalue for="5" value=" To further tackle the above issue, some researches~\cite{SR3+,murata2023gibbsddrm,blinddps,wang2023exploiting-stable-sr,wei2023raindiffusion,wang2023dr2} have extended the diffusion model to handle blind/real-world image restoration by incorporating real-world distortion simulation, kernel estimation, domain translation, and distortion invariant learning." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="1504.08083" id="987">
-        <attvalues>
-          <attvalue for="5" value=" On the whole, all the detectors that use deep CNN can be divided into two categories: two-stage approaches \cite{girshick2015fast, ren2015faster, lin2017feature,dai2016r, he2017mask, cai2018cascade, Cao_2020_CVPR, pang2019libra,li2019scale} and one-stage approaches \cite{liu2016ssd, fu2017dssd, redmon2016you, lin2017focal, zhang2018single, zhao2019m2det, redmon2016you,wang2020single, wu2019iou, Zhu_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="1605.06409" id="988">
-        <attvalues>
-          <attvalue for="5" value=" On the whole, all the detectors that use deep CNN can be divided into two categories: two-stage approaches \cite{girshick2015fast, ren2015faster, lin2017feature,dai2016r, he2017mask, cai2018cascade, Cao_2020_CVPR, pang2019libra,li2019scale} and one-stage approaches \cite{liu2016ssd, fu2017dssd, redmon2016you, lin2017focal, zhang2018single, zhao2019m2det, redmon2016you,wang2020single, wu2019iou, Zhu_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="1901.01892" id="989">
-        <attvalues>
-          <attvalue for="5" value=" On the whole, all the detectors that use deep CNN can be divided into two categories: two-stage approaches \cite{girshick2015fast, ren2015faster, lin2017feature,dai2016r, he2017mask, cai2018cascade, Cao_2020_CVPR, pang2019libra,li2019scale} and one-stage approaches \cite{liu2016ssd, fu2017dssd, redmon2016you, lin2017focal, zhang2018single, zhao2019m2det, redmon2016you,wang2020single, wu2019iou, Zhu_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="1512.02325" id="990">
-        <attvalues>
-          <attvalue for="5" value=" Most of the anchor based detectors, such as SSD \cite{liu2016ssd}, FPN \cite{lin2017feature}, handle this problem by building the feature pyramid and directly setting different scales of anchors on each feature map cell in multiple feature layers, which makes each level feature map perform its duty." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="2008.03511" id="991">
-        <attvalues>
-          <attvalue for="5" value=" On the whole, all the detectors that use deep CNN can be divided into two categories: two-stage approaches \cite{girshick2015fast, ren2015faster, lin2017feature,dai2016r, he2017mask, cai2018cascade, Cao_2020_CVPR, pang2019libra,li2019scale} and one-stage approaches \cite{liu2016ssd, fu2017dssd, redmon2016you, lin2017focal, zhang2018single, zhao2019m2det, redmon2016you,wang2020single, wu2019iou, Zhu_2019_CVPR}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.11021" target="1612.03144" id="992">
-        <attvalues>
-          <attvalue for="5" value=" Most of the anchor based detectors, such as SSD \cite{liu2016ssd}, FPN \cite{lin2017feature}, handle this problem by building the feature pyramid and directly setting different scales of anchors on each feature map cell in multiple feature layers, which makes each level feature map perform its duty." />
-        </attvalues>
-      </edge>
-      <edge source="2102.10202" target="1702.06451" id="993">
-        <attvalues>
-          <attvalue for="5" value=" Camera calibration models and estimates a camera's intrinsic and extrinsic parameters, and is an essential first step for many robotic and computer vision applications~\cite{liu2017calibration,sochor2017traffic,wang2017calibration, ren2013techniques}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.10202" target="1907.04096" id="994">
-        <attvalues>
-          <attvalue for="5" value="~\cite{rojtberg2018efficient} proposed pose selection for interactive calibration which depends on a good pose initialization." />
-        </attvalues>
-      </edge>
-      <edge source="2406.07937" target="2209.12435" id="995">
-        <attvalues>
-          <attvalue for="5" value=" proposed the STD \cite{yuan2023std} algorithm, which directly extracts planar boundary feature points from 3D point clouds to construct feature triangles." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13115" target="1506.02640" id="996">
-        <attvalues>
-          <attvalue for="5" value=" Recent one-stage object detectors \cite{yolov1, ssd, yolov3, efficientdet, yolov4} achieve good performance on datasets such as MS COCO \cite{coco} and PASCAL VOC \cite{pascalvoc}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13115" target="1405.0312" id="997">
-        <attvalues>
-          <attvalue for="5" value=" Recent one-stage object detectors \cite{yolov1, ssd, yolov3, efficientdet, yolov4} achieve good performance on datasets such as MS COCO \cite{coco} and PASCAL VOC \cite{pascalvoc}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13115" target="1704.04224" id="998">
-        <attvalues>
-          <attvalue for="5" value=" Many methods \cite{smn, relationnetwork, iterativevr, detr, vityolo} have been proposed to mimic human reasoning ability in object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2308.13004" target="1701.01081" id="999">
-        <attvalues>
-          <attvalue for="5" value="~\cite{chao2018salgan360} employed cubemap projection and fine-tuned the 2D image saliency model SalGAN~\cite{Pan_2017_SalGAN} on each cube face." />
-        </attvalues>
-      </edge>
-      <edge source="2308.13004" target="2011.10600" id="1000">
-        <attvalues>
-          <attvalue for="5" value="~\cite{dahou2021atsal} proposed a two-stream architecture to compute global and local saliency in omnidirectional videos." />
-        </attvalues>
-      </edge>
-      <edge source="2308.13004" target="1912.09390" id="1001">
-        <attvalues>
-          <attvalue for="5" value="~\cite{eder2020tangent} proposed tangent image representations, which use gnomonic projection to map a spherical image into multiple overlapping patches, where each patch is tangent to the faces of an icosahedron." />
-        </attvalues>
-      </edge>
-      <edge source="2308.13004" target="2102.05095" id="1002">
-        <attvalues>
-          <attvalue for="5" value=" This is the first work that employs tangent images for omnidirectional saliency prediction, which is also capable of modelling the temporal dimension, motivated by the recent video transformer architectures, eg, ~\cite{gberta_2021_ICML}." />
-        </attvalues>
-      </edge>
-      <edge source="2308.13004" target="1710.10755" id="1003">
-        <attvalues>
-          <attvalue for="5" value=" Our extensive experiments demonstrate the effectiveness of our proposed SalViT360~model against the state-of-the-art on VR-EyeTracking~\cite{xu2018gaze}, PVS-HMEM~\cite{xu2018predicting}, and 360-AV-HM~\cite{9301766} datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14745" target="2203.16220" id="1004">
-        <attvalues>
-          <attvalue for="5" value=" Although some works ~\cite{TarDAL, UMF-CMGR} pointed out that exploiting structural information could help improve the performance of IVIF methods, these methods encounter a challenge regarding structure inconsistency between the fused and source images." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14745" target="2205.11876" id="1005">
-        <attvalues>
-          <attvalue for="5" value="~\ref{figure22222}, the tree structure is clearly showed in the visible image, but absent in the fused image of UMF~\cite{UMF-CMGR}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.11876" target="1612.01925" id="1006">
-        <attvalues>
-          <attvalue for="5" value=" More often than not, existing widely-used image alignment methods~\cite{flownet,voxelmorph} perform pixel-wise and feature-level alignment by explicitly estimating the deformation field between the distorted image and its reference." />
-        </attvalues>
-      </edge>
-      <edge source="2205.11876" target="1802.02604" id="1007">
-        <attvalues>
-          <attvalue for="5" value=" More often than not, existing widely-used image alignment methods~\cite{flownet,voxelmorph} perform pixel-wise and feature-level alignment by explicitly estimating the deformation field between the distorted image and its reference." />
-        </attvalues>
-      </edge>
-      <edge source="2205.11876" target="2004.14557" id="1008">
-        <attvalues>
-          <attvalue for="5" value=" These obstacles gave rise to the development of cross-modality medical image translation~\cite{medireg,LiuLZFL20,Liu_pami_21} and cross-modality Re-ID tasks~\cite{AlignGAN,D2RL}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.11876" target="1611.07004" id="1009">
-        <attvalues>
-          <attvalue for="5" value=" The former leverages the cGAN~\cite{pix2pix} to perform NeuroImage-to-NeuroImage translation, while the latter learns the cross-modality match in pedestrian images of a person across disjoint camera views by an RGB-to-IR (infrared) translation." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1507.06550" id="1010">
-        <attvalues>
-          <attvalue for="5" value=" With the development of Convolutional Neural Networks (CNNs), most of recent pose estimation frameworks are based on CNNs\cite{huang2017coarse,carreira2016human,chu2017multi,lifshitz2016human,belagiannis2017recurrent,ramakrishna2014pose} and achieve superior performance." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1702.07432" id="1011">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1603.08212" id="1012">
-        <attvalues>
-          <attvalue for="5" value=" With the development of Convolutional Neural Networks (CNNs), most of recent pose estimation frameworks are based on CNNs\cite{huang2017coarse,carreira2016human,chu2017multi,lifshitz2016human,belagiannis2017recurrent,ramakrishna2014pose} and achieve superior performance." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1605.02914" id="1013">
-        <attvalues>
-          <attvalue for="5" value=" With the development of Convolutional Neural Networks (CNNs), most of recent pose estimation frameworks are based on CNNs\cite{huang2017coarse,carreira2016human,chu2017multi,lifshitz2016human,belagiannis2017recurrent,ramakrishna2014pose} and achieve superior performance." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1603.06937" id="1014">
-        <attvalues>
-          <attvalue for="5" value=" The performance of OpenPose\cite{cao2017realtime} and Hourglass\cite{newell2016stacked} demonstrate that there is only about 0." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1703.06870" id="1015">
-        <attvalues>
-          <attvalue for="5" value=" Top-Down methods will generate human boundary boxes\cite{fang2017rmpe} or proposals\cite{he2017mask} in the first step and then estimate joints positions for each proposal, the processing speed is related to the number of boundary boxes or proposals in one image." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1612.00137" id="1016">
-        <attvalues>
-          <attvalue for="5" value=" Top-Down methods will generate human boundary boxes\cite{fang2017rmpe} or proposals\cite{he2017mask} in the first step and then estimate joints positions for each proposal, the processing speed is related to the number of boundary boxes or proposals in one image." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1711.07319" id="1017">
-        <attvalues>
-          <attvalue for="5" value=" For example, the Online Hard Example Mining (OHEM)\cite{chen2018cascaded} and Graph Neural Networks (GNNs)\cite{zhang2019human} have been applied for single person pose refinement, however, these methods are hard to implement in the multi-person case directly because of the uncertainty of joints numbers." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1701.01779" id="1018">
-        <attvalues>
-          <attvalue for="5" value=" The PifPaf\cite{kreiss2019pifpaf} proposes a combination of auxiliary predictions by combining G-RMI\cite{papandreou2017towards} proposed offsets and their modified middle-level offsets\cite{papandreou2018personlab}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1804.06208" id="1019">
-        <attvalues>
-          <attvalue for="5" value=" These methods to generate multi-person pose could be divided into two types: Top-Down methods\cite{newell2016stacked,he2017mask,fang2017rmpe,chen2018cascaded,papandreou2017towards,xiao2018simple,li2019crowdpose,li2019rethinking,zhang2019human,chou2018self} and Bottom-Up methods\cite{cao2017realtime,cao2018openpose,newell2017associative,kocabas2018multiposenet,papandreou2018personlab,nie2019single,kreiss2019pifpaf,varadarajan2018greedy,gkioxari2013articulated, liu2020resolution}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1812.00324" id="1020">
-        <attvalues>
-          <attvalue for="5" value=" These methods to generate multi-person pose could be divided into two types: Top-Down methods\cite{newell2016stacked,he2017mask,fang2017rmpe,chen2018cascaded,papandreou2017towards,xiao2018simple,li2019crowdpose,li2019rethinking,zhang2019human,chou2018self} and Bottom-Up methods\cite{cao2017realtime,cao2018openpose,newell2017associative,kocabas2018multiposenet,papandreou2018personlab,nie2019single,kreiss2019pifpaf,varadarajan2018greedy,gkioxari2013articulated, liu2020resolution}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1901.00148" id="1021">
-        <attvalues>
-          <attvalue for="5" value=" Adopting the same loss function in a multi-stage network leads to the unbalanced learning difficulty for different stages, there are few new features for the final several stages to learn for refining the performance\cite{li2019rethinking}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1901.01760" id="1022">
-        <attvalues>
-          <attvalue for="5" value=" For example, the Online Hard Example Mining (OHEM)\cite{chen2018cascaded} and Graph Neural Networks (GNNs)\cite{zhang2019human} have been applied for single person pose refinement, however, these methods are hard to implement in the multi-person case directly because of the uncertainty of joints numbers." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1707.02439" id="1023">
-        <attvalues>
-          <attvalue for="5" value=" These methods to generate multi-person pose could be divided into two types: Top-Down methods\cite{newell2016stacked,he2017mask,fang2017rmpe,chen2018cascaded,papandreou2017towards,xiao2018simple,li2019crowdpose,li2019rethinking,zhang2019human,chou2018self} and Bottom-Up methods\cite{cao2017realtime,cao2018openpose,newell2017associative,kocabas2018multiposenet,papandreou2018personlab,nie2019single,kreiss2019pifpaf,varadarajan2018greedy,gkioxari2013articulated, liu2020resolution}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1611.08050" id="1024">
-        <attvalues>
-          <attvalue for="5" value=" These proposals are network-agnostic and could be implemented for common Bottom-Up\cite{cao2017realtime,papandreou2018personlab,kocabas2018multiposenet,kreiss2019pifpaf,wang2018magnify} frameworks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1812.08008" id="1025">
-        <attvalues>
-          <attvalue for="5" value=" But not all information (pixels value) of auxiliary predictions has been used in the post-processing algorithm\cite{cao2017realtime,cao2018openpose,wang2008multiple,newell2017associative,nie2019single,kreiss2019pifpaf}, which means different regions of auxiliary predictions have different importance but be treated the same way by simple loss functions." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1611.05424" id="1026">
-        <attvalues>
-          <attvalue for="5" value=" Excepting generate several auxiliary predictions, most State-Of-The-Art (SOTA) pose networks use multi-stage structures\cite{cao2017realtime,newell2017associative,nie2019single} to refine predictions." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1807.04067" id="1027">
-        <attvalues>
-          <attvalue for="5" value=" These proposals are network-agnostic and could be implemented for common Bottom-Up\cite{cao2017realtime,papandreou2018personlab,kocabas2018multiposenet,kreiss2019pifpaf,wang2018magnify} frameworks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1803.08225" id="1028">
-        <attvalues>
-          <attvalue for="5" value=" These proposals are network-agnostic and could be implemented for common Bottom-Up\cite{cao2017realtime,papandreou2018personlab,kocabas2018multiposenet,kreiss2019pifpaf,wang2018magnify} frameworks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1908.09220" id="1029">
-        <attvalues>
-          <attvalue for="5" value=" Excepting generate several auxiliary predictions, most State-Of-The-Art (SOTA) pose networks use multi-stage structures\cite{cao2017realtime,newell2017associative,nie2019single} to refine predictions." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1903.06593" id="1030">
-        <attvalues>
-          <attvalue for="5" value=" These proposals are network-agnostic and could be implemented for common Bottom-Up\cite{cao2017realtime,papandreou2018personlab,kocabas2018multiposenet,kreiss2019pifpaf,wang2018magnify} frameworks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1708.09182" id="1031">
-        <attvalues>
-          <attvalue for="5" value=" These methods to generate multi-person pose could be divided into two types: Top-Down methods\cite{newell2016stacked,he2017mask,fang2017rmpe,chen2018cascaded,papandreou2017towards,xiao2018simple,li2019crowdpose,li2019rethinking,zhang2019human,chou2018self} and Bottom-Up methods\cite{cao2017realtime,cao2018openpose,newell2017associative,kocabas2018multiposenet,papandreou2018personlab,nie2019single,kreiss2019pifpaf,varadarajan2018greedy,gkioxari2013articulated, liu2020resolution}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1207.0580" id="1032">
-        <attvalues>
-          <attvalue for="5" value=" Since Deep Neural Networks (DNNs)\cite{hinton2012improving} have achieved superior performance in prior knowledge learning, it is expanded into various fields." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1602.00134" id="1033">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1411.4280" id="1034">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1406.2984" id="1035">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1312.4659" id="1036">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1508.00271" id="1037">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1609.01743" id="1038">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1708.01101" id="1039">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1705.00389" id="1040">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1803.09894" id="1041">
-        <attvalues>
-          <attvalue for="5" value=" Especially, the Conventional Neural Networks\cite{krizhevsky2012imagenet} have been shown to outperform DNNs on this task\cite{wei2016convolutional,ouyang2014multi,tompson2015efficient,tompson2014joint,yang2011articulated,toshev2014deeppose,fragkiadaki2015recurrent,bulat2016human,chu2017multi,yang2017learning,chen2017adversarial,tang2018deeply,ke2018multi} as CNNs can capture spatial information in shared weights simultaneously." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1707.05388" id="1042">
-        <attvalues>
-          <attvalue for="5" value=" At the same time, the MicroSort COCO keypoints detection dataset\cite{ruggero2017benchmarking} and the benchmark has been published, The OpenPose\cite{cao2017realtime}, which combined part-based methods and CNNs, archived first place in the COCO keypoints task in 2016." />
-        </attvalues>
-      </edge>
-      <edge source="2110.10734" target="1708.02002" id="1043">
-        <attvalues>
-          <attvalue for="5" value=" The auxiliary predictions are high-level supervision that defines the network outputs' types; the loss functions are lower-level supervision that defines how does the network generate expectant outputs\cite{lin2017focal}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1601.05484" id="1044">
-        <attvalues>
-          <attvalue for="5" value=" Estimating the 6DoF pose of an object is an important problem with applications in various domains like robotics~\cite{correll2016analysis}, augmented reality~\cite{arpose} and autonomous driving~\cite{song2019apollocar3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1811.12222" id="1045">
-        <attvalues>
-          <attvalue for="5" value=" Estimating the 6DoF pose of an object is an important problem with applications in various domains like robotics~\cite{correll2016analysis}, augmented reality~\cite{arpose} and autonomous driving~\cite{song2019apollocar3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1711.10006" id="1046">
-        <attvalues>
-          <attvalue for="5" value=" Some studies~\cite{kehl2017ssd}\cite{xiang2018posecnn} attempted to regress the 6DoF pose directly from the image, however, these were not as competitive as recent two stage approaches." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1703.10896" id="1047">
-        <attvalues>
-          <attvalue for="5" value=" \cite{rad2017bb8}\cite{tekin2018real}\cite{hu2019segmentation} used a CNN based architecture to segment out regions containing the object and regress the 2D keypoints from those regions." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1711.08848" id="1048">
-        <attvalues>
-          <attvalue for="5" value=" \cite{rad2017bb8}\cite{tekin2018real}\cite{hu2019segmentation} used a CNN based architecture to segment out regions containing the object and regress the 2D keypoints from those regions." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1812.02541" id="1049">
-        <attvalues>
-          <attvalue for="5" value=" Their end-to-end trainable model showed improved results compared to the two stage approach as validated with two state of the art correspondence estimators~\cite{hu2019segmentation}~\cite{peng2019pvnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1812.11788" id="1050">
-        <attvalues>
-          <attvalue for="5" value=" We follow up on their model with~\cite{peng2019pvnet} as the correspondence estimator as it shows superior performance and refer to it as SSPE." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11078" target="1911.08324" id="1051">
-        <attvalues>
-          <attvalue for="5" value="~\cite{hu2020single} proposed to replace the non-differentiable RANSAC based pose estimator with a trainable neural network to estimate the 6DoF object pose." />
-        </attvalues>
-      </edge>
-      <edge source="2110.03479" target="1712.01259" id="1052">
-        <attvalues>
-          <attvalue for="5" value=" Most of the existing methods usually ignore the underlying mathematical formulation of the camera model and instead propose an end-to-end framework to directly estimate the desired parameters~\cite{workman2015deepfocal,rong2016radial,hold2018perceptual, lopez2019deep, zhai2016detecting, detone2016deep,bogdan2018deepcalib,zhang2020deepptz,workman2016horizon,barreto2006unifying}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.03479" target="1604.02129" id="1053">
-        <attvalues>
-          <attvalue for="5" value=" Most of the existing methods usually ignore the underlying mathematical formulation of the camera model and instead propose an end-to-end framework to directly estimate the desired parameters~\cite{workman2015deepfocal,rong2016radial,hold2018perceptual, lopez2019deep, zhai2016detecting, detone2016deep,bogdan2018deepcalib,zhang2020deepptz,workman2016horizon,barreto2006unifying}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.03479" target="1606.03798" id="1054">
-        <attvalues>
-          <attvalue for="5" value=" \item Unlike existing methods, instead of designing an end-to-end solution to directly estimate the desired parameters~\cite{detone2016deep}, we proposed a new representation that represents camera model equations as a neural network in a multi-task learning (MTL) framework." />
-        </attvalues>
-      </edge>
-      <edge source="2110.03479" target="1711.03938" id="1055">
-        <attvalues>
-          <attvalue for="5" value=" We addressed this limitation by generating a synthetic dataset from two towns in CARLA~\cite{Dosovitskiy17} simulation consisting of $49$ different camera settings." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="1603.07485" id="1056">
-        <attvalues>
-          <attvalue for="5" value=" In previous works~\cite{Khoreva_2017_CVPR_simple_does_it, Hsu_2019_NIPS_BBTP, Lee_2021_CVPR_BBAM, Tian_2021_CVPR_boxinst}, foreground is separated based on image appearance information." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="2012.02310" id="1057">
-        <attvalues>
-          <attvalue for="5" value=" For example, BoxInst~\cite{Tian_2021_CVPR_boxinst}, which is the state-of-the-art approach, has addressed the separation problem by segmenting regions based on the color similarity of local pixel pairs." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="1910.05395" id="1058">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, further fine-grained architectures, such as multi-level feature maps~\cite{Rashed_2019_ICCV_FuseMODNet} and feature pyramid networks~\cite{Mohamed_2021_IV_instance_motion_segmentation}, have recently been introduced." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="2008.07008" id="1059">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, further fine-grained architectures, such as multi-level feature maps~\cite{Rashed_2019_ICCV_FuseMODNet} and feature pyramid networks~\cite{Mohamed_2021_IV_instance_motion_segmentation}, have recently been introduced." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="2012.03400" id="1060">
-        <attvalues>
-          <attvalue for="5" value=" Several recent approaches for the instance segmentation have also explored the use of temporal information by aggregating features across frames~\cite{Fu_2021_AAAI_CompFeat, Bertasius_2020_CVPR_MaskProp} or using 3D convolutions~\cite{Voigtlaender_2019_CVPR_mots, Athar_2020_ECCV_StemSeg}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="1902.03604" id="1061">
-        <attvalues>
-          <attvalue for="5" value=" Several recent approaches for the instance segmentation have also explored the use of temporal information by aggregating features across frames~\cite{Fu_2021_AAAI_CompFeat, Bertasius_2020_CVPR_MaskProp} or using 3D convolutions~\cite{Voigtlaender_2019_CVPR_mots, Athar_2020_ECCV_StemSeg}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="2103.12886" id="1062">
-        <attvalues>
-          <attvalue for="5" value=" Previous work~\cite{Liu_2021_CVPR_temporal_mask_consistency} attempted to use motion information for instance segmentation in a weakly supervised manner." />
-        </attvalues>
-      </edge>
-      <edge source="2202.13006" target="1905.04804" id="1063">
-        <attvalues>
-          <attvalue for="5" value=" (3) Extensive evaluation on the YouTube-VIS 2019~\cite{yang_ICCV_2019_VIS} benchmark dataset demonstrates that the proposed method outperforms the state-of-the-art approach even with less complex pre-trained backbones." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15903" target="1911.06634" id="1064">
-        <attvalues>
-          <attvalue for="5" value=" More recently, the endeavor to solve the more common and practically significant scenario of a single input image has received a lot of appreciation \cite{li2014single}, \cite{shih2015reflection}, \cite{li2020single}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15903" target="1708.03474" id="1065">
-        <attvalues>
-          <attvalue for="5" value=" To deal with the problem's ill-posedness, the recent learning-based method has utilized different auxiliary information as prior and constraint \cite{fan2017generic}, \cite{wan2018crrn}, \cite{lyu2019reflection}, \cite{sun2019multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15903" target="1805.11802" id="1066">
-        <attvalues>
-          <attvalue for="5" value=" For example, in \cite{wan2018crrn} Wan et al." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15903" target="1508.06576" id="1067">
-        <attvalues>
-          <attvalue for="5" value=" \\Next, we reintroduced the well-known style loss \cite{gatys2015neural} as feature co-occurrence loss for reflection removal." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1512.02134" id="1068">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1905.07443" id="1069">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2010.13501" id="1070">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2203.02146" id="1071">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2203.11483" id="1072">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1810.02695" id="1073">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1904.06587" id="1074">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2103.07094" id="1075">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1904.09099" id="1076">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1909.03751" id="1077">
-        <attvalues>
-          <attvalue for="5" value=" Most of these methods either regress the disparity from the 3D cost volume upon 2D features (AMNet~\cite{du2019amnet}, AcfNet~\cite{zhang2020adaptive}, OptStereo~\cite{wang2021pvstereo}, GANet-deep~\cite{ganet}, CSPN~\cite{cheng2019learning}, LEAStereo~\cite{cheng2020hierarchical}, and ACVNet~\cite{xu2022attention}), or directly from the 2D feature maps (DispNet~\cite{sceneflow}, AutoDispNet~\cite{iccv19_autodispnet})." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2109.07547" id="1078">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2003.12039" id="1079">
-        <attvalues>
-          <attvalue for="5" value=" More recently, methods based on the strategy of excellent work on optical flow estimation, RAFT~\cite{teed2020raft}, iteratively update the disparity field using recurrent modules with the features sampled from the correlation volume, eg, RaftStereo~\cite{lipson2021raft} and CREStereo~\cite{li2022practical}, establishing the state of the arts." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1907.01341" id="1080">
-        <attvalues>
-          <attvalue for="5" value="~\cite{ranftl2020towards} revealed that training with mixed datasets is powerful even for zero-shot cross-dataset generalization." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="2003.14338" id="1081">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1804.06534" id="1082">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.12785" target="1912.06704" id="1083">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we collect seven public available datasets, eg, Sceneflow~\cite{sceneflow}, CreStereo~\cite{li2022practical}, Tartan~Air~\cite{tartanair2020iros}, Falling Things~\cite{tremblay2018falling}, Sintel~Stereo~\cite{butler2012naturalistic}, HR-VS~\cite{yang2019hsm} and InStereo2K~\cite{bao2020instereo2k}, and mix them for pre-training the RaftStereo~\cite{lipson2021raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="1506.01497" id="1084">
-        <attvalues>
-          <attvalue for="5" value=" Traditionally, effective object detectors were developed by collecting a large amount of annotated training data and training the model offline, a process that could take days or weeks and result in models tightly coupled to specific datasets or domains, such as Faster-RCNN~\cite{fasterrcnn}, YOLO~\cite{yolo} and etc on COCO dataset~\cite{cocodataset}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="1506.02640" id="1085">
-        <attvalues>
-          <attvalue for="5" value=" Traditionally, effective object detectors were developed by collecting a large amount of annotated training data and training the model offline, a process that could take days or weeks and result in models tightly coupled to specific datasets or domains, such as Faster-RCNN~\cite{fasterrcnn}, YOLO~\cite{yolo} and etc on COCO dataset~\cite{cocodataset}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="1405.0312" id="1086">
-        <attvalues>
-          <attvalue for="5" value=" Traditionally, effective object detectors were developed by collecting a large amount of annotated training data and training the model offline, a process that could take days or weeks and result in models tightly coupled to specific datasets or domains, such as Faster-RCNN~\cite{fasterrcnn}, YOLO~\cite{yolo} and etc on COCO dataset~\cite{cocodataset}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2112.03857" id="1087">
-        <attvalues>
-          <attvalue for="5" value=" These new models, like GLIP~\cite{glip} and Grounding-DINO~\cite{g-dino}, are capable of detecting wide range of objects." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2303.05499" id="1088">
-        <attvalues>
-          <attvalue for="5" value=" For example, the state-of-the-art open-set model Grounding-DINO\cite{g-dino}, primarily trained on large-scale natural scene datasets, achieves less than 1\% mAP on the aerial dataset DOTA~\cite{dota}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="1711.10398" id="1089">
-        <attvalues>
-          <attvalue for="5" value=" We construct an aerial domain adaptation benchmark with DOTA~\cite{dota} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2005.11401" id="1090">
-        <attvalues>
-          <attvalue for="5" value=" Notably, the retrieval-augmented paradigm has proven effective for large language models (LLMs) in addressing forgetting and hallucination in novel text domains (eg, retrieval-augmented generation (RAG~\cite{rag}) and KNN-LM~\cite{knn-lm})." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="1911.00172" id="1091">
-        <attvalues>
-          <attvalue for="5" value=" Notably, the retrieval-augmented paradigm has proven effective for large language models (LLMs) in addressing forgetting and hallucination in novel text domains (eg, retrieval-augmented generation (RAG~\cite{rag}) and KNN-LM~\cite{knn-lm})." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2301.07094" id="1092">
-        <attvalues>
-          <attvalue for="5" value=" There are also existing works on retrieval-augmented visual models with three main categories: (1) Generalizing vision tasks by retrieving task-related images and tuning an additional task-specific encoder (eg, REACT~\cite{learning-customized-visual-models-with-retrieval-augmented-knowledge})." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2202.11233" id="1093">
-        <attvalues>
-          <attvalue for="5" value=" (2) Solving long-tail classification-only problem by a huge augmented knowledge base~\cite{rac}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.10716" target="2404.05687" id="1094">
-        <attvalues>
-          <attvalue for="5" value=" (3) Open-vocabulary object detection, which requires a complex multi-step training process and LLMs to function~\cite{retrieval-augmented-open-vocabulary-object-detection}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2407.05913" id="1095">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have emerged as powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, offering a versatile framework for representing and analyzing complex visual data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2001.06807" id="1096">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have emerged as powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, offering a versatile framework for representing and analyzing complex visual data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2407.05924" id="1097">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have emerged as powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, offering a versatile framework for representing and analyzing complex visual data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2106.03772" id="1098">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have emerged as powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, offering a versatile framework for representing and analyzing complex visual data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2107.01319" id="1099">
-        <attvalues>
-          <attvalue for="5" value=" These approaches leverage the inherent structure and relationships within images \cite{wang2015robust,tinghuai2016method,xing2021learning} and videos \cite{wang2010video,wang2014wide,chen2020fine}, enabling more sophisticated and context-aware analysis." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2003.00392" id="1100">
-        <attvalues>
-          <attvalue for="5" value=" These approaches leverage the inherent structure and relationships within images \cite{wang2015robust,tinghuai2016method,xing2021learning} and videos \cite{wang2010video,wang2014wide,chen2020fine}, enabling more sophisticated and context-aware analysis." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2108.00580" id="1101">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2001.07108" id="1102">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="1606.02280" id="1103">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2007.07020" id="1104">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05913" target="2203.03145" id="1105">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models can capture spatial, temporal, and semantic dependencies crucial for various tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.07409" target="2003.10152" id="1106">
-        <attvalues>
-          <attvalue for="5" value=" There are a lot of works~\cite{2020Object, li2020improving, chen2018encoderdecoder, liu2021swin, wu2020ainnoseg, long2015fully,wang2020solov2, he2018mask, tao2020hierarchical, lin2019face,ruan2018devil} studied face parsing and scene parsing." />
-        </attvalues>
-      </edge>
-      <edge source="2106.07409" target="1809.05996" id="1107">
-        <attvalues>
-          <attvalue for="5" value=" There are a lot of works~\cite{2020Object, li2020improving, chen2018encoderdecoder, liu2021swin, wu2020ainnoseg, long2015fully,wang2020solov2, he2018mask, tao2020hierarchical, lin2019face,ruan2018devil} studied face parsing and scene parsing." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04887" target="1710.05172" id="1108">
-        <attvalues>
-          <attvalue for="5" value=" Starting from the pair-wise CoSOD model \cite{CSpair}, researchers have designed different unsupervised approaches for inter-image relationship extraction, such as clustering \cite{CCS}, rank constraint \cite{SWCS}, similarity matching \cite{ECS,icme18}, and depth cue \cite{MFLP,crm2019tmm,crm2019tc}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04887" target="1811.06679" id="1109">
-        <attvalues>
-          <attvalue for="5" value=" Starting from the pair-wise CoSOD model \cite{CSpair}, researchers have designed different unsupervised approaches for inter-image relationship extraction, such as clustering \cite{CCS}, rank constraint \cite{SWCS}, similarity matching \cite{ECS,icme18}, and depth cue \cite{MFLP,crm2019tmm,crm2019tc}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04887" target="1711.01371" id="1110">
-        <attvalues>
-          <attvalue for="5" value=" Starting from the pair-wise CoSOD model \cite{CSpair}, researchers have designed different unsupervised approaches for inter-image relationship extraction, such as clustering \cite{CCS}, rank constraint \cite{SWCS}, similarity matching \cite{ECS,icme18}, and depth cue \cite{MFLP,crm2019tmm,crm2019tc}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04887" target="2003.06167" id="1111">
-        <attvalues>
-          <attvalue for="5" value=" In the feature decoding of the CoSOD task, existing up-sampling or deconvolution based methods \cite{CSMG,GCAGC} ignore the maintenance of inter-image consistency, which may lead to the inconsistency of co-salient objects among different images and introduce additional artifacts." />
-        </attvalues>
-      </edge>
-      <edge source="2011.04887" target="1707.07381" id="1112">
-        <attvalues>
-          <attvalue for="5" value=" Previous studies adopted the recurrent learning \cite{RCNet} and concatenation-convolution \cite{GCS} strategies to model the inter-image relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2007.13393" target="1812.03828" id="1113">
-        <attvalues>
-          <attvalue for="5" value=" Many deep 3D implicit field reconstruction works~\cite{mescheder2019occupancy} \cite{chen2019learning} explore the use of global shape encoding." />
-        </attvalues>
-      </edge>
-      <edge source="2007.13393" target="1905.10711" id="1114">
-        <attvalues>
-          <attvalue for="5" value=" Usually all the sampled points are projected to the images \cite{xu2019disn}, and hence points in the back use features of the points that occlude them." />
-        </attvalues>
-      </edge>
-      <edge source="2007.13393" target="1812.02822" id="1115">
-        <attvalues>
-          <attvalue for="5" value=" Many deep 3D implicit field reconstruction works~\cite{mescheder2019occupancy} \cite{chen2019learning} explore the use of global shape encoding." />
-        </attvalues>
-      </edge>
-      <edge source="2007.13393" target="1903.03322" id="1116">
-        <attvalues>
-          <attvalue for="5" value=" Unlike previous works \cite{wang20193dn} \cite{yao2019front2back} that focus on the design of loss function, detection or encoding of symmetry, our method naturally integrates into the pixel-to-mesh alignment framework." />
-        </attvalues>
-      </edge>
-      <edge source="2007.13393" target="1912.10589" id="1117">
-        <attvalues>
-          <attvalue for="5" value=" Recently Front2Back \cite{yao2019front2back} addresses such issues by detecting reflective symmetries from the data and synthesizing the opposite orthographic view." />
-        </attvalues>
-      </edge>
-      <edge source="2007.11876" target="1807.07987" id="1118">
-        <attvalues>
-          <attvalue for="5" value=" Similar to most modern image analysis solutions, our method builds on a Convolutional Neural Network (CNN)~\cite{Lecun2015a}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.11876" target="1703.06870" id="1119">
-        <attvalues>
-          <attvalue for="5" value=" In earlier works, several CNNs have been designed to address the object detection problem, and among the most accurate, Mask~R-CNN~\cite{He2017} has been trained to detect a large variety of objects, including balls." />
-        </attvalues>
-      </edge>
-      <edge source="2007.11876" target="1807.03247" id="1120">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, regression of object coordinates is known to be poorly addressed by CNNs~\cite{Liu2018}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.11876" target="1706.05587" id="1121">
-        <attvalues>
-          <attvalue for="5" value=" Primarily, we propose to formulate the ball detection problem as a segmentation problem, for which CNNs are known to be quite effective~\cite{Chen2017a}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1405.0312" id="1122">
-        <attvalues>
-          <attvalue for="5" value=" The majority of works in object detection mainly describe results on two datasets: Pascal VOC \cite{everingham2011pascal} and MS-COCO \cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1904.00853" id="1123">
-        <attvalues>
-          <attvalue for="5" value=" \cite{goldman2019precise} proposed a dataset with the following key features: 1) a high number of objects on an image, 2) objects are densely packed." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1805.00123" id="1124">
-        <attvalues>
-          <attvalue for="5" value=" Considering this information, we might reference an example of a similar task from a different domain \cite{shao2018crowdhuman} where both RetinaNet \cite{lin2017focal} and Faster-RCNN \cite{ren2015faster} were taking as baseline solutions and showed good results." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1708.02002" id="1125">
-        <attvalues>
-          <attvalue for="5" value=" Considering this information, we might reference an example of a similar task from a different domain \cite{shao2018crowdhuman} where both RetinaNet \cite{lin2017focal} and Faster-RCNN \cite{ren2015faster} were taking as baseline solutions and showed good results." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1506.01497" id="1126">
-        <attvalues>
-          <attvalue for="5" value=" Considering this information, we might reference an example of a similar task from a different domain \cite{shao2018crowdhuman} where both RetinaNet \cite{lin2017focal} and Faster-RCNN \cite{ren2015faster} were taking as baseline solutions and showed good results." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1611.10012" id="1127">
-        <attvalues>
-          <attvalue for="5" value=" For now there are plenty of such toolboxes based on different deep learning frameworks: Tensorflow Object Detection API \cite{huang2016speedaccuracy}, Detectron \cite{Detectron2018} and Detectron2 \cite{wu2019detectron2}, MMDetection \cite{chen2019mmdetection}, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07825" target="1906.07155" id="1128">
-        <attvalues>
-          <attvalue for="5" value=" For now there are plenty of such toolboxes based on different deep learning frameworks: Tensorflow Object Detection API \cite{huang2016speedaccuracy}, Detectron \cite{Detectron2018} and Detectron2 \cite{wu2019detectron2}, MMDetection \cite{chen2019mmdetection}, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="1906.06543" id="1129">
-        <attvalues>
-          <attvalue for="5" value=" Depth information plays a fundamental role in a wide range of computer vision applications, such as 3D reconstruction~\cite{han2019image, peng2021shape}, semantic scene understanding~\cite{wang2021salient, xu2022weakly, wang2022learning} and autonomous driving~\cite{du2021ago, wang2021progressive}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="1904.09146" id="1130">
-        <attvalues>
-          <attvalue for="5" value=" Depth information plays a fundamental role in a wide range of computer vision applications, such as 3D reconstruction~\cite{han2019image, peng2021shape}, semantic scene understanding~\cite{wang2021salient, xu2022weakly, wang2022learning} and autonomous driving~\cite{du2021ago, wang2021progressive}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="2208.11658" id="1131">
-        <attvalues>
-          <attvalue for="5" value=" Depth information plays a fundamental role in a wide range of computer vision applications, such as 3D reconstruction~\cite{han2019image, peng2021shape}, semantic scene understanding~\cite{wang2021salient, xu2022weakly, wang2022learning} and autonomous driving~\cite{du2021ago, wang2021progressive}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="2103.07094" id="1132">
-        <attvalues>
-          <attvalue for="5" value=" Over the past decades, the performance of stereo matching algorithms has improved markedly~\cite{wang2021pvstereo, shankar2022learned}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="1512.03385" id="1133">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances of deep learning algorithms have revolutionized the area of computer vision; tremendous progress has been achieved in a variety of domains, such as image classification~\cite{he2016deep, dosovitskiy2021an}, object localization~\cite{zhang2021weakly, wu2022edn} and semantic segmentation~\cite{strudel2021segmenter, zhao2018psanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="2104.07918" id="1134">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances of deep learning algorithms have revolutionized the area of computer vision; tremendous progress has been achieved in a variety of domains, such as image classification~\cite{he2016deep, dosovitskiy2021an}, object localization~\cite{zhang2021weakly, wu2022edn} and semantic segmentation~\cite{strudel2021segmenter, zhao2018psanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="2105.05633" id="1135">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances of deep learning algorithms have revolutionized the area of computer vision; tremendous progress has been achieved in a variety of domains, such as image classification~\cite{he2016deep, dosovitskiy2021an}, object localization~\cite{zhang2021weakly, wu2022edn} and semantic segmentation~\cite{strudel2021segmenter, zhao2018psanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09598" target="2104.01530" id="1136">
-        <attvalues>
-          <attvalue for="5" value=" GDSR is no exception: a myriad of neural networks have been developed and advanced the state of the art~\cite{AHMF, zhao2021discrete, GraphSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07807" target="cs/0503076" id="1137">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{Meingast_Geometric_2005}, Meingast et al introduced the geometric model for RS cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07807" target="1904.06770" id="1138">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Fraundorfer_Minimal_2010} fused information from external sensors like IMU to estimate the relative motion while Lee et al \cite{Lee_Gyroscope_2019} simplified the pose estimation problem by using the gyroscope." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07807" target="1605.00475" id="1139">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by relative pose estimation for monocular camera \cite{Dai_epipolar_2016}, we exploit the epipolar geometry to solve the relative pose from consecutive RS frames and present the first relative pose estimation algorithm for stereo RS cameras as shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07807" target="1907.00276" id="1140">
-        <attvalues>
-          <attvalue for="5" value=" Existing relative pose estimation methods for stereo cameras such as \cite{Vakhitov_2018_line_point} are all designed for GS cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2006.07807" target="1903.03943" id="1141">
-        <attvalues>
-          <attvalue for="5" value="\cite{Zhuang_Rolling_shutter_aware_2017} introduced optical flow based on RS successive frames, which proposed an 8-point algorithm for constant motion, and 9-point algorithm for constant acceleration motion, then they achieved SfM as well as image rectification." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="1511.02919" id="1142">
-        <attvalues>
-          <attvalue for="5" value=" ``In-the-wild&quot; collections, including LIVE Challenge \cite{ghadiyaram2015massive}, KonIQ-10k \cite{hosu2020koniq}, and PaQ-2-PiQ \cite{ying2020patches}, offer a better representation of real-world conditions, but their uncontrolled data collection and subjective labeling make them less suited for rigorous digital camera evaluations." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="1910.06180" id="1143">
-        <attvalues>
-          <attvalue for="5" value=" ``In-the-wild&quot; collections, including LIVE Challenge \cite{ghadiyaram2015massive}, KonIQ-10k \cite{hosu2020koniq}, and PaQ-2-PiQ \cite{ying2020patches}, offer a better representation of real-world conditions, but their uncontrolled data collection and subjective labeling make them less suited for rigorous digital camera evaluations." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="1912.10088" id="1144">
-        <attvalues>
-          <attvalue for="5" value=" ``In-the-wild&quot; collections, including LIVE Challenge \cite{ghadiyaram2015massive}, KonIQ-10k \cite{hosu2020koniq}, and PaQ-2-PiQ \cite{ying2020patches}, offer a better representation of real-world conditions, but their uncontrolled data collection and subjective labeling make them less suited for rigorous digital camera evaluations." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="2304.05772" id="1145">
-        <attvalues>
-          <attvalue for="5" value=" This is often achieved through a semantics-aware multitasking framework \cite{chahine2023image, fang2020perceptual, feng2023learning, huang2020multi, su2020blindly}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="1609.04757" id="1146">
-        <attvalues>
-          <attvalue for="5" value=" BIQA methods, based on convolutional neural networks (CNN) \cite{kang2014convolutional, kim2016fully, zhang2018blind}, have shown marked improvements over their classical counterparts, due to their ability to extract perceptual quality information \cite{ghadiyaram2017perceptual, mittal2012no, moorthy2011blind, saad2012blind, ye2012unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.09178" target="2105.14550" id="1147">
-        <attvalues>
-          <attvalue for="5" value=" Given that image quality is inherently subjective and varies under different conditions, numerous studies underscore the significance of integrating semantic information into the assessment of image quality, tackling a challenge known as domain shift \cite{sun2021blind, zerman2017extensive, zhang2020learning, zhang2021uncertainty}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12950" target="1702.05448" id="1148">
-        <attvalues>
-          <attvalue for="5" value=" We also provide a careful analysis of the HICO-DET dataset \cite{chao2018learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.13509" target="1602.00763" id="1149">
-        <attvalues>
-          <attvalue for="5" value=" In particular, several MOT methods~\cite{SORT, DeepSORT, bytetrack, cao2022observation} that perform well on MOT17~\cite{MOT16}, may experience a significant performance drop on the DanceTrack~\cite{sun2022dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.13509" target="1703.07402" id="1150">
-        <attvalues>
-          <attvalue for="5" value=" Although MOT studies have been greatly developed~\cite{SORT,DeepSORT,bytetrack, Li_2022_CVPR}, a new challenge has recently attracted attention: unlike conventional MOT tasks that contain objects with distinct appearances and regular motions, MOT tasks that cover animals, group dancers, and sports players, may have indistinguishable appearances and irregular motions, which could cause existing MOT methods to fail." />
-        </attvalues>
-      </edge>
-      <edge source="2211.13509" target="2110.06864" id="1151">
-        <attvalues>
-          <attvalue for="5" value=" Although MOT studies have been greatly developed~\cite{SORT,DeepSORT,bytetrack, Li_2022_CVPR}, a new challenge has recently attracted attention: unlike conventional MOT tasks that contain objects with distinct appearances and regular motions, MOT tasks that cover animals, group dancers, and sports players, may have indistinguishable appearances and irregular motions, which could cause existing MOT methods to fail." />
-        </attvalues>
-      </edge>
-      <edge source="2211.13509" target="1603.00831" id="1152">
-        <attvalues>
-          <attvalue for="5" value=" In particular, several MOT methods~\cite{SORT, DeepSORT, bytetrack, cao2022observation} that perform well on MOT17~\cite{MOT16}, may experience a significant performance drop on the DanceTrack~\cite{sun2022dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.13509" target="2111.14690" id="1153">
-        <attvalues>
-          <attvalue for="5" value=" In particular, several MOT methods~\cite{SORT, DeepSORT, bytetrack, cao2022observation} that perform well on MOT17~\cite{MOT16}, may experience a significant performance drop on the DanceTrack~\cite{sun2022dancetrack}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07843" target="2103.08129" id="1154">
-        <attvalues>
-          <attvalue for="5" value=" \cite{kadam2021r} for point cloud registration." />
-        </attvalues>
-      </edge>
-      <edge source="2004.05275" target="1904.11111" id="1155">
-        <attvalues>
-          <attvalue for="5" value="} \cite{li2019learning} attempted to estimate dense depth maps using the multi-view stereo (MVS) method from video clips captured in stationary scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2004.05275" target="1901.04111" id="1156">
-        <attvalues>
-          <attvalue for="5" value=" This is achieved by adopting an approximation to the optimization objective in \cite{dong2019fast}, where both geometric constraints and appearance similarities are taken into account." />
-        </attvalues>
-      </edge>
-      <edge source="2004.05275" target="1612.03153" id="1157">
-        <attvalues>
-          <attvalue for="5" value=" Estimating 3D skeletons from predicted 2D poses of multiple views has been studied previously in several settings, eg the synchronized multi-camera lab environment \cite{dong2019fast,joo2017panoptic}, the synthesized sports video \cite{bridgeman2019multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.05275" target="1710.10000" id="1158">
-        <attvalues>
-          <attvalue for="5" value=" Besides, pose tracking methods like \cite{andriluka2018posetrack} can be a potential option when we need to determine which 2D poses in different frames corresponds to the same person." />
-        </attvalues>
-      </edge>
-      <edge source="2004.05275" target="1903.06593" id="1159">
-        <attvalues>
-          <attvalue for="5" value=" To accomplish this goal, we train a monocular multi-person 3D pose network, which is a modification of the network proposed in \cite{kreiss2019pifpaf}, from multiple frames using the 3D supervisions obtained by MVM." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2110.10394" id="1160">
-        <attvalues>
-          <attvalue for="5" value=" Many applications~\cite{chen2021new,wang2021deep} require images as detailed as real-world scenes, which cannot be captured by standard Low Dynamic Range (LDR) cameras\mycomment{ (much cheaper than HDR-enabled camera systems)}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2108.07978" id="1161">
-        <attvalues>
-          <attvalue for="5" value=" Many applications~\cite{chen2021new,wang2021deep} require images as detailed as real-world scenes, which cannot be captured by standard Low Dynamic Range (LDR) cameras\mycomment{ (much cheaper than HDR-enabled camera systems)}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2004.01179" id="1162">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2005.07335" id="1163">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="1710.07480" id="1164">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2210.15897" id="1165">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2309.02020" id="1166">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2211.11270" id="1167">
-        <attvalues>
-          <attvalue for="5" value=" Methods based on single-exposed LDR images often produce unwanted color, contrast, saturation, hue, luminance, brightness, and radiance levels in the resulting HDR images~\cite{santos2020single,eilertsen2017hdr,li2019hdrnet,liu2020single,le2023single,guo2023single,zou2023rawhdr,guo2022lhdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2007.01628" id="1168">
-        <attvalues>
-          <attvalue for="5" value=" Some methods~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR} utilize multi-exposed LDR images (generally $2$ or $3$) as input but often produce many artifacts in the resultant HDR images~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2310.01840" id="1169">
-        <attvalues>
-          <attvalue for="5" value=" Some methods~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR} utilize multi-exposed LDR images (generally $2$ or $3$) as input but often produce many artifacts in the resultant HDR images~\cite{niu2021hdr,cai2018learning,ren2023robust,SelfHDR}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2309.03827" id="1170">
-        <attvalues>
-          <attvalue for="5" value="~\ref{fig:teaser} depicts the clarity of the images generated by the proposed method compared to the recent state-of-the-art~\cite{barua2023arthdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2003.08934" id="1171">
-        <attvalues>
-          <attvalue for="5" value=" Recent methods employ Neural Radiance Fields~\cite{mildenhall2021nerf,huang2022hdr} drawing inspiration from physics/optics to replicate implicit color and radiance fields involved in the general camera pipeline and enable photo-realistic HDR view synthesis~\cite{mildenhall2022nerf, huang2022hdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2111.14451" id="1172">
-        <attvalues>
-          <attvalue for="5" value=" Recent methods employ Neural Radiance Fields~\cite{mildenhall2021nerf,huang2022hdr} drawing inspiration from physics/optics to replicate implicit color and radiance fields involved in the general camera pipeline and enable photo-realistic HDR view synthesis~\cite{mildenhall2022nerf, huang2022hdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2111.13679" id="1173">
-        <attvalues>
-          <attvalue for="5" value=" Recent methods employ Neural Radiance Fields~\cite{mildenhall2021nerf,huang2022hdr} drawing inspiration from physics/optics to replicate implicit color and radiance fields involved in the general camera pipeline and enable photo-realistic HDR view synthesis~\cite{mildenhall2022nerf, huang2022hdr}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="2307.02814" id="1174">
-        <attvalues>
-          <attvalue for="5" value=" Dalal \etal~\cite{dalal2023single} used Diffusion Models and exposure loss to replicate the reverse of the camera pipeline process through a conditional diffusion architecture with a noise induction process without requiring any classifier." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="1805.08318" id="1175">
-        <attvalues>
-          <attvalue for="5" value=" Attention-based methods have also been explored~\cite{nemoto2015visual,li2019hdrnet,zhang2019self,yan2019attention,chen2023improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2402.06692" target="1904.10293" id="1176">
-        <attvalues>
-          <attvalue for="5" value=" Attention-based methods have also been explored~\cite{nemoto2015visual,li2019hdrnet,zhang2019self,yan2019attention,chen2023improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="1608.03066" id="1177">
-        <attvalues>
-          <attvalue for="5" value=" Both object detector and tracker were employed to either impose spatio-temporal coherence \cite{ZhangCLWX15,drayer2016object} or learn an appearance model \cite{wang2016semi} for encoding the appearance variation of semantic objects." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="1606.02280" id="1178">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2407.05913" id="1179">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have become powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, providing a versatile framework for modeling contextual relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="1504.01013" id="1180">
-        <attvalues>
-          <attvalue for="5" value=" Pairwise contextual relationships have been investigated in image segmentation \cite{lin2016efficient} and object detection \cite{gidaris2015object,bai2017multiple} tasks, whose importance has been highlighted in effectively resolving semantic ambiguities." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="1704.00138" id="1181">
-        <attvalues>
-          <attvalue for="5" value=" Pairwise contextual relationships have been investigated in image segmentation \cite{lin2016efficient} and object detection \cite{gidaris2015object,bai2017multiple} tasks, whose importance has been highlighted in effectively resolving semantic ambiguities." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2001.06807" id="1182">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have become powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, providing a versatile framework for modeling contextual relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2407.05924" id="1183">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have become powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, providing a versatile framework for modeling contextual relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2106.03772" id="1184">
-        <attvalues>
-          <attvalue for="5" value=" Graphical models have become powerful tools in computer vision \cite{wang2010multi,qi20173d,wang2017submodular,wang2019zero,wang2019graph,yang2021learning}, providing a versatile framework for modeling contextual relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2107.01319" id="1185">
-        <attvalues>
-          <attvalue for="5" value=" These approaches exploit the inherent structure and relationships within images \cite{wang2015robust,tinghuai2016method,xing2021learning} and videos \cite{wang2010video,wang2014wide,chen2020fine}, facilitating more sophisticated and context-aware analysis." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2003.00392" id="1186">
-        <attvalues>
-          <attvalue for="5" value=" These approaches exploit the inherent structure and relationships within images \cite{wang2015robust,tinghuai2016method,xing2021learning} and videos \cite{wang2010video,wang2014wide,chen2020fine}, facilitating more sophisticated and context-aware analysis." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2108.00580" id="1187">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2001.07108" id="1188">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2007.07020" id="1189">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.06247" target="2203.03145" id="1190">
-        <attvalues>
-          <attvalue for="5" value=" By representing visual elements as nodes and their interactions as edges, graphical models capture spatial, temporal, and semantic dependencies essential for tasks such as visual information retrieval \cite{hu2013markov}, stylization \cite{WangCSCG10,wang2011stylized,wang2013learnable}, object detection \cite{WangW16,tinghuai2016apparatus,zhao2021graphfpn}, scene understanding \cite{WangW14,wang2017cross,wang2020spectral,tinghuai2020watermark,deng2021generative}, and image or video segmentation \cite{wang2015weakly,wang2016semi,wang2016primary,tinghuai2017method,tinghuai2018method1,ZhuWAK19,zhu2019cross,tinghuai2020semantic,lu2020video,wang2021end}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="2004.04730" id="1191">
-        <attvalues>
-          <attvalue for="5" value=" 3D convolutional neural-nets (CNNs) have been used extensively for video classification tasks~\cite{feichtenhofer2020x3d, tran2019video, lin2019tsm}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1706.03762" id="1192">
-        <attvalues>
-          <attvalue for="5" value=" However, their computational inefficiency has led the researchers to look for alternatives such as self-attention based Transformer architectures~\cite{vaswani2017attention}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="2010.11929" id="1193">
-        <attvalues>
-          <attvalue for="5" value="~\cite{bertasius2021space} extended the idea of~\cite{dosovitskiy2020image} to propose Timesformer for video classification by using self-attention blocks to obtain spatiotemporal contextualized feature representations from a sequence of frame-level patches." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1711.07971" id="1194">
-        <attvalues>
-          <attvalue for="5" value=" Capturing long-term dependencies across frames is crucial for video understanding as has been demonstrated by~\cite{wang2018non, wu2019long}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1810.00530" id="1195">
-        <attvalues>
-          <attvalue for="5" value="~\cite{kmiec2018learnable} used Transformer encoder along with NetVLAD blocks to get effective multimodal feature representations for large scale video understanding." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1812.02707" id="1196">
-        <attvalues>
-          <attvalue for="5" value="~\cite{girdhar2019video} proposed ‘action Transformers’ where features extracted using 3D CNN are aggregated and fed to a self-attention block to leverage the spatio-temporal information around a person for action localization." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="2102.05095" id="1197">
-        <attvalues>
-          <attvalue for="5" value="~\cite{bertasius2021space} extended the idea of~\cite{dosovitskiy2020image} to propose Timesformer for video classification by using self-attention blocks to obtain spatiotemporal contextualized feature representations from a sequence of frame-level patches." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="2008.05828" id="1198">
-        <attvalues>
-          <attvalue for="5" value="~\cite{pande2020importance, child2019generating, beltagy2020longformer} have explored the importance of local attention maps for various language understanding tasks and proposed computationally efficient Transformer models." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1701.06538" id="1199">
-        <attvalues>
-          <attvalue for="5" value=" We further propose to combine the global and local attention maps using gating mechanisms~\cite{shazeer2017outrageously}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="2004.08178" id="1200">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have explored gating in Transformer based models~\cite{chai2020highway, fedus2021switch}, but they are not motivated by gating in attention space." />
-        </attvalues>
-      </edge>
-      <edge source="2110.13992" target="1609.08675" id="1201">
-        <attvalues>
-          <attvalue for="5" value=" Finally, we perform large scale experiments on the Youtube8M dataset~\cite{abu2016youtube} and show the advantage of our model over the baseline self-attention based model for video classification." />
-        </attvalues>
-      </edge>
-      <edge source="2306.11414" target="2203.17054" id="1202">
-        <attvalues>
-          <attvalue for="5" value=" Based on previous perception tasks including 3D object detection and semantic map segmentation, current multi-camera 3D perception methods mainly lies in two types: 1) lift-splat-shoot(LSS)-based \cite{huang2022bevdet4d,huang2021bevdet,philion2020lift}, lifting 2D image features to plausible 3D volume space via implicit or explicit depth estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2306.11414" target="2112.11790" id="1203">
-        <attvalues>
-          <attvalue for="5" value=" Based on previous perception tasks including 3D object detection and semantic map segmentation, current multi-camera 3D perception methods mainly lies in two types: 1) lift-splat-shoot(LSS)-based \cite{huang2022bevdet4d,huang2021bevdet,philion2020lift}, lifting 2D image features to plausible 3D volume space via implicit or explicit depth estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2306.11414" target="2008.05711" id="1204">
-        <attvalues>
-          <attvalue for="5" value=" Based on previous perception tasks including 3D object detection and semantic map segmentation, current multi-camera 3D perception methods mainly lies in two types: 1) lift-splat-shoot(LSS)-based \cite{huang2022bevdet4d,huang2021bevdet,philion2020lift}, lifting 2D image features to plausible 3D volume space via implicit or explicit depth estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2306.11414" target="2303.09551" id="1205">
-        <attvalues>
-          <attvalue for="5" value=" 2) Transformer-based \cite{wei2023surroundocc,li2023voxformer,li2022bevformer}, which define 3D queries in 3D volume space and use transformer attention mechanism to query corresponding 2D image features." />
-        </attvalues>
-      </edge>
-      <edge source="2306.11414" target="2302.12251" id="1206">
-        <attvalues>
-          <attvalue for="5" value=" 2) Transformer-based \cite{wei2023surroundocc,li2023voxformer,li2022bevformer}, which define 3D queries in 3D volume space and use transformer attention mechanism to query corresponding 2D image features." />
-        </attvalues>
-      </edge>
-      <edge source="2101.11251" target="1906.10925" id="1207">
-        <attvalues>
-          <attvalue for="5" value=" The results show that the accuracy of e-ACJ detection is better than most event camera corner detection algorithms, such as eHarris \cite{eharris}, eFAST \cite{efast} and FA-Harris \cite{li2019fa}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="1809.03327" id="1208">
-        <attvalues>
-          <attvalue for="5" value=" Video Object Segmentation (VOS) aims to track and segment target objects in a video sequence, starting from mask annotations in the initial frame~\cite{ytvos2018, davis2017, feelvos, aot}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2208.01924" id="1209">
-        <attvalues>
-          <attvalue for="5" value=" With the increasing volume of video data, VOS technology has broad applications in fields like autonomous driving, augmented reality, and interactive video editing~\cite{perclip, bao2018cnnmrf}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="1904.00607" id="1210">
-        <attvalues>
-          <attvalue for="5" value=" VOS methods~\cite{STM, stcn} typically perform video object segmentation by comparing the test frame with previous frames." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2308.09903" id="1211">
-        <attvalues>
-          <attvalue for="5" value=" These correlated features are then utilized to accurately predict the target masks~\cite{SimVOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2106.02638" id="1212">
-        <attvalues>
-          <attvalue for="5" value=" To handle the issue of target appearance changes over time, some methods~\cite{aot, xmem} incorporate a memory module that stores samples of the targets to capture their evolving appearances." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2212.06826" id="1213">
-        <attvalues>
-          <attvalue for="5" value=" Although object query representations~\cite{ISVOS, cutie} improve ID association accuracy, they perform poorly in sequences with significant target appearance changes and small targets." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2302.01872" id="1214">
-        <attvalues>
-          <attvalue for="5" value=" On one hand, the MOSE~\cite{mose} (coMplex video Object SEgmentation) dataset is a comprehensive collection designed to provide challenging scenarios with occluded and crowded objects." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16431" target="2211.10181" id="1215">
-        <attvalues>
-          <attvalue for="5" value=" LVOS~\cite{Hong_2023_ICCV} (Long-term Video Object Segmentation) is the first densely annotated long-term video object segmentation dataset, which aims to provide a benchmark for the development and evaluation of long-term VOS models." />
-        </attvalues>
-      </edge>
-      <edge source="2002.01154" target="1810.05964" id="1216">
-        <attvalues>
-          <attvalue for="5" value=" Distinctive and robust representation of texture is the key for various multimedia applications such as image representation~\cite{al2017curvelet}, texture retrieval~\cite{al2019curvelettextureretrieval}, face recognition\cite{zhao2017dynamic}, image quality assessment~\cite{temel2019perceptual,temel2016csv}, image/texture segmentation~\cite{vo2010study}, dynamic texture/scene recognition~\cite{arashloo2014dynamic,zhao2017dynamic}, texture/color style transfer~\cite{lo2016example}, and seismic interpretation\cite{long2018comparative}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.01154" target="1810.06464" id="1217">
-        <attvalues>
-          <attvalue for="5" value=" Distinctive and robust representation of texture is the key for various multimedia applications such as image representation~\cite{al2017curvelet}, texture retrieval~\cite{al2019curvelettextureretrieval}, face recognition\cite{zhao2017dynamic}, image quality assessment~\cite{temel2019perceptual,temel2016csv}, image/texture segmentation~\cite{vo2010study}, dynamic texture/scene recognition~\cite{arashloo2014dynamic,zhao2017dynamic}, texture/color style transfer~\cite{lo2016example}, and seismic interpretation\cite{long2018comparative}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.01154" target="1812.08263" id="1218">
-        <attvalues>
-          <attvalue for="5" value=" Distinctive and robust representation of texture is the key for various multimedia applications such as image representation~\cite{al2017curvelet}, texture retrieval~\cite{al2019curvelettextureretrieval}, face recognition\cite{zhao2017dynamic}, image quality assessment~\cite{temel2019perceptual,temel2016csv}, image/texture segmentation~\cite{vo2010study}, dynamic texture/scene recognition~\cite{arashloo2014dynamic,zhao2017dynamic}, texture/color style transfer~\cite{lo2016example}, and seismic interpretation\cite{long2018comparative}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.01154" target="1812.04183" id="1219">
-        <attvalues>
-          <attvalue for="5" value=" Texture descriptors~\cite{bay2006surf,hu2016completed,liu2016evaluation, hu2017sselbp,liu2015fusing,mehta2016texture, allili2014texture}, which are robust against rotations and translations of images, are able to provide discriminative features." />
-        </attvalues>
-      </edge>
-      <edge source="2002.01154" target="1403.4334" id="1220">
-        <attvalues>
-          <attvalue for="5" value=" Covariance descriptors modeling the second-order statistics of images perform well on material categorization~\cite{harandi2014bregman}\cite{wang2012covariance}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05735" target="1801.03924" id="1221">
-        <attvalues>
-          <attvalue for="5" value=" The shortcomings of pixel-based similarity metrics have been studied substantially in the image generation community \cite{Zhang2018}, where the introduction of deep similarity metrics approximating human visual perception has improved the generation of photo-realistic images \cite{Czolbe2020, hou2017deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05735" target="2006.15057" id="1222">
-        <attvalues>
-          <attvalue for="5" value=" The shortcomings of pixel-based similarity metrics have been studied substantially in the image generation community \cite{Zhang2018}, where the introduction of deep similarity metrics approximating human visual perception has improved the generation of photo-realistic images \cite{Czolbe2020, hou2017deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05735" target="1805.04605" id="1223">
-        <attvalues>
-          <attvalue for="5" value=" As registration models are generative models \cite{dalca2018unsuperiveddiff}, we expect these similarity metrics to improve registration as well." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05735" target="1907.09200" id="1224">
-        <attvalues>
-          <attvalue for="5" value=" Current attempts at using learned similarity metrics for image registration require ground truth transformations \cite{Haskins2019a} or limit the input to the registration model \cite{Lee2019a}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02842" target="2103.16871" id="1225">
-        <attvalues>
-          <attvalue for="5" value=" developed histogram of orientated phase congruency (HOPC) \cite{ye2017robust,ye2018local}, in which an extended phase congruency model is designed for feature description." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1506.02640" id="1226">
-        <attvalues>
-          <attvalue for="5" value=" But almost all existing object detectors consider the close-world assumption that the test sets and training sets contain the same data categories \cite{redmon2016you,lin2017focal,ren2015faster,tian2019fcos}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1708.02002" id="1227">
-        <attvalues>
-          <attvalue for="5" value=" But almost all existing object detectors consider the close-world assumption that the test sets and training sets contain the same data categories \cite{redmon2016you,lin2017focal,ren2015faster,tian2019fcos}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1506.01497" id="1228">
-        <attvalues>
-          <attvalue for="5" value=" proposed the first open-world object detection model, named ORE \cite{joseph2021towards}, based on the Faster-RCNN \cite{ren2015faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1904.01355" id="1229">
-        <attvalues>
-          <attvalue for="5" value=" But almost all existing object detectors consider the close-world assumption that the test sets and training sets contain the same data categories \cite{redmon2016you,lin2017focal,ren2015faster,tian2019fcos}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="2103.02603" id="1230">
-        <attvalues>
-          <attvalue for="5" value=" proposed the first open-world object detection model, named ORE \cite{joseph2021towards}, based on the Faster-RCNN \cite{ren2015faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1805.03438" id="1231">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="2004.02434" id="1232">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="2103.00953" id="1233">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="2110.11013" id="1234">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="2011.00178" id="1235">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by work on open set recognition \cite{yang2018robust,miller2021class,yang2020convolutional,chen2021adversarial,xia2021spatial,chen2020learning}, we introduce a prototype branch to cluster known classes." />
-        </attvalues>
-      </edge>
-      <edge source="2302.11757" target="1801.09414" id="1236">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, we also introduce a Cosine Similarity-Based classifier (CSC) to help form tighter clusters \cite{wang2018cosface}." />
-        </attvalues>
-      </edge>
-      <edge source="1911.12706" target="1810.09496" id="1237">
-        <attvalues>
-          <attvalue for="5" value=" Previous work includes using the images of other cameras to help reduce the number of required corresponding points to compute epipolar geometry, \cite{8451727,satorecovering} and the vast literature on multiview geometry and pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11104" target="1804.07723" id="1238">
-        <attvalues>
-          <attvalue for="5" value=" Consequently, most previous works \cite{liu2018image, yi2020contextual, cao2021learning, rombach2022high, zeng2020high, suvorov2022resolution, zeng2022aggregated, 9195134} reporting object removal results present only qualitative results." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11104" target="1801.03924" id="1239">
-        <attvalues>
-          <attvalue for="5" value=" LPIPS \cite{zhang2018unreasonable}, PSNR \cite{wang2004image}, SSIM \cite{wang2004image}, and P-IDS \cite{zhao2021comodgan} require a pair of original and completed images, while FID \cite{heusel2017gans} and U-IDS \cite{zhao2021comodgan} can be measured using unpaired data." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11104" target="2103.10428" id="1240">
-        <attvalues>
-          <attvalue for="5" value=" LPIPS \cite{zhang2018unreasonable}, PSNR \cite{wang2004image}, SSIM \cite{wang2004image}, and P-IDS \cite{zhao2021comodgan} require a pair of original and completed images, while FID \cite{heusel2017gans} and U-IDS \cite{zhao2021comodgan} can be measured using unpaired data." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11104" target="1706.08500" id="1241">
-        <attvalues>
-          <attvalue for="5" value=" LPIPS \cite{zhang2018unreasonable}, PSNR \cite{wang2004image}, SSIM \cite{wang2004image}, and P-IDS \cite{zhao2021comodgan} require a pair of original and completed images, while FID \cite{heusel2017gans} and U-IDS \cite{zhao2021comodgan} can be measured using unpaired data." />
-        </attvalues>
-      </edge>
-      <edge source="2209.09195" target="1512.04150" id="1242">
-        <attvalues>
-          <attvalue for="5" value=" Class Activation Mapping (CAM) methods are a popular method for WSOL, allowing to highlight image regions linked to a particular class using features from the penultimate layer~\cite{zhou2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.09195" target="2201.02445" id="1243">
-        <attvalues>
-          <attvalue for="5" value=" However, CAMs provides coarse information on object location for a given input image, often resulting in poor localization performance because they tend to focus on the most discriminative regions shared among objects of the same class~\cite{negevsbelharbi2022,belharbi2022f}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.09195" target="2109.07069" id="1244">
-        <attvalues>
-          <attvalue for="5" value=" However, CAMs provides coarse information on object location for a given input image, often resulting in poor localization performance because they tend to focus on the most discriminative regions shared among objects of the same class~\cite{negevsbelharbi2022,belharbi2022f}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1702.07432" id="1245">
-        <attvalues>
-          <attvalue for="5" value=" Due to the recent advances in Deep Neural Networks (DNN), the task of 2D human pose estimation a great improvement in results~\cite{chu2017multi,cao2018openpose,newell2016stacked}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1812.08008" id="1246">
-        <attvalues>
-          <attvalue for="5" value=" Due to the recent advances in Deep Neural Networks (DNN), the task of 2D human pose estimation a great improvement in results~\cite{chu2017multi,cao2018openpose,newell2016stacked}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1603.06937" id="1247">
-        <attvalues>
-          <attvalue for="5" value=" Due to the recent advances in Deep Neural Networks (DNN), the task of 2D human pose estimation a great improvement in results~\cite{chu2017multi,cao2018openpose,newell2016stacked}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1609.05317" id="1248">
-        <attvalues>
-          <attvalue for="5" value=" In addition, DNNs~\cite{zhou2016deep} trained on such datasets have difficulties generalizing well to environments where a lot of variation is present, eg, scenarios in the wild." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1704.02447" id="1249">
-        <attvalues>
-          <attvalue for="5" value="\\ (2) Increasing the robustness of 3D human pose estimation algorithms, specifically~\cite{zhou2017towards}, with respect to challenging camera viewpoints thanks to data augmentation with 3DBodyTex." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1803.09722" id="1250">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many works focused on the challenging problem of 3D human pose estimation in the wild~\cite{zhou2017towards,yang20183d,pavllo20193d,rogez2019lcr}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1811.11742" id="1251">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many works focused on the challenging problem of 3D human pose estimation in the wild~\cite{zhou2017towards,yang20183d,pavllo20193d,rogez2019lcr}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.09989" target="1803.00455" id="1252">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many works focused on the challenging problem of 3D human pose estimation in the wild~\cite{zhou2017towards,yang20183d,pavllo20193d,rogez2019lcr}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="2210.08061" id="1253">
-        <attvalues>
-          <attvalue for="5" value=" As an example, \cite{objectdetectionmotion} generates supervisory boxes for an open-world LiDAR detector via offline object extraction using high quality scene flow estimates from Neural Scene Flow Prior (NSFP) \cite{nsfp}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="2111.01253" id="1254">
-        <attvalues>
-          <attvalue for="5" value=" As an example, \cite{objectdetectionmotion} generates supervisory boxes for an open-world LiDAR detector via offline object extraction using high quality scene flow estimates from Neural Scene Flow Prior (NSFP) \cite{nsfp}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="2012.07541" id="1255">
-        <attvalues>
-          <attvalue for="5" value=" This runtime feature formulation is similar to \cite{Zhai2020FlowMOT3M}'s use of scene flow from FlowNet3D~\cite{flownet3d} as an input primitive for their multi-object tracking pipeline; although FlowNet3D is fast enough for online processing of subsampled point clouds, its supervised feedforward formulation requires significant in-domain human annotations." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="1806.01411" id="1256">
-        <attvalues>
-          <attvalue for="5" value=" This runtime feature formulation is similar to \cite{Zhai2020FlowMOT3M}'s use of scene flow from FlowNet3D~\cite{flownet3d} as an input primitive for their multi-object tracking pipeline; although FlowNet3D is fast enough for online processing of subsampled point clouds, its supervised feedforward formulation requires significant in-domain human annotations." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="2301.00493" id="1257">
-        <attvalues>
-          <attvalue for="5" value="1 GPU months of compute) for the popular Argoverse~2~\cite{argoverse2} and Waymo Open~\cite{waymoopen} autonomous vehicle datasets, alongside our code and trained model weights, to facilitate further research." />
-        </attvalues>
-      </edge>
-      <edge source="2305.10424" target="1912.04838" id="1258">
-        <attvalues>
-          <attvalue for="5" value="1 GPU months of compute) for the popular Argoverse~2~\cite{argoverse2} and Waymo Open~\cite{waymoopen} autonomous vehicle datasets, alongside our code and trained model weights, to facilitate further research." />
-        </attvalues>
-      </edge>
-      <edge source="2407.01436" target="2306.02851" id="1259">
-        <attvalues>
-          <attvalue for="5" value=" The CVPR24 occupancy and flow prediction competition \cite{tong2023scene} focuses on developing new algorithms that predict occupancy and flow solely from camera input during inference, offering a significant platform for advancing state-of-the-art 3D occupancy and flow prediction algorithms." />
-        </attvalues>
-      </edge>
-      <edge source="2407.01436" target="2303.03991" id="1260">
-        <attvalues>
-          <attvalue for="5" value=" 3D occupancy and flow prediction \cite{tong2023scene,wang2023openoccupancy,tian2024occ3d,liu2023fully} are critical components of autonomous driving perception systems." />
-        </attvalues>
-      </edge>
-      <edge source="2407.01436" target="2304.14365" id="1261">
-        <attvalues>
-          <attvalue for="5" value=" 3D occupancy and flow prediction \cite{tong2023scene,wang2023openoccupancy,tian2024occ3d,liu2023fully} are critical components of autonomous driving perception systems." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2104.10078" id="1262">
-        <attvalues>
-          <attvalue for="5" value=" Our approach outperforms current state-of-the-art methods~\cite{Mildenhall_2020_nerf,OechsleP_2021_unisurf,yariv2021volsdf} under a sparse set of input views." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2106.10689" id="1263">
-        <attvalues>
-          <attvalue for="5" value=" Recently, implicit neural representation (INR) with coordinated-based learning has been driving significant advantages in producing high-fidelity 3D shapes~\cite{OechsleP_2021_unisurf,Wang_2021_neus,yariv2021volsdf} and photo-realistic images~\cite{Mildenhall_2020_nerf,Martinbrualla2020nerfw,Barron2021mipnerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2106.12052" id="1264">
-        <attvalues>
-          <attvalue for="5" value=" Our approach outperforms current state-of-the-art methods~\cite{Mildenhall_2020_nerf,OechsleP_2021_unisurf,yariv2021volsdf} under a sparse set of input views." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2003.08934" id="1265">
-        <attvalues>
-          <attvalue for="5" value=" Our approach outperforms current state-of-the-art methods~\cite{Mildenhall_2020_nerf,OechsleP_2021_unisurf,yariv2021volsdf} under a sparse set of input views." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2008.02268" id="1266">
-        <attvalues>
-          <attvalue for="5" value=" Recently, implicit neural representation (INR) with coordinated-based learning has been driving significant advantages in producing high-fidelity 3D shapes~\cite{OechsleP_2021_unisurf,Wang_2021_neus,yariv2021volsdf} and photo-realistic images~\cite{Mildenhall_2020_nerf,Martinbrualla2020nerfw,Barron2021mipnerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2103.13415" id="1267">
-        <attvalues>
-          <attvalue for="5" value=" Recently, implicit neural representation (INR) with coordinated-based learning has been driving significant advantages in producing high-fidelity 3D shapes~\cite{OechsleP_2021_unisurf,Wang_2021_neus,yariv2021volsdf} and photo-realistic images~\cite{Mildenhall_2020_nerf,Martinbrualla2020nerfw,Barron2021mipnerf}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="1711.00937" id="1268">
-        <attvalues>
-          <attvalue for="5" value=" Fortunately, learning the prior descriptors or prototypes from a large scale dataset is a long-studied topic from the early Bag-of-Words (BoW) in image classification, scene recognition~\cite{li_2005_bow_scene_recog} and visual SLAM~\cite{Kim_2011_slam}, Vector Quantized (VQ) codebook in data compression~\cite{Perlmutter_1994_vq}, to the recent neural discrete codebook representation in image synthesis~\cite{van_2017_vqvae,van_2019_vqvae2,Esser_2021_taming}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="1906.00446" id="1269">
-        <attvalues>
-          <attvalue for="5" value=" Fortunately, learning the prior descriptors or prototypes from a large scale dataset is a long-studied topic from the early Bag-of-Words (BoW) in image classification, scene recognition~\cite{li_2005_bow_scene_recog} and visual SLAM~\cite{Kim_2011_slam}, Vector Quantized (VQ) codebook in data compression~\cite{Perlmutter_1994_vq}, to the recent neural discrete codebook representation in image synthesis~\cite{van_2017_vqvae,van_2019_vqvae2,Esser_2021_taming}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2012.09841" id="1270">
-        <attvalues>
-          <attvalue for="5" value=" Fortunately, learning the prior descriptors or prototypes from a large scale dataset is a long-studied topic from the early Bag-of-Words (BoW) in image classification, scene recognition~\cite{li_2005_bow_scene_recog} and visual SLAM~\cite{Kim_2011_slam}, Vector Quantized (VQ) codebook in data compression~\cite{Perlmutter_1994_vq}, to the recent neural discrete codebook representation in image synthesis~\cite{van_2017_vqvae,van_2019_vqvae2,Esser_2021_taming}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="1911.10127" id="1271">
-        <attvalues>
-          <attvalue for="5" value=" We perform extensive experiments on different scene reconstruction benchmarks, including DTU~\cite{Jensen_2014_dtu}, BlendedMVS~\cite{Yao_2020_blendmvs} and the full head reconstruction benchmark H3DS~\cite{Ramon_2021_h3ds}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11170" target="2107.12512" id="1272">
-        <attvalues>
-          <attvalue for="5" value=" We perform extensive experiments on different scene reconstruction benchmarks, including DTU~\cite{Jensen_2014_dtu}, BlendedMVS~\cite{Yao_2020_blendmvs} and the full head reconstruction benchmark H3DS~\cite{Ramon_2021_h3ds}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.04055" target="2011.15126" id="1273">
-        <attvalues>
-          <attvalue for="5" value=" Sparse landmarks detection is an important problem for face detection applications \cite{amos2016openface}, face tracking with landmarks alignment as a sub-task for 3D face model fitting~\cite{bickel2007multi, dib2021practical} or to guide video synthesis for faces ~\cite{wang2020facevid2vid}, other body parts~\cite{chan2019everybody}, among several others." />
-        </attvalues>
-      </edge>
-      <edge source="2104.04055" target="1808.07371" id="1274">
-        <attvalues>
-          <attvalue for="5" value=" Sparse landmarks detection is an important problem for face detection applications \cite{amos2016openface}, face tracking with landmarks alignment as a sub-task for 3D face model fitting~\cite{bickel2007multi, dib2021practical} or to guide video synthesis for faces ~\cite{wang2020facevid2vid}, other body parts~\cite{chan2019everybody}, among several others." />
-        </attvalues>
-      </edge>
-      <edge source="2104.04055" target="1805.10483" id="1275">
-        <attvalues>
-          <attvalue for="5" value="~\cite{wu2018look} approach this problem with a focus on the boundary, taking advantage of well defined face boundary lines along which the landmarks reside." />
-        </attvalues>
-      </edge>
-      <edge source="2104.04055" target="2101.09866" id="1276">
-        <attvalues>
-          <attvalue for="5" value="~\cite{dong2018supervision} note that frame-to-frame landmark detection should ideally resemble the presence of physical markers and present an approach that uses optical flow, and later a triangulation-based approach ~\cite{dong2020supervision} that exploits the temporal information inherent in video data." />
-        </attvalues>
-      </edge>
-      <edge source="2104.04055" target="1703.10593" id="1277">
-        <attvalues>
-          <attvalue for="5" value="~\cite{zhu2017unpaired}, we pose this problem as an unpaired image translation problem, with an image generator network that translates from marked-to-unmarked images, while our novel landmark deformation prediction network performs the reverse translation." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04628" target="1902.09738" id="1278">
-        <attvalues>
-          <attvalue for="5" value=" High performing methods often require more costly system setups such as Lidar sensors \cite{lidar3, lidar4, lidar1, lidar2} for precise depth measurements or stereo cameras \cite{s1, s2, s3, s4} for stereo depth estimation, and are often more computationally expensive." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04628" target="2103.16237" id="1279">
-        <attvalues>
-          <attvalue for="5" value=" Based on the extensive and insightful analyses made by the MonoDLE method~\cite{monodle} in the KITTI benchmark, one main challenge of improving the overall performance in monocular 3D object detection lies in inferring the 3D center location with high accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04628" target="1904.07850" id="1280">
-        <attvalues>
-          <attvalue for="5" value=" It adopts the anchor-offset formulation proposed in the CenterNet~\cite{centernet} in learning the projected 3D center based on the 2D bounding box center (eg, the anchor), and proposes a simple yet effective method that facilitates better overall performance (Fig." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1610.04121" id="1281">
-        <attvalues>
-          <attvalue for="5" value=" This kind of method can be formulated as a three-step pipeline including matching cost calculation \cite{Hernandez16:PCS}, cost aggregation/optimization \cite{Hirschmuller08:PAMI, Zhang07:PAMI}, and disparity refinement \cite{Barron16:ECCV, Min14:TIP}." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1511.03296" id="1282">
-        <attvalues>
-          <attvalue for="5" value=" This kind of method can be formulated as a three-step pipeline including matching cost calculation \cite{Hernandez16:PCS}, cost aggregation/optimization \cite{Hirschmuller08:PAMI, Zhang07:PAMI}, and disparity refinement \cite{Barron16:ECCV, Min14:TIP}." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1406.2283" id="1283">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Eigen14:NIPS} combined a coarse global prediction network based on the entire image with a refinement network, and proposed a scale-invariant L2-norm loss function which is widely utilized in depth prediction." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1606.00373" id="1284">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Laina16:3DV} developed an up-projection decoder and achieved higher accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1609.03677" id="1285">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Godard17:CVPR} proposed an unsupervised method to enforce disparity consistency between the left and right images, achieving similar depth quality compared with supervised methods." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1804.00607" id="1286">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Li18:CVPR} presented a large depth dataset called MegaDepth, which is generated from collected multi-view internet photos, the hourglass model trained on this dataset shows good results on wild test images." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1412.1897" id="1287">
-        <attvalues>
-          <attvalue for="5" value=" Pure DNN-based methods usually have low robustness for scenarios which are not covered by training dataset \cite{Nguyen18:ECCV, Su18:ECCV} (as shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="1909.03375" target="1604.03901" id="1288">
-        <attvalues>
-          <attvalue for="5" value=" All of them employ the hourglass model \cite{Chen16:NIPS}, which has an encoder-decoder structure with skip connections." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="1809.03327" id="1289">
-        <attvalues>
-          <attvalue for="5" value=" Video Object Segmentation (VOS) focuses on tracking and segmenting target objects within a video sequence, beginning with mask annotations in the initial frame~\cite{ytvos2018, davis2017, feelvos, aot}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="2208.01924" id="1290">
-        <attvalues>
-          <attvalue for="5" value=" This technique has significant potential across various applications, especially given the increasing volume of video content in areas such as autonomous driving, augmented reality, and interactive video editing~\cite{perclip, bao2018cnnmrf}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="1904.00607" id="1291">
-        <attvalues>
-          <attvalue for="5" value=" VOS methods~\cite{STM, stcn} generally achieve video object segmentation by comparing the test frame with previous frames." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="2308.09903" id="1292">
-        <attvalues>
-          <attvalue for="5" value=" These correlated features are then used to predict the target masks accurately~\cite{SimVOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="2106.02638" id="1293">
-        <attvalues>
-          <attvalue for="5" value=" To handle variations in target appearances over time, certain methods~\cite{aot, xmem} incorporates a memory module that stores samples of the targets as their appearances change." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="2212.06826" id="1294">
-        <attvalues>
-          <attvalue for="5" value=" Second, although object queries~\cite{ISVOS, cutie} improve ID association accuracy, they perform poorly in sequences with significant target appearance changes and very small targets." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04600" target="2302.01872" id="1295">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the performance of current methods in realistic environments, The MOSE~\cite{mose} (coMplex video Object SEgmentation) dataset is a comprehensive collection designed to provide complex and challenging scenarios with occluded and crowded objects." />
-        </attvalues>
-      </edge>
-      <edge source="1908.08453" target="1807.03039" id="1296">
-        <attvalues>
-          <attvalue for="5" value=" In particular, based on the recent Glow architecture \cite{Kingma2018Glow:Convolutions}, we construct a normalizing flow model which is conditioned on critical variables, such as intensity, camera type, and gain settings (eg, ISO)." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1406.2283" id="1297">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2105.14021" id="1298">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2104.00877" id="1299">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1904.05822" id="1300">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1907.01341" id="1301">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2009.00743" id="1302">
-        <attvalues>
-          <attvalue for="5" value=" Supervised depth estimation approaches~\cite{Eigen2014,Miangoleh2021Boosting,chen2021s,GargDualPixelsICCV2019,Ranftl2020,lee2021patch,aich2021bidirectional} can predict dense depth maps but require costly ground truth depth labels." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1609.03677" id="1303">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, self-supervised approaches require no labelled data~\cite{lee2019real,multview,Godard17,tankovich2021hitnet,kendall2017end,zhang2018activestereonet,zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr} and are performing competitively." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2007.12140" id="1304">
-        <attvalues>
-          <attvalue for="5" value=" Previous works~\cite{choi2020safenet,tankovich2021hitnet,liebel2019multidepth,lu_icra} have shown that training a single depth model benefits from multiple regression or classification objectives." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1703.04309" id="1305">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, self-supervised approaches require no labelled data~\cite{lee2019real,multview,Godard17,tankovich2021hitnet,kendall2017end,zhang2018activestereonet,zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr} and are performing competitively." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1807.06009" id="1306">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, self-supervised approaches require no labelled data~\cite{lee2019real,multview,Godard17,tankovich2021hitnet,kendall2017end,zhang2018activestereonet,zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr} and are performing competitively." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1704.07813" id="1307">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, the models are trained within a standard self-supervised monocular depth estimation (SDE) framework~\cite{zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr,watson2021temporal}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1806.01260" id="1308">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, the models are trained within a standard self-supervised monocular depth estimation (SDE) framework~\cite{zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr,watson2021temporal}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2110.09482" id="1309">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, the models are trained within a standard self-supervised monocular depth estimation (SDE) framework~\cite{zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr,watson2021temporal}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2012.07356" id="1310">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by prior works that train a student depth network with a trained teacher network~\cite{lyu2020hr,poggi2020uncertainty}, we extend the single-task SDE framework to a multi-task setting by introducing a self-distillation scheme associated with a regression objective." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2104.14540" id="1311">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, the models are trained within a standard self-supervised monocular depth estimation (SDE) framework~\cite{zhou_sfmlearner,monodepth2,zhou2020constant,zhou_diffnet,lyu2020hr,watson2021temporal}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2010.02893" id="1312">
-        <attvalues>
-          <attvalue for="5" value=" Previous works~\cite{choi2020safenet,tankovich2021hitnet,liebel2019multidepth,lu_icra} have shown that training a single depth model benefits from multiple regression or classification objectives." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1907.11111" id="1313">
-        <attvalues>
-          <attvalue for="5" value=" Previous works~\cite{choi2020safenet,tankovich2021hitnet,liebel2019multidepth,lu_icra} have shown that training a single depth model benefits from multiple regression or classification objectives." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="2005.06209" id="1314">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by prior works that train a student depth network with a trained teacher network~\cite{lyu2020hr,poggi2020uncertainty}, we extend the single-task SDE framework to a multi-task setting by introducing a self-distillation scheme associated with a regression objective." />
-        </attvalues>
-      </edge>
-      <edge source="2111.09692" target="1705.07115" id="1315">
-        <attvalues>
-          <attvalue for="5" value=" Instead of manually tuning weights of loss terms, inspired by Kendall~\cite{kendall2018multi}, we propose two uncertainty modelling strategies to calculate uncertainty for the self-distillation task and the image reconstruction task respectively." />
-        </attvalues>
-      </edge>
-      <edge source="1910.00882" target="1607.02565" id="1316">
-        <attvalues>
-          <attvalue for="5" value=" In direct methods based on photometric consistency, the motion between different frames is estimated by minimizing the photometric error, as used in \cite{engel2014lsd,engel2018direct, newcombe2011dtam}." />
-        </attvalues>
-      </edge>
-      <edge source="1910.00882" target="1502.00956" id="1317">
-        <attvalues>
-          <attvalue for="5" value=" al combined both methods to realize real-time robust visual odometry in \cite{engel2018direct}; \cite{mur2015orb} uses geometric methods for the front-end of a SLAM system and the optimization method for the back-end." />
-        </attvalues>
-      </edge>
-      <edge source="1910.00882" target="1607.07405" id="1318">
-        <attvalues>
-          <attvalue for="5" value=" In addition, recently deep learning gained attention in pose estimation \cite{handa2016gvnn,costante2016exploring}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1603.06937" id="1319">
-        <attvalues>
-          <attvalue for="5" value=" The problem of multi-person pose estimation from images has been extensively studied in top-down paradigms~\cite{HRNet,SimpleBaseLine,SHG} that formulate the problem as single-person pose estimation with an off-the-shelf person detector with an impressively high AP (e." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1902.09212" id="1320">
-        <attvalues>
-          <attvalue for="5" value=" The vanilla center-offset regression method utilizes the HRNet-W32~\cite{HRNet} as the feature backbone to directly predict keypoints center heatmap and the offset vectors." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1804.06208" id="1321">
-        <attvalues>
-          <attvalue for="5" value="g, 56\% on the COCO-2017 validation dataset~\cite{SimpleBaseLine})." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1611.08050" id="1322">
-        <attvalues>
-          <attvalue for="5" value=" Bottom-up pose estimation approaches pay much attention on estimating pose parameters from any given multi-person image instead of using the cropped single-person ones, which poses several challenges on accurately identifying the learned bottom-level keypoints as person parts and grouping them into different individuals by learning part affinity fields~\cite{paf,openpose}, part association fields~\cite{pifpaf}, associative embedding~\cite{asso_embedding}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1812.08008" id="1323">
-        <attvalues>
-          <attvalue for="5" value=" Bottom-up pose estimation approaches pay much attention on estimating pose parameters from any given multi-person image instead of using the cropped single-person ones, which poses several challenges on accurately identifying the learned bottom-level keypoints as person parts and grouping them into different individuals by learning part affinity fields~\cite{paf,openpose}, part association fields~\cite{pifpaf}, associative embedding~\cite{asso_embedding}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1903.06593" id="1324">
-        <attvalues>
-          <attvalue for="5" value=" Bottom-up pose estimation approaches pay much attention on estimating pose parameters from any given multi-person image instead of using the cropped single-person ones, which poses several challenges on accurately identifying the learned bottom-level keypoints as person parts and grouping them into different individuals by learning part affinity fields~\cite{paf,openpose}, part association fields~\cite{pifpaf}, associative embedding~\cite{asso_embedding}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1611.05424" id="1325">
-        <attvalues>
-          <attvalue for="5" value=" Bottom-up pose estimation approaches pay much attention on estimating pose parameters from any given multi-person image instead of using the cropped single-person ones, which poses several challenges on accurately identifying the learned bottom-level keypoints as person parts and grouping them into different individuals by learning part affinity fields~\cite{paf,openpose}, part association fields~\cite{pifpaf}, associative embedding~\cite{asso_embedding}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1904.07850" id="1326">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many researchers attempted to learning center-offsets~\cite{centernet-zhou,CenterNet-duan,ART-pose,DEKR-pose} for pose estimation as its intrinsic simplicity and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="1904.08189" id="1327">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many researchers attempted to learning center-offsets~\cite{centernet-zhou,CenterNet-duan,ART-pose,DEKR-pose} for pose estimation as its intrinsic simplicity and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="2006.15480" id="1328">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many researchers attempted to learning center-offsets~\cite{centernet-zhou,CenterNet-duan,ART-pose,DEKR-pose} for pose estimation as its intrinsic simplicity and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2109.03622" target="2104.02300" id="1329">
-        <attvalues>
-          <attvalue for="5" value=" Recently, many researchers attempted to learning center-offsets~\cite{centernet-zhou,CenterNet-duan,ART-pose,DEKR-pose} for pose estimation as its intrinsic simplicity and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2303.07141" target="1405.0312" id="1330">
-        <attvalues>
-          <attvalue for="5" value=" Although many large-scale datasets \cite{lin2014microsoft,sun2020scalability,doering2022posetrack21} have been introduced in recent years, no dataset primarily targets robotic perception tasks in social navigation environments, and thus rarely reflects the specific challenges found in human-robot interaction and robot navigation in crowded human environments." />
-        </attvalues>
-      </edge>
-      <edge source="2303.07141" target="1912.04838" id="1331">
-        <attvalues>
-          <attvalue for="5" value=" Although many large-scale datasets \cite{lin2014microsoft,sun2020scalability,doering2022posetrack21} have been introduced in recent years, no dataset primarily targets robotic perception tasks in social navigation environments, and thus rarely reflects the specific challenges found in human-robot interaction and robot navigation in crowded human environments." />
-        </attvalues>
-      </edge>
-      <edge source="2303.07141" target="2210.11940" id="1332">
-        <attvalues>
-          <attvalue for="5" value=" The JRDB-Pose dataset \cite{vendrow2022jrdb} aims at bridging this gap and sets up a new benchmark for pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.07141" target="1902.09212" id="1333">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, we adopt a classical top-down method-HRNet \cite{sun2019deep} for human pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1609.04802" id="1334">
-        <attvalues>
-          <attvalue for="5" value=" With the emergence of deep learning, convolutional neural network (CNN) based SR methods (eg, SRCNN \cite{SRCNN}, SRGAN \cite{SRRESNET}, EDSR \cite{EDSR}, ESRGAN \cite{ESRGAN} and RCAN \cite{RCAN}) inherit the powerful capacity of deep learning and have achieved remarkable performance improvements." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1707.02921" id="1335">
-        <attvalues>
-          <attvalue for="5" value=" With the emergence of deep learning, convolutional neural network (CNN) based SR methods (eg, SRCNN \cite{SRCNN}, SRGAN \cite{SRRESNET}, EDSR \cite{EDSR}, ESRGAN \cite{ESRGAN} and RCAN \cite{RCAN}) inherit the powerful capacity of deep learning and have achieved remarkable performance improvements." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1809.00219" id="1336">
-        <attvalues>
-          <attvalue for="5" value=" With the emergence of deep learning, convolutional neural network (CNN) based SR methods (eg, SRCNN \cite{SRCNN}, SRGAN \cite{SRRESNET}, EDSR \cite{EDSR}, ESRGAN \cite{ESRGAN} and RCAN \cite{RCAN}) inherit the powerful capacity of deep learning and have achieved remarkable performance improvements." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1807.02758" id="1337">
-        <attvalues>
-          <attvalue for="5" value=" With the emergence of deep learning, convolutional neural network (CNN) based SR methods (eg, SRCNN \cite{SRCNN}, SRGAN \cite{SRRESNET}, EDSR \cite{EDSR}, ESRGAN \cite{ESRGAN} and RCAN \cite{RCAN}) inherit the powerful capacity of deep learning and have achieved remarkable performance improvements." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1904.03378" id="1338">
-        <attvalues>
-          <attvalue for="5" value=" To mitigate this issue, several real SR datasets have been recently built, City 100~\cite{city100} and SR-RAW~\cite{zoomlearn}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1905.05169" id="1339">
-        <attvalues>
-          <attvalue for="5" value=" To mitigate this issue, several real SR datasets have been recently built, City 100~\cite{city100} and SR-RAW~\cite{zoomlearn}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="1904.00523" id="1340">
-        <attvalues>
-          <attvalue for="5" value=" \cite{cai2019toward} released another real image SR dataset, named RealSR, which was captured from two DSLR cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2008.01928" id="1341">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, this challenge concerns the real-world SISR, which poses two challenging issues \cite{CDC}: (1) more complex degradation against bicubic downsampling, and (2) diverse degradation processes among devices, aiming to learn a generic model to super-resolve LR images captured in practical scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2009.12798" id="1342">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2010.01110" id="1343">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2011.04994" id="1344">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2011.04988" id="1345">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2009.06943" id="1346">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2009.12987" id="1347">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12072" target="2009.06290" id="1348">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation\cite{elhelou2020aim_relighting}, image extreme inpainting\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline\cite{ignatov2020aim_ISP}, rendering realistic bokeh\cite{ignatov2020aim_bokeh}, real image super-resolution\cite{wei2020aim_realSR}, efficient super-resolution\cite{zhang2020aim_efficientSR}, video temporal super-resolution\cite{son2020aim_VTSR} and video extreme super-resolution\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="1910.08131" id="1349">
-        <attvalues>
-          <attvalue for="5" value=" Recently, datasets limited to interior scenes~\cite{murmann2019dataset}, underexposed images enhanced by professionals~\cite{wang2019underexposed}, and rendered images with randomized light directions~\cite{xu2018deep} have been proposed, but none serve the benchmarking needs for image relighting, namely, having all $M\times N$ combinations of $M$ scenes and $N$ illumination settings." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="1705.01156" id="1350">
-        <attvalues>
-          <attvalue for="5" value=" For instance, IIW~\cite{bell2014intrinsic} and SAW~\cite{kovacs2017shading} contain human-labeled reflectance and shading annotations, and BigTime~\cite{li2018learning} contains time-lapse data of scenes illuminated under varying light conditions." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="1804.00582" id="1351">
-        <attvalues>
-          <attvalue for="5" value=" For instance, IIW~\cite{bell2014intrinsic} and SAW~\cite{kovacs2017shading} contain human-labeled reflectance and shading annotations, and BigTime~\cite{li2018learning} contains time-lapse data of scenes illuminated under varying light conditions." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="1905.00824" id="1352">
-        <attvalues>
-          <attvalue for="5" value=" Multiple methods are recently being developed for relighting~\cite{sun2019single,nagano2019deep,dherse2020scene}, and the prior literature on intrinsic images, which disentangle surface reflectance from lighting, is rich~\cite{weiss2001deriving,tappen2003recovering,finlayson2004intrinsic,shen2011intrinsic,barron2012color,bell2014intrinsic}, notably for applications such as relighting~\cite{bousseau2009user} and normalization~\cite{matsushita2004illumination}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2006.02333" id="1353">
-        <attvalues>
-          <attvalue for="5" value=" Multiple methods are recently being developed for relighting~\cite{sun2019single,nagano2019deep,dherse2020scene}, and the prior literature on intrinsic images, which disentangle surface reflectance from lighting, is rich~\cite{weiss2001deriving,tappen2003recovering,finlayson2004intrinsic,shen2011intrinsic,barron2012color,bell2014intrinsic}, notably for applications such as relighting~\cite{bousseau2009user} and normalization~\cite{matsushita2004illumination}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2001.10773" id="1354">
-        <attvalues>
-          <attvalue for="5" value=" Such virtual datasets have proven useful in multiple applications to augment even the training datasets containing real images, for instance the vKitti data~\cite{cabon2020virtual}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2009.12798" id="1355">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2010.01110" id="1356">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2011.04994" id="1357">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2011.04988" id="1358">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2009.06943" id="1359">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2009.12987" id="1360">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.12798" target="2009.06290" id="1361">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the AIM 2020 associated challenges on: scene relighting and illumination estimation~\cite{elhelou2020aim_relighting}, image extreme inpainting~\cite{ntavelis2020aim_inpainting}, learned image signal processing pipeline~\cite{ignatov2020aim_ISP}, rendering realistic bokeh~\cite{ignatov2020aim_bokeh}, real image super-resolution~\cite{wei2020aim_realSR}, efficient super-resolution~\cite{zhang2020aim_efficientSR}, video temporal super-resolution~\cite{son2020aim_VTSR} and video extreme super-resolution~\cite{fuoli2020aim_VXSR}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12327" target="2112.08936" id="1362">
-        <attvalues>
-          <attvalue for="5" value=" Running these models on this adverse data is a widely applicable problem, for example for self-driving cars, models that need to operate in geographical extremes, or models that must be highly robust~\cite{zhang2023perception}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.08919" target="1802.02611" id="1363">
-        <attvalues>
-          <attvalue for="5" value=" Since the dataset of the VIPriors challenges is a subset of cityscape \cite{cityscape}, several state-of-the-art semantic segmentation techniques are selected as the candidates, such as DeepLabV3+ \cite{deeplab}, Densely Connected Neural Architecture Search (DCNAS) \cite{dcans}, Object-Contextual Representation (OCR) \cite{hrnet}, Height-Driven Attention Networks (HANet) \cite{hanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.08919" target="1604.01685" id="1364">
-        <attvalues>
-          <attvalue for="5" value=" Since the dataset of the VIPriors challenges is a subset of cityscape \cite{cityscape}, several state-of-the-art semantic segmentation techniques are selected as the candidates, such as DeepLabV3+ \cite{deeplab}, Densely Connected Neural Architecture Search (DCNAS) \cite{dcans}, Object-Contextual Representation (OCR) \cite{hrnet}, Height-Driven Attention Networks (HANet) \cite{hanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.08919" target="2003.11883" id="1365">
-        <attvalues>
-          <attvalue for="5" value=" Since the dataset of the VIPriors challenges is a subset of cityscape \cite{cityscape}, several state-of-the-art semantic segmentation techniques are selected as the candidates, such as DeepLabV3+ \cite{deeplab}, Densely Connected Neural Architecture Search (DCNAS) \cite{dcans}, Object-Contextual Representation (OCR) \cite{hrnet}, Height-Driven Attention Networks (HANet) \cite{hanet}." />
-        </attvalues>
-      </edge>
-      <edge source="2007.08919" target="2003.05128" id="1366">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by HANet \cite{hanet}, we propose the edge-preserving loss to introduce more images prior to making the network can be well-trained on the limited dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2007.08919" target="2004.08955" id="1367">
-        <attvalues>
-          <attvalue for="5" value=" \item To have better feature representation powerful, we introduce the ResNeSt \cite{resnest} to the HANet as the backbone without dramatically increasing the number of parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2110.03249" target="1510.05970" id="1368">
-        <attvalues>
-          <attvalue for="5" value=" More recently, the focus of the community shifted towards supervised deep learning methods \cite{zbontar2016stereo, kendall2017end, zhang2019ga, chang2018pyramid, luo2016efficient, pang2017cascade, eigen2014depth, eigen2015predicting, fu2018deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02572" target="1912.04838" id="1369">
-        <attvalues>
-          <attvalue for="5" value=" Online Multiple Object Tracking (MOT) plays an important role in video surveillance~\cite{oh2011large}, autonomous driving~\cite{sun2020scalability}, and many other fields~\cite{zhang2020multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02572" target="1909.12605" id="1370">
-        <attvalues>
-          <attvalue for="5" value=" To mitigate the above issues, methods based on JDE paradigm are becoming prevalent, which learn both detection and re-ID embedding extraction in a shared model~\cite{wang2020towards}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02572" target="2004.01888" id="1371">
-        <attvalues>
-          <attvalue for="5" value=" In particular, FairMOT~\cite{zhang_wang_wang_zeng_liu_2021} achieves a good balance between tracking performance and efficiency by adopting a point-based framework." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02572" target="1809.04427" id="1372">
-        <attvalues>
-          <attvalue for="5" value=" It follows MOTDT~\cite{Chen_Ai_Zhuang_Shang_2018} and uses a hierarchical track-detection association method, which combines Kalman filter, re-ID embeddings, and Intersection-over-Union (IOU) metrics to facilitate the matching of tracks with detections." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02572" target="2201.01297" id="1373">
-        <attvalues>
-          <attvalue for="5" value=" OUTrack~\cite{liu_chen_chu_yuan_liu_zhang_yu_2022} optimizes the visual embedding by the unsupervised reID learning which does not require any (pseudo) identity information nor suffer from the scalability issue." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2308.02670" id="1374">
-        <attvalues>
-          <attvalue for="5" value=" Among these, RGB-D SLAM, which provides a depth map as a prior, is particularly useful for many applications such as AR/VR~\cite{edi_wang}, autonomous driving perception~\cite{feng2023robotic, feng2023subsurface, zhang2020manipulator, gao2024decentralized, gao2024adaptive}, 3D reconstruction~\cite{Chen_2023_CVPR, icra23_weihan, ral_jinglun}, novel view synthesis~\cite{Li_2024_CVPR} and navigation~\cite{icia_jinglun,ral_zheng} due to its potential to generate a dense 3D map." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2106.01907" id="1375">
-        <attvalues>
-          <attvalue for="5" value=" Among these, RGB-D SLAM, which provides a depth map as a prior, is particularly useful for many applications such as AR/VR~\cite{edi_wang}, autonomous driving perception~\cite{feng2023robotic, feng2023subsurface, zhang2020manipulator, gao2024decentralized, gao2024adaptive}, 3D reconstruction~\cite{Chen_2023_CVPR, icra23_weihan, ral_jinglun}, novel view synthesis~\cite{Li_2024_CVPR} and navigation~\cite{icia_jinglun,ral_zheng} due to its potential to generate a dense 3D map." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2304.00152" id="1376">
-        <attvalues>
-          <attvalue for="5" value=" Among these, RGB-D SLAM, which provides a depth map as a prior, is particularly useful for many applications such as AR/VR~\cite{edi_wang}, autonomous driving perception~\cite{feng2023robotic, feng2023subsurface, zhang2020manipulator, gao2024decentralized, gao2024adaptive}, 3D reconstruction~\cite{Chen_2023_CVPR, icra23_weihan, ral_jinglun}, novel view synthesis~\cite{Li_2024_CVPR} and navigation~\cite{icia_jinglun,ral_zheng} due to its potential to generate a dense 3D map." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2312.16812" id="1377">
-        <attvalues>
-          <attvalue for="5" value=" Among these, RGB-D SLAM, which provides a depth map as a prior, is particularly useful for many applications such as AR/VR~\cite{edi_wang}, autonomous driving perception~\cite{feng2023robotic, feng2023subsurface, zhang2020manipulator, gao2024decentralized, gao2024adaptive}, 3D reconstruction~\cite{Chen_2023_CVPR, icra23_weihan, ral_jinglun}, novel view synthesis~\cite{Li_2024_CVPR} and navigation~\cite{icia_jinglun,ral_zheng} due to its potential to generate a dense 3D map." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2203.03762" id="1378">
-        <attvalues>
-          <attvalue for="5" value=" Bayesian methods are important in SLAM for providing a framework of handling uncertainty and being able to make predictions from incomplete or noisy data~\cite{zhuang2022defending, zhuang2022robust}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2405.05526" id="1379">
-        <attvalues>
-          <attvalue for="5" value=" ~\cite{ming2024benchmarking} in NeRF-based SLAM systems." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2210.02657" id="1380">
-        <attvalues>
-          <attvalue for="5" value=" Edge computing ~\cite{li2023predictive, zong2022cocktail} involves processing data near the source of data generation, such as on the robot itself or on a nearby device in the network." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2308.04079" id="1381">
-        <attvalues>
-          <attvalue for="5" value=" In the realm of 3D generation ~\cite{tang2023dreamgaussian}, it also adapts 3D Gaussian splatting into generative settings for 3D content creation, significantly reducing the generation time of optimization-based 2D lifting methods In this paper, we aim to leverage the fast rendering performance and promising explicit 3D scene representation of 3DGS~\cite{kerbl3Dgaussians}, along with the accurate pose estimation strategy from Stereo-NEC~\cite{wang2024stereo}, to generate dense 3D reconstructions while maintaining accurate camera pose estimation~\cite{song2024eta}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2207.10606" id="1382">
-        <attvalues>
-          <attvalue for="5" value=" There have been many recent advancements and applications of 3DGS, such as in the capture of static scenes where 3DGS introduces an innovative differentiable renderer known as Fuzzy Metaballs ~\cite{keselman2022approximate}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2310.08528" id="1383">
-        <attvalues>
-          <attvalue for="5" value=" In various vision-related tasks, particularly dynamic scene capture ~\cite{wu20244d}, 3DGS constructs an efficient Gaussian splatting framework that accommodates changes in Gaussian motion and shape over time, significantly improving dynamic scene capture." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2309.16653" id="1384">
-        <attvalues>
-          <attvalue for="5" value=" In the realm of 3D generation ~\cite{tang2023dreamgaussian}, it also adapts 3D Gaussian splatting into generative settings for 3D content creation, significantly reducing the generation time of optimization-based 2D lifting methods In this paper, we aim to leverage the fast rendering performance and promising explicit 3D scene representation of 3DGS~\cite{kerbl3Dgaussians}, along with the accurate pose estimation strategy from Stereo-NEC~\cite{wang2024stereo}, to generate dense 3D reconstructions while maintaining accurate camera pose estimation~\cite{song2024eta}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.05635" target="2403.07225" id="1385">
-        <attvalues>
-          <attvalue for="5" value=" In the realm of 3D generation ~\cite{tang2023dreamgaussian}, it also adapts 3D Gaussian splatting into generative settings for 3D content creation, significantly reducing the generation time of optimization-based 2D lifting methods In this paper, we aim to leverage the fast rendering performance and promising explicit 3D scene representation of 3DGS~\cite{kerbl3Dgaussians}, along with the accurate pose estimation strategy from Stereo-NEC~\cite{wang2024stereo}, to generate dense 3D reconstructions while maintaining accurate camera pose estimation~\cite{song2024eta}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="1904.07734" id="1386">
-        <attvalues>
-          <attvalue for="5" value=" Incremental learning aims to address the challenge of catastrophic forgetting \cite{van2019three} that occurs when learning from dynamic data distributions." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2302.00487" id="1387">
-        <attvalues>
-          <attvalue for="5" value=" Typically, the process of learning new distributions leads to a reduction in the model's ability to retain knowledge of previously learned distributions \cite{wang2024comprehensive}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="1708.06977" id="1388">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="1904.00781" id="1389">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2012.15439" id="1390">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2204.02136" id="1391">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2307.12427" id="1392">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2312.09139" id="1393">
-        <attvalues>
-          <attvalue for="5" value=" Existing Incremental learning methods can be classified into distillation-based \cite{shmelkov2017incremental,li2019rilod,peng2021sid,feng2022overcoming}, parameter-restriction-based \cite{liu2023augmented}, and replay-based \cite{kim2024class, liucontinual}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="2402.17323" id="1394">
-        <attvalues>
-          <attvalue for="5" value=" Previous works \cite{yang2023pseudo,kim2024sddgr} introduce generative models to generate samples of absent classes, thereby ensuring knowledge transfer." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04829" target="1912.08795" id="1395">
-        <attvalues>
-          <attvalue for="5" value=" Compared with these methods, inversing \cite{yin2020dreaming} the detector to generate samples not only captures distinctive features but also eliminates the requirement for additional generative models, as shown in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2403.00592" id="1396">
-        <attvalues>
-          <attvalue for="5" value=" Understanding 3D scenes is critical for diverse applications, from autonomous vehicles navigating urban environments to robotic manipulation tasks~\cite{an2024rethinking,cai2023spsd}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="1810.04805" id="1397">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2005.14165" id="1398">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="1911.05722" id="1399">
-        <attvalues>
-          <attvalue for="5" value=" Based on this setting, we deploy MoCo~\cite{he2020momentum}, and BYOL~\cite{grill2020bootstrap} style CL with Point-MAE~\cite{PointMAE} to point cloud pre-training with ViTs structure." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2002.05709" id="1400">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2011.10566" id="1401">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2103.00020" id="1402">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2112.02413" id="1403">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2102.05918" id="1404">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning (SSL) has emerged as a key solution, the effectiveness of which has already been validated in natural language processing~\cite{devlin2018bert,radford2018improving,brown2020language}, computer vision~\cite{he2020momentum,chen2020simple,chen2021exploring,zhao2024denoising}, and multimodal learning~\cite{radford2021learning,zhang2022pointclip,jia2021scaling}, enabling informative feature representations from unlabeled input data." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2111.14819" id="1405">
-        <attvalues>
-          <attvalue for="5" value=" As a result, it is widely utilized in pre-training for 3D point clouds, demonstrating comparative or even better performance compared to its supervised counterpart in downstream tasks such as classification, segmentation, and detection~\cite{PointBERT, MaskPoint,qi2023contrast,ren2023masked,liu2020grouped}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2302.02318" id="1406">
-        <attvalues>
-          <attvalue for="5" value=" As a result, it is widely utilized in pre-training for 3D point clouds, demonstrating comparative or even better performance compared to its supervised counterpart in downstream tasks such as classification, segmentation, and detection~\cite{PointBERT, MaskPoint,qi2023contrast,ren2023masked,liu2020grouped}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2205.12551" id="1407">
-        <attvalues>
-          <attvalue for="5" value=" As a result, it is widely utilized in pre-training for 3D point clouds, demonstrating comparative or even better performance compared to its supervised counterpart in downstream tasks such as classification, segmentation, and detection~\cite{PointBERT, MaskPoint,qi2023contrast,ren2023masked,liu2020grouped}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2203.11183" id="1408">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, using MAE with a Chamfer constraint for point cloud reconstruction can lead to sub-optimal solutions~\cite{MaskPoint}, as ground truth points are sampled from the original point clouds, causing inconsistencies when forcing predicted points to match them." />
-        </attvalues>
-      </edge>
-      <edge source="2407.05862" target="2203.06604" id="1409">
-        <attvalues>
-          <attvalue for="5" value=" Based on this setting, we deploy MoCo~\cite{he2020momentum}, and BYOL~\cite{grill2020bootstrap} style CL with Point-MAE~\cite{PointMAE} to point cloud pre-training with ViTs structure." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01606" target="1804.02142" id="1410">
-        <attvalues>
-          <attvalue for="5" value=" Recently, spectral clustering based methods have shown remarkable results in segmenting motions in challenging dynamic environment containing significant motion degeneracy and complex scene structures \cite{xu_motion_2018, jiang_what_2021, xi_multi-motion_2022, lin_multi-motion_2023, huang_motion_2023}, largely thanks to its ability of synergetically fusing multiple types of motion cues together." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2306.15880" id="1411">
-        <attvalues>
-          <attvalue for="5" value=" Following zero-shot settings, OVD models are trained on base categories but require predicting both base and novel categories within a large-scale language vocabulary~\cite{survey1}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2103.00020" id="1412">
-        <attvalues>
-          <attvalue for="5" value=" Since the Cap4M dataset~\cite{radford2021learning} used by Grounding-DINO is not open-source, we have opted for the GRIT and V3Det datasets as substitutes in our study." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2308.16182" id="1413">
-        <attvalues>
-          <attvalue for="5" value=" The primary aim of REC task is to accurately identify the target designated by a given textual description and subsequently demarcate its position utilizing a bounding box~\cite{he2023grec}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2303.05499" id="1414">
-        <attvalues>
-          <attvalue for="5" value=" Among these grounding models, Grounding-DINO~\cite{liu2023grounding} has been made as a main-stream architecture with superior performance." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2203.03605" id="1415">
-        <attvalues>
-          <attvalue for="5" value=" Based on a closed-set detector DINO~\cite{zhang2022dino}, Grounding-DINO-Large achieves state-of-the-art zero-shot performance on COCO~\cite{lin2015microsoft}(mAP 52." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1405.0312" id="1416">
-        <attvalues>
-          <attvalue for="5" value=" Based on a closed-set detector DINO~\cite{zhang2022dino}, Grounding-DINO-Large achieves state-of-the-art zero-shot performance on COCO~\cite{lin2015microsoft}(mAP 52." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1906.07155" id="1417">
-        <attvalues>
-          <attvalue for="5" value=" Given that Grounding-DINO has demonstrated superior precision across the aforementioned three downstream tasks, yet is not entirely open-source (with only test and demo codes available), we rebuild the Grounding-DINO model utilizing the MMDetection toolbox~\cite{mmdetection} within the OpenMMLab project, adhering to the official test codes of Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2306.14824" id="1418">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2304.03752" id="1419">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1608.00272" id="1420">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1511.02283" id="1421">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1902.09506" id="1422">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1505.04870" id="1423">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2104.12763" id="1424">
-        <attvalues>
-          <attvalue for="5" value=" Based on the Grounding-DINO framework, we propose to apply more datasets for pretraining, including COCO, Objects365~\cite{9009553}, GRIT~\cite{peng2023kosmos2}, V3Det~\cite{wang2023v3det}, RefCOCO~\cite{kazemzadeh-etal-2014-referitgame}, RefCOCO+~\cite{yu2016modeling}, RefCOCOg~\cite{mao2016generation}, GQA~\cite{hudson2018gqa} / Flickr30k Entities~\cite{flickrentitiesijcv}(combination also named as Golden-G dataset~\cite{kamath2021mdetr}), results in a stronger Grounding-DINO-based model which we call MM-Grounding-DINO." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="1908.03195" id="1425">
-        <attvalues>
-          <attvalue for="5" value=" We further extend all available benchmarks for OVD, PG and REC evaluation, including COCO, LVIS~\cite{gupta2019lvis}, RefCOCO/+/g, Flickr30k Entities, ODinW13/35~\cite{li2022elevater}, gRefCOCO~\cite{liu2023gres} and Description Detection Dataset($D^3$) ~\cite{xie2023described}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2204.08790" id="1426">
-        <attvalues>
-          <attvalue for="5" value=" We further extend all available benchmarks for OVD, PG and REC evaluation, including COCO, LVIS~\cite{gupta2019lvis}, RefCOCO/+/g, Flickr30k Entities, ODinW13/35~\cite{li2022elevater}, gRefCOCO~\cite{liu2023gres} and Description Detection Dataset($D^3$) ~\cite{xie2023described}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2306.00968" id="1427">
-        <attvalues>
-          <attvalue for="5" value=" We further extend all available benchmarks for OVD, PG and REC evaluation, including COCO, LVIS~\cite{gupta2019lvis}, RefCOCO/+/g, Flickr30k Entities, ODinW13/35~\cite{li2022elevater}, gRefCOCO~\cite{liu2023gres} and Description Detection Dataset($D^3$) ~\cite{xie2023described}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.02361" target="2307.12813" id="1428">
-        <attvalues>
-          <attvalue for="5" value=" We further extend all available benchmarks for OVD, PG and REC evaluation, including COCO, LVIS~\cite{gupta2019lvis}, RefCOCO/+/g, Flickr30k Entities, ODinW13/35~\cite{li2022elevater}, gRefCOCO~\cite{liu2023gres} and Description Detection Dataset($D^3$) ~\cite{xie2023described}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.01307" target="1704.08141" id="1429">
-        <attvalues>
-          <attvalue for="5" value=" Due to the requirements for communicating feature descriptors, coding standards like compact descriptors for visual search (CDVS)~\cite{CDVS} and compact descriptors for video analysis (CDVA)~\cite{CDVA} were developed." />
-        </attvalues>
-      </edge>
-      <edge source="2102.01307" target="2001.03569" id="1430">
-        <attvalues>
-          <attvalue for="5" value=" Such collaborative coding problem is known as video coding for machines (VCM)~\cite{VCM}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.01307" target="2001.02915" id="1431">
-        <attvalues>
-          <attvalue for="5" value=" Addition to this, (ii) it is possible to reconstruct a full-resolution frame from the obtained feature descriptors alone; unlike the approach in~\cite{VCM-image-coding} that require additional reference pixels along with edge based feature descriptors in this regard." />
-        </attvalues>
-      </edge>
-      <edge source="2102.01307" target="1611.03718" id="1432">
-        <attvalues>
-          <attvalue for="5" value=" For instance, in~\cite{hierarchical} a method is proposed for performing hierarchical object detection in images through deep reinforcement learning~\cite{mnih2015}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.01307" target="1801.02298" id="1433">
-        <attvalues>
-          <attvalue for="5" value=" It was shown that cuboids have excellent global commonality modeling capabilities, are object centric and computationally efficient~\cite{cuboid-dicta2,cuboid-csvt,ashek-icassp,ashek-icassp2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1711.01589" id="1434">
-        <attvalues>
-          <attvalue for="5" value=" The first category deals with the time series of joints and extracts features, such as Dynamic Time Warping (DTW), from these time series \cite{ghodsi2018simultaneous}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1612.09401" id="1435">
-        <attvalues>
-          <attvalue for="5" value=" Some of the action recognition methods are based on convolutional networks \cite{ji20123d,wang2018action}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1906.02590" id="1436">
-        <attvalues>
-          <attvalue for="5" value=" For example, histograms of 3D joint locations (HOJ3D) utilizes Linear Discriminant Analysis (LDA) which is equivalent to Fisher Discriminant Analysis (FDA) \cite{ghojogh2019linear}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1708.06227" id="1437">
-        <attvalues>
-          <attvalue for="5" value=" LDA is also used in \cite{mokari2020recognizing} for recognizing involuntary actions." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1906.09436" id="1438">
-        <attvalues>
-          <attvalue for="5" value=" Some examples of these methods are Principal Component Analysis (PCA) \cite{jolliffe2011principal,ghojogh2019unsupervised}, Supervised PCA (SPCA) \cite{barshan2011supervised}, and FDA \cite{ghojogh2019fisher}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1903.11240" id="1439">
-        <attvalues>
-          <attvalue for="5" value=" This method is based on basic subspace learning approaches which make use of generalized eigenvalue problem \cite{ghojogh2019eigenvalue}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1906.03148" id="1440">
-        <attvalues>
-          <attvalue for="5" value=" Some examples are eigenfaces \cite{turk1991face}, Fisherfaces \cite{belhumeur1997eigenfaces}, supervised eigenfaces \cite{barshan2011supervised,ghojogh2019unsupervised}, and double supervised eigenfaces \cite{ghojogh2020roweis}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.15736" target="1910.05437" id="1441">
-        <attvalues>
-          <attvalue for="5" value=" Roweis Discriminant Analysis (RDA) \cite{ghojogh2019roweis,ghojogh2020roweis} is a family of infinite number of subspace learning methods based on generalized eigenvalue problem." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1606.00915" id="1442">
-        <attvalues>
-          <attvalue for="5" value=" D{eep} Convolutional Neural Networks (DCNNs) excel at a wide range of image recognition tasks~\cite{7780459, 7485869, Shelhamer2017}, such as semantic segmentation~\cite{7913730, Shelhamer2017, zhao2017pspnet} and panoptic segmentation~\cite{2018arXiv180100868K,2019arXiv190205093Y, 2019arXiv190103784X, 2019arXiv190102446K}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1801.00868" id="1443">
-        <attvalues>
-          <attvalue for="5" value=" D{eep} Convolutional Neural Networks (DCNNs) excel at a wide range of image recognition tasks~\cite{7780459, 7485869, Shelhamer2017}, such as semantic segmentation~\cite{7913730, Shelhamer2017, zhao2017pspnet} and panoptic segmentation~\cite{2018arXiv180100868K,2019arXiv190205093Y, 2019arXiv190103784X, 2019arXiv190102446K}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1902.05093" id="1444">
-        <attvalues>
-          <attvalue for="5" value=" D{eep} Convolutional Neural Networks (DCNNs) excel at a wide range of image recognition tasks~\cite{7780459, 7485869, Shelhamer2017}, such as semantic segmentation~\cite{7913730, Shelhamer2017, zhao2017pspnet} and panoptic segmentation~\cite{2018arXiv180100868K,2019arXiv190205093Y, 2019arXiv190103784X, 2019arXiv190102446K}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1712.04837" id="1445">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation studies the tasks of assigning a class label to each pixel of an image, while instance segmentation~\cite{Chen2018MaskLabIS} detects and segment each object instance." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1604.01685" id="1446">
-        <attvalues>
-          <attvalue for="5" value="5h on average” on the Cityscapes dataset~\cite{Cordts2016Cityscapes}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1409.0575" id="1447">
-        <attvalues>
-          <attvalue for="5" value=" To investigate the general purpose of low level DCNN features, we compare DCNN trained on ImageNet~\cite{ILSVRC15} and COCO~\cite{10." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1604.05144" id="1448">
-        <attvalues>
-          <attvalue for="5" value=" Second, they can be used inside any weakly or semi-supervised learning framework for semantic (and panoptic) segmentation, which only requires initial scribbles~\cite{Lin2016ScribbleSupSC}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1711.09081" id="1449">
-        <attvalues>
-          <attvalue for="5" value=" Recently, $4$ extreme points~\cite{Man+18, extremeclick} clicking is used as an alternative which shows superior result." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1803.09693" id="1450">
-        <attvalues>
-          <attvalue for="5" value=" Modern annotation tools often adopt interactive deep learning based methods, including Polygon-RNN++~\cite{acuna2018efficient} and Curve-GCN~\cite{2019arXiv190306874L}, which allows the annotator to click on the boundaries." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1903.06874" id="1451">
-        <attvalues>
-          <attvalue for="5" value=" Modern annotation tools often adopt interactive deep learning based methods, including Polygon-RNN++~\cite{acuna2018efficient} and Curve-GCN~\cite{2019arXiv190306874L}, which allows the annotator to click on the boundaries." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1812.01888" id="1452">
-        <attvalues>
-          <attvalue for="5" value=" Notably, a recent approach~\cite{agustsson2019interactive} enables both $4$ extreme points clicking and scribbles correction." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1503.01640" id="1453">
-        <attvalues>
-          <attvalue for="5" value=" ~\cite{Dai2015BoxSupEB, khoreva_CVPR17, Li_2018_ECCV} claim that weakly iteratively trained by just bounding boxes and image tags, the DCNN can achieve $95\%$ segmentation score compared to fully supervised on VOC." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1803.07351" id="1454">
-        <attvalues>
-          <attvalue for="5" value=" The authors of~\cite{DBLP-1803-07351} formulate the problem as an ILP and try to solve the global optimum, but only to a reduced image size." />
-        </attvalues>
-      </edge>
-      <edge source="2212.06756" target="1606.06135" id="1455">
-        <attvalues>
-          <attvalue for="5" value=" MRF with connectivity prior has been studied in~\cite{Rempfler2016TheMC, 5828, 4587440} and formulated as an ILP, which can be solved by any ILP solver~\cite{doi:10." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01187" target="2303.15935" id="1456">
-        <attvalues>
-          <attvalue for="5" value=" Recent advancement in large pre-trained models \cite{liu2023summary,zhao2023brain} inspired a series of developments of foundational models in computer vision \cite{willemink2022toward,zhou2023comprehensive}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01187" target="2302.09419" id="1457">
-        <attvalues>
-          <attvalue for="5" value=" Recent advancement in large pre-trained models \cite{liu2023summary,zhao2023brain} inspired a series of developments of foundational models in computer vision \cite{willemink2022toward,zhou2023comprehensive}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01187" target="2304.02643" id="1458">
-        <attvalues>
-          <attvalue for="5" value=" However, simple prompts such as a single point will lead to ambiguities in the SAM model, which is also discussed in the original paper of SAM \cite{kirillov2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.01187" target="2211.16327" id="1459">
-        <attvalues>
-          <attvalue for="5" value=" SAM represents the paradigm shift in artificial intelligence towards large-scale, general-purposed &quot;foundation&quot; models \cite{yuan2023power,zhang2023biomedgpt}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.03431" target="2009.13077" id="1460">
-        <attvalues>
-          <attvalue for="5" value=" Recent developed methods \cite{wang2020DMCount, wang2021uniformity, song2021rethinking} and datasets \cite{7780439, 6619173, 10." />
-        </attvalues>
-      </edge>
-      <edge source="2206.03431" target="1704.03593" id="1461">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, in addition to semantic scene understanding \cite{le2018segmentation} and video temporal modeling \cite{duong2019learning, duong2029cvpr_automatic, Quach_2021_CVPR, Truong_2022_CVPR}, some self-training methods appear to utilize existing datasets with labels, eg source domain, and perform counting on more open-set scenarios, eg target domain, \cite{8578662,liu2020unsupervised} by transfer learning and domain adaptation techniques." />
-        </attvalues>
-      </edge>
-      <edge source="2206.03431" target="2008.05383" id="1462">
-        <attvalues>
-          <attvalue for="5" value=" \cite{liu2020unsupervised} enable knowledge distillation between both regression-based and detection-based models by formulating the mutual transformation of outputs." />
-        </attvalues>
-      </edge>
-      <edge source="2206.03431" target="1811.12833" id="1463">
-        <attvalues>
-          <attvalue for="5" value=" While general self learning methods improve the generalization capability by attempting to estimate pseudo ground-truths or distillation learning from a teacher network, a few approaches investigate a new direction to narrow the domain shift from entropy feedback of the target domain, especially in the semantic segmentation task \cite{vu2018advent}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1506.01497" id="1464">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1712.00726" id="1465">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1506.02640" id="1466">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1708.02002" id="1467">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1912.02424" id="1468">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="2006.02334" id="1469">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="2103.09460" id="1470">
-        <attvalues>
-          <attvalue for="5" value=" In the last decades, there are many great progresses in object detection based on the CNN~\cite{fastercnn,cascadercnn,yolo,retinanet,atss,detectors,yolof}." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="2005.12872" id="1471">
-        <attvalues>
-          <attvalue for="5" value="~\cite{detr} propose the DETR which is a new paradigm of object detection based on the transformer." />
-        </attvalues>
-      </edge>
-      <edge source="2109.07107" target="1512.03385" id="1472">
-        <attvalues>
-          <attvalue for="5" value="2 AP with 19 FPS when using a single ResNet50-DC5~\cite{resnet} feature for training 50 epochs." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="1409.1556" id="1473">
-        <attvalues>
-          <attvalue for="5" value=" State-of-the-art solutions have shown impressive results in real-world settings (see \ref{sec:literature}) with their remarkable performance heavily relying on advances in deep learning architectures \cite{simonyan2014very,he2016deep} and the availability of large datasets for training." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="1512.03385" id="1474">
-        <attvalues>
-          <attvalue for="5" value=" State-of-the-art solutions have shown impressive results in real-world settings (see \ref{sec:literature}) with their remarkable performance heavily relying on advances in deep learning architectures \cite{simonyan2014very,he2016deep} and the availability of large datasets for training." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="1705.05301" id="1475">
-        <attvalues>
-          <attvalue for="5" value=" However, the constancy of the appearance across images has been exploited in the context of hand or human pose estimation in recent works~\cite{panteleris2017back,Moon_2020_ECCV_DeepHandMesh,bogo2014faust}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="2008.08213" id="1476">
-        <attvalues>
-          <attvalue for="5" value=" However, the constancy of the appearance across images has been exploited in the context of hand or human pose estimation in recent works~\cite{panteleris2017back,Moon_2020_ECCV_DeepHandMesh,bogo2014faust}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="2007.08501" id="1477">
-        <attvalues>
-          <attvalue for="5" value=" Most rendering components invoked in the literature can be mapped to Pytorch3D~\cite{ravi2020pytorch3d}, which implements Neural Rendering~\cite{kato2018renderer} and Soft Rasterization~\cite{liu2020general}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.05509" target="1711.07566" id="1478">
-        <attvalues>
-          <attvalue for="5" value=" Most rendering components invoked in the literature can be mapped to Pytorch3D~\cite{ravi2020pytorch3d}, which implements Neural Rendering~\cite{kato2018renderer} and Soft Rasterization~\cite{liu2020general}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04323" target="2106.09672" id="1479">
-        <attvalues>
-          <attvalue for="5" value=" Accordingly, attention to research on copy detection, which is a task to determine whether an image is a modified copy of any image in a database, has increased~\cite{ISC2021,Jiang2014VCDBALCopy,Christlein2012AnEOCopy}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04323" target="1208.3665" id="1480">
-        <attvalues>
-          <attvalue for="5" value=" Accordingly, attention to research on copy detection, which is a task to determine whether an image is a modified copy of any image in a database, has increased~\cite{ISC2021,Jiang2014VCDBALCopy,Christlein2012AnEOCopy}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.04323" target="2104.00298" id="1481">
-        <attvalues>
-          <attvalue for="5" value=" Our training pipeline has multiple steps inspired by progressive learning~\cite{EfficientNetV2}, which is a model training technique that increases input image resolution and regularization as a training step proceeds." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2401.11734" id="1482">
-        <attvalues>
-          <attvalue for="5" value="6 million deaths per year (an increase of 73\%)~\cite{bernal2012towards,wu2024colorectal}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2401.13560" id="1483">
-        <attvalues>
-          <attvalue for="5" value=" Colonoscopy plays a crucial role in detecting polyps and serves as a screening tool for CRC~\cite{xing2024segmamba,xing2022nestedformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2208.14876" id="1484">
-        <attvalues>
-          <attvalue for="5" value=" Colonoscopy plays a crucial role in detecting polyps and serves as a screening tool for CRC~\cite{xing2024segmamba,xing2022nestedformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2105.08468" id="1485">
-        <attvalues>
-          <attvalue for="5" value=" Recently, emerging methods have utilized deep learning techniques to automatically detect polyps in video, from popular convolutional neural networks to attention mechanism~\cite{tajbakhsh2015automated,puyal2020endoscopic,ji2021progressively,ji2022video,yang2023mammodg,shaharabany2023autosam,ali2024objective,yang2024vivim}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2203.14291" id="1486">
-        <attvalues>
-          <attvalue for="5" value=" Polyps can be diagnosed into different categories (hyperplastic polyp, invasive carcinoma, \etc) depending on the size and morphological features~\cite{ji2022video}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2308.01057" id="1487">
-        <attvalues>
-          <attvalue for="5" value=" Recently, emerging methods have utilized deep learning techniques to automatically detect polyps in video, from popular convolutional neural networks to attention mechanism~\cite{tajbakhsh2015automated,puyal2020endoscopic,ji2021progressively,ji2022video,yang2023mammodg,shaharabany2023autosam,ali2024objective,yang2024vivim}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2306.06370" id="1488">
-        <attvalues>
-          <attvalue for="5" value=" Recently, emerging methods have utilized deep learning techniques to automatically detect polyps in video, from popular convolutional neural networks to attention mechanism~\cite{tajbakhsh2015automated,puyal2020endoscopic,ji2021progressively,ji2022video,yang2023mammodg,shaharabany2023autosam,ali2024objective,yang2024vivim}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2401.15753" id="1489">
-        <attvalues>
-          <attvalue for="5" value=" Recently, emerging methods have utilized deep learning techniques to automatically detect polyps in video, from popular convolutional neural networks to attention mechanism~\cite{tajbakhsh2015automated,puyal2020endoscopic,ji2021progressively,ji2022video,yang2023mammodg,shaharabany2023autosam,ali2024objective,yang2024vivim}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2201.11793" id="1490">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have shown impressive performance in many vision tasks, such as image deblurring~\cite{kawar2022denoising}, super-resolution~\cite{li2022srdiff,gao2023implicit}, and anomaly detection~\cite{wyatt2022anoddpm,chen2023diffusiondet}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2104.14951" id="1491">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have shown impressive performance in many vision tasks, such as image deblurring~\cite{kawar2022denoising}, super-resolution~\cite{li2022srdiff,gao2023implicit}, and anomaly detection~\cite{wyatt2022anoddpm,chen2023diffusiondet}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2303.16491" id="1492">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have shown impressive performance in many vision tasks, such as image deblurring~\cite{kawar2022denoising}, super-resolution~\cite{li2022srdiff,gao2023implicit}, and anomaly detection~\cite{wyatt2022anoddpm,chen2023diffusiondet}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2211.09788" id="1493">
-        <attvalues>
-          <attvalue for="5" value=" Diffusion models have shown impressive performance in many vision tasks, such as image deblurring~\cite{kawar2022denoising}, super-resolution~\cite{li2022srdiff,gao2023implicit}, and anomaly detection~\cite{wyatt2022anoddpm,chen2023diffusiondet}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2006.11239" id="1494">
-        <attvalues>
-          <attvalue for="5" value=" The denoising diffusion probabilistic model (DDPM)~\cite{ho2020denoising} is a generative model based on a Markov chain." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2306.09004" id="1495">
-        <attvalues>
-          <attvalue for="5" value=" By representing the diffusion process of image intensity values over successive iterations, diffusion models adeptly grasp the intrinsic structure and texture within images, facilitating the distinct segregation of regions of interest from the background~\cite{amit2023annotator}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2211.00611" id="1496">
-        <attvalues>
-          <attvalue for="5" value=" While several pioneers~\cite{wu2024medsegdiff,xing2023diff,wolleb2022diffusion,yang2023diffmic} applied DDPM to medical image classification and segmentation, there is no diffusion-based work on medical video lesion segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2303.10326" id="1497">
-        <attvalues>
-          <attvalue for="5" value=" While several pioneers~\cite{wu2024medsegdiff,xing2023diff,wolleb2022diffusion,yang2023diffmic} applied DDPM to medical image classification and segmentation, there is no diffusion-based work on medical video lesion segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2112.03145" id="1498">
-        <attvalues>
-          <attvalue for="5" value=" While several pioneers~\cite{wu2024medsegdiff,xing2023diff,wolleb2022diffusion,yang2023diffmic} applied DDPM to medical image classification and segmentation, there is no diffusion-based work on medical video lesion segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2303.10610" id="1499">
-        <attvalues>
-          <attvalue for="5" value=" While several pioneers~\cite{wu2024medsegdiff,xing2023diff,wolleb2022diffusion,yang2023diffmic} applied DDPM to medical image classification and segmentation, there is no diffusion-based work on medical video lesion segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2409.07238" target="2303.17559" id="1500">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by the nature of diffusion models~\cite{ji2023ddp}, in this paper, 1) we present the first diffusion network, dubbed as Diff-VPS, for the VPS task." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2307.09481" id="1501">
-        <attvalues>
-          <attvalue for="5" value=" Image synthesis~\cite{anydoor,DiffUTE} aims to synthesize foreground objects from one image into another image to produce realistic, visually consistent synthesized images." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2305.10825" id="1502">
-        <attvalues>
-          <attvalue for="5" value=" Image synthesis~\cite{anydoor,DiffUTE} aims to synthesize foreground objects from one image into another image to produce realistic, visually consistent synthesized images." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2106.02853" id="1503">
-        <attvalues>
-          <attvalue for="5" value=" While traditional image harmonization methods~\cite{Mat2016} are mainly based on low-level color and statistical illumination information, the development of deep learning techniques has made deep image harmonization methods mainstream and achieve better results~\cite{RAIN,Bargainnet,MM23_HDNet,DCCF}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2009.09169" id="1504">
-        <attvalues>
-          <attvalue for="5" value=" While traditional image harmonization methods~\cite{Mat2016} are mainly based on low-level color and statistical illumination information, the development of deep learning techniques has made deep image harmonization methods mainstream and achieve better results~\cite{RAIN,Bargainnet,MM23_HDNet,DCCF}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2211.08639" id="1505">
-        <attvalues>
-          <attvalue for="5" value=" While traditional image harmonization methods~\cite{Mat2016} are mainly based on low-level color and statistical illumination information, the development of deep learning techniques has made deep image harmonization methods mainstream and achieve better results~\cite{RAIN,Bargainnet,MM23_HDNet,DCCF}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2207.04788" id="1506">
-        <attvalues>
-          <attvalue for="5" value=" While traditional image harmonization methods~\cite{Mat2016} are mainly based on low-level color and statistical illumination information, the development of deep learning techniques has made deep image harmonization methods mainstream and achieve better results~\cite{RAIN,Bargainnet,MM23_HDNet,DCCF}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="1703.00069" id="1507">
-        <attvalues>
-          <attvalue for="5" value=" The former employs an encoder-decoder framework to learn pixel-to-pixel translation~\cite{DIH,SSAM}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="1907.06406" id="1508">
-        <attvalues>
-          <attvalue for="5" value=" The former employs an encoder-decoder framework to learn pixel-to-pixel translation~\cite{DIH,SSAM}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12729" target="2304.02643" id="1509">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, to avoid training an additional segmentation network, we leverage the powerful Segment Anything Model (SAM)~\cite{SAM} to obtain semantic priors." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12080" target="2207.10269" id="1510">
-        <attvalues>
-          <attvalue for="5" value=" We show that {GenCrop} yields competitive results against fully supervised approaches~\cite{hcic} on the existing datasets~\cite{flms,fcdb,sacd} (under quantitative metrics such as Intersection-over-Union and boundary displacement~\cite{hcic}), while being superior to the best weakly/unsupervised method~\cite{vfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12080" target="1904.04441" id="1511">
-        <attvalues>
-          <attvalue for="5" value=" High-quality solutions to this problem are based on supervised learning, from large datasets of manual annotations created specifically for cropping~\cite{gaic,cpc,sacd}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12080" target="1701.01480" id="1512">
-        <attvalues>
-          <attvalue for="5" value=" We show that {GenCrop} yields competitive results against fully supervised approaches~\cite{hcic} on the existing datasets~\cite{flms,fcdb,sacd} (under quantitative metrics such as Intersection-over-Union and boundary displacement~\cite{hcic}), while being superior to the best weakly/unsupervised method~\cite{vfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2312.12080" target="1702.00503" id="1513">
-        <attvalues>
-          <attvalue for="5" value=" We show that {GenCrop} yields competitive results against fully supervised approaches~\cite{hcic} on the existing datasets~\cite{flms,fcdb,sacd} (under quantitative metrics such as Intersection-over-Union and boundary displacement~\cite{hcic}), while being superior to the best weakly/unsupervised method~\cite{vfn}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01300" target="2403.15209" id="1514">
-        <attvalues>
-          <attvalue for="5" value=" Multispectral pedestrian detection plays a critical role in many real-world applications that require both day/night operations, such as smart surveillance cameras (CCTVs), search and rescue (SAR) autopilots, and autonomous vehicles (AVs)~\cite{kim2022defending, hwang2015multispectral,choi2018kaist, c:25,gonzalez2016pedestrian, 9419080, cao2021handcrafted, qingyun2022crossmodality, zhou2020improving, kim2022map, zhang2019weakly,kim2024mscotdet}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01300" target="2006.04315" id="1515">
-        <attvalues>
-          <attvalue for="5" value=" Despite its notable progress, an overlooked factor in multispectral pedestrian detection is the modality bias problem that occurs in multimodal models (eg, Visual Question Answering) ~\cite{niu2021counterfactual, cadene2019rubi,zhang2023layout, kv2020reducing, wen2021debiased}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01300" target="2103.03493" id="1516">
-        <attvalues>
-          <attvalue for="5" value=" Multimodal models are known to often leverage spurious correlations between a certain modality and answers due to statistical biases in datasets~\cite{yang2021causal, zang2023discovering, nan2021interventional, liu2022show, huang2022deconfounded}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01300" target="1810.03649" id="1517">
-        <attvalues>
-          <attvalue for="5" value=" For example, a Visual Question Answering (VQA) model may blindly answer “tennis” for the question “What sports?” just referring to the most co-occurring textual QA pairs (eg, linguistic bias) in the train data~\cite{ramakrishnan2018overcoming, niu2021counterfactual}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.01300" target="2202.05306" id="1518">
-        <attvalues>
-          <attvalue for="5" value=" Models exploiting statistical bias in datasets often demonstrate poor generalization ability to out-of-distribution data, rarely providing proper multimodal evidence for prediction~\cite{wu2022characterizing, gat2020removing}." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1406.4216" id="1519">
-        <attvalues>
-          <attvalue for="5" value=" Therefore distance metric learning \cite{PRDC,LFDA:CVPR,SSSVM,rPcca,KISSME,LOMO,MLAPG,Zheng:nfst,NK3ML} is used as a second stage in person re-ID systems to improve the discrimination." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1603.02139" id="1520">
-        <attvalues>
-          <attvalue for="5" value=" Therefore distance metric learning \cite{PRDC,LFDA:CVPR,SSSVM,rPcca,KISSME,LOMO,MLAPG,Zheng:nfst,NK3ML} is used as a second stage in person re-ID systems to improve the discrimination." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1807.10908" id="1521">
-        <attvalues>
-          <attvalue for="5" value=" Therefore distance metric learning \cite{PRDC,LFDA:CVPR,SSSVM,rPcca,KISSME,LOMO,MLAPG,Zheng:nfst,NK3ML} is used as a second stage in person re-ID systems to improve the discrimination." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1706.03725" id="1522">
-        <attvalues>
-          <attvalue for="5" value=" There exist re-ranking methods \cite{Symantic,song:scalableManifold,SHaPE,Re-ranking:k-reciprocal} to be applied in the testing phase of a trained person re-ID system, to further improve the accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1703.08359" id="1523">
-        <attvalues>
-          <attvalue for="5" value=" There exist re-ranking methods \cite{Symantic,song:scalableManifold,SHaPE,Re-ranking:k-reciprocal} to be applied in the testing phase of a trained person re-ID system, to further improve the accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1701.08398" id="1524">
-        <attvalues>
-          <attvalue for="5" value=" There exist re-ranking methods \cite{Symantic,song:scalableManifold,SHaPE,Re-ranking:k-reciprocal} to be applied in the testing phase of a trained person re-ID system, to further improve the accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="1910.03923" target="1704.01719" id="1525">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has also been applied for person re-ID and shown good results on large training datasets \cite{Beyond:triplet_loss, Zhun:BodyLatentParts, Zhou:PointtoSetSimilarity, SpindleNet}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1703.03400" id="1526">
-        <attvalues>
-          <attvalue for="5" value=" Most existing methods either fall into the meta-learning based~\cite{optimization-as-model,MAML,Meta-SGD} or the metric-based ~\cite{relationnet,protonet,feat,match-net} paradigms." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1707.09835" id="1527">
-        <attvalues>
-          <attvalue for="5" value=" Most existing methods either fall into the meta-learning based~\cite{optimization-as-model,MAML,Meta-SGD} or the metric-based ~\cite{relationnet,protonet,feat,match-net} paradigms." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1711.06025" id="1528">
-        <attvalues>
-          <attvalue for="5" value=" Instead of fixing a pre-defined metric, Relation Network~\cite{relationnet} learns a deep distance metric to compare each query-support image pair." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1703.05175" id="1529">
-        <attvalues>
-          <attvalue for="5" value=" ProtoNet~\cite{protonet} first constructs a prototype for each category, by taking the mean of all the corresponding training examples, and then classifies test images based on the nearest prototype." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1812.03664" id="1530">
-        <attvalues>
-          <attvalue for="5" value=" The recently introduced FEAT model~\cite{feat} applies a transformer~\cite{DBLP:conf/nips/VaswaniSPUJGKP17} to contextualize the image features relative to the support set in a given task." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1606.04080" id="1531">
-        <attvalues>
-          <attvalue for="5" value=" Beyond one-shot learning, \cite{match-net} introduced an episode-based training strategy and proposed Matching Network, an FSL model which combines a weighted nearest-neighbor classifier with an attention mechanism." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1706.03762" id="1532">
-        <attvalues>
-          <attvalue for="5" value=" The recently introduced FEAT model~\cite{feat} applies a transformer~\cite{DBLP:conf/nips/VaswaniSPUJGKP17} to contextualize the image features relative to the support set in a given task." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1902.07104" id="1533">
-        <attvalues>
-          <attvalue for="5" value=" AM3~\cite{AM3} uses a pre-trained embedding of these category names, using the GloVe word embedding model \cite{DBLP:conf/emnlp/PenningtonSM14}, to predict visual prototypes." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="2005.13826" id="1534">
-        <attvalues>
-          <attvalue for="5" value=" TRAML~\cite{traml} also uses GloVe vectors, but instead uses them to get prior information about which categories may be most difficult to separate." />
-        </attvalues>
-      </edge>
-      <edge source="2102.00801" target="1810.04805" id="1535">
-        <attvalues>
-          <attvalue for="5" value=" In addition to using GloVe vectors for representing the category names, we also experiment with embeddings that are obtained using the BERT language model \cite{DBLP:conf/naacl/DevlinCLT19}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="1906.04155" id="1536">
-        <attvalues>
-          <attvalue for="5" value=" C{onsumer} digital cameras (eg, point-and-shoot, DSLR, mirrorless cameras and smartphones) are not tools designed for scientific imaging, eg, they are not scientific light-measuring devices~\cite{burggraaff2019standardized,akkaynak2014use,kim2012new}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="1703.10593" id="1537">
-        <attvalues>
-          <attvalue for="5" value=" Typically, research utilizing consumer camera imagery focuses on the development of filters or algorithms that alter the visual appearance of images~\cite{paris2009fast, guo2020zero,zhu2017unpaired,bychkovsky2011learning}, or on the understanding of scene content~\cite{deng2009imagenet,lowe1999object,xiao2010sun} and structure~\cite{mildenhall2021nerf,lee2009geometric} with downstream goals like recognizing, tracking, or counting objects." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="2003.08934" id="1538">
-        <attvalues>
-          <attvalue for="5" value=" Typically, research utilizing consumer camera imagery focuses on the development of filters or algorithms that alter the visual appearance of images~\cite{paris2009fast, guo2020zero,zhu2017unpaired,bychkovsky2011learning}, or on the understanding of scene content~\cite{deng2009imagenet,lowe1999object,xiao2010sun} and structure~\cite{mildenhall2021nerf,lee2009geometric} with downstream goals like recognizing, tracking, or counting objects." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="1703.09470" id="1539">
-        <attvalues>
-          <attvalue for="5" value=" If the camera's color bias could be eliminated, the recovery of scene reflectance (eg, for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc) and/or scene illumination (eg, for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc) could be achieved simpler and faster, and with more consistent results across cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="2106.04989" id="1540">
-        <attvalues>
-          <attvalue for="5" value=" If the camera's color bias could be eliminated, the recovery of scene reflectance (eg, for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc) and/or scene illumination (eg, for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc) could be achieved simpler and faster, and with more consistent results across cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="2104.13365" id="1541">
-        <attvalues>
-          <attvalue for="5" value=" If the camera's color bias could be eliminated, the recovery of scene reflectance (eg, for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc) and/or scene illumination (eg, for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc) could be achieved simpler and faster, and with more consistent results across cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="2011.11890" id="1542">
-        <attvalues>
-          <attvalue for="5" value=" If the camera's color bias could be eliminated, the recovery of scene reflectance (eg, for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc) and/or scene illumination (eg, for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc) could be achieved simpler and faster, and with more consistent results across cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11549" target="2006.12709" id="1543">
-        <attvalues>
-          <attvalue for="5" value=" If the camera's color bias could be eliminated, the recovery of scene reflectance (eg, for material property estimation~\cite{munoz2011bssrdf,debevec2004estimating,yu1999inverse}, spectral super-resolution~\cite{oh2016yourself,galliani2017learned}, etc) and/or scene illumination (eg, for computational color constancy~\cite{barnard2002camera,cheng2014illuminant,lo2021clcc}, image relighting~\cite{el2021ntire}, data augmentation~\cite{afifi2021cross,afifi2021cie,lou2015color} etc) could be achieved simpler and faster, and with more consistent results across cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2005.14313" target="1704.07813" id="1544">
-        <attvalues>
-          <attvalue for="5" value=", \cite{zhou2017unsupervised} proposed an unsupervised depth estimation end-to-end framework from video sequence where view synthesis act as the supervisory signal." />
-        </attvalues>
-      </edge>
-      <edge source="2005.14313" target="1802.05522" id="1545">
-        <attvalues>
-          <attvalue for="5" value=", \cite{mahjourian2018unsupervised} introduced a 3D loss to enforce consistency of the estimated 3D point cloud." />
-        </attvalues>
-      </edge>
-      <edge source="2005.14313" target="1803.02276" id="1546">
-        <attvalues>
-          <attvalue for="5" value=", \cite{yin2018geonet} jointly learned monocular depth, optical flow and ego motion from videos in unsupervised manner." />
-        </attvalues>
-      </edge>
-      <edge source="2005.14313" target="1806.01260" id="1547">
-        <attvalues>
-          <attvalue for="5" value=", \cite{godard2019digging} achieved state of the art results in KITTI rectified dataset with minimum reprojection loss, full-resolution mutli-scale sampling and a mask that ignores training pixels that violate camera motion assumptions." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2102.12145" id="1548">
-        <attvalues>
-          <attvalue for="5" value=" Some methods can even produce pose estimation at high throughput \cite{gdr, crt6d}, be specially robust to symmetries \cite{surfemb} and can even surpass synthetic to real domain gaps \cite{self6d, juil}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2111.13489" id="1549">
-        <attvalues>
-          <attvalue for="5" value=" Some methods can even produce pose estimation at high throughput \cite{gdr, crt6d}, be specially robust to symmetries \cite{surfemb} and can even surpass synthetic to real domain gaps \cite{self6d, juil}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2004.06468" id="1550">
-        <attvalues>
-          <attvalue for="5" value=" Some methods can even produce pose estimation at high throughput \cite{gdr, crt6d}, be specially robust to symmetries \cite{surfemb} and can even surpass synthetic to real domain gaps \cite{self6d, juil}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="1612.00796" id="1551">
-        <attvalues>
-          <attvalue for="5" value=" Retraining new objects on existing models might lead to catastrophic forgetting if not handled properly~\cite{catastrophic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="1906.05105" id="1552">
-        <attvalues>
-          <attvalue for="5" value=" Category level approaches~\cite{posefromshape, nocs} try to generalize up to a category, where each target object can be obtained with a simple deformation of a canonical model and all share semantic keypoints (ex." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="1711.04061" id="1553">
-        <attvalues>
-          <attvalue for="5" value=" Older methods have some very hard constraints such as needing masking and depth maps at test-time \cite{ppf}, a full colored 3D model of the target object \cite{deepim}, need to be fine-tuned on images of the same dataset for difficulties of overcoming domain gap~\cite{gen6d} or rely on an extremely expensive test-time optimization through renderers~\cite{inerf, nerfpose, latentfusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="1804.00175" id="1554">
-        <attvalues>
-          <attvalue for="5" value=" On top of these changes, we employ a 3D-refinement technique based on zoom refinement used on instance-level pose estimation~\cite{deepim,crt6d,cosypose}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2204.10776" id="1555">
-        <attvalues>
-          <attvalue for="5" value=" Older methods have some very hard constraints such as needing masking and depth maps at test-time \cite{ppf}, a full colored 3D model of the target object \cite{deepim}, need to be fine-tuned on images of the same dataset for difficulties of overcoming domain gap~\cite{gen6d} or rely on an extremely expensive test-time optimization through renderers~\cite{inerf, nerfpose, latentfusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2012.05877" id="1556">
-        <attvalues>
-          <attvalue for="5" value=" Older methods have some very hard constraints such as needing masking and depth maps at test-time \cite{ppf}, a full colored 3D model of the target object \cite{deepim}, need to be fine-tuned on images of the same dataset for difficulties of overcoming domain gap~\cite{gen6d} or rely on an extremely expensive test-time optimization through renderers~\cite{inerf, nerfpose, latentfusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2205.12257" id="1557">
-        <attvalues>
-          <attvalue for="5" value=" Particularly, OnePose~\cite{onepose} and OnePose++~\cite{onepose++} make use of existing state of the art pre-trained descriptor extractors on top of which a pose estimation pipeline is built." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2301.07673" id="1558">
-        <attvalues>
-          <attvalue for="5" value=" We also introduce a new efficient image to object attention layer which we call IO-Layer, which reduces both parameters when compared to the modules used by OnePose++~\cite{onepose++} and also separates the two input modalities, an image and a pointcloud, allowing the model to optimize weights specifically for either image or pointcloud keypoints." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2210.11718" id="1559">
-        <attvalues>
-          <attvalue for="5" value=" On top of these changes, we employ a 3D-refinement technique based on zoom refinement used on instance-level pose estimation~\cite{deepim,crt6d,cosypose}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.01382" target="2008.08465" id="1560">
-        <attvalues>
-          <attvalue for="5" value=" On top of these changes, we employ a 3D-refinement technique based on zoom refinement used on instance-level pose estimation~\cite{deepim,crt6d,cosypose}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2010.11929" id="1561">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies show the effectiveness of transformer models at scale~\cite{dosovitskiy2020image,zhai2022scaling,liu2022swin}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2106.04560" id="1562">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies show the effectiveness of transformer models at scale~\cite{dosovitskiy2020image,zhai2022scaling,liu2022swin}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2111.09883" id="1563">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies show the effectiveness of transformer models at scale~\cite{dosovitskiy2020image,zhai2022scaling,liu2022swin}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2104.10972" id="1564">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2106.08254" id="1565">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2111.06377" id="1566">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2111.09886" id="1567">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2202.03026" id="1568">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2208.06366" id="1569">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2208.10442" id="1570">
-        <attvalues>
-          <attvalue for="5" value=" With encoder pretraining on large-scale data~\cite{deng2009imagenet,ridnik2021imagenet}, the models~\cite{bao2021beit,he2022masked,xie2022simmim,chen2022context,peng2022beit,wang2022image} are able to achieve superior results on various vision tasks, including object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="2203.03605" id="1571">
-        <attvalues>
-          <attvalue for="5" value=" With supervised encoder-decoder pretraining on a large-scale dataset, Object365~\cite{shao2019objects365}, DINO~\cite{zhang2022dino} achieves a state-of-the-art result on COCO~\cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.03594" target="1405.0312" id="1572">
-        <attvalues>
-          <attvalue for="5" value="5$ mAP on COCO test-dev~\cite{lin2014microsoft} (Table~\ref{tab:leaderboard} and Table~\ref{tab:coco_dev}), setting a new record for COCO object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="1901.05555" id="1573">
-        <attvalues>
-          <attvalue for="5" value=" Many tail classes are associated with limited samples, while a few head categories occupy most of the instances \cite{CB-loss, OLTR, PriorLT, icasspunbalance}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2004.00431" id="1574">
-        <attvalues>
-          <attvalue for="5" value=" Some effective proposals replenish the tail samples via generation or optimization with the help of head instances\cite{M2m, Bagoftricks}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2007.10740" id="1575">
-        <attvalues>
-          <attvalue for="5" value=" The re-weighting ones punish different categories with data number relevant weight or logit bias\cite{BalCE, LADE-loss}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2104.00466" id="1576">
-        <attvalues>
-          <attvalue for="5" value=" Although the aforementioned methods have greatly mitigated the LT problem, the conclusions hold on the ResNet-based backbones \cite{MiSLAS, IB-loss}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2010.11929" id="1577">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, many transformer-based backbones \cite{vit} have surpassed the performance of CNN." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2204.07118" id="1578">
-        <attvalues>
-          <attvalue for="5" value=" DeiT\cite{deit3} proposes an effective receipt to train ViT with limited data, and MAE\cite{MAE} adopts a masked autoencoder to pre-train the ViT." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2111.06377" id="1579">
-        <attvalues>
-          <attvalue for="5" value=" DeiT\cite{deit3} proposes an effective receipt to train ViT with limited data, and MAE\cite{MAE} adopts a masked autoencoder to pre-train the ViT." />
-        </attvalues>
-      </edge>
-      <edge source="2302.14284" target="2111.03874" id="1580">
-        <attvalues>
-          <attvalue for="5" value=" Although some works show models' prediction distribution by visualization qualitatively\cite{PriorLT, MiSLAS, NCL}, a metric is required to evaluate it quantitatively." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="1709.03410" id="1581">
-        <attvalues>
-          <attvalue for="5" value=" These methods typically apply a two-branch framework~\cite{RN4, RN5,RN10} with a shared frozen pre-trained backbone, where a support branch is used to learn the object prototypes from very few labeled images (support images) and a query branch is utilized to make predictions for a query image on conditioned of the support prototypes." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="1903.02351" id="1582">
-        <attvalues>
-          <attvalue for="5" value=" These methods typically apply a two-branch framework~\cite{RN4, RN5,RN10} with a shared frozen pre-trained backbone, where a support branch is used to learn the object prototypes from very few labeled images (support images) and a query branch is utilized to make predictions for a query image on conditioned of the support prototypes." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2008.03898" id="1583">
-        <attvalues>
-          <attvalue for="5" value=" Under this framework, masked average pooling (MAP) \cite{ RN5, PANet, RN10} and clustering-based aggregation methods \cite{RN6, ASGNet, Part-aware-prototype} are two popular approaches to learning the representations for support images." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2008.01449" id="1584">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, methods such as \cite{PFENet, ASGNet, MLC} calculate prior maps, or similarity metrics, based on the query and support features to define their correlations." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2108.03032" id="1585">
-        <attvalues>
-          <attvalue for="5" value=" Other methods, including \cite{CWTans, CyCTR, HSNet}, feed support and query features into a decoder (eg, 4D convolution or transformer) to perform local region matching between query and support images." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2010.11929" id="1586">
-        <attvalues>
-          <attvalue for="5" value=" Building on this insight, we observe that self-attention in the Vision Transformer (ViT) \cite{ViT} and cross-attention in \cite{crossattention} can be utilized to capture contextual information of images during token dependency construction." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2207.00784" id="1587">
-        <attvalues>
-          <attvalue for="5" value=" Building on this insight, we observe that self-attention in the Vision Transformer (ViT) \cite{ViT} and cross-attention in \cite{crossattention} can be utilized to capture contextual information of images during token dependency construction." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="2106.02320" id="1588">
-        <attvalues>
-          <attvalue for="5" value=" Unlike CyCTR \cite{CyCTR} which employs Transformer blocks to pass support features to the query decoder, our approach emphasizes the importance of consistent mutual relations between query and support features." />
-        </attvalues>
-      </edge>
-      <edge source="2308.11201" target="1405.0312" id="1589">
-        <attvalues>
-          <attvalue for="5" value=" We evaluated the meta-learning ability of our model on two public FSS benchmarks, PASCAL-$5^i$ \cite{VOC} and COCO-$20^i$ \cite{COCO}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1906.02295" id="1590">
-        <attvalues>
-          <attvalue for="5" value=" Progressive NAPSAC~\cite{barath2019progressive} combines both and adds gradual convergence to uniform spatial sampling." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1803.07469" id="1591">
-        <attvalues>
-          <attvalue for="5" value=" Reflecting the inherent uncertainty of the threshold estimate, MAGSAC~\cite{barath2019magsac} marginalizes the quality function over a range of noise levels." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1912.05909" id="1592">
-        <attvalues>
-          <attvalue for="5" value=" MAGSAC++~\cite{Barath_2020_CVPR} proposes an iterative re-weighted least-squares optimization of the {\sftb} model with weights calculated from the inlier probability of points." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1706.00984" id="1593">
-        <attvalues>
-          <attvalue for="5" value=" Graph-Cut RANSAC \cite{barath2018graph}, in its local optimization, exploits the fact that real-world data tend to form spatial structures." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1503.02619" id="1594">
-        <attvalues>
-          <attvalue for="5" value=" In the repeated runs on datasets EVD~\cite{mishkin2015mods}, HPatches~\cite{Balntas_2017_CVPR}, PhotoTourism~\cite{phototourism} and Kusvod2~\cite{lebeda2012fixing}, it never failed." />
-        </attvalues>
-      </edge>
-      <edge source="2106.10240" target="1704.05939" id="1595">
-        <attvalues>
-          <attvalue for="5" value=" In the repeated runs on datasets EVD~\cite{mishkin2015mods}, HPatches~\cite{Balntas_2017_CVPR}, PhotoTourism~\cite{phototourism} and Kusvod2~\cite{lebeda2012fixing}, it never failed." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="2007.00643" id="1596">
-        <attvalues>
-          <attvalue for="5" value=" Object detectors play a crucial role in goal-oriented exploration \cite{chaplot2020object,cartillier2020semantic,chaplot2020learning}, and recent studies have demonstrated the relationship between object detection improvements and the reliable semantic mapping of environments \cite{chaplot2020object}, making them of significant importance in the field of robotics exploration." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="2004.05155" id="1597">
-        <attvalues>
-          <attvalue for="5" value=" Object detectors play a crucial role in goal-oriented exploration \cite{chaplot2020object,cartillier2020semantic,chaplot2020learning}, and recent studies have demonstrated the relationship between object detection improvements and the reliable semantic mapping of environments \cite{chaplot2020object}, making them of significant importance in the field of robotics exploration." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="1908.02983" id="1598">
-        <attvalues>
-          <attvalue for="5" value=" In self-training, a pre-trained model provides pseudo-labels for the unlabeled data, which may be refined by the introduction of additional noise \cite{arazo2019,xie2020selftraining,chen2020big}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="1911.04252" id="1599">
-        <attvalues>
-          <attvalue for="5" value=" In self-training, a pre-trained model provides pseudo-labels for the unlabeled data, which may be refined by the introduction of additional noise \cite{arazo2019,xie2020selftraining,chen2020big}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="2006.10029" id="1600">
-        <attvalues>
-          <attvalue for="5" value=" In self-training, a pre-trained model provides pseudo-labels for the unlabeled data, which may be refined by the introduction of additional noise \cite{arazo2019,xie2020selftraining,chen2020big}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="2112.01001" id="1601">
-        <attvalues>
-          <attvalue for="5" value=" \cite{chaplot2021seal} introduced a scenario where an agent explores the environment and collects data autonomously, proposing an approach to collect consistent pseudo-labels in the environment." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="1703.06870" id="1602">
-        <attvalues>
-          <attvalue for="5" value=" We implement our approach by equipping an agent with a pre-trained object detector, in this case MaskRCNN \cite{he2018mask}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="1503.03832" id="1603">
-        <attvalues>
-          <attvalue for="5" value=" In our approach, we utilize a contrastive loss function \cite{facenet2015} to ensure that features of an object are similar across its different views while features of different objects are distinguishable." />
-        </attvalues>
-      </edge>
-      <edge source="2302.10624" target="2002.05709" id="1604">
-        <attvalues>
-          <attvalue for="5" value=" Our method can be considered a generalization of the cropping strategy that is commonly utilized in self-supervised learning \cite{chen2020simple}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.12369" target="1708.06500" id="1605">
-        <attvalues>
-          <attvalue for="5" value=" Deep Neural Networks (DNNs) learn the mapping between the single RGB images and their corresponding depth maps to solve MDE, and show good performance on indoor and outdoor benchmarks~\cite{Uhrig2017THREEDV, Silberman:ECCV12}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.12369" target="1605.02305" id="1606">
-        <attvalues>
-          <attvalue for="5" value=" Classification Approaches for Regression (CAR) \cite{cao2017estimating, li2018monocular, fu2018deep, diaz2019soft, yang2019inferring, bhat2021adabins} have emerged recently in the spotlight among MDE algorithms." />
-        </attvalues>
-      </edge>
-      <edge source="2202.12369" target="1703.04977" id="1607">
-        <attvalues>
-          <attvalue for="5" value=" DNNs are prone to two kinds of uncertainty: aleatoric uncertainty and epistemic uncertainty~\cite{kendall2017uncertainties}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.12369" target="2005.06209" id="1608">
-        <attvalues>
-          <attvalue for="5" value=" Some works proposed to estimate the uncertainty of MDE DNNs by using an auxiliary network~\cite{Poggi_CVPR_2020, yu21bmvc}, or ensembling~\cite{2017simple}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.12369" target="1612.01474" id="1609">
-        <attvalues>
-          <attvalue for="5" value=" Some works proposed to estimate the uncertainty of MDE DNNs by using an auxiliary network~\cite{Poggi_CVPR_2020, yu21bmvc}, or ensembling~\cite{2017simple}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1808.01265" id="1610">
-        <attvalues>
-          <attvalue for="5" value=" Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1708.07819" id="1611">
-        <attvalues>
-          <attvalue for="5" value=" Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1901.01415" id="1612">
-        <attvalues>
-          <attvalue for="5" value=" Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1910.03997" id="1613">
-        <attvalues>
-          <attvalue for="5" value=" Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1802.10349" id="1614">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1804.08286" id="1615">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1711.11556" id="1616">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1811.12833" id="1617">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1903.12212" id="1618">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2003.00867" id="1619">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2108.03267" id="1620">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2007.09222" id="1621">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2004.05498" id="1622">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1909.00589" id="1623">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2008.12197" id="1624">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2007.08702" id="1625">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2105.00097" id="1626">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2103.05254" id="1627">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2107.09600" id="1628">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2101.10979" id="1629">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods mainly align domains in an adversarial~\cite{tsai2018learning,Zhang_2018_CVPR,Chen_2018_CVPR,vu2019advent,chang2019all,Kim_2020_CVPR,li2020content,truong2021bimal,wang2020classes,yang2020fda} or a self-training~\cite{Choi_2019_ICCV,mei2020instance,tranheden2021dacs,araslanov2021self,guo2021metacorrection,gao2021dsp,zhang2021prototypical} manner, regardless of how the domain gap is caused." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1903.01530" id="1630">
-        <attvalues>
-          <attvalue for="5" value=" One solution is to close this gap by defogging real foggy images, using an existing defogging method~\cite{ren2018deep,de2019learning,zhang2018densely,ren2016single,he2010single,berman2016non,ren2018gated}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1803.08396" id="1631">
-        <attvalues>
-          <attvalue for="5" value=" One solution is to close this gap by defogging real foggy images, using an existing defogging method~\cite{ren2018deep,de2019learning,zhang2018densely,ren2016single,he2010single,berman2016non,ren2018gated}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1804.00213" id="1632">
-        <attvalues>
-          <attvalue for="5" value=" One solution is to close this gap by defogging real foggy images, using an existing defogging method~\cite{ren2018deep,de2019learning,zhang2018densely,ren2016single,he2010single,berman2016non,ren2018gated}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="1810.05716" id="1633">
-        <attvalues>
-          <attvalue for="5" value=" They act as noise to hinder the domain adaptation to some extent~\cite{2018Does}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.00484" target="2006.15618" id="1634">
-        <attvalues>
-          <attvalue for="5" value=" Another solution is to add synthetic fog to clear images and learn with these synthetic foggy images and annotations of clear images in a supervised manner~\cite{sakaridis2018model,sakaridis2018semantic,2019Purely,dai2020curriculum,gong2021analogical}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.15475" target="2006.00497" id="1635">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, some other metrics, eg, PointSSIM\cite{pointSSIM}, GraphSIM\cite{GraphSIM}, MS-GraphSIM\cite{MSGraphSIM} and MPED\cite{MPED}, construct local neighborhoods to involve the structural features, which usually have better performances due to the fact that HVS is more sensitive to structural information." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1703.06870" id="1636">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1803.01534" id="1637">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1903.12174" id="1638">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="2004.04446" id="1639">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1912.04488" id="1640">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning-based methods have achieved promising results for image instance segmentation over the past few years, such as Mask R-CNN~\cite{he2017mask}, PANet~\cite{liu2018path}, TensorMask~\cite{chen2019tensormask}, CenterMask~\cite{wang2020centermask}, SOLO series~\cite{wang2020solo, wang2020solov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1810.04805" id="1641">
-        <attvalues>
-          <attvalue for="5" value=" In addition, transformers have made enormous strides in NLP\cite{devlin2018bert, radford2019language}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1906.05909" id="1642">
-        <attvalues>
-          <attvalue for="5" value=" There are quite a bit of works applying transformers to computer vision\cite{ramachandran2019stand, Zhao_2020_CVPR, carion2020end}, because transformers can capture the non-local and relational nature of images." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="2103.14030" id="1643">
-        <attvalues>
-          <attvalue for="5" value=" Especially, Swin Transformer\cite{liu2021swin} has been widely used for many computer vision tasks and achieved successful results, such as detection and segmentation tasks on COCO." />
-        </attvalues>
-      </edge>
-      <edge source="2209.13899" target="1903.00241" id="1644">
-        <attvalues>
-          <attvalue for="5" value=" And then we employed a strong segmentation model, and add mask scoring~\cite{huang2019mask} to the mask head." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2010.11929" id="1645">
-        <attvalues>
-          <attvalue for="5" value=" Vision Transformers (ViT)~\cite{dosovitskiy2020image} for dense prediction~\cite{zheng2021rethinking,ranftl2021vision} have achieved impressive results in tasks including medical image segmentation~\cite{hatamizadeh2022unetr}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2012.15840" id="1646">
-        <attvalues>
-          <attvalue for="5" value=" Vision Transformers (ViT)~\cite{dosovitskiy2020image} for dense prediction~\cite{zheng2021rethinking,ranftl2021vision} have achieved impressive results in tasks including medical image segmentation~\cite{hatamizadeh2022unetr}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2103.13413" id="1647">
-        <attvalues>
-          <attvalue for="5" value=" Vision Transformers (ViT)~\cite{dosovitskiy2020image} for dense prediction~\cite{zheng2021rethinking,ranftl2021vision} have achieved impressive results in tasks including medical image segmentation~\cite{hatamizadeh2022unetr}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="1908.07919" id="1648">
-        <attvalues>
-          <attvalue for="5" value=" In general, high-resolution features~\cite{wang2020deep} preserving details are always desirable for precise segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="1706.03762" id="1649">
-        <attvalues>
-          <attvalue for="5" value=" However, because of the quadratic computation complexity in self-attention~\cite{vaswani2017attention}, doubling the resolution per dimension in a 3D volume can lead to an 8$\times$ longer sequence and hence 64$\times$ more computation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2111.06377" id="1650">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, we adapt sampling methods in classification, eg, EViT~\cite{liang2021evit} and DynamicViT~\cite{rao2021dynamicvit}, to sparse encoding, and masked image modeling (MIM)~\cite{he2021masked,bao2021beit} to token completion." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2111.11429" id="1651">
-        <attvalues>
-          <attvalue for="5" value=" Considering ViT's flexibility and great potential in masked image modeling~\cite{he2021masked,li2021benchmarking}, we explore acceleration algorithms based on the standard ViT." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2106.02034" id="1652">
-        <attvalues>
-          <attvalue for="5" value=" Previous sparse token classification models~\cite{rao2021dynamicvit,liang2021evit} do not require token completion." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2106.08254" id="1653">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, we adapt sampling methods in classification, eg, EViT~\cite{liang2021evit} and DynamicViT~\cite{rao2021dynamicvit}, to sparse encoding, and masked image modeling (MIM)~\cite{he2021masked,bao2021beit} to token completion." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="1901.10517" id="1654">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by subset sampling~\cite{xie2019reparameterizable,li2021differentiable,cordonnier2021differentiable}, the intractable gradients of the topK operation are approximated through a perturbed continuous score distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2108.04657" id="1655">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by subset sampling~\cite{xie2019reparameterizable,li2021differentiable,cordonnier2021differentiable}, the intractable gradients of the topK operation are approximated through a perturbed continuous score distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2104.03059" id="1656">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by subset sampling~\cite{xie2019reparameterizable,li2021differentiable,cordonnier2021differentiable}, the intractable gradients of the topK operation are approximated through a perturbed continuous score distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06522" target="2106.05735" id="1657">
-        <attvalues>
-          <attvalue for="5" value=" We evaluate our method on two relatively sparse 3D medical image segmentation datasets, the CT Abdomen Multi-organ Segmentation (BTCV~\cite{landman2015miccai}, N=30) dataset and the MRI Brain Tumor Segmentation (MSD BraTS~\cite{antonelli2021medical}, N=484) dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2001.06810" id="1658">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-based methods~\cite{cosnet,cfam,f2net}, use only appearance features extracted from multiple frames." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2111.02368" id="1659">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-based methods~\cite{cosnet,cfam,f2net}, use only appearance features extracted from multiple frames." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2012.02534" id="1660">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-based methods~\cite{cosnet,cfam,f2net}, use only appearance features extracted from multiple frames." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2003.04253" id="1661">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-motion based methods~\cite{matnet,rtnet,dtt,amc,tmo} use the target frame as well as corresponding optical flow (called &quot;motion cue&quot;), which is calculated using optical flow networks~\cite{flownet,raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2209.03138" id="1662">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-motion based methods~\cite{matnet,rtnet,dtt,amc,tmo} use the target frame as well as corresponding optical flow (called &quot;motion cue&quot;), which is calculated using optical flow networks~\cite{flownet,raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="1612.01925" id="1663">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-motion based methods~\cite{matnet,rtnet,dtt,amc,tmo} use the target frame as well as corresponding optical flow (called &quot;motion cue&quot;), which is calculated using optical flow networks~\cite{flownet,raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="2003.12039" id="1664">
-        <attvalues>
-          <attvalue for="5" value=" Appearance-motion based methods~\cite{matnet,rtnet,dtt,amc,tmo} use the target frame as well as corresponding optical flow (called &quot;motion cue&quot;), which is calculated using optical flow networks~\cite{flownet,raft}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.04376" target="1512.03385" id="1665">
-        <attvalues>
-          <attvalue for="5" value=" TAF aligns the information of adjacent frames to the target frame to propagate contextual information in multi-layer features extracted from each residual block of ResNet~\cite{resnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.18299" target="2211.05642" id="1666">
-        <attvalues>
-          <attvalue for="5" value=" Concretely, we have recently shown that the surface normal could be reconstructed at each specularity~\cite{makki2023normal, makki2023elliptical}, with a two-fold ambiguity." />
-        </attvalues>
-      </edge>
-      <edge source="2209.01355" target="2001.05566" id="1667">
-        <attvalues>
-          <attvalue for="5" value=" For example, semantic segmentation is applied to medical image analysis, autonomous vehicles, video surveillance, augmented reality, and many other areas \cite{minaee2021image}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.01355" target="1802.01436" id="1668">
-        <attvalues>
-          <attvalue for="5" value="265-intra \cite{sullivan2012overview} have achieved good Rate-distortion performance in human perception while some learned end-to-end image compression methods \cite{balle2018variational, cheng2020learned} also achieved comparable performance." />
-        </attvalues>
-      </edge>
-      <edge source="2209.01355" target="1803.06131" id="1669">
-        <attvalues>
-          <attvalue for="5" value="$ \cite{wang2022learning} proposed a channel selection method and an entropy estimation method to improve the results in \cite{torfason2018towards}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.01355" target="2107.08373" id="1670">
-        <attvalues>
-          <attvalue for="5" value="$ \cite{choi2021scalable} further verified this by using an information theory method and proposed a compression domain-based approach which is verified on object detection and semantic segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2209.01355" target="2112.09300" id="1671">
-        <attvalues>
-          <attvalue for="5" value="$ \cite{Bai2022AAAI} utilized a transformer to improve image classification based on compressed domain." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1704.00675" id="1672">
-        <attvalues>
-          <attvalue for="5" value=" The experiments on the DAVIS16~\cite{perazzi2016benchmark} and DAVIS17~\cite{pont20172017} datasets show that the proposed method improves the $\mathcal{J\&amp;F}$ measure of three representative VOS models, OSVOS~\cite{caelles2017one}, RGMP~\cite{oh2018fast}, STM~\cite{oh2019video}, by $7." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1611.05198" id="1673">
-        <attvalues>
-          <attvalue for="5" value=" The experiments on the DAVIS16~\cite{perazzi2016benchmark} and DAVIS17~\cite{pont20172017} datasets show that the proposed method improves the $\mathcal{J\&amp;F}$ measure of three representative VOS models, OSVOS~\cite{caelles2017one}, RGMP~\cite{oh2018fast}, STM~\cite{oh2019video}, by $7." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1706.09364" id="1674">
-        <attvalues>
-          <attvalue for="5" value=" Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time~\cite{caelles2017one,voigtlaender2017online,maninis2018video,meinhardt2020make}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1709.06031" id="1675">
-        <attvalues>
-          <attvalue for="5" value=" Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time~\cite{caelles2017one,voigtlaender2017online,maninis2018video,meinhardt2020make}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="2012.01866" id="1676">
-        <attvalues>
-          <attvalue for="5" value=" Online learning-based methods fine-tune a model on the ground-truth mask of the first frame at test time~\cite{caelles2017one,voigtlaender2017online,maninis2018video,meinhardt2020make}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1612.02646" id="1677">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1612.05478" id="1678">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1703.09554" id="1679">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1908.06647" id="1680">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1904.00607" id="1681">
-        <attvalues>
-          <attvalue for="5" value=" The experiments on the DAVIS16~\cite{perazzi2016benchmark} and DAVIS17~\cite{pont20172017} datasets show that the proposed method improves the $\mathcal{J\&amp;F}$ measure of three representative VOS models, OSVOS~\cite{caelles2017one}, RGMP~\cite{oh2018fast}, STM~\cite{oh2019video}, by $7." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="2003.08333" id="1682">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="2109.11404" id="1683">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="2106.05210" id="1684">
-        <attvalues>
-          <attvalue for="5" value=" Propagation-based methods use predicted masks from the past frames to guide the current prediction~\cite{perazzi2017learning,jampani2017video,xiao2018monet,oh2018fast,khoreva2019lucid}, and Matching-based methods perform feature matching in embedding space to segment the target object~\cite{wang2019ranet,oh2019video,yang2020collaborative,seong2021hierarchical,cheng2021stcn}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.01197" target="1910.10895" id="1685">
-        <attvalues>
-          <attvalue for="5" value=" Some methods~\cite{oh2018fast,yang2019anchor} address the scale-inconsistency problem by averaging the predictions of the inputs with multiple sizes." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09252" target="1903.08094" id="1686">
-        <attvalues>
-          <attvalue for="5" value=" As such, recovering 3D information about real-world objects allows the exploration of many relevant applications, including self-driving cars~\cite{Wang:ACCV:2018, Schonbein:IROS:2014}, robot navigation~\cite{Moreau:SITIS:2012,Fernandez-Labrador:LRA:2020}, virtual tourism~\cite{Fangi:ISPRS:2018,Kim:JCVIU:2015}, infrastructure inspection~\cite{Pathak:IST:2016,Pathak:JCMSI:2017}, archaeological~\cite{Pagani:VAST:2011,Gava:VISAPP:2015} and architectural modeling~\cite{Fangi:ISPRS:2018,Giovanni:ECCV:2016}, city planning~\cite{Kim:IVMSPW:2013,Kuhn:GCPR:2017}, and 3D cinema~\cite{Azevedo:TCSVT:2019,Thatte:VCIP:2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09252" target="1901.01848" id="1687">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, most of the image processing and computer vision algorithms developed in the past -- which were designed to work with perspective images -- tend to fail when directly applied to omnidirectional media, regardless of their objective~\cite{Cruz-Mota:IJCV:2012,Su:NIPS:2017,Tateno:ECCV:2018,Azevedo:TCSVT:2019}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09252" target="1701.08493" id="1688">
-        <attvalues>
-          <attvalue for="5" value=" In theory, 3D information can only be inferred from two or more captures of the scene, as in typical multi-view stereo \cite{Seitz:CVPR:2006} or structure from motion \cite{Ozyesil:AN:2017} approaches." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09252" target="1902.10904" id="1689">
-        <attvalues>
-          <attvalue for="5" value=" It is worth mentioning that the existing nomenclature for 360 images is vast, and the terms omnidirectional \cite{Won:ICRA:2019, Wegner:ICIP:2018}, spherical \cite{Silveira:CVPR:2019, Eder:3DV:2019}, full panorama \cite{Pagani:ICCV:2011, Im:ECCV:2016}, and 360\degree \cite{Zioulis:3DV:2019, Huang:VR:2017} images/videos/cameras are often used as synonyms." />
-        </attvalues>
-      </edge>
-      <edge source="2401.09252" target="1912.09390" id="1690">
-        <attvalues>
-          <attvalue for="5" value=" As such, omnidirectional media present heavy {non-affine} distortions that are inherent to the spherical camera model~\cite{Eder:CVPR:2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1311.2524" id="1691">
-        <attvalues>
-          <attvalue for="5" value=" Object scale has been widely considered in model design, as various detection paradigms~\cite{rcnn}, augmentation schemes~\cite{augmentation}, and modules~\cite{sod} are proposed to improve small object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1511.06523" id="1692">
-        <attvalues>
-          <attvalue for="5" value=" As the needs of real-world applications in a wide variety of scenarios arise~\cite{widerface, aerial}, the significance of research regarding a particular topic elevates." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1711.11575" id="1693">
-        <attvalues>
-          <attvalue for="5" value=" The works on improving specific aspects~\cite{relation, scale} of object detection, such as detecting dense objects~\cite{florence2018dense, densebox} and small objects~\cite{augmentation}, boost the practical value of object detection and consequently inspire further advances~\cite{cascade, augfpn}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1806.08756" id="1694">
-        <attvalues>
-          <attvalue for="5" value=" The works on improving specific aspects~\cite{relation, scale} of object detection, such as detecting dense objects~\cite{florence2018dense, densebox} and small objects~\cite{augmentation}, boost the practical value of object detection and consequently inspire further advances~\cite{cascade, augfpn}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1906.11172" id="1695">
-        <attvalues>
-          <attvalue for="5" value=" Object scale has been widely considered in model design, as various detection paradigms~\cite{rcnn}, augmentation schemes~\cite{augmentation}, and modules~\cite{sod} are proposed to improve small object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1712.00726" id="1696">
-        <attvalues>
-          <attvalue for="5" value=" The works on improving specific aspects~\cite{relation, scale} of object detection, such as detecting dense objects~\cite{florence2018dense, densebox} and small objects~\cite{augmentation}, boost the practical value of object detection and consequently inspire further advances~\cite{cascade, augfpn}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1708.02002" id="1697">
-        <attvalues>
-          <attvalue for="5" value=" \cite{retina} propose the focal loss to tackle dense object detection and prompt it to become a common practice for classification loss in object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1811.08883" id="1698">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by previous works~\cite{diagnose,rethinking,survey}, which provide in-depth ablations, analyses, and insights regarding object detection algorithms, we start with diagnosing and analyzing existing methods for object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2011.08529" target="1907.09408" id="1699">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by previous works~\cite{diagnose,rethinking,survey}, which provide in-depth ablations, analyses, and insights regarding object detection algorithms, we start with diagnosing and analyzing existing methods for object detection." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="2001.04193" id="1700">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1610.02984" id="1701">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1604.02531" id="1702">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1711.08184" id="1703">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1910.09777" id="1704">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1809.05996" id="1705">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1808.00157" id="1706">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1705.07206" id="1707">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1904.04536" id="1708">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="2003.04845" id="1709">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1703.05446" id="1710">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1612.00137" id="1711">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1705.03098" id="1712">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1710.10000" id="1713">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1701.01779" id="1714">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1605.02914" id="1715">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="2012.13392" id="1716">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="2103.10455" id="1717">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1705.02757" id="1718">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1702.05693" id="1719">
-        <attvalues>
-          <attvalue for="5" value=" Human-centric perception remains a core focus within the computer vision and machine learning communities, encompassing a wide array of research tasks and applications such as person ReID~\cite{ye2021deep,zheng2016person,zheng2015scalable,wang2020comprehensive,zheng2017person,wu2019deep,zhang2017alignedreid}, human parsing~\cite{li2020self,ruan2019devil,gong2018instance,li2017multiple,gong2019graphonomy,wang2020hierarchical,gong2017look}, human pose estimation~\cite{fang2017rmpe,chen20173d,martinez2017simple,andriluka2018posetrack,papandreou2017towards,belagiannis2017recurrent,zheng2023deep,zheng20213d}, and pedestrian detection~\cite{liu2019high,gawande2020pedestrian,mao2017can,zhang2017citypersons,lan2018pedestrian,iftikhar2022deep,roszyk2022adopting}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="1906.11300" id="1720">
-        <attvalues>
-          <attvalue for="5" value=" Despite significant advancements, these algorithms are inherently data-intensive and frequently encounter overfitting issues~\cite{ying2019overview,bejani2021systematic,bartlett2020benign,chen2020robust,zhang2021lbcf,hosseini2020tried,zhang2022towards}, where models excel with training data but falter on unseen test data." />
-        </attvalues>
-      </edge>
-      <edge source="2403.08650" target="2209.06053" id="1721">
-        <attvalues>
-          <attvalue for="5" value=" Despite significant advancements, these algorithms are inherently data-intensive and frequently encounter overfitting issues~\cite{ying2019overview,bejani2021systematic,bartlett2020benign,chen2020robust,zhang2021lbcf,hosseini2020tried,zhang2022towards}, where models excel with training data but falter on unseen test data." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1703.10896" id="1722">
-        <attvalues>
-          <attvalue for="5" value=" Only keypoints are detected in \cite{rad2017bb8} and \cite{tekin2018real}, in the latter case using a YOLO-like \cite{redmon2016you} backbone." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1711.08848" id="1723">
-        <attvalues>
-          <attvalue for="5" value=" Only keypoints are detected in \cite{rad2017bb8} and \cite{tekin2018real}, in the latter case using a YOLO-like \cite{redmon2016you} backbone." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1506.02640" id="1724">
-        <attvalues>
-          <attvalue for="5" value=" Only keypoints are detected in \cite{rad2017bb8} and \cite{tekin2018real}, in the latter case using a YOLO-like \cite{redmon2016you} backbone." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1711.00199" id="1725">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{xiang2017posecnn} a prediction is made for the object center, followed by a regression towards the rotation." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1812.11788" id="1726">
-        <attvalues>
-          <attvalue for="5" value=" The method in \cite{peng2019pvnet} uses a compromising approach where a semi-dense set of keypoints are predicted over the image." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1711.10006" id="1727">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{kehl2017ssd}, an SSD-like detector \cite{liu2016ssd} provides initial detections, which are refined using depth information." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1512.02325" id="1728">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{kehl2017ssd}, an SSD-like detector \cite{liu2016ssd} provides initial detections, which are refined using depth information." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1901.04780" id="1729">
-        <attvalues>
-          <attvalue for="5" value=" Initial segmentations are used in \cite{wang2019densefusion} and finally the pose is refined using PointNet \cite{qi2017pointnet}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.09057" target="1612.00593" id="1730">
-        <attvalues>
-          <attvalue for="5" value=" Our method relies on a PointNet backbone \cite{qi2017pointnet} to achieve this, and we show that in many of the tested circumstances, this significantly increases detection rates and localization accuracy during full 6 DoF pose estimation of objects." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1806.01261" id="1731">
-        <attvalues>
-          <attvalue for="5" value=" \end{enumerate} Relation to previous work: Iterative graph based methods have a long history in machine learning and are currently enjoying a fast-growing interest~\cite{battaglia2018relational, pmlr-v70-gilmer17a}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1312.6203" id="1732">
-        <attvalues>
-          <attvalue for="5" value=" Initial methods generalizing conv nets to the case of graph structured data~\cite{DBLP:journals/corr/BrunaZSL13_bruna_lecun_spectral,henaff2015deep,defferrard2016convolutional} learn in the spectral domain of the graph." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1506.05163" id="1733">
-        <attvalues>
-          <attvalue for="5" value=" Initial methods generalizing conv nets to the case of graph structured data~\cite{DBLP:journals/corr/BrunaZSL13_bruna_lecun_spectral,henaff2015deep,defferrard2016convolutional} learn in the spectral domain of the graph." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1606.09375" id="1734">
-        <attvalues>
-          <attvalue for="5" value=" Initial methods generalizing conv nets to the case of graph structured data~\cite{DBLP:journals/corr/BrunaZSL13_bruna_lecun_spectral,henaff2015deep,defferrard2016convolutional} learn in the spectral domain of the graph." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1609.02907" id="1735">
-        <attvalues>
-          <attvalue for="5" value=" They are approximated~\cite{kipf2017semi} by message passing based on linear operations~\cite{duvenaud2015convolutional} or MLPs~\cite{battaglia2016interaction}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1509.09292" id="1736">
-        <attvalues>
-          <attvalue for="5" value=" They are approximated~\cite{kipf2017semi} by message passing based on linear operations~\cite{duvenaud2015convolutional} or MLPs~\cite{battaglia2016interaction}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1612.00222" id="1737">
-        <attvalues>
-          <attvalue for="5" value=" They are approximated~\cite{kipf2017semi} by message passing based on linear operations~\cite{duvenaud2015convolutional} or MLPs~\cite{battaglia2016interaction}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1810.00826" id="1738">
-        <attvalues>
-          <attvalue for="5" value=" Aggregation of messages needs permutation invariant operators such as max or sum, the last one being proved superior in \cite{xu2018how_powerful}, with attention mechanism~\cite{velickovic2018graph_gat} as an alternative." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1710.10903" id="1739">
-        <attvalues>
-          <attvalue for="5" value=" Aggregation of messages needs permutation invariant operators such as max or sum, the last one being proved superior in \cite{xu2018how_powerful}, with attention mechanism~\cite{velickovic2018graph_gat} as an alternative." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1511.05493" id="1740">
-        <attvalues>
-          <attvalue for="5" value=" Recurrence is used in graph neural nets~\cite{li2016gated} to tackle symbolic tasks with single input and sequential language output." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1807.03819" id="1741">
-        <attvalues>
-          <attvalue for="5" value=" It is improved in~\cite{dehghani2018universal_transformer} by performing operations recurrently." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1706.03762" id="1742">
-        <attvalues>
-          <attvalue for="5" value=" Long-range dependencies in sequential language are captured in~\cite{vaswani2017attention} with a self-attention model." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1806.01822" id="1743">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{NIPS2018_7960_transformer} new information is incorporated into the existing memory by self-attention using a temporary new node." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1706.01427" id="1744">
-        <attvalues>
-          <attvalue for="5" value=" Initial node information could come from each local spatio-temporal point in convolutional feature maps~\cite{NIPS2017_7082_ralational_cuburi, wang2018non_local} or from features corresponding to entities detected by external methods, such as objects~\cite{wang2018videos_gupta2, Ghosh2018StackedSG_B, tsai2019video_C} or skeletons~\cite{yan2018spatial_A}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1806.01810" id="1745">
-        <attvalues>
-          <attvalue for="5" value=" Initial node information could come from each local spatio-temporal point in convolutional feature maps~\cite{NIPS2017_7082_ralational_cuburi, wang2018non_local} or from features corresponding to entities detected by external methods, such as objects~\cite{wang2018videos_gupta2, Ghosh2018StackedSG_B, tsai2019video_C} or skeletons~\cite{yan2018spatial_A}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1801.07455" id="1746">
-        <attvalues>
-          <attvalue for="5" value=" A 1D convolution is used in \cite{yan2018spatial_A} to temporally connect only the nodes corresponding to the same skeleton joint and recently~\cite{Ghosh2018StackedSG_B} send messages between nodes corresponding to the same entities, while we recurrently update in time the state of each node." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1806.06157" id="1747">
-        <attvalues>
-          <attvalue for="5" value=" Also, the approach in \cite{Baradel_2018_ECCV_orn} is to extract objects and form relations between objects from pairs of time steps randomly chosen." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1711.07971" id="1748">
-        <attvalues>
-          <attvalue for="5" value=" All space-time positions in the input volume are connected in~\cite{wang2018non_local, wang2018videos_gupta2,chen20182_a2nets}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1811.10575" id="1749">
-        <attvalues>
-          <attvalue for="5" value=" A 1D convolution is used in \cite{yan2018spatial_A} to temporally connect only the nodes corresponding to the same skeleton joint and recently~\cite{Ghosh2018StackedSG_B} send messages between nodes corresponding to the same entities, while we recurrently update in time the state of each node." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1512.00567" id="1750">
-        <attvalues>
-          <attvalue for="5" value=" The work in \cite{szegedy2016incv2, chollet2017xception} confirm our hypothesis that features could be more efficiently processed by factorization into simpler operations." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1510.00562" id="1751">
-        <attvalues>
-          <attvalue for="5" value=" The models in~\cite{sun2015human_factorised, xie2018rethinking, tran2018closer} factorize 3D convolutions into 2D spatial and 1D temporal convolutions." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1711.08496" id="1752">
-        <attvalues>
-          <attvalue for="5" value=" For spatio-temporal processing, some methods, which do not use explicit graph modeling, encode frames individually using 2D convolutions and aggregate them in different ways~\cite{karpathy2014large, yue2015beyond, donahue2015long}; others form relations as functions (MLPs) over sets of frames~\cite{zhou2018temporal_trn_torralba} or use 3D convolution inflated from existing 2D convolutional networks~\cite{carreira2017quo}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1705.07750" id="1753">
-        <attvalues>
-          <attvalue for="5" value=" For spatio-temporal processing, some methods, which do not use explicit graph modeling, encode frames individually using 2D convolutions and aggregate them in different ways~\cite{karpathy2014large, yue2015beyond, donahue2015long}; others form relations as functions (MLPs) over sets of frames~\cite{zhou2018temporal_trn_torralba} or use 3D convolution inflated from existing 2D convolutional networks~\cite{carreira2017quo}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1406.2199" id="1754">
-        <attvalues>
-          <attvalue for="5" value=" Optical flow could be used as input to a separate branch of a 2D ConvNet \cite{simonyan2014two_stream} or used as part of the model to guide the kernel of 3D convolutions~\cite{NIPS2018_7489_trajectory}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.05582" target="1506.04214" id="1755">
-        <attvalues>
-          <attvalue for="5" value=" To cover both spatial and temporal dimensions simultaneously, Convolutional LSTM~\cite{Shi2015ConvLSTM} can be used, augmented with additional memory~\cite{Wang2017ST-LSTM} or self-attention in order to update LSTM hidden states~\cite{wang2018eidetic}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.02573" target="2006.13256" id="1756">
-        <attvalues>
-          <attvalue for="5" value=" The EPIC-KITCHENS-100 dataset is a large-scale video benchmark, capturing daily cooking activities from egocentric perspective~\cite{damen2020rescaling}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.02573" target="2001.09691" id="1757">
-        <attvalues>
-          <attvalue for="5" value=" To the best of our knowledge, MM-SADA~\cite{munro2020multi} is the first attempt on UDA for fine-grained action recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2206.02573" target="1907.12743" id="1758">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, we adapt the TA3N~\cite{chen2019temporal}, an existing method for video domain adaptation, to learn a fine-grained action recognition model that can be adapted to the unlabeled target domain." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02715" target="1502.00956" id="1759">
-        <attvalues>
-          <attvalue for="5" value=" Estimating planar homographic transformations between consecutive video frames is a fundamental image task and is an essential part of many multimedia applications, including video management, robot navigation, content-based video retrieval, intelligent drones, self-driving vehicles, etc The estimated homography matrix, for example, can be used for image stitching~\cite{brown2007automatic}, image completion~\cite{huang2014image}, monocular SLAM~\cite{mur2015orb}, camera calibration~\cite{zhang2000flexible}, 3d scene reconstruction~\cite{zhang19963d}, and camera tracking~\cite{simon2000markerless}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02715" target="1606.03798" id="1760">
-        <attvalues>
-          <attvalue for="5" value=" Such an unsupervised method has better adaptability and stronger performance than the supervised one~\cite{detone2016deep} but may suffer from severe over-fitting issues." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02715" target="1709.03966" id="1761">
-        <attvalues>
-          <attvalue for="5" value="~\cite{nguyen2018unsupervised} introduce an unsupervised method that does not need labels of homography matrices to train the deep networks, as reviewed in Section~\ref{sect:one}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.05149" target="1803.08326" id="1762">
-        <attvalues>
-          <attvalue for="5" value=" Starting from the classic Gray World algorithm \cite{buchsbaum1980spatial} assuming the global averaged vector of the captured scene is achromatic, a large selection of AWB methods have been proposed, including but not limited to Gray Edge \cite{van2007edge}, Gray Pixels \cite{yanlin2019vissap, yanlin2019arxiv}, Random Forest-based AWB \cite{cheng2015effective}, Corrected-Moment \cite{finlayson2013corrected} and Convolutional Color Constancy \cite{barron2015convolutional, barron2017fourier}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.05149" target="1507.00410" id="1763">
-        <attvalues>
-          <attvalue for="5" value=" Starting from the classic Gray World algorithm \cite{buchsbaum1980spatial} assuming the global averaged vector of the captured scene is achromatic, a large selection of AWB methods have been proposed, including but not limited to Gray Edge \cite{van2007edge}, Gray Pixels \cite{yanlin2019vissap, yanlin2019arxiv}, Random Forest-based AWB \cite{cheng2015effective}, Corrected-Moment \cite{finlayson2013corrected} and Convolutional Color Constancy \cite{barron2015convolutional, barron2017fourier}." />
-        </attvalues>
-      </edge>
-      <edge source="2010.05149" target="1504.04548" id="1764">
-        <attvalues>
-          <attvalue for="5" value=" \cite{bianco2015color} initially shows the potential of applying CNN for illumination estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2010.05149" target="1607.03856" id="1765">
-        <attvalues>
-          <attvalue for="5" value=" Then more effective and efficient CNN variants are designed, like AlexNet+SVM \cite{yanlin2016icpr}, DS-Net \cite{shi2016eccv}, FC4-Net \cite{hu2017cvpr}, RCC \cite{yanlin2017iccv}, BCC \cite{qian2020benchmark} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2010.05149" target="1912.11180" id="1766">
-        <attvalues>
-          <attvalue for="5" value=" The up-to-date works C4-Net \cite{yu2020aaai}, ICDF \cite{xu2020end} achieved nearly-saturated results over these two datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2301.03796" target="1810.03173" id="1767">
-        <attvalues>
-          <attvalue for="5" value=" Local contrast measure (LCM) \cite{chen2013local}, improved local contrast measure (ILCM) \cite{han2014robust}, relative local contrast measure (RLCM) \cite{han2018infrared}, Tri-Layer local constrast method (TLLCM) \cite{han2019localb}, novel local contrast descriptor (NLCD) \cite{qin2019infrared}, absolute directional mean difference (ADMD) \cite{moradi2020fast}, and weighted strengthened local contrast measure (WSLCM) \cite{han2020infrared} are the most effective IRSTDs in the literature." />
-        </attvalues>
-      </edge>
-      <edge source="2301.03796" target="2009.14530" id="1768">
-        <attvalues>
-          <attvalue for="5" value=" ACM \cite{dai2021asymmetric} devised feature fusion modules within the encoder-decoder framework to enhance the representation of low and deep semantic features, resulting in a more effective feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2301.03796" target="2012.08573" id="1769">
-        <attvalues>
-          <attvalue for="5" value=" ALCNet \cite{dai2021attentional} employs shift operations on semantic tensors to simulate local contrast, aiming to extract specific local information about the target." />
-        </attvalues>
-      </edge>
-      <edge source="2301.03796" target="2106.00487" id="1770">
-        <attvalues>
-          <attvalue for="5" value=" &#09;In \cite{li2022dense}, the authors introduce a dense nested interactive module (DNIM) to maintain the information of small targets in deep layers through progressive interaction." />
-        </attvalues>
-      </edge>
-      <edge source="2301.03796" target="2304.01484" id="1771">
-        <attvalues>
-          <attvalue for="5" value=" &#09;&#09;In \cite{ying2023mapping}, the authors proposed a label evolution framework called Label Evolution with Single Point Supervision (LESPS) to address the labor-intensive nature of fully supervised training for infrared small target detection in CNNs." />
-        </attvalues>
-      </edge>
-      <edge source="2210.09220" target="2010.11929" id="1772">
-        <attvalues>
-          <attvalue for="5" value=" Visual inspection of the basis functions in the ViT paper \cite{DBLP:journals/corr/abs-2010-11929} show them to bear at least a passing resemblance to visual receptive fields in higher animals \cite{hubel1963shape}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.09220" target="2102.02000" id="1773">
-        <attvalues>
-          <attvalue for="5" value=" In this paper the visual interest operator chosen is the ST-transform \cite{DBLP:journals/corr/abs-2102-02000}, returning the connected dark/light boundary chains as geometrically filterable candidate saccade points." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1501.00092" id="1774">
-        <attvalues>
-          <attvalue for="5" value=" Deep regression neural network (NN)-based methods that can accurately predict real- or complex-valued output have been rapidly gaining popularity in a wide range of computational imaging and computer vision applications including image denoising \cite{Vincent&amp;etal:10JMLR, Xie&amp;Xu&amp;Chen:12NIPS, Zhang&amp;etal:17TIP}, image deblurring \cite{Xu&amp;etal:14NIPS}, image super-resolution \cite{Dong&amp;etal:16TPAMI,Kim&amp;Lee&amp;Lee:16CVPR}, light-field reconstruction \cite{Chun&amp;etal:20PAMI, Huang&amp;etal:20ICASSP}, object localization \cite{Szegedy&amp;Toshev&amp;Erhan:13NIPS}, end-to-end autonomous driving \cite{Bojarski&amp;etal:16NIPS-DLS}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1511.04587" id="1775">
-        <attvalues>
-          <attvalue for="5" value=" Deep regression neural network (NN)-based methods that can accurately predict real- or complex-valued output have been rapidly gaining popularity in a wide range of computational imaging and computer vision applications including image denoising \cite{Vincent&amp;etal:10JMLR, Xie&amp;Xu&amp;Chen:12NIPS, Zhang&amp;etal:17TIP}, image deblurring \cite{Xu&amp;etal:14NIPS}, image super-resolution \cite{Dong&amp;etal:16TPAMI,Kim&amp;Lee&amp;Lee:16CVPR}, light-field reconstruction \cite{Chun&amp;etal:20PAMI, Huang&amp;etal:20ICASSP}, object localization \cite{Szegedy&amp;Toshev&amp;Erhan:13NIPS}, end-to-end autonomous driving \cite{Bojarski&amp;etal:16NIPS-DLS}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1604.07316" id="1776">
-        <attvalues>
-          <attvalue for="5" value=" Deep regression neural network (NN)-based methods that can accurately predict real- or complex-valued output have been rapidly gaining popularity in a wide range of computational imaging and computer vision applications including image denoising \cite{Vincent&amp;etal:10JMLR, Xie&amp;Xu&amp;Chen:12NIPS, Zhang&amp;etal:17TIP}, image deblurring \cite{Xu&amp;etal:14NIPS}, image super-resolution \cite{Dong&amp;etal:16TPAMI,Kim&amp;Lee&amp;Lee:16CVPR}, light-field reconstruction \cite{Chun&amp;etal:20PAMI, Huang&amp;etal:20ICASSP}, object localization \cite{Szegedy&amp;Toshev&amp;Erhan:13NIPS}, end-to-end autonomous driving \cite{Bojarski&amp;etal:16NIPS-DLS}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1711.10925" id="1777">
-        <attvalues>
-          <attvalue for="5" value=" All the aforementioned methods except for \cite{Ulyanov&amp;etal:18CVPR} have been developed based on some noise assumptions including pixel-wise independent noise \cite{Krull&amp;etal:19CVPR, Xie&amp;Wang&amp;Ji:20NIPS, Huang&amp;eta:21CVPR}, zero-mean noise \cite{Lehtinen&amp;etal:18ICML, Batson&amp;Royer:19ICML, Xie&amp;Wang&amp;Ji:20NIPS}, and \dquotes{weak} noise \cite{Xu&amp;etal:20TIP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1901.11365" id="1778">
-        <attvalues>
-          <attvalue for="5" value=" Section~\ref{sec:result} demonstrates the outperforming performance of the proposed SSRL approach over the several existing self-supervised denoising methods, Noise2Self \cite{Batson&amp;Royer:19ICML}, Noise2Inverse \cite{Hendriksen&amp;etal:20TCI} or Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, Noisy-As-Claen~\cite{Xu&amp;etal:20TIP}, and Self2Self~\cite{Quan&amp;etal:20CVPR}, with low-dose CT and camera image denoising experiments using both real and synthetic datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1811.10980" id="1779">
-        <attvalues>
-          <attvalue for="5" value=" All the aforementioned methods except for \cite{Ulyanov&amp;etal:18CVPR} have been developed based on some noise assumptions including pixel-wise independent noise \cite{Krull&amp;etal:19CVPR, Xie&amp;Wang&amp;Ji:20NIPS, Huang&amp;eta:21CVPR}, zero-mean noise \cite{Lehtinen&amp;etal:18ICML, Batson&amp;Royer:19ICML, Xie&amp;Wang&amp;Ji:20NIPS}, and \dquotes{weak} noise \cite{Xu&amp;etal:20TIP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="2010.11971" id="1780">
-        <attvalues>
-          <attvalue for="5" value=" Section~\ref{sec:result} demonstrates the outperforming performance of the proposed SSRL approach over the several existing self-supervised denoising methods, Noise2Self \cite{Batson&amp;Royer:19ICML}, Noise2Inverse \cite{Hendriksen&amp;etal:20TCI} or Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, Noisy-As-Claen~\cite{Xu&amp;etal:20TIP}, and Self2Self~\cite{Quan&amp;etal:20CVPR}, with low-dose CT and camera image denoising experiments using both real and synthetic datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1803.04189" id="1781">
-        <attvalues>
-          <attvalue for="5" value=" All the aforementioned methods except for \cite{Ulyanov&amp;etal:18CVPR} have been developed based on some noise assumptions including pixel-wise independent noise \cite{Krull&amp;etal:19CVPR, Xie&amp;Wang&amp;Ji:20NIPS, Huang&amp;eta:21CVPR}, zero-mean noise \cite{Lehtinen&amp;etal:18ICML, Batson&amp;Royer:19ICML, Xie&amp;Wang&amp;Ji:20NIPS}, and \dquotes{weak} noise \cite{Xu&amp;etal:20TIP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="2001.11801" id="1782">
-        <attvalues>
-          <attvalue for="5" value=" Section~\ref{sec:result} demonstrates the outperforming performance of the proposed SSRL approach over the several existing self-supervised denoising methods, Noise2Self \cite{Batson&amp;Royer:19ICML}, Noise2Inverse \cite{Hendriksen&amp;etal:20TCI} or Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, Noisy-As-Claen~\cite{Xu&amp;etal:20TIP}, and Self2Self~\cite{Quan&amp;etal:20CVPR}, with low-dose CT and camera image denoising experiments using both real and synthetic datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="2101.02824" id="1783">
-        <attvalues>
-          <attvalue for="5" value=" Section~\ref{sec:result} demonstrates the outperforming performance of the proposed SSRL approach over the several existing self-supervised denoising methods, Noise2Self \cite{Batson&amp;Royer:19ICML}, Noise2Inverse \cite{Hendriksen&amp;etal:20TCI} or Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, Noise2Same \cite{Xie&amp;Wang&amp;Ji:20NIPS}, Noisy-As-Claen~\cite{Xu&amp;etal:20TIP}, and Self2Self~\cite{Quan&amp;etal:20CVPR}, with low-dose CT and camera image denoising experiments using both real and synthetic datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1803.01314" id="1784">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by Noise2Noise, several self-supervised image denoising methods such as Noise2Inverse \cite{Hendriksen&amp;etal:20TCI}, Neighbor2Neighbor \cite{Huang&amp;eta:21CVPR}, and \cite{Soltanayev&amp;Chun:18NIPS, Moran&amp;etal:20CVPR, Xu&amp;etal:20TIP} emulate pairs of two independent noisy images, by applying partitioning or adding simulated noise to single noisy measurements." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="1906.00651" id="1785">
-        <attvalues>
-          <attvalue for="5" value=" All the aforementioned methods are blind in a sense that one does not need to estimate noise parameters, whereas some works estimate statistical parameters of noise, such as noise histogram \cite{Krull&amp;etal:20FCS} and parameters of Gaussian mixture noise model \cite{Prakash&amp;Krull&amp;Jug:21ICLR}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.04821" target="2006.06072" id="1786">
-        <attvalues>
-          <attvalue for="5" value=" All the aforementioned methods are blind in a sense that one does not need to estimate noise parameters, whereas some works estimate statistical parameters of noise, such as noise histogram \cite{Krull&amp;etal:20FCS} and parameters of Gaussian mixture noise model \cite{Prakash&amp;Krull&amp;Jug:21ICLR}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00272" target="1505.00880" id="1787">
-        <attvalues>
-          <attvalue for="5" value=" Prior work \cite{mvcnn15, RotationNet2018, TCL2018, ModelNet40} has employed multi-view deep networks to learn aggregated multi-view representations capturing the variability in object appearance under different pose transformations." />
-        </attvalues>
-      </edge>
-      <edge source="2404.17170" target="2004.05508" id="1788">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, Zhu\cite{9156932} proposes a model that employs meta-learning to capture shared prior knowledge among different distortions." />
-        </attvalues>
-      </edge>
-      <edge source="2404.17170" target="2207.02595" id="1789">
-        <attvalues>
-          <attvalue for="5" value=" There already exist some data-efficient methods for quality assessment, such as FAST-VQA\cite{Wu2022FASTVQAEE} with random-crop." />
-        </attvalues>
-      </edge>
-      <edge source="2404.17170" target="1905.07481" id="1790">
-        <attvalues>
-          <attvalue for="5" value=" Tran\cite{9070152} makes some theoretical works and gradually finds some multidimensional properties of CL." />
-        </attvalues>
-      </edge>
-      <edge source="2404.17170" target="2207.11972" id="1791">
-        <attvalues>
-          <attvalue for="5" value=" A recent work \cite{9841016} by Chong Mou et al." />
-        </attvalues>
-      </edge>
-      <edge source="1907.03326" target="1810.03783" id="1792">
-        <attvalues>
-          <attvalue for="5" value=" Zhuo \etal \cite{zhuo2018unsupervised} build salient motion masks that are further combined with &quot;objectness&quot; masks in order to generate the final segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="1703.05884" id="1793">
-        <attvalues>
-          <attvalue for="5" value=" Afterwards, tracking benchmarks are introduced to evaluate different aspects of tracking such as, the impact of color information~\cite{TempleColor}, fast target motion~\cite{Galoogahi_2017_ICCV_NFS} as well as tracking in aerial imagery~\cite{Mueller_2016_ECCV_UAV123}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="1803.10794" id="1794">
-        <attvalues>
-          <attvalue for="5" value=" More recently, the tracking community has focused on constructing datasets~\cite{2018_Muller_Trackingnet, Fan_2019_CVPR_Lasot,Huang_2021_TPAMI_GOT10k} with large-scale training splits, to benefit from task-specific deep learning." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="1810.11981" id="1795">
-        <attvalues>
-          <attvalue for="5" value=" Among these, GOT-10K~\cite{Huang_2021_TPAMI_GOT10k} comprises a large collection of shorter videos, whereas LaSOT~\cite{Fan_2019_CVPR_Lasot} focuses on longer sequences." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="1809.07845" id="1796">
-        <attvalues>
-          <attvalue for="5" value=" Among these, GOT-10K~\cite{Huang_2021_TPAMI_GOT10k} comprises a large collection of shorter videos, whereas LaSOT~\cite{Fan_2019_CVPR_Lasot} focuses on longer sequences." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="1503.01313" id="1797">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, there exist dedicated benchmarks, such as the VOT series~\cite{Kristan_2016_TPAMI_VOT} associated with annual tracking challenge competitions." />
-        </attvalues>
-      </edge>
-      <edge source="2208.06888" target="2302.02814" id="1798">
-        <attvalues>
-          <attvalue for="5" value=" The best method, MixFormer-22k \cite{Cui_2022_CVPR_Mixformer} which employs an ImageNet-22K pre-trained backbone achieves an AUC score of only 56." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2104.10729" id="1799">
-        <attvalues>
-          <attvalue for="5" value=" Low-light image enhancement (LLIE) is a critical task in computer vision and is essential for various applications ranging from surveillance to autonomous driving\cite{li2021low}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2404.08965" id="1800">
-        <attvalues>
-          <attvalue for="5" value=" Images captured in low-light environments often suffer from low contrast and loss of detail, making downstream tasks such as object or text detection, semantic segmentation, and others highly challenging\cite{xu2024seeing}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2403.02879" id="1801">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, to further enhance various visual applications in poor environments, low-light image enhancement tasks have received extensive attention from researchers\cite{he2024zero,xue2024low,singh2024illumination}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2401.03788" id="1802">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, to further enhance various visual applications in poor environments, low-light image enhancement tasks have received extensive attention from researchers\cite{he2024zero,xue2024low,singh2024illumination}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2209.02197" id="1803">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, some methods\cite{zhang2023multi,zhang2023lrt,xu2023low,cai2023retinexformer} significantly improve the performance of low-light image enhancement by learning the mapping between low-light and normal images in a data-driven way." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2305.05839" id="1804">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, some methods\cite{zhang2023multi,zhang2023lrt,xu2023low,cai2023retinexformer} significantly improve the performance of low-light image enhancement by learning the mapping between low-light and normal images in a data-driven way." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2303.06705" id="1805">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, some methods\cite{zhang2023multi,zhang2023lrt,xu2023low,cai2023retinexformer} significantly improve the performance of low-light image enhancement by learning the mapping between low-light and normal images in a data-driven way." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2006.11239" id="1806">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the diffusion model \cite{ho2020denoising,nichol2021improved} has received much attention for its remarkable performance in generative tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2102.09672" id="1807">
-        <attvalues>
-          <attvalue for="5" value=" Recently, the diffusion model \cite{ho2020denoising,nichol2021improved} has received much attention for its remarkable performance in generative tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2304.01247" id="1808">
-        <attvalues>
-          <attvalue for="5" value=" \cite{fei2023generative} introduced the diffusion model into the low-light image enhancement task to improve the recovery of image details and textures in low-light conditions." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2310.17577" id="1809">
-        <attvalues>
-          <attvalue for="5" value=" \ref{sample}, our method shows a significant improvement over the current state-of-the-art method \cite{hou2024global}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2404.19756" id="1810">
-        <attvalues>
-          <attvalue for="5" value=" \par The recent introduction of Kolmogorov-Arnold Networks\cite{liu2024kan} has raised hopes of opening the black box of traditional networks\cite{yu2023white}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2306.01129" id="1811">
-        <attvalues>
-          <attvalue for="5" value=" \par The recent introduction of Kolmogorov-Arnold Networks\cite{liu2024kan} has raised hopes of opening the black box of traditional networks\cite{yu2023white}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03404" target="2406.02918" id="1812">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2024u} was the first to introduce KANs into visual tasks, reformulating U-Net as U-KAN to improve medical image segmentation and generation." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04712" target="2008.06963" id="1813">
-        <attvalues>
-          <attvalue for="5" value=" In existing methods~\cite{zhao2020not, he2020guided}, external networks are used or an attention map is obtained by performing additional operations upon query and gallery image to find the target." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04712" target="1904.04975" id="1814">
-        <attvalues>
-          <attvalue for="5" value=" In order to obtain only the target features, most methods~\cite{he2019foreground, miao2019pose} use attention maps or pose information generated by other additional networks such as pose estimation and human parsing networks." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04712" target="1711.09349" id="1815">
-        <attvalues>
-          <attvalue for="5" value=" Unlike the previous method~\cite{sun2018beyond} of simply concatenating the part features, which causes feature misalignment, our model predicts the weights through relational information between various features and corrects irrelevant information to create the final representation." />
-        </attvalues>
-      </edge>
-      <edge source="2004.11969" target="1708.03852" id="1816">
-        <attvalues>
-          <attvalue for="5" value=" Existing VIO methods \cite{mourikis2007multi,qin2018vins,leutenegger2015keyframe} can estimate pose accurately but only provide a sparse map of 3D points." />
-        </attvalues>
-      </edge>
-      <edge source="2004.11969" target="1903.01067" id="1817">
-        <attvalues>
-          <attvalue for="5" value=" Our proposed method utilizes structural line to improves the correctness and robustness of plane detection and mesh generation compared to the method involving only sparse points\cite{rosinol2019incremental}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.11969" target="1711.08654" id="1818">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, line features incorporated into the SLAM system can improve pose accuracy and the robustness under illumination-changing scenes \cite{he2018pl,pumarola2017pl,zuo2017robust}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.11969" target="1909.04102" id="1819">
-        <attvalues>
-          <attvalue for="5" value=" Planar constraints used in these SLAM system are more often relies on the depth sensor\cite{ataer2013tracking,kaess2015simultaneous,yang2019tightly,zuo2019lic}." />
-        </attvalues>
-      </edge>
-      <edge source="2004.11969" target="1805.05876" id="1820">
-        <attvalues>
-          <attvalue for="5" value="\cite{yang2018observability} investigated the observability of the VIO using heterogeneous features of points, lines, and planes." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14997" target="1804.08361" id="1821">
-        <attvalues>
-          <attvalue for="5" value=" These techniques can be broadly categorized into three groups: auto-encoder (AE)-based \cite{li2018densefuse, li2021rfn, Reviewer2-3, 9502544, 9127964, 9349250}, convolutional neural network (CNN)-based \cite{liu2021smoa, ZHANG202099, xu2020u2fusion, di2022unsupervised, li2021different, liu2022coconet}, and generative adversarial network (GAN)-based methods \cite{ma2019fusiongan, ma2020ganmcc, 9894670, liu2022target, Reviewer2-1, Reviewer2-2}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14997" target="2003.09210" id="1822">
-        <attvalues>
-          <attvalue for="5" value=" DIDFuse~\cite{Reviewer2-3} pioneered the application of a decomposition network with a dual-stream structure to extract high and low-frequency features." />
-        </attvalues>
-      </edge>
-      <edge source="2309.14997" target="2211.14461" id="1823">
-        <attvalues>
-          <attvalue for="5" value=" Subsequently, SwinFusion~\cite{Reviewer1-1} and CDDFuse~\cite{Reviewer2-4} incorporate the Transformer structure, which models long-range dependencies, into the fusion network." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1406.4216" id="1824">
-        <attvalues>
-          <attvalue for="5" value=" It is a kernalized (non-linear) counterpart of XQDA\cite{LOMO}, which is one of the most popularly applied metric learning method in re-ID literature." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1807.10908" id="1825">
-        <attvalues>
-          <attvalue for="5" value=" Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1603.02139" id="1826">
-        <attvalues>
-          <attvalue for="5" value=" Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1806.09695" id="1827">
-        <attvalues>
-          <attvalue for="5" value=" Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1910.03923" id="1828">
-        <attvalues>
-          <attvalue for="5" value=" Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1910.03905" id="1829">
-        <attvalues>
-          <attvalue for="5" value=" Hence the distance metric learning methods\cite{NK3ML,LOMO,rPcca,Zheng:nfst,IRS,SSSVM,MFML,SemiNK3ML} are used to learn a better similarity measure such that, irrespective of the view, same class samples are closer and distinct class samples are well separated." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1709.05165" id="1830">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, though deep learning methods \cite{ImprDeep,MuDeep,PTGAN,Beyond:triplet_loss, DGD, TCP, SpindleNet, SLSTM, SCNN} have made good improvement in re-ID performance, they have a fundamental limitation in practical deployment as they need a large, annotated training data." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1711.08565" id="1831">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, though deep learning methods \cite{ImprDeep,MuDeep,PTGAN,Beyond:triplet_loss, DGD, TCP, SpindleNet, SLSTM, SCNN} have made good improvement in re-ID performance, they have a fundamental limitation in practical deployment as they need a large, annotated training data." />
-        </attvalues>
-      </edge>
-      <edge source="1909.11316" target="1704.01719" id="1832">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, though deep learning methods \cite{ImprDeep,MuDeep,PTGAN,Beyond:triplet_loss, DGD, TCP, SpindleNet, SLSTM, SCNN} have made good improvement in re-ID performance, they have a fundamental limitation in practical deployment as they need a large, annotated training data." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04613" target="2002.05709" id="1833">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised contrastive models such as SimCLR \cite{chen2020simple} and MoCo \cite{he2020momentum} have been shown to approach or surpass the performance of supervised models when representations are transferred to downstream image classification, object detection, and semantic segmentation tasks \cite{ericsson2021well}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04613" target="1911.05722" id="1834">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised contrastive models such as SimCLR \cite{chen2020simple} and MoCo \cite{he2020momentum} have been shown to approach or surpass the performance of supervised models when representations are transferred to downstream image classification, object detection, and semantic segmentation tasks \cite{ericsson2021well}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04613" target="2011.13377" id="1835">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised contrastive models such as SimCLR \cite{chen2020simple} and MoCo \cite{he2020momentum} have been shown to approach or surpass the performance of supervised models when representations are transferred to downstream image classification, object detection, and semantic segmentation tasks \cite{ericsson2021well}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.04613" target="2102.08318" id="1836">
-        <attvalues>
-          <attvalue for="5" value=" In particular, we conduct an empirical study of the state-of-the-art detection pretext task InsLoc \cite{yang2021instance} and strategically alter the views created from InsLoc's data augmentation pipeline by adjusting cropping, adding shortcut-reducing appearance augmentations, and integrating saliency-based object priors." />
-        </attvalues>
-      </edge>
-      <edge source="2306.02894" target="2106.01226" id="1837">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we propose the recyclable semi-supervised method\cite{chen2021semi}\cite{hung2018adversarial} based on multi-model ensemble." />
-        </attvalues>
-      </edge>
-      <edge source="2306.02894" target="1802.07934" id="1838">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we propose the recyclable semi-supervised method\cite{chen2021semi}\cite{hung2018adversarial} based on multi-model ensemble." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1611.09571" id="1839">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1609.01064" id="1840">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1411.1045" id="1841">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1804.01793" id="1842">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1704.06904" id="1843">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1809.01125" id="1844">
-        <attvalues>
-          <attvalue for="5" value=" Not only is this a common assumption to all the CNN-based methods \cite{SALICON,TORONTO,MIT300,DEEPGAZE2,SAMRes,MLNet,DEEPGAZE1,PDP}, but it also can be used in an extremely wide range of applications \cite{Zund13,Ancuti11,Zhang16,Wang17,Jia17b,Hu18,Li18}." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1604.03605" id="1845">
-        <attvalues>
-          <attvalue for="5" value=" The study \cite{Bylinskii19} shows a centered Gaussian distribution can only achieve an S-AUC score of $0." />
-        </attvalues>
-      </edge>
-      <edge source="2002.10540" target="1704.08615" id="1846">
-        <attvalues>
-          <attvalue for="5" value=" Later this idea was extended to a metric-tailored design \cite{Matthias18}, saliency models and saliency maps are decoupled so that one model can output different metric specific maps." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1609.03677" id="1847">
-        <attvalues>
-          <attvalue for="5" value=" Depth estimation attempts to infer the distance between the objects and the camera in the captured scene, which is a classical but important vision problem for many upstream 3D applications, such as scene reconstruction, semantic understanding, robotics navigation, etc Traditionally, researchers focus on the problem of monocular depth estimation \cite{monodepth2017}, which learns the traditional 2D content captured by typical pinhole projection model-based cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1904.08103" id="1848">
-        <attvalues>
-          <attvalue for="5" value=" To obtain the novel information (360$^\circ$ sensing) of a 3D scene, they integrate those single views based on multi-view consistency \cite{gmvs2019,yao2018mvsnet} and structure-from-motion (SfM) \cite{sfm2018,schonberger2016structure}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1804.02505" id="1849">
-        <attvalues>
-          <attvalue for="5" value=" To obtain the novel information (360$^\circ$ sensing) of a 3D scene, they integrate those single views based on multi-view consistency \cite{gmvs2019,yao2018mvsnet} and structure-from-motion (SfM) \cite{sfm2018,schonberger2016structure}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1406.2283" id="1850">
-        <attvalues>
-          <attvalue for="5" value=" However, as the best way to display the panorama image is using the sphere (a non-Euclidean space), the existing studies of monocular depth estimation based on convolutional neural networks (CNNs) \cite{eigen2014depth,laina2016deeper,zhou2017unsupervised} cannot be directly applied to its depth task." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1606.00373" id="1851">
-        <attvalues>
-          <attvalue for="5" value=" However, as the best way to display the panorama image is using the sphere (a non-Euclidean space), the existing studies of monocular depth estimation based on convolutional neural networks (CNNs) \cite{eigen2014depth,laina2016deeper,zhou2017unsupervised} cannot be directly applied to its depth task." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1704.07813" id="1852">
-        <attvalues>
-          <attvalue for="5" value=" However, as the best way to display the panorama image is using the sphere (a non-Euclidean space), the existing studies of monocular depth estimation based on convolutional neural networks (CNNs) \cite{eigen2014depth,laina2016deeper,zhou2017unsupervised} cannot be directly applied to its depth task." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1901.03861" id="1853">
-        <attvalues>
-          <attvalue for="5" value=" Equirectangular projection provides a wide FoV mimicking a peripheral vision but introduces distortion \cite{sun2019horizonnet,su2017learning,zioulis2018omnidepth}, while cube map projection provides a smaller and non-distorted FoV mimicking the foveal vision but introduces discontinuity \cite{cheng2018cube,jiang2021unifuse,jin2020geometric}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13714" target="1807.09620" id="1854">
-        <attvalues>
-          <attvalue for="5" value=" Equirectangular projection provides a wide FoV mimicking a peripheral vision but introduces distortion \cite{sun2019horizonnet,su2017learning,zioulis2018omnidepth}, while cube map projection provides a smaller and non-distorted FoV mimicking the foveal vision but introduces discontinuity \cite{cheng2018cube,jiang2021unifuse,jin2020geometric}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.13107" target="1808.04560" id="1855">
-        <attvalues>
-          <attvalue for="5" value=" Prior works mainly rely on the paired data for supervised learning~\cite{Chen2018Retinex, Wang_2019_CVPR, kind++, liu2021ruas}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.13107" target="2012.15052" id="1856">
-        <attvalues>
-          <attvalue for="5" value=" &#09;For example, both DPE~\cite{Chen_2018_CVPR_DPE} and QAGAN~\cite{QAGAN} adopt the two-way GAN~\cite{CycleGAN2017} framework to implement the user-oriented image enhancement with unpaired training set." />
-        </attvalues>
-      </edge>
-      <edge source="2112.13107" target="1703.10593" id="1857">
-        <attvalues>
-          <attvalue for="5" value=" &#09;For example, both DPE~\cite{Chen_2018_CVPR_DPE} and QAGAN~\cite{QAGAN} adopt the two-way GAN~\cite{CycleGAN2017} framework to implement the user-oriented image enhancement with unpaired training set." />
-        </attvalues>
-      </edge>
-      <edge source="2112.13107" target="1906.06972" id="1858">
-        <attvalues>
-          <attvalue for="5" value=" &#09;EnlightenGAN~\cite{jiang2021enlightengan} is the first to successfully apply the unpaired learning for low-light image enhancement." />
-        </attvalues>
-      </edge>
-      <edge source="2112.13107" target="2001.06826" id="1859">
-        <attvalues>
-          <attvalue for="5" value="~\cite{Zero-DCE} recently propose to train a lightweight CNN to estimate the image-specific and pixel-wise curves for dynamic range adjustment without the requirement of any paired or unpaired data." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1702.05464" id="1860">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised Domain Adaptation (UDA) approaches tackle the domain shift problem by aligning the training (source) and test (target) distributions, and can be roughly divided into statistical matching ~\cite{dip, manifold, distribution, DDC, JDA, DAN, DANN,ADDA,jingjing} or adversarial learning~\cite{DANN,ADDA,JAN,DRCN} methods." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1505.07818" id="1861">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised Domain Adaptation (UDA) approaches tackle the domain shift problem by aligning the training (source) and test (target) distributions, and can be roughly divided into statistical matching ~\cite{dip, manifold, distribution, DDC, JDA, DAN, DANN,ADDA,jingjing} or adversarial learning~\cite{DANN,ADDA,JAN,DRCN} methods." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1605.06636" id="1862">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised Domain Adaptation (UDA) approaches tackle the domain shift problem by aligning the training (source) and test (target) distributions, and can be roughly divided into statistical matching ~\cite{dip, manifold, distribution, DDC, JDA, DAN, DANN,ADDA,jingjing} or adversarial learning~\cite{DANN,ADDA,JAN,DRCN} methods." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1607.03516" id="1863">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised Domain Adaptation (UDA) approaches tackle the domain shift problem by aligning the training (source) and test (target) distributions, and can be roughly divided into statistical matching ~\cite{dip, manifold, distribution, DDC, JDA, DAN, DANN,ADDA,jingjing} or adversarial learning~\cite{DANN,ADDA,JAN,DRCN} methods." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1804.10427" id="1864">
-        <attvalues>
-          <attvalue for="5" value=" While promising, existing OUDA approaches~\cite{ATI,OSBP,mahsa,STA,Attract-UTS} lack an essential theoretical analysis of the aforementioned partial risks and the upper bound for the target risk, thus omitting potential solutions for improvement and leading to a biased solution." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1805.12277" id="1865">
-        <attvalues>
-          <attvalue for="5" value=" While promising, existing OUDA approaches~\cite{ATI,OSBP,mahsa,STA,Attract-UTS} lack an essential theoretical analysis of the aforementioned partial risks and the upper bound for the target risk, thus omitting potential solutions for improvement and leading to a biased solution." />
-        </attvalues>
-      </edge>
-      <edge source="2006.12087" target="1908.01925" id="1866">
-        <attvalues>
-          <attvalue for="5" value=" While promising, existing OUDA approaches~\cite{ATI,OSBP,mahsa,STA,Attract-UTS} lack an essential theoretical analysis of the aforementioned partial risks and the upper bound for the target risk, thus omitting potential solutions for improvement and leading to a biased solution." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1405.0312" id="1867">
-        <attvalues>
-          <attvalue for="5" value=" Public datasets, such as PASCAL VOC\cite{Everingham}, MS COCO\cite{lin}, ImageNet\cite{deng} can be used to train a general object detector." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1804.06516" id="1868">
-        <attvalues>
-          <attvalue for="5" value=" One popular way for overcoming these problems is rendering scenes and objects using 3D models \cite{tremblay}\cite{prakash}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1810.10093" id="1869">
-        <attvalues>
-          <attvalue for="5" value=" One popular way for overcoming these problems is rendering scenes and objects using 3D models \cite{tremblay}\cite{prakash}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1702.07836" id="1870">
-        <attvalues>
-          <attvalue for="5" value=" superimpose 2D images of textured object models into images of real indoor environments \cite{georgakis}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1708.01642" id="1871">
-        <attvalues>
-          <attvalue for="5" value=" present a method to make detectors ignore the artifacts during training \cite{dwibedi}." />
-        </attvalues>
-      </edge>
-      <edge source="2207.07867" target="1909.11972" id="1872">
-        <attvalues>
-          <attvalue for="5" value=" \cite{yun} introduces an approach to diversify the foreground with GAN-based seed images in order to balance domain gaps." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1804.08361" id="1873">
-        <attvalues>
-          <attvalue for="5" value=" They compared this approach with a strategy involving training a novel network architecture (DenseFuse) \cite{li2018densefuse}." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1808.10072" id="1874">
-        <attvalues>
-          <attvalue for="5" value=" The methods most commonly used for image fusion are multi-scale transforms \cite{chen2020infrared}\cite{vishwakarma2018image} and representation learning based methods \cite{li2018densefuse}\cite{borsoi2019super}." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1804.08355" id="1875">
-        <attvalues>
-          <attvalue for="5" value=" In contrast to another LRR-based fusion method~\cite{li2017multi}, which calculates the low-rank coefficients for each image, in our method, the LatLRR problem is only solved once in the training phase to learn a projection matrix $L$." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1704.05240" id="1876">
-        <attvalues>
-          <attvalue for="5" value=" Other options include the joint sparse representation \cite{liu2017infrared} and cosparse representation \cite{gao2017image}." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1712.07384" id="1877">
-        <attvalues>
-          <attvalue for="5" value=" \cite{prabhakar2017deepfuse} proposed a novel CNN-based fusion framework for a multi-exposure image fusion task." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1804.06992" id="1878">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018infrared}\cite{li2019infrared} proposed two novel fusion frameworks based on a pretrained network(VGG-19 \cite{simonyan2014very} and ResNet-50 \cite{he2016deep})." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1806.07119" id="1879">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018infrared}\cite{li2019infrared} proposed two novel fusion frameworks based on a pretrained network(VGG-19 \cite{simonyan2014very} and ResNet-50 \cite{he2016deep})." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1409.1556" id="1880">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018infrared}\cite{li2019infrared} proposed two novel fusion frameworks based on a pretrained network(VGG-19 \cite{simonyan2014very} and ResNet-50 \cite{he2016deep})." />
-        </attvalues>
-      </edge>
-      <edge source="1811.02291" target="1512.03385" id="1881">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018infrared}\cite{li2019infrared} proposed two novel fusion frameworks based on a pretrained network(VGG-19 \cite{simonyan2014very} and ResNet-50 \cite{he2016deep})." />
-        </attvalues>
-      </edge>
-      <edge source="1909.12939" target="1706.07567" id="1882">
-        <attvalues>
-          <attvalue for="5" value=" A wide variety of approaches have been proposed for distance metric learning such as contrastive loss \cite{hadsell2006dimensionality}, triplet loss \cite{wu2017sampling, ge2018deep} and NormSoftmax loss \cite{zhai2018making}." />
-        </attvalues>
-      </edge>
-      <edge source="1909.12939" target="1810.06951" id="1883">
-        <attvalues>
-          <attvalue for="5" value=" (3) Inspired by \cite{ge2018deep}, We propose an online anchor nearest neighbor sampling (OANNS) method which addresses the batch sampling issue for large scale training data." />
-        </attvalues>
-      </edge>
-      <edge source="2402.17319" target="2206.08367" id="1884">
-        <attvalues>
-          <attvalue for="5" value=" It introduces SHIFT dataset \cite{sun2022shift}, a multi-task driving dataset featuring the most important perception tasks under a variety of conditions and with a comprehensive sensor setup." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2311.11465" id="1885">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1605.06211" id="1886">
-        <attvalues>
-          <attvalue for="5" value=" Long \etal~\cite{long2015fully} transpose feature extraction layers to a segmentation task and update parameters by fine tuning." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1505.04597" id="1887">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1511.00561" id="1888">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2303.11717" id="1889">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2304.06488" id="1890">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation methods using convolutional neural networks \cite{zhang2023understanding} have tried to identify visual texture and corresponding surface material type of objects \cite{long2015fully} \cite{ronneberger2015u} \cite{badrinarayanan2017segnet} \cite{zhang2023complete} \cite{zhang2023one}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1612.01105" id="1891">
-        <attvalues>
-          <attvalue for="5" value=" At the same time, it designs a novel structure that combines low-level and high-level semantics, which is extensively utilized by following semantic segmentation networks such as \cite{zhao2017pyramid}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1711.07971" id="1892">
-        <attvalues>
-          <attvalue for="5" value=" Non-local operators \cite{wang2018non} based on a self-attention mechanism \cite{chorowski2015attention} are applied in semantic segmentation \cite{fu2019dual}\cite{babiloni2020tesa}\cite{zhang2023faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1506.07503" id="1893">
-        <attvalues>
-          <attvalue for="5" value=" Non-local operators \cite{wang2018non} based on a self-attention mechanism \cite{chorowski2015attention} are applied in semantic segmentation \cite{fu2019dual}\cite{babiloni2020tesa}\cite{zhang2023faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1809.02983" id="1894">
-        <attvalues>
-          <attvalue for="5" value=" Non-local operators \cite{wang2018non} based on a self-attention mechanism \cite{chorowski2015attention} are applied in semantic segmentation \cite{fu2019dual}\cite{babiloni2020tesa}\cite{zhang2023faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2306.14289" id="1895">
-        <attvalues>
-          <attvalue for="5" value=" Non-local operators \cite{wang2018non} based on a self-attention mechanism \cite{chorowski2015attention} are applied in semantic segmentation \cite{fu2019dual}\cite{babiloni2020tesa}\cite{zhang2023faster}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2112.04108" id="1896">
-        <attvalues>
-          <attvalue for="5" value=" Song \etal~\cite{song2022fully} propose fully attentional blocks to encode spatial and channel attentions in similarity graphs with high computational efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2003.13948" id="1897">
-        <attvalues>
-          <attvalue for="5" value=" Xie \etal~\cite{xie2020segmenting} propose a new benchmark for glass object segmentation and considers the characteristics of transparent object boundary features." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2108.11117" id="1898">
-        <attvalues>
-          <attvalue for="5" value=" Recently, glass surface object segmentation methods are proposed including transparent objects \cite{xie2020segmenting} \cite{zheng2022glassnet} \cite{he2021enhanced} \cite{he2021enhanced} and strongly reflective objects such as mirror \cite{guan2022learning} \cite{mei2021depth} \cite{yang2019my} \cite{han2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2103.15734" id="1899">
-        <attvalues>
-          <attvalue for="5" value=" EBLNet \cite{he2021enhanced} extracts boundary features employing a non-local approach." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1908.09101" id="1900">
-        <attvalues>
-          <attvalue for="5" value=" Recently, glass surface object segmentation methods are proposed including transparent objects \cite{xie2020segmenting} \cite{zheng2022glassnet} \cite{he2021enhanced} \cite{he2021enhanced} and strongly reflective objects such as mirror \cite{guan2022learning} \cite{mei2021depth} \cite{yang2019my} \cite{han2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2305.00278" id="1901">
-        <attvalues>
-          <attvalue for="5" value=" Recently, glass surface object segmentation methods are proposed including transparent objects \cite{xie2020segmenting} \cite{zheng2022glassnet} \cite{he2021enhanced} \cite{he2021enhanced} and strongly reflective objects such as mirror \cite{guan2022learning} \cite{mei2021depth} \cite{yang2019my} \cite{han2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2101.08461" id="1902">
-        <attvalues>
-          <attvalue for="5" value=" Xie \etal~\cite{xie2021segmenting} employ attentional mechanisms to explore relationships among glass objects of different categories." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1806.05376" id="1903">
-        <attvalues>
-          <attvalue for="5" value=" RCAM\cite{lin2021rich} guides the network to identify reflections based on the feature information obtained from the reflection removal network\cite{zhang2018single}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2301.07431" id="1904">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, external boundaries have played a crucial role in salient object segmentation\cite{wei2020f3net}\cite{zhu2023sharp}\cite{qiao2023robustness}, especially in medical imaging, such as CT image segmentation\cite{ghadimi2015skull} \cite{shojaii2005automatic} \cite{bai2016automatic}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="2306.07713" id="1905">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, external boundaries have played a crucial role in salient object segmentation\cite{wei2020f3net}\cite{zhu2023sharp}\cite{qiao2023robustness}, especially in medical imaging, such as CT image segmentation\cite{ghadimi2015skull} \cite{shojaii2005automatic} \cite{bai2016automatic}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.00212" target="1512.03385" id="1906">
-        <attvalues>
-          <attvalue for="5" value=" To this end, we construct Internal-External Boundary Attention Module (IEBAM) with step-by-step supervision improving the feature fusion of residual networks\cite{he2016deep}." />
-        </attvalues>
-      </edge>
-      <edge source="1810.03243" target="1807.07987" id="1907">
-        <attvalues>
-          <attvalue for="5" value=" Recently, convolutional neural network (CNN) \cite{lecun2015deep} is revolutionizing objection detection field, like Mask R-CNN \cite{he2017mask} and YOLO \cite{redmon2018yolov3}." />
-        </attvalues>
-      </edge>
-      <edge source="1810.03243" target="1703.06870" id="1908">
-        <attvalues>
-          <attvalue for="5" value=" Recently, convolutional neural network (CNN) \cite{lecun2015deep} is revolutionizing objection detection field, like Mask R-CNN \cite{he2017mask} and YOLO \cite{redmon2018yolov3}." />
-        </attvalues>
-      </edge>
-      <edge source="1810.03243" target="1804.02767" id="1909">
-        <attvalues>
-          <attvalue for="5" value=" Recently, convolutional neural network (CNN) \cite{lecun2015deep} is revolutionizing objection detection field, like Mask R-CNN \cite{he2017mask} and YOLO \cite{redmon2018yolov3}." />
-        </attvalues>
-      </edge>
-      <edge source="1810.03243" target="1608.07470" id="1910">
-        <attvalues>
-          <attvalue for="5" value=" In addition, both \cite{fornaciari2014fast} and \cite{jia2017fast} require at least three arc segments to recover the ellipse model, which might disable the algorithms when handling the incomplete ellipses." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2201.08625" id="1911">
-        <attvalues>
-          <attvalue for="5" value=" The 'VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning' workshop \cite{VIP2th}\cite{VIP3th} introduces a challenge, urging participants to build generalizable models by fusing dataset-specific prior knowledge in resource-scarce contexts." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2305.19688" id="1912">
-        <attvalues>
-          <attvalue for="5" value=" The 'VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning' workshop \cite{VIP2th}\cite{VIP3th} introduces a challenge, urging participants to build generalizable models by fusing dataset-specific prior knowledge in resource-scarce contexts." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2110.00470" id="1913">
-        <attvalues>
-          <attvalue for="5" value=" In VIPriors instance segmentation challenge, \cite{2021sota}\cite{2022sota} introduced task-specific copy-paste augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2211.11282" id="1914">
-        <attvalues>
-          <attvalue for="5" value=" In VIPriors instance segmentation challenge, \cite{2021sota}\cite{2022sota} introduced task-specific copy-paste augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2107.06278" id="1915">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning-based instance segmentation methods have garnered widespread recognition, exemplified by approaches such as Cascade-MaskRCNN\cite{CascadeRCNN}, Maskformer\cite{Maskformer}, queryinst\cite{QueryInst}, and CBNet\cite{CBNet}, among others." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2105.01928" id="1916">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning-based instance segmentation methods have garnered widespread recognition, exemplified by approaches such as Cascade-MaskRCNN\cite{CascadeRCNN}, Maskformer\cite{Maskformer}, queryinst\cite{QueryInst}, and CBNet\cite{CBNet}, among others." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2107.00420" id="1917">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning-based instance segmentation methods have garnered widespread recognition, exemplified by approaches such as Cascade-MaskRCNN\cite{CascadeRCNN}, Maskformer\cite{Maskformer}, queryinst\cite{QueryInst}, and CBNet\cite{CBNet}, among others." />
-        </attvalues>
-      </edge>
-      <edge source="2403.11572" target="2012.07177" id="1918">
-        <attvalues>
-          <attvalue for="5" value=" Copy-paste augmentation\cite{CopyPaste} emerges as an effective augmentation strategy for instance segmentation task." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1711.10733" id="1919">
-        <attvalues>
-          <attvalue for="5" value=" The registration problem has endured thorough investigation from optimization aspects \cite{Bernard_2018_CVPR,bes1992method,dym2017ds++,forstner2017efficient,iglesias2020global,kezurer2015tight,le2019sdrsac,segal2009generalized,yang2019polynomial}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1705.06148" id="1920">
-        <attvalues>
-          <attvalue for="5" value=" The registration problem has endured thorough investigation from optimization aspects \cite{Bernard_2018_CVPR,bes1992method,dym2017ds++,forstner2017efficient,iglesias2020global,kezurer2015tight,le2019sdrsac,segal2009generalized,yang2019polynomial}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1904.03483" id="1921">
-        <attvalues>
-          <attvalue for="5" value=" The registration problem has endured thorough investigation from optimization aspects \cite{Bernard_2018_CVPR,bes1992method,dym2017ds++,forstner2017efficient,iglesias2020global,kezurer2015tight,le2019sdrsac,segal2009generalized,yang2019polynomial}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1903.08588" id="1922">
-        <attvalues>
-          <attvalue for="5" value=" The registration problem has endured thorough investigation from optimization aspects \cite{Bernard_2018_CVPR,bes1992method,dym2017ds++,forstner2017efficient,iglesias2020global,kezurer2015tight,le2019sdrsac,segal2009generalized,yang2019polynomial}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1603.08182" id="1923">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there are many development in 3D deep learning techniques \cite{3dmatch,deng20193d,choy2019fully,yang2020learning,valsesia2020learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1904.04281" id="1924">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there are many development in 3D deep learning techniques \cite{3dmatch,deng20193d,choy2019fully,yang2020learning,valsesia2020learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1904.12099" id="1925">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there are many development in 3D deep learning techniques \cite{3dmatch,deng20193d,choy2019fully,yang2020learning,valsesia2020learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="2005.01014" id="1926">
-        <attvalues>
-          <attvalue for="5" value=" There is also some combination of conventional registration optimization strategies and deep learning techniques in an end-to-end framework \cite{huang2020feature,choy2020deep,aoki2019pointnetlk,wang2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="2004.11540" id="1927">
-        <attvalues>
-          <attvalue for="5" value=" There is also some combination of conventional registration optimization strategies and deep learning techniques in an end-to-end framework \cite{huang2020feature,choy2020deep,aoki2019pointnetlk,wang2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1905.03304" id="1928">
-        <attvalues>
-          <attvalue for="5" value=" There is also some combination of conventional registration optimization strategies and deep learning techniques in an end-to-end framework \cite{huang2020feature,choy2020deep,aoki2019pointnetlk,wang2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1608.05143" id="1929">
-        <attvalues>
-          <attvalue for="5" value=" Data fusion of these different kinds of 3D sensors combines their advantages and is a cross-source point cloud registration problem \cite{huang2017coarse,huang2017systematic,huang2016coarse}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.02690" target="1610.07324" id="1930">
-        <attvalues>
-          <attvalue for="5" value=" Data fusion of these different kinds of 3D sensors combines their advantages and is a cross-source point cloud registration problem \cite{huang2017coarse,huang2017systematic,huang2016coarse}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04979" target="2306.03413" id="1931">
-        <attvalues>
-          <attvalue for="5" value=" However, existing methods \cite{zhang2023dvis, zhang2023dvis++} face several challenges: they struggle with finely segmenting object edges, often misclassifying regions of the same object with nearly unchanged visual features in the original image as different categories." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04979" target="2310.03744" id="1932">
-        <attvalues>
-          <attvalue for="5" value=" In the post-processing stage, we utilized the large multimodal model \cite{liu2023improved, liu2024visual, achiam2023gpt} to correct some of the video segmentation results." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="1902.09513" id="1933">
-        <attvalues>
-          <attvalue for="5" value=" Recently, matching-based methods \cite{match1,chen3,match2} have achieved great success in semi-supervised VOS, achieving object segmentation through pixel-level matching between query frames and memory frames." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2003.08333" id="1934">
-        <attvalues>
-          <attvalue for="5" value=" Recently, matching-based methods \cite{match1,chen3,match2} have achieved great success in semi-supervised VOS, achieving object segmentation through pixel-level matching between query frames and memory frames." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="1904.00607" id="1935">
-        <attvalues>
-          <attvalue for="5" value=" After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2010.06349" id="1936">
-        <attvalues>
-          <attvalue for="5" value=" Among these methods, memory-based networks \cite{1,2,3,qihe,4,5,6,7,chen2,9,chen1,10,mem1,mem2,mem3} attracted a lot of attention." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2103.12934" id="1937">
-        <attvalues>
-          <attvalue for="5" value=" Firstly, the existing methods \cite{3,7,10} investigate the space-time coherence between adjacent video frames to exclude distractor objects, however the coherence is often completely destroyed due to occlusion, fast motion and irregular deformation." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2109.11404" id="1938">
-        <attvalues>
-          <attvalue for="5" value=" After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2106.02638" id="1939">
-        <attvalues>
-          <attvalue for="5" value=" After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2205.03761" id="1940">
-        <attvalues>
-          <attvalue for="5" value=" After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2304.11840" id="1941">
-        <attvalues>
-          <attvalue for="5" value=" Firstly, the existing methods \cite{3,7,10} investigate the space-time coherence between adjacent video frames to exclude distractor objects, however the coherence is often completely destroyed due to occlusion, fast motion and irregular deformation." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2106.05210" id="1942">
-        <attvalues>
-          <attvalue for="5" value=" Particularly, the latest related state-of-the-art methods, known as STCN \cite{9} and XMem \cite{10}, share multiple memory stores for all objects to capture different spatial-temporal contexts, have achieved prominent performance." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2207.07115" id="1943">
-        <attvalues>
-          <attvalue for="5" value=" Firstly, the existing methods \cite{3,7,10} investigate the space-time coherence between adjacent video frames to exclude distractor objects, however the coherence is often completely destroyed due to occlusion, fast motion and irregular deformation." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04042" target="2208.01924" id="1944">
-        <attvalues>
-          <attvalue for="5" value=" After STM \cite{1}, there are many variants \cite{4,5,6,7,chen1,mem1,mem2,mem3} developed aiming to improve accuracy, reduce memory usage, and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1306.5151" id="1945">
-        <attvalues>
-          <attvalue for="5" value=" &#09;\item We conduct comprehensive experiments on the three commonly used FGVC datasets, eg, CUB-$200$-$2011$~\cite{birds}, FGVC-Aircraft~\cite{air}， and Stanford Cars~\cite{cars} datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1903.06150" id="1946">
-        <attvalues>
-          <attvalue for="5" value=" The existing methods~\cite{TASN,SENet,RA-CNN,CBAM} usually utilize different attention mechanisms to generally adjust the distributions of the attention weights for balancing the contributions of feature maps extracted from each part." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1809.00287" id="1947">
-        <attvalues>
-          <attvalue for="5" value=" The existing approaches can be roughly divided into two classes: ($1$) searching the informative regions that contribute the most to the classification task~\cite{anno,TASN,NTS} and ($2$) paying more attention to extract high-order features from the images~\cite{BP,BT,mamcloss,mc}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1807.09915" id="1948">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, for the latter one, the powerful deep networks~\cite{BP,BT} were employed for feature extraction, and different loss functions~\cite{mamcloss,mc} were designed for constraining these networks to improve the discrimination of the extracted features." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1911.03621" id="1949">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, for the latter one, the powerful deep networks~\cite{BP,BT} were employed for feature extraction, and different loss functions~\cite{mamcloss,mc} were designed for constraining these networks to improve the discrimination of the extracted features." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1806.05372" id="1950">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, for the latter one, the powerful deep networks~\cite{BP,BT} were employed for feature extraction, and different loss functions~\cite{mamcloss,mc} were designed for constraining these networks to improve the discrimination of the extracted features." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="2002.04264" id="1951">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, for the latter one, the powerful deep networks~\cite{BP,BT} were employed for feature extraction, and different loss functions~\cite{mamcloss,mc} were designed for constraining these networks to improve the discrimination of the extracted features." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1709.01507" id="1952">
-        <attvalues>
-          <attvalue for="5" value=" The existing methods~\cite{TASN,SENet,RA-CNN,CBAM} usually utilize different attention mechanisms to generally adjust the distributions of the attention weights for balancing the contributions of feature maps extracted from each part." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1807.06521" id="1953">
-        <attvalues>
-          <attvalue for="5" value=" The existing methods~\cite{TASN,SENet,RA-CNN,CBAM} usually utilize different attention mechanisms to generally adjust the distributions of the attention weights for balancing the contributions of feature maps extracted from each part." />
-        </attvalues>
-      </edge>
-      <edge source="2101.09666" target="1610.02391" id="1954">
-        <attvalues>
-          <attvalue for="5" value=" Gradient-weighted class activation mapping (Grad-CAM)~\cite{cam} is usually introduced to illustrate attentions of the networks with heat maps and visualize the attentions in each part by weighted averaging channels, which can be used for guiding the networks to focus on more efficient parts and discard the redundant information for the classification." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1511.04510" id="1955">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1809.05996" id="1956">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1708.03383" id="1957">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1512.08086" id="1958">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="2005.10411" id="1959">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="2103.05569" id="1960">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1703.08580" id="1961">
-        <attvalues>
-          <attvalue for="5" value=" A tremendous amount of research efforts~\cite{liang2016semantic1,ruan2019devil,zhao2021ordinal,xia2017joint, zhang2014part,huang2016part,he2017weakly,wei2018mask,huang2020interpretable} has been proposed to solve this important problem, which can also be applied for downstream applications~\cite{bickel2020impacts,sun2022fair1m,pakhomov2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="1406.2375" id="1962">
-        <attvalues>
-          <attvalue for="5" value=" Conventional machine learning techniques build explicit structures for parsing and understanding these fine-grained objects,~\eg, graph and tree structures for part segmentation~\cite{zhu2008unsupervised,hsieh2010segmentation,wang2011learning,car2014parsing}, and part learning in fine-grained recognition~\cite{zhang2012pose,branson2011strong}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.13693" target="2111.06119" id="1963">
-        <attvalues>
-          <attvalue for="5" value=" Besides these improvements compared to other survey papers~\cite{zhao2017survey,wei2021fine}, in this paper, we start from the novel view of~part relationship learning and regard it as the correspondences of different fine-grained sub-tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="1908.07007" id="1964">
-        <attvalues>
-          <attvalue for="5" value=" Predictive models extrapolate pixels out from crops \cite{teterwak2019boundless, wang2019wide, van2019image}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="2002.05709" id="1965">
-        <attvalues>
-          <attvalue for="5" value=" Even the latest self-supervised efforts depend on crops for contrastive learning to induce rich visual representations \cite{chen2020simple,he2020momentum,pathak2016context,selvaraju2020casting}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="1911.05722" id="1966">
-        <attvalues>
-          <attvalue for="5" value=" Even the latest self-supervised efforts depend on crops for contrastive learning to induce rich visual representations \cite{chen2020simple,he2020momentum,pathak2016context,selvaraju2020casting}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="1604.07379" id="1967">
-        <attvalues>
-          <attvalue for="5" value=" Even the latest self-supervised efforts depend on crops for contrastive learning to induce rich visual representations \cite{chen2020simple,he2020momentum,pathak2016context,selvaraju2020casting}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="2012.04630" id="1968">
-        <attvalues>
-          <attvalue for="5" value=" Even the latest self-supervised efforts depend on crops for contrastive learning to induce rich visual representations \cite{chen2020simple,he2020momentum,pathak2016context,selvaraju2020casting}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="1505.05192" id="1969">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, extensive effort goes into preventing neural networks from learning such shortcuts enabled by image crops \cite{doersch2015unsupervised, noroozi2017representation}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="2004.06130" id="1970">
-        <attvalues>
-          <attvalue for="5" value=" Historically, asking fundamental questions has spurred significant insight into core computer vision problems, such as invariances to scale \cite{burt1983laplacian}, asymmetries in time \cite{pickup2014seeing}, the speediness of videos \cite{benaim2020speednet}, and visual chirality \cite{lin2020visual}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.11831" target="2006.09512" id="1971">
-        <attvalues>
-          <attvalue for="5" value=" Historically, asking fundamental questions has spurred significant insight into core computer vision problems, such as invariances to scale \cite{burt1983laplacian}, asymmetries in time \cite{pickup2014seeing}, the speediness of videos \cite{benaim2020speednet}, and visual chirality \cite{lin2020visual}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.07510" target="2006.01423" id="1972">
-        <attvalues>
-          <attvalue for="5" value=" Human Pose Estimation (HPE) is a widely deployed computer vision application \cite{chen2020monocular}, along with semantic segmentation \cite{segdasvisapp19}, and object tracking and trajectory prediction \cite{sridevi2021object}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.07510" target="1901.06580" id="1973">
-        <attvalues>
-          <attvalue for="5" value=" Human Pose Estimation (HPE) is a widely deployed computer vision application \cite{chen2020monocular}, along with semantic segmentation \cite{segdasvisapp19}, and object tracking and trajectory prediction \cite{sridevi2021object}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.07510" target="1702.05693" id="1974">
-        <attvalues>
-          <attvalue for="5" value=" To address the issue of the non-availability of pose annotations during training in the CityPersons \cite{zhang2017citypersons} dataset, we consider a related dataset (MS-COCO \cite{lin2014microsoft}) to get adequate supervision specific to PPE during training." />
-        </attvalues>
-      </edge>
-      <edge source="2206.07510" target="1405.0312" id="1975">
-        <attvalues>
-          <attvalue for="5" value=" To address the issue of the non-availability of pose annotations during training in the CityPersons \cite{zhang2017citypersons} dataset, we consider a related dataset (MS-COCO \cite{lin2014microsoft}) to get adequate supervision specific to PPE during training." />
-        </attvalues>
-      </edge>
-      <edge source="2110.06697" target="1401.0166" id="1976">
-        <attvalues>
-          <attvalue for="5" value=" Image fusion has been motivated by the need to improve visual representations, visualisation, scene understanding and situational awareness in multi-sensor and multi-camera applications such as remote sensing~\cite{GHASSEMIAN201675}, medicine~\cite{james2014medical} and surveillance~\cite{paramanandham2018multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.06697" target="2007.00328" id="1977">
-        <attvalues>
-          <attvalue for="5" value=" Due to the requirements of needing training data these methods are often domain focused with state-of-the-art results reported for IR/Visible fusion~\cite{li2020nestfuse, li2018infrared}, remote sensing (multispectral)~\cite{shao2018remote} and multi-focus areas~\cite{tang2018pixel, amin2019ensemble}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.06697" target="1804.06992" id="1978">
-        <attvalues>
-          <attvalue for="5" value=" More recently, state-of-the-art image fusion techniques have focused on the use of network based methods~\cite{javan2021review, hermessi2021multimodal,li2020nestfuse,li2018infrared,shao2018remote,tang2018pixel, amin2019ensemble,xu2020u2fusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1405.0312" id="1979">
-        <attvalues>
-          <attvalue for="5" value=" In this work, by using the Faster R-CNN-FPN\cite{lin2017feature,ren2015faster} and ResNet\cite{he2016deep} as backbone, PBRnet brings about 3 point of the $mAP$ gains on COCO2014 benchmark\cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1711.07240" id="1980">
-        <attvalues>
-          <attvalue for="5" value=" Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1409.1556" id="1981">
-        <attvalues>
-          <attvalue for="5" value=" Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1409.4842" id="1982">
-        <attvalues>
-          <attvalue for="5" value=" Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1512.03385" id="1983">
-        <attvalues>
-          <attvalue for="5" value=" In this work, by using the Faster R-CNN-FPN\cite{lin2017feature,ren2015faster} and ResNet\cite{he2016deep} as backbone, PBRnet brings about 3 point of the $mAP$ gains on COCO2014 benchmark\cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1602.07261" id="1984">
-        <attvalues>
-          <attvalue for="5" value=" Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1611.05431" id="1985">
-        <attvalues>
-          <attvalue for="5" value=" Object detection\cite{deng2009imagenet,lin2014microsoft,everingham2010pascal} serves as a fundamental task with many applications in computer vision, and has been rapidly improved in recent years based on the deep learning models\cite{peng2018megdet,krizhevsky2012imagenet,simonyan2014very,szegedy2015going,he2016deep,szegedy2017inception,xie2017aggregated}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1504.08083" id="1986">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1311.2524" id="1987">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1506.01497" id="1988">
-        <attvalues>
-          <attvalue for="5" value=" In this work, by using the Faster R-CNN-FPN\cite{lin2017feature,ren2015faster} and ResNet\cite{he2016deep} as backbone, PBRnet brings about 3 point of the $mAP$ gains on COCO2014 benchmark\cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1605.06409" id="1989">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1703.06870" id="1990">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1604.03540" id="1991">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1708.02002" id="1992">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1703.06211" id="1993">
-        <attvalues>
-          <attvalue for="5" value=" Among them Faster R-CNN is a classical two stage detection framework which has been most widely used and is the basis of many advanced models\cite{girshick2015fast,girshick2014rich,ren2015faster,singh2018analysis,dai2016r,gidaris2015object,he2017mask,shrivastava2016training,lin2017focal,dai2017deformable}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1606.04446" id="1994">
-        <attvalues>
-          <attvalue for="5" value=" Iterative bounding box regression\cite{gidaris2016attend} claims that a single workflow is not sufficient for detection and iteratively applies additional post-processing steps." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1712.00726" id="1995">
-        <attvalues>
-          <attvalue for="5" value=" It achieves a comparable performance to the Cascade R-CNN\cite{cai2018cascade} but introduces much less extra memory and parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1912.04260" id="1996">
-        <attvalues>
-          <attvalue for="5" value=" Recently, Side-Aware Boundary Localization (SABL)\cite{wang2019side} demonstrates that local areas are more suitable for boundary refinement, and proposes a lightweight two step bucketing scheme for localization which searches the bucket where the boundary resides and then predicts the offsets of the bucket." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1612.03144" id="1997">
-        <attvalues>
-          <attvalue for="5" value=" In this work, by using the Faster R-CNN-FPN\cite{lin2017feature,ren2015faster} and ResNet\cite{he2016deep} as backbone, PBRnet brings about 3 point of the $mAP$ gains on COCO2014 benchmark\cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1904.02701" id="1998">
-        <attvalues>
-          <attvalue for="5" value=" Feature Pyramid Network(FPN)\cite{lin2017feature,pang2019libra,qin2019thundernet,liu2018path} is an effective backbone which uses a top-down architecture with lateral connections to build an in-network feature pyramid structure." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1803.01534" id="1999">
-        <attvalues>
-          <attvalue for="5" value=" Feature Pyramid Network(FPN)\cite{lin2017feature,pang2019libra,qin2019thundernet,liu2018path} is an effective backbone which uses a top-down architecture with lateral connections to build an in-network feature pyramid structure." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04541" target="1803.06799" id="2000">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, as pointed in \cite{cheng2018revisiting}, localization requires translational variant features, the information in the lower-level feature map is more sensitive to displacement and translation as the smaller stride compare with that in the higher-level feature maps." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1901.08225" id="2001">
-        <attvalues>
-          <attvalue for="5" value=" Such signal distortion and detail loss often lead to the failure of many computer vision systems~\cite{Bae2019aaai,HuangLYWRH2018AAAI,yang2020mutualnet,yu2021towards, xu2021rethinking,zhong2021grayscale,huang2021occluded,xu2021exploring}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1909.12978" id="2002">
-        <attvalues>
-          <attvalue for="5" value=" Such signal distortion and detail loss often lead to the failure of many computer vision systems~\cite{Bae2019aaai,HuangLYWRH2018AAAI,yang2020mutualnet,yu2021towards, xu2021rethinking,zhong2021grayscale,huang2021occluded,xu2021exploring}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="2011.03822" id="2003">
-        <attvalues>
-          <attvalue for="5" value=" Such signal distortion and detail loss often lead to the failure of many computer vision systems~\cite{Bae2019aaai,HuangLYWRH2018AAAI,yang2020mutualnet,yu2021towards, xu2021rethinking,zhong2021grayscale,huang2021occluded,xu2021exploring}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="2007.16124" id="2004">
-        <attvalues>
-          <attvalue for="5" value=" Such signal distortion and detail loss often lead to the failure of many computer vision systems~\cite{Bae2019aaai,HuangLYWRH2018AAAI,yang2020mutualnet,yu2021towards, xu2021rethinking,zhong2021grayscale,huang2021occluded,xu2021exploring}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1707.02880" id="2005">
-        <attvalues>
-          <attvalue for="5" value=" When $\theta(\cdot)$ is simplified as the pixel-wise summation~\cite{gharbi2017deep}, the effort is to learn the perturbation $I_N$ and then subtract it from the observed image $I_{L}$." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1905.04161" id="2006">
-        <attvalues>
-          <attvalue for="5" value=" With this kind of scheme, an image can be enhanced either by estimating and adjusting the illumination map~\cite{li2018lightennet,wang2019underexposed}, or by learning joint features of these two components~\cite{Zhang2019Kindling}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="2003.06792" id="2007">
-        <attvalues>
-          <attvalue for="5" value=" More specifically, since the degradation additionally damages the texture, color, and contrast of normal-light images, we argue that subtracting the perturbation $I_N$ from its low-light input $I_{L}$ directly (scheme one) cannot fully recover multiple kinds of fine details and color distribution (refer to MIRNet~\cite{zamir2020learning} in Figure~\ref{fig:sample})." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1808.04560" id="2008">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, adjusting the illumination map to enhance reflectance $I_R$ (scheme two) cannot fully keep the visual naturalness of the predicted normal-light image (refer to RetinexNet~\cite{wei2018deep} and DeepUPE~\cite{wang2019underexposed} in Figure~\ref{fig:sample})." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1807.11458" id="2009">
-        <attvalues>
-          <attvalue for="5" value=" More specifically, inspired by the impressive results of GAN in image synthesis and translation~\cite{GatysEB15,bulat2018to}, we train a parametric generator to learn the degradation factors from the low-light input in the first stage, denoted as $\phi(I_{L})$." />
-        </attvalues>
-      </edge>
-      <edge source="2103.10621" target="1612.03716" id="2010">
-        <attvalues>
-          <attvalue for="5" value=" \item We also introduce two novel low/normal-light datasets (COCO24700 for training and COCO1000 for evaluation) based on COCO~\cite{caesar2018coco} dataset, which can be used for the joint low-light image enhancement and detection tasks under extreme night-time degradation scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2104.08894" id="2011">
-        <attvalues>
-          <attvalue for="5" value=" Understanding the complex probability distribution of the data is essential for image authenticity and quality analysis, but is challenging due to its high dimensionality and intricate domain variations~ \cite{manifold_dimension,manifold_geo}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1904.06151" id="2012">
-        <attvalues>
-          <attvalue for="5" value=" Understanding the complex probability distribution of the data is essential for image authenticity and quality analysis, but is challenging due to its high dimensionality and intricate domain variations~ \cite{manifold_dimension,manifold_geo}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2102.12092" id="2013">
-        <attvalues>
-          <attvalue for="5" value=" \revision{ Nevertheless, the phenomenon that image embeddings encoded using methods such as a pretrained CLIP encoder~\cite{dall_e} lie within a narrow cone of the unit sphere instead of the entire sphere~\cite{cone_1,cone_2}, which degrades the aforementioned pattern of probability distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1907.12009" id="2014">
-        <attvalues>
-          <attvalue for="5" value=" \revision{ Nevertheless, the phenomenon that image embeddings encoded using methods such as a pretrained CLIP encoder~\cite{dall_e} lie within a narrow cone of the unit sphere instead of the entire sphere~\cite{cone_1,cone_2}, which degrades the aforementioned pattern of probability distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2211.02487" id="2015">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, it is important to compute the probability in the latent space to indicate whether the corresponding image is in a high-density region of the space~ \cite{manifold_mle,manifold_flow_flow,manifold_flow_gan,nf_prob,manifold_gan,manifold_hyperbolic,manifold_flow_beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1705.08868" id="2016">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, it is important to compute the probability in the latent space to indicate whether the corresponding image is in a high-density region of the space~ \cite{manifold_mle,manifold_flow_flow,manifold_flow_gan,nf_prob,manifold_gan,manifold_hyperbolic,manifold_flow_beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1912.02762" id="2017">
-        <attvalues>
-          <attvalue for="5" value="} Specifically, we use normalising flow (NF)~\cite{vae_nf,nf_prob} and diffusion models (DMs)~\cite{ddpm,ddpm_understand,ddim} as image generators." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2207.05468" id="2018">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, it is important to compute the probability in the latent space to indicate whether the corresponding image is in a high-density region of the space~ \cite{manifold_mle,manifold_flow_flow,manifold_flow_gan,nf_prob,manifold_gan,manifold_hyperbolic,manifold_flow_beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1912.11035" id="2019">
-        <attvalues>
-          <attvalue for="5" value=" Some works train a discriminator with positive (real) and negative (synthetic) examples in the manner of contrastive learning~\cite{detect_gen_by_real} or analyse their frequency differences~\cite{frequency}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1505.05770" id="2020">
-        <attvalues>
-          <attvalue for="5" value=" \revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2} and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have been explored in model training." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2006.11239" id="2021">
-        <attvalues>
-          <attvalue for="5" value="} Specifically, we use normalising flow (NF)~\cite{vae_nf,nf_prob} and diffusion models (DMs)~\cite{ddpm,ddpm_understand,ddim} as image generators." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2208.11970" id="2022">
-        <attvalues>
-          <attvalue for="5" value="} Specifically, we use normalising flow (NF)~\cite{vae_nf,nf_prob} and diffusion models (DMs)~\cite{ddpm,ddpm_understand,ddim} as image generators." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2010.02502" id="2023">
-        <attvalues>
-          <attvalue for="5" value="} Specifically, we use normalising flow (NF)~\cite{vae_nf,nf_prob} and diffusion models (DMs)~\cite{ddpm,ddpm_understand,ddim} as image generators." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2110.07579" id="2024">
-        <attvalues>
-          <attvalue for="5" value=" \revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2} and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have been explored in model training." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2010.00029" id="2025">
-        <attvalues>
-          <attvalue for="5" value=" \revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2} and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have been explored in model training." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2108.05301" id="2026">
-        <attvalues>
-          <attvalue for="5" value=" \revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2} and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have been explored in model training." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2106.08462" id="2027">
-        <attvalues>
-          <attvalue for="5" value=" \revision{In prior works, NF for exact likelihood estimation~\cite{vae_nf,exact_1,exact_2} and with hierarchical structure~\cite{hir_structure_1,hir_structure_2,hir_structure_3} have been explored in model training." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1312.6114" id="2028">
-        <attvalues>
-          <attvalue for="5" value=" In this work, \revision{we disentangle semantics in the latent space by using a variational autoencoder (VAE)~\cite{kingma2013auto} in the framework of emergent languages (EL)~ \cite{havrylov2017emergence,el_control,el_communication,el_dynamic,el_miccai}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1705.11192" id="2029">
-        <attvalues>
-          <attvalue for="5" value=" In this work, \revision{we disentangle semantics in the latent space by using a variational autoencoder (VAE)~\cite{kingma2013auto} in the framework of emergent languages (EL)~ \cite{havrylov2017emergence,el_control,el_communication,el_dynamic,el_miccai}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2108.01828" id="2030">
-        <attvalues>
-          <attvalue for="5" value=" In this work, \revision{we disentangle semantics in the latent space by using a variational autoencoder (VAE)~\cite{kingma2013auto} in the framework of emergent languages (EL)~ \cite{havrylov2017emergence,el_control,el_communication,el_dynamic,el_miccai}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2007.09448" id="2031">
-        <attvalues>
-          <attvalue for="5" value=" In this work, \revision{we disentangle semantics in the latent space by using a variational autoencoder (VAE)~\cite{kingma2013auto} in the framework of emergent languages (EL)~ \cite{havrylov2017emergence,el_control,el_communication,el_dynamic,el_miccai}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2004.01255" id="2032">
-        <attvalues>
-          <attvalue for="5" value=" \revision{Although some VAE variant models such as $\beta$-TCVAE~\cite{tcvae}, GuidedVAE~\cite{guidedvae}, and DCVAE~\cite{dcvae} achieve qualified semantic disentanglement results, we mainly focus on understanding the effectiveness of the emergent language framework for VAE based disentanglement inspired by \cite{el_model} and emphasizing the feasibility of our GridVAE (with mixture of Gaussian priors) under such an EL framework to study semantic distributions on the image manifold." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2011.10063" id="2033">
-        <attvalues>
-          <attvalue for="5" value=" \revision{Although some VAE variant models such as $\beta$-TCVAE~\cite{tcvae}, GuidedVAE~\cite{guidedvae}, and DCVAE~\cite{dcvae} achieve qualified semantic disentanglement results, we mainly focus on understanding the effectiveness of the emergent language framework for VAE based disentanglement inspired by \cite{el_model} and emphasizing the feasibility of our GridVAE (with mixture of Gaussian priors) under such an EL framework to study semantic distributions on the image manifold." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2210.00482" id="2034">
-        <attvalues>
-          <attvalue for="5" value=" \revision{Although some VAE variant models such as $\beta$-TCVAE~\cite{tcvae}, GuidedVAE~\cite{guidedvae}, and DCVAE~\cite{dcvae} achieve qualified semantic disentanglement results, we mainly focus on understanding the effectiveness of the emergent language framework for VAE based disentanglement inspired by \cite{el_model} and emphasizing the feasibility of our GridVAE (with mixture of Gaussian priors) under such an EL framework to study semantic distributions on the image manifold." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1712.09665" id="2035">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2005.09257" id="2036">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="2005.10884" id="2037">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1812.00292" id="2038">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1706.06083" id="2039">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02881" target="1608.04644" id="2040">
-        <attvalues>
-          <attvalue for="5" value="} We also evaluate their semantic robustness on such a manifold against adversarial and patch attacks~ \cite{patch_adv,patch_check,patch_gan,patch_guard,patch_sentinet,PGD_attack,Ensemble_Adversarial_Training_Attacks_defenses,Towards_Evaluating_Robustness_Neural_Networks} and defend against the same attacks using semantic consistency with a purification loss." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1903.12261" id="2041">
-        <attvalues>
-          <attvalue for="5" value=" It was shown that the current models are vulnerable to noisy input \cite{corruptions,fourier}, novel weather conditions \cite{wintercoming}, and background changes \cite{background}, which creates safety considerations for models deployed to the real world, eg, autonomous driving or medical applications." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1906.08988" id="2042">
-        <attvalues>
-          <attvalue for="5" value=" It was shown that the current models are vulnerable to noisy input \cite{corruptions,fourier}, novel weather conditions \cite{wintercoming}, and background changes \cite{background}, which creates safety considerations for models deployed to the real world, eg, autonomous driving or medical applications." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1907.07484" id="2043">
-        <attvalues>
-          <attvalue for="5" value=" It was shown that the current models are vulnerable to noisy input \cite{corruptions,fourier}, novel weather conditions \cite{wintercoming}, and background changes \cite{background}, which creates safety considerations for models deployed to the real world, eg, autonomous driving or medical applications." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2006.09994" id="2044">
-        <attvalues>
-          <attvalue for="5" value=" It was shown that the current models are vulnerable to noisy input \cite{corruptions,fourier}, novel weather conditions \cite{wintercoming}, and background changes \cite{background}, which creates safety considerations for models deployed to the real world, eg, autonomous driving or medical applications." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1811.12231" id="2045">
-        <attvalues>
-          <attvalue for="5" value=" To improve visual recognition models’ robustness it was proposed to use data augmentations which change objects' appearance, for example by using style-transfer \cite{cnnbiased,towards} data augmentation or color distortions \cite{simclr,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2002.05709" id="2046">
-        <attvalues>
-          <attvalue for="5" value=" To improve visual recognition models’ robustness it was proposed to use data augmentations which change objects' appearance, for example by using style-transfer \cite{cnnbiased,towards} data augmentation or color distortions \cite{simclr,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2106.00076" id="2047">
-        <attvalues>
-          <attvalue for="5" value=" Sampling based-methods were shown to obtain very good results in terms of accuracy, out-of-domain robustness and improving model predictive uncertainty \cite{medconf,canyoutrust,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1706.04599" id="2048">
-        <attvalues>
-          <attvalue for="5" value=" Additionally, the current models are often overconfident in their predictions \cite{calibration}, and the problem becomes more evident with out-of-domain data \cite{canyoutrust,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1906.02530" id="2049">
-        <attvalues>
-          <attvalue for="5" value=" Sampling based-methods were shown to obtain very good results in terms of accuracy, out-of-domain robustness and improving model predictive uncertainty \cite{medconf,canyoutrust,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1911.13273" id="2050">
-        <attvalues>
-          <attvalue for="5" value=" Sampling based-methods were shown to obtain very good results in terms of accuracy, out-of-domain robustness and improving model predictive uncertainty \cite{medconf,canyoutrust,gta}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1506.02142" id="2051">
-        <attvalues>
-          <attvalue for="5" value=" Several methods were developed to reduce the high computational cost, such as test-time dropout \cite{Gal} or batch ensemble \cite{batchensemble}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2002.06715" id="2052">
-        <attvalues>
-          <attvalue for="5" value=" Several methods were developed to reduce the high computational cost, such as test-time dropout \cite{Gal} or batch ensemble \cite{batchensemble}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1511.06314" id="2053">
-        <attvalues>
-          <attvalue for="5" value=" Another competitive approach is the m-heads model \cite{mheads}, which can be viewed as an ensemble with parameter sharing in the first layers of the network." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1612.01474" id="2054">
-        <attvalues>
-          <attvalue for="5" value=" Those methods, however, do not always match ensembling accuracy, as the success of sampling-based methods lies in the diversity of the predictions, which is a challenging problem \cite{deepensemble,pitfalls}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2002.06470" id="2055">
-        <attvalues>
-          <attvalue for="5" value=" Those methods, however, do not always match ensembling accuracy, as the success of sampling-based methods lies in the diversity of the predictions, which is a challenging problem \cite{deepensemble,pitfalls}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1803.03635" id="2056">
-        <attvalues>
-          <attvalue for="5" value=" These methods were inspired by the compression methods, which show that it is possible to remove even 90\% of the weights, without affecting the final model accuracy \cite{nipsprune,ticket}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="1902.05522" id="2057">
-        <attvalues>
-          <attvalue for="5" value=" For example, \cite{superposition,supermasks} use a single model in the multi-task setting, and the latter approach retrieves a subnetwork (from the main model) to efficiently solve the target task." />
-        </attvalues>
-      </edge>
-      <edge source="2111.13065" target="2006.14769" id="2058">
-        <attvalues>
-          <attvalue for="5" value=" For example, \cite{superposition,supermasks} use a single model in the multi-task setting, and the latter approach retrieves a subnetwork (from the main model) to efficiently solve the target task." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1808.08319" id="2059">
-        <attvalues>
-          <attvalue for="5" value=" The first challenge was organized in 2017~\cite{hodan2017sixd} and the results were published in~\cite{hodan2018bop}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="2009.07378" id="2060">
-        <attvalues>
-          <attvalue for="5" value=" For the 2020 challenge, we then provided participants with 350K PBR training images (see~\cite{hodan2020bop} for examples), which helped the DNN-based methods to achieve noticeably higher accuracy and to finally catch up with the PPF-based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1703.06870" id="2061">
-        <attvalues>
-          <attvalue for="5" value=" To create an opportunity for detector-agnostic comparison of pose estimation methods and to allow participants to focus only on the pose estimation stage, we also provided default detections and segmentations from Mask R-CNN~\cite{he2017mask} trained for CosyPose~\cite{labbe2020cosypose}, the winning method in 2020." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="2008.08465" id="2062">
-        <attvalues>
-          <attvalue for="5" value=" To create an opportunity for detector-agnostic comparison of pose estimation methods and to allow participants to focus only on the pose estimation stage, we also provided default detections and segmentations from Mask R-CNN~\cite{he2017mask} trained for CosyPose~\cite{labbe2020cosypose}, the winning method in 2020." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1701.05498" id="2063">
-        <attvalues>
-          <attvalue for="5" value=" The largest improvements were achieved on challenging industry-relevant datasets ITODD~\cite{drost2017introducing} and T-LESS~\cite{hodan2017tless}, and on the HB dataset~\cite{kaskman2019homebreweddb} which includes diverse objects captured under various levels of occlusion." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="2109.00512" id="2064">
-        <attvalues>
-          <attvalue for="5" value=" Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are established~\cite{newcombe2011kinectfusion,reizenstein2021common} and promising approaches for transparent and highly specular objects are emerging~\cite{wu2018full,Munkberg_2022_CVPR,hasselgren2022shape,verbin2022ref}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1805.03482" id="2065">
-        <attvalues>
-          <attvalue for="5" value=" Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are established~\cite{newcombe2011kinectfusion,reizenstein2021common} and promising approaches for transparent and highly specular objects are emerging~\cite{wu2018full,Munkberg_2022_CVPR,hasselgren2022shape,verbin2022ref}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="2111.12503" id="2066">
-        <attvalues>
-          <attvalue for="5" value=" Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are established~\cite{newcombe2011kinectfusion,reizenstein2021common} and promising approaches for transparent and highly specular objects are emerging~\cite{wu2018full,Munkberg_2022_CVPR,hasselgren2022shape,verbin2022ref}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="2112.03907" id="2067">
-        <attvalues>
-          <attvalue for="5" value=" Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are established~\cite{newcombe2011kinectfusion,reizenstein2021common} and promising approaches for transparent and highly specular objects are emerging~\cite{wu2018full,Munkberg_2022_CVPR,hasselgren2022shape,verbin2022ref}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1711.10006" id="2068">
-        <attvalues>
-          <attvalue for="5" value=" DNN-based methods need large amounts of annotated training images, which had been typically obtained by OpenGL rendering of the 3D object models on random backgrounds~\cite{kehl2017ssd,hinterstoisser2017pre}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1710.10710" id="2069">
-        <attvalues>
-          <attvalue for="5" value=" DNN-based methods need large amounts of annotated training images, which had been typically obtained by OpenGL rendering of the 3D object models on random backgrounds~\cite{kehl2017ssd,hinterstoisser2017pre}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1902.03334" id="2070">
-        <attvalues>
-          <attvalue for="5" value=" However, as suggested in~\cite{hodan2019photorealistic}, the evident domain gap between these ``render\;\&amp;\;paste'' training images and real test images limits the potential of the DNN-based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1911.01911" id="2071">
-        <attvalues>
-          <attvalue for="5" value=" To reduce the gap between the synthetic and real domains and thus to bring fresh air to the DNN world, we joined the development of BlenderProc~\cite{denninger2019blenderproc,denninger2020blenderproc}, an open-source, physically-based renderer (PBR)." />
-        </attvalues>
-      </edge>
-      <edge source="2302.13075" target="1904.03167" id="2072">
-        <attvalues>
-          <attvalue for="5" value=" The largest improvements were achieved on challenging industry-relevant datasets ITODD~\cite{drost2017introducing} and T-LESS~\cite{hodan2017tless}, and on the HB dataset~\cite{kaskman2019homebreweddb} which includes diverse objects captured under various levels of occlusion." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1808.08319" id="2073">
-        <attvalues>
-          <attvalue for="5" value=" The first challenge was organized in 2017~\cite{hodan2017sixd} and the results were published in~\cite{hodan2018bop}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1701.05498" id="2074">
-        <attvalues>
-          <attvalue for="5" value=" While capturing real images of objects under various conditions and annotating the images with 6D object poses requires a significant human effort~\cite{hodan2017tless}, the 3D models are either available before the physical objects, which is often the case for manufactured objects, or can be reconstructed at an admissible cost." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1805.03482" id="2075">
-        <attvalues>
-          <attvalue for="5" value=" Approaches for reconstructing 3D models of opaque, matte and moderately specular objects are well established~\cite{newcombe2011kinectfusion} and promising approaches for transparent and highly specular objects are emerging~\cite{qian20163d,wu2018full,godard2015multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1711.10006" id="2076">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, DNN-based methods require large amounts of annotated training images, which have been typically obtained by OpenGL rendering of the 3D object models on top of random backgrounds~\cite{kehl2017ssd,rad2017bb8,hinterstoisser2017pre,dwibedi2017cut}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1703.10896" id="2077">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, DNN-based methods require large amounts of annotated training images, which have been typically obtained by OpenGL rendering of the 3D object models on top of random backgrounds~\cite{kehl2017ssd,rad2017bb8,hinterstoisser2017pre,dwibedi2017cut}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1710.10710" id="2078">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, DNN-based methods require large amounts of annotated training images, which have been typically obtained by OpenGL rendering of the 3D object models on top of random backgrounds~\cite{kehl2017ssd,rad2017bb8,hinterstoisser2017pre,dwibedi2017cut}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1708.01642" id="2079">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, DNN-based methods require large amounts of annotated training images, which have been typically obtained by OpenGL rendering of the 3D object models on top of random backgrounds~\cite{kehl2017ssd,rad2017bb8,hinterstoisser2017pre,dwibedi2017cut}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1902.03334" id="2080">
-        <attvalues>
-          <attvalue for="5" value=" However, as suggested in~\cite{hodan2019photorealistic}, the evident domain gap between these ``render\;\&amp;\;paste'' training images and real test images presumably limits the potential of the DNN-based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1911.01911" id="2081">
-        <attvalues>
-          <attvalue for="5" value=" To reduce the gap between the synthetic and real domains and thus to bring fresh air to the DNN-based methods, we have created BlenderProc4BOP~\cite{denninger2019blenderproc,denninger2020blenderproc}, an open-source and light-weight physically-based renderer (PBR)." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="2008.08465" id="2082">
-        <attvalues>
-          <attvalue for="5" value=" Three of the top five methods, including the top-performing one, are single-view variants of CosyPose, a DNN-based method by Labb{\'e}~\etal~\cite{labbe2020cosypose}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="2011.05669" id="2083">
-        <attvalues>
-          <attvalue for="5" value=" The second is a hybrid DNN+PPF method by K\&quot;onig and Drost~\cite{koenig2020hybrid}, and the fourth is Pix2Pose, a DNN-based method by Park \etal~\cite{park2019pix2pose}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.07378" target="1908.07433" id="2084">
-        <attvalues>
-          <attvalue for="5" value=" The second is a hybrid DNN+PPF method by K\&quot;onig and Drost~\cite{koenig2020hybrid}, and the fourth is Pix2Pose, a DNN-based method by Park \etal~\cite{park2019pix2pose}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1701.05498" id="2085">
-        <attvalues>
-          <attvalue for="5" value=" For example, datasets like T-LESS~\cite{hodan2017tless} and LineMOD~\cite{hinterstoisserModelBasedTraining2013} cover textureless and target-specific object types in particular scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1711.10006" id="2086">
-        <attvalues>
-          <attvalue for="5" value=" However, the fact that 3D models of the objects are available and training data can be synthesized by rendering them has been used in only a few studies, most notably using such detectors as SSD6D~\cite{kehlSSD6DMakingRGBbased2017b}, AAE~\cite{sundermeyerImplicit3DOrientation2018} and DPOD~\cite{zakharov2019dpod}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1902.01275" id="2087">
-        <attvalues>
-          <attvalue for="5" value=" However, the fact that 3D models of the objects are available and training data can be synthesized by rendering them has been used in only a few studies, most notably using such detectors as SSD6D~\cite{kehlSSD6DMakingRGBbased2017b}, AAE~\cite{sundermeyerImplicit3DOrientation2018} and DPOD~\cite{zakharov2019dpod}." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1506.02640" id="2088">
-        <attvalues>
-          <attvalue for="5" value=" It is remarkable that all deep learning 6DoF object detectors trained either on real or synthetic data use a single neural network per object, in contrast to 2D object detectors, such as YOLO~\cite{redmonYouOnlyLook2015}, SSD~\cite{liuSSDSingleShot2016} or R-CNNs~\cite{girshickRichFeatureHierarchies2014a, girshickFastRcnn2015, renFasterRCNNRealTime2015, heMaskRCNN2017}, which use one network for all object classes." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1512.02325" id="2089">
-        <attvalues>
-          <attvalue for="5" value=" It is remarkable that all deep learning 6DoF object detectors trained either on real or synthetic data use a single neural network per object, in contrast to 2D object detectors, such as YOLO~\cite{redmonYouOnlyLook2015}, SSD~\cite{liuSSDSingleShot2016} or R-CNNs~\cite{girshickRichFeatureHierarchies2014a, girshickFastRcnn2015, renFasterRCNNRealTime2015, heMaskRCNN2017}, which use one network for all object classes." />
-        </attvalues>
-      </edge>
-      <edge source="1904.03167" target="1311.2524" id="2090">
-        <attvalues>
-          <attvalue for="5" value=" It is remarkable that all deep learning 6DoF object detectors trained either on real or synthetic data use a single neural network per object, in contrast to 2D object detectors, such as YOLO~\cite{redmonYouOnlyLook2015}, SSD~\cite{liuSSDSingleShot2016} or R-CNNs~\cite{girshickRichFeatureHierarchies2014a, girshickFastRcnn2015, renFasterRCNNRealTime2015, heMaskRCNN2017}, which use one network for all object classes." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2303.08774" id="2091">
-        <attvalues>
-          <attvalue for="5" value=" With the emergence of large language models (\eg, ChatGPT \cite{openai2023gpt4}), it has been demonstrated that training these models on vast amounts of data can effectively enable them to learn common tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.02643" id="2092">
-        <attvalues>
-          <attvalue for="5" value=" In the field of computer vision, the Segment Anything Model (SAM) \cite{kirillov2023segany} is the pioneering foundation model that adopts this paradigm." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.12620" id="2093">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.13785" id="2094">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.13973" id="2095">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2305.00293" id="2096">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.04709" id="2097">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.05750" id="2098">
-        <attvalues>
-          <attvalue for="5" value=" Subsequent studies have assessed its generalization capabilities across diverse domains and scenarios, including medical images \cite{he2023accuracy,wu2023medical,zhang2023customized,roy2023sam,hu2023skinsam,li2023polyp}, camouflaged object detection \cite{tang2023can}, salient object segmentation \cite{ji2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.06790" id="2099">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, SAM is also used as a foundation model for promoting other computer vision tasks such as semantic segmentation \cite{lin2018multi,lin2019zigzagnet,lin2022tagnet,shen2020ranet}, image editing \cite{yu2023inpaint,xie2023edit}, style transfer \cite{liu2023any}, object counting \cite{ma2023can}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.14006" id="2100">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, SAM is also used as a foundation model for promoting other computer vision tasks such as semantic segmentation \cite{lin2018multi,lin2019zigzagnet,lin2022tagnet,shen2020ranet}, image editing \cite{yu2023inpaint,xie2023edit}, style transfer \cite{liu2023any}, object counting \cite{ma2023can}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.09728" id="2101">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, SAM is also used as a foundation model for promoting other computer vision tasks such as semantic segmentation \cite{lin2018multi,lin2019zigzagnet,lin2022tagnet,shen2020ranet}, image editing \cite{yu2023inpaint,xie2023edit}, style transfer \cite{liu2023any}, object counting \cite{ma2023can}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2304.10817" id="2102">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, SAM is also used as a foundation model for promoting other computer vision tasks such as semantic segmentation \cite{lin2018multi,lin2019zigzagnet,lin2022tagnet,shen2020ranet}, image editing \cite{yu2023inpaint,xie2023edit}, style transfer \cite{liu2023any}, object counting \cite{ma2023can}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.16220" target="2305.06422" id="2103">
-        <attvalues>
-          <attvalue for="5" value=" A few notable works \cite{wang2023empirical, zhang2023attack} have explored the evaluation of SAM's robustness against corruption and adversarial attacks respectively." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05669" target="1808.08319" id="2104">
-        <attvalues>
-          <attvalue for="5" value=" Many of the recently published approaches for 6DoF object detection, especially on the BOP benchmark~\cite{hodan2018bop}, follow a two-stage pipeline." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05669" target="1908.07433" id="2105">
-        <attvalues>
-          <attvalue for="5" value=" Graph-based networks~\cite{gao2018occlusion}), and the kind of pose estimation they employ (direct regression of pose parameters~\cite{gao2018occlusion}, regression of 3D model coordinates of the scene points usually followed by PnP~\cite{park2019pix2pose,li2019cdpn,DBLP:journals/corr/abs-1902-11020,hodan2020epos}, or employing a codebook for estimating the rotation of an object~\cite{Sundermeyer_2020_CVPR,sundermeyer2020augmented})." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05669" target="1908.00151" id="2106">
-        <attvalues>
-          <attvalue for="5" value=" Graph-based networks~\cite{gao2018occlusion}), and the kind of pose estimation they employ (direct regression of pose parameters~\cite{gao2018occlusion}, regression of 3D model coordinates of the scene points usually followed by PnP~\cite{park2019pix2pose,li2019cdpn,DBLP:journals/corr/abs-1902-11020,hodan2020epos}, or employing a codebook for estimating the rotation of an object~\cite{Sundermeyer_2020_CVPR,sundermeyer2020augmented})." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05669" target="1808.05498" id="2107">
-        <attvalues>
-          <attvalue for="5" value=" Graph-based networks~\cite{gao2018occlusion}), and the kind of pose estimation they employ (direct regression of pose parameters~\cite{gao2018occlusion}, regression of 3D model coordinates of the scene points usually followed by PnP~\cite{park2019pix2pose,li2019cdpn,DBLP:journals/corr/abs-1902-11020,hodan2020epos}, or employing a codebook for estimating the rotation of an object~\cite{Sundermeyer_2020_CVPR,sundermeyer2020augmented})." />
-        </attvalues>
-      </edge>
-      <edge source="2011.05669" target="2004.00605" id="2108">
-        <attvalues>
-          <attvalue for="5" value=" Graph-based networks~\cite{gao2018occlusion}), and the kind of pose estimation they employ (direct regression of pose parameters~\cite{gao2018occlusion}, regression of 3D model coordinates of the scene points usually followed by PnP~\cite{park2019pix2pose,li2019cdpn,DBLP:journals/corr/abs-1902-11020,hodan2020epos}, or employing a codebook for estimating the rotation of an object~\cite{Sundermeyer_2020_CVPR,sundermeyer2020augmented})." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1605.06211" id="2109">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1512.03385" id="2110">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1505.04597" id="2111">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1412.7062" id="2112">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1606.00915" id="2113">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1706.05587" id="2114">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1612.01105" id="2115">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1802.02611" id="2116">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1711.07971" id="2117">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1809.02983" id="2118">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1804.03821" id="2119">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1811.11721" id="2120">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1908.07678" id="2121">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1809.00916" id="2122">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1909.02651" id="2123">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2004.01547" id="2124">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1909.09408" id="2125">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1907.13426" id="2126">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2103.13413" id="2127">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2107.06278" id="2128">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2007.10035" id="2129">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="1907.05740" id="2130">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2203.07160" id="2131">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2112.01527" id="2132">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2312.11872" id="2133">
-        <attvalues>
-          <attvalue for="5" value=" Since the introduction of Fully Convolutional Networks (FCNs~\cite{cFCN}) and Residual Networks (ResNets~\cite{cResnet}), supervised deep learning models~\cite{cUNet,cDeepLabv1,cDeepLab,cDeepLabV3,cPSPNet,cDeepLabV3Plus,cNonLocal,cDualAttention,cDenseASPP,cExFuse,cCCNet,cANNN,cOCNet,cCFNet,cCPN,cKSAC,cA2Net,cACFNet,cEMANet,cOCR,cDPT,cMaskFormer,cDecoupleSegNet,cGSCNN,cCAA,cCAR,cMask2Former,cDDP,cSegDeformer,cSAR} have achieved remarkable success in the field of semantic segmentation on traditional benchmark datasets (e." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14686" target="2304.02643" id="2134">
-        <attvalues>
-          <attvalue for="5" value=" SAM (Segment-anything~\cite{cSAM}) is a vision foundation model trained on a massive-scale dataset (SA-1B)." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1804.06423" id="2135">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018deep} use a pre-trained VGG16 network for feature extraction and pass them to a correlation layer to find the similarities among a pair of images as inspired by Flownet \cite{dosovitskiy2015flownet}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1504.06852" id="2136">
-        <attvalues>
-          <attvalue for="5" value=" \cite{li2018deep} use a pre-trained VGG16 network for feature extraction and pass them to a correlation layer to find the similarities among a pair of images as inspired by Flownet \cite{dosovitskiy2015flownet}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1810.06859" id="2137">
-        <attvalues>
-          <attvalue for="5" value=" \cite{chen2018semantic} developed a similar architecture to perform deep object co-segmentation by passing the global image features obtained from pre-trained VGG16 model to a channel-wise attention model which trains a fully connected network." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1911.12950" id="2138">
-        <attvalues>
-          <attvalue for="5" value=" Recently, \cite{zhang2020deep} developed a unique deep learning architecture which uses HRNet\cite{sun2019deep} pre-trained on ImageNet as a backbone to extract image features and then capture the correlation between features using a spatial modulator to generate a mask which can localize the common foreground object." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1902.09212" id="2139">
-        <attvalues>
-          <attvalue for="5" value=" Recently, \cite{zhang2020deep} developed a unique deep learning architecture which uses HRNet\cite{sun2019deep} pre-trained on ImageNet as a backbone to extract image features and then capture the correlation between features using a spatial modulator to generate a mask which can localize the common foreground object." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12828" target="1810.08393" id="2140">
-        <attvalues>
-          <attvalue for="5" value=" In the proposed approach we use deep ResNet features and pre-trained DGC Net\cite{melekhov2019dgc} to establish global and local correspondences, respectively." />
-        </attvalues>
-      </edge>
-      <edge source="2407.07816" target="2004.08566" id="2141">
-        <attvalues>
-          <attvalue for="5" value=" We cover a wide range of topics explored since the last surveys \cite{poggi2021synergies,laga2020survey}, including novel architectures, multi-modal approaches, and cutting-edge techniques designed to address the aforementioned critical challenges." />
-        </attvalues>
-      </edge>
-      <edge source="2407.07816" target="2006.02535" id="2142">
-        <attvalues>
-          <attvalue for="5" value=" We cover a wide range of topics explored since the last surveys \cite{poggi2021synergies,laga2020survey}, including novel architectures, multi-modal approaches, and cutting-edge techniques designed to address the aforementioned critical challenges." />
-        </attvalues>
-      </edge>
-      <edge source="2407.07816" target="2011.02910" id="2143">
-        <attvalues>
-          <attvalue for="5" value=" Transformer-based \cite{Li_2021_ICCV_STTR} and iterative refinement \cite{lipson2021raft} architectures are prime examples of how the field has evolved, demonstrating the potential for further improvements in accuracy and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2407.07816" target="2109.07547" id="2144">
-        <attvalues>
-          <attvalue for="5" value=" Transformer-based \cite{Li_2021_ICCV_STTR} and iterative refinement \cite{lipson2021raft} architectures are prime examples of how the field has evolved, demonstrating the potential for further improvements in accuracy and efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="2107.11820" id="2145">
-        <attvalues>
-          <attvalue for="5" value=" As this typically involves the evaluation of intractable integrals, sampling \cite{Robert04,luengo2020survey} or approximation \cite{Evans95} strategies are used." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1601.00670" id="2146">
-        <attvalues>
-          <attvalue for="5" value=" Another family of approach consists in adopting the so-called variational Bayesian approximation paradigm \cite{Foxtutorial,David2016}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1505.00273" id="2147">
-        <attvalues>
-          <attvalue for="5" value=" See, for example, \cite{Yorsa2017,Pereyra2016, Marnissi2020} for applications of such ideas in the context of large-scale image processing." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1711.07064" id="2148">
-        <attvalues>
-          <attvalue for="5" value=" Let us mention DeblurGAN \cite{Kupyn2018}, based on conditional generative adversarial networks and a multi-component loss function, SRCNN \cite{Dong2016} and its extended version, DBSRCNN \cite{Albluwi2018}, relying on a CNN architecture, SelfDeblur \cite{Ren2020} combining an optimization-based method with two GAN networks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1501.00092" id="2149">
-        <attvalues>
-          <attvalue for="5" value=" Let us mention DeblurGAN \cite{Kupyn2018}, based on conditional generative adversarial networks and a multi-component loss function, SRCNN \cite{Dong2016} and its extended version, DBSRCNN \cite{Albluwi2018}, relying on a CNN architecture, SelfDeblur \cite{Ren2020} combining an optimization-based method with two GAN networks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1908.02197" id="2150">
-        <attvalues>
-          <attvalue for="5" value=" Let us mention DeblurGAN \cite{Kupyn2018}, based on conditional generative adversarial networks and a multi-component loss function, SRCNN \cite{Dong2016} and its extended version, DBSRCNN \cite{Albluwi2018}, relying on a CNN architecture, SelfDeblur \cite{Ren2020} combining an optimization-based method with two GAN networks." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1912.10557" id="2151">
-        <attvalues>
-          <attvalue for="5" value=" An emerging set of methods, in the field of inverse problems in signal/image processing, performs algorithm unrolling \cite{Vishal2021}, where an iterative method (eg, an optimization algorithm) is unrolled as layers of a neural network." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="2012.13247" id="2152">
-        <attvalues>
-          <attvalue for="5" value=" These methods are also closely related to plug-and-play techniques where a trained neural network is employed as the denoiser \cite{Pesquet2020, Kamilov2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1808.07526" id="2153">
-        <attvalues>
-          <attvalue for="5" value=" Theoretical results assessing the stability and robustness of unrolling techniques can be found in \cite{Pesquet2020,Patrick2020,Emilie2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="2105.15044" id="2154">
-        <attvalues>
-          <attvalue for="5" value=" Theoretical results assessing the stability and robustness of unrolling techniques can be found in \cite{Pesquet2020,Patrick2020,Emilie2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2110.07202" target="1312.6114" id="2155">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, they are backbones of variational autoencoders \cite{Kingma2014} and also constitute methods of choice for training Bayesian neural networks \cite{Jospin2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="1710.11151" id="2156">
-        <attvalues>
-          <attvalue for="5" value=" &#09;Thus, previous VCM research either sacrifices the condition of uncompressed input data~\cite{choi2018, galteri2018} or evaluates the coding frameworks on single images~\cite{fischer2020_ICIP, fischer2020_FRDO, le2021_ICASSP, fischer2021_ICASSP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="2203.05927" id="2157">
-        <attvalues>
-          <attvalue for="5" value=" &#09;Thus, previous VCM research either sacrifices the condition of uncompressed input data~\cite{choi2018, galteri2018} or evaluates the coding frameworks on single images~\cite{fischer2020_ICIP, fischer2020_FRDO, le2021_ICASSP, fischer2021_ICASSP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="2203.05890" id="2158">
-        <attvalues>
-          <attvalue for="5" value=" &#09;More recent work focuses on coding for M2M communication, where a neural network analyzes the decoded image instead of a human observer~\cite{choi2018,galteri2018,fischer2020_ICIP,fischer2020_FRDO, le2021_ICASSP, fischer2021_ICASSP}." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="1703.07684" id="2159">
-        <attvalues>
-          <attvalue for="5" value="~\cite{luc2017} and Nabavi~et~al." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="1807.07946" id="2160">
-        <attvalues>
-          <attvalue for="5" value="~\cite{nabavi2018FutureSS} take the predictions of a state-of-the-art semantic segmentation network as pseudo GT to evaluate their future semantic segmentation models." />
-        </attvalues>
-      </edge>
-      <edge source="2205.06519" target="1604.01685" id="2161">
-        <attvalues>
-          <attvalue for="5" value="~\cite{couprie2018} applied pseudo GT annotations for the task of future instance segmentation for the Cityscapes~\cite{cordts2016} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1905.13258" id="2162">
-        <attvalues>
-          <attvalue for="5" value=" In the realm of visual scene analysis, multi-target tracking is one of the most important low-level computer vision problems that provides a backbone to many high level tasks like autonomous driving \cite{khan2019survey, li2019generic}, action recognition \cite{ullah2017human, wang2018generative, ullah2019stacked}, behavior analysis \cite{marsden2017resnetcrowd, ullah2016crowd}, anomaly detection\cite{wang2018generative, ullah2019hybrid, ullah2018anomalous}, crowd management \cite{ullah2017density, ullah2017density}, and sports players analysis \cite{thomas2017computer, khan2019person}, to name a few." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1705.10698" id="2163">
-        <attvalues>
-          <attvalue for="5" value=" In the realm of visual scene analysis, multi-target tracking is one of the most important low-level computer vision problems that provides a backbone to many high level tasks like autonomous driving \cite{khan2019survey, li2019generic}, action recognition \cite{ullah2017human, wang2018generative, ullah2019stacked}, behavior analysis \cite{marsden2017resnetcrowd, ullah2016crowd}, anomaly detection\cite{wang2018generative, ullah2019hybrid, ullah2018anomalous}, crowd management \cite{ullah2017density, ullah2017density}, and sports players analysis \cite{thomas2017computer, khan2019person}, to name a few." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1506.02640" id="2164">
-        <attvalues>
-          <attvalue for="5" value=" With the advancement in object detection algorithms \cite{redmon2016you, khan2019dimension}, the tracking-by-detection paradigm becomes the most suitable for tracking multiple objects in a visual scene." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1902.03604" id="2165">
-        <attvalues>
-          <attvalue for="5" value=" Until now, almost all the tracking algorithms use the whole body detection \cite{voigtlaender2019mots, ullah2019single, schulter2017deep, ullah2017hierarchical, milan2014continuous, sun2019deep, ullah2018directed, bae2017confidence, ullah2016hog, son2017multi, chu2017online, ullah2018deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1706.08482" id="2166">
-        <attvalues>
-          <attvalue for="5" value=" \cite{schulter2017deep} modeled multi-target tracking as a network flow graph." />
-        </attvalues>
-      </edge>
-      <edge source="2006.06134" target="1708.02843" id="2167">
-        <attvalues>
-          <attvalue for="5" value=" \cite{chu2017online} proposed a spatial-temporal attention mechanism for occulation handling and the interaction among different targets." />
-        </attvalues>
-      </edge>
-      <edge source="1911.00756" target="1808.07784" id="2168">
-        <attvalues>
-          <attvalue for="5" value=" &#09;Learning a probabilistic model for sequential data is a key step towards solving a lot of interesting problems, including analysis and deconstruction of auditory sequences \cite{turner2010statistical}, predicting the next piece of information given previously recorded data such as video frames \cite{jayaraman2018time} and text \cite{xie2017neural}, and controlling an agent to perform specific tasks (model-based reinforcement learning \cite{deisenroth2011pilco})." />
-        </attvalues>
-      </edge>
-      <edge source="1911.00756" target="1711.09534" id="2169">
-        <attvalues>
-          <attvalue for="5" value=" &#09;Learning a probabilistic model for sequential data is a key step towards solving a lot of interesting problems, including analysis and deconstruction of auditory sequences \cite{turner2010statistical}, predicting the next piece of information given previously recorded data such as video frames \cite{jayaraman2018time} and text \cite{xie2017neural}, and controlling an agent to perform specific tasks (model-based reinforcement learning \cite{deisenroth2011pilco})." />
-        </attvalues>
-      </edge>
-      <edge source="1911.00756" target="1605.06432" id="2170">
-        <attvalues>
-          <attvalue for="5" value="&#09;To this end, one of the more successful solutions proposed in the past for modeling dynamical systems is DVBF \cite{karl2016deep} - a framework for learning a State-Space Model given sequential observations from the environment in an unsupervised manner." />
-        </attvalues>
-      </edge>
-      <edge source="1906.01123" target="1607.08022" id="2171">
-        <attvalues>
-          <attvalue for="5" value="~\cite{ulyanov2016instance} and Jonson et al." />
-        </attvalues>
-      </edge>
-      <edge source="1906.01123" target="1603.08155" id="2172">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue they extended traditional methods~\cite{gatys2016image} and \cite{johnson2016perceptual} with a regularizer, forcing preservation of the depth map of the content image." />
-        </attvalues>
-      </edge>
-      <edge source="1906.01123" target="1703.06868" id="2173">
-        <attvalues>
-          <attvalue for="5" value=" Later architectures, such as AdaIN~\cite{huang2017arbitrary} and other (\cite{ghiasi2017exploring}, \cite{li2017universal}), allowed transferring arbitrary style without training a separate network but lacked rendering quality due to failure to preserve the depth map of the content image." />
-        </attvalues>
-      </edge>
-      <edge source="1906.01123" target="1705.06830" id="2174">
-        <attvalues>
-          <attvalue for="5" value=" Later architectures, such as AdaIN~\cite{huang2017arbitrary} and other (\cite{ghiasi2017exploring}, \cite{li2017universal}), allowed transferring arbitrary style without training a separate network but lacked rendering quality due to failure to preserve the depth map of the content image." />
-        </attvalues>
-      </edge>
-      <edge source="1906.01123" target="1705.08086" id="2175">
-        <attvalues>
-          <attvalue for="5" value=" Later architectures, such as AdaIN~\cite{huang2017arbitrary} and other (\cite{ghiasi2017exploring}, \cite{li2017universal}), allowed transferring arbitrary style without training a separate network but lacked rendering quality due to failure to preserve the depth map of the content image." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2112.00995" id="2176">
-        <attvalues>
-          <attvalue for="5" value=" SwinTrack~\cite{lin2022swintrack} stands out by leveraging transformers for feature extraction and fusion, enabling seamless interaction between the target object and the search area during tracking." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2302.02814" id="2177">
-        <attvalues>
-          <attvalue for="5" value=" Meanwhile, MixFormer~\cite{cui2022mixformer} adopts a novel approach by integrating template and test samples in the backbone network and directly regressing bounding boxes using a simple head, eliminating the need for post-processing." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2303.10100" id="2178">
-        <attvalues>
-          <attvalue for="5" value=" Hence, tracking methods that utilize masks tend to achieve higher accuracy owing to their pixel-based alignment mechanism~\cite{li2023unified, cheng2023segment, zhou2022survey, li2022locality, yang2021collaborative}." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2106.02638" id="2179">
-        <attvalues>
-          <attvalue for="5" value=" Over the years, the Associating Objects with Transformers (AOT)~\cite{yang2021associating} series has demonstrated commendable performance in video object segmentation (VOS) tasks, prompting the natural extension of applying it to video object tracking (VOT) tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2304.02643" id="2180">
-        <attvalues>
-          <attvalue for="5" value=" Fortunately, the introduction of the Segment Anything Model (SAM)~\cite{kirillov2023segment} has significantly alleviated this challenge by enabling the conversion of bounding boxes into masks with remarkable efficacy." />
-        </attvalues>
-      </edge>
-      <edge source="2307.02508" target="2012.06815" id="2181">
-        <attvalues>
-          <attvalue for="5" value=" We first convert the bounding boxes to masks in reference frames with the help of SAM and Alpha-Refine~\cite{yan2021alpha}, and then feed the mask and frames into the VOS model." />
-        </attvalues>
-      </edge>
-      <edge source="2005.11546" target="1808.01992" id="2182">
-        <attvalues>
-          <attvalue for="5" value=" Contour shape alignment with noisy image observations is a fundamental, but challenging, problem in computer vision and graphics fields, with diverse applications including skeleton/silhouette alignment \cite{alexiadis2011evaluating} (for animation re-targeting), semantic boundary alignment \cite{yu2018simultaneous} and shape-to-scan alignment \cite{hirshberg2012coregistration} etc For instance, consider the first row of Figure \ref{fig_intro}." />
-        </attvalues>
-      </edge>
-      <edge source="2005.11546" target="1604.05592" id="2183">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by the strong invasion and great success of deep convolutional neural networks (ConvNets) in various vision tasks, some recent works \cite{kanazawa2016warpnet,miao2018dilated,guan2019deformable,hanocka2018alignet,de2017end} have designed ConvNet based architectures for shape alignment/registration and shown impressive results on the datasets with limited misalignments." />
-        </attvalues>
-      </edge>
-      <edge source="2005.11546" target="1712.01651" id="2184">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by the strong invasion and great success of deep convolutional neural networks (ConvNets) in various vision tasks, some recent works \cite{kanazawa2016warpnet,miao2018dilated,guan2019deformable,hanocka2018alignet,de2017end} have designed ConvNet based architectures for shape alignment/registration and shown impressive results on the datasets with limited misalignments." />
-        </attvalues>
-      </edge>
-      <edge source="2005.11546" target="1804.08497" id="2185">
-        <attvalues>
-          <attvalue for="5" value=" Motivated by the strong invasion and great success of deep convolutional neural networks (ConvNets) in various vision tasks, some recent works \cite{kanazawa2016warpnet,miao2018dilated,guan2019deformable,hanocka2018alignet,de2017end} have designed ConvNet based architectures for shape alignment/registration and shown impressive results on the datasets with limited misalignments." />
-        </attvalues>
-      </edge>
-      <edge source="2005.11546" target="1704.06065" id="2186">
-        <attvalues>
-          <attvalue for="5" value=" We believe that this is due to (a) direct prediction of alignment warp in one-go at input resolution and (b) training with proximity-insensitive pixel-level similarity metrics (such as normalized cross correlation \cite{de2017end}) that might result in non-informative gradients as the spatial overlap between contours may not be significant." />
-        </attvalues>
-      </edge>
-      <edge source="2303.15848" target="2005.05999" id="2187">
-        <attvalues>
-          <attvalue for="5" value=" AODNet~\cite{aod}, FDMHN~\cite{fdmhn}, PFFNet~\cite{pff}, GCANet~\cite{tang2019single}, MGBL~\cite{mgbl}, DehazeFlow~\cite{li2021dehazeflow}, UDDN~\cite{liu2022image} and PSD~\cite{psd} can be dehazed in real time on a single 24G RAM RTX3090 GPU." />
-        </attvalues>
-      </edge>
-      <edge source="2303.15848" target="1810.02283" id="2188">
-        <attvalues>
-          <attvalue for="5" value=" AODNet~\cite{aod}, FDMHN~\cite{fdmhn}, PFFNet~\cite{pff}, GCANet~\cite{tang2019single}, MGBL~\cite{mgbl}, DehazeFlow~\cite{li2021dehazeflow}, UDDN~\cite{liu2022image} and PSD~\cite{psd} can be dehazed in real time on a single 24G RAM RTX3090 GPU." />
-        </attvalues>
-      </edge>
-      <edge source="2303.15848" target="2112.08088" id="2189">
-        <attvalues>
-          <attvalue for="5" value=" AODNet~\cite{aod}, FDMHN~\cite{fdmhn}, PFFNet~\cite{pff}, GCANet~\cite{tang2019single}, MGBL~\cite{mgbl}, DehazeFlow~\cite{li2021dehazeflow}, UDDN~\cite{liu2022image} and PSD~\cite{psd} can be dehazed in real time on a single 24G RAM RTX3090 GPU." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1712.03917" id="2190">
-        <attvalues>
-          <attvalue for="5" value=" As commercial depth cameras have been released and become more common, depth-based hand pose estimation methods have attracted significant research interest in recent decades~\cite{yuan2018depth}." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1705.09606" id="2191">
-        <attvalues>
-          <attvalue for="5" value="~\cite{madadi2017end} designed a hierarchically tree-like, structured CNN using five branches to model the five fingers and an additional branch to model the palm orientation." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1711.07399" id="2192">
-        <attvalues>
-          <attvalue for="5" value=" The proposed HCRNN operates approximately 80, seven, and three times faster than V2V-PoseNet~\cite{moon2018v2v}, Point-to-Point~\cite{ge2018point}, and A2J~\cite{xiong2019a2j}, respectively." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1908.09999" id="2193">
-        <attvalues>
-          <attvalue for="5" value=" The proposed HCRNN operates approximately 80, seven, and three times faster than V2V-PoseNet~\cite{moon2018v2v}, Point-to-Point~\cite{ge2018point}, and A2J~\cite{xiong2019a2j}, respectively." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1609.09698" id="2194">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, in terms of inference speed, the proposed HCRNN is in the 2nd place behind Feedback~\cite{oberweger2015training}, while HCRNN improves the estimation accuracy of Feedback about 40\%." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07424" target="1507.05717" id="2195">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the recent study in which sequential features of a sequence-like object are obtained in a single image~\cite{shi2016end}, we first extract the sequential features of joints by applying the combination of convolutional and fully-connected~(FC) layers to the feature from the encoder network." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11141" target="2106.09672" id="2196">
-        <attvalues>
-          <attvalue for="5" value=" Image representations are a critical building block of computer vision applications~\cite{douze20212021}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11141" target="1612.06321" id="2197">
-        <attvalues>
-          <attvalue for="5" value=" Traditionally, research on image embedding learning has been conducted with a focus on per-domain models~\cite{araujo_retrieval, radenovic2018fine, google_landmark}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.11141" target="2001.05027" id="2198">
-        <attvalues>
-          <attvalue for="5" value=" Then the image retrieval task\cite{cao2020unifying, araujo2020deep, radenovic2018fine} considers an index-reference database of images $Z=\{z_1, z_2, \dots, z_n\}$, and a given a query image $x$, we calculate \begin{equation} \underset{Z}{\mathrm{argmin}} \lVert \phi(x) - \phi(z_i) \rVert^{2}_{2} \end{equation} finally retrieve the top-$k$ most similar images (eg those that minimize the previous equation)." />
-        </attvalues>
-      </edge>
-      <edge source="2011.02172" target="1705.03098" id="2199">
-        <attvalues>
-          <attvalue for="5" value=" \ifjapanese 近年の深層学習の研究の発展に伴い、3次元姿勢推定の精度も大きく向上している。 その中でも、タスクを画像からの2次元姿勢推定と、推定された2次元姿勢からの3次元姿勢推定に分割するアプローチの有効性が確認されている\cite{martinez2017simple}。2次元姿勢推定には多様性の高いデータセットを用いて学習された既製の2次元姿勢推定器を利用できる点、また3次元姿勢推定の際に画像の外観情報を敢えて使用しないことで過学習を軽減している点が本アプローチの有効性に寄与していると考えられる。 また、近年では動画における時系列情報の活用を図る研究が増えつつある。 それらの研究では、フレーム毎に推定された2次元姿勢をモデルの入力とするアプローチが主流となっている\cite{rayat2018exploiting,pavllo20193d,cai2019exploiting}。 In recent years, there has been an increasing number of studies on the use of temporal information in video." />
-        </attvalues>
-      </edge>
-      <edge source="2011.02172" target="1811.11742" id="2200">
-        <attvalues>
-          <attvalue for="5" value=" \ifjapanese 本稿では動画を入力した3次元姿勢推定において、動画の各フレームに対して3次元姿勢を推定し、それら複数フレームの推定結果を用いて最終的な3次元姿勢を推定するアプローチを提案する。 本手法は2次元姿勢よりも情報量の多い画像からの方が精度良く関節点の奥行きを推定できるという仮定に基づいている。2次元姿勢を中間的に使用する場合とは異なり、関節点の奥行きに関する情報を途中で損なうことのない推論が可能である。 本アプローチにおいて、前段と後段それぞれのモデルには任意のものを使用することができる。我々は、前段モデルには2次元、3次元データセットを併用した学習が可能なIntegral Regression \cite{sun2018integral}を採用し、後段モデルには2次元姿勢からの3次元姿勢推定で有効性が確認されている1次元畳み込みニューラルネットワーク\cite{pavllo20193d}を使用する。 公開データセットを用いた評価実験により提案手法の有効性を示す。 In this paper, we propose a novel 3D pose estimation approach that first estimates a 3D pose for each frame (first stage) and then aggregates the multi-frame predictions for estimating the final 3D poses (second stage)." />
-        </attvalues>
-      </edge>
-      <edge source="2011.02172" target="1711.08229" id="2201">
-        <attvalues>
-          <attvalue for="5" value=" \ifjapanese 本稿では動画を入力した3次元姿勢推定において、動画の各フレームに対して3次元姿勢を推定し、それら複数フレームの推定結果を用いて最終的な3次元姿勢を推定するアプローチを提案する。 本手法は2次元姿勢よりも情報量の多い画像からの方が精度良く関節点の奥行きを推定できるという仮定に基づいている。2次元姿勢を中間的に使用する場合とは異なり、関節点の奥行きに関する情報を途中で損なうことのない推論が可能である。 本アプローチにおいて、前段と後段それぞれのモデルには任意のものを使用することができる。我々は、前段モデルには2次元、3次元データセットを併用した学習が可能なIntegral Regression \cite{sun2018integral}を採用し、後段モデルには2次元姿勢からの3次元姿勢推定で有効性が確認されている1次元畳み込みニューラルネットワーク\cite{pavllo20193d}を使用する。 公開データセットを用いた評価実験により提案手法の有効性を示す。 In this paper, we propose a novel 3D pose estimation approach that first estimates a 3D pose for each frame (first stage) and then aggregates the multi-frame predictions for estimating the final 3D poses (second stage)." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1503.02619" id="2202">
-        <attvalues>
-          <attvalue for="5" value=" Robust estimation of two-view geometry is a fundamental problem in numerous computer vision applications, \eg, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, multi-model fitting~\cite{isack2012energy,pham2014interacting,barath2018multi}, initial pose recovery of Structure-from-Motion~\cite{schoenberger2016sfm,schoenberger2016mvs} and Simultaneous Localization and Mapping (SLAM)~\cite{mur2015orb} pipelines." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1706.00827" id="2203">
-        <attvalues>
-          <attvalue for="5" value=" Robust estimation of two-view geometry is a fundamental problem in numerous computer vision applications, \eg, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, multi-model fitting~\cite{isack2012energy,pham2014interacting,barath2018multi}, initial pose recovery of Structure-from-Motion~\cite{schoenberger2016sfm,schoenberger2016mvs} and Simultaneous Localization and Mapping (SLAM)~\cite{mur2015orb} pipelines." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1502.00956" id="2204">
-        <attvalues>
-          <attvalue for="5" value=" Robust estimation of two-view geometry is a fundamental problem in numerous computer vision applications, \eg, wide baseline matching~\cite{pritchett1998wide,matas2004robust,mishkin2015mods}, multi-model fitting~\cite{isack2012energy,pham2014interacting,barath2018multi}, initial pose recovery of Structure-from-Motion~\cite{schoenberger2016sfm,schoenberger2016mvs} and Simultaneous Localization and Mapping (SLAM)~\cite{mur2015orb} pipelines." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1803.07469" id="2205">
-        <attvalues>
-          <attvalue for="5" value=" More recently, MAGSAC~\cite{barath2019magsac} and MAGSAC++~\cite{barath2020magsac++} have been proposed to alleviate the dependence on manual threshold selection by marginalizing over an acceptable range of noise scales." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1912.05909" id="2206">
-        <attvalues>
-          <attvalue for="5" value=" Third, as a technical contribution, we combine two state-of-the-art algorithms, \ie, MAGSAC++~\cite{barath2020magsac++} and NG-RANSAC~\cite{brachmann2019neural}, to improve the relative pose estimation accuracy on a wide range of scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="2106.10240" id="2207">
-        <attvalues>
-          <attvalue for="5" value=" As shown in the recent survey~\cite{ma2021image}, MAGSAC++ is currently the most accurate RANSAC variant as in \cite{ivashechkin2021vsac}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1905.04132" id="2208">
-        <attvalues>
-          <attvalue for="5" value=" Third, as a technical contribution, we combine two state-of-the-art algorithms, \ie, MAGSAC++~\cite{barath2020magsac++} and NG-RANSAC~\cite{brachmann2019neural}, to improve the relative pose estimation accuracy on a wide range of scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1711.05971" id="2209">
-        <attvalues>
-          <attvalue for="5" value=" The first paper on the topic, Context Normalization Networks (PointCN)~\cite{cne2018} proposes the use of PointNet (MLP) with batch normalization as a context mechanism." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1908.04964" id="2210">
-        <attvalues>
-          <attvalue for="5" value=" The OANet algorithm~\cite{oanet2019} introduces differentiable pooling and unpooling blocks for correspondence filtering." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="2101.00591" id="2211">
-        <attvalues>
-          <attvalue for="5" value=" More recently, CLNet~\cite{clnet2021} introduces several algorithmic and architectural improvements to remove gross outliers with iterative pruning." />
-        </attvalues>
-      </edge>
-      <edge source="2111.14093" target="1712.07629" id="2212">
-        <attvalues>
-          <attvalue for="5" value=" Additionally, we show that this extra information can be obtained for any features, \eg SuperPoint~\cite{detone2018superpoint}, allowing to seamlessly integrate the proposed loss with recent detectors." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1808.09102" id="2213">
-        <attvalues>
-          <attvalue for="5" value=" Instead of learning to generate the attention map,~\cite{liu2018localization} produces attribute-specific features by means of the class activation map (CAM)~\cite{zhou2016learning}, which is initially designed to build a generic localizable deep representation." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1901.05742" id="2214">
-        <attvalues>
-          <attvalue for="5" value=" They can be basically abstracted as emphasizing on better attribute localization to mitigate the accuracy drop from inferring on irrelevant area~\cite{liu2018localization,DBLP:conf/aaai/JiaGHCH22}, or involving additional supervision or information, like pose keypoints~\cite{liu2018localization} and pedestrian video clip~\cite{chen2019temporal,2020Pedestrian}, to guide PAR under explicit assumptions regarding body topological structure or temporal context, etc." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1707.02240" id="2215">
-        <attvalues>
-          <attvalue for="5" value=" For work striving to enhance the attribute localization, ~\cite{fabbri2017generative,2017Learning} split the body image vertically, by a manually defined fixed strategy, into three parts and feeds each part into an individual network for feature extracting." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1710.06555" id="2216">
-        <attvalues>
-          <attvalue for="5" value=" For work striving to enhance the attribute localization, ~\cite{fabbri2017generative,2017Learning} split the body image vertically, by a manually defined fixed strategy, into three parts and feeds each part into an individual network for feature extracting." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1709.09930" id="2217">
-        <attvalues>
-          <attvalue for="5" value=" As a further step,~\cite{liu2017hydraplus} proposed the HydraPlus-Net, learning to locate attributes of different scale with a multi-directional attention modules." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1512.04150" id="2218">
-        <attvalues>
-          <attvalue for="5" value=" Instead of learning to generate the attention map,~\cite{liu2018localization} produces attribute-specific features by means of the class activation map (CAM)~\cite{zhou2016learning}, which is initially designed to build a generic localizable deep representation." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="2107.03576" id="2219">
-        <attvalues>
-          <attvalue for="5" value=" Albeit lightweight, our method establishes the-state-of-art performance on various realistic datasets like the PETAzs and RAPzs~\cite{jia2021rethinking}, with considerable margins over previous approaches." />
-        </attvalues>
-      </edge>
-      <edge source="2307.15252" target="1705.10659" id="2220">
-        <attvalues>
-          <attvalue for="5" value=" Seeking for better feature learning of information exchanges, a thread of literature~\cite{2016Video,wang2017discovering,2017Multi,2020Correlation,2022Label2Label} aims to enhance the modeling of interdependencies among attributes, under the hypothesis that such correlations provide a contextual constraint complementary to visual attributes recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2212.12668" target="2211.01566" id="2221">
-        <attvalues>
-          <attvalue for="5" value=" \end{enumerate} Mitigating these drawbacks, we build on our previous work \cite{eapen2022narpa} and propose a differentiable rendering pipeline for pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="2001.04193" id="2222">
-        <attvalues>
-          <attvalue for="5" value=" Despite the success of deep learning, one of the main difficulties resides in the lack of large publicly available Re-ID datasets for training~\cite{deepSurvey2020}, mainly because manually labeling images is a very difficulty task, especially in open-world scenarios~\cite{airport}, where the datasets may increase dynamically." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1605.09653" id="2223">
-        <attvalues>
-          <attvalue for="5" value=" Despite the success of deep learning, one of the main difficulties resides in the lack of large publicly available Re-ID datasets for training~\cite{deepSurvey2020}, mainly because manually labeling images is a very difficulty task, especially in open-world scenarios~\cite{airport}, where the datasets may increase dynamically." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1406.4216" id="2224">
-        <attvalues>
-          <attvalue for="5" value=" Aiming at further improving the quality of the results, rather than using traditional distance measures, researchers have proposed metric learning approaches for Re-ID~\cite{paperLOMO, paperLargeScaleML, paperModelFeatDistClass, paperKernelizedReidML, zhang2016learning}, most of them based on supervised models." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1512.03385" id="2225">
-        <attvalues>
-          <attvalue for="5" value=" Among the works, existing networks have been trained for Re-ID~\cite{paperRESNET, paperMobileNetv2}, and some new architectures have been proposed solely focusing on Re-ID~\cite{paperOSNET, paperOSNET-IBN-AIN, paperMLFN, paperHACNN, paperPCB}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1905.00953" id="2226">
-        <attvalues>
-          <attvalue for="5" value=" Among the works, existing networks have been trained for Re-ID~\cite{paperRESNET, paperMobileNetv2}, and some new architectures have been proposed solely focusing on Re-ID~\cite{paperOSNET, paperOSNET-IBN-AIN, paperMLFN, paperHACNN, paperPCB}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="2004.03547" id="2227">
-        <attvalues>
-          <attvalue for="5" value=" As an alternative solution, unsupervised approaches~\cite{paperARN, paperECN, paperMAR, paperTAUDL, paperUTAL, paperUnsCrossDom2019, paperDECAMEL,PaperCrossSimTIP20,PaperUnsReIDSimLenCVPR20} have been attracting a lot of attention from the research community, especially because, once labeled data are not required, the methods become more suitable for real-world scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="2002.05046" id="2228">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods often suffer from performance degradation and often require a large proportion of expensive cross-view pairwise labeling~\cite{paperIntraCam2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="2107.13904" id="2229">
-        <attvalues>
-          <attvalue for="5" value=" There are approaches that mitigate these issues; among them there is the Intra-Camera Supervised (ICS)~\cite{paperIntraCam2021, paperIntraCam2019}, Multi-Task Multi-Label (MATE)~\cite{paperIntraCam2021, paperIntraCam2019}, and Cross-camera Feature Prediction~\cite{paperCrossCamSup2021}, which were recently proposed." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1904.03832" id="2230">
-        <attvalues>
-          <attvalue for="5" value=" Weakly supervised Re-ID is very challenging, since it is rather difficult to model the considerable variances across camera views (eg occlusion and illumination) without using strong label data~\cite{weaklyreid2019, paperIntraCam2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1904.03845" id="2231">
-        <attvalues>
-          <attvalue for="5" value=" Recently,~\cite{wang2020weakly} proposed to replace image-level annotations with bag-level annotations." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1804.09347" id="2232">
-        <attvalues>
-          <attvalue for="5" value=" As an alternative solution, unsupervised approaches~\cite{paperARN, paperECN, paperMAR, paperTAUDL, paperUTAL, paperUnsCrossDom2019, paperDECAMEL,PaperCrossSimTIP20,PaperUnsReIDSimLenCVPR20} have been attracting a lot of attention from the research community, especially because, once labeled data are not required, the methods become more suitable for real-world scenarios." />
-        </attvalues>
-      </edge>
-      <edge source="2304.14321" target="1701.08398" id="2233">
-        <attvalues>
-          <attvalue for="5" value=" In~\cite{cuhk03_new_protocol}, the original distances among images are improved by calculating Jaccard correlation scores of each ranked list." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="1610.02984" id="2234">
-        <attvalues>
-          <attvalue for="5" value=" Person re-identification (Re-ID) is the task of retrieving images of a specified person under non-overlapping cameras \cite{zheng2016person}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2103.11568" id="2235">
-        <attvalues>
-          <attvalue for="5" value=" Existing unsupervised works usually adopt pseudo-label-based methods, which alternate between generating pseudo labels by clustering \cite{lin2019bottom,dai2022cluster}, softened labels \cite{yu2019unsupervised,lin2020unsupervised} or label refinement \cite{cho2022part} and training the model with the generated labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="1903.06325" id="2236">
-        <attvalues>
-          <attvalue for="5" value=" Existing unsupervised works usually adopt pseudo-label-based methods, which alternate between generating pseudo labels by clustering \cite{lin2019bottom,dai2022cluster}, softened labels \cite{yu2019unsupervised,lin2020unsupervised} or label refinement \cite{cho2022part} and training the model with the generated labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2004.03547" id="2237">
-        <attvalues>
-          <attvalue for="5" value=" SSL \cite{lin2020unsupervised} proposes a cross-camera encouragement term to reduce intra-camera negative pairs by increasing the dissimilarity to image pairs under the same camera." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2203.14675" id="2238">
-        <attvalues>
-          <attvalue for="5" value=" Existing unsupervised works usually adopt pseudo-label-based methods, which alternate between generating pseudo labels by clustering \cite{lin2019bottom,dai2022cluster}, softened labels \cite{yu2019unsupervised,lin2020unsupervised} or label refinement \cite{cho2022part} and training the model with the generated labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2004.09228" id="2239">
-        <attvalues>
-          <attvalue for="5" value=" MMCL \cite{wang2020unsupervised} trains the Re-ID model to recognize each image as multiple labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2006.02713" id="2240">
-        <attvalues>
-          <attvalue for="5" value=" SPCL \cite{ge2020self} proposes a unified contrastive learning framework to seek reliable labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2106.06133" id="2241">
-        <attvalues>
-          <attvalue for="5" value=" RLCC \cite{zhang2021refining} exploits cluster consensus to estimate pseudo-label similarity and refines pseudo-labels." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2012.10674" id="2242">
-        <attvalues>
-          <attvalue for="5" value=" CAP \cite{wang2021camera} splits each cluster into multiple camera-aware proxies according to the camera ID." />
-        </attvalues>
-      </edge>
-      <edge source="2304.12634" target="2103.11658" id="2243">
-        <attvalues>
-          <attvalue for="5" value=" IICS \cite{xuan2021intra} alternates between intra-camera and inter-camera iterative training to optimize feature encoder." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1612.00593" id="2244">
-        <attvalues>
-          <attvalue for="5" value=" However, the effectiveness of neural network-based methods is often limited by their high data requirements \cite{liu2019deep}, high computational costs \cite{qi2017pointnet}, and vulnerability to out-of-distribution samples, eg, adversarial attacks \cite{liao2018defense,basu2014detecting,shifat2021radon}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1611.07759" id="2245">
-        <attvalues>
-          <attvalue for="5" value=" Point sets provide valuable insights about object geometry, making them useful for a variety of applications, including object detection, recognition, segmentation, and tracking in fields such as robotics, autonomous vehicles, virtual reality, and computer vision, among others \cite{qi2017pointnet,zhao2019pointweb,li2018pointcnn,chen2017multi,zhou2018voxelnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1711.06396" id="2246">
-        <attvalues>
-          <attvalue for="5" value=" Point sets provide valuable insights about object geometry, making them useful for a variety of applications, including object detection, recognition, segmentation, and tracking in fields such as robotics, autonomous vehicles, virtual reality, and computer vision, among others \cite{qi2017pointnet,zhao2019pointweb,li2018pointcnn,chen2017multi,zhou2018voxelnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="2104.03670" id="2247">
-        <attvalues>
-          <attvalue for="5" value=" They represent the surface geometry of an object in an N-dimensional space as a set of points, obtained using various scanning technologies such as LiDAR or photogrammetry \cite{xu2021toward,wang2020computational}, or by sampling a continuous probability density function over an N-D space \cite{zhou20213d}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="2004.03327" id="2248">
-        <attvalues>
-          <attvalue for="5" value=" However, modeling the set structure space for classification presents significant challenges due to the sparsity and noise in data, the accumulation of spatial deformations (such as affine deformations) in real-world point set data, and the high dimensionality of point sets, among other factors \cite{pomerleau2015review,wang2020cascaded,qi2017pointnet,zeng20193d}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1803.07252" id="2249">
-        <attvalues>
-          <attvalue for="5" value=" However, modeling the set structure space for classification presents significant challenges due to the sparsity and noise in data, the accumulation of spatial deformations (such as affine deformations) in real-world point set data, and the high dimensionality of point sets, among other factors \cite{pomerleau2015review,wang2020cascaded,qi2017pointnet,zeng20193d}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="2112.05872" id="2250">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, defining a metric or distance function for point set classification is challenging due to the permutation invariant nature of point sets resulting from the arbitrary order of points in a set \cite{qi2017pointnet,lu2021slosh}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1711.02512" id="2251">
-        <attvalues>
-          <attvalue for="5" value=" Over the last few decades, point set classification methods have evolved from relying on feature engineering \cite{radenovic2018fine,wang2020deep,acharya2018covariance,zhang2019fspool} to utilizing deep neural networks to learn representations and use them in classification tasks \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1904.06836" id="2252">
-        <attvalues>
-          <attvalue for="5" value=" Over the last few decades, point set classification methods have evolved from relying on feature engineering \cite{radenovic2018fine,wang2020deep,acharya2018covariance,zhang2019fspool} to utilizing deep neural networks to learn representations and use them in classification tasks \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1805.04855" id="2253">
-        <attvalues>
-          <attvalue for="5" value=" Over the last few decades, point set classification methods have evolved from relying on feature engineering \cite{radenovic2018fine,wang2020deep,acharya2018covariance,zhang2019fspool} to utilizing deep neural networks to learn representations and use them in classification tasks \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1906.02795" id="2254">
-        <attvalues>
-          <attvalue for="5" value=" Over the last few decades, point set classification methods have evolved from relying on feature engineering \cite{radenovic2018fine,wang2020deep,acharya2018covariance,zhang2019fspool} to utilizing deep neural networks to learn representations and use them in classification tasks \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1801.07829" id="2255">
-        <attvalues>
-          <attvalue for="5" value=" They have demonstrated to achieve high accuracy in several classification tasks, and are also suitable for parallel implementation using graphical processing units (GPUs) \cite{li2018pointcnn,qi2017pointnet,wang2019dynamic}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="1712.02976" id="2256">
-        <attvalues>
-          <attvalue for="5" value=" However, the effectiveness of neural network-based methods is often limited by their high data requirements \cite{liu2019deep}, high computational costs \cite{qi2017pointnet}, and vulnerability to out-of-distribution samples, eg, adversarial attacks \cite{liao2018defense,basu2014detecting,shifat2021radon}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.10015" target="2004.03669" id="2257">
-        <attvalues>
-          <attvalue for="5" value=" However, the effectiveness of neural network-based methods is often limited by their high data requirements \cite{liu2019deep}, high computational costs \cite{qi2017pointnet}, and vulnerability to out-of-distribution samples, eg, adversarial attacks \cite{liao2018defense,basu2014detecting,shifat2021radon}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="2204.10776" id="2258">
-        <attvalues>
-          <attvalue for="5" value=" To this end, some unseen object pose estimation methods have been recently proposed \cite{Gen6D, MegaPose, GigaPose, SAM-6D, FoundationPose}, which can be further classified into CAD model-based and manual reference view-based methods." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="1711.00199" id="2259">
-        <attvalues>
-          <attvalue for="5" value=" However, these methods exhibit deficiencies in accuracy and robustness when confronted with complex scenes\cite{xiang2017posecnn, wang2019densefusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="1901.04780" id="2260">
-        <attvalues>
-          <attvalue for="5" value=" Early methods were mainly instance-level \cite{wang2019densefusion, peng2019pvnet, he2020pvn3d, Li2019CDPN, Zakharov2019DPOD}, trained to estimate the pose of specific object instances." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="1911.04231" id="2261">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, many instance-level methods\cite{he2020pvn3d, he2021ffb6d} require CAD models of the objects." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="1901.02970" id="2262">
-        <attvalues>
-          <attvalue for="5" value=" \cite{NOCS} proposed the first category-level object pose and size estimation method." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="1905.06658" id="2263">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Du_Grasping_review} exclusively examined object pose estimation within the context of the robotic grasping task, which limits the readers to understand object pose estimation only from the perspective of a single specific application." />
-        </attvalues>
-      </edge>
-      <edge source="2405.07801" target="2105.14291" id="2264">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Fan_monocular_review} provided valuable insights into RGB image-based object pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1808.04134" id="2265">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in spite of several works on Chinese handwritten characters recognition \cite{8563234,9085435,8869841}, there are no reported works on performance of deep learning for Chinese handwritten numbers." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1811.06347" id="2266">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in spite of several works on Chinese handwritten characters recognition \cite{8563234,9085435,8869841}, there are no reported works on performance of deep learning for Chinese handwritten numbers." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1807.04899" id="2267">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, analysis dictionaries, which was recently introduced, rely on the assumption that the input data can be transformed into a latent sparse subspace by its corresponding dictionary \cite{8745700}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1905.11543" id="2268">
-        <attvalues>
-          <attvalue for="5" value=" For instance, an analysis discriminative dictionary learning has been proposed in \cite{8038251} to process two-dimensional images." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1809.04185" id="2269">
-        <attvalues>
-          <attvalue for="5" value=" A method called deep micro-dictionary learning and coding network was proposed in \cite{8658671} which includes most of the standard deep learning layers (pooling, fully, connected, input/output, etc)." />
-        </attvalues>
-      </edge>
-      <edge source="2003.11700" target="1907.03211" id="2270">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{1446833}, scalability and speed of deep learning were combined with dictionary learning to significantly reduce the number of parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2107.11810" target="1903.07047" id="2271">
-        <attvalues>
-          <attvalue for="5" value=" \item[--] We provide example-derivations and open-source code which serve as a tutorial for applying~\cite{AKKSZ19} to a range of outlier removal problems in computer vision." />
-        </attvalues>
-      </edge>
-      <edge source="2305.11365" target="1611.05267" id="2272">
-        <attvalues>
-          <attvalue for="5" value=" Automated detection and segmentation of human activities from an egocentric perspective have numerous applications in fields such as mixed reality, human behavior analysis, and robotics \cite{lea2017temporal,kwon2022context}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.11365" target="2204.12223" id="2273">
-        <attvalues>
-          <attvalue for="5" value=" Automated detection and segmentation of human activities from an egocentric perspective have numerous applications in fields such as mixed reality, human behavior analysis, and robotics \cite{lea2017temporal,kwon2022context}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.11365" target="2203.14104" id="2274">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in action segmentation models have shown promising results through the use of cutting-edge visual language feature representations \cite{li2022bridge, wang2021actionclip}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1406.2283" id="2275">
-        <attvalues>
-          <attvalue for="5" value=" Despite the remarkable progress of deep monocular depth estimation in recent years~\cite{eigen2014depth,godard2017unsupervised,fu2018deep}, it cannot estimate a correct depth map without sufficient contextual information due to the lack of a physical depth cue, for instance, in a scene without the ground." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1609.03677" id="2276">
-        <attvalues>
-          <attvalue for="5" value=" Despite the remarkable progress of deep monocular depth estimation in recent years~\cite{eigen2014depth,godard2017unsupervised,fu2018deep}, it cannot estimate a correct depth map without sufficient contextual information due to the lack of a physical depth cue, for instance, in a scene without the ground." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1806.02446" id="2277">
-        <attvalues>
-          <attvalue for="5" value=" Despite the remarkable progress of deep monocular depth estimation in recent years~\cite{eigen2014depth,godard2017unsupervised,fu2018deep}, it cannot estimate a correct depth map without sufficient contextual information due to the lack of a physical depth cue, for instance, in a scene without the ground." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1805.08318" id="2278">
-        <attvalues>
-          <attvalue for="5" value=" In order to estimate a correct depth map under shift-variant PSFs, we add positional information as an additional branch by a self-attention mechanism~\cite{zhang2018self}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1703.04977" id="2279">
-        <attvalues>
-          <attvalue for="5" value=" To handle various uncertainties, we improve the loss function based on Bayesian deep learning~\cite{kendall2017uncertainties} for stabilizing the training." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00329" target="1805.03096" id="2280">
-        <attvalues>
-          <attvalue for="5" value=" Second, our method requires much computation time, for example, about 50 seconds (NVIDIA Geforce GTX 1080 ti), because of the patch-based architecture~\cite{bailer2018fast}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10110" target="1812.03982" id="2281">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, much progresses~\cite{Fan2019ShiftingMA,Feichtenhofer2019SlowFastNF} have been witnessed in handling unconstrained videos, but there remains a large room to improve that has not yet been adequately explored." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10110" target="2003.12039" id="2282">
-        <attvalues>
-          <attvalue for="5" value=" Motion (\eg, optical flow~\cite{Teed2020RAFTRA} and trajectory~\cite{Keuper2015MotionTS}) and appearance features (\eg, color~\cite{Wei2012GeodesicSU} and superpixel~\cite{Wang2015SaliencyawareGV} segmentation) are both crucial cues for understanding the dynamic salient object with an unconstrained background." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10110" target="2105.08468" id="2283">
-        <attvalues>
-          <attvalue for="5" value=" Several works have been made to develop spatial-temporal convolution neural networks (CNNs) for learning discriminative appearance and motion features, in which recurrent memory~\cite{Li2018FlowGR,Song2018PyramidDD} and 3D convolution~\cite{ji2021pnsnet} are frequently used." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10110" target="1909.07061" id="2284">
-        <attvalues>
-          <attvalue for="5" value=" Thus, recent cutting-edge methods (\eg,\cite{Li2019MotionGA}) encourage utilising the motion-dominated features to guide the encoding/decoding of appearance-dominated features, achieving satisfactory results." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1312.4400" id="2285">
-        <attvalues>
-          <attvalue for="5" value=" NIN \cite{NIN} for the first time replaced traditional flattening-based fully-connected (FC) layers with global average pooling (GAP), which reduces model size and enables input images of arbitrary size." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1409.4842" id="2286">
-        <attvalues>
-          <attvalue for="5" value=" GoogLeNet \cite{googlenet} extracts and fuses multi-scale convolutional features to enrich feature representation." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1512.03385" id="2287">
-        <attvalues>
-          <attvalue for="5" value=" ResNet \cite{resnet} introduces residual learning to facilitate optimization of deep networks, improving model performance noticeably." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1605.07146" id="2288">
-        <attvalues>
-          <attvalue for="5" value=" Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1611.05431" id="2289">
-        <attvalues>
-          <attvalue for="5" value=" Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1608.06993" id="2290">
-        <attvalues>
-          <attvalue for="5" value=" Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1704.06904" id="2291">
-        <attvalues>
-          <attvalue for="5" value=" Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="2004.08955" id="2292">
-        <attvalues>
-          <attvalue for="5" value=" Later on, more variants and derivatives of ResNet are proposed to make further improvements by increasing network width \cite{wideresnet}, adopting multi-path group convolution \cite{ResNeXt}, extending skip connection to dense connections \cite{huang2017densely}, and introducing attention mechanism \cite{wang2017residual,zhang2020resnest}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1704.04861" id="2293">
-        <attvalues>
-          <attvalue for="5" value=" Besides, networks were also specially designed for lightweight deployment by using more efficient convolution or convolutional blocks \cite{mobilenets,squeezenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1710.09412" id="2294">
-        <attvalues>
-          <attvalue for="5" value=" Mixup \cite{mixup} blends two images and their corresponding labels with a random proportion as training samples." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1708.04552" id="2295">
-        <attvalues>
-          <attvalue for="5" value=" Cutout \cite{cutout} randomly removes a rectangular patch from images to guide the network to learn from local views." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1905.04899" id="2296">
-        <attvalues>
-          <attvalue for="5" value=" CutMix \cite{cutmix} randomly cuts and pastes rectangular patches among training images with their corresponding labels mixed up in proportion to patch area." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1512.04150" id="2297">
-        <attvalues>
-          <attvalue for="5" value=" CAM \cite{CAM} produces class activation maps that highlight class-discriminative regions of images with GAP and a single-layer classifier head." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1610.02391" id="2298">
-        <attvalues>
-          <attvalue for="5" value=" Grad-CAM \cite{Grad-CAM} extends such object localization ability to arbitrary CNN architectures by computing gradients of the logits with respect to intermediate feature maps." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1709.01507" id="2299">
-        <attvalues>
-          <attvalue for="5" value=" SENet \cite{SENet} built a Squeeze-and-Excitation module to selectively enhance or depress features in channel dimension." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1807.06521" id="2300">
-        <attvalues>
-          <attvalue for="5" value=" CBAM \cite{cbam} sequentially stacked two modules for channel-wise and spatial attentions respectively." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1903.06586" id="2301">
-        <attvalues>
-          <attvalue for="5" value=" SKNet \cite{SKNet} proposed a multi-branch attention module for adaptive convolution kernel size selection." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12943" target="1904.11492" id="2302">
-        <attvalues>
-          <attvalue for="5" value=" GCNet \cite{gcnet} improved model performance by designing an efficient self-attention block that captures global context information." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="1703.07402" id="2303">
-        <attvalues>
-          <attvalue for="5" value=" In the past few years, the MOT task is dominated by the tracking-by-detection (TBD) paradigm \cite{wojke2017simple, du2022strongsort}, which performs detection per frame and formulates the MOT problem as a data association task." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2004.01177" id="2304">
-        <attvalues>
-          <attvalue for="5" value=" We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline, eg, SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2203.05482" id="2305">
-        <attvalues>
-          <attvalue for="5" value=" proposes Model Soups to average weights of multiple models to improve the classification accuracy \cite{wortsman2022model}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="1610.02242" id="2306">
-        <attvalues>
-          <attvalue for="5" value=" To estimate more stable and accurate pseudo labels for semi-supervised image classification, Temporal Ensembling \cite{laine2016temporal} aggregates the predictions of multiple previous network evaluations into an ensemble prediction." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="1910.13302" id="2307">
-        <attvalues>
-          <attvalue for="5" value=" For the object detection task, Soft-NMS \cite{bodla1704improving} and WBF \cite{solovyev2021weighted} are widely used to combine results from multiple detectors." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2007.15576" id="2308">
-        <attvalues>
-          <attvalue for="5" value=" proposes the Layer-wise Aggregation Discriminative Model (LADM) \cite{peng2020dense}, which uses the weighted average of predictions from three softmax layers to judge whether a detection box represents a person or not." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2202.11983" id="2309">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by SoftNMS, TrackNMS is designed in GIAOTracker \cite{du2021giaotracker} to fuse multiple tracking results." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="1609.01775" id="2310">
-        <attvalues>
-          <attvalue for="5" value=" Instead, the instance-based metrics, eg, MOTA \cite{bernardin2008evaluating}, IDF1\cite{ristani2016performance} and HOTA \cite{luiten2021hota}, are more common and reasonable evaluation metrics for the MOT task." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2009.07736" id="2311">
-        <attvalues>
-          <attvalue for="5" value=" Instead, the instance-based metrics, eg, MOTA \cite{bernardin2008evaluating}, IDF1\cite{ristani2016performance} and HOTA \cite{luiten2021hota}, are more common and reasonable evaluation metrics for the MOT task." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="1603.00831" id="2312">
-        <attvalues>
-          <attvalue for="5" value=" We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline, eg, SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2105.11595" id="2313">
-        <attvalues>
-          <attvalue for="5" value=" We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline, eg, SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2012.15460" id="2314">
-        <attvalues>
-          <attvalue for="5" value=" We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline, eg, SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2004.01888" id="2315">
-        <attvalues>
-          <attvalue for="5" value=" We conduct experiments on the MOT17 dataset \cite{milan2016mot16} with multiple state-of-the-art (SOTA) trackers as baseline, eg, SiamMOT\cite{shuai2021siammot}, CenterTrack\cite{zhou2020tracking}, TransTrack\cite{sun2020transtrack}, and FairMOT\cite{zhang2021fairmot}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.05278" target="2202.13514" id="2316">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, it can be integrated with other post-processing algorithms, eg, interpolation \cite{du2022strongsort}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1906.08332" id="2317">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning methods have made a significant progress on pedestrian Re-ID task \cite{Luo_2019_Strong_TMM,Luo_2019_CVPR_Workshops,sun2018beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1903.07071" id="2318">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning methods have made a significant progress on pedestrian Re-ID task \cite{Luo_2019_Strong_TMM,Luo_2019_CVPR_Workshops,sun2018beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1711.09349" id="2319">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, deep learning methods have made a significant progress on pedestrian Re-ID task \cite{Luo_2019_Strong_TMM,Luo_2019_CVPR_Workshops,sun2018beyond}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1711.07027" id="2320">
-        <attvalues>
-          <attvalue for="5" value=" Based on the situation, domain adaptation has attracted much attention on the pedestrian Re-ID task \cite{deng2018image,zhang2019self,fu2019self,chen2019instance,song2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1907.13315" id="2321">
-        <attvalues>
-          <attvalue for="5" value=" Based on the situation, domain adaptation has attracted much attention on the pedestrian Re-ID task \cite{deng2018image,zhang2019self,fu2019self,chen2019instance,song2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1811.10144" id="2322">
-        <attvalues>
-          <attvalue for="5" value=" Based on the situation, domain adaptation has attracted much attention on the pedestrian Re-ID task \cite{deng2018image,zhang2019self,fu2019self,chen2019instance,song2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1807.11334" id="2323">
-        <attvalues>
-          <attvalue for="5" value=" Based on the situation, domain adaptation has attracted much attention on the pedestrian Re-ID task \cite{deng2018image,zhang2019self,fu2019self,chen2019instance,song2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2012.13498" target="1812.02162" id="2324">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in VisDA-2020, a synthetic dataset are provided as the labeled source domain \cite{sun2019dissecting} and a real-world dataset is adopted as the target domain, where exists a large domain gap." />
-        </attvalues>
-      </edge>
-      <edge source="2003.04883" target="1702.06451" id="2325">
-        <attvalues>
-          <attvalue for="5" value=" The works described in \cite{SOCHOR201787} and \cite{Giannakeris_2018_CVPR_Workshops} are examples in the first category, where vanishing points are detected on the considered 2D scene and the camera parameters are computed to enable the measurement of distances on the road plane, hence the speed of the vehicle of interest." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1801.07424" id="2326">
-        <attvalues>
-          <attvalue for="5" value=" Our method ranks first in the largest and most diverse video saliency dataset, such as DHF1K~\cite{7wang2018revisiting}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1506.04214" id="2327">
-        <attvalues>
-          <attvalue for="5" value=" For example, ACLNet~\cite{7wang2018revisiting} encodes static saliency features through attention mechanism, and then learns dynamic saliency through ConvLSTM~\cite{8shi2015convolutional}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1907.01869" id="2328">
-        <attvalues>
-          <attvalue for="5" value=" SalEMA~\cite{9linardos2019simple} uses exponential moving average instead of LSTM to extract temporal features for video saliency detection." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1908.05786" id="2329">
-        <attvalues>
-          <attvalue for="5" value=" We put away the previous deconvolution and unpooling operation of 3D fully convolutional encoder-decoder~\cite{11min2019tased} and completely adopt the 3D convolution and the trilinear upsampling." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1612.03144" id="2330">
-        <attvalues>
-          <attvalue for="5" value=" We consider the influence of time, space and scale and establish a temporal-spatial feature pyramid similar to FPN \cite{4422}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1412.7062" id="2331">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1606.00915" id="2332">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1706.05587" id="2333">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1802.02611" id="2334">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1611.06612" id="2335">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1511.00561" id="2336">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1605.06211" id="2337">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.04213" target="1505.04597" id="2338">
-        <attvalues>
-          <attvalue for="5" value=" Some studies on the semantic segmentation of 2D network show that the convolution with the upsampling in decoder~\cite{15chen2014semantic,16chen2017deeplab,17chen2017rethinking,19chen2018encoder,18lin2017refinenet} can obtain better results, compared with some methods, which adopted the deconvolution or unpooling~\cite{14badrinarayanan2017segnet,12long2015fully,13ronneberger2015u}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04044" target="1802.03931" id="2339">
-        <attvalues>
-          <attvalue for="5" value=" The first kind of method, similar to image compression, measures distortion by the Mean Squared Error (MSE) between the reconstructed feature and the original feature \cite{choi2018deep, suzuki2022deep, kim2023end, liu2023learnt}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04044" target="2306.16670" id="2340">
-        <attvalues>
-          <attvalue for="5" value=" For example, in \cite{kim2023end}, the feature MSE is used for multi-layer feature compression." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04044" target="2105.10089" id="2341">
-        <attvalues>
-          <attvalue for="5" value=" In the second kind of method, researchers incorporate semantic information into the distortion measurement \cite{choi2021latent, chen2019lossy, chen2020toward, ikusan2021rate, hu2020sensitivity}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.04044" target="2007.11797" id="2342">
-        <attvalues>
-          <attvalue for="5" value=" In the third kind of method, researchers adopt a more direct semantic information constraint to reconstructed features \cite{singh2020end, yang2021video, zhang2021MSFC, yan2021SSSIC, alvar2019multi}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2105.12555" id="2343">
-        <attvalues>
-          <attvalue for="5" value=" By dividing an image into meaningful regions based on specific semantic criteria, image segmentation enables a wide array of downstream tasks in both natural and medical domains, such as camouflaged object detection~\cite{IJCAI21_C2FNet,CVPR22_ZoomNet}, salient object detection~\cite{AAAI20_F3Net,CVPR23_FEDER}, marine animal segmentation~\cite{JOE23_MASNet,TCSVT21_MAS3K}, mirror detection~\cite{AAAI23_HetNet,CVPR22_SAMirror}, and polyp segmentation~\cite{MICCAI20_PraNet,PR23_CFANet}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2203.02688" id="2344">
-        <attvalues>
-          <attvalue for="5" value=" By dividing an image into meaningful regions based on specific semantic criteria, image segmentation enables a wide array of downstream tasks in both natural and medical domains, such as camouflaged object detection~\cite{IJCAI21_C2FNet,CVPR22_ZoomNet}, salient object detection~\cite{AAAI20_F3Net,CVPR23_FEDER}, marine animal segmentation~\cite{JOE23_MASNet,TCSVT21_MAS3K}, mirror detection~\cite{AAAI23_HetNet,CVPR22_SAMirror}, and polyp segmentation~\cite{MICCAI20_PraNet,PR23_CFANet}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2211.15644" id="2345">
-        <attvalues>
-          <attvalue for="5" value=" By dividing an image into meaningful regions based on specific semantic criteria, image segmentation enables a wide array of downstream tasks in both natural and medical domains, such as camouflaged object detection~\cite{IJCAI21_C2FNet,CVPR22_ZoomNet}, salient object detection~\cite{AAAI20_F3Net,CVPR23_FEDER}, marine animal segmentation~\cite{JOE23_MASNet,TCSVT21_MAS3K}, mirror detection~\cite{AAAI23_HetNet,CVPR22_SAMirror}, and polyp segmentation~\cite{MICCAI20_PraNet,PR23_CFANet}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2006.11392" id="2346">
-        <attvalues>
-          <attvalue for="5" value=" By dividing an image into meaningful regions based on specific semantic criteria, image segmentation enables a wide array of downstream tasks in both natural and medical domains, such as camouflaged object detection~\cite{IJCAI21_C2FNet,CVPR22_ZoomNet}, salient object detection~\cite{AAAI20_F3Net,CVPR23_FEDER}, marine animal segmentation~\cite{JOE23_MASNet,TCSVT21_MAS3K}, mirror detection~\cite{AAAI23_HetNet,CVPR22_SAMirror}, and polyp segmentation~\cite{MICCAI20_PraNet,PR23_CFANet}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2304.02643" id="2347">
-        <attvalues>
-          <attvalue for="5" value=" Among these VFMs, a notable example is the Segment Anything Model (SAM1)~\cite{ICCV23_SAM1} and its successor, Segment Anything 2 (SAM2)~\cite{SAM2}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2408.00714" id="2348">
-        <attvalues>
-          <attvalue for="5" value=" Among these VFMs, a notable example is the Segment Anything Model (SAM1)~\cite{ICCV23_SAM1} and its successor, Segment Anything 2 (SAM2)~\cite{SAM2}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2401.10229" id="2349">
-        <attvalues>
-          <attvalue for="5" value=" The emergence of vision foundation models (VFMs)~\cite{ICCV23_SAM1,SAM2,ICCV23_SegGPT,CVPR24_OMGSeg} has introduced significant potential in the field of image segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2304.13785" id="2350">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2406.00480" id="2351">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2406.20076" id="2352">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2406.15764" id="2353">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2305.03048" id="2354">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2305.13310" id="2355">
-        <attvalues>
-          <attvalue for="5" value=" To adapt SAM to downstream tasks, several approaches have been proposed, including the use of adapters~\cite{ICCVW23_SAMAdapter,arXiv23_SAMed} for parameter-efficient fine-tuning and the integration of additional conditional inputs such as text prompts~\cite{CVPR24_AlignSAM,arXiv24_EVFSAM,MICCAI24_TPDRSeg} or in-context samples~\cite{ICLR24_PerSAM,ICLR24_Matcher}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="1505.04597" id="2356">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the strong segmentation capabilities of U-Net~\cite{MICCAI15_UNet} and its variants~\cite{TMI19_UNetPP,MedIA24_TransUNet,ECCVW22_SwinUNet}, some researchers have explored the possibility of transforming SAM into a U-shaped architecture~\cite{MICCAI24_DeSAM,MLMI23_MammoSAM}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="1912.05074" id="2357">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the strong segmentation capabilities of U-Net~\cite{MICCAI15_UNet} and its variants~\cite{TMI19_UNetPP,MedIA24_TransUNet,ECCVW22_SwinUNet}, some researchers have explored the possibility of transforming SAM into a U-shaped architecture~\cite{MICCAI24_DeSAM,MLMI23_MammoSAM}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2105.05537" id="2358">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the strong segmentation capabilities of U-Net~\cite{MICCAI15_UNet} and its variants~\cite{TMI19_UNetPP,MedIA24_TransUNet,ECCVW22_SwinUNet}, some researchers have explored the possibility of transforming SAM into a U-shaped architecture~\cite{MICCAI24_DeSAM,MLMI23_MammoSAM}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.08870" target="2010.11929" id="2359">
-        <attvalues>
-          <attvalue for="5" value=" However, these efforts have often been limited by the plain structure of the vanilla ViT encoder~\cite{ICLR21_ViT}, which lacks the hierarchy needed for more sophisticated segmentation tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2201.10700" id="2360">
-        <attvalues>
-          <attvalue for="5" value=" Developing image deblurring models to transform blurred images into sharp ones can greatly improve downstream tasks such as object detection, autonomous driving, and visual navigation\cite{d2}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2205.01649" id="2361">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2106.03106" id="2362">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2202.09652" id="2363">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2111.11745" id="2364">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2111.09881" id="2365">
-        <attvalues>
-          <attvalue for="5" value=" Deep learning has excelled in image deblurring tasks \cite{d3,d4,db1,db2,db3}, with models like MIRNet-v2 \cite{d3} and Uformer \cite{d4} setting new benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="1612.02177" id="2366">
-        <attvalues>
-          <attvalue for="5" value=" These models achieve impressive PSNR scores on benchmarks like SIDD \cite{d5} and GoPro \cite{d6}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2204.04676" id="2367">
-        <attvalues>
-          <attvalue for="5" value=" introduced NAFNet \cite{d7}, a streamlined and nonlinear activation free network that maintains state-of-the-art (SOTA) performance while reducing computational overhead." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2304.02643" id="2368">
-        <attvalues>
-          <attvalue for="5" value=" The motivation for using SAM as a semantic prior is based on its robustness to blurred images, which is due to SAM's pretraining on the expansive SA-1B dataset \cite{d8}, encompassing a staggering 10 billion masks and 11 million images." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2306.15870" id="2369">
-        <attvalues>
-          <attvalue for="5" value=" Some studies have applied SAM priors to image restoration tasks like dehazing \cite{hazing} by using grayscale coding and channel expansion, highlighting their utility in low-level visual tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02270" target="2204.08179" id="2370">
-        <attvalues>
-          <attvalue for="5" value=" Extensively experiment on RealBlurJ \cite{realblur}, REDS \cite{reds}, and ReLoBlur \cite{relo} show that SAM priors improve deblurring performance and mitigate &quot;mode collapse&quot;." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06281" target="1809.10821" id="2371">
-        <attvalues>
-          <attvalue for="5" value=" Salient object detection (SOD)~\cite{2019-SPL-EACNNet}\cite{2018-SPL-BGFANet}\cite{2016-SPL-DCMC} aims to identify object-level regions with strong visual impact that attract the attention of the human visual system most in an image~\cite{what-is-SOD1}\cite{2009-CVPR-Fm}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06281" target="1710.05174" id="2372">
-        <attvalues>
-          <attvalue for="5" value=" With the development of deep learning, deep features have replaced hand-crafted features~\cite{2014-ICIMCS-DES}\cite{2016-SPL-DCMC}\cite{2017-ICCVW-CDCP} as a powerful tool for SOD, and promote it to better serve application-oriented tasks~\cite{2010-ACMTOG-RepFinder}\cite{2009-ACMTOG-Sketch2Photo}\cite{2009-CVPR-tracking-SBDT}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.06281" target="2007.08074" id="2373">
-        <attvalues>
-          <attvalue for="5" value=" \cite{2020-ECCV-GateNet} adopts the gated mechanism to suppress different level features in different degrees." />
-        </attvalues>
-      </edge>
-      <edge source="2303.05109" target="1604.04574" id="2374">
-        <attvalues>
-          <attvalue for="5" value=" Reconstruction-based methods \cite{hasan2016learning, luo2017remembering, gong2019memorizing, park2020learning} usually train AutoEncoders (AEs) on normal data and expect abnormal data to incur larger reconstruction errors at test time, making abnormal data detectable from normal ones." />
-        </attvalues>
-      </edge>
-      <edge source="2303.05109" target="1904.02639" id="2375">
-        <attvalues>
-          <attvalue for="5" value=" However, such approaches are highly dependent on the local context of frame sequence, and some studies \cite{gong2019memorizing, liu2021hybrid} have shown that AEs trained only on normal data sometimes reconstruct anomalies well, which leads to poor performance." />
-        </attvalues>
-      </edge>
-      <edge source="2303.05109" target="2108.06852" id="2376">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, inspired by SOTA methods that use multiple modalities \cite{liu2021hybrid, yan2018abnormal, nguyen2019anomaly, cai2021appearance}, we first extract the representative features of appearance and motion in normal events by a universal two-stream encoder." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10192" target="1601.07661" id="2377">
-        <attvalues>
-          <attvalue for="5" value=" Besides, DCP is also broadly used as a standard baseline of the performance evaluation in recent works using deep learning \cite{cai2016dehazenet, li2017aod, ren2018gated, qu2019enhanced, zhang2018densely, dong2020fd}." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10192" target="1804.00213" id="2378">
-        <attvalues>
-          <attvalue for="5" value=" \cite{ren2018gated} designed a coarse-to-fine fusion network, which recovers the clear image directly from the hazy one." />
-        </attvalues>
-      </edge>
-      <edge source="2105.10192" target="1801.00631" id="2379">
-        <attvalues>
-          <attvalue for="5" value=" Not only a large labeled dataset is required, but also the training process is time-consuming, let alone the problems of interpretability and robustness \cite{marcus2018deep}, which is required to be employed for productization." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03000" target="1909.12828" id="2380">
-        <attvalues>
-          <attvalue for="5" value=" Early works on 3D human pose estimation have using convolutional neural networks (CNN)~\cite{kolotouros2019learning, cheng2019occlusion, Sun2018integral, Martinez_2017_ICCV,zhou2017towards, NibaliHMP19} have focused on using a single image as input." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03000" target="1704.02447" id="2381">
-        <attvalues>
-          <attvalue for="5" value=" Early works on 3D human pose estimation have using convolutional neural networks (CNN)~\cite{kolotouros2019learning, cheng2019occlusion, Sun2018integral, Martinez_2017_ICCV,zhou2017towards, NibaliHMP19} have focused on using a single image as input." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03000" target="1905.05754" id="2382">
-        <attvalues>
-          <attvalue for="5" value=" To cope with noise in the 2D pose estimate two strategies exist: (1) defining a robust 3D triangulation algorithm~\cite{iskakov2019learnable, Joo2017panoptic, zhou2019hemlets} or (2) refining the 2D pose estimates prior to doing the 3D triangulation~\cite{multiviewpose, pavlakos2017harvesting, tome2018rethinking}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03000" target="1909.01203" id="2383">
-        <attvalues>
-          <attvalue for="5" value=" To cope with noise in the 2D pose estimate two strategies exist: (1) defining a robust 3D triangulation algorithm~\cite{iskakov2019learnable, Joo2017panoptic, zhou2019hemlets} or (2) refining the 2D pose estimates prior to doing the 3D triangulation~\cite{multiviewpose, pavlakos2017harvesting, tome2018rethinking}." />
-        </attvalues>
-      </edge>
-      <edge source="1804.06039" target="1502.02766" id="2384">
-        <attvalues>
-          <attvalue for="5" value=" Same as many other vision tasks, the performance of face detection has been substantially improved by Convolutional Neural Network (CNN) \cite{farfade2015multi, ren2015faster, Liu2015SSD, li2015convolutional, yang2015facial, zhang2016joint, jiang2017face, Hu_2017_CVPR, Najibi_2017_ICCV}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="1701.03572" id="2385">
-        <attvalues>
-          <attvalue for="5" value=" Apart from casual hand-held photography, the need for video stabilization also arises in endoscopy \cite{Karargyris2014}, underwater imaging \cite{Oreifej2011} and aerial photography from drones/helicopters \cite{Lim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="1805.09817" id="2386">
-        <attvalues>
-          <attvalue for="5" value=" The work uses frame-to-frame optical flow as initial input and requires a number of pre-processing steps to: (1) identify regions with moving objects from the optical flow fields using a variety of segmentation masks for typical foreground objects obtained from \cite{Zhou2018_ACM}, (2) identify regions of inaccurate optical flow, and (3) inpaint all such regions using the PCA-based approach from \cite{Wulff2015}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="1909.02641" id="2387">
-        <attvalues>
-          <attvalue for="5" value=" Similar in spirit to \cite{Choi2020}, work in \cite{Liu_2021_ICCV, Xu_2021_ICCV} perform full-frame video stabilization by bringing in border-based frame inpainting." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="1505.04597" id="2388">
-        <attvalues>
-          <attvalue for="5" value=" The resulting warped frames are passed through a U-Net \cite{Ronneberger2015} to generate the $i^{\textrm{th}}$ intermediate `stabilized' frame." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="1512.03385" id="2389">
-        <attvalues>
-          <attvalue for="5" value=" To prevent this, the intermediate stabilized frames are also passed through a Resnet \cite{He2016}." />
-        </attvalues>
-      </edge>
-      <edge source="2210.13769" target="2102.06205" id="2390">
-        <attvalues>
-          <attvalue for="5" value=" However, the approach in \cite{Liu_2021_ICCV} is computationally expensive." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1904.05160" id="2391">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1910.09217" id="2392">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1912.02413" id="2393">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1409.1556" id="2394">
-        <attvalues>
-          <attvalue for="5" value=", deep learning) can not be directly applied due to their notorious demand of a large number of annotated data~\cite{simonyan2014very,he2016deep,Krizhevsky2017ImageNet}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1512.03385" id="2395">
-        <attvalues>
-          <attvalue for="5" value=", deep learning) can not be directly applied due to their notorious demand of a large number of annotated data~\cite{simonyan2014very,he2016deep,Krizhevsky2017ImageNet}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1606.04080" id="2396">
-        <attvalues>
-          <attvalue for="5" value=" Next, we empirically validate our approach on two standard benchmark datasets --- {Mini}ImageNet~\cite{VinyalsBLKW16Matching} and {Tiered}ImageNet~\cite{Ren2018Meta}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1904.04232" id="2397">
-        <attvalues>
-          <attvalue for="5" value=" Few-shot learning~(FSL)~\cite{VinyalsBLKW16Matching,chen2019closer} is mindful of the limited data per tail concept (\ie." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1703.05175" id="2398">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1807.05960" id="2399">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1812.03664" id="2400">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1703.03400" id="2401">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1803.02999" id="2402">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1810.09502" id="2403">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1801.05401" id="2404">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1904.03758" id="2405">
-        <attvalues>
-          <attvalue for="5" value=" Instance embedding~\cite{VinyalsBLKW16Matching,SnellSZ17Prototypical,Rusu2018Meta,YeHZS2018Learning}, model initialization~\cite{FinnAL17Model,Nichol2018On,Antoniou2018How}, image generator~\cite{Wang2018Low}, and optimization flow~\cite{Sachin2017,Lee2019Meta} act as popular meta-knowledge and usually incorporates with FSL." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1606.02819" id="2406">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1901.05555" id="2407">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1906.07413" id="2408">
-        <attvalues>
-          <attvalue for="5" value=" One naive GFSL solution is to train a single classifier over the imbalanced long-tail distribution~\cite{Hariharan2017Low,WangRH17Learning,Liu2019Large,Zhou2020BBN}, and re-balance it~\cite{Cui2019Class,Cao2019Margin,Kang2020Decoupling}." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1706.08840" id="2409">
-        <attvalues>
-          <attvalue for="5" value=" As a result, it allows backward knowledge transfer~\cite{lopez2017gradient} --- new knowledge learned from novel few-shot training examples can benefit the existing many-shot classifiers." />
-        </attvalues>
-      </edge>
-      <edge source="1906.02944" target="1803.00676" id="2410">
-        <attvalues>
-          <attvalue for="5" value=" Next, we empirically validate our approach on two standard benchmark datasets --- {Mini}ImageNet~\cite{VinyalsBLKW16Matching} and {Tiered}ImageNet~\cite{Ren2018Meta}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1909.03459" id="2411">
-        <attvalues>
-          <attvalue for="5" value=" GeoNet\cite{li2019blind} employs CNN to predict distortion flow in images without prior distortion-type knowledge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2011.13607" id="2412">
-        <attvalues>
-          <attvalue for="5" value=" PCL\cite{yu2021pcls} uses perspective crop layers in CNNs to correct perspective distortions for 3D pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2104.12464" id="2413">
-        <attvalues>
-          <attvalue for="5" value=" A cascaded deep structure network\cite{tan2021practical} corrects wide-angle portrait distortions without calibrated camera parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1905.07515" id="2414">
-        <attvalues>
-          <attvalue for="5" value=" Another study\cite{zhao2019learning} predicts per-pixel displacement for face portrait undistortion." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2212.03239" id="2415">
-        <attvalues>
-          <attvalue for="5" value=" PerspectiveNet\cite{jin2023perspective} and ParamNet\cite{jin2023perspective} predict perspective fields and derive camera calibration parameters, respectively." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1804.04784" id="2416">
-        <attvalues>
-          <attvalue for="5" value=" Methods for fisheye image distortion correction are also developed\cite{yang2023innovating,yin2018fisheyerecnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2303.13796" id="2417">
-        <attvalues>
-          <attvalue for="5" value=" Zolly\cite{wang2023zolly} corrects perspective distortion by adjusting focal lengths in human mesh reconstruction tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2007.10323" id="2418">
-        <attvalues>
-          <attvalue for="5" value=" One possible approach is to focus on developing task-specific methods where PD is noticeable, such as crowd counting\cite{yang2020weakly, idrees2018composition}, object re-identification\cite{ayala2019vehicle}, autonomous vehicle\cite{wang2020pillar}, and recognizing texts in scene\cite{phan2013recognizing}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2111.15056" id="2419">
-        <attvalues>
-          <attvalue for="5" value=" Further, \cite{cho2021camera} meta-learning based approach to adapt 3D human pose estimation to varying degrees of camera distortion." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2110.00620" id="2420">
-        <attvalues>
-          <attvalue for="5" value=" SPEC\cite{kocabas2021spec} tackled PD by estimating the camera parameters directly from in-the-wild images to improve the accuracy of 3D human pose predictions." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2211.01866" id="2421">
-        <attvalues>
-          <attvalue for="5" value=" MPD demonstrates improved performance on existing benchmarks, ImagNet-E \cite{li2023imagenet}, and ImageNet-X \cite{idrissi2022imagenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2303.17096" id="2422">
-        <attvalues>
-          <attvalue for="5" value=" MPD demonstrates improved performance on existing benchmarks, ImagNet-E \cite{li2023imagenet}, and ImageNet-X \cite{idrissi2022imagenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2002.02917" id="2423">
-        <attvalues>
-          <attvalue for="5" value=" Due to its conformal nature (preserving angles during transformation), \cite{zhou2021data} utilized the Möbius transform for eight specific transformations, including various twists and spreads, as a data augmentation strategy to enhance model performance on standard data distributions." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2201.12212" id="2424">
-        <attvalues>
-          <attvalue for="5" value=" In other works, Möbius convolution\cite{mitchel2022mobius}, the authors design a Möbius-equivalent convolution operator in spherical CNN for shape classification and image segmentation tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2308.08114" id="2425">
-        <attvalues>
-          <attvalue for="5" value=" OmniZoomer\cite{cao2023omnizoomer} integrates Möbius transformations into a neural network, enabling movement and variable zoom on omnidirectional images while addressing the challenges of blur and aliasing associated with such transformations." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2203.10554" id="2426">
-        <attvalues>
-          <attvalue for="5" value=" MöbiusGCN\cite{azizi20223d} addresses the limitation of existing graph convolution network {GCNs} in explicitly encoding transformations between joints in 3D human pose estimation." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2002.05709" id="2427">
-        <attvalues>
-          <attvalue for="5" value=" We employ ImageNet-PD in our proposed MPD method and extensively investigate it across supervised and self-supervised learning methods (SimCLR \cite{chen2020simple}, DINO \cite{caron2021emerging})." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2104.14294" id="2428">
-        <attvalues>
-          <attvalue for="5" value=" We employ ImageNet-PD in our proposed MPD method and extensively investigate it across supervised and self-supervised learning methods (SimCLR \cite{chen2020simple}, DINO \cite{caron2021emerging})." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1906.11942" id="2429">
-        <attvalues>
-          <attvalue for="5" value=" It demonstrates effective knowledge transfer in fish-eye image recognition, improving performance by $3\%$ on the VOC-360\cite{fu2019datasets} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2303.11855" id="2430">
-        <attvalues>
-          <attvalue for="5" value=" MPD combined with a transformer-based self-supervised method, Clip-ReIdent\cite{habel2022clip}, excels in person re-identification, achieving mean average precision of 97." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="2208.08190" id="2431">
-        <attvalues>
-          <attvalue for="5" value="30 with and without re-ranking, respectively, on the DeepSportradar dataset\cite{van2022deepsportradar}." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1506.01497" id="2432">
-        <attvalues>
-          <attvalue for="5" value=" Also, MPD adapted for object detection with FasterRCNN \cite{ren2015faster} and FCOS \cite{9010746} methods gain up to 3\% performance improvement on COCO \cite{lin2014microsoft} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1904.01355" id="2433">
-        <attvalues>
-          <attvalue for="5" value=" Also, MPD adapted for object detection with FasterRCNN \cite{ren2015faster} and FCOS \cite{9010746} methods gain up to 3\% performance improvement on COCO \cite{lin2014microsoft} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2405.02296" target="1405.0312" id="2434">
-        <attvalues>
-          <attvalue for="5" value=" Also, MPD adapted for object detection with FasterRCNN \cite{ren2015faster} and FCOS \cite{9010746} methods gain up to 3\% performance improvement on COCO \cite{lin2014microsoft} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2303.09801" target="1612.03144" id="2435">
-        <attvalues>
-          <attvalue for="5" value=" For instance, FPN~\cite{FPN} used a pyramid structure with bottom-up and top-down pathways, and lateral connections to extract multi-scale features." />
-        </attvalues>
-      </edge>
-      <edge source="2303.09801" target="2110.06550" id="2436">
-        <attvalues>
-          <attvalue for="5" value="~\cite{park2022saliency} introduces a context fusion decoder network consisting of a context module to extract global information of an image and a feature fusion module to refine the features from encoder and decoder." />
-        </attvalues>
-      </edge>
-      <edge source="2303.09801" target="1609.02907" id="2437">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by a previously published graph convolution network~\cite{gnn}, we designed AGCM to preserve permutation invariance even if the order of the nodes is altered." />
-        </attvalues>
-      </edge>
-      <edge source="2303.09801" target="1801.07829" id="2438">
-        <attvalues>
-          <attvalue for="5" value=" In particular, when forming an adaptive graph, the constructed prototype feature vectors are refined using a graph architecture (more specifically, EdgeConv~\cite{EdgeConv}) to generate refined feature vectors, where each prototype feature vector is considered as a node." />
-        </attvalues>
-      </edge>
-      <edge source="2303.12217" target="1906.11241" id="2439">
-        <attvalues>
-          <attvalue for="5" value=" For instance, how should we define an IGM for black hole imaging without having ever seen a direct image of a black hole or knowing what one should look like? Moreover, classical approaches that utilize hand-crafted IGMs \cite{TV-ROF} are prone to human bias \cite{akiyama2019first4}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1511.03296" id="2440">
-        <attvalues>
-          <attvalue for="5" value=" It has been a useful tool for many image processing and computer vision tasks, such as depth map super-resolution~\cite{JBU, DJFR}, scale-space filtering~\cite{sdf, RGF}, cross-modality image restoration~\cite{GF, mugf, guide-filter-SR2019}, structure-texture separation~\cite{texture2012, texture2013, RTV}, image semantic segmentation~\cite{FBS, DKN} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1605.09673" id="2441">
-        <attvalues>
-          <attvalue for="5" value=" Although with increased flexibility thanks to their adaptive nature, ~\cite{DFN} and \cite{PanNet} still suffer from the same drawback as~\cite{GF, bilateral} that only the guidance information is considered in filters design." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1904.05373" id="2442">
-        <attvalues>
-          <attvalue for="5" value=" Although with increased flexibility thanks to their adaptive nature, ~\cite{DFN} and \cite{PanNet} still suffer from the same drawback as~\cite{GF, bilateral} that only the guidance information is considered in filters design." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1710.04200" id="2443">
-        <attvalues>
-          <attvalue for="5" value=" It implies that instead of performing regression on guidance only, as done in ~\cite{GF, bilateral}, we should perform dual regression on both the guidance and the target, and combine them adaptively in a smarter manner instead of simple concatenation or element-wise multiplication, as done in ~\cite{DJFR, DKN}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1910.08373" id="2444">
-        <attvalues>
-          <attvalue for="5" value="~\cite{DKN} proposes to use spatially variant weighted averages, where the set of neighbors and the corresponding kernel weights are learned in an end-to-end manner." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1910.04066" id="2445">
-        <attvalues>
-          <attvalue for="5" value=" However, for some scenarios, such as cross-modality image restoration~\cite{CUNet} and guided super-resolution~\cite{GSPRT}, multi-modal data has significantly different characteristics due to the difference of sensing principle, making the guidance not always trustworthy." />
-        </attvalues>
-      </edge>
-      <edge source="2112.06401" target="1904.01501" id="2446">
-        <attvalues>
-          <attvalue for="5" value=" However, for some scenarios, such as cross-modality image restoration~\cite{CUNet} and guided super-resolution~\cite{GSPRT}, multi-modal data has significantly different characteristics due to the difference of sensing principle, making the guidance not always trustworthy." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1701.08493" id="2447">
-        <attvalues>
-          <attvalue for="5" value=" Digital RGB cameras have been used to estimate 3D geometry using Structure-from-Motion (SfM) \cite{sfm_2017}, single-frame monocular depth estimation \cite{monocular_Godard_2019_ICCV}, and 3D pose estimation of known objects \cite{tremblay2018corl:dope}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1806.01260" id="2448">
-        <attvalues>
-          <attvalue for="5" value=" Digital RGB cameras have been used to estimate 3D geometry using Structure-from-Motion (SfM) \cite{sfm_2017}, single-frame monocular depth estimation \cite{monocular_Godard_2019_ICCV}, and 3D pose estimation of known objects \cite{tremblay2018corl:dope}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1809.10790" id="2449">
-        <attvalues>
-          <attvalue for="5" value=" Digital RGB cameras have been used to estimate 3D geometry using Structure-from-Motion (SfM) \cite{sfm_2017}, single-frame monocular depth estimation \cite{monocular_Godard_2019_ICCV}, and 3D pose estimation of known objects \cite{tremblay2018corl:dope}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1604.00999" id="2450">
-        <attvalues>
-          <attvalue for="5" value=" In contrast, depth cameras are predominant when imaging indoor scenes \cite{Silberman:NYUv2,dai2017scannet}, household objects \cite{ycb}, and other applications \cite{rgbd_datasets}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1808.00769" id="2451">
-        <attvalues>
-          <attvalue for="5" value=" The shortcomings of depth sensors have been previously addressed with depth completion \cite{depth_comp_unit,Ma19icra,NLSPN,plugNplay19,sajjan2019cleargrasp} on single frames." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1807.00275" id="2452">
-        <attvalues>
-          <attvalue for="5" value=" The shortcomings of depth sensors have been previously addressed with depth completion \cite{depth_comp_unit,Ma19icra,NLSPN,plugNplay19,sajjan2019cleargrasp} on single frames." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="2007.10042" id="2453">
-        <attvalues>
-          <attvalue for="5" value=" The shortcomings of depth sensors have been previously addressed with depth completion \cite{depth_comp_unit,Ma19icra,NLSPN,plugNplay19,sajjan2019cleargrasp} on single frames." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1910.02550" id="2454">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, specular reflections and transparent recyclable objects \cite{sajjan2019cleargrasp} may induce large regions of erroneous values that need to be identified and corrected." />
-        </attvalues>
-      </edge>
-      <edge source="2108.04706" target="1702.04405" id="2455">
-        <attvalues>
-          <attvalue for="5" value=" Depth cameras, on the other hand, may yield large connected regions with missing or invalid values, for example at window panes \cite{Silberman:NYUv2,dai2017scannet}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.13820" target="1501.00092" id="2456">
-        <attvalues>
-          <attvalue for="5" value=" Recently, however, CNNs have demonstrated their utility for such inverse problems\cite{Xie_2012,Dong_16}." />
-        </attvalues>
-      </edge>
-      <edge source="2003.13820" target="1607.08194" id="2457">
-        <attvalues>
-          <attvalue for="5" value=" As shown in the work of Papyan \etal~\cite{Papyan_2017}, this implicitly encodes the assumption that $\y$ is convolutionally compressible." />
-        </attvalues>
-      </edge>
-      <edge source="1911.07932" target="1405.0312" id="2458">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, as the publicly available datasets are small, we generate a new dataset of forged images using deep semantic inpainting algorithm from COCO \cite{coco} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1611.07004" id="2459">
-        <attvalues>
-          <attvalue for="5" value=" To this end, we first design our whole network following the U-Net architecture \cite{isola2017image}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1809.00219" id="2460">
-        <attvalues>
-          <attvalue for="5" value=" Early studies learn mappings directly by Generating Adversarial Networks (GANs), and have shown great success in various applications \cite{wang2018esrgan, chang2018pairedcyclegan}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1805.11145" id="2461">
-        <attvalues>
-          <attvalue for="5" value=" Recently, exemplar based image translation \cite{ma2018exemplar,zhang2020cocosnet,luo2022memory}, where an exemplar image is used to control the style of translated images, has attracted a lot of attention." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2004.05571" id="2462">
-        <attvalues>
-          <attvalue for="5" value=" Previous works \cite{zhang2020cocosnet, zhou2021cocosnetv2, zhan2022marginal} have used the vanilla attention mechanism \cite{vaswani2017attention} for learning full correspondence." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1611.07308" id="2463">
-        <attvalues>
-          <attvalue for="5" value=" Early pioneering works \cite{kipf2016variational, park2019SPADE,huang2018multimodal} attempt to globally control the style of generated images." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1804.04732" id="2464">
-        <attvalues>
-          <attvalue for="5" value=" Early pioneering works \cite{kipf2016variational, park2019SPADE,huang2018multimodal} attempt to globally control the style of generated images." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2209.11047" id="2465">
-        <attvalues>
-          <attvalue for="5" value=" In these methods, the quality of generated images relies heavily on the learned correspondence \cite{seo2022midms}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1706.03762" id="2466">
-        <attvalues>
-          <attvalue for="5" value=" Besides, the Feed-Forward Network (FFN) \cite{vaswani2017attention} in vanilla transformers neglects contextual correlations inside an image." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2010.11929" id="2467">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the great success of Transformers \cite{vaswani2017attention,dosovitskiy2020image,liu2021swin,heo2021rethinking}, we first devise a Masked and Adaptive Transformer (MAT) for learning accurate cross-domain correspondence and executing context-aware feature augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2103.14030" id="2468">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the great success of Transformers \cite{vaswani2017attention,dosovitskiy2020image,liu2021swin,heo2021rethinking}, we first devise a Masked and Adaptive Transformer (MAT) for learning accurate cross-domain correspondence and executing context-aware feature augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2103.16302" id="2469">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the great success of Transformers \cite{vaswani2017attention,dosovitskiy2020image,liu2021swin,heo2021rethinking}, we first devise a Masked and Adaptive Transformer (MAT) for learning accurate cross-domain correspondence and executing context-aware feature augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2201.03545" id="2470">
-        <attvalues>
-          <attvalue for="5" value=" We thus replace FFN by an adaptive convolution block \cite{liu2022convnext}, where the coordinate attention \cite{hou2021coordinate} and depthwise separable convolution \cite{chollet2017xception} are used to capture contextual correlations and to improve efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2103.02907" id="2471">
-        <attvalues>
-          <attvalue for="5" value=" We thus replace FFN by an adaptive convolution block \cite{liu2022convnext}, where the coordinate attention \cite{hou2021coordinate} and depthwise separable convolution \cite{chollet2017xception} are used to capture contextual correlations and to improve efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="1610.02357" id="2472">
-        <attvalues>
-          <attvalue for="5" value=" We thus replace FFN by an adaptive convolution block \cite{liu2022convnext}, where the coordinate attention \cite{hou2021coordinate} and depthwise separable convolution \cite{chollet2017xception} are used to capture contextual correlations and to improve efficiency." />
-        </attvalues>
-      </edge>
-      <edge source="2303.17123" target="2205.09542" id="2473">
-        <attvalues>
-          <attvalue for="5" value=" \cite{zhang2020cast} propose a similar CSL method, where the target exemplar is used as a positive sample, and the other exemplars as negative ones." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1604.07379" id="2474">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1501.00092" id="2475">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1511.04587" id="2476">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1603.08511" id="2477">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1705.02999" id="2478">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1703.06868" id="2479">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1812.02342" id="2480">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision and graphics there are many image-to-image translation tasks, including inpainting \cite{pathak2016context,iizuka2017globally}, super resolution \cite{dong2016image,kim2016accurate}, colorization \cite{zhang2016colorful,zhang2017real}, style transfer \cite{gatys2016image,huang2017arbitrary,park2018arbitrary} and so on." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1611.07004" id="2481">
-        <attvalues>
-          <attvalue for="5" value=" In many cases, given a paired dataset, it is possible to solve the problem with conditional image translation \cite{isola2017image,wang2017high,li2017alice}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1711.11585" id="2482">
-        <attvalues>
-          <attvalue for="5" value=" In many cases, given a paired dataset, it is possible to solve the problem with conditional image translation \cite{isola2017image,wang2017high,li2017alice}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1709.01215" id="2483">
-        <attvalues>
-          <attvalue for="5" value=" In many cases, given a paired dataset, it is possible to solve the problem with conditional image translation \cite{isola2017image,wang2017high,li2017alice}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1704.02510" id="2484">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1703.10593" id="2485">
-        <attvalues>
-          <attvalue for="5" value=" CycleGAN \cite{zhu2017unpaired}, the most representative unsupervised image translation method, failed to change the high-level semantic meaning due to the network structure specialized for style transfer." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1703.05192" id="2486">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1611.02200" id="2487">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1703.00848" id="2488">
-        <attvalues>
-          <attvalue for="5" value=" To solve the unsupervised image-to-image translation problem, UNIT \cite{liu2017unsupervised} made a shared-latent space assumption." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1711.05139" id="2489">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1711.09020" id="2490">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1804.04732" id="2491">
-        <attvalues>
-          <attvalue for="5" value=" MUNIT \cite{huang2018multimodal} proposed a multimodal unsupervised image-to-image translation framework." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1712.06909" id="2492">
-        <attvalues>
-          <attvalue for="5" value=" Recent works have yielded impressive results in the GANs-based unsupervised image-to-image translation \cite{yi2017dualgan,zhu2017unpaired,kim2017learning,taigman2016unsupervised,liu2017unsupervised,royer2017xgan,choi2017stargan,huang2018multimodal,anoosheh2017combogan}." />
-        </attvalues>
-      </edge>
-      <edge source="1901.08242" target="1805.08318" id="2493">
-        <attvalues>
-          <attvalue for="5" value=" Recently, SAGAN \cite{zhang2018self} showed that the self-attention module is complementary to convolutions and helps with modeling long range, multi-level dependencies across image regions." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2307.14889" id="2494">
-        <attvalues>
-          <attvalue for="5" value=" Applications of human pose estimation span a wide range, including predicting pedestrian behavior and trajectory on roads with autonomous vehicles \cite{bauer_2023_weakly, kress_2019_human, wang_leverage_2019, zanfir_2023_hum3dil, Zheng_2022_multimodal}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2303.11638" id="2495">
-        <attvalues>
-          <attvalue for="5" value=" How can we address this \cite{geng_human_2023, yang_camerapose_2023, zhang_neuromorphic_2023}? (4) Can we use transformers \cite{einfalt_uplift_2023, qian_hstformer_2023, xu_vitpose_2022, zhao_dpit_2022}? (5) Vision-based pose estimation is unreliable." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2210.06110" id="2496">
-        <attvalues>
-          <attvalue for="5" value=" How can we address this \cite{geng_human_2023, yang_camerapose_2023, zhang_neuromorphic_2023}? (4) Can we use transformers \cite{einfalt_uplift_2023, qian_hstformer_2023, xu_vitpose_2022, zhao_dpit_2022}? (5) Vision-based pose estimation is unreliable." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2301.00250" id="2497">
-        <attvalues>
-          <attvalue for="5" value=" Can we use other more robust modes of data \cite{geng_densepose_2022, lee_hupr_2023}? An important caveat is that many of these methods require labor-intensive pose labeling, which limits scalability in training data." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="1405.0312" id="2498">
-        <attvalues>
-          <attvalue for="5" value="7 million labeled keypoints \cite{lin2014microsoft}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2111.15124" id="2499">
-        <attvalues>
-          <attvalue for="5" value=" To make use of the vast amount of unlabeled data, we look to self-supervised models, which frame classification and regression as reconstruction problems \cite{cao_-bed_2022, li_geometry-driven_2020, schmidtke_unsupervised_2021, wan_self-supervised_2019}, where given some parts of the input space (source), the model reconstructs other parts of the input space (target)." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2109.09150" id="2500">
-        <attvalues>
-          <attvalue for="5" value=" Hence, rather than optimizing directly for the task, self-supervision learns indirectly by optimizing for reconstruction, yielding representations that surpass the generalizability of those learned via supervised learning \cite{tendle_study_2021}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2006.08373" id="2501">
-        <attvalues>
-          <attvalue for="5" value=" Self-supervised learning holds much potential for autonomous driving, facilitating greater scalability in training data, improving robustness, and enabling lifelong-learning \cite{berscheid2020self, deng2020self}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2108.07777" id="2502">
-        <attvalues>
-          <attvalue for="5" value=" SOTA methods in this space typically rely on multi-view geometry \cite{bouazizi2021self, gholami2022self, Kocabas_2019_self, Wandt_2021_CVPR}, unpaired pose data \cite{Kundu_2020_self, sosa2023self}, or synthetic datasets with pose labels \cite{Kundu_2022_uncertainty}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2004.04400" id="2503">
-        <attvalues>
-          <attvalue for="5" value=" SOTA methods in this space typically rely on multi-view geometry \cite{bouazizi2021self, gholami2022self, Kocabas_2019_self, Wandt_2021_CVPR}, unpaired pose data \cite{Kundu_2020_self, sosa2023self}, or synthetic datasets with pose labels \cite{Kundu_2022_uncertainty}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2203.15293" id="2504">
-        <attvalues>
-          <attvalue for="5" value=" SOTA methods in this space typically rely on multi-view geometry \cite{bouazizi2021self, gholami2022self, Kocabas_2019_self, Wandt_2021_CVPR}, unpaired pose data \cite{Kundu_2020_self, sosa2023self}, or synthetic datasets with pose labels \cite{Kundu_2022_uncertainty}." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2105.04154" id="2505">
-        <attvalues>
-          <attvalue for="5" value=" Using these insights, we develop an efficient model pipeline that exceeds the baseline \cite{schmidtke_unsupervised_2021} performance, using a training set that is less than one-third the size of the original, along with a new formulation of matrix transformation that implements coarse-to-fine learning and data augmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2311.02815" target="2007.10603" id="2506">
-        <attvalues>
-          <attvalue for="5" value=" However, without labels, there is no guarantee that predictions will reflect ground truth \cite{shu_feature-metric_2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="1804.06919" id="2507">
-        <attvalues>
-          <attvalue for="5" value=" Accurate optical flow contributes significantly to various downstream tasks such as video compression~\cite{wu2018video, lu2019dvc}, video denoising and blur removal~\cite{buades2016patch, yuan2020efficient, zhang2021deep}, action recognition~\cite{lee2018motion, cai2019temporal}, and VFI stands as one of these applications." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="1807.10037" id="2508">
-        <attvalues>
-          <attvalue for="5" value=" Accurate optical flow contributes significantly to various downstream tasks such as video compression~\cite{wu2018video, lu2019dvc}, video denoising and blur removal~\cite{buades2016patch, yuan2020efficient, zhang2021deep}, action recognition~\cite{lee2018motion, cai2019temporal}, and VFI stands as one of these applications." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="2205.14620" id="2509">
-        <attvalues>
-          <attvalue for="5" value=" While utilizing flow-based methods is a common practice in VFI~\cite{kong2022ifrnet, lu2022video, huang2022real, li2023amt}, leveraging inter-frames to train optical flow models is relatively less explored." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="1504.06852" id="2510">
-        <attvalues>
-          <attvalue for="5" value=" To address data scarcity in optical flow training: data augmentation \cite{sun2021autoflow, jeong2023distractflow}, data generation \cite{dosovitskiy2015flownet, sun2021autoflow, han2022realflow}, and semi-supervised learning \cite{jeong2022imposing, im2022semi, han2022realflow,jeong2023distractflow} have been explored." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="2104.14544" id="2511">
-        <attvalues>
-          <attvalue for="5" value=" To address data scarcity in optical flow training: data augmentation \cite{sun2021autoflow, jeong2023distractflow}, data generation \cite{dosovitskiy2015flownet, sun2021autoflow, han2022realflow}, and semi-supervised learning \cite{jeong2022imposing, im2022semi, han2022realflow,jeong2023distractflow} have been explored." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="2204.07262" id="2512">
-        <attvalues>
-          <attvalue for="5" value=" To address data scarcity in optical flow training: data augmentation \cite{sun2021autoflow, jeong2023distractflow}, data generation \cite{dosovitskiy2015flownet, sun2021autoflow, han2022realflow}, and semi-supervised learning \cite{jeong2022imposing, im2022semi, han2022realflow,jeong2023distractflow} have been explored." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="2303.14078" id="2513">
-        <attvalues>
-          <attvalue for="5" value=" Most of the current data augmentation methods prioritize modifying the existing frames, eg,~\cite{jeong2023distractflow}, as illustrated in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2403.18092" target="2207.11075" id="2514">
-        <attvalues>
-          <attvalue for="5" value=" Notably, RealFlow\cite{han2022realflow} stands out by synthesizing a new second frame via forward warping." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2302.01872" id="2515">
-        <attvalues>
-          <attvalue for="5" value=" However, the challenging nature of MOSE\cite{r9} still poses several obstacles to overcome." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2308.08544" id="2516">
-        <attvalues>
-          <attvalue for="5" value=" The PVUW 2024 workshop challenge includes two new tracks, Complex Video Object Segmentation Track based on MOSE\cite{r9} and Motion Expression guided Video Segmentation track based on MeViS\cite{ding2023mevis}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1512.06735" id="2517">
-        <attvalues>
-          <attvalue for="5" value=" As an important branch of the VOS task, semi-supervised VOS aims at tracking and segmenting agnostic objects given only the first-frame annotations, which has been widely applied in autonomous driving\cite{r1}, video editing\cite{r2}, automatic data annotation\cite{r3}, and universal video segmentation\cite{r4}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2103.07941" id="2518">
-        <attvalues>
-          <attvalue for="5" value=" As an important branch of the VOS task, semi-supervised VOS aims at tracking and segmenting agnostic objects given only the first-frame annotations, which has been widely applied in autonomous driving\cite{r1}, video editing\cite{r2}, automatic data annotation\cite{r3}, and universal video segmentation\cite{r4}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2209.12118" id="2519">
-        <attvalues>
-          <attvalue for="5" value=" As an important branch of the VOS task, semi-supervised VOS aims at tracking and segmenting agnostic objects given only the first-frame annotations, which has been widely applied in autonomous driving\cite{r1}, video editing\cite{r2}, automatic data annotation\cite{r3}, and universal video segmentation\cite{r4}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2309.03903" id="2520">
-        <attvalues>
-          <attvalue for="5" value=" As an important branch of the VOS task, semi-supervised VOS aims at tracking and segmenting agnostic objects given only the first-frame annotations, which has been widely applied in autonomous driving\cite{r1}, video editing\cite{r2}, automatic data annotation\cite{r3}, and universal video segmentation\cite{r4}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1611.05198" id="2521">
-        <attvalues>
-          <attvalue for="5" value=" Due to the advantages over other VOS methods\cite{other1,other2,other3,other4}, the memory-based paradigm has been paid much attention by the research community." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1612.02646" id="2522">
-        <attvalues>
-          <attvalue for="5" value=" Due to the advantages over other VOS methods\cite{other1,other2,other3,other4}, the memory-based paradigm has been paid much attention by the research community." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1904.00607" id="2523">
-        <attvalues>
-          <attvalue for="5" value=" As one of the most successful early attempts, Space-Time Memory network (STM)\cite{r5} stores the past frames with object masks into the memory and performs pixel-level matching between the encoded key of query frame and memory." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2106.05210" id="2524">
-        <attvalues>
-          <attvalue for="5" value=" Space-Time Correspondence Network (STCN)\cite{r6} is developed from STM, it encodes the key features from frames without masks and replace dot product with L2 similarity in the affinity for memory reading." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2207.07115" id="2525">
-        <attvalues>
-          <attvalue for="5" value=" XMem\cite{r7} introduces three indendent memory banks: a sensory memory, a working memory and a long-term memory." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2310.12982" id="2526">
-        <attvalues>
-          <attvalue for="5" value=" To solve the mismatch problem in pixel-level matching, Cutie\cite{r8} proposes the object memory and object transformer for bidirectional information interaction." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1809.03327" id="2527">
-        <attvalues>
-          <attvalue for="5" value=" Apart from higher disappearance-reappearance rates and heavier occlusions than previous VOS datasets such as DAVIS\cite{r10} and YouTubeVOS\cite{r11}, MOSE has many tiny and similar objects, which may confuse VOS models." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="2112.01527" id="2528">
-        <attvalues>
-          <attvalue for="5" value=" To mitigate the above issues, we combine Mask2Former\cite{r12} and motion blur as data augmentation to enhance semantic representation during the training process." />
-        </attvalues>
-      </edge>
-      <edge source="2406.08192" target="1405.0312" id="2529">
-        <attvalues>
-          <attvalue for="5" value=" We also introduce COCO\cite{r13} to further enrich the pretraining data, with the purpose of enhancing semantic learning in the early stage of training and improve the segmentation accuracy of tiny and similar objects." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1606.04080" id="2530">
-        <attvalues>
-          <attvalue for="5" value=" The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1703.05175" id="2531">
-        <attvalues>
-          <attvalue for="5" value=" The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1703.03400" id="2532">
-        <attvalues>
-          <attvalue for="5" value=" The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1711.06025" id="2533">
-        <attvalues>
-          <attvalue for="5" value=" The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1706.03466" id="2534">
-        <attvalues>
-          <attvalue for="5" value=" The few-shot learning literature has mainly focused on the classification task such as~\cite{koch2015siamese}\cite{vinyals2016matching}\cite{snell2017prototypical}\cite{qi2017learning}\cite{finn2017model}\cite{ravi2016optimization}\cite{sung2018learning}\cite{qiao2018few}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1709.03410" id="2535">
-        <attvalues>
-          <attvalue for="5" value=" Recently, solutions for few-shot object segmentation which learns pixel-wise classification based on few labelled samples have emerged~\cite{shaban2017one}\cite{rakelly2018conditional}\cite{dong2018few}\cite{wang2019panet}\cite{zhang2019pyramid}\cite{zhang2019canet}\cite{siam2019amp}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1908.06391" id="2536">
-        <attvalues>
-          <attvalue for="5" value=" A few recent works~\cite{rakelly2018conditional}\cite{zhang2019canet}\cite{wang2019panet} started to conduct experiments using weak annotations in terms of scribbles, bounding boxes." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1903.02351" id="2537">
-        <attvalues>
-          <attvalue for="5" value=" A few recent works~\cite{rakelly2018conditional}\cite{zhang2019canet}\cite{wang2019panet} started to conduct experiments using weak annotations in terms of scribbles, bounding boxes." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1310.4546" id="2538">
-        <attvalues>
-          <attvalue for="5" value=" The use of neural word embeddings pretrained on GoogleNews~\cite{mikolov2013distributed} and visual embeddings from pretrained networks on ImageNet~\cite{deng2009imagenet} allows to build such a model." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="2001.06810" id="2539">
-        <attvalues>
-          <attvalue for="5" value=" Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1704.00675" id="2540">
-        <attvalues>
-          <attvalue for="5" value=" Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1612.07217" id="2541">
-        <attvalues>
-          <attvalue for="5" value=" Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1701.05384" id="2542">
-        <attvalues>
-          <attvalue for="5" value=" Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1807.09190" id="2543">
-        <attvalues>
-          <attvalue for="5" value=" Concurrent to few-shot object segmentation, video object segmentation(VOS) has been extensively researched~\cite{Pont-Tuset_arXiv_2017}\cite{Perazzi2016}\cite{lu2019see}\cite{tokmakov2016learning}\cite{jain2017fusionseg}\cite{luiten2018premvos}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1803.08006" id="2544">
-        <attvalues>
-          <attvalue for="5" value=" In a recent work~\cite{khoreva2018video} video object segmentation using language expression has been studied." />
-        </attvalues>
-      </edge>
-      <edge source="1912.08936" target="1809.03327" id="2545">
-        <attvalues>
-          <attvalue for="5" value=" To this end we use Youtube-VOS dataset~\cite{xu2018youtube} which is a large-scale VOS dataset with 65 different object categories, the originally provided training data is further split into 5 different folds with 13 novel classes per fold." />
-        </attvalues>
-      </edge>
-      <edge source="2102.12670" target="2001.11584" id="2546">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there has been an effort to implement a learning-based method specifically for ellipse detection~\cite{cnnellipse}." />
-        </attvalues>
-      </edge>
-      <edge source="2102.12670" target="1608.07470" id="2547">
-        <attvalues>
-          <attvalue for="5" value=" The authors of~\cite{jin2018} have developed a Convolutional Neural Network trained with over 5,000 images collected from the pattern moving at a maximum speed of $15$\,km/h at various heights to select (validate) the desired ellipse from all the ellipses detected using the method in~\cite{jia2017}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1703.03400" id="2548">
-        <attvalues>
-          <attvalue for="5" value=" The most successful demonstrations have been on datasets such as mini-ImageNet \cite{finn2017model}, ImageNet \cite{deng2009imagenet}, and Omniglot \cite{lake2019omniglot}---datasets comprised of images typically containing centered, non-occluded objects commonly appearing in the foreground." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1902.03477" id="2549">
-        <attvalues>
-          <attvalue for="5" value=" The most successful demonstrations have been on datasets such as mini-ImageNet \cite{finn2017model}, ImageNet \cite{deng2009imagenet}, and Omniglot \cite{lake2019omniglot}---datasets comprised of images typically containing centered, non-occluded objects commonly appearing in the foreground." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="2003.04052" id="2550">
-        <attvalues>
-          <attvalue for="5" value=" Related Work Given saturation of performance on the standard few-shot benchmarks, recent works have extended few-shot tasks to more difficult settings, eg object classes with confounding texture \cite{azad2020texture}, cluttered scene segmentation \cite{michaelis2018one, fortin2019few}, long tailed distributions~\cite{wertheimer_few-shot_2019}, and segmentation of homogeneous object clusters ~\cite{wu2018annotation}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1803.09597" id="2551">
-        <attvalues>
-          <attvalue for="5" value=" Related Work Given saturation of performance on the standard few-shot benchmarks, recent works have extended few-shot tasks to more difficult settings, eg object classes with confounding texture \cite{azad2020texture}, cluttered scene segmentation \cite{michaelis2018one, fortin2019few}, long tailed distributions~\cite{wertheimer_few-shot_2019}, and segmentation of homogeneous object clusters ~\cite{wu2018annotation}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1904.08502" id="2552">
-        <attvalues>
-          <attvalue for="5" value=" We include comparison with the Few-Shot Localization (FSL) method presented in \cite{wertheimer_few-shot_2019} in our experimental evaluations." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1802.00383" id="2553">
-        <attvalues>
-          <attvalue for="5" value=" Related Work Given saturation of performance on the standard few-shot benchmarks, recent works have extended few-shot tasks to more difficult settings, eg object classes with confounding texture \cite{azad2020texture}, cluttered scene segmentation \cite{michaelis2018one, fortin2019few}, long tailed distributions~\cite{wertheimer_few-shot_2019}, and segmentation of homogeneous object clusters ~\cite{wu2018annotation}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1703.05175" id="2554">
-        <attvalues>
-          <attvalue for="5" value=" Enhancements to Prototypical Networks (ProtoNets)~\cite{snell_prototypical_2017} in particular have been proposed for difficult few-shot tasks involving, for instance, inhomogeneous noisy datasets~\cite{fort2017gaussian}, domain adaptation~\cite{Pan_2019_CVPR}, and relation classification in text~\cite{gao2019hybrid}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1708.02735" id="2555">
-        <attvalues>
-          <attvalue for="5" value=" Enhancements to Prototypical Networks (ProtoNets)~\cite{snell_prototypical_2017} in particular have been proposed for difficult few-shot tasks involving, for instance, inhomogeneous noisy datasets~\cite{fort2017gaussian}, domain adaptation~\cite{Pan_2019_CVPR}, and relation classification in text~\cite{gao2019hybrid}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.03496" target="1904.11227" id="2556">
-        <attvalues>
-          <attvalue for="5" value=" Enhancements to Prototypical Networks (ProtoNets)~\cite{snell_prototypical_2017} in particular have been proposed for difficult few-shot tasks involving, for instance, inhomogeneous noisy datasets~\cite{fort2017gaussian}, domain adaptation~\cite{Pan_2019_CVPR}, and relation classification in text~\cite{gao2019hybrid}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1606.00915" id="2557">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1804.09337" id="2558">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1511.07122" id="2559">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1811.11721" id="2560">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1809.05996" id="2561">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1910.09777" id="2562">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1908.09798" id="2563">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1605.06211" id="2564">
-        <attvalues>
-          <attvalue for="5" value=" Current state-of-the-art semantic segmentation approaches ~\cite{chen2017deeplab,yu2018learning, zhao2017pyramid,yu2015multi,peng2017large,huang2019ccnet,ruan2019devil,li2020self,cheng2019spgnet, cheng2020panoptic} based on the fully convolutional network (FCN)~\cite{long2015fully} have made remarkable progress." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1806.02479" id="2565">
-        <attvalues>
-          <attvalue for="5" value=" For face parsing, CNN-based features are imported into network to extract multi-scale or independent-part facial features~\cite{liu2015multi,luo2012hierarchical,zhou2015inter}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="1906.01342" id="2566">
-        <attvalues>
-          <attvalue for="5" value=" propose two branches with the local-based for inner facial components and the global based for outer facial ones~\cite{lin2019face}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="2007.11240" id="2567">
-        <attvalues>
-          <attvalue for="5" value=" propose to learn graph representations over facial images, which model the relations between regions~\cite{te2020edge}." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="2010.11929" id="2568">
-        <attvalues>
-          <attvalue for="5" value=" Transformers~\cite{dosovitskiy2020image,nicolas2020end,hugo2020training,liu2021swin} have achieve excellent performance on a wide range of visual tasks including image-level classification, object detection, and semantic segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="2005.12872" id="2569">
-        <attvalues>
-          <attvalue for="5" value=" Transformers~\cite{dosovitskiy2020image,nicolas2020end,hugo2020training,liu2021swin} have achieve excellent performance on a wide range of visual tasks including image-level classification, object detection, and semantic segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="2103.14030" id="2570">
-        <attvalues>
-          <attvalue for="5" value=" Transformers~\cite{dosovitskiy2020image,nicolas2020end,hugo2020training,liu2021swin} have achieve excellent performance on a wide range of visual tasks including image-level classification, object detection, and semantic segmentation." />
-        </attvalues>
-      </edge>
-      <edge source="2106.08650" target="2106.03650" id="2571">
-        <attvalues>
-          <attvalue for="5" value=" And Shuffle Transformer~\cite{huang2021shuffle} is confirmed as a strong backbone for many vision tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2408.16190" target="2003.12039" id="2572">
-        <attvalues>
-          <attvalue for="5" value="Kautz_PWC-Net_2018}, and RAFT \cite{TeedDeng_RAFT_2020} have seen significant improvement in accuracy and computation time over their knowledge-driven counterparts \cite{Zhai." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="1908.03737" id="2573">
-        <attvalues>
-          <attvalue for="5" value=" Given a query item in one modality, cross-modal retrieval is the task of retrieving similar items in another modality~\cite{zeng2020deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="1909.02701" id="2574">
-        <attvalues>
-          <attvalue for="5" value=" We focus on \acfi{ICR}~\cite{li2019visual, diao2021similarity, lee2018stacked,verma-2020-using}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2003.00708" id="2575">
-        <attvalues>
-          <attvalue for="5" value=" We focus on \acfi{ICR}~\cite{li2019visual, diao2021similarity, lee2018stacked,verma-2020-using}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2101.01368" id="2576">
-        <attvalues>
-          <attvalue for="5" value=" Most ICR methods work with a separate encoder for each modality to map the input data to a representation in a shared latent space~\cite{diao2021similarity, li2019visual, faghri2018vse++, jia2021scaling, lee2018stacked}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2102.05918" id="2577">
-        <attvalues>
-          <attvalue for="5" value=" A lot of recent work on ICR relies on \begin{enumerate*}[label=(\arabic*)] &#09;\item pre-training on large amounts of data \cite{jia2021scaling, radford2021learning, li2020oscar}, and &#09;\item more sophisticated (and data-hungry) model architectures \cite{messina2021transformer, faghri2018vse++, li2019visual, diao2021similarity, lee2018stacked, chen2020intriguing}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2004.09144" id="2578">
-        <attvalues>
-          <attvalue for="5" value=" A lot of recent work on ICR relies on \begin{enumerate*}[label=(\arabic*)] &#09;\item pre-training on large amounts of data \cite{jia2021scaling, radford2021learning, li2020oscar}, and &#09;\item more sophisticated (and data-hungry) model architectures \cite{messina2021transformer, faghri2018vse++, li2019visual, diao2021similarity, lee2018stacked, chen2020intriguing}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="1707.05612" id="2579">
-        <attvalues>
-          <attvalue for="5" value=" To learn the similarity between a query and candidate representations, most ICR work relies on the standard Triplet loss with semi-hard negatives (Triplet SH)~\cite{faghri2018vse++, li2019visual, messina2021transformer, diao2021similarity, lee2018stacked, chen2020intriguing, chen2020imram} or on the cross-entropy based NT-Xent~\cite{chen2020adaptive, jia2021scaling} loss." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2003.03669" id="2580">
-        <attvalues>
-          <attvalue for="5" value=" To learn the similarity between a query and candidate representations, most ICR work relies on the standard Triplet loss with semi-hard negatives (Triplet SH)~\cite{faghri2018vse++, li2019visual, messina2021transformer, diao2021similarity, lee2018stacked, chen2020intriguing, chen2020imram} or on the cross-entropy based NT-Xent~\cite{chen2020adaptive, jia2021scaling} loss." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2003.08505" id="2581">
-        <attvalues>
-          <attvalue for="5" value=" Following~\cite{musgrave2020metric}, we evaluate the three loss functions on fixed methods, with different datasets, and with a fixed training regime (eg, training hyper-parameters) to verify which loss function uses the given training data as effectively as possible." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2007.12163" id="2582">
-        <attvalues>
-          <attvalue for="5" value=" \end{enumerate*} We expect SmoothAP to result in the highest performance based on the findings in in the context of image retrieval \cite{brown2020smooth} and in representation learning~\cite{varamesh2020self}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="1411.2539" id="2583">
-        <attvalues>
-          <attvalue for="5" value=" Hence, can newly introduced metric learning approaches, that is, alternative loss functions, be used to increase the performance of ICR methods? We compare three loss function for the ICR task: \begin{enumerate*}[label=(\arabic*)] \item the Triplet loss \cite{kiros2014unifying}, including semi-hard negative mining, \item NT-Xent loss~\cite{chen2020simple}, and \item SmoothAP~\cite{brown2020smooth}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2002.05709" id="2584">
-        <attvalues>
-          <attvalue for="5" value=" Hence, can newly introduced metric learning approaches, that is, alternative loss functions, be used to increase the performance of ICR methods? We compare three loss function for the ICR task: \begin{enumerate*}[label=(\arabic*)] \item the Triplet loss \cite{kiros2014unifying}, including semi-hard negative mining, \item NT-Xent loss~\cite{chen2020simple}, and \item SmoothAP~\cite{brown2020smooth}." />
-        </attvalues>
-      </edge>
-      <edge source="2202.07474" target="2010.07258" id="2585">
-        <attvalues>
-          <attvalue for="5" value=" \end{enumerate*} We expect SmoothAP to result in the highest performance based on the findings in in the context of image retrieval \cite{brown2020smooth} and in representation learning~\cite{varamesh2020self}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1612.00593" id="2586">
-        <attvalues>
-          <attvalue for="5" value=" Deep neural networks such as PointNet\cite{pointnet}, DGCNN\cite{DGCNN} have been proposed for better performances on point cloud related tasks, with the help of larger datasets such as the ModelNet\cite{modelnet} and Semantic3D\cite{semantic3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1801.07829" id="2587">
-        <attvalues>
-          <attvalue for="5" value=" Deep neural networks such as PointNet\cite{pointnet}, DGCNN\cite{DGCNN} have been proposed for better performances on point cloud related tasks, with the help of larger datasets such as the ModelNet\cite{modelnet} and Semantic3D\cite{semantic3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1406.5670" id="2588">
-        <attvalues>
-          <attvalue for="5" value=" Deep neural networks such as PointNet\cite{pointnet}, DGCNN\cite{DGCNN} have been proposed for better performances on point cloud related tasks, with the help of larger datasets such as the ModelNet\cite{modelnet} and Semantic3D\cite{semantic3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1704.03847" id="2589">
-        <attvalues>
-          <attvalue for="5" value="6 million points Oakland Dataset\cite{oakland}, the 215 million points Stanford Large-scale 3D Indoor Spaces Dataset (S3DIS) dataset\cite{S3DIS} and the 4 billion points Semantic3D\cite{semantic3d}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1908.08854" id="2590">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, point cloud semantic segmentation is of great interests in the applications of autonomous driving, robotics and remote sensing\cite{xie2020linking}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1603.09246" id="2591">
-        <attvalues>
-          <attvalue for="5" value=" This self-supervised approach has found success in designing ``pretext&quot; tasks, such as jigsaw puzzle reassembly\cite{jigsaw}, image clustering\cite{imageClustering} and image rotation prediction\cite{rotation} etc, by training deep learning models for feature extraction without labels being involved." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1807.05520" id="2592">
-        <attvalues>
-          <attvalue for="5" value=" This self-supervised approach has found success in designing ``pretext&quot; tasks, such as jigsaw puzzle reassembly\cite{jigsaw}, image clustering\cite{imageClustering} and image rotation prediction\cite{rotation} etc, by training deep learning models for feature extraction without labels being involved." />
-        </attvalues>
-      </edge>
-      <edge source="2201.04833" target="1811.11387" id="2593">
-        <attvalues>
-          <attvalue for="5" value=" This self-supervised approach has found success in designing ``pretext&quot; tasks, such as jigsaw puzzle reassembly\cite{jigsaw}, image clustering\cite{imageClustering} and image rotation prediction\cite{rotation} etc, by training deep learning models for feature extraction without labels being involved." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1504.08289" id="2594">
-        <attvalues>
-          <attvalue for="5" value=" Early works on localization-based methods often adopt part detectors trained on part annotations and then extract part-specific features for fine-grained classification \cite{xu2015augmenting, xiao2015application,simon2015neural,zhang2016picking}." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1806.05372" id="2595">
-        <attvalues>
-          <attvalue for="5" value=" \cite{sun2018multi} proposes a soft attenion-based model that generates semantic region features." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1711.07971" id="2596">
-        <attvalues>
-          <attvalue for="5" value=" Recently, non-local (NL) module~\cite{wang2018non} is proposed to model the long-range dependencies in one layer, via self-attention mechanism to form an attention map." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1810.13125" id="2597">
-        <attvalues>
-          <attvalue for="5" value=" \cite{yue2018compact,cao2019gcnet} show the NL module could bring excellent improvement in image recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1904.11492" id="2598">
-        <attvalues>
-          <attvalue for="5" value=" \cite{yue2018compact,cao2019gcnet} show the NL module could bring excellent improvement in image recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1606.08572" id="2599">
-        <attvalues>
-          <attvalue for="5" value=" However, for fine-grained classification task, multi-scale features are vital because object parts have various sizes and shapes in images~\cite{zhao2017diversified,luo2019cross,he2015spatial}." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1909.04412" id="2600">
-        <attvalues>
-          <attvalue for="5" value=" However, for fine-grained classification task, multi-scale features are vital because object parts have various sizes and shapes in images~\cite{zhao2017diversified,luo2019cross,he2015spatial}." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1406.4729" id="2601">
-        <attvalues>
-          <attvalue for="5" value=" However, for fine-grained classification task, multi-scale features are vital because object parts have various sizes and shapes in images~\cite{zhao2017diversified,luo2019cross,he2015spatial}." />
-        </attvalues>
-      </edge>
-      <edge source="2005.09153" target="1512.03385" id="2602">
-        <attvalues>
-          <attvalue for="5" value=" We evaluate our CNL method on three datasets of fine-grained classification, eg CUB-200-2011~\cite{wah2011caltech}, Stanford Dogs~\cite{khosla2011novel} and Stanford Cars~\cite{he2016deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1902.03604" id="2603">
-        <attvalues>
-          <attvalue for="5" value=" \cite{voigtlaender2019mots} propose a new baseline approach, Track R-CNN, which addresses detection, tracking, and segmentation via a unified convolutional neural network." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1506.01497" id="2604">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies on object detection and segmentation achieve impressive performance with various advanced structures \cite{ren2015faster,he2017mask}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1703.06870" id="2605">
-        <attvalues>
-          <attvalue for="5" value=" With the given detection results, the sequences are sent into a variant of Mask R-CNN \cite{he2017mask} to obtain the embedding features as region proposals of RPN." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1503.03832" id="2606">
-        <attvalues>
-          <attvalue for="5" value=" FaceNet \cite{schroff2015facenet} proposes a metric learning strategy." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1909.12605" id="2607">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, a multi-scaled pedestrian detector \cite{jde} is proposed to allow target detection and appearance features to be learned in a shared model." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1811.07258" id="2608">
-        <attvalues>
-          <attvalue for="5" value=" \cite{wang2019exploit} propose the TrackletNet Tracking (TNT) framework that jointly considers appearance and geometric features to form a tracklet-based graph model." />
-        </attvalues>
-      </edge>
-      <edge source="2006.13458" target="1712.07920" id="2609">
-        <attvalues>
-          <attvalue for="5" value=" CAMOT \cite{ovsep2018track} exploits stereo information to perform mask-based tracking of generic objects on the KITTI dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12533" target="2004.02215" id="2610">
-        <attvalues>
-          <attvalue for="5" value="2980251}, super resolution~\cite{Jin_2020_CVPR,meng2020high}, etc One of the most important tasks of the LF camera is depth estimation~\cite{jeon2015accurate,tao2013depth,wanner2012globally,8985549,Huang_2017_ICCV,rogge2020depth,8263242}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.12533" target="2003.13094" id="2611">
-        <attvalues>
-          <attvalue for="5" value="2980251}, super resolution~\cite{Jin_2020_CVPR,meng2020high}, etc One of the most important tasks of the LF camera is depth estimation~\cite{jeon2015accurate,tao2013depth,wanner2012globally,8985549,Huang_2017_ICCV,rogge2020depth,8263242}." />
-        </attvalues>
-      </edge>
-      <edge source="2104.10847" target="1810.10658" id="2612">
-        <attvalues>
-          <attvalue for="5" value=" For smoothing or refinement of the initial homography estimation, some works in the literature such as\cite{chen2019sports}, use Lucas-Kanade algorithm \cite{baker2004lucas}, which is based on tracking feature patches in the video frames." />
-        </attvalues>
-      </edge>
-      <edge source="2104.10847" target="2003.14109" id="2613">
-        <attvalues>
-          <attvalue for="5" value=" Generally, the state-of-the-art procedures for localizing the sports fields \cite{citraro2020realtime, jiang2020optimizing, sha2020endtoend} utilize deep structures for registering the video frames on the model of the sport filed, by minimizing a distance metric between the ground-truth and predicted points (or areas) of a frame on the model (or vice versa)." />
-        </attvalues>
-      </edge>
-      <edge source="2104.10847" target="1909.08034" id="2614">
-        <attvalues>
-          <attvalue for="5" value=" A frame-based hockey ice-rink localization is performed by inspiration from \cite{jiang2020optimizing}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.04238" target="2007.15156" id="2615">
-        <attvalues>
-          <attvalue for="5" value=" Others adopt a multiple exposure fusion approach~\cite{jinno2012multiple, pradeep2012robust,xu2022multi,zhang2021benchmarking}, which combines the multi-exposed LDR images to estimate the irradiance value for each pixel." />
-        </attvalues>
-      </edge>
-      <edge source="2211.04238" target="1711.08937" id="2616">
-        <attvalues>
-          <attvalue for="5" value="~\cite{wu2018deep} proposed the first end-to-end deep neural network without optical flow alignment for HDR reconstruction, which includes three encoder networks, a merger network, and a decoder network." />
-        </attvalues>
-      </edge>
-      <edge source="2211.04238" target="1904.10293" id="2617">
-        <attvalues>
-          <attvalue for="5" value="~\cite{yan2019attention} were the first to use attention networks (AHDRNet) aiming to highlight or align large motions in the features prior to the merger network, which can then suppress any ghosting artifacts." />
-        </attvalues>
-      </edge>
-      <edge source="2211.04238" target="2106.01439" id="2618">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, this work inspired several following works, which adapted their proposed architecture; for example, see several solutions in the recent NTIRE 2021 and 2022 challenges on HDR~\cite{eduardo2021ntire, perez2022ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="1501.00092" id="2619">
-        <attvalues>
-          <attvalue for="5" value=" This paper considers image deconvolution methodology for situations in which there is no suitable training data available to use as ground truth (with regards to methodology for situations with abundant training data available, we refer the reader to the recent works \cite{dong2015image, kupyn2018deblurgan, albluwi2018image,monga2021algorithm,pesquet2021learning,li2020efficient,bertocchi2020deep} based on deep learning strategies)." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="1912.10557" id="2620">
-        <attvalues>
-          <attvalue for="5" value=" This paper considers image deconvolution methodology for situations in which there is no suitable training data available to use as ground truth (with regards to methodology for situations with abundant training data available, we refer the reader to the recent works \cite{dong2015image, kupyn2018deblurgan, albluwi2018image,monga2021algorithm,pesquet2021learning,li2020efficient,bertocchi2020deep} based on deep learning strategies)." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="2012.13247" id="2621">
-        <attvalues>
-          <attvalue for="5" value=" This paper considers image deconvolution methodology for situations in which there is no suitable training data available to use as ground truth (with regards to methodology for situations with abundant training data available, we refer the reader to the recent works \cite{dong2015image, kupyn2018deblurgan, albluwi2018image,monga2021algorithm,pesquet2021learning,li2020efficient,bertocchi2020deep} based on deep learning strategies)." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="1812.04276" id="2622">
-        <attvalues>
-          <attvalue for="5" value=" This paper considers image deconvolution methodology for situations in which there is no suitable training data available to use as ground truth (with regards to methodology for situations with abundant training data available, we refer the reader to the recent works \cite{dong2015image, kupyn2018deblurgan, albluwi2018image,monga2021algorithm,pesquet2021learning,li2020efficient,bertocchi2020deep} based on deep learning strategies)." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="0910.4887" id="2623">
-        <attvalues>
-          <attvalue for="5" value=" There are many regularisation strategies available to make these problems well-posed, offering a wide range of possibilities in terms of estimation accuracy and computational efficiency (see, eg, \cite{afonso2010fast,beck2009fast,laumont2022bayesian,chantas2006bayesian, guerrero2006deblurring, bar2006image})." />
-        </attvalues>
-      </edge>
-      <edge source="2403.04536" target="1303.3866" id="2624">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, in a manner akin to blind problems, semi-blind problems can be solved by joint MAP estimation to determine the unknown image and the parameters of $h$ from $y$ \cite{almeida2009blind}, by marginal MAP or MMSE estimation (following the marginalisation of the parameters of $h$) \cite{orieux2010bayesian,Park2014}, or by estimating the parameter of $h$ by MMLE followed by empirical Bayesian MAP estimation of $x^\star$ given $y$ and the estimated blur parameters." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2202.10261" id="2625">
-        <attvalues>
-          <attvalue for="5" value=" In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2205.12358" id="2626">
-        <attvalues>
-          <attvalue for="5" value=" In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2210.10620" id="2627">
-        <attvalues>
-          <attvalue for="5" value=" In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2203.02654" id="2628">
-        <attvalues>
-          <attvalue for="5" value=" In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2211.13090" id="2629">
-        <attvalues>
-          <attvalue for="5" value=" In response to this situation, the field of copy detection has experienced a swift expansion in recent years, with research on image and video copy detection~\cite{SSCD2022,wang2023benchmark,fernandez2022active,Jiang2014VCDBALCopy,he2022largeVCSL,he2023transvcl} gaining significant attention." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2106.09672" id="2630">
-        <attvalues>
-          <attvalue for="5" value=" Notably, the Image Similarity Challenge at NeurIPS'21 (ISC2021) was organized as a competition for image copy detection~\cite{ISC2021,papakipos2022resultsISC}, followed by the recent organization of the Meta AI Video Similarity Challenge at CVPR’23 (VSC2022) focusing on video copy detection." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2202.04007" id="2631">
-        <attvalues>
-          <attvalue for="5" value=" Notably, the Image Similarity Challenge at NeurIPS'21 (ISC2021) was organized as a competition for image copy detection~\cite{ISC2021,papakipos2022resultsISC}, followed by the recent organization of the Meta AI Video Similarity Challenge at CVPR’23 (VSC2022) focusing on video copy detection." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11964" target="2112.04323" id="2632">
-        <attvalues>
-          <attvalue for="5" value=" Our solution can be summarized as follows: \begin{enumerate} \item We propose a simple yet powerful video copy detection pipeline, based on an image copy detection model (ISC-dt1~\cite{ISC-dt1})." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1703.02910" id="2633">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="2103.16130" id="2634">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="2004.04699" id="2635">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="2103.10374" id="2636">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1901.10609" id="2637">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1604.03540" id="2638">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1708.02002" id="2639">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1904.02701" id="2640">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1608.07441" id="2641">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="2101.05036" id="2642">
-        <attvalues>
-          <attvalue for="5" value=" The main objective of these works was to either perform active learning mainly to reduce the labelling and training cost \cite{Gal2017-vh,Choi2021-zg,Roy2018-lv, Aghdam2019-dx,Haussmann2020-vg,Yu2021-fe,Feng2019-yt}, or to improve the training mechanism~\cite{OHEM,retinanet,LibraRCNN,ovsampler}, perform metric learning~\cite{Suh2019-tf,Bucher2016-mp}, or even to improve the reliability of object detectors towards out-of-distribution samples~\cite{RegressionUncOD,VOS}." />
-        </attvalues>
-      </edge>
-      <edge source="2209.11559" target="1806.01768" id="2643">
-        <attvalues>
-          <attvalue for="5" value=" Note that the widely used uncertainty metrics that are often used to quantify ``hardness'', such as entropy~\cite{ShannonEntropy} and Dempster-Shafer~\cite{sensoy2018evidential}, to name a few, are completely agnostic to such requirements." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="2001.00179" id="2644">
-        <attvalues>
-          <attvalue for="5" value=" \item Type of deepfakes: We focus on popular types of facial deepfakes~\cite{tolosana2020deepfakes}, comprising identity swaps, expression swaps, and entire face synthesis, whose datasets can be found or easily generated." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="1901.08971" id="2645">
-        <attvalues>
-          <attvalue for="5" value=" To overcome this challenge, the use of transfer learning and fine-tuning has become a commonly used approach~\cite{rossler2019faceforensics++, nguyen2019capsule, ferrer2020deepfake, zhao2021multi, sun2022dual} in which researchers and practitioners leverage pre-trained backbones from computer vision (CV)-related models trained on ImageNet~\cite{deng2009imagenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="1409.1556" id="2646">
-        <attvalues>
-          <attvalue for="5" value=" While ImageNet has played a vital role in the success of several deep learning models, such as VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, XceptionNet~\cite{chollet2017xception}, and EfficientNet~\cite{tan2019efficientnet}, the effectiveness of the pre-trained backbones for the DFD task remains uncertain." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="1512.03385" id="2647">
-        <attvalues>
-          <attvalue for="5" value=" While ImageNet has played a vital role in the success of several deep learning models, such as VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, XceptionNet~\cite{chollet2017xception}, and EfficientNet~\cite{tan2019efficientnet}, the effectiveness of the pre-trained backbones for the DFD task remains uncertain." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="1610.02357" id="2648">
-        <attvalues>
-          <attvalue for="5" value=" While ImageNet has played a vital role in the success of several deep learning models, such as VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, XceptionNet~\cite{chollet2017xception}, and EfficientNet~\cite{tan2019efficientnet}, the effectiveness of the pre-trained backbones for the DFD task remains uncertain." />
-        </attvalues>
-      </edge>
-      <edge source="2310.00922" target="1905.11946" id="2649">
-        <attvalues>
-          <attvalue for="5" value=" While ImageNet has played a vital role in the success of several deep learning models, such as VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, XceptionNet~\cite{chollet2017xception}, and EfficientNet~\cite{tan2019efficientnet}, the effectiveness of the pre-trained backbones for the DFD task remains uncertain." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1512.02325" id="2650">
-        <attvalues>
-          <attvalue for="5" value=" Many past studies\cite{ssd,fpn,retinanet,yolov2} have shown that feature maps in a feature pyramid can capture an object's visual features at different scales." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1612.03144" id="2651">
-        <attvalues>
-          <attvalue for="5" value=" This way makes FPN~\cite{fpn} significantly improve object detection performance and become a standard component of most SoTA object detectors~\cite{yolov1,prb}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1708.02002" id="2652">
-        <attvalues>
-          <attvalue for="5" value=" Many past studies\cite{ssd,fpn,retinanet,yolov2} have shown that feature maps in a feature pyramid can capture an object's visual features at different scales." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1612.08242" id="2653">
-        <attvalues>
-          <attvalue for="5" value=" Many past studies\cite{ssd,fpn,retinanet,yolov2} have shown that feature maps in a feature pyramid can capture an object's visual features at different scales." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1506.02640" id="2654">
-        <attvalues>
-          <attvalue for="5" value=" This way makes FPN~\cite{fpn} significantly improve object detection performance and become a standard component of most SoTA object detectors~\cite{yolov1,prb}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="2012.01724" id="2655">
-        <attvalues>
-          <attvalue for="5" value=" This way makes FPN~\cite{fpn} significantly improve object detection performance and become a standard component of most SoTA object detectors~\cite{yolov1,prb}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1409.1556" id="2656">
-        <attvalues>
-          <attvalue for="5" value=" Then, the SFPN is integrated on VGG-16~\cite{vgg} and MobileNetV2~\cite{mobilenetv2}, respectively, and evaluated in the MS-COCO~\cite{mscoco} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1801.04381" id="2657">
-        <attvalues>
-          <attvalue for="5" value=" Then, the SFPN is integrated on VGG-16~\cite{vgg} and MobileNetV2~\cite{mobilenetv2}, respectively, and evaluated in the MS-COCO~\cite{mscoco} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2203.02445" target="1405.0312" id="2658">
-        <attvalues>
-          <attvalue for="5" value=" Then, the SFPN is integrated on VGG-16~\cite{vgg} and MobileNetV2~\cite{mobilenetv2}, respectively, and evaluated in the MS-COCO~\cite{mscoco} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1605.06211" id="2659">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1706.05587" id="2660">
-        <attvalues>
-          <attvalue for="5" value=" PSPNet~\cite{zhao2017pspnet}, Deeplabv3~\cite{deeplabv3}, Deeplabv3+~\cite{deeplabv3plus} capture information at multiple scales by either applying pooling operations with different kernel size and they called it pyramid pooling module (PPM) or employing dilated convolution with different rate and that was called Atrous Spatial Pyramid Pooling (ASPP)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1611.05431" id="2661">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation~\cite{long2015fully, deeplabv3plus, zhao2017pspnet,deeplabv3}, and similar to image classification~\cite{xie2017aggregated,resnet,szegedy2015going} and object detection~\cite{yolo,lin2017focal}, has seen considerable progress due to the employment of deep learning architectures, especially convolutional neural networks (CNN)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1512.03385" id="2662">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1409.4842" id="2663">
-        <attvalues>
-          <attvalue for="5" value=" Most of the mentioned approaches are not efficient for real-time applications as they employed large backbone networks such as GoogleNet~\cite{szegedy2015going}, Xcepetion~\cite{simonyan2014very}, or ResNet~\cite{resnet}, or employed a large CNN architectures for both the encoder or decoder sides." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1612.08242" id="2664">
-        <attvalues>
-          <attvalue for="5" value=" \\ In summary, our main contributions are: \begin{itemize} \item LiteSeg, a real time competitive architecture is presented and tested with three different backbone networks, Darknet19~\cite{yolo}, MobileNetV2~\cite{mobilenetv2}, and ShuffleNet~\cite{shufflenet}, achieving performance 70." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1708.02002" id="2665">
-        <attvalues>
-          <attvalue for="5" value=" Semantic segmentation~\cite{long2015fully, deeplabv3plus, zhao2017pspnet,deeplabv3}, and similar to image classification~\cite{xie2017aggregated,resnet,szegedy2015going} and object detection~\cite{yolo,lin2017focal}, has seen considerable progress due to the employment of deep learning architectures, especially convolutional neural networks (CNN)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1505.04597" id="2666">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1802.02611" id="2667">
-        <attvalues>
-          <attvalue for="5" value=" PSPNet~\cite{zhao2017pspnet}, Deeplabv3~\cite{deeplabv3}, Deeplabv3+~\cite{deeplabv3plus} capture information at multiple scales by either applying pooling operations with different kernel size and they called it pyramid pooling module (PPM) or employing dilated convolution with different rate and that was called Atrous Spatial Pyramid Pooling (ASPP)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1612.01105" id="2668">
-        <attvalues>
-          <attvalue for="5" value=" PSPNet~\cite{zhao2017pspnet}, Deeplabv3~\cite{deeplabv3}, Deeplabv3+~\cite{deeplabv3plus} capture information at multiple scales by either applying pooling operations with different kernel size and they called it pyramid pooling module (PPM) or employing dilated convolution with different rate and that was called Atrous Spatial Pyramid Pooling (ASPP)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1803.06815" id="2669">
-        <attvalues>
-          <attvalue for="5" value=" Alternatively, ESPNet~\cite{espnet} proposed an efficient module called efficient spatial pyramid (ESP), which uses point wise convolution and spatial pyramid of dilated convolution." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1606.02147" id="2670">
-        <attvalues>
-          <attvalue for="5" value=" Numerous attempts have been investigated in providing lightweight semantic segmentation architectures, such as ERFNet~\cite{erfnet}, ESPNet~\cite{espnet}, Enet~\cite{enet}, CCC~\cite{ccc2}, and DSNet~\cite{dsnet}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1812.07049" id="2671">
-        <attvalues>
-          <attvalue for="5" value=" Numerous attempts have been investigated in providing lightweight semantic segmentation architectures, such as ERFNet~\cite{erfnet}, ESPNet~\cite{espnet}, Enet~\cite{enet}, CCC~\cite{ccc2}, and DSNet~\cite{dsnet}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1409.1556" id="2672">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1411.4734" id="2673">
-        <attvalues>
-          <attvalue for="5" value=" For example, the work in~\cite{eigen2015predicting} proposed a multi-scale network which employed a different three scale to generate a fine, high resolution predictions." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1505.04366" id="2674">
-        <attvalues>
-          <attvalue for="5" value=" Another solution was proposed by ~\cite{noh2015learning}, in which a more complex deconvolution network was used to produce high resolution predictions instead of the used one in~\cite{long2015fully} which used a single bilinear interpolation layer." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1606.00915" id="2675">
-        <attvalues>
-          <attvalue for="5" value=" A different approach~\cite{chen2018deeplab} employed dilated convolution to increase the receptive field without any increase the in number of parameters and computational cost, followed by bilinear interpolation layers to scale up the feature maps to the input image size." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1210.5644" id="2676">
-        <attvalues>
-          <attvalue for="5" value=" Then a conditional random field (CRF)~\cite{krahenbuhl2011efficient} was used as a post processing to refine the result image." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1803.02758" id="2677">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1801.04381" id="2678">
-        <attvalues>
-          <attvalue for="5" value=" \\ In summary, our main contributions are: \begin{itemize} \item LiteSeg, a real time competitive architecture is presented and tested with three different backbone networks, Darknet19~\cite{yolo}, MobileNetV2~\cite{mobilenetv2}, and ShuffleNet~\cite{shufflenet}, achieving performance 70." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1707.01083" id="2679">
-        <attvalues>
-          <attvalue for="5" value=" \\ In summary, our main contributions are: \begin{itemize} \item LiteSeg, a real time competitive architecture is presented and tested with three different backbone networks, Darknet19~\cite{yolo}, MobileNetV2~\cite{mobilenetv2}, and ShuffleNet~\cite{shufflenet}, achieving performance 70." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06683" target="1511.07122" id="2680">
-        <attvalues>
-          <attvalue for="5" value=" RTSeg~\cite{rtseg} provided a decoupled encoder-decoder architecture which allows to plug any encoder (eg, VGG16~\cite{simonyan2014very}, MobileNet~\cite{mobilenetv2}, ShuffleNet~\cite{shufflenet}, ResNet18~\cite{resnet}) or decoder (eg, UNet~\cite{ronneberger2015u}, Dilation~\cite{yu2015multi}, SkipNet~\cite{long2015fully}) architectures independently." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1707.05388" id="2681">
-        <attvalues>
-          <attvalue for="5" value=" The problem of multi-person pose estimation is to localize the 2D skeleton keypoints (body joints) for all persons given a single image \cite{Ronchi2017Benchmarking, Papandreou2018PersonLab, li2020simple}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1603.06937" id="2682">
-        <attvalues>
-          <attvalue for="5" value=" For instance, Associative Embedding employs Hourglass Networks \cite{Newell2016Stacked} to infer the identity tags for all candidate keypoints." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1902.09212" id="2683">
-        <attvalues>
-          <attvalue for="5" value=" Some recent work can address the problem of single-person pose estimation extremely well thanks to the development of convolutional neural networks (CNNs) specially designed for human pose estimation such as Hourglass Network \cite{Newell2016Stacked, law2018cornernet, li2020simple} and HRNet \cite{sun2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1910.06278" id="2684">
-        <attvalues>
-          <attvalue for="5" value=" Some recent top-down approaches \cite{zhang2020distribution, Chen2017Cascaded, Papandreou2017Towards, sun2019deep, He2017Mask} have achieved high accuracy." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1611.08050" id="2685">
-        <attvalues>
-          <attvalue for="5" value=" By contrast, Part Affinity Fields \cite{Cao2017Realtime}, PersonLab \cite{Papandreou2018PersonLab} and PIFPAF \cite{kreiss2019pifpaf} are greedy grouping approaches, associating adjacent nodes in a human skeleton tree independently rather than finding the global solution of a global graph matching problem." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1312.4659" id="2686">
-        <attvalues>
-          <attvalue for="5" value=" With the arrival of the deep learning era, DeepPose \cite{Toshev2013DeepPose} for the first time uses CNNs to regress the Cartesian coordinates of a fixed number of person keypoints directly." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1803.08225" id="2687">
-        <attvalues>
-          <attvalue for="5" value=" By contrast, Part Affinity Fields \cite{Cao2017Realtime}, PersonLab \cite{Papandreou2018PersonLab} and PIFPAF \cite{kreiss2019pifpaf} are greedy grouping approaches, associating adjacent nodes in a human skeleton tree independently rather than finding the global solution of a global graph matching problem." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1904.07850" id="2688">
-        <attvalues>
-          <attvalue for="5" value=" We use Hourglass-104 \cite{law2018cornernet} as the inference model and select CenterNet \cite{Zhou:2019ta} as the baseline approach." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1903.06593" id="2689">
-        <attvalues>
-          <attvalue for="5" value=" By contrast, Part Affinity Fields \cite{Cao2017Realtime}, PersonLab \cite{Papandreou2018PersonLab} and PIFPAF \cite{kreiss2019pifpaf} are greedy grouping approaches, associating adjacent nodes in a human skeleton tree independently rather than finding the global solution of a global graph matching problem." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1708.02002" id="2690">
-        <attvalues>
-          <attvalue for="5" value=" However, special techniques are essential to make their approaches work well, eg, leveraging focal loss \cite{Lin2017Focal} for the classification task and Laplace-based L1 loss \cite{kendall2017uncertainties} for the offset regression task \cite{kreiss2019pifpaf}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1703.04977" id="2691">
-        <attvalues>
-          <attvalue for="5" value=" However, special techniques are essential to make their approaches work well, eg, leveraging focal loss \cite{Lin2017Focal} for the classification task and Laplace-based L1 loss \cite{kendall2017uncertainties} for the offset regression task \cite{kreiss2019pifpaf}." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1406.2984" id="2692">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Tompson2014Joint} and many later researchers use CNNs to predict the Gaussian response heatmaps of person keypoints and then obtain the keypoint coordinates by finding the local maximum responses in the heatmaps." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1605.03170" id="2693">
-        <attvalues>
-          <attvalue for="5" value=" Prior work, such as DeeperCut \cite{Insafutdinov2016DeeperCut}, Associative Embedding \cite{Newell2017Associative} and CenterNet \cite{Zhou:2019ta}, presents different approaches to encode the global keypoint grouping information." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1611.05424" id="2694">
-        <attvalues>
-          <attvalue for="5" value=" Prior work, such as DeeperCut \cite{Insafutdinov2016DeeperCut}, Associative Embedding \cite{Newell2017Associative} and CenterNet \cite{Zhou:2019ta}, presents different approaches to encode the global keypoint grouping information." />
-        </attvalues>
-      </edge>
-      <edge source="2107.03098" target="1808.01244" id="2695">
-        <attvalues>
-          <attvalue for="5" value=" We use Hourglass-104 \cite{law2018cornernet} as the inference model and select CenterNet \cite{Zhou:2019ta} as the baseline approach." />
-        </attvalues>
-      </edge>
-      <edge source="2112.14931" target="2003.01060" id="2696">
-        <attvalues>
-          <attvalue for="5" value=" It has been a major topic of research in computer vision for the past few decades due to the significant role it plays in various machine vision applications, including scene reconstruction~\cite{da2019dense, kim20133d, ma20153d, pathak20163d, pathak2016dense, schonbein2014omnidirectional, Yang_2019_CVPR, Fernandez_RAL_2020, Fangzheng_CSVT_2021}, view synthesis~\cite{lie2017key, jiang2017light}, navigation~\cite{yang2020d3vo, zhan2020visual, xue2019beyond}, and tracking~\cite{carrio2018drone, kart2019object}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.14931" target="1808.00259" id="2697">
-        <attvalues>
-          <attvalue for="5" value=" It has been a major topic of research in computer vision for the past few decades due to the significant role it plays in various machine vision applications, including scene reconstruction~\cite{da2019dense, kim20133d, ma20153d, pathak20163d, pathak2016dense, schonbein2014omnidirectional, Yang_2019_CVPR, Fernandez_RAL_2020, Fangzheng_CSVT_2021}, view synthesis~\cite{lie2017key, jiang2017light}, navigation~\cite{yang2020d3vo, zhan2020visual, xue2019beyond}, and tracking~\cite{carrio2018drone, kart2019object}." />
-        </attvalues>
-      </edge>
-      <edge source="2112.14931" target="1803.08669" id="2698">
-        <attvalues>
-          <attvalue for="5" value=" Owing to the development of deep learning frameworks in stereo depth estimation~\cite{chang2018pyramid, kendall2017end, Zhang_2019_CVPR}, the recent work in \cite{wang2020360sd} employed a convolutional neural network framework based on two camera configurations in a top-bottom manner~\cite{li2008binocular, kim20133d}, showing satisfactory results." />
-        </attvalues>
-      </edge>
-      <edge source="2112.14931" target="1909.08112" id="2699">
-        <attvalues>
-          <attvalue for="5" value=" ~\cite{zioulis2019spherical} designed a self-supervised learning framework that synthesis three views using estimated disparity maps from the convolutional neural network." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2406.13007" id="2700">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.10343" id="2701">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.11159" id="2702">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.16687" id="2703">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2405.09923" id="2704">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.16223" id="2705">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.11313" id="2706">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2405.09923" target="2404.14248" id="2707">
-        <attvalues>
-          <attvalue for="5" value=" \end{itemize} This challenge is one of the NTIRE 2024 Workshop associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2402.07220" id="2708">
-        <attvalues>
-          <attvalue for="5" value=" This NTIRE 2024 Short-form UGC Video Quality Assessment Challenge is held to promote related research on S-UGC VQA, and develop the powerful S-UGC VQA benchmark to assist the technique evolution of S-UGC video creation, compression, or processing, etc In this competition, we utilize the KVQ dataset~\cite{lu2024kvq} collected from the Kwai platform to evaluate the submitted methods." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2308.00729" id="2709">
-        <attvalues>
-          <attvalue for="5" value=" Apart from that, there are also some works exploring the potential of large cross-modality foundation models~\cite{ADAQA,maxwell,wu2023qQ-Align,CLIP} for VQA." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2206.09853" id="2710">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2401.08522" id="2711">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2309.17014" id="2712">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2104.14115" id="2713">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2207.14489" id="2714">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2307.16813" id="2715">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2304.06440" id="2716">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="1512.03385" id="2717">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2207.08124" id="2718">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="1908.00375" id="2719">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2106.13230" id="2720">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2205.04264" id="2721">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2207.06177" id="2722">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2204.14047" id="2723">
-        <attvalues>
-          <attvalue for="5" value=" Recently, amounts of studies~\cite{ADAQA, BVQA, GSTVQA, VSFA, starVQA,discovqa, Dover,lu2024AIGC-VQA,yu2024video,li2023freqalign,liu2022liqa,lu2022styleam,DBLP:conf/mm/YuanKZSW23,DBLP:conf/mm/LiuWYSTZWL23,DBLP:conf/cvpr/ZhaoYSW23,DBLP:conf/cvpr/ZhaoYSLW23} have been proposed to assess the perceptual quality of long UGC videos or images by excavating the capability of pre-trained backbones, \ieno, ResNet~\cite{resnet,liu2022sourceBIQA,VSFA}, and Transformer~\cite{transformer,3dswin,liu2022swiniqa,lu2022rtn,simpleVQA,wu2022fastFastVQA,Yu2024ntire}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2207.02595" id="2724">
-        <attvalues>
-          <attvalue for="5" value=" Since the scarcity of the VQA database, some well-designed sampling strategies~\cite{wu2022fastFastVQA,Dover}, \egno, fragment sampling, are introduced to reduce the computational complexity of the VQA method and enhance the diversity of VQA database." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2305.12726" id="2725">
-        <attvalues>
-          <attvalue for="5" value=" Apart from that, there are also some works exploring the potential of large cross-modality foundation models~\cite{ADAQA,maxwell,wu2023qQ-Align,CLIP} for VQA." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2312.17090" id="2726">
-        <attvalues>
-          <attvalue for="5" value=" Apart from that, there are also some works exploring the potential of large cross-modality foundation models~\cite{ADAQA,maxwell,wu2023qQ-Align,CLIP} for VQA." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2103.00020" id="2727">
-        <attvalues>
-          <attvalue for="5" value=" Apart from that, there are also some works exploring the potential of large cross-modality foundation models~\cite{ADAQA,maxwell,wu2023qQ-Align,CLIP} for VQA." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="1904.06457" id="2728">
-        <attvalues>
-          <attvalue for="5" value=" However, the above works are only designed and validated on general long UGC VQA databases, \eg, LSVQ~\cite{ying2021patchLSVQ}, YouTube-UGC~\cite{youtubeUGC}, and KoNViD-1k~\cite{KoNViD-1k}, lacking the perception capability for short-form UGC VQA." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2406.13007" id="2729">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.10343" id="2730">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.11159" id="2731">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.16687" id="2732">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2405.09923" id="2733">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.16223" id="2734">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.11313" id="2735">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2404.11313" target="2404.14248" id="2736">
-        <attvalues>
-          <attvalue for="5" value=" This challenge is one of the NTIRE 2024 Workshop~ associated challenges on: dense and non-homogeneous dehazing~\cite{ntire2024dehazing}, night photography rendering~\cite{ntire2024night}, blind compressed image enhancement~\cite{ntire2024compressed}, shadow removal~\cite{ntire2024shadow}, efficient super resolution~\cite{ntire2024efficientsr}, image super resolution ($\times$4)~\cite{ntire2024srx4}, light field image super-resolution~\cite{ntire2024lightfield}, stereo image super-resolution~\cite{ntire2024stereosr}, HR depth from images of specular and transparent surfaces~\cite{ntire2024depth}, bracketing image restoration and enhancement~\cite{ntire2024bracketing}, portrait quality assessment~\cite{ntire2024QA_portrait}, quality assessment for AI-generated content~\cite{ntire2024QA_AI}, restore any image model (RAIM) in the wild~\cite{ntire2024raim}, RAW image super-resolution~\cite{ntire2024rawsr}, short-form UGC video quality assessment~\cite{ntire2024QA_UGC}, low light enhancement~\cite{ntire2024lowlight}, and RAW burst alignment and ISP challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2012.15779" target="1507.00410" id="2737">
-        <attvalues>
-          <attvalue for="5" value=" Over time, the best accuracy used to be achieved by learning-based methods such as Bayesian learning~\cite{gehler2008bayesian}, spatio-spectral learning~\cite{chakrabarti2012color}, illumination solution space restriction~\cite{banic2015color,banic2015using,banic2015acolor}, using color moments~\cite{finlayson2013corrected}, regression trees with simple features from color distribution statistics~\cite{cheng2015effective}, spatial localizations~\cite{barron2015convolutional,barron2017fast}, convolutional neural networks~\cite{bianco2015color,shi2016deep,hu2017fc4,oh2017approaching}, genetic algorithms~\cite{koscevic2019color}, etc Thousands of other scientific papers about the problem of illumination estimation have been written, but it is still not solved to a satisfactory level due to the following reasons: \begin{itemize} \item Volumes of publicly available scientific datasets are not large enough to cover all various cases of illumination." />
-        </attvalues>
-      </edge>
-      <edge source="2012.15779" target="1611.07596" id="2738">
-        <attvalues>
-          <attvalue for="5" value=" Over time, the best accuracy used to be achieved by learning-based methods such as Bayesian learning~\cite{gehler2008bayesian}, spatio-spectral learning~\cite{chakrabarti2012color}, illumination solution space restriction~\cite{banic2015color,banic2015using,banic2015acolor}, using color moments~\cite{finlayson2013corrected}, regression trees with simple features from color distribution statistics~\cite{cheng2015effective}, spatial localizations~\cite{barron2015convolutional,barron2017fast}, convolutional neural networks~\cite{bianco2015color,shi2016deep,hu2017fc4,oh2017approaching}, genetic algorithms~\cite{koscevic2019color}, etc Thousands of other scientific papers about the problem of illumination estimation have been written, but it is still not solved to a satisfactory level due to the following reasons: \begin{itemize} \item Volumes of publicly available scientific datasets are not large enough to cover all various cases of illumination." />
-        </attvalues>
-      </edge>
-      <edge source="2012.15779" target="1504.04548" id="2739">
-        <attvalues>
-          <attvalue for="5" value=" Over time, the best accuracy used to be achieved by learning-based methods such as Bayesian learning~\cite{gehler2008bayesian}, spatio-spectral learning~\cite{chakrabarti2012color}, illumination solution space restriction~\cite{banic2015color,banic2015using,banic2015acolor}, using color moments~\cite{finlayson2013corrected}, regression trees with simple features from color distribution statistics~\cite{cheng2015effective}, spatial localizations~\cite{barron2015convolutional,barron2017fast}, convolutional neural networks~\cite{bianco2015color,shi2016deep,hu2017fc4,oh2017approaching}, genetic algorithms~\cite{koscevic2019color}, etc Thousands of other scientific papers about the problem of illumination estimation have been written, but it is still not solved to a satisfactory level due to the following reasons: \begin{itemize} \item Volumes of publicly available scientific datasets are not large enough to cover all various cases of illumination." />
-        </attvalues>
-      </edge>
-      <edge source="2012.15779" target="1608.07951" id="2740">
-        <attvalues>
-          <attvalue for="5" value=" Over time, the best accuracy used to be achieved by learning-based methods such as Bayesian learning~\cite{gehler2008bayesian}, spatio-spectral learning~\cite{chakrabarti2012color}, illumination solution space restriction~\cite{banic2015color,banic2015using,banic2015acolor}, using color moments~\cite{finlayson2013corrected}, regression trees with simple features from color distribution statistics~\cite{cheng2015effective}, spatial localizations~\cite{barron2015convolutional,barron2017fast}, convolutional neural networks~\cite{bianco2015color,shi2016deep,hu2017fc4,oh2017approaching}, genetic algorithms~\cite{koscevic2019color}, etc Thousands of other scientific papers about the problem of illumination estimation have been written, but it is still not solved to a satisfactory level due to the following reasons: \begin{itemize} \item Volumes of publicly available scientific datasets are not large enough to cover all various cases of illumination." />
-        </attvalues>
-      </edge>
-      <edge source="2007.02419" target="1804.02767" id="2741">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there have been major developments in this domain (\cite{yolov3}, \cite{ren2015faster}, \cite{he2017mask}) but the diversity of this problem poses new challenges motivating active research in this direction." />
-        </attvalues>
-      </edge>
-      <edge source="2007.02419" target="1506.01497" id="2742">
-        <attvalues>
-          <attvalue for="5" value=" Recently, there have been major developments in this domain (\cite{yolov3}, \cite{ren2015faster}, \cite{he2017mask}) but the diversity of this problem poses new challenges motivating active research in this direction." />
-        </attvalues>
-      </edge>
-      <edge source="2007.02419" target="1706.03762" id="2743">
-        <attvalues>
-          <attvalue for="5" value=" Recently, self-attention based transformer architectures \cite{vaswani2017attention} have gained popularity which eliminated the use of Recurrent Model like RNN for modeling contextual information." />
-        </attvalues>
-      </edge>
-      <edge source="2007.02419" target="1406.2031" id="2744">
-        <attvalues>
-          <attvalue for="5" value=" We use the PASCAL-Part 2010 Dataset (\cite{chen2014detect}) that contains segmentation masks for object and its various semantic parts." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="1705.01583" id="2745">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, 3D Human Pose Estimation (3DHPE) has garnered widespread attention due to its significant applications in fields such as action recognition\cite{liu2018recognizing}, virtual reality\cite{mehta2017vnect}, and human-computer interaction\cite{preece1994human}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="1603.06937" id="2746">
-        <attvalues>
-          <attvalue for="5" value=" Substantial progress has been achieved in 3D pose estimation from 2D pose inputs, facilitated by powerful 2D pose estimators\cite{newell2016stacked,chen2018cascaded,sun2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="1711.07319" id="2747">
-        <attvalues>
-          <attvalue for="5" value=" Substantial progress has been achieved in 3D pose estimation from 2D pose inputs, facilitated by powerful 2D pose estimators\cite{newell2016stacked,chen2018cascaded,sun2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="1902.09212" id="2748">
-        <attvalues>
-          <attvalue for="5" value=" Substantial progress has been achieved in 3D pose estimation from 2D pose inputs, facilitated by powerful 2D pose estimators\cite{newell2016stacked,chen2018cascaded,sun2019deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="1811.11742" id="2749">
-        <attvalues>
-          <attvalue for="5" value=" A considerable body of work has focused on deterministic models\cite{pavllo20193d,zou2021modulated,zhang2022mixste,zhao2023poseformerv2,cai2023htnet,kang2023double} aimed at inferring the most likely pose directly from given input data." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2203.00859" id="2750">
-        <attvalues>
-          <attvalue for="5" value=" A considerable body of work has focused on deterministic models\cite{pavllo20193d,zou2021modulated,zhang2022mixste,zhao2023poseformerv2,cai2023htnet,kang2023double} aimed at inferring the most likely pose directly from given input data." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2303.17472" id="2751">
-        <attvalues>
-          <attvalue for="5" value=" A considerable body of work has focused on deterministic models\cite{pavllo20193d,zou2021modulated,zhang2022mixste,zhao2023poseformerv2,cai2023htnet,kang2023double} aimed at inferring the most likely pose directly from given input data." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2302.09790" id="2752">
-        <attvalues>
-          <attvalue for="5" value=" Consequently, we propose a diffusion-based\cite{ho2020denoising,song2020denoising} refinement framework, called DRPose, aimed at refining 3D poses generated by deterministic models\cite{cai2023htnet,kang2023double} to bring the average predictions of probabilistic models closer to the real distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2308.05298" id="2753">
-        <attvalues>
-          <attvalue for="5" value=" Consequently, we propose a diffusion-based\cite{ho2020denoising,song2020denoising} refinement framework, called DRPose, aimed at refining 3D poses generated by deterministic models\cite{cai2023htnet,kang2023double} to bring the average predictions of probabilistic models closer to the real distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2107.13788" id="2754">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue, probabilistic models\cite{wehrbein2021probabilistic,oikarinen2021graphmdn,li2022mhformer,ci2023gfpose} have been introduced, aiming to capture the uncertainty in pose estimation by generating multiple pose hypotheses." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2010.13668" id="2755">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue, probabilistic models\cite{wehrbein2021probabilistic,oikarinen2021graphmdn,li2022mhformer,ci2023gfpose} have been introduced, aiming to capture the uncertainty in pose estimation by generating multiple pose hypotheses." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2111.12707" id="2756">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue, probabilistic models\cite{wehrbein2021probabilistic,oikarinen2021graphmdn,li2022mhformer,ci2023gfpose} have been introduced, aiming to capture the uncertainty in pose estimation by generating multiple pose hypotheses." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2212.08641" id="2757">
-        <attvalues>
-          <attvalue for="5" value=" To address this issue, probabilistic models\cite{wehrbein2021probabilistic,oikarinen2021graphmdn,li2022mhformer,ci2023gfpose} have been introduced, aiming to capture the uncertainty in pose estimation by generating multiple pose hypotheses." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2006.11239" id="2758">
-        <attvalues>
-          <attvalue for="5" value=" Consequently, we propose a diffusion-based\cite{ho2020denoising,song2020denoising} refinement framework, called DRPose, aimed at refining 3D poses generated by deterministic models\cite{cai2023htnet,kang2023double} to bring the average predictions of probabilistic models closer to the real distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2401.04921" target="2010.02502" id="2759">
-        <attvalues>
-          <attvalue for="5" value=" Consequently, we propose a diffusion-based\cite{ho2020denoising,song2020denoising} refinement framework, called DRPose, aimed at refining 3D poses generated by deterministic models\cite{cai2023htnet,kang2023double} to bring the average predictions of probabilistic models closer to the real distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2002.05709" id="2760">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="1911.05722" id="2761">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2110.15348" id="2762">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2303.06562" id="2763">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2306.04272" id="2764">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2310.18904" id="2765">
-        <attvalues>
-          <attvalue for="5" value=" Contrastive learning has to be the state-of-the-art method for self-supervised representation learning across many domains \cite{simclr,moco,wang2021residual,guo2023contranorm,zhang2023on,zhang2023tricontrastive}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2104.02057" id="2766">
-        <attvalues>
-          <attvalue for="5" value=" However, there still remains a noticeable gap in performance compared to its supervised counterparts \cite{mocov3,cui2023rethinking}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2306.04160" id="2767">
-        <attvalues>
-          <attvalue for="5" value=" However, there still remains a noticeable gap in performance compared to its supervised counterparts \cite{mocov3,cui2023rethinking}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2202.06464" id="2768">
-        <attvalues>
-          <attvalue for="5" value=" Among many attempts to close this gap, a recent surge of interest lies in leveraging high-quality generative models to boost contrastive learning \cite{wu2022synthetic,wang2022unified,azizi2023synthetic,tian2023stablerep}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2203.13455" id="2769">
-        <attvalues>
-          <attvalue for="5" value=" Among many attempts to close this gap, a recent surge of interest lies in leveraging high-quality generative models to boost contrastive learning \cite{wu2022synthetic,wang2022unified,azizi2023synthetic,tian2023stablerep}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2304.08466" id="2770">
-        <attvalues>
-          <attvalue for="5" value=" Among many attempts to close this gap, a recent surge of interest lies in leveraging high-quality generative models to boost contrastive learning \cite{wu2022synthetic,wang2022unified,azizi2023synthetic,tian2023stablerep}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2306.00984" id="2771">
-        <attvalues>
-          <attvalue for="5" value=" Among many attempts to close this gap, a recent surge of interest lies in leveraging high-quality generative models to boost contrastive learning \cite{wu2022synthetic,wang2022unified,azizi2023synthetic,tian2023stablerep}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2006.11239" id="2772">
-        <attvalues>
-          <attvalue for="5" value=" Given an unlabeled dataset, \eg CIFAR-10, one can train a generative model on it (\eg DDPM \cite{ho2020denoising}) to generate a lot of synthetic samples, and then perform contrastive learning on the combination of the real and generated data." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2005.10243" id="2773">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, we discover an intriguing phenomenon that weaker data augmentation, although harmful in standard contrastive learning \cite{nips/Tian0PKSI20,luo2023rethinking}, can be very helpful with data inflation." />
-        </attvalues>
-      </edge>
-      <edge source="2403.12448" target="2303.01289" id="2774">
-        <attvalues>
-          <attvalue for="5" value=" For the latter, we discover an intriguing phenomenon that weaker data augmentation, although harmful in standard contrastive learning \cite{nips/Tian0PKSI20,luo2023rethinking}, can be very helpful with data inflation." />
-        </attvalues>
-      </edge>
-      <edge source="2201.10766" target="1512.03385" id="2775">
-        <attvalues>
-          <attvalue for="5" value=" Large scale benchmark datasets like ImageNet\cite{deng2009imagenet} that were constructed with single class label annotation have propelled rapid advances in the image classification task\cite{resnets, huang2017densely, tan2019efficientnet, zhai2021scaling}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.10766" target="1409.1556" id="2776">
-        <attvalues>
-          <attvalue for="5" value=" Over the last decade, several network architectures and training procedures were proposed to yield very high classification accuracies \cite{simonyan2014very, resnets, tan2019efficientnet, vits}." />
-        </attvalues>
-      </edge>
-      <edge source="2201.10766" target="1512.04150" id="2777">
-        <attvalues>
-          <attvalue for="5" value=" One of the most popular class of approaches are saliency methods\cite{gradcam,cam,smoothgrad, integrated-grads} that use model gradients to produce a saliency map corresponding to the most influential input regions that yielded the resulting prediction." />
-        </attvalues>
-      </edge>
-      <edge source="2201.10766" target="1706.03825" id="2778">
-        <attvalues>
-          <attvalue for="5" value=" One of the most popular class of approaches are saliency methods\cite{gradcam,cam,smoothgrad, integrated-grads} that use model gradients to produce a saliency map corresponding to the most influential input regions that yielded the resulting prediction." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2107.04782" id="2779">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{few-shot-action-recognition}, the action recognition problem is considered as two separate problems, action duration misalignment, and action evolution misalignment." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2105.04823" id="2780">
-        <attvalues>
-          <attvalue for="5" value=" Video Classification \cite{few-shot-classification, few-shot-classiciation2} and action detection \cite{action-detection, action-detection2, action-detection3} are other examples of the applications of video alignment that have received attention in recent years." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1612.00558" id="2781">
-        <attvalues>
-          <attvalue for="5" value=" Video Classification \cite{few-shot-classification, few-shot-classiciation2} and action detection \cite{action-detection, action-detection2, action-detection3} are other examples of the applications of video alignment that have received attention in recent years." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1904.07846" id="2782">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1901.02598" id="2783">
-        <attvalues>
-          <attvalue for="5" value=" In the field of video alignment, self-supervised \cite{TCC, aligning_in_space_and_time, learning-by-aligning-videos-in-time, learning-to-align-sequential-actions, ctw} and weakly-supervised methods \cite{D3TW, representation-learning-via-global-temporal-alignment, weakly3, weakly4} have been presented in recent years." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1906.11415" id="2784">
-        <attvalues>
-          <attvalue for="5" value=" Since DTW is not derivable and cannot be implemented using neural networks, in these works, some modified types of DTW, such as soft-dtw, which are derivable, have been used \cite{few-shot-classiciation2, D3TW }." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2007.04515" id="2785">
-        <attvalues>
-          <attvalue for="5" value=" Unlike \cite{TCC}, \cite{aligning_in_space_and_time} deals with correspondence learning in both time and space domains based on cross-cycle stability." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2103.17260" id="2786">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{learning-by-aligning-videos-in-time}, the network is trained based on a cost function including two terms, soft-dtw and temporal regularization." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2105.05217" id="2787">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="2111.09301" id="2788">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{learning-to-align-sequential-actions}, another look at the subject of video alignment is given." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1603.08561" id="2789">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1704.06888" id="2790">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2304.06841" target="1212.0402" id="2791">
-        <attvalues>
-          <attvalue for="5" value=" To evaluate the effectiveness of the proposed features, we compared their performance with existing self-supervised \cite{SAL,TCN,TCC} and weakly supervised \cite{representation-learning-via-global-temporal-alignment} methods in phase classification tasks on the PennAction \cite{penn} and a subset of the UCF101 \cite{UCF}, known as UCF9, datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2204.02311" id="2792">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, large language models (LLMs)~\cite{chowdhery2023palm,touvron2023llama,zhang2022opt} have sparked a revolution in natural language processing (NLP)." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2302.13971" id="2793">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, large language models (LLMs)~\cite{chowdhery2023palm,touvron2023llama,zhang2022opt} have sparked a revolution in natural language processing (NLP)." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2205.01068" id="2794">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, large language models (LLMs)~\cite{chowdhery2023palm,touvron2023llama,zhang2022opt} have sparked a revolution in natural language processing (NLP)." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2103.00020" id="2795">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of LLMs, vision foundational models such as CLIP~\cite{DBLP:conf/icml/RadfordKHRGASAM21}, DINOv2~\cite{DBLP:journals/corr/abs-2304-07193}, BLIP~\cite{DBLP:conf/icml/0001LXH22}, and SAM~\cite{kirillov2023segment} have also emerged." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2304.07193" id="2796">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of LLMs, vision foundational models such as CLIP~\cite{DBLP:conf/icml/RadfordKHRGASAM21}, DINOv2~\cite{DBLP:journals/corr/abs-2304-07193}, BLIP~\cite{DBLP:conf/icml/0001LXH22}, and SAM~\cite{kirillov2023segment} have also emerged." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2201.12086" id="2797">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of LLMs, vision foundational models such as CLIP~\cite{DBLP:conf/icml/RadfordKHRGASAM21}, DINOv2~\cite{DBLP:journals/corr/abs-2304-07193}, BLIP~\cite{DBLP:conf/icml/0001LXH22}, and SAM~\cite{kirillov2023segment} have also emerged." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2304.02643" id="2798">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of LLMs, vision foundational models such as CLIP~\cite{DBLP:conf/icml/RadfordKHRGASAM21}, DINOv2~\cite{DBLP:journals/corr/abs-2304-07193}, BLIP~\cite{DBLP:conf/icml/0001LXH22}, and SAM~\cite{kirillov2023segment} have also emerged." />
-        </attvalues>
-      </edge>
-      <edge source="2407.21596" target="2408.00714" id="2799">
-        <attvalues>
-          <attvalue for="5" value=" To further extend SAM’s applicability, SAM2~\cite{ravi2024sam2} is introduced, designed to handle both image and video segmentation tasks within a unified architecture." />
-        </attvalues>
-      </edge>
-      <edge source="2111.07529" target="1703.06870" id="2800">
-        <attvalues>
-          <attvalue for="5" value=" In this work, we propose a video instance segmentation algorithm based on the Mask R-CNN pipeline~\cite{he2017Mask}." />
-        </attvalues>
-      </edge>
-      <edge source="2111.07529" target="1905.04804" id="2801">
-        <attvalues>
-          <attvalue for="5" value=" Third, our method outperforms the conventional Mask R-CNN methods on the on the YouTube-VIS dataset~\cite{yang2019Video}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="2001.05566" id="2802">
-        <attvalues>
-          <attvalue for="5" value=" It has been widely studied in all kinds of applications \cite{minaee2021image} since 1965." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1505.04597" id="2803">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, with the development of the deep learning in medical image analysis \cite{wang2017detecting,esteva2017dermatologist,wang2018context,wang2020simultaneous}, deep neural networks (DNNs) have been used to successfully tackle a variety of image segmentation problems in a fully-supervised manner \cite{ronneberger2015u,long2015fully,chen2018encoder}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1605.06211" id="2804">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, with the development of the deep learning in medical image analysis \cite{wang2017detecting,esteva2017dermatologist,wang2018context,wang2020simultaneous}, deep neural networks (DNNs) have been used to successfully tackle a variety of image segmentation problems in a fully-supervised manner \cite{ronneberger2015u,long2015fully,chen2018encoder}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1802.02611" id="2805">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, with the development of the deep learning in medical image analysis \cite{wang2017detecting,esteva2017dermatologist,wang2018context,wang2020simultaneous}, deep neural networks (DNNs) have been used to successfully tackle a variety of image segmentation problems in a fully-supervised manner \cite{ronneberger2015u,long2015fully,chen2018encoder}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1904.05044" id="2806">
-        <attvalues>
-          <attvalue for="5" value=" Several types of annotations have been investigated, including image labels \cite{ahn2019weakly,wang2020self}, points \cite{bearman2016s}, scribbles \cite{lin2016scribblesup}, and bounding boxes \cite{rajchl2016deepcut,kervadec2020bounding}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="2004.04581" id="2807">
-        <attvalues>
-          <attvalue for="5" value=" Several types of annotations have been investigated, including image labels \cite{ahn2019weakly,wang2020self}, points \cite{bearman2016s}, scribbles \cite{lin2016scribblesup}, and bounding boxes \cite{rajchl2016deepcut,kervadec2020bounding}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1604.05144" id="2808">
-        <attvalues>
-          <attvalue for="5" value=" Several types of annotations have been investigated, including image labels \cite{ahn2019weakly,wang2020self}, points \cite{bearman2016s}, scribbles \cite{lin2016scribblesup}, and bounding boxes \cite{rajchl2016deepcut,kervadec2020bounding}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="1605.07866" id="2809">
-        <attvalues>
-          <attvalue for="5" value=" \cite{rajchl2016deepcut} designed an iterative optimization approach for image segmentation, in which a neural network classifier was trained using bounding box supervision." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="2004.06816" id="2810">
-        <attvalues>
-          <attvalue for="5" value=" However, most methods in the literature, if not all, achieve high segmentation performance using tight bounding box supervision, and decrease the performance greatly when supervised by loose bounding boxes \cite{kervadec2020bounding}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="2110.00934" id="2811">
-        <attvalues>
-          <attvalue for="5" value=" In the end, the segmentation assistance is conducted by combining the loss derived from the polar transformation based MIL into the loss in \cite{wang2021bounding}." />
-        </attvalues>
-      </edge>
-      <edge source="2301.12053" target="2110.00943" id="2812">
-        <attvalues>
-          <attvalue for="5" value=" Recently, in our previous study we developed a WSIS approach using tight bounding box supervision by exploiting the properties of tight bounding boxes for MIL formulation \cite{wang2021bounding} and achieved state-of-the-art segmentation performance \cite{wang2021bounding,wang2022cdrnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1607.00470" id="2813">
-        <attvalues>
-          <attvalue for="5" value=" For instance, monocular SLAM systems~\cite{taketomi2017visual,younes2017keyframe,saputra2018visual} rely on homographies when detecting pure rotational camera movements, planar scenes, and scenes with far objects." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1801.09005" id="2814">
-        <attvalues>
-          <attvalue for="5" value=" The generated images cover a larger field-of-view and are useful in various applications, \eg image-based localization~\cite{Arth2011}, SLAM~\cite{Lemaire2007,Ji2020}, autonomous driving~\cite{Wang2020}, sport broadcasting~\cite{chen2018two}, surveillance~\cite{yang2019panoramic}, and augmented and virtual reality~\cite{jethwa1998real,macquarrie2017cinematic}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="2003.01587" id="2815">
-        <attvalues>
-          <attvalue for="5" value=" First, similarly as in most algorithms working with pairs, feature points are detected and matched~\cite{SIFT2004,ORB2011,brown2007automatic,schoenberger2016vote,IMC2020}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1711.05971" id="2816">
-        <attvalues>
-          <attvalue for="5" value=" They are then often filtered by the widely-used second nearest neighbors (SNN) ratio~\cite{lowe1999object,SIFT2004} or by deep learned filtering methods~\cite{cne2018,acne2020,dfe2018,clnet2021}, to remove gross outliers and, therefore, improve the robust estimation procedure that follows." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="2101.00591" id="2817">
-        <attvalues>
-          <attvalue for="5" value=" They are then often filtered by the widely-used second nearest neighbors (SNN) ratio~\cite{lowe1999object,SIFT2004} or by deep learned filtering methods~\cite{cne2018,acne2020,dfe2018,clnet2021}, to remove gross outliers and, therefore, improve the robust estimation procedure that follows." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1706.00984" id="2818">
-        <attvalues>
-          <attvalue for="5" value=" Given that recent robust estimators, \eg \cite{barath2018graph}, report lower than $0." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1704.05939" id="2819">
-        <attvalues>
-          <attvalue for="5" value=" The HPatches dataset~\cite{hpatches2017} was introduced in form of local patches for benchmarking descriptors and metric learning methods, later extended with images and homographies." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1503.02619" id="2820">
-        <attvalues>
-          <attvalue for="5" value=" The homographies are derived from hand-labeled correspondences that stem from multiple local feature detectors paired with an affine view synthesis procedure~\cite{Mishkin2015MODS} and RootSIFT descriptor~\cite{RootSIFT2012} matching." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="2003.08505" id="2821">
-        <attvalues>
-          <attvalue for="5" value=" Otherwise, as it happens in other fields, the seemingly rapid progress might be an artifact of tuning the algorithms on the test data, or an artifact of the flawed evaluation procedure~\cite{musgrave2020metric,pGT2021,goyal2021revisiting}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="2109.00524" id="2822">
-        <attvalues>
-          <attvalue for="5" value=" Otherwise, as it happens in other fields, the seemingly rapid progress might be an artifact of tuning the algorithms on the test data, or an artifact of the flawed evaluation procedure~\cite{musgrave2020metric,pGT2021,goyal2021revisiting}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="2106.05304" id="2823">
-        <attvalues>
-          <attvalue for="5" value=" Otherwise, as it happens in other fields, the seemingly rapid progress might be an artifact of tuning the algorithms on the test data, or an artifact of the flawed evaluation procedure~\cite{musgrave2020metric,pGT2021,goyal2021revisiting}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.09997" target="1711.06704" id="2824">
-        <attvalues>
-          <attvalue for="5" value=" As the forth contribution, we demonstrate that the dataset can be used to evaluate the uncertainty of partially or fully affine covariant features detectors~\cite{SIFT2004,AffNet2018}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09564" target="1711.10105" id="2825">
-        <attvalues>
-          <attvalue for="5" value=" Following the matrix case, the nuclear norm of the unfolding matrices has been used to efficiently recover different types of tensor ranks, see \cite{song2019tensor} for a comprehensive review on this topic." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09564" target="1603.06038" id="2826">
-        <attvalues>
-          <attvalue for="5" value=" Estimating the unknown elements of an incomplete data tensor is known as {tensor completion} and has played important roles in many machine learning problems, eg image/video completion \cite{song2019tensor} and recommender systems \cite{frolov2017tensor}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.09564" target="0805.4471" id="2827">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, tensor completion has its basis from matrix completion \cite{candes2009exact}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.11395" target="1806.02279" id="2828">
-        <attvalues>
-          <attvalue for="5" value=" The commonly concerned generic shape priors contain star shape \cite{2019star-convex,2022star,Veksler2008StarSP}, convexity \cite{Gorelick2016convexity,luo2019convex}, connectivity \cite{2019connecvtivity}, constant width \cite{Parameteric2022}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.11395" target="1809.00254" id="2829">
-        <attvalues>
-          <attvalue for="5" value=" The commonly concerned generic shape priors contain star shape \cite{2019star-convex,2022star,Veksler2008StarSP}, convexity \cite{Gorelick2016convexity,luo2019convex}, connectivity \cite{2019connecvtivity}, constant width \cite{Parameteric2022}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.11395" target="1509.02122" id="2830">
-        <attvalues>
-          <attvalue for="5" value=" The minimization problems for the region methods are NP-hard and only approximation solutions can be obtained by penalization and linearization techniques because the associated problems are combination optimization issues \cite{Gorelick2016convexity,gorelick2017multi,royer2016convexity}." />
-        </attvalues>
-      </edge>
-      <edge source="2203.11395" target="2003.09600" id="2831">
-        <attvalues>
-          <attvalue for="5" value=" This method is generalized to multiple convex objects \cite{luo2019convex}, convex ring \cite{2022Luo}, 3D convex object \cite{2021Luo} characterization, and applied to compute convex hull of given sets with and without outliers \cite{li2021new}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1812.03085" id="2832">
-        <attvalues>
-          <attvalue for="5" value=" An updated comprehensive compendium and comparison of CC algorithm performances can be found in \cite{cheng2014illuminant,Lou2015DeepColor,das2018color,Yan2018MultipleIE}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1704.00090" id="2833">
-        <attvalues>
-          <attvalue for="5" value="\cite{gardner2017learning} introduced a method to convert low dynamic range (LDR) indoor images to high dynamic range (HDR) images, first they used a deep network to localize the light source in LDR image environmental map and then they used another network with these annoted LDR images to convert them to HDR image." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1611.06403" id="2834">
-        <attvalues>
-          <attvalue for="5" value=" \cite{hold2017deep} introduced a method to convert outdoor LDR images to HDR images." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1910.08812" id="2835">
-        <attvalues>
-          <attvalue for="5" value=" Later on, Geoffroy et al \cite{gardner2019deep} extended their previous idea for indoor lighting but replaced the environmental maps with light geometric and photometric properties." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1905.00824" id="2836">
-        <attvalues>
-          <attvalue for="5" value=" \cite{Sun2019SingleIP} introduced an encoder-decoder based network to relight a single portrait image, the encoder predicts the input image environmental map and an embedding for the scene information, while the decoder builds a new version of the image scene with the target environmental map and obtains a relighted portrait." />
-        </attvalues>
-      </edge>
-      <edge source="2009.08941" target="1910.08131" id="2837">
-        <attvalues>
-          <attvalue for="5" value=" \cite{murmann2019dataset} have introduced the Multi-illuminant dataset of $1000$ scenes each one acquired under $25$ different light position conditions, and they used a deep network to predict a right sided illuminated image from its corresponding left sided illuminated image." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1803.07441" id="2838">
-        <attvalues>
-          <attvalue for="5" value=" Earlier hand-crafted feature representation and classical machine learning based methods were utilized for face recognition and analysis \cite{saber1998frontal}, \cite{wang2002facial}, \cite{deniz2003face}, \cite{ahonen2004face}, \cite{he2005face}, \cite{deniz2011face}, \cite{ldop}, \cite{chakraborty2018centre}, \cite{fdlbp}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="2201.00511" id="2839">
-        <attvalues>
-          <attvalue for="5" value=" Earlier hand-crafted feature representation and classical machine learning based methods were utilized for face recognition and analysis \cite{saber1998frontal}, \cite{wang2002facial}, \cite{deniz2003face}, \cite{ahonen2004face}, \cite{he2005face}, \cite{deniz2011face}, \cite{ldop}, \cite{chakraborty2018centre}, \cite{fdlbp}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1709.06508" id="2840">
-        <attvalues>
-          <attvalue for="5" value=" Earlier hand-crafted feature representation and classical machine learning based methods were utilized for face recognition and analysis \cite{saber1998frontal}, \cite{wang2002facial}, \cite{deniz2003face}, \cite{ahonen2004face}, \cite{he2005face}, \cite{deniz2011face}, \cite{ldop}, \cite{chakraborty2018centre}, \cite{fdlbp}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1807.07987" id="2841">
-        <attvalues>
-          <attvalue for="5" value=" DNNs can learn the important features automatically from the image data in a hierarchical fashion at different levels of abstractness \cite{lecun2015deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1409.1556" id="2842">
-        <attvalues>
-          <attvalue for="5" value=" Since then several variants of CNNs have been proposed for different applications of computer vision and image processing \cite{vgg}, \cite{resnet}, \cite{fasterrcnn}, \cite{imageretrieval}, \cite{feng2022iris}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1512.03385" id="2843">
-        <attvalues>
-          <attvalue for="5" value=" Since then several variants of CNNs have been proposed for different applications of computer vision and image processing \cite{vgg}, \cite{resnet}, \cite{fasterrcnn}, \cite{imageretrieval}, \cite{feng2022iris}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1506.01497" id="2844">
-        <attvalues>
-          <attvalue for="5" value=" Since then several variants of CNNs have been proposed for different applications of computer vision and image processing \cite{vgg}, \cite{resnet}, \cite{fasterrcnn}, \cite{imageretrieval}, \cite{feng2022iris}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="2012.00641" id="2845">
-        <attvalues>
-          <attvalue for="5" value=" Since then several variants of CNNs have been proposed for different applications of computer vision and image processing \cite{vgg}, \cite{resnet}, \cite{fasterrcnn}, \cite{imageretrieval}, \cite{feng2022iris}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1409.0575" id="2846">
-        <attvalues>
-          <attvalue for="5" value=" The key success of deep learning models was observed due to the availability of powerful computational machines, such as Graphical Processing Units (GPUs) which can perform heavy parallel computation, and availability of large-scale datasets \cite{imagenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1710.08092" id="2847">
-        <attvalues>
-          <attvalue for="5" value=" The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1503.03832" id="2848">
-        <attvalues>
-          <attvalue for="5" value=" The recent advancements in face recognition research, with the help of deep learning methods, can be observed from the outstanding performance of deep learning models \cite{srivastava2019performance}, such as VGGFace \cite{vggface}, \cite{vggface2} and FaceNet \cite{facenet}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1704.08063" id="2849">
-        <attvalues>
-          <attvalue for="5" value=" So far, several deep learning models have been developed by the researchers for face recognition by introducing different CNNs, different objective functions, and different other strategies \cite{deepface}, \cite{sphereface}, \cite{arcface}, \cite{hmloss}, \cite{mishra2021multiscale}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1801.07698" id="2850">
-        <attvalues>
-          <attvalue for="5" value=" So far, several deep learning models have been developed by the researchers for face recognition by introducing different CNNs, different objective functions, and different other strategies \cite{deepface}, \cite{sphereface}, \cite{arcface}, \cite{hmloss}, \cite{mishra2021multiscale}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1908.09747" id="2851">
-        <attvalues>
-          <attvalue for="5" value=" So far, several deep learning models have been developed by the researchers for face recognition by introducing different CNNs, different objective functions, and different other strategies \cite{deepface}, \cite{sphereface}, \cite{arcface}, \cite{hmloss}, \cite{mishra2021multiscale}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1804.06655" id="2852">
-        <attvalues>
-          <attvalue for="5" value=" A detailed discussion on the advancements of deep learning based face recogntion models can be found in a recent survey published by Wang and Deng in \cite{wang2021deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1411.7923" id="2853">
-        <attvalues>
-          <attvalue for="5" value=" The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1611.01484" id="2854">
-        <attvalues>
-          <attvalue for="5" value=" The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1607.08221" id="2855">
-        <attvalues>
-          <attvalue for="5" value=" The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="2103.04098" id="2856">
-        <attvalues>
-          <attvalue for="5" value=" The popular face datasets include Labelled Faces in Wild \cite{lfw}, CASIA WebFace \cite{casiawebface}, VGGFace \cite{vggface}, VGGFace2 \cite{vggface2}, UMDFace \cite{umdface}, Microsoft Celeb (MS-Celeb-1M) \cite{msceleb} and WebFace \cite{webface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1810.01898" id="2857">
-        <attvalues>
-          <attvalue for="5" value=" Some other face datasets include multi-face dataset \cite{dubey2018multi}, YouTube Face Dataset \cite{wolf2011face}, CelebFaces Attributes Dataset (CelebA) \cite{liu2015faceattributes}, etc These existing datasets were generally developed by mining the images of celebrities from the public domain." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1411.7766" id="2858">
-        <attvalues>
-          <attvalue for="5" value=" Some other face datasets include multi-face dataset \cite{dubey2018multi}, YouTube Face Dataset \cite{wolf2011face}, CelebFaces Attributes Dataset (CelebA) \cite{liu2015faceattributes}, etc These existing datasets were generally developed by mining the images of celebrities from the public domain." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1805.02283" id="2859">
-        <attvalues>
-          <attvalue for="5" value=" The DocFace performs the matching of ID photos with live face photos/selfies \cite{shi2018docface}." />
-        </attvalues>
-      </edge>
-      <edge source="2302.07245" target="1809.05620" id="2860">
-        <attvalues>
-          <attvalue for="5" value=" The DocFace+ supports multiple ID/selfie per class for ID-Selfie matching \cite{shi2019docface+}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="2003.00387" id="2861">
-        <attvalues>
-          <attvalue for="5" value=" In addition, scene graphs are used for a wide range of vision applications, such as image retrieval~\cite{johnson2015image,qi2017online}, image captioning~\cite{chen2020say}, visual reasoning~\cite{shi2019explainable}, visual answering~\cite{hildebrandt2020scene}, and robotics~\cite{amiri2022reasoning}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="1812.01855" id="2862">
-        <attvalues>
-          <attvalue for="5" value=" In addition, scene graphs are used for a wide range of vision applications, such as image retrieval~\cite{johnson2015image,qi2017online}, image captioning~\cite{chen2020say}, visual reasoning~\cite{shi2019explainable}, visual answering~\cite{hildebrandt2020scene}, and robotics~\cite{amiri2022reasoning}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="2007.01072" id="2863">
-        <attvalues>
-          <attvalue for="5" value=" In addition, scene graphs are used for a wide range of vision applications, such as image retrieval~\cite{johnson2015image,qi2017online}, image captioning~\cite{chen2020say}, visual reasoning~\cite{shi2019explainable}, visual answering~\cite{hildebrandt2020scene}, and robotics~\cite{amiri2022reasoning}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="2202.10432" id="2864">
-        <attvalues>
-          <attvalue for="5" value=" In addition, scene graphs are used for a wide range of vision applications, such as image retrieval~\cite{johnson2015image,qi2017online}, image captioning~\cite{chen2020say}, visual reasoning~\cite{shi2019explainable}, visual answering~\cite{hildebrandt2020scene}, and robotics~\cite{amiri2022reasoning}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="1602.07332" id="2865">
-        <attvalues>
-          <attvalue for="5" value=" The visual genome dataset~\cite{krishna2017visual} is the most common and popular dataset for scene graphs, with around $10^5$ annotated images." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="2007.05756" id="2866">
-        <attvalues>
-          <attvalue for="5" value=" The automatic generation approaches can only generate limited scene graphs on commonly used object categories and predicates, and have difficulty for rare and previously unseen compositions~\cite{knyazev2021generative,knyazev2020graph}." />
-        </attvalues>
-      </edge>
-      <edge source="2211.16697" target="2005.08230" id="2867">
-        <attvalues>
-          <attvalue for="5" value=" The automatic generation approaches can only generate limited scene graphs on commonly used object categories and predicates, and have difficulty for rare and previously unseen compositions~\cite{knyazev2021generative,knyazev2020graph}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2003.08934" id="2868">
-        <attvalues>
-          <attvalue for="5" value=" Recent success of deep learning techniques in 2D domain has sparked a surge of interest in higher dimensional problems, such as 3D computer vision tasks and their application fields in VR/AR, movie production, games etc For instance, the neural radiance field (NeRF)~\cite{nerf} demonstrates that Multi-Layer Perceptron (MLP) neural networks can represent a scene by implicitly encoding the geometry and appearance into the parameter of networks." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2011.13084" id="2869">
-        <attvalues>
-          <attvalue for="5" value=" While NeRF was initially designed for static content, subsequent dynamic works either condition the neural field with addtional temporal input~\cite{nsff,xian2021space,nrf4}, or jointly optimize a deformation field and a neural radiance network~\cite {d-nerf, nerfies, NR-NeRF}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2011.12950" id="2870">
-        <attvalues>
-          <attvalue for="5" value=" While NeRF was initially designed for static content, subsequent dynamic works either condition the neural field with addtional temporal input~\cite{nsff,xian2021space,nrf4}, or jointly optimize a deformation field and a neural radiance network~\cite {d-nerf, nerfies, NR-NeRF}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2012.09790" id="2871">
-        <attvalues>
-          <attvalue for="5" value=" While NeRF was initially designed for static content, subsequent dynamic works either condition the neural field with addtional temporal input~\cite{nsff,xian2021space,nrf4}, or jointly optimize a deformation field and a neural radiance network~\cite {d-nerf, nerfies, NR-NeRF}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2106.12052" id="2872">
-        <attvalues>
-          <attvalue for="5" value=" To address this problem, several works~\cite{volsdf,neus,NeuralWarp,dneus} employ the signed distance function(SDF) for implicit surface representation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2106.10689" id="2873">
-        <attvalues>
-          <attvalue for="5" value=" To address this problem, several works~\cite{volsdf,neus,NeuralWarp,dneus} employ the signed distance function(SDF) for implicit surface representation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2112.09648" id="2874">
-        <attvalues>
-          <attvalue for="5" value=" To address this problem, several works~\cite{volsdf,neus,NeuralWarp,dneus} employ the signed distance function(SDF) for implicit surface representation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2211.11320" id="2875">
-        <attvalues>
-          <attvalue for="5" value=" To address this problem, several works~\cite{volsdf,neus,NeuralWarp,dneus} employ the signed distance function(SDF) for implicit surface representation." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2102.06199" id="2876">
-        <attvalues>
-          <attvalue for="5" value=" Although these methods~\cite{anerf, peng2022animatable, chen2021accurate, neuralactor,peng2021neural,dd-nerf} have demonstrated impressive efficacy in view synthesis and reconstruction in terms of the human body, the requirement of human priors like the SMPL model hinders them from generalizing to other dynamic scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2012.15838" id="2877">
-        <attvalues>
-          <attvalue for="5" value=" Although these methods~\cite{anerf, peng2022animatable, chen2021accurate, neuralactor,peng2021neural,dd-nerf} have demonstrated impressive efficacy in view synthesis and reconstruction in terms of the human body, the requirement of human priors like the SMPL model hinders them from generalizing to other dynamic scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2303.00050" target="2112.12390" id="2878">
-        <attvalues>
-          <attvalue for="5" value=" Although these methods~\cite{anerf, peng2022animatable, chen2021accurate, neuralactor,peng2021neural,dd-nerf} have demonstrated impressive efficacy in view synthesis and reconstruction in terms of the human body, the requirement of human priors like the SMPL model hinders them from generalizing to other dynamic scenes." />
-        </attvalues>
-      </edge>
-      <edge source="2005.10353" target="1710.00925" id="2879">
-        <attvalues>
-          <attvalue for="5" value=" \item We demonstrate that simple modifications to HopeNet~\cite{ruiz2018fine} can achieve $29\%$ improvement over the original network and $5-13\%$ improvement over the current state-of-the-art FSANet~\cite{yang2019fsa} for narrow range HPE from RGB images in the AFLW2000~\cite{zhu2016face} and BIWI~\cite{fanelli_IJCV} datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2005.10353" target="1905.08609" id="2880">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, though methods such as~\cite{ruiz2018fine,shao2019improving} perform well, they may be prohibitively large networks for mobile and embedded platforms." />
-        </attvalues>
-      </edge>
-      <edge source="2005.10353" target="1612.03153" id="2881">
-        <attvalues>
-          <attvalue for="5" value=" \item We detail an automated labeling process for the CMU Panoptic Dataset~\cite{joo2017panoptic} allowing it to be used for training and validation data in full-range HPE." />
-        </attvalues>
-      </edge>
-      <edge source="2005.10353" target="1511.07212" id="2882">
-        <attvalues>
-          <attvalue for="5" value=" \item We demonstrate that simple modifications to HopeNet~\cite{ruiz2018fine} can achieve $29\%$ improvement over the original network and $5-13\%$ improvement over the current state-of-the-art FSANet~\cite{yang2019fsa} for narrow range HPE from RGB images in the AFLW2000~\cite{zhu2016face} and BIWI~\cite{fanelli_IJCV} datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2109.10123" target="1807.11699" id="2883">
-        <attvalues>
-          <attvalue for="5" value=" Recent developments \cite{SegStereo}, \cite{SemForDipsEstimation} in the field revealed that stereo matching and semantic segmentation are intrinsically related, and both pieces of information need to be considered in an integrated manner to succeed in challenging applications such as robotics and autonomous navigation." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1610.07584" id="2884">
-        <attvalues>
-          <attvalue for="5" value=" Some work \cite{wu2016learning,choy20163d,wu20153d} have considered voxel-based representations and used 3D CNN to reconstruct the 3D shape from a single image." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1604.00449" id="2885">
-        <attvalues>
-          <attvalue for="5" value=" Some work \cite{wu2016learning,choy20163d,wu20153d} have considered voxel-based representations and used 3D CNN to reconstruct the 3D shape from a single image." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1905.03678" id="2886">
-        <attvalues>
-          <attvalue for="5" value=" Recently, 3D reconstruction based on deep learning has achieved remarkable results in many directions, such as 3D shape reconstruction from single view or multiple views \cite{wu2016learning,choy20163d,tatarchenko2019single} and shape completion \cite{xu2019disn}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1905.10711" id="2887">
-        <attvalues>
-          <attvalue for="5" value=" Recently, some work\cite{xu2019disn,mescheder2019occupancy,chen2019learning} implicitly represented the 3D surface as the continuous decision boundary of a deep neural network classifier." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1804.10975" id="2888">
-        <attvalues>
-          <attvalue for="5" value=" Generally, most of the 3D reconstruction work is based on CNN encoder-decoder architecture\cite{choy20163d,richter2018matryoshka}." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1809.05068" id="2889">
-        <attvalues>
-          <attvalue for="5" value=" For example, if voxels\cite{wu2018learning,zhang2018learning}are expected to be the final representation, 3D CNN will be selected as the decoder." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1812.11166" id="2890">
-        <attvalues>
-          <attvalue for="5" value=" For example, if voxels\cite{wu2018learning,zhang2018learning}are expected to be the final representation, 3D CNN will be selected as the decoder." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1804.01654" id="2891">
-        <attvalues>
-          <attvalue for="5" value="}$ \cite{wang2018pixel2mesh} used graph convolutional neural network \cite{scarselli2008graph} to make an ellipsoid template gradually become the target object, but the result was usually limited to the spherical topology." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1612.00603" id="2892">
-        <attvalues>
-          <attvalue for="5" value="}$ \cite{fan2017point}introduced point clouds as an output representation for 3D reconstruction." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1406.5670" id="2893">
-        <attvalues>
-          <attvalue for="5" value=" Some work \cite{wu2016learning,choy20163d,wu20153d} have considered voxel-based representations and used 3D CNN to reconstruct the 3D shape from a single image." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1703.09438" id="2894">
-        <attvalues>
-          <attvalue for="5" value="}$ \cite{tatarchenko2017octree} performed hierarchical partitioning of the output space for computational and storage efficiency, which helped predict higher resolution 3D shapes." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1812.03828" id="2895">
-        <attvalues>
-          <attvalue for="5" value=" Recently, some work\cite{xu2019disn,mescheder2019occupancy,chen2019learning} implicitly represented the 3D surface as the continuous decision boundary of a deep neural network classifier." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="1812.02822" id="2896">
-        <attvalues>
-          <attvalue for="5" value=" Recently, some work\cite{xu2019disn,mescheder2019occupancy,chen2019learning} implicitly represented the 3D surface as the continuous decision boundary of a deep neural network classifier." />
-        </attvalues>
-      </edge>
-      <edge source="2011.10776" target="2003.10739" id="2897">
-        <attvalues>
-          <attvalue for="5" value="}$ \cite{li2020dynamic} and Xu ${et}$ ${al." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2111.01628" id="2898">
-        <attvalues>
-          <attvalue for="5" value=" Fine-grained image classification \cite{wang2019survey,rong2021human,zhuang2020learning}, which involves distinguishing between visually similar classes, plays a crucial role in various applications such as species identification, product categorization, and medical diagnostics." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2002.10191" id="2899">
-        <attvalues>
-          <attvalue for="5" value=" Fine-grained image classification \cite{wang2019survey,rong2021human,zhuang2020learning}, which involves distinguishing between visually similar classes, plays a crucial role in various applications such as species identification, product categorization, and medical diagnostics." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2305.07611" id="2900">
-        <attvalues>
-          <attvalue for="5" value=" Despite the remarkable success of deep learning in computer vision \cite{abdullahi2024systematic,rayed2024deep,lai2023multimodal}, achieving high accuracy in fine-grained classification remains challenging due to the scarcity of labeled data and the subtlety of distinguishing features \cite{guo2024fine}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2401.17766" id="2901">
-        <attvalues>
-          <attvalue for="5" value=" Despite the remarkable success of deep learning in computer vision \cite{abdullahi2024systematic,rayed2024deep,lai2023multimodal}, achieving high accuracy in fine-grained classification remains challenging due to the scarcity of labeled data and the subtlety of distinguishing features \cite{guo2024fine}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="1610.02242" id="2902">
-        <attvalues>
-          <attvalue for="5" value=" SSL methods, including pseudo-labeling \cite{lee2013pseudo} and consistency regularization \cite{laine2016temporal}, have shown promise in improving model performance with limited supervision." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="1909.13719" id="2903">
-        <attvalues>
-          <attvalue for="5" value=" Standard data augmentation techniques \cite{cubuk2020randaugment} \cite{cubuk2019autoaugmentlearningaugmentationpolicies} can disrupt critical visual cues, and fine-grained image features are destroyed." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="1805.09501" id="2904">
-        <attvalues>
-          <attvalue for="5" value=" Standard data augmentation techniques \cite{cubuk2020randaugment} \cite{cubuk2019autoaugmentlearningaugmentationpolicies} can disrupt critical visual cues, and fine-grained image features are destroyed." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2104.00679" id="2905">
-        <attvalues>
-          <attvalue for="5" value=" Image region mixing may also overlook the fine details essential for accurate classification \cite{su2021realistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2008.00299" id="2906">
-        <attvalues>
-          <attvalue for="5" value=" These phases utilize CAMs\cite{jiang2021layercam,muhammad2020eigen,zhang2024opti} to accurately estimate the semantic content\cite{chen2022class} and generate refined labels that capture the essential details necessary for fine-grained classification." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2301.07002" id="2907">
-        <attvalues>
-          <attvalue for="5" value=" These phases utilize CAMs\cite{jiang2021layercam,muhammad2020eigen,zhang2024opti} to accurately estimate the semantic content\cite{chen2022class} and generate refined labels that capture the essential details necessary for fine-grained classification." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2203.00962" id="2908">
-        <attvalues>
-          <attvalue for="5" value=" These phases utilize CAMs\cite{jiang2021layercam,muhammad2020eigen,zhang2024opti} to accurately estimate the semantic content\cite{chen2022class} and generate refined labels that capture the essential details necessary for fine-grained classification." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2103.00550" id="2909">
-        <attvalues>
-          <attvalue for="5" value=" By focusing on semantic-level information, our approach effectively addresses the limitations of standard data augmentation and image-mixing techniques in preserving critical fine-grained features, we have conducted extensive experiments on two commonly used datasets for fine-grained classification, and the results show that our method far exceeds the most advanced and representative semi-supervised methods\cite{yang2022survey,ouali2020overview} at present, with a 13\% improvement in accuracy over the fully supervised model on the CUB\_200\_2011 dataset using 20\% labeled data, and similar results to supervised learning using only 30\% labeled data." />
-        </attvalues>
-      </edge>
-      <edge source="2409.03192" target="2006.05278" id="2910">
-        <attvalues>
-          <attvalue for="5" value=" By focusing on semantic-level information, our approach effectively addresses the limitations of standard data augmentation and image-mixing techniques in preserving critical fine-grained features, we have conducted extensive experiments on two commonly used datasets for fine-grained classification, and the results show that our method far exceeds the most advanced and representative semi-supervised methods\cite{yang2022survey,ouali2020overview} at present, with a 13\% improvement in accuracy over the fully supervised model on the CUB\_200\_2011 dataset using 20\% labeled data, and similar results to supervised learning using only 30\% labeled data." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1505.04597" id="2911">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1704.07711" id="2912">
-        <attvalues>
-          <attvalue for="5" value=" Examples of such algorithms are active contours \cite{kass1988snakes}, region-growing \cite{nock2004statistical}, conditional random fields (CRFs) \cite{plath2009multi}, and sparse reconstruction based methods \cite{minaee2019admm}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1605.06211" id="2913">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1612.01105" id="2914">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1803.08904" id="2915">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1804.03821" id="2916">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1802.02611" id="2917">
-        <attvalues>
-          <attvalue for="5" value=" When we tried to establish an adversarial network with DeepLabv3+ \cite{chen2018encoder} as segmentor, we could not manage to stabilize the network regardless of remedies we pulled in." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1801.00868" id="2918">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1908.07919" id="2919">
-        <attvalues>
-          <attvalue for="5" value=" Recent advances in deep learning and convolutional neural networks (CNNs) revolutionized this field resulting in state-of-the-art image segmentation algorithms such as FCN \cite{long2015fully}, U-Net \cite{ronneberger2015u}, PSPNet \cite{zhao2017pyramid}, EncNet \cite{zhang2018context}, Exfuse \cite{zhang2018exfuse}, DeepLabv3+ \cite{chen2018encoder}, PS and Panoptic DeepLab \cite{kirillov2019panoptic, cheng2019panoptic}, HRNet \cite{wang2020deep} and many other elegant architectures that considerably outperformed the traditional signal processing based methods addressing the same challenge." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1606.07230" id="2920">
-        <attvalues>
-          <attvalue for="5" value=" For instance, CRFs can be used to model pixel relationships and enforce label consistency between pixels \cite{liu2017deep, chen2017deeplab, shen2017semantic, liu2017deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1803.10335" id="2921">
-        <attvalues>
-          <attvalue for="5" value=" An alternative approach is extracting pixel affinity information from images and fusing them back to predicted label maps \cite{ke2018adaptive}; this comes at the cost of extra model branches and larger memory requirements." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1910.12037" id="2922">
-        <attvalues>
-          <attvalue for="5" value=" We think a conditional adversarial approach similar to \cite{luc2016semantic, souly2017semi} has the capacity to capture these pixel inter-dependencies and correlations in a more general (and not only local) fashion when compared to methods proposed in \cite{zhao2019region, zhao2019correlation}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1611.08408" id="2923">
-        <attvalues>
-          <attvalue for="5" value=" We think a conditional adversarial approach similar to \cite{luc2016semantic, souly2017semi} has the capacity to capture these pixel inter-dependencies and correlations in a more general (and not only local) fashion when compared to methods proposed in \cite{zhao2019region, zhao2019correlation}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="2005.04311" id="2924">
-        <attvalues>
-          <attvalue for="5" value=" In medical imaging, there are studies proposing (generative) adversarial networks for image segmentation \cite{imran2020progressive, chen2018semantic}; however, their aim is tackling domain shift in semi-supervised settings and they mostly employ U-Net type segmentors given their solid performance in this context." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1701.07875" id="2925">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, plugging state-of-the-art semantic segmentation models in an adversarial setting is prone to the well-known divergence and mode collapse issues \cite{arjovsky2017wasserstein, goodfellow2014explaining}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1907.08610" id="2926">
-        <attvalues>
-          <attvalue for="5" value=" The proposed solution (LoAd) takes inspiration from the ``lookahead optimizer'' \cite{zhang2019lookahead} and allows the semantic segmentation adversarial network to go ahead and actually diverge to some extent, then inspired by DAGGER in imitation learning \cite{ross2011reduction} aggregates the degraded label maps, and steps backward to use these new sets of information and avoid further divergence." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1011.0686" id="2927">
-        <attvalues>
-          <attvalue for="5" value=" The proposed solution (LoAd) takes inspiration from the ``lookahead optimizer'' \cite{zhang2019lookahead} and allows the semantic segmentation adversarial network to go ahead and actually diverge to some extent, then inspired by DAGGER in imitation learning \cite{ross2011reduction} aggregates the degraded label maps, and steps backward to use these new sets of information and avoid further divergence." />
-        </attvalues>
-      </edge>
-      <edge source="2006.11227" target="1412.6572" id="2928">
-        <attvalues>
-          <attvalue for="5" value=" Notably, our label map aggregation strategy is different from collecting adversarial examples to retrain and stabilize generative adversarial networks (GANs) \cite{goodfellow2014explaining, liu2019rob}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2304.02643" id="2929">
-        <attvalues>
-          <attvalue for="5" value=" Building on these developments, the Segment Anything Model (SAM)~\cite{kirillov2023segment} stands out breakthrough in scene segmentation with large vision models." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.00714" id="2930">
-        <attvalues>
-          <attvalue for="5" value=" More recently, the release of SAM2~\cite{ravi2024sam} further overcomes the limitation that SAM does not handle video content well." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.02924" id="2931">
-        <attvalues>
-          <attvalue for="5" value="}~\cite{lian2024evaluation} assessed its instance segmentation performance in underwater environments, while Yan {et al." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.00874" id="2932">
-        <attvalues>
-          <attvalue for="5" value=" A variety of evaluations have recently emerged to examine the segmentation performance of SAM2 in different scenarios~\cite{lian2024evaluation, yan2024biomedical,zhu2024medical,ma2024segment,tang2024evaluating,lou2024zero,yu2024sam}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.03322" id="2933">
-        <attvalues>
-          <attvalue for="5" value="}~\cite{ma2024segment} conducted a comprehensive benchmark of SAM2 across 11 medical image modalities and videos, highlighting its strengths and weaknesses compared to SAM and MedSAM." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2407.21596" id="2934">
-        <attvalues>
-          <attvalue for="5" value="}~\cite{tang2024evaluating} compared SAM2 and SAM on the camouflage object detection benchmark." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.01648" id="2935">
-        <attvalues>
-          <attvalue for="5" value=" A variety of evaluations have recently emerged to examine the segmentation performance of SAM2 in different scenarios~\cite{lian2024evaluation, yan2024biomedical,zhu2024medical,ma2024segment,tang2024evaluating,lou2024zero,yu2024sam}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.04593" id="2936">
-        <attvalues>
-          <attvalue for="5" value=" A variety of evaluations have recently emerged to examine the segmentation performance of SAM2 in different scenarios~\cite{lian2024evaluation, yan2024biomedical,zhu2024medical,ma2024segment,tang2024evaluating,lou2024zero,yu2024sam}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2408.03286" id="2937">
-        <attvalues>
-          <attvalue for="5" value="}~\cite{yan2024biomedical} explored its effectiveness in endoscopic and microscopic images." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="1704.03604" id="2938">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we evaluate the performance of SAM2 in class-agnostic instance-level segmentation tasks, focusing on three distinct scenarios: Salient Instance Segmentation (SIS)~\cite{li2017instance}, Camouflaged Instance Segmentation (CIS)~\cite{pei2022osformer}, and Shadow Instance Detection (SID)~\cite{wang2022instance}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2207.02255" id="2939">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we evaluate the performance of SAM2 in class-agnostic instance-level segmentation tasks, focusing on three distinct scenarios: Salient Instance Segmentation (SIS)~\cite{li2017instance}, Camouflaged Instance Segmentation (CIS)~\cite{pei2022osformer}, and Shadow Instance Detection (SID)~\cite{wang2022instance}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2207.04614" id="2940">
-        <attvalues>
-          <attvalue for="5" value=" In this paper, we evaluate the performance of SAM2 in class-agnostic instance-level segmentation tasks, focusing on three distinct scenarios: Salient Instance Segmentation (SIS)~\cite{li2017instance}, Camouflaged Instance Segmentation (CIS)~\cite{pei2022osformer}, and Shadow Instance Detection (SID)~\cite{wang2022instance}." />
-        </attvalues>
-      </edge>
-      <edge source="2409.02567" target="2203.03041" id="2941">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, We also thoroughly evaluate SAM2 on the high-resolution dichotomous image segmentation (DIS) benchmark~\cite{qin2022highly} to analyze its ability to segment granular target structures." />
-        </attvalues>
-      </edge>
-      <edge source="2206.14164" target="1811.03264" id="2942">
-        <attvalues>
-          <attvalue for="5" value=" Assuming Gaussian noises, the next best pose is directly estimated in \cite{estimate_next_01} by minimizing both the calibration and corner uncertainty." />
-        </attvalues>
-      </edge>
-      <edge source="2206.14164" target="2102.10202" id="2943">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{estimate_next_03}, a score function is proposed to estimate an optimal pose set in offline mode." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="1911.11763" id="2944">
-        <attvalues>
-          <attvalue for="5" value=" Following the matching-based approaches used for same-modal registration in images \cite{sarlin2020superglue, 2021LoFTR, Xu_2022_CVPR, xie2024deepmatcher} or point clouds \cite{lu2021hregnet, huang2021predator, qin2022geometric, wu2023sacf}, the mainstream pipeline (as illustrated in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2107.11992" id="2945">
-        <attvalues>
-          <attvalue for="5" value=" Following the matching-based approaches used for same-modal registration in images \cite{sarlin2020superglue, 2021LoFTR, Xu_2022_CVPR, xie2024deepmatcher} or point clouds \cite{lu2021hregnet, huang2021predator, qin2022geometric, wu2023sacf}, the mainstream pipeline (as illustrated in Fig." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2105.00896" id="2946">
-        <attvalues>
-          <attvalue for="5" value=" \ref{fig1} (b)) for image-to-point cloud registration is to first extract the 2D-3D correspondences with pseudo-siamese neural networks \cite{xia2021pseudo}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="1904.09742" id="2947">
-        <attvalues>
-          <attvalue for="5" value=" Recent algorithms such as 2D3D-MatchNet \cite{feng20192d3d}, CorrI2P \cite{ren2022corri2p} and CFI2P \cite{yao2023cfi2p} fall into this category." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2207.05483" id="2948">
-        <attvalues>
-          <attvalue for="5" value=" Recent algorithms such as 2D3D-MatchNet \cite{feng20192d3d}, CorrI2P \cite{ren2022corri2p} and CFI2P \cite{yao2023cfi2p} fall into this category." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2006.05525" id="2949">
-        <attvalues>
-          <attvalue for="5" value=" 1) Analogous to the concept of hard label in knowledge distillation \cite{gou2021knowledge}, a correspondence can be seen as a hard result that only retains the information of the matched point-pixel pair and discards the beneficial information between other points or pixels." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2104.03501" id="2950">
-        <attvalues>
-          <attvalue for="5" value=" DeepI2P \cite{li2021deepi2p} casts the registration process into a two-stage point classification and pose optimization framework." />
-        </attvalues>
-      </edge>
-      <edge source="2408.02392" target="2003.03026" id="2951">
-        <attvalues>
-          <attvalue for="5" value=" 4) We achieve state-of-the-art performance in registration accuracy and recall on the KITTI-Odometry \cite{geiger2013vision} and Apollo-DaoxiangLake \cite{zhou2020da4ad} datasets." />
-        </attvalues>
-      </edge>
-      <edge source="2402.18175" target="2303.04654" id="2952">
-        <attvalues>
-          <attvalue for="5" value=" Existing spatially variant PSF estimation methods are classified as optimization-based methods~\cite{mannan2016blur, mannan2016good, joshi2008psf, mosleh2015camera, delbracio2012non} and learning-based methods~\cite{yang2023aberration, tseng2021differentiable}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02861" target="1904.01382" id="2953">
-        <attvalues>
-          <attvalue for="5" value=" Moreover, some of these works~\cite{hosu2019effective, xiong2023federated} require input images in full resolution and additional feature extraction stages in advance, which lack efficiency and practical applicability." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02861" target="2207.12396" id="2954">
-        <attvalues>
-          <attvalue for="5" value=" The vision-language models used are either pre-trained solely on general image-text pairs~\cite{wang2023exploring}, or further pre-trained on aesthetic image-text pairs~\cite{ke2023vila}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02861" target="2303.14302" id="2955">
-        <attvalues>
-          <attvalue for="5" value=" The vision-language models used are either pre-trained solely on general image-text pairs~\cite{wang2023exploring}, or further pre-trained on aesthetic image-text pairs~\cite{ke2023vila}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.02861" target="2301.12597" id="2956">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by BLIP-2~\cite{li2023blip}, a vision-language pre-training model that uses learnable queries to align vision and language pre-trained features extracted from frozen unimodal models, we propose the \underline{I}mage \underline{A}esthetics \underline{A}ssessment via \underline{L}earnable \underline{Q}ueries ({\tt{IAA-LQ}}) approach." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="2303.05338" id="2957">
-        <attvalues>
-          <attvalue for="5" value=" Fine-grained recognition stands as a cornerstone in the field of computer vision\cite{xu2023mmcosine}, tasked with the discernment of subtle differences between closely related subclasses within a category." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="2103.07976" id="2958">
-        <attvalues>
-          <attvalue for="5" value=" This capability is paramount for applications ranging from biodiversity conservation to medical diagnostics\cite{he2022transfg}, where the accurate identification of species or conditions can have profound implications." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="2210.01439" id="2959">
-        <attvalues>
-          <attvalue for="5" value=" Despite certain progress in related research, existing methods are often hampered by an erroneous focus on background elements, which detracts from the extraction of key discriminative cues from the subject itself\cite{zha2023boosting}." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="1610.02391" id="2960">
-        <attvalues>
-          <attvalue for="5" value=" As illustrated by the Grad-CAM\cite{selvaraju2017grad} visualisation in Figure \ref{fig_cam}, there is an undesired focus on non-subject content, regardless of whether CNN-based or Transformer-based backbones are used." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="2304.02643" id="2961">
-        <attvalues>
-          <attvalue for="5" value=" By leveraging the capabilities of SAM\cite{kirillov2023segment} and Detic\cite{zhou2022detecting}, we have constructed a dataset that isolates the foreground subjects, thereby providing a more refined and accurate environment for model training and evaluation." />
-        </attvalues>
-      </edge>
-      <edge source="2408.01998" target="2201.02605" id="2962">
-        <attvalues>
-          <attvalue for="5" value=" By leveraging the capabilities of SAM\cite{kirillov2023segment} and Detic\cite{zhou2022detecting}, we have constructed a dataset that isolates the foreground subjects, thereby providing a more refined and accurate environment for model training and evaluation." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1703.06376" id="2963">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1605.01224" id="2964">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1411.4568" id="2965">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1611.07571" id="2966">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1603.09114" id="2967">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.09300" target="1511.04273" id="2968">
-        <attvalues>
-          <attvalue for="5" value=" Most survey papers compare traditional methods \cite{li2015survey,salahat2017recent}, whilst papers that introduce CNN based detectors compare their own method to one or more traditional approaches \cite{lenc2016learning,verdie2015,savinov2017quad,yi2016,moo2016learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1904.08405" id="2969">
-        <attvalues>
-          <attvalue for="5" value=" This event-driven scheme operating at the pixel level leads to (i)~a temporal resolution on the order of microseconds, which helps alleviate motion blur, and (ii)~a power consumption on the order of milliwatts thanks to sparsity~\cite{event_survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1512.03385" id="2970">
-        <attvalues>
-          <attvalue for="5" value=" As the standard neural network model for computer vision tasks, convolutional neural networks (CNNs) have demonstrated excellent performance for scenarios such as object recognition or detection \cite{alex,resnet,yolotiny,mobilenetv3,dualconv,cnn_survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1905.02244" id="2971">
-        <attvalues>
-          <attvalue for="5" value=" As the standard neural network model for computer vision tasks, convolutional neural networks (CNNs) have demonstrated excellent performance for scenarios such as object recognition or detection \cite{alex,resnet,yolotiny,mobilenetv3,dualconv,cnn_survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="2202.07481" id="2972">
-        <attvalues>
-          <attvalue for="5" value=" As the standard neural network model for computer vision tasks, convolutional neural networks (CNNs) have demonstrated excellent performance for scenarios such as object recognition or detection \cite{alex,resnet,yolotiny,mobilenetv3,dualconv,cnn_survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="2004.02806" id="2973">
-        <attvalues>
-          <attvalue for="5" value=" As the standard neural network model for computer vision tasks, convolutional neural networks (CNNs) have demonstrated excellent performance for scenarios such as object recognition or detection \cite{alex,resnet,yolotiny,mobilenetv3,dualconv,cnn_survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1904.08245" id="2974">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, current event-based vision systems usually rely on a straightforward conversion of event streams into images by accumulating events from each pixel within a given time window, which is also known as the dense-frame approach \cite{hugnet_ref_13,time_slice,dense_frame}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1708.09011" id="2975">
-        <attvalues>
-          <attvalue for="5" value=" Indeed, current event-based vision systems usually rely on a straightforward conversion of event streams into images by accumulating events from each pixel within a given time window, which is also known as the dense-frame approach \cite{hugnet_ref_13,time_slice,dense_frame}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="2308.14419" id="2976">
-        <attvalues>
-          <attvalue for="5" value=" As solving this challenge requires departing from frame-based vision, graph representations of event-based data have recently started being explored \cite{evs, aegnn, hugnet}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1609.02907" id="2977">
-        <attvalues>
-          <attvalue for="5" value=" While this sparse event graph is well suited for processing by graph neural networks (GNNs) \cite{gcn, spline}, the standard approach consisting of first constructing the full event graph, and then processing it with a GNN, fails to exploit the event-driven nature of the data and is bound to millisecond-level latencies in dedicated hardware~\cite{flowgnn, blockgnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="2204.13103" id="2978">
-        <attvalues>
-          <attvalue for="5" value=" While this sparse event graph is well suited for processing by graph neural networks (GNNs) \cite{gcn, spline}, the standard approach consisting of first constructing the full event graph, and then processing it with a GNN, fails to exploit the event-driven nature of the data and is bound to millisecond-level latencies in dedicated hardware~\cite{flowgnn, blockgnn}." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="2203.17149" id="2979">
-        <attvalues>
-          <attvalue for="5" value=" To solve this challenge, techniques to construct and process the event graph in a dynamic fashion (\ie on a per-event basis) have recently been proposed~\cite{aegnn, hugnet}, thereby enabling both sparsity-aware and event-driven processing." />
-        </attvalues>
-      </edge>
-      <edge source="2404.19489" target="1803.07913" id="2980">
-        <attvalues>
-          <attvalue for="5" value=" \item We finally deploy \prjname~on a Xilinx KV260 Ultrascale$+$ MPSoC platform and evaluate our design with the real-world dataset N-CARS \cite{ncars} for car recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2212.02319" target="1502.00956" id="2981">
-        <attvalues>
-          <attvalue for="5" value=" There are today mature methods that do SfM and SLAM efficiently \cite{mur2015orb,schoenberger2016sfm,schoenberger2016mvs}." />
-        </attvalues>
-      </edge>
-      <edge source="2212.02319" target="1707.09092" id="2982">
-        <attvalues>
-          <attvalue for="5" value=" However, they do not scale well for very large scenes, they are often not stable over time \cite{sattler2018benchmarking}, and are bad for downstream tasks such as interpretation and recognition." />
-        </attvalues>
-      </edge>
-      <edge source="2212.02319" target="2101.01909" id="2983">
-        <attvalues>
-          <attvalue for="5" value=" Working with lines in images is quite difficult, but line segment detectors have matured over the last years, \cite{xu2021line,dai2021fully,von2012lsd,pautrat2021sold2} and we foresee that they will soon be applicable for many general computer vision tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2212.02319" target="2104.11207" id="2984">
-        <attvalues>
-          <attvalue for="5" value=" Working with lines in images is quite difficult, but line segment detectors have matured over the last years, \cite{xu2021line,dai2021fully,von2012lsd,pautrat2021sold2} and we foresee that they will soon be applicable for many general computer vision tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2212.02319" target="2104.03362" id="2985">
-        <attvalues>
-          <attvalue for="5" value=" Working with lines in images is quite difficult, but line segment detectors have matured over the last years, \cite{xu2021line,dai2021fully,von2012lsd,pautrat2021sold2} and we foresee that they will soon be applicable for many general computer vision tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2008.11588" target="1705.06950" id="2986">
-        <attvalues>
-          <attvalue for="5" value=" Video action classification is a highly emerging research topic in computer vision \cite{kay2017kinetics, carreira2018short, sigurdsson2016hollywood} due to its potential wide range of applications." />
-        </attvalues>
-      </edge>
-      <edge source="2008.11588" target="1804.02748" id="2987">
-        <attvalues>
-          <attvalue for="5" value=" There are several video action classification datasets in the literature, among them, egocentric ones, eg those recorded from a first person point of view, have become of crucial importance \cite{damen2018scaling}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.11588" target="1908.00867" id="2988">
-        <attvalues>
-          <attvalue for="5" value=" This ego-motion might hinder the performance of modern CNNs trained to recognize actions in videos \cite{price2019evaluation,feichtenhofer2019slowfast, wang2018non,wang2019baidu} as these are focused on extracting the representative temporal features defining an action." />
-        </attvalues>
-      </edge>
-      <edge source="2008.11588" target="1812.03982" id="2989">
-        <attvalues>
-          <attvalue for="5" value=" This ego-motion might hinder the performance of modern CNNs trained to recognize actions in videos \cite{price2019evaluation,feichtenhofer2019slowfast, wang2018non,wang2019baidu} as these are focused on extracting the representative temporal features defining an action." />
-        </attvalues>
-      </edge>
-      <edge source="2008.11588" target="1906.09383" id="2990">
-        <attvalues>
-          <attvalue for="5" value=" This ego-motion might hinder the performance of modern CNNs trained to recognize actions in videos \cite{price2019evaluation,feichtenhofer2019slowfast, wang2018non,wang2019baidu} as these are focused on extracting the representative temporal features defining an action." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="2007.04316" id="2991">
-        <attvalues>
-          <attvalue for="5" value=" And it is applied to video security surveillance, personality recommendation, social media \cite{1-7proencca2021uu,yin2020fan}, etc However, most previous studies \cite{cao2020rank,icmeage} are limited to estimating a single attribute, or learning separate model for each face attribute." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="1911.11680" id="2992">
-        <attvalues>
-          <attvalue for="5" value=" And it is applied to video security surveillance, personality recommendation, social media \cite{1-7proencca2021uu,yin2020fan}, etc However, most previous studies \cite{cao2020rank,icmeage} are limited to estimating a single attribute, or learning separate model for each face attribute." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="1901.07884" id="2993">
-        <attvalues>
-          <attvalue for="5" value=" And it is applied to video security surveillance, personality recommendation, social media \cite{1-7proencca2021uu,yin2020fan}, etc However, most previous studies \cite{cao2020rank,icmeage} are limited to estimating a single attribute, or learning separate model for each face attribute." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="2203.17156" id="2994">
-        <attvalues>
-          <attvalue for="5" value=" And it is applied to video security surveillance, personality recommendation, social media \cite{1-7proencca2021uu,yin2020fan}, etc However, most previous studies \cite{cao2020rank,icmeage} are limited to estimating a single attribute, or learning separate model for each face attribute." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="1311.2901" id="2995">
-        <attvalues>
-          <attvalue for="5" value=" The hierarchical nature that CNNs possess provides a medium for expressing the correlation of heterogeneous face attributes\cite{zeiler2014visualizing}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.00561" target="1706.00906" id="2996">
-        <attvalues>
-          <attvalue for="5" value=" Specifically, we eschew the conventional approach of manually adjusting multi-task loss weights as observed in previously studies\cite{han2017heterogeneous}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2104.04638" id="2997">
-        <attvalues>
-          <attvalue for="5" value=" 3D reconstruction and relighting are of great importance in human digitization, especially in supporting realistic rendering in varying virtual environments, that can be widely applied in AR/VR \cite{ma2021pixel, orts2016holoportation}, holographic communication \cite{lawrence2021project, zhang2022virtualcube}, movie and gaming industry \cite{debevec2000pursuing}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="1901.00049" id="2998">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="1905.05172" id="2999">
-        <attvalues>
-          <attvalue for="5" value=" This method suffers from occlusion ambiguity, where some views may well be occluded, and mixing their features with visible ones causes inefficient feature utilization, thus penalizing the reconstruction quality \cite{saito2019pifu}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2104.05289" id="3000">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2007.13988" id="3001">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2004.00452" id="3002">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2007.03858" id="3003">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2204.08906" id="3004">
-        <attvalues>
-          <attvalue for="5" value=" Starting from Siclope \cite{natsume2019siclope} and PIFu \cite{saito2019pifu}, a series of methods \cite{hong2021stereopifu} improve the reconstruction performance in speed \cite{li2020monocular, feng2022fof}, quality \cite{saito2020pifuhd}, robustness \cite{zheng2021pamir, zheng2019deephuman} and light decoupling \cite{alldieck2022photorealistic}." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2007.11432" id="3005">
-        <attvalues>
-          <attvalue for="5" value=" Human templates such as SMPL \cite{loper2015smpl} can serve as effective guidance \cite{bhatnagar2020ipnet, zheng2021deepmulticap, peng2021neural, verbin2022ref}, but introduce additional template alignment errors and therefore cannot guarantee complete occlusion awareness." />
-        </attvalues>
-      </edge>
-      <edge source="2304.11900" target="2105.01859" id="3006">
-        <attvalues>
-          <attvalue for="5" value=" Function4D \cite{yu2021function4d} leverages the truncated Projective Signed Distance Function (PSDF) for visibility indication, but its level of details is susceptible to depth noise." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2006.11339" id="3007">
-        <attvalues>
-          <attvalue for="5" value=" VPSNet\cite{vps} integrated a temporal pixel-level fusion module and an object tracking head into a image panoptic segmentation network UPSNet\cite{upsnet} to achieve panoptic video results." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="1905.04804" id="3008">
-        <attvalues>
-          <attvalue for="5" value=" Video panoptic segmentation can be seen as a fusion of video semantic segmentation and video instance segmentation \cite{vis}." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="1901.03784" id="3009">
-        <attvalues>
-          <attvalue for="5" value=" VPSNet\cite{vps} integrated a temporal pixel-level fusion module and an object tracking head into a image panoptic segmentation network UPSNet\cite{upsnet} to achieve panoptic video results." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2204.04656" id="3010">
-        <attvalues>
-          <attvalue for="5" value=" Video K-Net\cite{video-knet} utilizes cross-temporal kernel interaction based on K-Net \cite{knet} to simultaneously segment and track both &quot;things&quot; and &quot;stuff&quot; in videos." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2106.14855" id="3011">
-        <attvalues>
-          <attvalue for="5" value=" Video K-Net\cite{video-knet} utilizes cross-temporal kernel interaction based on K-Net \cite{knet} to simultaneously segment and track both &quot;things&quot; and &quot;stuff&quot; in videos." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2012.05258" id="3012">
-        <attvalues>
-          <attvalue for="5" value=" ViP-DeepLab\cite{vip-deeplab}, adapted from Panoptic-DeepLab\cite{panoptic-deeplab} achieves temporal alignment by utilizing offset predictions for consecutive frames, ensuring consistent instance grouping across the video sequence." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="1911.10194" id="3013">
-        <attvalues>
-          <attvalue for="5" value=" ViP-DeepLab\cite{vip-deeplab}, adapted from Panoptic-DeepLab\cite{panoptic-deeplab} achieves temporal alignment by utilizing offset predictions for consecutive frames, ensuring consistent instance grouping across the video sequence." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2012.00759" id="3014">
-        <attvalues>
-          <attvalue for="5" value=" Considering that earlier methods necessitated the use of several distinct networks and sophisticated post-processing, Max-DeepLab\cite{max-deeplab} proposed the first end-to-end model for panoptic segmentation that applies a pixel and memory dual-path modeling." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2306.03413" id="3015">
-        <attvalues>
-          <attvalue for="5" value=" Recently, DVIS \cite{dvis} introduces a decoupled framework that divides video segmentation into three independent sub-tasks: image segmentation, referring tracking and temporal refinement." />
-        </attvalues>
-      </edge>
-      <edge source="2406.04002" target="2312.13305" id="3016">
-        <attvalues>
-          <attvalue for="5" value=" DVIS++\cite{dvis++} introduced denoising training and contrastive learning strategies into DVIS, resulting in a significant improvement and achieving state-of-the-art performance on VIPSeg\cite{vipseg} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2003.07733" id="3017">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1806.01196" id="3018">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2001.01173" id="3019">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2102.05447" id="3020">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1203.6722" id="3021">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1806.08472" id="3022">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2003.08061" id="3023">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2003.04092" id="3024">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="2007.02157" id="3025">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1901.00488" id="3026">
-        <attvalues>
-          <attvalue for="5" value=" 3D dense face alignment is essential for many face related tasks, eg, recognition~\cite{taigman2014deepface,booth20163d,guo2020learning,guo2018face,cao2020informative,xu2021searching}, animation~\cite{cao20133d}, avatar retargeting~\cite{cao2018stabilized}, tracking~\cite{xiong2015global}, attribute classification~\cite{bettadapura2012face,guo2017multi,guo2018dominant}, image restoration~\cite{yang2013structured,cao2019towards,cao2018learning}, anti-spoofing~\cite{wang2020deep,yu2020searching,qin2019learning,yu2020face,guo2019improving}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1511.07212" id="3027">
-        <attvalues>
-          <attvalue for="5" value="2ms) on multiple CPU cores (i5-8259U processor), (ii) accurate: By dynamically optimizing 3DMM parameters through a novel meta-optimization strategy combining the fast WPDC and VDC, we surpass the state-of-the-art results~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face} under a strict computation burden in inference, and (iii) stable: In a mini-batch, one still image is transformed slightly and smoothly into a short synthetic video, involving both in-plane and out-of-plane rotations, which provides temporal information of adjacent frames for training." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1709.01442" id="3028">
-        <attvalues>
-          <attvalue for="5" value=" Cascaded structures~\cite{zhu2016face,liu2017dense,zhu2019face} are always adopted to progressively update the parameters but the computation cost is increased linearly with the number of cascaded stages." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1612.04904" id="3029">
-        <attvalues>
-          <attvalue for="5" value=" Recent studies are mainly divided into two categories: 3D Morphable Model (3DMM) parameters regression~\cite{jourabloo2016large,zhu2016face,liu2016joint,liu2017dense,tuan2017regressing,zhu2019face,3ddfa_cleardusk} and dense vertices regression~\cite{jackson2017large,feng2018joint}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1804.01005" id="3030">
-        <attvalues>
-          <attvalue for="5" value="2ms) on multiple CPU cores (i5-8259U processor), (ii) accurate: By dynamically optimizing 3DMM parameters through a novel meta-optimization strategy combining the fast WPDC and VDC, we surpass the state-of-the-art results~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face} under a strict computation burden in inference, and (iii) stable: In a mini-batch, one still image is transformed slightly and smoothly into a short synthetic video, involving both in-plane and out-of-plane rotations, which provides temporal information of adjacent frames for training." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1703.07834" id="3031">
-        <attvalues>
-          <attvalue for="5" value="2ms) on multiple CPU cores (i5-8259U processor), (ii) accurate: By dynamically optimizing 3DMM parameters through a novel meta-optimization strategy combining the fast WPDC and VDC, we surpass the state-of-the-art results~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face} under a strict computation burden in inference, and (iii) stable: In a mini-batch, one still image is transformed slightly and smoothly into a short synthetic video, involving both in-plane and out-of-plane rotations, which provides temporal information of adjacent frames for training." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1803.07835" id="3032">
-        <attvalues>
-          <attvalue for="5" value="2ms) on multiple CPU cores (i5-8259U processor), (ii) accurate: By dynamically optimizing 3DMM parameters through a novel meta-optimization strategy combining the fast WPDC and VDC, we surpass the state-of-the-art results~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face} under a strict computation burden in inference, and (iii) stable: In a mini-batch, one still image is transformed slightly and smoothly into a short synthetic video, involving both in-plane and out-of-plane rotations, which provides temporal information of adjacent frames for training." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1603.06937" id="3033">
-        <attvalues>
-          <attvalue for="5" value=" However, the resolution of reconstructed faces relies on the size of the feature map and these methods rely on heavy networks like hourglass~\cite{newell2016stacked} or its variants, which are slow and memory-consuming in inference." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1805.11714" id="3034">
-        <attvalues>
-          <attvalue for="5" value=" In addition to single image, 3D face applications on videos are becoming more and more popular~\cite{cao20133d,cao2018stabilized,kim2018deep,kim2019neural}, where reconstructing stable results across consecutive frames is important, but it is often ignored by recent methods~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1909.02518" id="3035">
-        <attvalues>
-          <attvalue for="5" value=" In addition to single image, 3D face applications on videos are becoming more and more popular~\cite{cao20133d,cao2018stabilized,kim2018deep,kim2019neural}, where reconstructing stable results across consecutive frames is important, but it is often ignored by recent methods~\cite{zhu2016face,jackson2017large,feng2018joint,zhu2019face}." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1608.05477" id="3036">
-        <attvalues>
-          <attvalue for="5" value=" Video-based training~\cite{peng2016recurrent,liu2018two,dong2018supervision,tai2018towards} is always adopted to improve the stability in 2D face alignment." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1807.00966" id="3037">
-        <attvalues>
-          <attvalue for="5" value=" Video-based training~\cite{peng2016recurrent,liu2018two,dong2018supervision,tai2018towards} is always adopted to improve the stability in 2D face alignment." />
-        </attvalues>
-      </edge>
-      <edge source="2009.09960" target="1811.00342" id="3038">
-        <attvalues>
-          <attvalue for="5" value=" Video-based training~\cite{peng2016recurrent,liu2018two,dong2018supervision,tai2018towards} is always adopted to improve the stability in 2D face alignment." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06185" target="1904.11622" id="3039">
-        <attvalues>
-          <attvalue for="5" value=" Visual relationship detection is a core computer vision task that has gained a lot of attention recently~\cite{chen2019scene,yang2018graph,zellers2018scenegraphs,zhang2019vrd}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06185" target="1808.00191" id="3040">
-        <attvalues>
-          <attvalue for="5" value=" Visual relationship detection is a core computer vision task that has gained a lot of attention recently~\cite{chen2019scene,yang2018graph,zellers2018scenegraphs,zhang2019vrd}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06185" target="1711.06640" id="3041">
-        <attvalues>
-          <attvalue for="5" value=" Visual relationship detection is a core computer vision task that has gained a lot of attention recently~\cite{chen2019scene,yang2018graph,zellers2018scenegraphs,zhang2019vrd}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06185" target="1903.02728" id="3042">
-        <attvalues>
-          <attvalue for="5" value=" Visual relationship detection is a core computer vision task that has gained a lot of attention recently~\cite{chen2019scene,yang2018graph,zellers2018scenegraphs,zhang2019vrd}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.06185" target="1811.00982" id="3043">
-        <attvalues>
-          <attvalue for="5" value=" The challenge dataset is based on the Open Images V5 dataset~\cite{kuznetsova2018open}, which contains 9 million images annotated with class labels, bounding boxes, segmentation masks and visual relationships." />
-        </attvalues>
-      </edge>
-      <edge source="2403.17994" target="2306.05422" id="3044">
-        <attvalues>
-          <attvalue for="5" value=" In this competition, we adopted a zero-shot strategy and explored three approaches: OmniMotion\cite{abs-2306-05422}, TAPIR\cite{abs-2306-08637}, and Cotraker\cite{abs-2307-07635}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.17994" target="2306.08637" id="3045">
-        <attvalues>
-          <attvalue for="5" value=" In this competition, we adopted a zero-shot strategy and explored three approaches: OmniMotion\cite{abs-2306-05422}, TAPIR\cite{abs-2306-08637}, and Cotraker\cite{abs-2307-07635}." />
-        </attvalues>
-      </edge>
-      <edge source="2403.17994" target="2307.07635" id="3046">
-        <attvalues>
-          <attvalue for="5" value=" In this competition, we adopted a zero-shot strategy and explored three approaches: OmniMotion\cite{abs-2306-05422}, TAPIR\cite{abs-2306-08637}, and Cotraker\cite{abs-2307-07635}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1502.05908" id="3047">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1803.06189" id="3048">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1803.11493" id="3049">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1605.05863" id="3050">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1807.11206" id="3051">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1703.07737" id="3052">
-        <attvalues>
-          <attvalue for="5" value=" It has many important applications, including image retrieval \cite{wohlhart2015learning,he2018triplet,grabner20183d}, face recognition \cite{wen2016discriminative}, visual tracking \cite{tao2016siamese} and person re-identification \cite{yu2018hard,hermans2017defense}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1807.05520" id="3053">
-        <attvalues>
-          <attvalue for="5" value=" For example, in DeepCluster \cite{caron2018deep}, clustering is used to generate pseudo labels for images." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1901.04596" id="3054">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1505.05192" id="3055">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1603.09246" id="3056">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1708.04552" id="3057">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1803.07728" id="3058">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2008.04378" target="1911.05722" id="3059">
-        <attvalues>
-          <attvalue for="5" value=" Various supervised learning methods have been developed to train networks to predict the relative position of two randomly sampled patches \cite{doersch2015unsupervised}, solve Jigsaw image puzzles \cite{noroozi2016unsupervised}, predict pixel values of missing image patches \cite{devries2017improved}, classify image rotations of four discrete angles \cite{gidaris2018unsupervised}, reconstruct image transforms \cite{zhang2019aet}, etc Once successfully trained by these pretext tasks, the baseline network should be able to generate discriminative features for subsequent tasks, such as image retrieval, classification, matching, etc \cite{he2019momentum}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2108.05988" id="3060">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the object detection model based on transformer architecture has become a very potential detector with its complete end-to-end characteristics, no post-processing and better generalization performance \cite{tvt,daformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2111.14887" id="3061">
-        <attvalues>
-          <attvalue for="5" value=" In recent years, the object detection model based on transformer architecture has become a very potential detector with its complete end-to-end characteristics, no post-processing and better generalization performance \cite{tvt,daformer}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="1812.04798" id="3062">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised domain adaptation aims to align a labeled source domain’s detector with an unlabeled target domain, bridging the domain gap \cite{SWDA,cross}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2111.13216" id="3063">
-        <attvalues>
-          <attvalue for="5" value=" Unsupervised domain adaptation aims to align a labeled source domain’s detector with an unlabeled target domain, bridging the domain gap \cite{SWDA,cross}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2107.12636" id="3064">
-        <attvalues>
-          <attvalue for="5" value=" Almost all existing transformer-based detectors for domain adaptation apply adversarial learning to extract common features between different domains \cite{SFA,AQT}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2207.12647" id="3065">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of recent study \cite{MTTrans,cross-modal,enhanced} using self-training method, we link mean-teacher mechanism \cite{mean} with adversarial learning to generate pseudo labels for target domain, thus mining the information inside the data of target domain." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="1703.01780" id="3066">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the success of recent study \cite{MTTrans,cross-modal,enhanced} using self-training method, we link mean-teacher mechanism \cite{mean} with adversarial learning to generate pseudo labels for target domain, thus mining the information inside the data of target domain." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2209.01589" id="3067">
-        <attvalues>
-          <attvalue for="5" value=" However, it ignores the inconsistency \cite{consistent} between classification and regression task in object detection, which means high confidence score do not always guarantee the localization accuracy \cite{soft}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2106.09018" id="3068">
-        <attvalues>
-          <attvalue for="5" value=" However, it ignores the inconsistency \cite{consistent} between classification and regression task in object detection, which means high confidence score do not always guarantee the localization accuracy \cite{soft}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="1708.07819" id="3069">
-        <attvalues>
-          <attvalue for="5" value=" As shown in Fig \ref{fig:imbalance}, taking foggy cityscapes \cite{foggy} dataset as an example, there is a serious category imbalance issue in data distribution." />
-        </attvalues>
-      </edge>
-      <edge source="2401.13264" target="2307.08095" id="3070">
-        <attvalues>
-          <attvalue for="5" value=" Furthermore, to address the issue of class imbalance, a Gaussian Mixture Model (GMM) \cite{semidetr, consistent} is implemented to generate adaptive threshold for each category, utilizing the scores of each category pseudo boxes." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1604.01093" id="3071">
-        <attvalues>
-          <attvalue for="5" value=" Over the past decade, approaches to fusion-based dense visual SLAM have demonstrated the ability to build high fidelity dense $3D$ models of an environment facilitating higher-level scene understanding, perception and planning \cite{kinectfusion}, \cite{kintinuous}, \cite{elasticfusion}, \cite{bundlefusion}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1502.00956" id="3072">
-        <attvalues>
-          <attvalue for="5" value=" The ORB-SLAM open-source SLAM frameworks \cite{orbslam,orbslam2,orbslam3} in particular represent the state-of-the-art in monocular SLAM." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="2007.11898" id="3073">
-        <attvalues>
-          <attvalue for="5" value=" The ORB-SLAM open-source SLAM frameworks \cite{orbslam,orbslam2,orbslam3} in particular represent the state-of-the-art in monocular SLAM." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1610.05949" id="3074">
-        <attvalues>
-          <attvalue for="5" value=" On the other hand, sparse and semi-dense monocular systems have found a large degree of success in these environments but are limited in their resulting sparse or partially dense representations \cite{orbslam}, \cite{orbslam}, \cite{orbslam3}, \cite{orb_vi}, \cite{lsd_slam}, \cite{svo}, \cite{svo2}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1812.07179" id="3075">
-        <attvalues>
-          <attvalue for="5" value=" High-resolution dense models may become more necessary for robot planning in the future to accommodate a broader range of scenarios (eg, analyzing surface geometry in unevenly paved roads \cite{dense_path_planning}, obstacle detection and avoidance \cite{lidar}, etc)." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1610.06475" id="3076">
-        <attvalues>
-          <attvalue for="5" value="}\par \add{The work most similar to \add{our aproach}\del{the current work} is the dense surfel-based fusion system of \cite{scalable_dense_surfel}, \del{in which a hybrid approach is proposed that combines the}\add{combining} camera tracking from a sparse SLAM system (ORB-SLAM2 \cite{orbslam2}) with a dense fusion-based back-end." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="2007.06676" id="3077">
-        <attvalues>
-          <attvalue for="5" value=" The novelty of our approach lies in the use of a dense depth prediction network \cite{unrectdepthnet} within a hybrid architecture, combining state-of-the-art sparse feature tracking and dense fusion-based visual mapping algorithms in a loosely coupled fashion." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1808.01900" id="3078">
-        <attvalues>
-          <attvalue for="5" value=" In comparison to this work, these systems are either not suited to fast, outdoor camera motion \cite{codeslam, deep_tam, cnn_slam} or do not produce dense models \cite{d3vo, deep_vo}}\add{Several systems have been presented that demonstrate hybrid approaches to monocular visual SLAM and VO \cite{deepfactors, codeslam, cnn_slam, d3vo}, while other approaches implement a fully learned SLAM pipeline \cite{deep_tam, deep_vo}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="2003.01060" id="3079">
-        <attvalues>
-          <attvalue for="5" value=" In comparison to this work, these systems have either not been designed with large-scale fast, outdoor camera motion in mind \cite{deepfactors, codeslam, deep_tam, cnn_slam}, or do not produce dense models \cite{d3vo, deep_vo}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="2001.05049" id="3080">
-        <attvalues>
-          <attvalue for="5" value=" In comparison to this work, these systems have either not been designed with large-scale fast, outdoor camera motion in mind \cite{deepfactors, codeslam, deep_tam, cnn_slam}, or do not produce dense models \cite{d3vo, deep_vo}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.07736" target="1909.04250" id="3081">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{scalable_dense_surfel}, \add{a sub-mapping approach is employed where} each surfel is anchored to a reference keyframe from the sparse system." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13070" target="2207.00419" id="3082">
-        <attvalues>
-          <attvalue for="5" value=" Therefore, it is not trivial to expand the image-based approaches directly to the video domain due to dynamic and fine-grained movements or actions \cite{schiappa2022self}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13070" target="2011.00362" id="3083">
-        <attvalues>
-          <attvalue for="5" value=" Compared with static images, video-specific pretext tasks (eg, predicting clip orders, time arrows, and paces) provide richer sources of “supervision” \cite{jaiswal2020survey}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.13070" target="1505.00295" id="3084">
-        <attvalues>
-          <attvalue for="5" value=" In order to avoid human labeling, Walker {et al} \cite{walker2015dense} proposed to estimate the motion information based on dense optical flow." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1804.02913" id="3085">
-        <attvalues>
-          <attvalue for="5" value=" The past decade has witnessed significant advances in deblurring, wherein major efforts have gone into designing priors that are apt for recovering the underlying undistorted image and the camera trajectory \cite{xu2010two,pan2016robust,fergus2006removing,shan2008high, cho2009fast,joshi2008psf,krishnan2009fast,krishnan2011blind,xu2010two, pan2014deblurring,pan2016blind,vasu2017local,yan2017image, chandramouli2010inferring, paramanand2011depth, paramanand2013non, vijay2013non, rao2014inferring,nimisha2018unsupervised,mohan2019unconstrained,purohit2020region,mohan2021deep,paramanand2014shape,rao2014harnessing, nimisha2017blur,vasu2017local,nimisha2018generating,vasu2018non,purohit2018learning,purohit2019bringing}." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1603.04771" id="3086">
-        <attvalues>
-          <attvalue for="5" value=" Few approaches~\cite{chakrabarti2016neural,schuler2013machine,schuler2016learning} have proposed hybrid algorithms where a Convolutional Neural Network (CNN) estimates the blur kernel, which is then used in an alternative optimization framework for recovering the latent image." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1406.7444" id="3087">
-        <attvalues>
-          <attvalue for="5" value=" Few approaches~\cite{chakrabarti2016neural,schuler2013machine,schuler2016learning} have proposed hybrid algorithms where a Convolutional Neural Network (CNN) estimates the blur kernel, which is then used in an alternative optimization framework for recovering the latent image." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1503.00593" id="3088">
-        <attvalues>
-          <attvalue for="5" value=" Few methods \cite{sun2015learning,gong2017motion} circumvent the segmentation stage by training CNNs to estimate locally linear blur kernels and feeding them to a non-uniform deblurring algorithm based on patch-level prior." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1612.02583" id="3089">
-        <attvalues>
-          <attvalue for="5" value=" Few methods \cite{sun2015learning,gong2017motion} circumvent the segmentation stage by training CNNs to estimate locally linear blur kernels and feeding them to a non-uniform deblurring algorithm based on patch-level prior." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1507.02438" id="3090">
-        <attvalues>
-          <attvalue for="5" value=" \cite{hyun2015generalized} further relaxed the constraint on the scene motion by parameterizing spatially varying blur kernel using optical flow." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1708.04208" id="3091">
-        <attvalues>
-          <attvalue for="5" value=" Few methods have also been proposed for burst image deblurring \cite{wieschollek2017learning,aittala2018burst} which utilize number of observations with independent blurs to restore a scene, but are not trained for general video deblurring." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1704.03285" id="3092">
-        <attvalues>
-          <attvalue for="5" value=" Online Video Deblurring (OVD)~\cite{hyun2017online} presents a faster design for video deblurring which does not require frame alignment." />
-        </attvalues>
-      </edge>
-      <edge source="1903.11394" target="1802.01770" id="3093">
-        <attvalues>
-          <attvalue for="5" value=" However, their performance is inferior to the state-of-the-art~\cite{tao2018scale} in several aspects." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="2101.02082" id="3094">
-        <attvalues>
-          <attvalue for="5" value=" Recently, significant progress on the DAR task has been achieved due to the advances in automation technologies and the application of deep learning methods \cite{DBLP:journals/itsm/RongHHLK22}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="2303.16727" id="3095">
-        <attvalues>
-          <attvalue for="5" value=" Several methods have been proposed for DAR, often building upon general human action recognition models that utilize 3D convolutional neural networks (CNNs) \cite{chen2023skateboardai} and vision transformers \cite{wang2023videomaev2}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="1811.08383" id="3096">
-        <attvalues>
-          <attvalue for="5" value=" Among them, the temporal shift module (TSM) provides an efficient solution by shifting features from neighbour frames \cite{DBLP:conf/iccv/LinGH19}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="2109.04021" id="3097">
-        <attvalues>
-          <attvalue for="5" value=" utilized average late fusion to detect driving behaviours with depth and IR modalities \cite{DBLP:conf/crv/KhanSSPA22}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="2210.09441" id="3098">
-        <attvalues>
-          <attvalue for="5" value=" employed a multi-scale channel attention module for the score fusion of depth and IR inputs \cite{DBLP:journals/corr/abs-2210-09441}." />
-        </attvalues>
-      </edge>
-      <edge source="2401.14838" target="2204.04734" id="3099">
-        <attvalues>
-          <attvalue for="5" value=" compared several late fusion strategies for DAR task \cite{DBLP:conf/ivs/RoitbergPMSSS22}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="2103.17001" id="3100">
-        <attvalues>
-          <attvalue for="5" value=" Surround-view systems employ four sensors to create a network with large overlapping zones to cover the car's near-field area \cite{eising2021near, kumar2022surround}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="1905.01489" id="3101">
-        <attvalues>
-          <attvalue for="5" value="\par WoodScape \cite{yogamani2019_woodscape} is a large dataset for $360^\degree$ sensing around an ego vehicle with four fisheye cameras." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="1901.09270" id="3102">
-        <attvalues>
-          <attvalue for="5" value=" It is built based on industrialization needs addressing the diversity challenges \cite{uricar2019challenges}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="2105.12763" id="3103">
-        <attvalues>
-          <attvalue for="5" value=" WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="1912.02249" id="3104">
-        <attvalues>
-          <attvalue for="5" value=" WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="2104.14042" id="3105">
-        <attvalues>
-          <attvalue for="5" value=" WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="1803.06192" id="3106">
-        <attvalues>
-          <attvalue for="5" value=" WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="2001.02223" id="3107">
-        <attvalues>
-          <attvalue for="5" value=" WoodScape dataset (public and private versions) has enabled research in various perception areas such as object detection~\cite{dahal2021online, rashedfisheyeyolo, rashed2021generalized, yahiaoui2019optimization}, trailer detection \cite{dahal2019deeptrailerassist}, soiling detection~\cite{uricar2021let, das2020tiledsoilingnet, uricar2019desoiling}, semantic segmentation \cite{cheke2022fisheyepixpro, sobh2021adversarial, dahal2021roadedgenet, klingner2022detecting, rashed2019motion}, weather classification \cite{dhananjaya2021weather}, depth prediction \cite{kumar2018monocular, kumar2018near, kumar2021svdistnet, kumar2020unrectdepthnet, kumar2020fisheyedistancenet, kumar2021fisheyedistancenet++, kumar2020syndistnet}, moving object detection \cite{siam2018modnet, yahiaoui2019fisheyemodnet, rashed2019motion, mohamed2021monocular}, SLAM \cite{tripathi2020trained, gallagher2021hybrid} and multi-task learning \cite{leang2020dynamic, kumar2021omnidet}." />
-        </attvalues>
-      </edge>
-      <edge source="2206.12912" target="2203.05056" id="3108">
-        <attvalues>
-          <attvalue for="5" value=" SynwoodScape \cite{sekkat2022synwoodscape} is a synthetic version of the Woodscape dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2305.14207" target="2304.02643" id="3109">
-        <attvalues>
-          <attvalue for="5" value=" Recently, a prominent model for image segmentation called the Segment Anything model (SAM) has been introduced~\cite{kirillov2023segment}." />
-        </attvalues>
-      </edge>
-      <edge source="2305.14207" target="2305.08196" id="3110">
-        <attvalues>
-          <attvalue for="5" value=" Notably, it is worth mentioning that while previous SAM-based projects~\cite{zhang2023comprehensive} like SSA~\cite{SSA}, Anything-3D~\cite{shen2023anything3d}, and SAM 3D~\cite{pointcept2023} primarily employ RGB images as inputs, we are the first to utilize SAM for directly segmenting rendered depth images." />
-        </attvalues>
-      </edge>
-      <edge source="2305.14207" target="2304.10261" id="3111">
-        <attvalues>
-          <attvalue for="5" value=" Notably, it is worth mentioning that while previous SAM-based projects~\cite{zhang2023comprehensive} like SSA~\cite{SSA}, Anything-3D~\cite{shen2023anything3d}, and SAM 3D~\cite{pointcept2023} primarily employ RGB images as inputs, we are the first to utilize SAM for directly segmenting rendered depth images." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1703.06870" id="3112">
-        <attvalues>
-          <attvalue for="5" value=" This is different from existing semantic segmentation~\cite{he2017mask} and instance segmentation~\cite{liu2018path,zhou2017scene}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1803.01534" id="3113">
-        <attvalues>
-          <attvalue for="5" value=" However, instance segmentation does not distinguish the differences among the buildings~\cite{zhou2017scene,liu2018path,bolya2019yolact}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1904.02689" id="3114">
-        <attvalues>
-          <attvalue for="5" value=" However, instance segmentation does not distinguish the differences among the buildings~\cite{zhou2017scene,liu2018path,bolya2019yolact}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1604.01685" id="3115">
-        <attvalues>
-          <attvalue for="5" value=" This differs from existing segmentation benchmarks~\cite{zhou2017scene,cordts2016cityscapes,lin2014microsoft,everingham2010pascal,geiger2012we}, where the recognition granularity is fixed in advance." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1405.0312" id="3116">
-        <attvalues>
-          <attvalue for="5" value=" This differs from existing segmentation benchmarks~\cite{zhou2017scene,cordts2016cityscapes,lin2014microsoft,everingham2010pascal,geiger2012we}, where the recognition granularity is fixed in advance." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="2001.05027" id="3117">
-        <attvalues>
-          <attvalue for="5" value=" For example, we find the SOTA spatial verification methods~\cite{cao2020unifying,Noh_2017_ICCV} give a high inlier number to some true query-index pairs but match the wrong regions." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1803.11285" id="3118">
-        <attvalues>
-          <attvalue for="5" value=" Thirdly, every positive image is guaranteed to be identifiable by people without considering any contextual visual information~\cite{radenovic2018revisiting}." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="2011.12172" id="3119">
-        <attvalues>
-          <attvalue for="5" value=" Pixel retrieval can also be useful in medical diagnosis~\cite{yang2014parallel}, geographical information systems~\cite{zhu2021vigor}, and image matting~\cite{wei2021improved,xu2017deep,zhang2019late} (Section ~\ref{sec:Applications})." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="2012.08323" id="3120">
-        <attvalues>
-          <attvalue for="5" value=" Pixel retrieval can also be useful in medical diagnosis~\cite{yang2014parallel}, geographical information systems~\cite{zhu2021vigor}, and image matting~\cite{wei2021improved,xu2017deep,zhang2019late} (Section ~\ref{sec:Applications})." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1703.03872" id="3121">
-        <attvalues>
-          <attvalue for="5" value=" Pixel retrieval can also be useful in medical diagnosis~\cite{yang2014parallel}, geographical information systems~\cite{zhu2021vigor}, and image matting~\cite{wei2021improved,xu2017deep,zhang2019late} (Section ~\ref{sec:Applications})." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="1612.06321" id="3122">
-        <attvalues>
-          <attvalue for="5" value=" For example, we find the SOTA spatial verification methods~\cite{cao2020unifying,Noh_2017_ICCV} give a high inlier number to some true query-index pairs but match the wrong regions." />
-        </attvalues>
-      </edge>
-      <edge source="2309.05438" target="2104.01538" id="3123">
-        <attvalues>
-          <attvalue for="5" value=" We find the dense and pixel-level approaches~\cite{min2021hypercorrelation, truong2021warp} helpful for the pixel retrieval task." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="2004.07728" id="3124">
-        <attvalues>
-          <attvalue for="5" value=" While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="1801.03924" id="3125">
-        <attvalues>
-          <attvalue for="5" value=" While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="2010.11929" id="3126">
-        <attvalues>
-          <attvalue for="5" value=" While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="2101.01097" id="3127">
-        <attvalues>
-          <attvalue for="5" value=" While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="2108.05997" id="3128">
-        <attvalues>
-          <attvalue for="5" value=" While the early examples employ convolutionals neural networks (CNNs) for predicting the image quality~\cite{DBLP:journals/corr/abs-2004-07728,DBLP:journals/corr/abs-1801-03924}, the more recent work considers self-attention and vision-transformers (ViT)~\cite{dosovitskiy2020image} based architectures~\cite{you2021transformer,ke2021musiq}." />
-        </attvalues>
-      </edge>
-      <edge source="2303.06907" target="2002.09140" id="3129">
-        <attvalues>
-          <attvalue for="5" value=" Only very recently, CNN-based models specific to omnidirection images haves been proposed~\cite{8702664,xu2020blind,9506044,9766332,9859468}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2104.14294" id="3130">
-        <attvalues>
-          <attvalue for="5" value=" We present an in-depth empirical evaluation of the adversarial robustness of self-supervised vision encoders~\cite{simsiam,caron2021dino} for downstream tasks beyond classification." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2111.06377" id="3131">
-        <attvalues>
-          <attvalue for="5" value=" Foundation models in the vision domain have even outperformed dedicated models on several tasks~\cite{caron2021dino,mae,oquab2024dinov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2304.07193" id="3132">
-        <attvalues>
-          <attvalue for="5" value=" Foundation models in the vision domain have even outperformed dedicated models on several tasks~\cite{caron2021dino,mae,oquab2024dinov2}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2006.04924" id="3133">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2010.13337" id="3134">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2111.01124" id="3135">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2207.10899" id="3136">
-        <attvalues>
-          <attvalue for="5" value=" Our main observation is that the state-of-the-art adversarial full fine-tuning of encoders~\cite{zhang2022decoupled}: (1) substantially lowers clean performance, (2) increases robustness only against the \EmbedAttack, and (3) remains ineffective in improving robustness against the task-specific \PGDAttacks." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2303.01289" id="3137">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="1605.06211" id="3138">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="1806.01260" id="3139">
-        <attvalues>
-          <attvalue for="5" value=" Despite their broad utility, the adversarial robustness of these models has only been explored for classification tasks with linear probing~\cite{NaseerPurifier2020CVPR,jiang2020robust,fan2021does,zhang2022decoupled,luo2023rethinking} while other common downstream tasks, such as semantic segmentation~\cite{long2017segmentation} and depth estimation~\cite{godard2019depthestimation}, remain unexplored." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2310.18936" id="3140">
-        <attvalues>
-          <attvalue for="5" value=" Recently, ~\cite{li2023adversarialnotbugs} showed that non-robust features extracted from adversarial examples for supervised models (and useful for classification) become largely useless when transferred to self-supervised learning paradigms." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2011.10566" id="3141">
-        <attvalues>
-          <attvalue for="5" value=" We present an in-depth empirical evaluation of the adversarial robustness of self-supervised vision encoders~\cite{simsiam,caron2021dino} for downstream tasks beyond classification." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="1706.06083" id="3142">
-        <attvalues>
-          <attvalue for="5" value=" We use attacks that operate in the encoder's embedding space (\EmbedAttack) and those that leverage direct access to the downstream task outputs (\PGDAttacks), \eg \PGD for classification~\cite{madry2018towards} or \SegPGD for semantic segmentation~\cite{gu2022segpgd}." />
-        </attvalues>
-      </edge>
-      <edge source="2407.12588" target="2207.12391" id="3143">
-        <attvalues>
-          <attvalue for="5" value=" We use attacks that operate in the encoder's embedding space (\EmbedAttack) and those that leverage direct access to the downstream task outputs (\PGDAttacks), \eg \PGD for classification~\cite{madry2018towards} or \SegPGD for semantic segmentation~\cite{gu2022segpgd}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1611.07004" id="3144">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="2008.11882" id="3145">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1808.06601" id="3146">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1808.05174" id="3147">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1908.07683" id="3148">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1908.09514" id="3149">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1603.08511" id="3150">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1705.02999" id="3151">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1501.00092" id="3152">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1804.07723" id="3153">
-        <attvalues>
-          <attvalue for="5" value=" These techniques leverage self-supervised video data and tailor models to specific tasks using relevant losses, \eg, reconstruction error~\cite{Liu2018,Yu2018,Kim2019} or style loss~\cite{Johnson2016,Gupta2017}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1801.07892" id="3154">
-        <attvalues>
-          <attvalue for="5" value=" Now consider a state-of-the-art image inpainting model, Contextual Attention~\cite{Yu2018}: this method produces realistic textures on the same example (but exhibits temporal inconsistency)." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1904.08060" id="3155">
-        <attvalues>
-          <attvalue for="5" value=" Developments in both large-scale datasets and deep neural networks (DNNs) have led to incredible advancements in image-to-image~\cite{Isola2017,Yang2018} and video-to-video~\cite{Wang2018,Bansal2018,Park2019,Chen2019} translation tasks such as color restoration~\cite{Zhang2016,Zhang2017}, super-resolution~\cite{Dong2015a,Dong2015}, inpainting~\cite{Liu2018,Yu2018,Hong2019}, and style transfer~\cite{Gatys2016,Johnson2016}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1603.08155" id="3156">
-        <attvalues>
-          <attvalue for="5" value=" These techniques leverage self-supervised video data and tailor models to specific tasks using relevant losses, \eg, reconstruction error~\cite{Liu2018,Yu2018,Kim2019} or style loss~\cite{Johnson2016,Gupta2017}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1905.01639" id="3157">
-        <attvalues>
-          <attvalue for="5" value=" HyperCon outperforms a prior state-of-the-art video consistency model~\cite{Lai2018} in terms of reducing flicker and adhering to the intended translation; when combined with a strong image inpainting method, it also produces better predictions than a state-of-the-art video inpainting model~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1604.08610" id="3158">
-        <attvalues>
-          <attvalue for="5" value=" The first incorporates temporal consistency directly into the method through optical flow-based losses~\cite{Kim2019,Ruder2016,Sajjadi2018,Gupta2017,Zhang2019} or network layers that operate on the time dimension, \eg, 3D convolutional layers~\cite{Chang2019} or recurrent layers~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1801.04590" id="3159">
-        <attvalues>
-          <attvalue for="5" value=" The first incorporates temporal consistency directly into the method through optical flow-based losses~\cite{Kim2019,Ruder2016,Sajjadi2018,Gupta2017,Zhang2019} or network layers that operate on the time dimension, \eg, 3D convolutional layers~\cite{Chang2019} or recurrent layers~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1705.02092" id="3160">
-        <attvalues>
-          <attvalue for="5" value=" These techniques leverage self-supervised video data and tailor models to specific tasks using relevant losses, \eg, reconstruction error~\cite{Liu2018,Yu2018,Kim2019} or style loss~\cite{Johnson2016,Gupta2017}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1909.07957" id="3161">
-        <attvalues>
-          <attvalue for="5" value=" The first incorporates temporal consistency directly into the method through optical flow-based losses~\cite{Kim2019,Ruder2016,Sajjadi2018,Gupta2017,Zhang2019} or network layers that operate on the time dimension, \eg, 3D convolutional layers~\cite{Chang2019} or recurrent layers~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1907.01131" id="3162">
-        <attvalues>
-          <attvalue for="5" value=" The first incorporates temporal consistency directly into the method through optical flow-based losses~\cite{Kim2019,Ruder2016,Sajjadi2018,Gupta2017,Zhang2019} or network layers that operate on the time dimension, \eg, 3D convolutional layers~\cite{Chang2019} or recurrent layers~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1808.00449" id="3163">
-        <attvalues>
-          <attvalue for="5" value=" HyperCon outperforms a prior state-of-the-art video consistency model~\cite{Lai2018} in terms of reducing flicker and adhering to the intended translation; when combined with a strong image inpainting method, it also produces better predictions than a state-of-the-art video inpainting model~\cite{Kim2019}." />
-        </attvalues>
-      </edge>
-      <edge source="1912.04950" target="1809.00461" id="3164">
-        <attvalues>
-          <attvalue for="5" value=" Whereas the video model was trained on about 5,000 examples from one of the largest video segmentation datasets to date~\cite{Xu2018}, the image model was trained on over 1,000,000 images~\cite{Zhou2018}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="1206.5538" id="3165">
-        <attvalues>
-          <attvalue for="5" value=" Extracting data representation that are compact, expressive, and meaningful is a long term goal in the field of machine learning~\cite{bengio2013representation}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="1603.09246" id="3166">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision, prior work addressed this by solving pretext tasks defined in a self-supervised manner such as denoising autoencoders~\cite{vincent2008extracting}, solving jigsaw puzzles~\cite{noroozi2016unsupervised}, or motion segmentation~\cite{pathak2017learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="1612.06370" id="3167">
-        <attvalues>
-          <attvalue for="5" value=" In computer vision, prior work addressed this by solving pretext tasks defined in a self-supervised manner such as denoising autoencoders~\cite{vincent2008extracting}, solving jigsaw puzzles~\cite{noroozi2016unsupervised}, or motion segmentation~\cite{pathak2017learning}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="2002.05709" id="3168">
-        <attvalues>
-          <attvalue for="5" value=" We validate our batch curation scheme in a small scale experiment by integrating it into the SimCLR~\cite{chen2020simple} model trained using CIFAR-10~\cite{krizhevsky2009learning} dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="1911.05722" id="3169">
-        <attvalues>
-          <attvalue for="5" value=" A specific pretext task, namely instance discrimination~\cite{wu2018unsupervised}, has recently emerged as the state-of-the-art for unsupervised visual representation learning, spawning a polyhedral of new models based on contrastive learning such as SimCLR~\cite{chen2020simple}, MoCo~\cite{he2020momentum} or SwAV~\cite{caron2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2108.08643" target="2006.09882" id="3170">
-        <attvalues>
-          <attvalue for="5" value=" A specific pretext task, namely instance discrimination~\cite{wu2018unsupervised}, has recently emerged as the state-of-the-art for unsupervised visual representation learning, spawning a polyhedral of new models based on contrastive learning such as SimCLR~\cite{chen2020simple}, MoCo~\cite{he2020momentum} or SwAV~\cite{caron2020unsupervised}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1610.02984" id="3171">
-        <attvalues>
-          <attvalue for="5" value=" Person ReID is a vital computer vision task that aims to recognize the same person across the images or video sequences taken by different cameras \cite{zheng2016person, ye2020deep}." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1803.09132" id="3172">
-        <attvalues>
-          <attvalue for="5" value=" Due to the success of convolutional neural networks (CNNs), a variety of learning-based methods \cite{chang2018multi, sun2018beyond, zheng2017unlabeled} have been proposed to address person ReID problem and achieved good performance on many benchmarks." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1907.08427" id="3173">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1701.07717" id="3174">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1904.07223" id="3175">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1904.03349" id="3176">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1712.02225" id="3177">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1905.04525" id="3178">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="2002.01625" id="3179">
-        <attvalues>
-          <attvalue for="5" value=" Nevertheless, with the presence of people occlusions \cite{huang2018adversarially,hou2019vrstc}, viewpoint changes \cite{zheng2017unlabeled,zheng2019joint}, pose changes \cite{zhu2019progressive,qian2018pose}, and even illumination changes \cite{zeng2019illumination,9190796}, person reID remains a very challenging task." />
-        </attvalues>
-      </edge>
-      <edge source="2101.04544" target="1908.06052" id="3180">
-        <attvalues>
-          <attvalue for="5" value=" To address the above problem, existing methods \cite{jiao2018deep, chen2019learning, mao2019resolution,li2019recover} either use super-resolution (SR) models or generative adversarial network (GAN) to convert LR images into their corresponding HR images, followed by a person ReID model." />
-        </attvalues>
-      </edge>
-      <edge source="2106.06649" target="1905.04804" id="3181">
-        <attvalues>
-          <attvalue for="5" value=" VIS, first introduced in the YoutubeVOS 2019 challenge \cite{vis}, aims to perform object detection, instance segmentation, and object tracking across video frames." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02971" target="2002.05709" id="3182">
-        <attvalues>
-          <attvalue for="5" value=" Because the number of negative samples will directly affect the training results, in view of this, SimCLR provides more negative samples by increasing the batch size \cite{b1}, while MoCo \cite{b2} and MoCo v2 \cite{b3} using momentum update mechanism keeps a queue as memory bank, which can dynamically maintain the count of negative samples." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02971" target="1911.05722" id="3183">
-        <attvalues>
-          <attvalue for="5" value=" Because the number of negative samples will directly affect the training results, in view of this, SimCLR provides more negative samples by increasing the batch size \cite{b1}, while MoCo \cite{b2} and MoCo v2 \cite{b3} using momentum update mechanism keeps a queue as memory bank, which can dynamically maintain the count of negative samples." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02971" target="2003.04297" id="3184">
-        <attvalues>
-          <attvalue for="5" value=" Because the number of negative samples will directly affect the training results, in view of this, SimCLR provides more negative samples by increasing the batch size \cite{b1}, while MoCo \cite{b2} and MoCo v2 \cite{b3} using momentum update mechanism keeps a queue as memory bank, which can dynamically maintain the count of negative samples." />
-        </attvalues>
-      </edge>
-      <edge source="2304.02971" target="2010.06682" id="3185">
-        <attvalues>
-          <attvalue for="5" value=" The study \cite{b4} found that only a small minority of negative samples are both necessary and sufficient for contrastive learning." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1711.09349" id="3186">
-        <attvalues>
-          <attvalue for="5" value=" Inspired by the PCB model \cite{Sun2018}, the feature map from each level is divided horizontally into multi-stripes which will be processed later in pairs with overlapping rather than individually." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1805.08805" id="3187">
-        <attvalues>
-          <attvalue for="5" value=" Secondly, instead of using the features from all multi-resolution levels for classification as in \cite{Wang2018}, only the features from the last two levels are considered." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1703.07737" id="3188">
-        <attvalues>
-          <attvalue for="5" value=" Hermans \etal \cite{Hermans2017} and Zhang \etal \cite{zhang2019learning} modified triplet loss to increase the training performance." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1807.11178" id="3189">
-        <attvalues>
-          <attvalue for="5" value=" Shen \etal \cite{GSRW} combined global features with random walk algorithm." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1802.08122" id="3190">
-        <attvalues>
-          <attvalue for="5" value=" Li \etal \cite{Li2018} proposed an attention-based model." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1903.07071" id="3191">
-        <attvalues>
-          <attvalue for="5" value=" Luo \etal \cite{Luo_2019} reported a strong CNN based model with bag of learning tricks including augmentation and regularization." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1903.09776" id="3192">
-        <attvalues>
-          <attvalue for="5" value=" Quan \etal \cite{Quan2019} introduced neural architecture search to PReID by focusing on searching the best CNN structure and applied the part-aware module in PReID search space that employs both part and global information." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1503.03832" id="3193">
-        <attvalues>
-          <attvalue for="5" value=" Two loss functions are widely used: triplet loss \cite{FaceNet} and cross-entropy loss \cite{wen2016discriminative}." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1812.06576" id="3194">
-        <attvalues>
-          <attvalue for="5" value=" Hermans \etal \cite{Hermans2017} and Zhang \etal \cite{zhang2019learning} modified triplet loss to increase the training performance." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1807.00537" id="3195">
-        <attvalues>
-          <attvalue for="5" value=" Fan \etal \cite{Fan2019} presented a classification model based on an extended version of the cross-entropy loss function and a warming-up learning rate to learn a hypersphere manifold embedding." />
-        </attvalues>
-      </edge>
-      <edge source="1910.12322" target="1512.03385" id="3196">
-        <attvalues>
-          <attvalue for="5" value=" First, based on the residual network (ResNet50) \cite{Resnet}, multiple levels are created each of which has different resolution." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1409.7618" id="3197">
-        <attvalues>
-          <attvalue for="5" value=" Some of these methods are based on detecting objects and associating unique ids to the same objects in a sequence of frames \cite{mot_lit} while the others focus on the tracking task exclusively \cite{object_tracking}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1810.03851" id="3198">
-        <attvalues>
-          <attvalue for="5" value=" The other approach in \cite{deep_attentive} suggests a reciprocative learning algorithm while utilizing visual attention." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1804.04273" id="3199">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{adversarial_}, adversarial learning is employed to find the most robust features of the targets in a form of a mask." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1809.07845" id="3200">
-        <attvalues>
-          <attvalue for="5" value=" Single object tracking, on the other hand, is of high importance for which plenty of researches are conducted in this area \cite{Lasot_}." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="2004.00305" id="3201">
-        <attvalues>
-          <attvalue for="5" value=" To start with, a meta-updater is proposed in \cite{LTMU} by which an online-update-based tracker is designed to accomplish the local tracking." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1912.08531" id="3202">
-        <attvalues>
-          <attvalue for="5" value=" Another method presented in \cite{Global} suggests a baseline for performing a global search (over a very large area)." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1809.04320" id="3203">
-        <attvalues>
-          <attvalue for="5" value=" They also utilize the SiamRPN-based network \cite{siamrpn} to produce some region candidates for the subsequent tasks." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1904.07220" id="3204">
-        <attvalues>
-          <attvalue for="5" value=" In \cite{DiMP}, however, the authors suggest to consider both the target and the background appearance information to predict the required model based on defining a discriminative learning loss." />
-        </attvalues>
-      </edge>
-      <edge source="2103.15105" target="1811.07628" id="3205">
-        <attvalues>
-          <attvalue for="5" value=" Previous to this method the same backbone had been employed by \cite{atom} which is basically proposed using a ResNet-18 architecture along with an online discriminative learning." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1605.06211" id="3206">
-        <attvalues>
-          <attvalue for="5" value=" Neural networks, now popularized as deep learning models, have achieved state-of-the-art results in areas such as object recognition and localization \cite{ILSVRC, COCO}, semantic segmentation \cite{FULLYCONV, SEMSEG}, RGB-D \cite{DCNF} and multiple view data analytics, etc However, most deep learning methodologies usually require very high data processing bandwidth, and part of their overwhelming success can be attributed to the availability of large-scale image datasets \cite{ILSVRC, DL3D}." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1502.07411" id="3207">
-        <attvalues>
-          <attvalue for="5" value=" As reported in the literature, such as \cite{DCNF, SPCNN}, it is feasible to adapt superpixel features to many deep convolutional neural network (DCNN) architectures to extract features from convolutional maps, making a potentially very expensive computation feasible." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1811.11168" id="3208">
-        <attvalues>
-          <attvalue for="5" value=" Related state-of-the-art research actively addressing the connection between feature space and feature spatial properties also includes Deformable ConverNets \cite{DEFORM}, which has allowed the region of interest to adapt to the geometric variations of objects." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1806.02705" id="3209">
-        <attvalues>
-          <attvalue for="5" value=" Prior researches \cite{DCNF} \cite{SPSEG} in the field of deep learning have used superpixels mainly to reduce the computational intensity." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1412.0774" id="3210">
-        <attvalues>
-          <attvalue for="5" value=" \cite{SPSEMSEG} superpixels were used to exploit statistical structure in the image and in the label space and to avoid complex and expensive inference." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1710.09829" id="3211">
-        <attvalues>
-          <attvalue for="5" value=" To this end, capsules \cite{CAPS} are experimented with to create a model that analyzes visual images by putting together pieces of superpixels and processing the superpixel vector features." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1505.04597" id="3212">
-        <attvalues>
-          <attvalue for="5" value=" Although the relational analysis appears to be similar to segmentation, it is worth pointing out that it is not feasible to compare with the state-of-art segmentation DCNN such as U-Net \cite{UNET} or various other segmentation networks \cite{pal2018capsdemm} \cite{lalonde2018capsules} that rely on it, because those types of neural networks are using segmentation labels for training while the proposed model has none, instead implicitly learning through many occurrences of the image objects given non-localized class labels only." />
-        </attvalues>
-      </edge>
-      <edge source="1908.00669" target="1804.04241" id="3213">
-        <attvalues>
-          <attvalue for="5" value=" Although the relational analysis appears to be similar to segmentation, it is worth pointing out that it is not feasible to compare with the state-of-art segmentation DCNN such as U-Net \cite{UNET} or various other segmentation networks \cite{pal2018capsdemm} \cite{lalonde2018capsules} that rely on it, because those types of neural networks are using segmentation labels for training while the proposed model has none, instead implicitly learning through many occurrences of the image objects given non-localized class labels only." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1912.10664" id="3214">
-        <attvalues>
-          <attvalue for="5" value=" However, a new question arises: Could we take better advantage of existing large datasets for a task-specific application, particularly when object sizes significantly differ between the datasets? SM algorithm\cite{Yu2020ScaleMF}, Random Scale Match (RSM) and Monotone Scale Match (MSM), gave simple yet effective ways." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1612.03144" id="3215">
-        <attvalues>
-          <attvalue for="5" value=" It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs)." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1712.00726" id="3216">
-        <attvalues>
-          <attvalue for="5" value=" It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs)." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1904.02701" id="3217">
-        <attvalues>
-          <attvalue for="5" value=" It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs)." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1708.02002" id="3218">
-        <attvalues>
-          <attvalue for="5" value=" It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs)." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1904.01355" id="3219">
-        <attvalues>
-          <attvalue for="5" value=" It has wide applications including surveillance \cite{Collins2000ASF}\cite{Haritaoglu2000W4RS}, driving assistance \cite{Dollr2009PedestrianDA} and maritime quick rescue \cite{Yu2020ScaleMF}, etc The research of detectors \cite{lin2017feature}\cite{Cai2018CascadeRD}\cite{Pang2019LibraRT}\cite{lin2017focal}\cite{Tian2019FCOSFC} has achieved significant progress with the rapid development of data-driven deep convolutional neural networks (CNNs)." />
-        </attvalues>
-      </edge>
-      <edge source="2102.03558" target="1405.0312" id="3220">
-        <attvalues>
-          <attvalue for="5" value=" With the insufficient data for a specific application, an alternative way is to pre-train a model on the extra-large datasets (eg, ImageNet \cite{deng2009imagenet}, COCO \cite{lin2014microsoft}), and then fine-tune the model on a task-specific dataset." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05196" target="1401.8261" id="3221">
-        <attvalues>
-          <attvalue for="5" value="a Long-Wave infrared (\SI{7}{\micro\metre} - \SI{14}{\micro\metre} wavelength), is regarded as a promising direction to extend imaging technology \cite{ghiass2014infrared}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05196" target="1706.01789" id="3222">
-        <attvalues>
-          <attvalue for="5" value=" The models being tested in this study include Deep Alignment Network (DAN) \cite{kowalski2017deep}, Multi-task Convolutional Neural Network (MTCNN) \cite{zhang2016joint}, and Multi-class Patch-based fully convolutional neural network (PBC) \cite{poster2019examination}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05196" target="1703.10593" id="3223">
-        <attvalues>
-          <attvalue for="5" value=" The transformation is achieved by developing a CycleGAN model \cite{Zhu_2017_ICCV}." />
-        </attvalues>
-      </edge>
-      <edge source="2006.05196" target="1505.04597" id="3224">
-        <attvalues>
-          <attvalue for="5" value=" The first stage of this solution is to train a U-Net \cite{Ronneberger2015} to outline the face of a given image." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="1903.12261" id="3225">
-        <attvalues>
-          <attvalue for="5" value=" Applying various filtering effects to raw images for data augmentation techniques can improve the generalizability of a model, but filtered images significantly degrade performance in major computer vision tasks~\cite{bianco2017artistic,hendrycks2019benchmarking} such as image classification~\cite{chen2015filter,wu2020recognizing}, detection, and segmentation~\cite{kinli2021instagram}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="1912.13000" id="3226">
-        <attvalues>
-          <attvalue for="5" value=" Applying various filtering effects to raw images for data augmentation techniques can improve the generalizability of a model, but filtered images significantly degrade performance in major computer vision tasks~\cite{bianco2017artistic,hendrycks2019benchmarking} such as image classification~\cite{chen2015filter,wu2020recognizing}, detection, and segmentation~\cite{kinli2021instagram}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="2104.05072" id="3227">
-        <attvalues>
-          <attvalue for="5" value=" CAIR achieves about 11x faster runtime compared to~\cite{kinli2021instagram,kinli2022patch} without performance penalty." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="1611.07004" id="3228">
-        <attvalues>
-          <attvalue for="5" value=" IFRNet consists of an encoder with VGGNet \cite{simonyan2014very} as a style extractor module and a decoder inspired by PatchGAN \cite{isola2017image}, regarding filter as additional style information." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="1703.06868" id="3229">
-        <attvalues>
-          <attvalue for="5" value=" Instagram filter removal network (IFRNet) \cite{kinli2021instagram} defines the problem of removing filters from photos as a reverse style transfer \cite{huang2017arbitrary}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="1409.1556" id="3230">
-        <attvalues>
-          <attvalue for="5" value=" IFRNet consists of an encoder with VGGNet \cite{simonyan2014very} as a style extractor module and a decoder inspired by PatchGAN \cite{isola2017image}, regarding filter as additional style information." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="2204.07486" id="3231">
-        <attvalues>
-          <attvalue for="5" value=" CAIR achieves about 11x faster runtime compared to~\cite{kinli2021instagram,kinli2022patch} without performance penalty." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="2007.15651" id="3232">
-        <attvalues>
-          <attvalue for="5" value="~\cite{kinli2022patch} employ a multi-layer patch-wise contrastive learning strategy \cite{park2020contrastive} and propose an isolated patch sampling module that distills the content and style information \cite{kinli2022patch}." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="2204.04676" id="3233">
-        <attvalues>
-          <attvalue for="5" value=" We propose a multi-scale color attention network based on NAFNet (nonlinear activation function free network)~\cite{chen2022simple} for removing Instagram filters." />
-        </attvalues>
-      </edge>
-      <edge source="2208.14039" target="2003.07761" id="3234">
-        <attvalues>
-          <attvalue for="5" value=" Second, a color attention module inspired by~\cite{zamir2020cycleisp} is proposed to utilize the color-informative features from multi-scale input images." />
-        </attvalues>
-      </edge>
-    </edges>
-  </graph>
-</gexf>